StreamSets Data Collector™ is a data ingestion infrastructure that simplifies the process of designing, testing, deploying and operating data pipelines. he tool enables users to migrate data from sources such as NoSQL, RDBMS and REST endpoints to another destination such as Amazon S3, a data lake or a data warehouse.
In addition to allowing the user to build a dynamic and agile data pipeline, StreamSets Data Collector™ can help clean and transform data that is not in an ideal format. Red Pill often uses StreamSets in cases where data is difficult to parse and organize; the Data Collector makes this part of the job easier with built-in tools for data storage and organization.