# Data Shaping with Pipelines

After data is extracted from a SingleStore Pipeline’s data source, it can be optionally shaped (modified). After shaping, the pipeline loads the data into one or more destination tables in SingleStore.

Some common data shaping operations that can be performed are:

* Lookups from other SingleStore tables (in addition to the destination table(s))
* Normalizing data
* Denormalizing data
* Adding computed columns
* Filtering data (excluding specific columns or records)
* Mapping data values from the data source to new values
* Splitting records from the data source into multiple destination tables
* Adding surrogate keys

Data modifications made during shaping are not written back to the data source, unless done explicitly in a transform (SingleStore Self-Managed only).

Ways to specify data shaping logic:

* In a `CREATE PIPELINE` statement.
* In a stored procedure that is called from the pipeline.
* In a transform that is called from the pipeline.

## Methods for Data Shaping with Pipelines

The details of each data shaping method are explained in the following table.

| Data Shaping Method            | Amount of Customization Logic Allowed | Ease of Use    | Comments                                                                                                                                                                                                                                                                                                                                                                                | Examples                                                                                                                                                                                                           |
| ------------------------------ | ------------------------------------- | -------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ |
| In a`CREATE PIPELINE`statement | Low                                   | Easiest        | **Pros**: Generally, runs the fastest of the three data shaping methods; transactional guarantees.**Cons**: Limited to SQL WHERE CLAUSE to perform filtering and SQL expressions to perform other shaping logic; can only load data into one target table; cannot perform lookups in other tables; a single row in the data source cannot generate multiple rows in the target table.   | `CREATE PIPELINE`uses the`LOAD DATA`clause to perform data shaping. See examples in the[LOAD DATA](https://docs.singlestore.com/db/v9.1/reference/sql-reference/data-manipulation-language-dml/load-data.md)topic. |
| Pipeline Stored Procedure      | Medium                                | More Difficult | **Pros**: Transactional guarantees; cons of specifying data shaping logic directly in your`CREATE PIPELINE`statement don’t apply.**Cons**: Limited to the capabilities of the PSQL language; cannot use third-party libraries.                                                                                                                                                          | See examples in[CREATE PIPELINE ... INTO PROCEDURE](https://docs.singlestore.com/db/v9.1/reference/sql-reference/pipelines-commands/create-pipeline-into-procedure.md)                                             |
| Transform                      | High                                  | Most Difficult | **Pros**: Can use any nearly any programming language and leverage third-party libraries.**Cons**: Not available inSingleStore Helios; runs slower than stored procedures for most use cases; deployment could be challenging; cannot insert into multiple tables; transactional guarantees apply to data written to`stdout`, only (refer to the guide in the next column for details). | See the guide[Writing a Transform to Use with a Pipeline](https://docs.singlestore.com/db/v9.1/load-data/about-singlestore-pipelines/pipeline-concepts/writing-a-transform-to-use-with-a-pipeline.md)              |

***

Modified at: October 23, 2023

Source: [/db/v9.1/load-data/about-singlestore-pipelines/pipeline-concepts/data-shaping-with-pipelines/](https://docs.singlestore.com/db/v9.1/load-data/about-singlestore-pipelines/pipeline-concepts/data-shaping-with-pipelines/)

(An index of the documentation is available at /llms.txt)