What is the difference between Data Factory and Synapse pipelines?

Azure Synapse Analytics includes a pipeline service that is based on Data Factory. They share the same underlying technology and most features are identical. The main difference is that Synapse pipelines are part of the unified Synapse workspace and have direct integration with Synapse SQL pools and Spark pools.

Can Data Factory connect to on-premises data sources?

Yes. Data Factory uses a Self-Hosted Integration Runtime (SHIR) to connect to on-premises databases, file systems, and other data sources behind a firewall. The SHIR acts as a bridge between Data Factory in the cloud and your on-premises infrastructure.

Azure Data Factory Pipeline Builder

ComputeAzure

Build copy activity pipeline JSON with source and sink configurations.

Data Factory Configuration

Build copy activity pipeline JSON with source and sink configurations.

Required Fields

namepropertiesproperties.activities

Generated Output

Output will appear here...

About This Tool

The Azure Data Factory Pipeline Builder helps you create Data Factory pipeline definitions with activities, datasets, linked services, and triggers. Data Factory orchestrates data movement and transformation across cloud and on-premises sources. This tool provides a structured interface for building pipelines with copy activities, data flows, conditional logic, and iteration, generating the JSON definition for deployment via ARM templates, Bicep, or the Data Factory SDK.

When to Use This Tool

•Build data ingestion pipelines that copy data from SQL databases, blob storage, or REST APIs into a data lake.
•Create ETL pipelines with data flow transformations for cleaning, aggregating, and reshaping data before loading into a data warehouse.
•Design pipelines with ForEach loops and conditional logic for processing variable numbers of files or tables.
•Generate pipeline definitions with tumbling window or schedule triggers for recurring data processing jobs.

Frequently Asked Questions

What is the difference between Data Factory and Synapse pipelines?: Azure Synapse Analytics includes a pipeline service that is based on Data Factory. They share the same underlying technology and most features are identical. The main difference is that Synapse pipelines are part of the unified Synapse workspace and have direct integration with Synapse SQL pools and Spark pools.
Can Data Factory connect to on-premises data sources?: Yes. Data Factory uses a Self-Hosted Integration Runtime (SHIR) to connect to on-premises databases, file systems, and other data sources behind a firewall. The SHIR acts as a bridge between Data Factory in the cloud and your on-premises infrastructure.

Disclaimer: This tool runs entirely in your browser. No data is sent to our servers. Always verify outputs before using them in production. AWS, Azure, and GCP are trademarks of their respective owners.

Azure Data Factory Pipeline Builder

ComputeAzure

Build copy activity pipeline JSON with source and sink configurations.

Data Factory Configuration

Build copy activity pipeline JSON with source and sink configurations.

Required Fields

namepropertiesproperties.activities

Generated Output

Output will appear here...

About This Tool

When to Use This Tool

•Build data ingestion pipelines that copy data from SQL databases, blob storage, or REST APIs into a data lake.
•Create ETL pipelines with data flow transformations for cleaning, aggregating, and reshaping data before loading into a data warehouse.
•Design pipelines with ForEach loops and conditional logic for processing variable numbers of files or tables.
•Generate pipeline definitions with tumbling window or schedule triggers for recurring data processing jobs.

Frequently Asked Questions

What is the difference between Data Factory and Synapse pipelines?: Azure Synapse Analytics includes a pipeline service that is based on Data Factory. They share the same underlying technology and most features are identical. The main difference is that Synapse pipelines are part of the unified Synapse workspace and have direct integration with Synapse SQL pools and Spark pools.
Can Data Factory connect to on-premises data sources?: Yes. Data Factory uses a Self-Hosted Integration Runtime (SHIR) to connect to on-premises databases, file systems, and other data sources behind a firewall. The SHIR acts as a bridge between Data Factory in the cloud and your on-premises infrastructure.