It provides multiple connectors to read and write data between different sources, such as cloud, on-premises, structured, and un-structured. Workflows contain the logic to transform and prepare data tables or files into different structures and layouts. Staging Tables can be used as either a source, or a synchronize data set when defining Azure Synapse pipelines.

Prerequisites

There are three main prerequisites to use Azure Synapse pipelines:

  1. A staging table must be present, as the Azure Synapse pipelines are required to read from or write into the staging table. Read Create a Staging Table for the Data Import to read more about a staging table.
  2. You also must have a connection string and authentication ID created by Anaplan's support team. The connection string is used in Azure Synapse when defining the synchronize data set.
  3. Optional: Azure Synapse connection string requirements. It is used if you automate the integration with Azure Synapse using a Workflow. If you select to use Azure Synapse triggers rather than a system Workflow to automate the execution of the Azure Synapse pipeline, this isn't required.

Connect to Staging Tables

Within the Azure Synapse workspace:

  1. Select the Data icon and select +. Now select Integration dataset.
  2. Select Azure SQL Database from the list of connectors.
  3. Enter a name and in the Linked service dropdown select + New.
  4. In the New Linked service property pane:
    1. Enter a name for the service.
    2. Select the Enter Manually button. Populate the properties with the details for the connection string and user name/password.
    3. Enter the name of the staging table that'll be used.
    4. Use the Test connection button to validate that the connection is successful.

Once the linked service is complete, you are now ready to use the staging tables as the synchronize destination when defining a pipeline. 

Automate the workflow (Optional step)

Create a Data Source connection if the Workflow objects are used to automate the execution of the Azure Synapse pipeline. The details required from the Azure Synapse environment to create a data source include:

  • SubscriptionId
  • TenantId
  • ApplicationId
  • AuthenticationKey

For example:

{"SubscriptionId": "", "TenantId": "111111-222222-33333-4444-5555", "ApplicationId": "111111-222222-33333-4444-5555", "AuthenticationKey": "111111-222222-33333-4444-5555"}

Create a data source

To create a data source:

  1. Select Configure and select + to create a new data source.
  2. Enter the data source name in Name.
  3. Select Azure Synapse from the dropdown in Provider.
  4. Enter the connection string in Details.
  5. Select Save to create the data source.
  6. Now select Save in the Connection Strings window to save the connection string for future use.

The next step is to define a Workflow to import or export data using the Azure Synapse pipeline.

See Automated Import Workflow Tasks to learn how to set up the workflow.