It provides multiple connectors to read and write data between different sources, such as cloud, on-premises, structured, and un-structured. Workflows contain the logic to transform and prepare data tables or files into different structures and layouts. The system's Staging Tables can be used as either a source, or a synchronize data set when defining Azure Synapse pipelines. 

Prerequisites

There are three main prerequisites to use Azure Data Factory pipelines:

  1. A Staging Table must be present, as the Azure Data Factory pipelines are required to read from or write into the staging table. Read Create a Staging Table for the Data Import to read more about a Staging Table.
  2. You also must have a connection string and authentication ID created by the support team. The connection string is used in Azure Synapse when defining the synchronize data set. Contact support to request an authentication ID be setup for you to access Financial Consolidation Staging Tables.
  3. Optional: Azure Data Factory connection string requirements. It is used if you automate the integration with Azure Data Factory using a Financial Consolidation workflow. If you select to use Azure Data Factory triggers rather than the workflow to automate the execution of the Azure Data Factory pipeline, this isn't required.

Connect to Staging tables

Within the Azure Data Factory workspace:

  1. Select the Data icon and select the "+" icon. Now select Integration dataset.
  2. Select Azure SQL Database from the list of connectors.
  3. Enter a name and in the Linked service dropdown select + New.
  4. In the New Linked service property pane:
    1. Enter a name for the service.
    2. Select the Enter Manually button. Populate the properties with the details for the connection string and user name/password.
    3. Enter the name of the staging table that'll be used.
    4. Use the Test connection button to validate that the connection is successful.

Once the linked service is complete, you are now ready to use the staging tables as the synchronize destination when defining a pipeline. 

Automate the Workflow (Optional step)

Create a Data Source connection if the Workflow objects are used to automate the execution of the Azure Data Factory pipeline. The details required from the Azure Synapse environment to create a Data Source include:

  • SubscriptionId
  • TenantId
  • ApplicationId
  • AuthenticationKey

For example:

{"SubscriptionId": "", "TenantId": "111111-222222-33333-4444-5555", "ApplicationId": "111111-222222-33333-4444-5555", "AuthenticationKey": "111111-222222-33333-4444-5555"}

Create a data source

To create a data source:

  1. Select Configure and remain in the Connection Strings window.
  2. Select + to create a new data source.
  3. Enter a name for the data source in Name.
  4. Select Azure Data Factory from the dropdown in Provider.
  5. Enter a connection string in Details.
  6. Select Save to create the data source.
  7. Now select Save in the Connection Strings window to save the connection string for future use.

The next step is to define a workflow to import or export data using the Azure Data Factory pipeline.

See Application Integration Tasks to learn how to set up the workflow.