You can create a pipeline that writes back data from an Anaplan Data Orchestrator dataset to ‌an SFTP server.

Prerequisites

You need a connection to an SFTP server to create a pipeline. Make sure you meet the prerequisites in these sections before you create a connection to an SFTP server and a writeback pipeline.

Create a connection to an SFTP server

To create a connection:

  1. Select Data Orchestrator from the top-left navigation menu.
  2. Choose a dataspace from the list.
  3. Select Connections from the left-side panel.
  4. Select Create connection.
  5. On the Create connection page, select SFTP, and then select Next.
    If you can't find the connector, enter a search term in the Find... field.
  6. On the Connection details page, enter these details and select Next:
    • Name: Create a name for your connection. The name can contain alphanumeric characters and underscores.
    • Description: Optionally, enter a description about your connection.
  7. On the Connection credentials page, enter your SFTP credentials and select Next:
    • Host: Enter your SFTP server hostname (for example, s-52d4b77da9example.server.transfer.us-east-1.amazonaws.com).
    • Port: Enter the SFTP port (for example, 22).
    • Username: Enter the SFTP server user name (for example, sftp-user).
    • Private Key: Enter the SSH private key (OpenSSH format) linked to the SFTP server.

      You can use the ssh-keygen -t rsa -b 4096 -f <keyfilename> command to generate a key pair. When you run the command, the system prompts you to set a passphrase. Press Enter twice to generate the unencrypted private key without a passphrase.
      This creates a public key (.pub) and a private key. The public key is linked to the SFTP server and the private key is used to make connections.

      Private key example:

-----BEGIN OPENSSH PRIVATE KEY-----
b3BlbnNzaC1rZXktdjEAAAAABG5vbmUAAAAEbm9uZQAAAAAAAAABAAABlwAAAAdzc2gtcn
... (Your private key content in Base64) ...
... multiple lines ...
-----END OPENSSH PRIVATE KEY-----

  1. After the connection test is complete, select Done.

Create a writeback pipeline

When you set up the writeback pipeline, you'll use your connection to export either a source dataset or a transformation view dataset (derived dataset) from Data Orchestrator to an SFTP server.

Note that the write option for SFTP writeback pipeline is automatically full replace.

To create a writeback pipeline:

  1. Select Data Orchestrator from the top-left navigation menu.
  2. Choose a dataspace from the list.
  3. Select Pipelines from the left-side panel.
  4. Select Create pipeline.
  5. Enter a Name for your pipeline and then select Create.
    You are taken to the pipeline designer view.
  6. Select the Source icon, and then complete these steps in the right-side panel:
    1. Select Anaplan from the Connection type dropdown. 
    2. Select Datasets from the Choose connection dropdown.
    3. Enter a new Label to change the source display name in the designer view.
      By default, the name of the dataset you selected displays as the label name.
    4. Select Source location > Source to choose a Data Orchestrator dataset, then select Confirm.
      The dataset is used as the source for your pipeline.
    5. Select Columns if you want to exclude columns in the writeback.
       By default, all ‌columns are included. You can also select a Cursor column for your source.
  7. Optionally, select the add icon that appears between the Source and Sink nodes.
    You can add steps to your pipeline to process data.
  8. Select the Sink icon, and then complete these steps in the right-side panel:
    1. Select SFTP from the Connection type dropdown.
    2. Select the SFTP connection you created from the Choose connection dropdown.
    3. Enter a new Label to change the sink name that displays in the designer view.
    4. Enter the file name followed by the .csv extension in the File path field.
      We recommend you include the date in the file name in this format YYMMDD (Y for year, M for month, and D for date). For example: SFTPfilename_260901.csv
    5. Select the Overwrite file checkbox to overwrite an existing file on the SFTP server, or leave it blank to create a new file:
      • Create new file: Leave the Overwrite file checkbox blank, and enter the new file name in the File path field.
        Note: If the file already exists, the pipeline fails and displays a message with this reason.
      • Overwrite existing file: Select the Overwrite file checkbox to overwrite an existing file on the SFTP server. Enter the name of the existing file in the File path field.
    6. Select a Column Separator from the dropdown:
      • Comma
      • Tab
      • Pipe
      • Semicolon
    7. Select a Text delimiter from the dropdown:
      • Double quote (")
      • Single quote (')
      • None
  9. Select Publish, and then select Run to execute the data transfer.

Verify the writeback in SFTP

Once the pipeline is successfully completed, log in to the SFTP server, and verify the CSV file is written back to the SFTP server.