You can create a pipeline that writes back data from an Anaplan Data Orchestrator dataset to ‌Google Cloud Storage (GCS).

Prerequisites

You need a connection to Google Cloud Storage to create a pipeline. Make sure you meet the prerequisites in these sections before you create a connection to GCS and a writeback pipeline.

Create a connection to Google Cloud Storage

To create a connection:

  1. Select Data Orchestrator from the top-left navigation menu.
  2. Choose a dataspace from the list.
  3. Select Connections from the left-side panel.
  4. Select Create connection.
  5. Select the Google Cloud Storage connector and then select Next.
    If you can't find the connector, enter a search term in the Find... field.
  6. Enter these details on the Connection details screen, and then select Next:
    • Name: Create a name for your connection. The name can contain alphanumeric characters and underscores.
    • Description: Optionally, enter a description about your connection.
  7. Enter your Google credentials on the Connection credentials screen, and then select Next:
    • GCP Service Account Email: Enter your GCP service account email (for example, XXXXXXXXXXX-compute@developer.gserviceaccount.com). 
    • GCP Private Key: Enter GCP private key
      For example:

-----BEGIN PRIVATE KEY-----

************************************************************

... (Your private key content in Base64) ...

... multiple lines ...

-----END PRIVATE KEY-----

    • Bucket name: The GCS bucket where your data is stored (for example, GCS-EXAMPLE-BUCKET).
  1. After the connection test is complete, select Done.

Create a writeback pipeline

When you set up the writeback pipeline, you'll use your connection to export either a source dataset or a transformation view dataset (derived dataset) from Data Orchestrator to Google Cloud Storage.

Note that the write option for GCS is automatically full replace.

To create a writeback pipeline:

  1. Select Data Orchestrator from the top-left navigation menu.
  2. Choose a dataspace from the list.
  3. Select Pipelines from the left-side panel.
  4. Select Create pipeline.
  5. Enter a Name for your pipeline and then select Create.
    You are taken to the pipeline designer view.
  6. Select the Source icon, and then complete these steps in the right-side panel:
    1. Select Anaplan from the Connection type dropdown. 
    2. Select Datasets from the Choose connection dropdown.
    3. Enter a new Label if you want to change the source display name in the designer view.
      By default, the name of the dataset you selected displays as the label name.
    4. Select Source location to choose a Data Orchestrator dataset, then select Confirm.
      The dataset is used as the source for your pipeline.
    5. Select Columns if you want to exclude columns in the writeback.
       By default, all ‌columns are included. You can also select a Cursor column for your source.
  7. Optionally, select the add icon that appears between the Source and Sink nodes.
    You can add steps to your pipeline to process data.
  8. Select the Sink icon, and then complete these steps in the right-side panel:
    1. Select GCS from the Connection type dropdown.
    2. Select the GCS connection you created from the Choose connection dropdown.
    3. Enter the file name followed by the .csv extension in the File path field.
      We recommend you include the date in the file name in this format YYMMDD (Y for year, M for month, and D for date). For example: GCSfilename_260901.csv
    4. Select the Overwrite file checkbox if you want to overwrite an existing file in GCS, or leave it blank to create a new file:
      • Create new file: Leave the Overwrite file checkbox blank, and enter the new file name in the File path field.
        Note: If the file already exists, the pipeline fails and displays a message with this reason.
      • Overwrite existing file: Select the Overwrite file checkbox to overwrite an existing file in GCS. Enter the name of the existing file in the File path field.
    5. Select a Column Separator from the dropdown:
      • Comma
      • Tab
      • Pipe
      • Semicolon
    6. Select a Text delimiter from the dropdown:
      • Double quote (")
      • Single quote (')
      • None
  9. Enter a new Label to change the sink name that displays in the designer view.
  10. Select Publish, and then select Run to execute the data transfer.

Verify the writeback in Google Cloud Storage

Once the pipeline is successfully completed, you can log in to the Google Cloud Platform to verify the writeback. Navigate to Google Cloud Storage and verify the CSV file is written back to your bucket.