Upload a file in chunks

Uploading the file in chunks.

This makes large-file transfers more resilient and allows the integration to retry individual chunks instead of restarting the entire upload.Use chunking when:

• you want to be able to resume the transfer after the connection is lost during an upload.
• you are extracting data from a database and want to push it to the server without holding all the results in memory.

You can split a file into a set of chunks using a shell command such as split in shell with this syntax: split -b [number of bytes] [path and filename] [prefix for output files] For example: split -b 15360 ArchivedOpportunities.csv chunk-

The first chunk might represent the entire file, or the file might consist of multiple chunks. If the file is larger than one megabyte, PUT it onto the server by streaming each "chunk" as an octet stream. To do this, specify the header Content-Type:application/octet-stream.

We recommend that you specify a chunk size no larger than 50 MB of uncompressed data with no more than 1,000 chunks per file. To reduce the upload time for larger chunks, compress the chunks. You can compress the chunks in a folder using a shell command. For example, this command compresses all files within the current folder and those below it if they don't have the file extension .gz:

find . -type f ! -name '*.gz' -exec gzip "{}" \;

If you have compressed your chunk files, specify the header Content-Type:application/x-gzip instead of Content-Type:application/octet-stream. This compresses each chunk with the gzip format.

The HTTP 204 No Content success response status code indicates that a request has succeeded.