Setting up downloads
To configure automatic downloads:- Create a field in your schema with the URL type.
- In the field settings, ensure Download file from URL is set to
True
. - Create a job that will save the download URL to a file in this field.
Note: File downloads happen asynchronously. It may take time for files to appear in your exports.
Retrieving files
File download links will be included in thefiles
array of your exports.
Example API response for file data:
Key fields
s3_url
: Pre-signed URL to retrieve the file from our S3 bucket.source_url
: Original URL of the file; points to our S3 bucket if no canonical source URL exists. If so, file_metadata.dynamic_download will be set to true.field
: Indicates which field in the output data the file relates to.
How are files downloaded?
Regular downloads
Regular downloads occur when files are directly accessible via a URL (e.g., direct PDF links).- The canonical URL of the file is used and saved
- Files are downloaded asynchronously via AWS Lambda using a dedicated download queue. Delays may occur.
Dynamic downloads
Dynamic downloads occur when there is no canonical URL available, typically triggered via JavaScript or requiring active session information.- Files are downloaded directly in the browser worker to guarantee accuracy.
- Because no canonical link is available, the link to the current page is used as the source URL.
- For more technical details, see Handling file downloading.