Generate a presigned URL for dataset file upload
POST/datasets/file
Generates an S3 presigned URL for uploading a file to the specified dataset.
Dataset requirements:
- DatasetType must be
internal - RegistrationStatus must be
completed - DatasourceType must be
api
Partition keys:
- Required when the dataset has PartitionKeys defined and is not Iceberg, Hudi, or Delta Lake.
- Each partition key (except
upload_date) must have a value in the request body. For example, if the dataset defines PartitionKeys as [region, year, upload_date], onlyregionandyearneed to be supplied. - Iceberg, Hudi, and Delta Lake datasets manage partitions internally — do not pass PartitionKeys.
- PartitionKeys values are used to build the S3 path.
upload_dateis auto-generated from the current epoch timestamp and appended to the path. Ifupload_dateis passed in the request body, it is ignored and overwritten with the system-generated value.
Schema uploads:
- When the dataset is in
pendingregistration status, this endpoint is used for sample schema file uploads. - In this case, DatasetType and DatasourceType checks are skipped.
File name handling:
- Special characters like (
& $ @ = ; : + , ? \ / { ^ } % \] > [ ~ < # | " '`) are replaced with underscores for S3 key compliance.
Request
Responses
- 200
- 400
- 500
Successfully generated S3 presigned URL
Response Headers
Access-Control-Allow-Origin
Access-Control-Allow-Methods
Access-Control-Allow-Headers
Invalid input - dataset not found, dataset type not internal, registration not completed, non-API datasource, missing partition keys, or insufficient permissions
Response Headers
Access-Control-Allow-Origin
Access-Control-Allow-Methods
Access-Control-Allow-Headers
Unexpected internal error during presigned URL generation
Response Headers
Access-Control-Allow-Origin
Access-Control-Allow-Methods
Access-Control-Allow-Headers