Skip to main content
Version: v3.0 print this page

DataPipelines

  1. Issue Description: Existing data pipelines utilizing file load validation or sync to S3 nodes fail to execute.

    Affected Versions:3.0

    Explanation: Existing datapipelines using either file load validation nodes or sync to s3 nodes will encounter errors due to missing arguments in underlying Glue jobs. The acl_resources_table and acl_resources_table_resourcetype_index arguments were missing in the post script which updates all data pipelines after a version upgrade. This issue does not affect new data pipelines.

    Workaround: Perform a dummy update on the affected data pipeline. This will automatically update the missing arguments in the underlying AWS Glue jobs, allowing the pipeline to execute successfully.