Skip to main content
print this page

Failed to ingest data into reload type of datasets with target location redshift(only in multi-tenant envs)

· 2 min read
Bug Reported
Bug Reported
Bug has been reported, and is being investigated
Fix In Progress
Fix In Progress
Bug identified and fix is in progress

In a multi-tenant environment(with non ra3 redshift nodes), Users were unable to process files in a reload type of dataset with target location as redshift when the selected domain is under a non-default tenant.

Ex: In a multi-tenant environment, User created a 'tenant01' tenant and created a domain 'domain01' under this tenant. Now user created a Reload type of redshift dataset and uploaded file to it. This will fail with below error.

Data load failed in previous run with error - Failed to rename temporary table after job success with error - DWH-1038 - Failed, ERROR: schema "abcd" does not exist

Affected Versions: 2.1 2.0 1.14 1.13

Fix Version: 2.2

Root cause(s)​

In 1.13, Redshift DATA APIs were introduced in Amorphic to improve the Redshift data copy process. Amorphic is creating redshift connection with the default tenant (database) instead of tenant where the dataset was created, which is leading to the 'schema does not exist' error as redshift is searching the table in default tenant instead of dataset specific tenant.

Note: Redshift does not allow cross-tenant queries in non-ra3 type of nodes.

Impact​

In a multi-tenant environment, Users will be unable to process files in a reload type of redshift dataset with domain created under non-default tenant.

Mitigation​

Use the ra3 family redshift nodes in a multi-tenant environment.

Timeline​

gantt
    title Timeline
    dateFormat YYYY-MM-DD
    tickInterval 5h
    axisFormat %b-%d
    todayMarker off
    section Tracker
        %% update the ticket number and date of bug report
        CLOUD-3217 : done, 2023-04-10, 0d
    section Identification
        Reported : crit, des1, 2023-04-10, 0d
    section Mitigation
        %% Update number of days took for each step below
        Bug fixed: crit, done, 2023-04-10, 1d
    section Delivery
        %% update the date of each step below
        testing completed: milestone, 2023-04-11, 1d
  • 2023-04-10: Bug reported / identified (CLOUD-3217)
  • 2023-04-10: Bug triaged
  • 2023-04-10: Bug fixed
  • 2023-04-11: Testing completed
  • 2023-06-01: Planned for Release v2.2