9 posts tagged with "v2.7"

Serverless Dataflows Occasionally Stuck in Deleting State​

Affected Versions: 2.7 3.0 3.0.3 3.0.6

Fix Version: 3.1

JDBC Bulkdataload Dataflows Issues​

Affected Versions: 2.7 3.0 3.0.3 3.0.6

Fix Version: 3.1

This report covers two related issues affecting JDBC bulkdataload dataflows:

• Individual source table reload not working for full load dataflows
• Endpoint testing failing immediately without retry on timeout.

Files with identical names from different folders may not upload correctly, causing some of your data to be lost during S3 ingestion.

Affected Versions: 2.7 3.0

Fix Version: 3.1

Root cause(s)​

When you upload files with the same name from different folders in your S3 bucket, Amorphic doesn't always process all of them correctly.

For example, if your S3 bucket has these files:

• reports/2025-01-01/daily_sales.csv
• reports/2025-01-02/daily_sales.csv
• reports/2025-01-03/daily_sales.csv

Amorphic sees them all as just daily_sales.csv and treats them as the same file. This means only the last file gets processed successfully - the others get overwritten and lost.

This happens because the system focuses on the filename only and ignores which folder each file came from. When multiple files are processed at the same time, they conflict with each other and some data gets lost.

Impact​

• Your data goes missing: Files with the same name get overwritten, and you lose data without any warning
• Unpredictable results: Which files get processed depends on timing, so results vary each time you run ingestion
• Misleading error messages: System provides file not found error message for the ingestion exception.
• Affects all S3 connections: This happens whether you use access keys or bucket policies to connect to S3
• Historical data loss: You might lose important historical records stored in date-organized folders

Mitigation​

What you can do right now:

1. Make filenames unique: Add dates, timestamps, or folder names to your filenames (like daily_sales_2025-01-01.csv)
2. Check your data: Review your ingestion results to make sure all expected data is present
3. Process folders one at a time: If possible, upload files from one folder at a time instead of all at once
4. Monitor carefully: Keep a close eye on your data uploads until the fix is available

What's available in version 3.1:

• All files will be processed correctly, even if they have the same name
• Better tracking to ensure no data is lost

Timeline​

• 2025-08-05: Users reported missing data after S3 ingestion
• 2025-08-07: Problem identified and analyzed - files with same names overwriting each other
• 2025-08-14: Solution developed and completed to handle identical filenames properly
• 2025-08-15: Solution available in version 3.1

Parquet file uploads fail when you change column data types in Redshift datasets.

Affected Versions: 2.7 3.0 3.0.3 3.0.6

Fix Version: 3.1

Upon deletion of a user from Amorphic, its associated metadata is also removed. But metadata pertaining to this said user is actually retained in the Role Managers list in the Roles metadata in the backend tables.

Affected Versions: 2.7 3.0 3.0.3 3.0.6

Fix Version: 3.0.6

Glossary term metadata still pertains in backend table even after parent dataset was deleted

Affected Versions: 2.7 3.0 3.0.3 3.0.6

Fix Version: 3.0.6

Users without editor access to Apps could still create new users and grant them App access.

Affected Versions: 2.6 2.6.1

Fix Version: 2.7

In amorphic we have different set of permissions to access different components and features. This permissions are given to an amorphic role and user who have access to these role can use those features. This bug is relate to an issue that is users were able to call specific APIs and use those features in Amoprhic even if their role doesn't have the required permission.

Affected Versions: 2.6.1 and below

Fix Version: 2.7

Scheduled backend job data metric collection is failing with error 'some pip modules could not be installed'.

Affected Versions: 2.6, 2.6.1

Fix Version: 2.7