Skip to main content
print this page

JDBC bulkdataload dataflows issues

· 3 min read
Fix Available
Fix Available
This bug has been fixed

JDBC Bulkdataload Dataflows Issues​

Affected Versions: 2.7 3.0 3.0.3 3.0.6

Fix Version: 3.1

This report covers two related issues affecting JDBC bulkdataload dataflows:

  • Individual source table reload not working for full load dataflows
  • Endpoint testing failing immediately without retry on timeout.

Root cause(s)​

1. Individual source table reload not working for full load dataflows​

The reload-table feature allows users to reload individual source tables within dataflows, but was not working for full load dataflows due to a timing issue with statistics collection. The specific problem was:

  • The reload feature only works when the dataflow is in running state
  • However, the system was fetching and displaying dataflow statistics only after the dataflow completed running
  • This created a gap where users could not access the reload functionality during the critical running state
  • As a result, the individual table reload feature was effectively unavailable for full load dataflows

The fix involved changing the timing of statistics collection to occur during dataflow execution rather than after completion, making the reload feature accessible when needed.

2. Endpoint testing failing immediately without retry on timeout​

The endpoint testing mechanism fails immediately when encountering connectivity issues or timeouts, without implementing proper retry logic. The specific problems include:

  • No retry mechanism when initial endpoint tests fail due to temporary issues
  • Immediate failure on first timeout without considering transient connectivity problems
  • Lack of proper timeout handling during endpoint validation
  • Missing exponential backoff or retry intervals for endpoint testing operations

Impact​

1. Individual source table reload not working for full load dataflows​

  • Users could not use the individual table reload feature for full load dataflows
  • Users had to restart entire dataflows instead of reloading specific tables, causing longer downtime
  • Loss of productivity due to inability to perform granular table-level operations

2. Endpoint testing failing immediately without retry on timeout​

  • Dataflow start fails immediately on temporary endpoint issues
  • Users cannot run dataflows in environments with occasional connectivity problems
  • Poor user experience with immediate failures instead of resilient testing

Mitigation​

Fix available

Fix is available in Amorphic version 3.1. The statistics collection timing has been optimized for table reload operations during dataflow execution. Endpoint testing now includes proper retry logic with configurable timeouts.

Timeline​

  • 2025-08-18: Bug reported/identified (CLOUD-6006)
  • 2025-08-18: Bug triaged and documented
  • 2025-08-19: Root cause analysis, fix development and testing completed
  • 2025-08-22: Solution merged and released in Version 3.1