Skip to main content
Version: v3.3 print this page

CICD on Amorphic

Continuous Integration and Continuous Delivery (CICD) is the practice of developing and releasing software in short, frequent cycles using automation pipelines. While CICD originated in software development, it has become increasingly essential in data engineering and data science as well.
By automating the build, test, and deployment process, teams can deliver releases faster and more reliably compared to manual methods.

Amorphic CICD extends this principle by allowing you to define Amorphic resources such as Datasets, Data Sources, ETL Jobs, and more as source code. These resources, along with supporting assets like ETL job scripts, can be bundled into a complete project definition. Projects can then be version-controlled in Git and deployed automatically to Amorphic through the Amorphic CICD platform.

This approach enables:

  • Version control of Amorphic as Code
  • Seamless migration of resources between environments
  • Integration with external automation tools (e.g., GitHub Actions, Bitbucket Pipelines)
info

The Amorphic CICD framework is compatible with Amorphic Data Cloud version 3.2 and later. For earlier versions, please refer to the legacy documentation or contact Amorphic Support for migration assistance.

When should I use Amorphic CICD?​

Amorphic CICD follows an Infrastructure-as-Code (IaC) model for managing resources. It is best suited for:

  • Complex projects with multiple contributors
  • Environments requiring consistent and repeatable deployments
  • Teams that need continuous integration and delivery of Amorphic resources

How does Amorphic CICD work?​

  1. Define resources as code
    Amorphic resources are described in JSON templates and organized in a project directory structure.

  2. Version control with Git
    These templates are stored and managed in a Git repository (Bitbucket, GitHub, or GitLab).

  3. Validate & deploy
    The Amorphic CICD Utils Python module validates the JSON templates and deploys them to the target Amorphic environment.

  4. Centralized automation
    The CICD framework provides infrastructure to run this process centrally, allowing multiple contributors to collaborate in the same Git repository.
    Whenever changes are pushed, the framework automatically executes the CICD Utils module to apply updates in Amorphic.