Repository Organization

Use the following repository conventions for Amorphic CICD (recommended, not enforced):

• Store all resource definitions as JSON under a root folder (commonly resources), organized by resource type (for example, datasets, datasources, jobs).
• Multiple resources can exist in one JSON file, but one resource per file in a resource-specific folder is recommended (for example, datasets/my_dataset/my_dataset.json) for readability and isolation.
• Keep supporting assets (such as ETL scripts) in the same folder as the resource JSON.
• Include a mandatory driver file named cicd.py (name cannot be changed); it is the deployment entry point for Amorphic CICD Utils.
• Include a README.md for contributor guidance.
• For centralized sharing, use a root-level access-management folder with a subdirectory per target (dev, qa, prod, …). Point ResourceManager at that subdirectory via access_management_dir (for example ./access-management/dev). Details: Resource access management.
• File/folder naming is flexible, but consistent and descriptive naming is strongly encouraged.

The example below illustrates a recommended directory structure for organizing Amorphic CICD repositories:

.
├── cicd.py
├── README.md
├── access-management
│   ├── dev
│   │   ├── team-a-shares.json
│   │   └── team-b-shares.json
│   ├── qa
│   │   └── shares.json
│   └── prod
│       └── shares.json
└── resources
    ├── datasources
    │   └── bulkload_v2
    │       └── bulkload_v2.json
    ├── datasets
    │   ├── cicd_dataset_1
    │   │   └── cicd_dataset_1.json
    │   └── cicd_dataset_2
    │       └── cicd_dataset_2.json
    ├── domains
    │   └── cicd_domain_1
    │       └── cicd_domain_1.json
    ├── jobs
    │   └── cicd_job_1
    │       ├── job_script.py
    │       └── cicd_job_1.json
    ├── parameters
    │   └── api_key_param
    │       └── api_key_param.json
    ├── schedules
    │   └── schedule_1
    │       └── schedule_1.json
    ├── roles
    │   └── role_1
    │       └── role_1.json
    ├── tags
    │   └── tag_1
    │       └── tag_1.json
    ├── saml_mappings
    │   └── saml_mapping_1
    │       └── saml_mapping_1.json
    ├── shared_libraries
    │   └── library_1
    │       └── library_1.json
    ├── data_pipelines
    │   └── data_pipeline_1
    │       └── data_pipeline_1.json
    ├── dataflows
    │   └── dataflow_1
    │       └── dataflow_1.json
    ├── guardrails
    │   └── guardrail_1
    │       └── guardrail_1.json
    ├── agents
    │   └── agent_1
    │       └── agent_1.json
    ├── knowledge-bases
    │   └── kb_1
    │       └── kb_1.json
    ├── datalabs
    │   └── datalab_1
    │       └── datalab_1.json
    ├── datalab-lifecycle-configurations
    │   └── dlc_1
    │       └── dlc_1.json
    ├── data-classifications
    │   └── classification_1
    │       └── classification_1.json
    ├── dashboards
    │   └── dashboard_1
    │       └── dashboard_1.json
    ├── projects
    │   └── project_1
    │       └── project_1.json
    ├── mail-servers
    │   └── mail_server_1
    │       └── mail_server_1.json
    └── cost-tags
        └── cost_tag_1
            └── cost_tag_1.json