Resource Deletion

Users can manage lifecycle the same way users can manage creation: resource definitions in JSON. To stop managing a resource through CICD and remove it from Amorphic, users just need to remove that resource’s block from the JSON file pipeline deploys.

Example

Suppose the resource creation JSON file includes both a domain and a dataset, as shown below.

{
  "rCustomerDomain": {
    "Type": "Domain",
    "Properties": {
      "DomainName": "customer_data_domain",
      "DisplayName": "Customer Data",
      "DomainDescription": "Customer analytics and reporting"
    }
  },
  "rCustomerDataset": {
    "Type": "Dataset",
    "Properties": {
      "DatasetName": "customer_events",
      "DatasetDescription": "Inbound customer events",
      "Domain": {
        "!DependsOn": "rCustomerDomain.DomainName"
      },
      "Keywords": ["Owner: data-team"],
      "DatasourceType": "api",
      "IsDataValidationEnabled": true,
      "SerDe": "OpenCSVSerde",
      "FileDelimiter": ",",
      "FileType": "csv",
      "IsDataCleanupEnabled": false,
      "IsDataProfilingEnabled": true,
      "LifeCyclePolicyStatus": "Disabled",
      "TargetLocation": "s3athena",
      "SkipFileHeader": true,
      "SkipRowCount": { "header": 1, "footer": 0 },
      "SkipLZProcess": false,
      "TableUpdate": "append",
      "DataMetricsCollectionOptions": { "IsMetricsCollectionEnabled": false },
      "DatasetType": "internal",
      "DatasetSchema": [
        {
          "name": "event_id",
          "description": "",
          "type": "varchar(64)",
          "is_not_null": false
        },
        {
          "name": "event_type",
          "description": "",
          "type": "varchar(128)",
          "is_not_null": false
        }
      ]
    }
  }
}

The top-level keys (rCustomerDomain, rCustomerDataset) are the resource names CICD uses to tell resources apart. Those names must stay stable while you want the same logical resource updated on each run.

Deleting a resource

• To delete a resource, remove its entire object from the resource JSON (adjust commas so the file stays valid JSON). Then run your usual CICD deploy.
• CICD compares what is in the repo now with what it already deployed. Anything that is gone from the JSON but was deployed before is treated as removed and will be deleted in Amorphic on that run.

You do not need to plan separate deploys or manually sequence deletes yourself for the common case: drop the definition(s), run CICD, and let the pipeline perform the deletions.

• Single resource — Remove that resource’s block and deploy; CICD deletes it.
• Several resources at once — If you remove multiple resources in one change (for example both rCustomerDataset and rCustomerDomain from the sample above), a single deploy runs the deletes for everything you removed—you do not need one deploy per resource.
• When the run fails — Usually this is one of two situations:
  • Another resource in your JSON still points at the one you removed — For example you deleted rCustomerDomain but rCustomerDataset still has !DependsOn on that domain. CICD needs every !DependsOn target to exist in the same deployment. Fix: Remove or change those references, or remove the dependent resource blocks as well (then deploy again).
  • Amorphic refuses the delete — The delete API can return an error if the resource is still in use inside Amorphic (for example a domain that still has datasets). Fix: Remove the things that use it in Amorphic first—often by deleting those resources through CICD in the same change set or an earlier deploy—and try again. The error message from Amorphic or CICD should point to what is blocking removal.