Datasources

Amorphic Datasources are external systems or repositories from which data is ingested into platform datasets. They enable users to connect to and pull data from a wide variety of sources, such as streams, SaaS applications, external APIs, and S3 buckets, into Amorphic for processing and analysis.

Below is the sample resource definition file for Datasources:

{
    "rDatasource": {
        "Type": "Datasource",
        "Properties": {
            "DatasourceConfig":{
                "JdbcURL":"jdbc:postgresql://hostname.com:5432/database",
                "Username":{
                    "!Environ": "DB_USERNAME"
                },
                "Password":{
                    "!Environ": "DB_PASSWORD"
                },
                "PublicAccessibility":"yes"
            },
            "DatasourceType":"jdbc",
            "DatasourceName":"pqsqldata",
            "Description":"Postgress as datasource",
            "Keywords":["DB : PostgreSQL","Owner: asysuser"],
            "IngestionType":"normaldataload",
            "DatasourceVersion":"5.0"
        }
    }
}

Note: Deployment dependencies

Datasource has dependencies on Tag.

Dependent resources should not be deleted before the primary resource; attempting to do so may lead to failures or inconsistencies during the deletion process.

Referencing this Resource

Below are the common keys that can be used in the DependsOn function to retrieve details of this resource.

Supported Keys

Key	Description
DatasourceId	Returns the DatasourceId value of this resource
DatasourceName	Returns the DatasourceName value of this resource

For additional supported keys, refer to the API definition document for the respective resource type.

Example

The following example shows how to retrieve the datasource ID from a datasource template and use it in a dataflow template:

{
  "rDataflow": {
    "Type": "Dataflow",
    "Properties": {
      "DatasourceId": {
        "!DependsOn": "rDatasource.DatasourceId"
      }
    }
  }
}

In the above JSON configuration, rDatasource is a logical resource that must be unique across the repository. It serves as a unique identifier for a resource among all the resources created.

The Properties section accepts a dictionary, which takes the Datasource configuration as input. The specific configuration used will depend on the DatasourceType and IngestionType specified.