Shared Libraries

Libraries are an extension of external job libraries. They are mainly used to maintain a central repository of organization-approved libraries/packages to be used across multiple Jobs or Data labs.

Files that need to be uploaded to a shared library must be specified under the Artifacts key using one of the two supported input modes: File or Folder.
Each path should be relative to the root resources directory.

---

Supported File Types

CICD supports the same file types that Amorphic supports for shared libraries. The currently supported types are:

.py , .zip , .egg , .jar , .whl

These apply when using File mode, each path in the list must point to a file of one of the above types.

---

Input Modes

File Mode

Use File mode when you want to upload one or more individual library files. Each entry in the list is a path to a single file of any supported type.

{
  "rLibrary": {
    "Type": "SharedLibraries",
    "Artifacts": {
      "Library": {
        "File": [
          "resources/shared-libraries/libraries.py",
          "resources/shared-libraries/utils.zip",
          "resources/shared-libraries/analytics.whl"
        ]
      }
    },
    "Properties": {
      "LibraryName": "cicd_library",
      "LibraryDescription": "Library created using CICD",
      "Packages": []
    }
  }
}

Folder Mode

Use Folder mode when you have a package spread across multiple files and sub-directories that you want to upload as a single unit. CICD automatically zips each folder before uploading, no manual zipping is required.

The uploaded file takes its name from the folder itself: a folder named etl_package is uploaded as etl_package.zip.

{
  "rLibrary": {
    "Type": "SharedLibraries",
    "Artifacts": {
      "Library": {
        "Folder": [
          "resources/shared-libraries/etl_package",
          "resources/shared-libraries/analytics_package"
        ]
      }
    },
    "Properties": {
      "LibraryName": "cicd_library",
      "LibraryDescription": "Library created using CICD",
      "Packages": []
    }
  }
}

Note: Older format still supported

If you are already using the older flat-list format shown below, there is no need to migrate — CICD handles it automatically. The structured File / Folder format is recommended for new definitions.

"Library": [
  "resources/shared-libraries/libraries.py"
]

---

State File

After a successful deployment, CICD records the artifact state for each resource. This state is used on subsequent runs to detect changes — only files that have been added, removed, or modified are re-uploaded; everything else is skipped.

File mode — state file entry:

"Artifacts": {
  "Library": {
    "File": {
      "resources/shared-libraries/libraries.py": {
        "Hash": "<sha256>",
        "S3Path": "s3://..."
      },
      "resources/shared-libraries/utils.zip": {
        "Hash": "<sha256>",
        "S3Path": "s3://..."
      }
    }
  }
}

Folder mode — state file entry:

"Artifacts": {
  "Library": {
    "Folder": {
      "resources/shared-libraries/etl_package": {
        "Hash": "<sha256>",
        "S3Path": "s3://.../etl_package.zip"
      },
      "resources/shared-libraries/analytics_package": {
        "Hash": "<sha256>",
        "S3Path": "s3://.../analytics_package.zip"
      }
    }
  }
}

---

Note: Deployment dependencies

SharedLibraries lists no other resource types under DependsOn in the CICD module configuration.

Dependent resources should not be deleted before the primary resource; attempting to do so may lead to failures or inconsistencies during the deletion process.

---

Referencing this Resource

Below are the common keys that can be used in the DependsOn function to retrieve details of this resource.

Supported Keys

Key	Description
LibraryName	Returns the LibraryName value of this resource
LibraryId	Returns the LibraryId value of this resource

For additional supported keys, refer to the API definition document for the respective resource type.

Example

The following example shows how to retrieve the shared library ID from a shared library template and use it in a job template:

{
  "rPythonJob": {
    "Type": "Job",
    "Properties": {
      "SharedLibraries": [
        {
          "!DependsOn": "rLibrary.LibraryId"
        }
      ]
    }
  }
}