System Datasets

The Amorphic application provides users with seven system datasets. These datasets are available in a domain with the postfix system. The prefix depends on the shortname and environment selected for the application. User can run SQL queries to analyze dataset contents and maintain a record of query executions.

Available System Datasets

sys_dynamodb_logs This dataset contains information about calls made to DynamoDB tables, such as the table name accessed, the function name used, the Lambda triggered, the time, and the API (if any) used to make the call.

Log delivery is facilitated by SQS, but depending on the size of the input and output payloads to DynamoDB, these APIs may not be logged due to SQS message size limitations.

sys_api_gateway_logs This dataset contains information about API calls made through the amorphic application. It includes details such as the API resource path used, as well as user information like usernames and email addresses.

sys_cloudfront_logs This dataset contains information about API calls made through CloudFront associated with the Amorphic application.

sys_cloudtrail_logs This dataset contains information about the CloudTrail logs associated with the Amorphic application, including event-related information.

sys_s3_buckets_logs This dataset contains information about the S3 calls made through the Amorphic application, including the S3 endpoint, requester, and bucket owner information.

sys_cost_analysis_table This system dataset stores cost allocation information. It is used to track costs allocated for each tag key and value pair for each service and resource in the Amorphic application.

The dataset contains the following columns:

• bill_billing_period_end_date
• bill_billing_period_start_date
• line_item_blended_cost
• line_item_blended_rate
• line_item_currency_code
• line_item_net_unblended_cost
• line_item_net_unblended_rate
• line_item_product_code
• line_item_resource_id
• line_item_unblended_cost
• line_item_unblended_rate
• line_item_usage_amount
• line_item_usage_end_date
• line_item_usage_start_date
• pricing_currency
• pricing_public_on_demand_cost
• pricing_public_on_demand_rate
• product
• product_name
• product_servicename
• product_region_code
• resource_tags

For more information about these columns, refer to the AWS Cost and Usage Reports Documentation.

Note

The sys_cost_analysis_table dataset is only available when the cost feature is enabled for your Amorphic application.

sys_observability_logs This dataset contains information for all actions performed on the Amorphic application. It includes details about when actions occurred, resource details, who triggered the operation, and other operation-specific information.

Following are the resources for which observability logs are tracked:

• Assets
• Connection appflows
• Connections
• Connections apps
• Cost management
• Dashboards
• Data classification
• Data quality checks
• Data sources
• Datasets
• Datasource sync job
• Deep search index
• Deep search index sync jobs
• Domains
• Glossary
• Hcls health imaging
• Hcls store jobs
• Hcls workflows
• Insights
• Jobs
• Models
• NL2SQL training document
• Notebook lcc
• Notebooks
• Parameters
• Playground
• Roles
• Schedules
• Stream consumers
• Stream data transformations
• Streams
• Studios
• System config
• Tenants
• Views
• Workflows

Keeping Data Up to Date

Amorphic provides users with the option to keep the underlying data up to date using the Update Partitions option, which can be found under the System Datasets tab in the Infra Management.

Note

The Update Partitions option is enabled only for the sys_api_gateway_logs, sys_dynamodb_logs, and sys_observability_logs datasets.