Cloud Custodian

The Cloud Custodian Plugin runs Cloud Custodian policies in dry-run mode, builds a full inventory baseline for each configured resource type, converts each resource/policy pair into a standardized per-resource payload, and then executes CCF OPA bundles against those payloads to generate evidence.

This page is for operators who run the plugin-cloud-custodian plugin with the CCF Agent, and for developers who write policies for it. It documents v0.3.0 and is generated from the plugin's README at v0.3.0.

How it works

Behavior Overview

Load Cloud Custodian policy YAML from config.
Parse top-level policies and iterate one policy entry per check.
Register runtime-derived subject templates for each configured resource type during runner-v2 Init.
Run one unfiltered inventory collection for each unique resource type.
Run each configured check with:

custodian run --dryrun -s <output-dir> <single-policy-file>

Compare each check's matched resources with the inventory baseline.
Build one standardized payload per resource per check. Matched resources are marked non_compliant; baseline resources not matched by that check are marked compliant.
Evaluate each OPA policy bundle path from agent EvalRequest.policyPaths.
Send evidence via the plugin gRPC helper (CreateEvidence).

Safety Model

This plugin always enforces read-only Cloud Custodian execution:

--dryrun is always used.
Mutating actions are not applied.
For AWS checks, the plugin runs with --region all to evaluate across all AWS regions by default unless aws_regions is configured.
With more than one region (including all), Cloud Custodian writes one resources.json per region (<output>/<region>/<policy>/resources.json). The plugin reads every one of them, for the inventory baseline and for each check.
On execution failures or timeouts, the plugin includes the tail of any generated custodian-run.log artifacts in the execution error.

Requirements

Operational Prerequisites

Cloud Custodian CLI must be installed and executable.
Cloud/provider credentials must be available in the plugin process environment (ambient credentials/profile/env vars). With aws_accounts, these are the hub's credentials, used to assume each account's role.

Configuration

All plugin config fields are strings (agent gRPC map<string,string> contract).

Key	Required	Description
`policies_yaml`	Conditionally	Inline Cloud Custodian policy YAML. Preferred over `policies_path` when both are set.
`policies_path`	Conditionally	Local path, `file://`, `http://`, or `https://` location for policy YAML. Used when `policies_yaml` is empty.
`custodian_binary`	No	Path/name of Cloud Custodian executable. Default: `custodian`.
`custodian_debug`	No	Boolean (`true`/`false`) toggle to pass `--debug` to Cloud Custodian. This increases Cloud Custodian diagnostic output on stderr. Default: `false`.
`custodian_verbose`	No	Boolean (`true`/`false`) toggle to pass `-v` to Cloud Custodian. This increases Cloud Custodian diagnostic output on stderr. Default: `false`.
`custodian_aws_api_trace`	No	Boolean (`true`/`false`) toggle to inject a temporary Python `sitecustomize.py` into the Custodian child process. Logs botocore API start/end/error events to stderr and `custodian-aws-api-trace.jsonl` in the check output directory. Default: `false`.
`custodian_network_diagnostics`	No	Boolean (`true`/`false`) toggle to run Go DNS/TLS preflight probes for relevant AWS service endpoints before Custodian starts and log child process TCP socket snapshots while Custodian is running. DNS/TLS probe failures are logged as warnings and surfaced in evidence as `execution.warnings`. Concrete `aws_regions` are narrowed to regions with reachable resource service endpoints before Custodian runs; if none are reachable for a check, that check is skipped, later checks continue, and the accumulated diagnostic warnings are returned after reachable evidence is submitted. If no concrete `aws_regions` are configured, service-derived probes are skipped unless `custodian_network_diagnostic_endpoints` is configured. For AWS resource types not mapped to diagnostic services, diagnostics warn and continue unless explicit endpoints are configured. Invalid configured endpoints still fail before execution. Default: `false`.
`custodian_network_diagnostic_endpoints`	No	Comma or whitespace separated list of additional endpoint hostnames, `host:port` values, or HTTPS URLs to DNS/TLS probe when `custodian_network_diagnostics` is enabled. Non-HTTPS URL schemes are rejected. Use this for AWS VPC endpoint DNS names such as `vpce-123.backup.eu-west-1.vpce.amazonaws.com`. Default: unset.
`custodian_log_tail_during_run`	No	Boolean (`true`/`false`) toggle to tail discovered `custodian-run.log` artifacts during the monitor loop, not only after process exit. Default: `false`.
`aws_regions`	No	Comma or whitespace separated AWS regions passed as repeated `--region` flags. Duplicate entries are removed while preserving order. Default: unset, which falls back to `--region all` for AWS checks.
`check_timeout_seconds`	No	Per-check timeout in seconds. Default: `300`.
`policy_labels`	No	JSON map of labels merged into generated evidence labels.
`resource_identity_fields`	No	JSON object mapping Cloud Custodian resource types to ordered identity field paths. Built-in defaults (see Resource Identity) are used after configured fields. Example: `{"aws.ec2":["InstanceId","Arn"]}`.
`debug_dump_payloads`	No	Boolean (`true`/`false`) toggle to write standardized resource payload JSON files for troubleshooting. Default: `false`.
`debug_payload_output_dir`	No	Directory where debug payload JSON files are written. If set, debug dumping is auto-enabled. Default when enabled without explicit path: `ccf-custodian-debug-payloads` under the OS temp directory (`$TMPDIR`, else `/tmp`), e.g. `/tmp/ccf-custodian-debug-payloads`. Use an absolute path: a relative value is still honoured but resolves against the plugin process working directory (which the agent may set per plugin) and logs a warning. The resolved absolute path is logged at `INFO` when dumping is enabled.
`preserve_execution_artifacts`	No	Boolean (`true`/`false`) toggle to keep the temporary Cloud Custodian execution root after a check execution failure for postmortem review. Default: `false`.
`aws_accounts`	No	JSON (or YAML) array of AWS accounts to scan: `{"id", "name", "role_arn", "role_name", "external_id", "regions", "use_ambient_credentials"}`, only `id` required. See Cross-Account Scanning. Default: unset, which scans the account of the plugin's own credentials.
`aws_assume_role_name`	No	Role name assumed in every account without its own `role_arn` or `role_name`. Only used with `aws_accounts`.
`aws_external_id`	No	External ID passed when assuming a role in an account without its own `external_id`. Only used with `aws_accounts`. Never logged.
`aws_partition`	No	Partition of the role ARNs built from role names (`aws`, `aws-cn`, `aws-us-gov`, ...). Only used with `aws_accounts`. Default: derived from the account's regions (`cn-` is `aws-cn`, `us-gov-` is `aws-us-gov`), else `aws`.
`aws_session_name_suffix`	No	Suffix of the STS session name, `CloudCustodian@<suffix>` (`C7N_SESSION_SUFFIX`). Only used with `aws_accounts`. Default: `ccf`.

Validation rules:

At least one of policies_yaml or policies_path must be provided.
custodian_binary must resolve on PATH (or as explicit executable path).
check_timeout_seconds must be a positive integer.
resource_identity_fields, when set, must be valid JSON and each resource type must include at least one field path.
Policy YAML must include top-level policies array.
With aws_accounts: every account id is 12 digits and listed once; each account has a role (role_arn, role_name or aws_assume_role_name) or use_ambient_credentials: true, but not its own role and ambient credentials together, and at most one account uses ambient credentials; a role_arn belongs to its account; an external ID is 2 to 1224 characters from [A-Za-z0-9_+=,.@:/-]; unknown account keys are rejected; every policy is an AWS policy. Without aws_accounts, the other aws_* account keys are ignored with a warning.

Example Agent Config (Inline YAML)

plugins:
  cloud_custodian:
    source: ./dist/plugin
    policies:
      - ./policy-bundle
    config:
      policies_yaml: |
        policies:
          - name: ec2-public-ip-check
            resource: aws.ec2
            filters:
              - type: value
                key: PublicIpAddress
                op: not-null
      custodian_binary: custodian
      check_timeout_seconds: "300"
      policy_labels: '{"team":"cloud-security","environment":"prod"}'

Example Agent Config (Path/URL Fallback)

plugins:
  cloud_custodian:
    source: ./dist/plugin
    policies:
      - ./policy-bundle
    config:
      policies_path: file:///etc/ccf/cloud-custodian.yaml
      custodian_binary: /usr/local/bin/custodian

Cross-Account Scanning

Without aws_accounts the plugin scans the account of its own (ambient) credentials, exactly as before. With aws_accounts, it scans only the listed accounts, one after the other, assuming a role in each through Cloud Custodian's --assume and --external-id. The account the plugin runs in (the hub) is scanned only when it is listed with use_ambient_credentials: true. Its id is checked against the accounts in the ARNs of the resources it finds: if they name another account, the hub fails and submits no evidence (the other accounts still run); if no ARN names an account, the plugin logs a warning that it can't verify the id.

For each account:

the role is the entry's role_arn, else arn:<partition>:iam::<id>:role/<role_name>, else the same with aws_assume_role_name;
the external ID is the entry's external_id, else aws_external_id;
the regions are the entry's regions, else aws_regions, else all.

plugins:
  cloud_custodian:
    source: ghcr.io/compliance-framework/plugin-cloud-custodian:<version>
    policies:
      - ghcr.io/compliance-framework/plugin-cloud-custodian-policies:<version>
    config:
      policies_path: file:///etc/ccf/cloud-custodian.yaml
      aws_regions: "eu-west-1,us-east-1"
      aws_assume_role_name: ccf-audit
      aws_external_id: "${env:CCF_CUSTODIAN_EXTERNAL_ID}"
      aws_accounts: |
        [
          {"id": "111111111111", "name": "hub", "use_ambient_credentials": true},
          {"id": "222222222222", "name": "prod", "role_name": "ccf-prod-audit", "external_id": "${env:CCF_PROD_EXTERNAL_ID}"},
          {"id": "333333333333", "name": "dev", "regions": ["eu-west-1"]},
          {"id": "444444444444", "name": "legacy", "role_arn": "arn:aws:iam::444444444444:role/security/ccf-audit"}
        ]

Here the hub uses the agent's own credentials; prod assumes ccf-prod-audit with its own external ID; dev assumes ccf-audit with the global external ID in eu-west-1 only; legacy assumes the given role with the global external ID. The agent resolves the ${env:...} placeholders before the plugin sees them.

What changes in accounts mode:

The account_id label is the scanned account, whatever the resource data says, and account_name is added when the account has a name. Both are reserved: policy_labels can't set them.
The payload gets "account": {"id": "...", "name": "..."}, and resource.account_id is the scanned account.
account_id is part of the evidence UUID seed and of the resource subject id (cloud-custodian-resource/<account>/<type>/<id>), and the subject templates' identity keys are provider, account_id, resource_type, resource_id. The same resource id in two accounts is two resources. Moving an existing deployment to aws_accounts therefore re-keys its evidence streams once. account_name is not part of the seed, so renaming an account keeps its evidence streams.
Each account has its own inventory baseline, Cloud Custodian cache file and output directory.

When an account fails (for example its role can't be assumed, which skips the account's remaining runs), the other accounts still run and submit their evidence. The run then fails with one error whose first line names every failed account, followed, per account, by its role, its errors and the tail of Cloud Custodian's output. The agent records that error as the plugin run's error (its log and the agent run evidence).

Regions

With all (no regions and no aws_regions), Cloud Custodian lists the enabled regions with the hub's credentials (ec2:DescribeRegions), not the target account's, so an opt-in region enabled only in a target account is missed. The plugin logs a warning for each such account; list the regions to scan them. On private networks without a route to the global STS endpoint, set C7N_USE_STS_REGIONAL=true in the agent's environment so Cloud Custodian assumes roles through the regional STS endpoint (for example a VPC endpoint).

IAM

The hub principal (the plugin's own credentials) needs sts:AssumeRole on the target roles. Narrow it to them, for example with an aws:ResourceOrgID condition or a list of the role ARNs, plus ec2:DescribeRegions when any account scans all regions.
Each target role trusts the hub principal with an sts:ExternalId condition, and can also require the session name:

{
  "Version": "2012-10-17",
  "Statement": [{
    "Effect": "Allow",
    "Principal": {"AWS": "arn:aws:iam::111111111111:role/ccf-agent"},
    "Action": "sts:AssumeRole",
    "Condition": {
      "StringEquals": {"sts:ExternalId": "<external id>"},
      "StringLike": {"sts:RoleSessionName": "CloudCustodian@ccf*"}
    }
  }]
}

The target role's permissions are the AWS managed policies SecurityAudit and job-function/ViewOnlyAccess, not ReadOnlyAccess, which can also read data (S3 objects, DynamoDB items, secrets).

Remote configuration

Never list aws_accounts, aws_assume_role_name, aws_external_id, aws_partition or aws_session_name_suffix in the agent's remote_config.overridable_config_flags (and don't use * there for this plugin): a remote overlay could then point the plugin at other accounts or roles without an operator's review.

Policies

The policy bundles that work with this plugin, at their newest releases. The controls are the ones the bundle's policies list in their metadata.

Bundle	Latest version	Controls covered	Compatible plugin lines
`plugin-cloud-custodian-policies`	`v0.2.2`	none listed	latest

Standardized Per-Resource OPA Input

Each resource/check iteration produces one payload with this shape:

{
  "schema_version": "v2",
  "source": "cloud-custodian",
  "check": {
    "name": "ec2-public-ip-check",
    "resource": "aws.ec2",
    "provider": "aws",
    "index": 0,
    "metadata": {}
  },
  "resource": {
    "id": "i-1234567890abcdef0",
    "type": "aws.ec2",
    "provider": "aws",
    "account_id": "123456789012",
    "region": "us-east-1",
    "identity_fields": {
      "InstanceId": "i-1234567890abcdef0"
    },
    "data": {"...": "..."}
  },
  "assessment": {
    "status": "non_compliant",
    "matched": true,
    "inventory_status": "baseline",
    "matched_resource_count": 3,
    "artifact_path": "/tmp/ccf-cloud-custodian-123/001-ec2-public-ip-check",
    "resources_path": "/tmp/ccf-cloud-custodian-123/001-ec2-public-ip-check/ec2-public-ip-check/resources.json"
  },
  "execution": {
    "status": "success",
    "dry_run": true,
    "exit_code": 0,
    "started_at": "2026-03-06T12:00:00Z",
    "ended_at": "2026-03-06T12:00:01Z",
    "duration_ms": 1000,
    "stdout": "...",
    "stderr": "",
    "error": "",
    "errors": []
  },
  "raw_policy": {
    "name": "ec2-public-ip-check",
    "resource": "aws.ec2",
    "filters": []
  }
}

Generated evidence labels include resource_id, resource_type, provider, and any available account_id/region (with aws_accounts: the scanned account's account_id, its account_name, and an account object in the payload, see Cross-Account Scanning). A region in the resource data is used as is; otherwise, when Cloud Custodian wrote per-region output, region is the region directory the resource came from. Global resource types (IAM, CloudFront, Route 53) and resources returned identically by every region (S3 buckets, which ListBuckets returns everywhere) are kept once and get no region from the directory. The resource subject includes the resource identifier as a link and a resource_id property. When the AWS resource identifier is an ARN, generated evidence includes an AWS Resource Explorer link using an exact id:<arn> search. AWS S3 bucket names are parsed as arn:aws:s3:::<bucket-name> for Resource Explorer links. AWS Route53 hosted zone identifiers such as /hostedzone/Z123 are normalized to full ARNs such as arn:aws:route53:::hostedzone/Z123.

assessment.inventory_status is baseline for resources found in the unfiltered inventory run. If a policy returns a resource that is not present in the baseline, the plugin still evaluates it as non_compliant and sets inventory_status to missing_from_baseline.

Resource Identity

resource.id (and the resource_id label) is the first non-empty field from, in order:

the type's resource_identity_fields, when configured;
the built-in field list for the AWS type below: the ARN first, then the natural id, following the resource metadata of Cloud Custodian 0.9.49;
the generic fields Arn, ARN, arn, Id, ID, id, InstanceId, instance_id, InstanceID, ResourceId, resource_id, Name, name;
a SHA-256 of the whole resource, as a last resort.

Type	Fields
`aws.app-elb`	`LoadBalancerArn`
`aws.backup-plan`	`BackupPlanArn`, `BackupPlanName`
`aws.backup-vault`	`BackupVaultArn`, `BackupVaultName`
`aws.cache-cluster`	`ARN`, `CacheClusterId`
`aws.distribution`	`ARN`, `Id`
`aws.dynamodb-table`	`TableArn`, `TableName`
`aws.ebs`	`VolumeId`
`aws.ec2`	`InstanceId`
`aws.ecs-service`	`serviceArn`
`aws.efs`	`FileSystemArn`, `FileSystemId`
`aws.eks`	`arn`, `name`
`aws.firewall`	`FirewallArn`, `FirewallName`
`aws.hostedzone`	`Id` (normalized to an ARN)
`aws.iam-group`	`Arn`, `GroupName`
`aws.iam-policy`	`Arn`, `PolicyId`
`aws.iam-role`	`Arn`, `RoleName`
`aws.iam-user`	`Arn`, `UserName`
`aws.kms-key`	`Arn`, `KeyId`
`aws.lambda`	`FunctionArn`, `FunctionName`
`aws.log-group`	`arn`, `logGroupName`
`aws.rds`	`DBInstanceArn`, `DBInstanceIdentifier`
`aws.rds-cluster`	`DBClusterArn`, `DBClusterIdentifier`
`aws.s3`	`Name`
`aws.secrets-manager`	`ARN`, `Name`
`aws.sns`	`TopicArn`
`aws.sqs`	`QueueArn`, `QueueUrl`
`aws.transfer-server`	`Arn`, `ServerId`
`aws.wafv2`	`ARN`, `Id`

The id never depends on the annotations Cloud Custodian adds to matched resources (c7n:MatchedFilters and the like), so a check's matched resource is the same resource as its inventory baseline record.

provider extraction rule:

aws.s3 -> aws
non <provider>.<resource> formats -> unknown

Policy Data

Any policy_data set on this plugin's block in the agent configuration is passed to every policy evaluation and is available to Rego as data.<key>:

plugins:
  cloud_custodian:
    # ...plugin settings as above...
    policy_data:
      allowed_regions:
        - eu-west-1
        - us-east-1

How it works​

Behavior Overview​

Safety Model​

Requirements​

Operational Prerequisites​

Configuration​

Example Agent Config (Inline YAML)​

Example Agent Config (Path/URL Fallback)​

Cross-Account Scanning​

Regions​

IAM​

Remote configuration​

Policies​

Standardized Per-Resource OPA Input​

Resource Identity​

Policy Data​