HTTP Collector

Fetches an HTTP endpoint and evaluates the configured policies against the response, for health monitoring and compliance validation.

This plugin is intended to be run as part of an agent to validate HTTP endpoints against defined compliance policies.

This page is for operators who run the plugin-http-collector plugin with the CCF Agent, and for developers who write policies for it. It documents v0.2.0 and is generated from the plugin's README at v0.2.0.

Configuration

The plugin supports the following configuration options:

• url (required): The HTTP endpoint URL to test
• method: HTTP method to use (default: "GET")
• timeout: Request timeout in milliseconds (default: 5000)
• basic_auth: Enable basic authentication (default: false)
• basic_auth_username: Username for basic auth
• basic_auth_password: Password for basic auth
• additional_headers: Additional headers in format "Key1: Value1;Key2: Value2"
• check_certificate: Validate SSL certificates (default: true)
• body_regex_pattern: Regex pattern to match against response body

Evidence collected

Subject

The plugin speaks plugin protocol v2. In Init it registers one subject template with the API, and the risk templates of the configured policies:

Field	Value
Name	http-endpoint
Type	COMPONENT
Title	HTTP Endpoint: {{ .method }} {{ .url }}
Description	HTTP endpoint {{ .url }} checked with {{ .method }}
Purpose	Represents an HTTP endpoint whose availability and response are checked
Identity labels	url, method

Label schema: url is the URL requested, method the HTTP method used (upper case). Every evidence carries both labels, so the API links it to one subject per endpoint and method. Agents that don't call Init (protocol v1) get the same evidence.

Policies

The policy bundles that work with this plugin, at their newest releases. The controls are the ones the bundle's policies list in their metadata.

Bundle	Latest version	Controls covered	Compatible plugin lines
plugin-http-collector-policies	v0.2.0	none listed	latest

The plugin evaluates the policy bundles listed under policies in the agent's plugin config, usually the plugin-http-collector-policies bundle:

plugins:
  health-checker:
    source: ghcr.io/compliance-framework/plugin-http-collector:<version>
    policies:
      - ghcr.io/compliance-framework/plugin-http-collector-policies:<version>
    config:
      url: https://example.com/health

Each run makes one request and evaluates every policy package in every bundle against it, producing one evidence per package. Without policies the plugin makes the request but produces no evidence.

Policies are Rego v1 and live under the compliance_framework.http_collector package. They report failures with a violation set, and set title, description and optionally labels (merged into the evidence labels):

package compliance_framework.http_collector.ensure_success_status

import rego.v1

violation contains {} if {
	not input.success
}

title := "HTTP endpoint should return successful status codes"
description := "HTTP endpoints should return 2xx status codes to ensure proper service functionality"
labels := {"severity": "high"}

Policy input

The input is the response, as JSON:

{
  "status_code": 200,
  "status": "200 OK",
  "headers": {
    "Content-Type": ["application/json"],
    "Content-Length": ["1234"]
  },
  "body": "response body content",
  "response_time_ms": 150,
  "success": true,
  "error": "error message if request failed",
  "matched_regex": true,
  "body_regex_pattern": "pattern used for matching"
}

• headers uses Go's canonical header names (Content-Type), each with a list of values.
• success is true for a 2xx status.
• error is set only when no response was received (the request could not be built or sent, or the body could not be read). Then success is false, status_code is 0 (unless the body read failed), and the plugin still evaluates the policies, so the failure is recorded as evidence, before Eval reports FAILURE.
• matched_regex and body_regex_pattern are set only when body_regex_pattern is configured (matched_regex is left out when false).

Policy data

policy_data from the agent config is passed to the policies as data.

Evidence

Each evidence carries:

• labels provider: http, url and method (with the policy's _policy and its own labels). Together with the policy they derive the evidence UUID, so the same endpoint and policy keep one evidence history across runs;
• props http_url, http_method, status_code, response_time_ms and success, plus error when the request failed and regex_pattern and regex_matched when body_regex_pattern is configured;
• the HTTP Health Check activity (client settings, request, response size and time), followed by the policy manager's activities;
• the endpoint URL as a component subject, and start and end times of the request.

The response itself (the policy input above) is recorded with the evidence's policy evaluation.