Execution Analysis

Technical reference. This page is for integrations, advanced configuration, or platform operations. For portal tasks, start with the user guide.

Execution analysis is the governed, metadata-only summary for a single run. It starts from the same execution, evidence, cost, retrieval, policy, subagent, and provenance records used by the portal, then returns bounded facts for a chosen workflow lens.

Use execution analysis before reading the deeper evidence graph. The analysis response answers the first triage questions without fetching raw prompt, completion, retrieval chunk, tool payload, secret, or provider payload content.

Surfaces

Lenses and includes

view selects the audience lens:

include controls optional sections. Repeat it or comma-separate values:

curl "$ORQESTRA_API_URL/v1/executions/$EXECUTION_ID/analysis?view=engineer&include=spans&include=evidence"

Available include values:

Span fields

Each entry in spans[] (from include=spans) can carry a canonical flow-step key and an optional flow scope:

The route serializes with exclude_none, so either optional field is absent (not JSON null) whenever it has no value. For a step-bearing span, an absent subexecution_id means the step belongs to the root flow. Treat an absent source_step_id as “no canonical reference” — never as “this is a pre-migration artifact, apply a display-id fallback.” source_step_id is absent for:

• any non-step span (execution, policy_decision, a retrieval event span, evidence_write, cost, and similar non-step kinds);
• a span whose stored id was malformed (non-string, empty, or over 256 characters) — defense-in-depth against corrupted stored evidence, not an authoring case;
• a pre-migration execution_trace.v2 artifact written before this field existed — the only case that is actually legacy.

Consumers that need to recover a span’s flow step should key on the tuple (subexecution_id, source_step_id) when source_step_id is present. The absent scope represents the root flow, while a present scope keeps a colliding child execution’s step distinct. span_id remains an opaque, sanitized display id and is not guaranteed to reverse to the original step id for step ids containing punctuation or longer than 120 characters.

Source refs are also resolved within the scope of their containing span; do not flatten them into a run-wide map keyed only by (type, ref_id). In particular, a telemetry_step ref uses the trace step id as ref_id. On a child-scoped span its optional label repeats that span’s subexecution_id, so a colliding root and child ref remain distinguishable when exported independently. The root ref keeps its existing absent-or-null label. Treat this label convention as specific to telemetry_step; labels on other source-ref types remain descriptive metadata rather than scope.

Operator triage

Start with the operator lens:

orqestractl execution analysis "$EXECUTION_ID" --view operator --include evidence

Check:

If evidence is missing or expired, do not infer payload contents. Use source refs and artifact metadata to decide whether to rerun, request approval, or open the deeper evidence graph.

Engineer debugging

Use the engineer lens when the run failed or produced a degraded output:

analysis = await client.executions.analysis(
    execution_id,
    view="engineer",
    include=["spans", "provenance", "cost", "graph_refs"],
)

Debug from facts in this order:

1. Read failure_analysis.failure_span_id and match it to a span in spans.
2. Confirm failure_analysis.basis contains source refs for the claim.
3. Inspect adjacent spans for retrieval, policy, tool, provider, or guardrail context.
4. Use cost.telemetry_summary for cost and latency context.
5. Follow graph_refs only when the analysis does not contain enough source refs to explain the failure.

redaction on a span is part of the answer. metadata_only, payload_redacted, payload_expired, and payload_not_captured mean the debugging path must use IDs, labels, status, timings, and source refs instead of raw payload content.

Compliance and risk review

Use the compliance lens for evidence readiness:

orqestractl --format json execution analysis "$EXECUTION_ID" \
  --view compliance \
  --include evidence \
  --include provenance \
  --include graph_refs

Review:

Treat the analysis response as the source for bounded summary facts. Treat the evidence graph as the deeper audit traversal. Treat raw artifact payloads as a separate permissioned read that is not required for normal analysis validation.

MCP workflow

AI Dev Kit agents should call analysis before graph traversal:

{
  "tool": "get_execution_analysis",
  "arguments": {
    "execution_id": "exec_123",
    "view": "engineer",
    "include": ["spans", "evidence", "provenance"]
  }
}

Only call get_evidence_graph when the analysis source refs are insufficient for the investigation. Only read artifact payloads when the user’s task and existing permissions require payload inspection.

Validation examples

Validation should assert governed metadata and completeness signals, not raw sensitive content. A validation helper can use the SDK response directly:

REQUIRED_SPANS = {"execution", "policy_decision", "tool_call"}
ALLOWED_REDACTIONS = {
    "metadata_only",
    "payload_available",
    "payload_redacted",
    "payload_expired",
    "payload_not_captured",
    "not_applicable",
}
 
 
def assert_analysis_complete_enough(analysis: dict) -> None:
    span_kinds = {span["kind"] for span in analysis.get("spans", [])}
    assert REQUIRED_SPANS <= span_kinds
 
    failure = analysis["failure_analysis"]
    assert failure["confidence"] in {"observed", "derived", "unknown"}
    if failure["confidence"] == "unknown":
        assert failure.get("unknown_reason")
    else:
        assert failure.get("basis")
 
    family_status = {row["family"]: row["status"] for row in analysis["evidence_status"]}
    assert family_status["step_trace"] in {
        "complete",
        "partial",
        "redacted",
        "expired",
        "missing",
        "not_required",
        "unknown",
    }
 
    for span in analysis.get("spans", []):
        assert span["redaction"] in ALLOWED_REDACTIONS
        assert "payload" not in span
        assert "prompt" not in span
        assert "completion" not in span
 
    assert "raw_provider_payload" not in analysis
    assert "secret" not in analysis

For retrieval, policy, cost, subagent, and provenance checks, assert that the metadata and source refs exist:

def assert_source_refs_are_metadata_only(analysis: dict) -> None:
    refs = []
    refs.extend(analysis["failure_analysis"].get("basis", []))
    refs.extend(analysis.get("graph_refs", {}).get("source_refs", []))
 
    for ref in refs:
        assert ref["type"] in {
            "execution",
            "evidence_artifact",
            "policy_decision",
            "decision_obligation",
            "retrieval_event",
            "release",
            "release_asset_binding",
            "subexecution",
            "context_lineage",
            "governance_event",
            "log_correlation",
        }
        assert ref["ref_id"]
        assert "payload" not in ref

These checks prove the trace/evidence contract has the expected shape while leaving raw content behind the existing artifact permission boundary.

Boundaries

• Workspace isolation and permissions are enforced by the control-plane API.
• Analysis reads do not widen access to another workspace.
• Analysis does not fetch artifact payloads by default.
• Redacted, expired, missing, not-required, and unknown states are distinct.
• Source facts and derived analysis are separate fields; do not present derived claims as observed facts.
• The portal route and API response are the source of truth for the analysis vocabulary.