Sitelet https://github.com/InternalDeveloperPlatform/pe-agentic-course/tree/main/module5
Skip to content

module5

Directory actions

More options

Directory actions

More options

Latest commit

 

History

History

module5

README.md

Module 5 — Intelligent CI/CD and Adaptive Delivery

What You Will Build

A two-agent CI/CD quality gate system:

  1. triage_agent.py (blocking gate) — runs before deploy, evaluates the release candidate against six configurable thresholds in quality-gates.json, and returns APPROVE, APPROVE_WITH_CONDITIONS, or REJECT.
  2. monitor.py (post-deploy watchdog) — runs after deploy on a timed check, evaluates live production signals, and decides whether to recommend an immediate rollback.

This is deliberate separation of concerns: the gate prevents bad deploys; the monitor catches what slips through.

Files

File Purpose
triage_agent.py Demo file — fully implemented pre-deploy quality gate agent
monitor.py Demo file — fully implemented post-deploy monitor agent
quality-gates.json Threshold configuration (edit this to change gate behaviour)
sample_data.json Sample pipeline results (agent input for both scripts)
agent-config.yml Model and output schema
solutions/solution.py Reference implementation — read this only after your own attempt

Setup

# From the repo root
export ANTHROPIC_API_KEY=your_key_here
python module1/verify_setup.py

Run the Quality Gate Agent

# Mock mode — see expected output shape:
python module5/triage_agent.py --mock

# Live call (evaluates sample_data.json against quality-gates.json):
ANTHROPIC_API_KEY=sk-... python module5/triage_agent.py

Run the Post-Deploy Monitor

# Mock mode:
python module5/monitor.py --mock

# Live call:
ANTHROPIC_API_KEY=sk-... python module5/monitor.py

Quality Gates (from quality-gates.json)

Gate Metric Threshold Blocks Deploy?
Unit Test Coverage coverage_pct ≥ 95% No
Branch Coverage coverage_branch_pct ≥ 80% No
SAST High Findings security_scan.high = 0 Yes
Lighthouse Score lighthouse_score ≥ 85 No
P95 Latency Delta latency_p95_delta_pct ≤ 10% Yes
Cost Per Request Delta cost_per_request_delta_pct ≤ 10% No

Edit quality-gates.json to adjust thresholds without changing any code.

Expected Output (Quality Gate — borderline case)

{
  "decision": "APPROVE_WITH_CONDITIONS",
  "confidence": "HIGH",
  "rationale": "All critical gates pass. Coverage 74.1% is below threshold but has not regressed. Friday deploy window elevates risk.",
  "blocking_issues": [],
  "conditions": [
    "Coverage must not regress below 74% in the next three PRs",
    "Deploy should target off-peak hours (before 14:00 UTC)"
  ],
  "risk_score": "MEDIUM",
  "recommended_deploy_window": "Before 14:00 UTC today or defer to Monday",
  "escalate": false
}

Full result saved to output/output_module5.json.

Exercise

Part A — Read and run the quality gate: triage_agent.py is fully implemented — both the SYSTEM_PROMPT and run_agent() are complete. Study the implementation before running it: read SYSTEM_PROMPT to understand the gate criteria, then trace how run_agent() loads sample_data.json, calls ask(), and prints the structured decision.

python module5/triage_agent.py --mock                     # shows expected output shape
ANTHROPIC_API_KEY=sk-... python module5/triage_agent.py   # live run against Claude

Part B — Threshold experiment: Edit quality-gates.json — lower the threshold for test_coverage from 95 to 70. Re-run triage_agent.py and observe how the gate decision changes. No code edit required, only the config file. This demonstrates the architecture: the agent reads thresholds as data, not as hardcoded logic.

See solutions/solution.py for a version with an extended six-dimension gate including a Change Risk dimension not present in the base implementation.

Key Takeaway

  • The gate and the monitor are intentionally two separate agents with two separate failure modes.
  • The gate is preventive — it blocks a bad deploy before it reaches production. The monitor is reactive — it catches the failures that slip through.
  • A single agent trying to do both jobs would conflate pre-deploy reasoning (static analysis of pipeline results) with post-deploy reasoning (live production signals).
  • Keeping them separate means you can tune each agent's confidence threshold and system prompt independently.
  • The monitor's rollback_recommended decision is a different kind of judgment than the gate's decision — one is about risk, the other is about live impact.

GitHub Actions

Workflow file: .github/workflows/module5-quality-gate.yml

Property Value
Workflow name Module 5 — Quality Gate
Trigger Push to module5/** or shared/**, or manual via Actions tab
Script run python module5/triage_agent.py
Output artifact module5-output → output/output_module5.json

The workflow runs the quality gate agent automatically on every push to module5/ or shared/. This mirrors how a real pre-deploy gate works: any change to the agent or its config triggers a fresh evaluation against the pipeline results in sample_data.json.

Note that monitor.py (the post-deploy watchdog) is not included in this workflow — it is designed to run on a timed schedule or post-deploy hook, not on code push. Run it locally to see the rollback decision.

Prerequisite: Add your API key as a repository secret named ANTHROPIC_API_KEY (Settings → Secrets and variables → Actions → New repository secret).

Success Criteria

  • triage_agent.py --mock runs cleanly and shows expected output shape
  • Live run returns decision of APPROVE, APPROVE_WITH_CONDITIONS, or REJECT
  • monitor.py returns rollback_recommended (boolean) with a severity
  • Editing quality-gates.json changes the gate decision without code changes
  • Full output saved to output/output_module5.json
  • GitHub Actions workflow completes and module5-output artifact is attached to the run
  • If stuck, see solutions/solution.py