Updated on 2026-08-14

This commit is contained in:
Tangem 2026-06-22 13:22:42 +05:00
parent f4987988d3
commit 5b27f5191e
13 changed files with 2186 additions and 0 deletions

View file

@ -0,0 +1,50 @@
# Agent Toolkit — approach & contents
A system for building **continuable, orchestratable** Claude Code subagents, with an
Android agent set and an analyzer to keep agents healthy.
## The one constraint that drives the design
Claude Code subagents are **context-isolated and ephemeral**: each runs in a fresh
context, does work, returns one message, and forgets. They can't see the parent
conversation or each other. Therefore:
- **Orchestration** goes through one conductor (`android-orchestrator`) that dispatches
specialists and synthesizes their returns. Specialists never talk to each other.
- **Context lives on disk**, not in chat. The root `CLAUDE.md` is the project's
architecture overview (modules, layers, dependency rules, entry points) — every agent
reads it on entry.
## The contract every agent follows
- **Entry:** read the root `CLAUDE.md` before doing anything.
- **Exit:** return the `HANDOFF` block (asked / did / state / impact / blockers / next /
how-to-verify).
This contract is the whole answer to "a user can resume at any time with minimal effort":
each HANDOFF block makes its step legible cold, so the orchestrator (and a human) can
synthesize where things stand and what to do next.
## Contents
```
agent-toolkit/
README.md ← this file (the approach)
RUBRIC.md ← 10-dimension agent quality spec
analyze_agents.py ← dependency-free linter that scores agents against the rubric
templates/
HANDOFF.md ← return-contract template
~/.claude/agents/
android-orchestrator.md ← conductor: plans, dispatches, synthesizes HANDOFFs
android-feature-builder.md ← implements within the architecture
android-code-reviewer.md ← Android-pitfall correctness review (read-only)
android-build-test.md ← Gradle build/test, iterate to green
android-architecture-guardian.md ← enforces boundaries & layering
agent-auditor.md ← meta-agent: audits/improves other agents via RUBRIC.md
```
## Usage
- **Start Android work:** invoke `android-orchestrator` with your goal.
- **Audit agents (tooling):** `python3 ~/.claude/agent-toolkit/analyze_agents.py`
- **Audit agents (judgment):** invoke `agent-auditor` for substance-level review + fixes.
## Extending to other stacks
The pattern is stack-agnostic. Clone the android-* set, swap the domain checklists
(build commands, framework pitfalls) in each specialist, keep the orchestrator,
contracts, and rubric unchanged.

View file

@ -0,0 +1,88 @@
# Agent Quality Rubric
A scoring spec for Claude Code subagents (`.claude/agents/*.md`). Each dimension is
scored **0 (absent) / 1 (partial) / 2 (solid)**. Max score = 20.
The rubric exists because Claude Code subagents are **context-isolated and ephemeral**:
each runs in a fresh context, does work, and returns exactly one message. They cannot
see the parent conversation or each other. Most agent-quality problems trace back to
authors forgetting this. The rubric is built to catch those problems.
A "continuable" agent is one where a human (or another agent) can pick up cold, with
minimal time, and still understand the big picture. Dimensions 46 protect that property.
---
## Dimensions
### 1. Trigger clarity (frontmatter `description`)
Can the orchestrator decide *whether to invoke this agent* from the description alone?
- **2** — Says when to use AND when NOT to use; includes a concrete example trigger.
- **1** — Says when to use, but no negative guidance or examples.
- **0** — Vague ("helps with code") or missing.
### 2. Tool scoping (frontmatter `tools`)
Least privilege. A read-only analyzer must not hold `Write`/`Edit`.
- **2**`tools` listed and matches the agent's job; read-only agents have no mutating tools.
- **1**`tools` listed but broader than needed.
- **0** — No `tools` field (silently inherits everything), or obvious over-grant.
### 3. Single responsibility
One clear job. Agents that "do everything" can't be orchestrated or audited.
- **2** — One crisp mandate; explicitly defers adjacent work to other agents.
- **1** — Mostly focused but with scope creep.
- **0** — Grab-bag of unrelated duties.
### 4. Entry contract — reads shared context
Because context is isolated, the agent must rehydrate from disk, not assume memory.
- **2** — Explicitly reads the root `CLAUDE.md` (or named inputs) as step one.
- **1** — Reads some context but not the project's architecture overview.
- **0** — Assumes it already knows the project; no entry read.
### 5. Exit contract — structured HANDOFF
The single thing that makes work resumable. Output must be legible cold.
- **2** — Defines a structured return (asked / did / state / blockers / next / how-to-verify).
- **1** — Returns a summary but unstructured.
- **0** — No defined output shape.
### 6. Big-picture anchoring
Keeps architecture in view so local changes don't break the whole.
- **2** — Reasons against the architecture in `CLAUDE.md`; flags structural impact in its HANDOFF.
- **1** — Mentions architecture but doesn't tie decisions to it.
- **0** — Purely local; no architectural awareness.
### 7. Guardrails & escalation
Knows its limits and stop conditions.
- **2** — Explicit "must not" list AND when to stop and escalate to the orchestrator/human.
- **1** — Some guardrails, no escalation path (or vice versa).
- **0** — None.
### 8. Self-verification
Tells how its own output should be checked.
- **2** — Concrete verification (run these tests / this build / these checks).
- **1** — Says "verify" without specifics.
- **0** — None.
### 9. Determinism of process
A repeatable procedure, not vibes.
- **2** — Numbered, ordered steps the agent follows every run.
- **1** — Loose guidance.
- **0** — Freeform.
### 10. Conciseness & specificity
No filler; concrete over abstract.
- **2** — Tight, every line earns its place, concrete nouns/paths.
- **1** — Some bloat or vague phrasing.
- **0** — Long, generic, or contradictory.
---
## Score bands
- **1820** — Production-ready. Orchestratable and continuable.
- **1317** — Usable; fix the 0/1 dimensions.
- **812** — Risky; likely breaks under orchestration or loses context.
- **07** — Rewrite.
## How to use
- Script: `python3 ~/.claude/agent-toolkit/analyze_agents.py <path-or-glob>`
- Meta-agent: invoke `agent-auditor` — it reads this rubric and proposes concrete edits.

View file

@ -0,0 +1,277 @@
#!/usr/bin/env python3
"""
analyze_agents.py grade Claude Code subagents against RUBRIC.md.
Heuristic, dependency-free linter. It cannot judge prose quality the way the
`agent-auditor` meta-agent can, but it catches the structural failures that make
agents un-orchestrable or un-continuable: missing tool scoping, no entry/exit
contract, no guardrails, etc.
Usage:
python3 analyze_agents.py # scan ./.claude/agents and ~/.claude/agents
python3 analyze_agents.py path/to/agent.md # one file
python3 analyze_agents.py 'dir/*.md' # a glob
python3 analyze_agents.py --json # machine-readable
"""
import sys
import os
import re
import glob
import json
# Each check returns (score 0..2, message). Mirrors RUBRIC.md dimensions.
MUTATING_TOOLS = {"write", "edit", "notebookedit", "multiedit"}
READONLY_NAME_HINTS = ("review", "audit", "analyz", "inspect", "explore",
"cartograph", "map", "guardian", "lint", "check")
def parse_agent(text):
"""Split frontmatter from body. Returns (meta, body).
Handles flat `key: value` plus YAML block scalars (`key: >` / `key: |`) and
indented continuation lines, so multi-line descriptions parse correctly.
"""
meta, body = {}, text
m = re.match(r"^---\s*\n(.*?)\n---\s*\n?(.*)$", text, re.DOTALL)
if not m:
return meta, body
raw, body = m.group(1), m.group(2)
lines = raw.splitlines()
i = 0
while i < len(lines):
line = lines[i]
if not line.strip() or line.lstrip().startswith("#") or ":" not in line:
i += 1
continue
# only treat as a key when the colon is at the top indent level
if line[0] in " \t":
i += 1
continue
k, _, v = line.partition(":")
key, v = k.strip().lower(), v.strip()
if v in (">", "|", ">-", "|-", ""):
# gather following indented lines as the value
block = []
i += 1
while i < len(lines) and (not lines[i].strip() or lines[i][:1] in " \t"):
block.append(lines[i].strip())
i += 1
meta[key] = " ".join(b for b in block if b).strip()
else:
# strip one layer of matching surrounding quotes, e.g. tools: "Read, Edit"
if len(v) >= 2 and v[0] == v[-1] and v[0] in "\"'":
v = v[1:-1]
meta[key] = v
i += 1
return meta, body
def has_any(text, *words):
low = text.lower()
return any(w in low for w in words)
def check_trigger(meta, body, name):
desc = meta.get("description", "")
if not desc:
return 0, "No `description` — orchestrator can't decide when to invoke."
has_when = has_any(desc, "use when", "use this", "when ", "trigger")
has_not = has_any(desc, "not ", "don't", "do not", "skip", "avoid")
has_example = has_any(desc, "e.g.", "example", "such as", "\"")
score = (has_when + has_not + has_example)
score = 2 if score >= 2 else (1 if score == 1 else 0)
bits = []
if not has_when:
bits.append("add explicit 'use when ...'")
if not has_not:
bits.append("add 'do NOT use for ...'")
if not has_example:
bits.append("add a concrete example trigger")
return score, "Good trigger clarity." if score == 2 else "; ".join(bits)
def check_tools(meta, body, name):
tools = meta.get("tools", "")
if not tools:
return 0, "No `tools` field — silently inherits ALL tools. Scope it."
toolset = {t.strip().lower() for t in re.split(r"[,\s]+", tools) if t.strip()}
readonly_named = any(h in name.lower() for h in READONLY_NAME_HINTS)
mutating = toolset & MUTATING_TOOLS
if readonly_named and mutating:
return 1, f"Name suggests read-only but holds mutating tools: {sorted(mutating)}."
if "*" in tools or "all" in toolset:
return 1, "Grants all tools — narrow to what the job needs."
return 2, "Tools are scoped."
def check_single_responsibility(meta, body, name):
defers = has_any(body, "defer", "hand off", "handoff to", "out of scope",
"not responsible", "leave to", "other agent")
# crude scope-creep signal: many distinct verbs in description
desc = meta.get("description", "").lower()
verbs = sum(desc.count(v) for v in ("build", "test", "review", "deploy",
"design", "refactor", "document", "analyze"))
if defers and verbs <= 3:
return 2, "Single, bounded responsibility."
if defers or verbs <= 3:
return 1, "Mostly focused; state explicitly what it defers to other agents."
return 0, "Looks like a grab-bag — split it or define one mandate."
def check_entry(meta, body, name):
reads_context = has_any(body, "claude.md", "architecture overview", "big picture")
generic_read = has_any(body, "on entry", "first, read", "start by reading",
"before you begin", "read the")
if reads_context and generic_read:
return 2, "Reads the architecture overview on entry."
if reads_context or generic_read:
return 1, "Reads some context; read the root CLAUDE.md as step one."
return 0, "No entry read — will assume context it doesn't have (isolation bug)."
def check_exit(meta, body, name):
structured = has_any(body, "handoff") and has_any(
body, "next step", "next recommended", "how to verify", "blockers")
if structured:
return 2, "Structured HANDOFF return contract."
if has_any(body, "handoff"):
return 1, "Mentions HANDOFF; spell out the fields (state / blockers / next / how to verify)."
return 0, "No exit contract — output won't be resumable."
def check_big_picture(meta, body, name):
architecture = has_any(body, "claude.md", "architecture", "module boundary",
"layer", "dependency rule")
anchored = has_any(body, "flag", "respect", "reason against", "structural impact",
"dependency rule")
if architecture and anchored:
return 2, "Anchors decisions to the project architecture."
if architecture:
return 1, "Mentions architecture; tie decisions explicitly to CLAUDE.md."
return 0, "No big-picture anchoring."
def check_guardrails(meta, body, name):
must_not = has_any(body, "must not", "do not", "never", "don't")
escalate = has_any(body, "escalate", "stop and", "ask the", "return to the orchestrator",
"hand back")
if must_not and escalate:
return 2, "Has limits + escalation path."
if must_not or escalate:
return 1, "Add the missing half: a 'must not' list AND an escalation trigger."
return 0, "No guardrails or stop conditions."
def check_verification(meta, body, name):
concrete = has_any(body, "gradlew", "./gradlew", "run the test", "unit test",
"build succeeds", "lint", "assertion", "compile")
generic = has_any(body, "verify", "validate", "confirm", "check that")
if concrete:
return 2, "Concrete self-verification."
if generic:
return 1, "Says verify but no concrete method."
return 0, "No self-verification."
def check_determinism(meta, body, name):
numbered = len(re.findall(r"^\s*\d+[\.\)]\s+", body, re.MULTILINE))
if numbered >= 3:
return 2, "Has an ordered procedure."
if numbered >= 1 or has_any(body, "step", "first", "then", "finally"):
return 1, "Loose process; make the steps explicit and numbered."
return 0, "No defined procedure."
def check_conciseness(meta, body, name):
words = len(body.split())
vague = sum(body.lower().count(p) for p in (
"as needed", "appropriate", "etc.", "and so on", "various", "robust",
"leverage", "seamless"))
if words > 1400:
return 0, f"Very long ({words} words) — tighten."
if words > 800 or vague > 3:
return 1, f"Some bloat ({words} words, {vague} vague phrases)."
return 2, f"Tight ({words} words)."
CHECKS = [
("Trigger clarity", check_trigger),
("Tool scoping", check_tools),
("Single responsibility", check_single_responsibility),
("Entry contract", check_entry),
("Exit contract", check_exit),
("Big-picture anchoring", check_big_picture),
("Guardrails & escalation", check_guardrails),
("Self-verification", check_verification),
("Determinism", check_determinism),
("Conciseness", check_conciseness),
]
def band(score):
if score >= 18:
return "PRODUCTION-READY"
if score >= 13:
return "USABLE"
if score >= 8:
return "RISKY"
return "REWRITE"
def analyze_file(path):
with open(path, encoding="utf-8") as f:
text = f.read()
meta, body = parse_agent(text)
name = meta.get("name", os.path.basename(path).rsplit(".", 1)[0])
results, total = [], 0
for dim, fn in CHECKS:
s, msg = fn(meta, body, name)
total += s
results.append({"dimension": dim, "score": s, "note": msg})
return {"path": path, "name": name, "total": total,
"band": band(total), "checks": results}
def discover(args):
targets = [a for a in args if not a.startswith("-")]
if targets:
files = []
for t in targets:
files.extend(glob.glob(os.path.expanduser(t)) if any(c in t for c in "*?[")
else [os.path.expanduser(t)])
return [f for f in files if f.endswith(".md")]
files = []
for d in (".claude/agents", os.path.expanduser("~/.claude/agents")):
files.extend(sorted(glob.glob(os.path.join(d, "*.md"))))
return files
def print_report(reports):
for r in reports:
print(f"\n{'='*68}\n{r['name']}{r['total']}/20 [{r['band']}]\n{r['path']}\n{'-'*68}")
for c in r["checks"]:
mark = {0: "", 1: "~", 2: ""}[c["score"]]
print(f" {mark} {c['dimension']:<26} {c['score']}/2 {c['note']}")
if len(reports) > 1:
print(f"\n{'='*68}\nSUMMARY")
for r in sorted(reports, key=lambda x: x["total"]):
print(f" {r['total']:>2}/20 [{r['band']:<16}] {r['name']}")
def main():
args = sys.argv[1:]
files = discover(args)
if not files:
print("No agent .md files found. Pass a path/glob, or run where "
".claude/agents exists.", file=sys.stderr)
sys.exit(1)
reports = [analyze_file(f) for f in files]
if "--json" in args:
print(json.dumps(reports, indent=2))
else:
print_report(reports)
if __name__ == "__main__":
main()

View file

@ -0,0 +1,24 @@
# HANDOFF block (the return contract)
> Every specialist returns this exact shape as its final message. It is what makes work
> resumable cold. Keep it short — links and paths over prose. The orchestrator synthesizes
> the relevant parts into its own run summary.
```
## HANDOFF — <agent-name><YYYY-MM-DD HH:MM>
**Asked:** one line — what this run was dispatched to do.
**Did:** bullet list of concrete actions. Reference files as path:line.
- …
**State now:** build = pass/fail · tests = N pass / M fail · what compiles, what doesn't.
**Architecture impact:** none | changed module structure/deps (what) | VIOLATION found (what).
**Blockers / open questions:** decisions or info needed before continuing. "none" if clean.
**Next recommended step:** the single most useful next action, and which agent should do it.
**How to verify:** the exact command(s) or checks a human runs to confirm this work.
```