54 lines
8.5 KiB
Markdown
54 lines
8.5 KiB
Markdown
---
|
|
name: skill-check
|
|
description: Routine compliance, script, hook, flow-efficiency, and cost-efficiency audit of the whole jsc skill set with no change request in hand. Sync every domain repo from the Gitea canonical marketplace, validate scripts and hook smoke, audit every skill against guidelines.md, then review parallelism, tool extraction, repeated interaction, redundant checks, misplaced gates, and avoidable token, sub-agent, API, scan, or interaction cost. Confirm compliance fixes and optimization suggestions before applying them, re-check until accepted fixes pass, then open a PR per affected repo via jsc-git pr. Use for periodic or on-demand skill-set checks; not for applying a change request (use skillset-update) or editing one skill (use skill-update).
|
|
---
|
|
|
|
# skill-check — audit compliance, flow efficiency, and cost efficiency
|
|
|
|
Single source of guidelines: [`../../references/guidelines.md`](../../references/guidelines.md).
|
|
|
|
## Flow
|
|
|
|
1. Run `tools/sync-domains.sh` to sync every domain repo of the Gitea canonical marketplace. Completion condition: the script exits 0 and prints one `domain<TAB>path` line per marketplace domain — exit 0 is the only code that means every repo is present and current. Exit 3 means some repos were not updated: reconcile every path named on stderr (commit or stash the dirty tree, or fix the failing pull) and rerun; when the user confirms a dirty tree is intentional local work, record that decision and continue on the local version — never read exit 3 as current. Exit 2 means a domain could not be cloned and exit 1 means the canonical marketplace was unreadable — resolve either before continuing.
|
|
2. Validate scripts and hooks before reading skill text:
|
|
1. For every synced domain repo, run `find {domain-path}/tools {domain-path}/hooks -type f -name '*.sh' -exec sh -n {} \;` for directories that exist. Report each script that fails with path and parser output. Missing `tools/` or `hooks/` directories are not failures.
|
|
2. For every shell script directly named by a touched or audited SKILL.md, confirm the script exists, is executable when it is meant to be called directly, and documents or implements every exit code the skill routes. Report evidence as `skill file:line -> script path`.
|
|
3. When the `jsc-hooks` domain is present, run `jsc-hooks/tools/wire-cli.sh smoke {cli}` for every CLI reported by `jsc-cli/tools/detect-clis.sh`. When no CLI is detected, run `jsc-hooks/tools/wire-cli.sh smoke codex` as the minimum hook behavior check and label it 「預設 hook smoke」 in the report. Use `smoke`, not `purge` or rewiring actions.
|
|
4. When a hook or script smoke fails, route it as a compliance failure with script name, exit code, output summary, and proposed fix. Do not continue to report the affected hook as compliant.
|
|
|
|
Completion condition: every domain has a script syntax verdict, every named script has an existence and exit-code-routing verdict, and the `jsc-hooks` domain has a hook smoke verdict.
|
|
3. Audit every skill of every domain against the guidelines.md audit checklist — this step MUST run as a sub agent, one sub agent per domain repo. Each sub agent reports its findings: skill, failed checklist item, evidence (file:line), proposed fix. Cover the checklist's four flow checks by name, not only the naming and language items:
|
|
1. Every step number, file path and section title the skill references — inside itself and in other files — really exists (the pointer points at something).
|
|
2. Every step ends in a checkable completion condition, with no vague wording.
|
|
3. Every external call (script, API, other skill) states what to do on failure and routes every exit code.
|
|
4. No gate the skill installs blocks the only path that lifts that gate.
|
|
|
|
Completion condition: every domain has an audit result that names a verdict for all checklist items, the four flow checks included.
|
|
4. Run a separate flow and cost optimization review after the compliance audit. Each aspect **MUST run as a sub agent**, and the six aspects may run in parallel:
|
|
|
|
| Aspect | Scope |
|
|
| --- | --- |
|
|
| 1 Parallelism | Steps that run in series today but have no data dependency and can run in parallel |
|
|
| 2 Tool extraction | SKILL.md text flows with clear inputs and outputs that should move to `tools/`, including hook-enforceable rules that still rely on prompts |
|
|
| 3 Repeated interaction | The same user question, repository fact, wiki page, API result, or file content being collected more than once |
|
|
| 4 Redundant checks | Completion conditions or verification steps that overlap, or a later step that necessarily covers an earlier check |
|
|
| 5 Gate timing | Gates that run too early or too late, causing wasted work before a block or blocking the only path that clears the gate |
|
|
| 6 Cost efficiency | Avoidable token, sub-agent, API, file-scan, full-repo audit, or user-interaction cost that can be reduced without weakening correctness |
|
|
|
|
Each optimization finding reports skill, aspect, evidence (file:line), current flow step count, proposed flow step count, what time or interaction it saves, what cost it saves, current cost driver, proposed cost driver, whether correctness decreases, and which protection would be weakened if any. Cost savings may be token volume, sub-agent count, API calls, file scans, full-repo audits, or user prompts. Keep optimization findings separate from compliance failures. Completion condition: every aspect has returned a verdict for every domain; aspects with no findings return 「無發現」.
|
|
5. Present compliance failures and optimization findings separately via the `jsc-ask:ask` decision tree:
|
|
- Compliance failure options: apply the proposed fix / skip / custom fix. Every option states its impact scope, for example skipping leaves the skill non-compliant until the next audit.
|
|
- Optimization options: apply / defer / custom. Any suggestion that weakens a protection must name the protection it removes and must not be applied unless the user explicitly accepts that tradeoff. Cost optimization may move, merge, cache, or narrow checks; it must not delete a compliance check only because it is expensive.
|
|
|
|
Completion condition: every compliance failure and every optimization finding has a recorded decision.
|
|
6. Apply the confirmed fixes and accepted optimizations — the file-change part MUST run as a sub agent, one sub agent per affected domain repo: modify the files per the recorded decision. Then run `tools/sync-skill-manifest.sh {domain-path}` directly (no sub agent needed) for each affected domain repo to refresh that domain README's 「Skills 目錄」 section and bump the version in all three manifests. Completion condition: every affected repo carries the changes and the manifest bump.
|
|
7. Sync the canonical marketplace — a **required** step, never optional. The canonical pair lives in `plugins/meta` and every domain repo carries a byte-identical copy, so a fix that leaves the copies apart makes some repos register a stale plugin set. Run `tools/sync-marketplace.sh {domain} {repo-url} {description}` once with an existing entry's own current values (rewriting the same entry is idempotent); the script rewrites both canonical files and copies them into every domain repo. Route each exit code:
|
|
- Exit 3 — written, but some domain repo is not present locally. Run `tools/sync-domains.sh`, then rerun this step.
|
|
- Exit 2 — usage error: the script takes exactly three arguments. Fix them and rerun.
|
|
- Exit 1 — missing python3, an unreadable canonical file, or a byte mismatch between copies. Read stderr, fix the named cause (install python3 for the first), then rerun.
|
|
- Exit 0 — every copy holds identical bytes; the script verifies that itself.
|
|
|
|
Completion condition: the script exits 0 and prints the touched paths.
|
|
8. Re-run the script and hook validation from step 2, re-check the guidelines.md audit checklist for every touched skill, then re-run the optimization aspect that produced each accepted optimization. On any compliance failure, **return to step 5**: confirm and fix again, until all accepted compliance fixes pass. On an accepted optimization that does not produce the promised step reduction or cost reduction, or still weakens correctness beyond the recorded decision, return to step 5 for a new decision. Completion condition: all script and hook checks pass, all checklist items pass, and every accepted optimization has a matching verification result.
|
|
9. Call `jsc-git:pr` once per affected domain repo to open a Push Request. Completion condition: every affected repo has a PR URL, and all URLs are reported in one table with the format in `references/pr-report.md`.
|