feat(doctor): 加入 CLI 實測流程 #37

Merged
admin merged 4 commits from feat/doctor-cli-runtime-tests/add-executable-checks into feat/doctor-cli-runtime-tests/main 2026-08-28 09:00:42 +00:00
5 changed files with 211 additions and 25 deletions
Showing only changes of commit 0b2c7b13a0 - Show all commits
+3 -1
View File
@@ -23,6 +23,7 @@ Marketplace 統一為 `jsc`(https://gitea.jsc.idv.tw/plugins/meta.git),安
| 工具 | 用途 |
| --- | --- |
| `tools/detect-clis.sh` | 列出已安裝的 AI CLI 與執行檔路徑(TSV:name / path / version;antigravity 的執行檔為 `agy`、kiro 為 `kiro-cli`) |
| `tools/test-clis.sh` | 實際呼叫已偵測到的 AI CLI,跑版本、說明頁與 plugin 清單等唯讀命令(`test-clis.sh [cli...]`);印出每項命令、結束碼、判定與輸出摘要,最後一行 `summary` 標出通過、降級、失敗、略過數。`JSC_CLI_TEST_TIMEOUT` 可調整單項命令逾時秒數,預設 10 秒 |
| `tools/deploy.sh` | 對單一 CLI 執行安裝、更新或解除安裝(`deploy.sh [-n] {mode} {cli} {domain}...`,mode 為 install / update / uninstall);印出每個指令與其結束碼,最後一行 `result` 標 ok 或 fail。`-n` 只印指令不執行。上表五個 CLI 的指令差異全部收在這支腳本裡。install 或 update 全數成功時,收尾轉呼叫 `jsc-hooks` 的 `restart-gate.sh require` 掛上重啟閘門,並印一行 `restart` 標出狀態檔位置;uninstall 不寫。尋找 `restart-gate.sh` 時優先用 `$JSC_HOME/current/jsc-hooks`、本地 clone 與 Kiro skills,最後才掃各 CLI 快取,避免部署收尾綁死單一 CLI 的版號路徑。狀態檔的路徑、格式與判讀全在 `restart-gate.sh`,這支腳本不自己拼——格式只留一個真實來源。站台取自 `GITEA_HOST`,本地 clone 目錄取自 `JSC_LOCAL_PLUGINS`,兩者的預設值見下表 |
| `tools/check-requires.sh` | `check-requires.sh {cli} {manifest}` 檢查 manifest 的 `jsc.requires` 最低版本。沒有宣告就通過;版本不符或缺相依 plugin 就回 `status=blocked`。`deploy.sh update` 在每個 domain 更新前呼叫它,不符就跳過該 domain 並列出原因 |
| `tools/write-guides.sh` | 產生這台機器專屬的更新指引 `$JSC_HOME/update-guide.md` 與移除指引 `$JSC_HOME/remove-guide.md`(`write-guides.sh [-n] {install\|update} {domain}...`),一輪部署跑一次。CLI 清單取自 `detect-clis.sh`,每支 CLI 的指令字面直接取自 `deploy.sh -n` 的輸出,所以指引寫的就是實際會跑的指令;kiro 走不走本地複製退路也依實際偵測結果標注 |
@@ -53,7 +54,7 @@ Marketplace 統一為 `jsc`(https://gitea.jsc.idv.tw/plugins/meta.git),安
### `doctor`
一次體檢執行環境,只讀不改。四項檢查:技能版本(`jsc-hooks/hooks/version-guard.sh report`)、Hook 接線(`jsc-hooks/tools/wire-cli.sh status`,唯讀子命令)、全域設定與自我設定(`tools/scan-config.sh` 比對 `tools/config-spec.tsv`)。每項各出一張表,整份結果寫進 wiki `CHECK_{HASH}`,`HASH` 取 `{主機名}/{登入帳號}`,只保留最新一次。修復交給 `/jsc-cli:setup`,體檢本身不動任何設定。
一次體檢執行環境,只讀不改。五項檢查:技能版本(`jsc-hooks/hooks/version-guard.sh report`)、Hook 接線(`jsc-hooks/tools/wire-cli.sh status`,唯讀子命令)、CLI 實測(`tools/test-clis.sh` 實際呼叫版本、說明頁與 plugin 清單)、全域設定與自我設定(`tools/scan-config.sh` 比對 `tools/config-spec.tsv`)。每項各出一張表,整份結果寫進 wiki `CHECK_{HASH}`,`HASH` 取 `{主機名}/{登入帳號}`,只保留最新一次。修復交給 `/jsc-cli:setup`,體檢本身不動任何設定。
### `setup`
@@ -72,6 +73,7 @@ Marketplace 統一為 `jsc`(https://gitea.jsc.idv.tw/plugins/meta.git),安
| `JSC_DEPLOY_DRYRUN` | 設為 `1` 等同 `deploy.sh -n`,只印指令不執行 | 照常執行 |
| `JSC_WIKI_REPO_CHECK` | 體檢頁 `CHECK_CONTENTS`、`CHECK_{HASH}` 所在的 `{owner}/{repo}` | 退回 `JSC_WIKI_REPO`;兩個都沒有就略過寫入,並把這一項列進待修 |
| `JSC_CONFIG_SPEC` | 改讀別份設定規格表(測試 `scan-config.sh` 時用) | 用 `tools/config-spec.tsv` |
| `JSC_CLI_TEST_TIMEOUT` | `test-clis.sh` 單項 CLI 實測命令的逾時秒數 | 用 `10` |
| `JSC_HOME` | hook 資料目錄,兩份指引與重啟狀態檔都寫在這裡 | 用 `~/.jsc` |
| `JSC_RESTART_GATE` | 設成 `off` 可略過部署後的重啟提示閘門(判讀在 `jsc-hooks`,`jsc-cli` 只負責寫狀態檔) | 照常提示重啟 |
+39 -20
View File
@@ -1,45 +1,64 @@
---
name: doctor
description: Health-check the execution environment in one pass and record the result, changing nothing. Four checks - plugin versions from jsc-hooks/hooks/version-guard.sh report, hook wiring from jsc-hooks/tools/wire-cli.sh status, global settings and current-directory settings from tools/scan-config.sh against tools/config-spec.tsv. Report one findings table per check, then write the whole run to wiki CHECK_{HASH} where HASH comes from {hostname}/{user}; the page keeps only the latest run. Use after installing or updating the skill set, when a skill fails on a settings or wiring problem, or before handing a machine over; not for applying fixes, which is jsc-cli:setup.
description: Health-check the execution environment in one pass and record the result, changing nothing. Five checks - plugin versions from jsc-hooks/hooks/version-guard.sh report, hook wiring from jsc-hooks/tools/wire-cli.sh status, executable CLI tests from tools/test-clis.sh, global settings and current-directory settings from tools/scan-config.sh against tools/config-spec.tsv. Report one findings table per check, then write the whole run to wiki CHECK_{HASH} where HASH comes from {hostname}/{user}; the page keeps only the latest run. Use after installing or updating the skill set, when a skill fails on a settings, wiring or CLI runtime problem, or before handing a machine over; not for applying fixes, which is jsc-cli:setup.
---
# doctor — execution environment health check
# doctor - execution environment health check
Read-only. Every command below either reads a file or asks Gitea; none of them writes a setting. That is the contract with `jsc-cli:setup`: doctor states the facts, setup changes things.
Read-only. Every command below either reads local state, calls a read-only CLI command, or asks Gitea. None of them writes a setting. That is the contract with `jsc-cli:setup`: doctor states the facts, setup changes things.
Collection (steps 1 to 4) **MUST run as a sub agent** — one sub agent for all four, returning the raw TSV lines. Only the report and the wiki write stay in the main agent.
Collection (steps 1 to 5) **MUST run as a sub agent** - one sub agent for all five, returning the raw TSV lines. Only the report and the wiki write stay in the main agent.
## 1. Skill versions
Run `jsc-hooks/hooks/version-guard.sh report`. It prints `{domain}<TAB>{本機}<TAB>{遠端}<TAB>{落後|最新|超前|查詢失敗}` per plugin, then `behind<TAB>{count}`.
Run `jsc-hooks/hooks/version-guard.sh report`. It prints `{domain}<TAB>{local}<TAB>{remote}<TAB>{落後|最新|超前|查詢失敗}` per plugin, then `behind<TAB>{count}`.
A report with no `{domain}` row, or one carrying `noregistry<TAB>{path}`, means this CLI has no local plugin registry. Report it as 無法驗證 — never as 最新. `behind<TAB>0` proves nothing when no domain row precedes it.
A report with no `{domain}` row, or one carrying `noregistry<TAB>{path}`, means this CLI has no local plugin registry. Report it as `無法驗證` - never as `最新`. `behind<TAB>0` proves nothing when no domain row precedes it.
Done when every installed domain has a status literal, or the CLI is reported as unverifiable.
## 2. Hook wiring
Run `jsc-hooks/tools/wire-cli.sh status {cli}` for every CLI that `tools/detect-clis.sh` found. Use `status` and nothing else: `wire-cli.sh` without a subcommand rewires, `purge` deletes, and `smoke` executes hooks — all three break the read-only contract.
Run `jsc-hooks/tools/wire-cli.sh status {cli}` for every CLI that `tools/detect-clis.sh` found. Use `status` and nothing else: `wire-cli.sh` without a subcommand rewires, `purge` deletes, and `smoke` executes hooks - all three break the read-only contract.
Exit codes: 0 wired, 1 degraded, 3 skipped (CLI not installed), 5 unwired. Each `item` line names one wiring point and whether it is present.
Exit codes: 0 wired, 1 degraded, 3 skipped because the CLI is not installed, 5 unwired. Each `item` line names one wiring point and whether it is present.
Only claude reaches `wired`. The other four have no pre-tool hook, so `degraded` is their healthy state — report the degradation reason as-is and never present it as a defect to fix.
Only claude reaches `wired`. The other four have no pre-tool hook, so `degraded` is their healthy state. Report the degradation reason as-is and never present it as a defect to fix.
Done when every detected CLI has a status and its missing items are listed.
## 3. Global settings
## 3. CLI runtime tests
Run `tools/test-clis.sh` with no CLI arguments. It calls `tools/detect-clis.sh`, then runs real read-only commands for every detected CLI.
Output:
- `test<TAB>{cli}<TAB>{test}<TAB>{command}<TAB>{exit-code}<TAB>{verdict}<TAB>{detail}`
- `summary<TAB>{ok}<TAB>{warn}<TAB>{fail}<TAB>{skipped}`
Verdicts: `ok`, `warn`, `fail`, `skipped`.
Exit codes: 0 completed, 2 usage error, 3 missing `detect-clis.sh`. Any other script exit code is itself a doctor finding.
Treat `fail` as a machine problem. Treat `warn` as degraded capability: name it in the report, but do not put it in the fix table unless the failing skill needs that feature. Treat `skipped` as no conclusion. Map the report labels to the template as `ok` -> `通過`, `warn` -> `降級`, `fail` -> `失敗`, and `skipped` -> `略過`.
Done when every detected CLI has at least a version test row and the summary line is read.
## 4. Global settings
Run `tools/scan-config.sh scan global`. It checks every `scope=global` row of `tools/config-spec.tsv` and prints `item<TAB>scope<TAB>required<TAB>actual<TAB>expect<TAB>fix<TAB>verdict`, closing with `summary<TAB>{missing}<TAB>{invalid}<TAB>{unset}<TAB>{skipped}`.
Verdicts: `ok`, `default` (unset, default works), `unset` (optional, feature degrades), `missing` (required, skills break), `invalid` (set but fails verification), `skipped` (offline).
Verdicts: `ok`, `default`, `unset`, `missing`, `invalid`, `skipped`.
Add `-o` when Gitea is unreachable; the Gitea-dependent rows then come back `skipped`. Report those rows as 未取得結論 and never as passes.
Map the report labels to the template as `ok` -> `通過`, `default` -> `走預設`, `unset` -> `未設定`, `missing` -> `缺漏`, `invalid` -> `設錯`, and `skipped` -> `略過`.
Also run `tools/scan-config.sh orphans` — variables used in the source but absent from the spec table. They are a maintenance note for the skill set, not a fault on this machine.
Add `-o` when Gitea is unreachable; the Gitea-dependent rows then come back `skipped`. Report those rows as inconclusive and never as passes.
Also run `tools/scan-config.sh orphans` - variables used in the source but absent from the spec table. They are a maintenance note for the skill set, not a fault on this machine.
Done when the summary line is read and every `missing` and `invalid` row is named.
## 4. Own settings
## 5. Own settings
Run `tools/scan-config.sh scan project` from the current working directory. Same output format, `scope=project` rows only.
@@ -49,23 +68,23 @@ When `.env` or `.envrc` exists, name the spec-table variables it overrides and s
Done when the scanned directory is stated and every project row has a verdict.
## 5. Report and record
## 6. Report and record
Report all four tables per `templates/check-page.md`. Then build the 待修項目 table from every `missing`, `invalid` and `unwired` item, plus every domain reported 落後. Order them `missing` → `invalid` → `unwired` → `落後`. Nothing wrong → one row reading 無.
Report all five tables per `templates/check-page.md`. Then build the `待修項目` table from every `missing`, `invalid`, `unwired` and CLI runtime `fail` item, plus every domain reported `落後`. Order them `missing` -> `invalid` -> `unwired` -> `runtime-fail` -> `落後`. Nothing wrong -> one row reading `無`.
Write the page through `jsc-gitea:wiki`:
- Wiki repo: `jsc-gitea/tools/gitea.sh wiki-repo CHECK`.
- Page name: `CHECK_` plus `gitea.sh hash-id "{hostname}/{user}"` — the host and the login account, not `{owner}/{repo}`. Doctor checks a machine, and it has to work in directories that are not repositories at all.
- Page name: `CHECK_` plus `gitea.sh hash-id "{hostname}/{user}"` - the host and the login account, not `{owner}/{repo}`. Doctor checks a machine, and it has to work in directories that are not repositories at all.
- Overwrite the whole page. This page type keeps only the latest run.
- Update `CHECK_CONTENTS` from `templates/check-contents.md` in the same pass.
`wiki-repo` exiting 3 means no wiki repo is configured for CHECK. Print the tables, skip the wiki write, and put `JSC_WIKI_REPO_CHECK` at the top of 待修項目 — that unset variable is itself a finding, so a failed write never fails the health check.
`wiki-repo` exiting 3 means no wiki repo is configured for CHECK. Print the tables, skip the wiki write, and put `JSC_WIKI_REPO_CHECK` at the top of the `待修項目` table - that unset variable is itself a finding, so a failed write never fails the health check.
Done when either the wiki page URL is reported, or the skipped write is reported together with the reason.
## 6. Hand off
## 7. Hand off
State the counts: required items missing, settings invalid, CLIs unwired, domains behind. Recommend `/jsc-cli:setup` when any of those is above zero. Never fix anything here.
State the counts: required items missing, settings invalid, CLIs unwired, CLI runtime failures, domains behind. Recommend `/jsc-cli:setup` when any of those is above zero. Never fix anything here.
Done when the counts are stated and the recommendation is given or explicitly withheld.
+3 -3
View File
@@ -2,6 +2,6 @@
> 由 `jsc-cli:doctor` 維護。每台執行環境一列;`HASH` 取 `{主機名}/{登入帳號}`,算法與其他頁面共用。
| 體檢頁 | 主機 | 帳號 | 必要項缺漏 | 設定錯誤 | 最後體檢 |
| --- | --- | --- | --- | --- | --- |
| [[CHECK_{HASH}]] | {hostname} | {使用者帳號} | {n} | {n} | {yyyy-MM-dd HH:mm} |
| 體檢頁 | 主機 | 帳號 | 必要項缺漏 | 設定錯誤 | CLI 實測失敗 | 最後體檢 |
| --- | --- | --- | --- | --- | --- | --- |
| [[CHECK_{HASH}]] | {hostname} | {使用者帳號} | {n} | {n} | {n} | {yyyy-MM-dd HH:mm} |
+8 -1
View File
@@ -13,6 +13,7 @@
| --- | --- | --- | --- |
| 技能版本 | {n} | {n} | {n} |
| Hook 接線 | {n} | {n} | {n} |
| CLI 實測 | {n} | {n} | {n} |
| 全域設定 | {n} | {n} | {n} |
| 自我設定 | {n} | {n} | {n} |
@@ -28,6 +29,12 @@
| --- | --- | --- | --- |
| {cli} | {wired、degraded、unwired、skipped} | {項目名,逗號分隔;無則寫「無」} | {降級原因或未偵測到執行檔} |
## CLI 實測
| CLI | 測試 | 指令 | 結束碼 | 判定 | 說明 |
| --- | --- | --- | --- | --- | --- |
| {cli} | {version、help、plugin-list} | {實際命令} | {結束碼} | {通過、降級、失敗、略過} | {輸出摘要或錯誤原因} |
## 全域設定
| 項目 | 必要 | 現況 | 期望 | 修法 | 判定 |
@@ -48,7 +55,7 @@
| 順序 | 項目 | 範圍 | 判定 | 修法 | 影響 |
| --- | --- | --- | --- | --- | --- |
| {n} | {變數、檔案、hook 或 domain} | {全域、自我} | {缺漏、設錯、未接線、落後} | {自動、詢問、手動} | {不修的話哪些技能跑不動} |
| {n} | {變數、檔案、hook、CLI 測試或 domain} | {全域、自我、CLI} | {缺漏、設錯、未接線、實測失敗、落後} | {自動、詢問、手動} | {不修的話哪些技能跑不動} |
## 未登錄變數
+158
View File
@@ -0,0 +1,158 @@
#!/usr/bin/env sh
# test-clis.sh — 實際呼叫已安裝的 AI CLI,找出靜態設定看不出的環境問題。
# 用法:
# test-clis.sh [cli...]
# 輸出(TSV):
# test<TAB>{cli}<TAB>{test}<TAB>{指令}<TAB>{結束碼}<TAB>{判定}<TAB>{說明}
# summary<TAB>{ok}<TAB>{warn}<TAB>{fail}<TAB>{skipped}
# 判定:
# ok 實際命令成功
# warn CLI 可用,但該功能在這支 CLI 或這個版本不是必要功能
# fail 命令失敗、逾時,或必要功能不存在
# skipped 未安裝或無法執行該項測試
# 結束碼:0=測試完成;2=用法錯誤;3=找不到 detect-clis.sh
set -u
HERE=$(CDPATH= cd -- "$(dirname -- "$0")" && pwd)
DETECT="$HERE/detect-clis.sh"
TIMEOUT_SECONDS="${JSC_CLI_TEST_TIMEOUT:-10}"
ok_count=0
warn_count=0
fail_count=0
skipped_count=0
[ -f "$DETECT" ] || { echo "找不到 detect-clis.sh:$DETECT" >&2; exit 3; }
usage() {
echo "用法:test-clis.sh [claude|codex|copilot|antigravity|kiro ...]" >&2
exit 2
}
cli_bin() {
case "$1" in
antigravity) printf 'agy' ;;
kiro) printf 'kiro-cli' ;;
claude|codex|copilot) printf '%s' "$1" ;;
*) return 1 ;;
esac
}
emit() { # cli test command code verdict detail
printf 'test\t%s\t%s\t%s\t%s\t%s\t%s\n' "$1" "$2" "$3" "$4" "$5" "$6"
case "$5" in
ok) ok_count=$((ok_count + 1)) ;;
warn) warn_count=$((warn_count + 1)) ;;
fail) fail_count=$((fail_count + 1)) ;;
skipped) skipped_count=$((skipped_count + 1)) ;;
esac
}
run_capture() { # $@=command
out_file=$(mktemp) || return 125
err_file=$(mktemp) || { rm -f "$out_file"; return 125; }
if command -v timeout >/dev/null 2>&1; then
timeout "$TIMEOUT_SECONDS" "$@" >"$out_file" 2>"$err_file"
else
"$@" >"$out_file" 2>"$err_file"
fi
code=$?
text=$(cat "$out_file" "$err_file" 2>/dev/null | tr '\r\n\t' ' ' | sed 's/[[:space:]][[:space:]]*/ /g; s/^ //; s/ $//; s/.*TOKEN[^ ]*/[secret]/g' | cut -c1-180)
rm -f "$out_file" "$err_file"
RUN_CODE=$code
RUN_TEXT=${text:-無輸出}
return 0
}
run_required() { # cli test command...
cli=$1
test_name=$2
shift 2
cmd_text="$*"
run_capture "$@"
code=$RUN_CODE
detail=$RUN_TEXT
if [ "$code" -eq 0 ]; then
emit "$cli" "$test_name" "$cmd_text" "$code" ok "$detail"
elif [ "$code" -eq 124 ]; then
emit "$cli" "$test_name" "$cmd_text" "$code" fail "命令逾時(${TIMEOUT_SECONDS} 秒)"
else
emit "$cli" "$test_name" "$cmd_text" "$code" fail "$detail"
fi
}
run_optional() { # cli test command...
cli=$1
test_name=$2
shift 2
cmd_text="$*"
run_capture "$@"
code=$RUN_CODE
detail=$RUN_TEXT
if [ "$code" -eq 0 ]; then
emit "$cli" "$test_name" "$cmd_text" "$code" ok "$detail"
elif [ "$code" -eq 124 ]; then
emit "$cli" "$test_name" "$cmd_text" "$code" fail "命令逾時(${TIMEOUT_SECONDS} 秒)"
else
emit "$cli" "$test_name" "$cmd_text" "$code" warn "$detail"
fi
}
has_cli() {
name=$1
"$DETECT" | awk -F '\t' -v name="$name" '$1 == name { found = 1 } END { exit found ? 0 : 1 }'
}
test_one() {
cli=$1
bin=$(cli_bin "$cli") || usage
path=$(command -v "$bin" 2>/dev/null || true)
if [ -z "$path" ]; then
emit "$cli" executable "$bin" "-" skipped "未偵測到執行檔"
return 0
fi
case "$cli" in
antigravity)
run_required "$cli" version "$path" --version
run_optional "$cli" help "$path" --help
run_required "$cli" plugin-list "$path" plugin list
;;
kiro)
run_required "$cli" version "$path" --version
run_optional "$cli" help "$path" --help-all
run_optional "$cli" plugin-list "$path" plugin list
;;
*)
run_required "$cli" version "$path" --version
run_optional "$cli" help "$path" --help
run_required "$cli" plugin-list "$path" plugin list
;;
esac
}
if [ "$#" -eq 0 ]; then
set -- $("$DETECT" | cut -f1)
fi
if [ "$#" -eq 0 ]; then
emit all executable "-" "-" skipped "未偵測到任何支援的 CLI"
printf 'summary\t%s\t%s\t%s\t%s\n' "$ok_count" "$warn_count" "$fail_count" "$skipped_count"
exit 0
fi
for cli in "$@"; do
case "$cli" in
claude|codex|copilot|antigravity|kiro)
if has_cli "$cli"; then
test_one "$cli"
else
bin=$(cli_bin "$cli") || usage
emit "$cli" executable "$bin" "-" skipped "未偵測到執行檔"
fi
;;
*) usage ;;
esac
done
printf 'summary\t%s\t%s\t%s\t%s\n' "$ok_count" "$warn_count" "$fail_count" "$skipped_count"
exit 0