feat(wiki): 目錄頁專用存取庫、HASH 去除截短與 H 前綴

What:CONTENTS 成為第 15 種頁面類型,解析鏈為 JSC_WIKI_REPO_CONTENTS 到
JSC_WIKI_REPO,刻意不退回型別變數。hash-id 拿掉 8 碼截短與 H 前綴改寫,只留大寫
轉換,輸出完整 40 碼;空輸入改成用法錯誤。新增 page-name.sh、wiki-contents.sh、
migrate-wiki.sh 與 wiki-delete 子命令。

Why:H 前綴會命中 16 個首碼裡的 13 個,還丟掉第 8 碼,把有效熵壓到 28 位元,而且
全庫查不到任何理由紀錄。目錄頁的整列 upsert 原本 14 處只有一處寫成程式,同一段判斷
做 14 次,錯一次就少一筆紀錄。

How:頁名樣式仍收 H 加 7 碼的舊頁,遷移期間讀得到舊頁。wiki-contents.sh 建新頁時
剝掉範本的示範列,否則每個目錄頁第一次建立都會留一列佔位死連結。migrate-wiki.sh
預設只印對照表,--apply 先寫新頁、確認寫成、才刪舊頁;孤兒頁只列不猜,因為 SHA-1
不可逆,新頁名只能靠候選鍵正推。

Who:jsc-gitea
This commit is contained in:
2026-09-02 11:02:07 +08:00
parent a3ce17f98d
commit c40b561589
13 changed files with 826 additions and 53 deletions
+435
View File
@@ -0,0 +1,435 @@
#!/usr/bin/env sh
# migrate-wiki.sh — 把既有 wiki 頁搬到新規則。
#
# 一次改兩件事:
# 目錄頁 {TYPE}_CONTENTS 換存取庫,頁名不變。
# 內容頁 {TYPE}_{舊 HASH} 換頁名,存取庫不變。
#
# 為什麼只做正推:SHA-1 不可逆,新頁名算不回舊頁名。所以拿候選鍵跑舊演算法,
# 對得上現有頁名才算配對成功。配不上就列成孤兒,交給人處理,絕不猜。
#
# 用法:
# migrate-wiki.sh [--apply] [--key <候選鍵>]...
# 不帶 --apply 時只印對照表與孤兒清單,不寫任何東西。
# --key 可重複,補充自動蒐集不到的候選鍵。
#
# 候選鍵來源:
# 目錄頁表格的存取庫、主機、帳號、工具、期間欄,以及這幾欄依序串成的組合鍵。
# 內容頁的 H1 標題(CHECK 頁的 H1 直接就是 {主機}/{帳號})。
# --key 補充的候選。
#
# --apply 的順序:先把每一頁寫到新位置,再改連結,最後才刪舊頁。
# 連結改寫不能省:[[...]] 只在同一個 wiki 內解析,頁名或存取庫一變,
# 連結就指向一個不存在的頁,而畫面上看不出異常。
# 先全部寫到新位置,wiki-url 才查得到每一個目的地的絕對網址。
# 刪除排在確認新頁讀得回來之後:先刪再寫,中間出錯就兩邊都沒有。
# 每一頁寫到新位置之前一定先讀目的地,只有結束碼 4 才准寫:目的地那一頁可能是
# 前一次搬到一半留下的,也可能是 wiki-contents.sh 剛建好的活頁,直接寫就是覆蓋。
#
# 連結改寫的範圍限制:只改寫被搬的那些頁自己的內容。沒被搬的頁——已經合規的頁、
# 目的地存取庫裡原有的頁——裡面指向被搬頁的 [[...]] 這支不動。要改寫全部,
# 得把每個存取庫的每一頁都讀回來重寫,代價是整批頁的讀寫都變成寫入風險。
# 所以改採列清單:報告的「引用被搬頁、但自己沒被搬」一節列出每一個引用方與它指到的頁名,
# 交給人改。清單不空時結束碼是 3。
#
# 逐列處理的迴圈一律從 fd 3 讀清單,不佔用 stdin。wiki-put 與 wiki-delete 的
# 人工確認要從終端讀一行,stdin 被清單檔佔走的話,每一次寫入都會被判成
# 「沒有互動終端」而拒絕。
# 結束碼: 0=全部搬完 1=有頁搬移失敗 2=用法錯誤,含 CONTENTS 存取庫未設定
# 3=有需人工處理的項目:孤兒頁、沒被搬的引用方、目的地已有內容的頁
set -eu
dir=$(CDPATH= cd -- "$(dirname -- "$0")" && pwd)
gitea="$dir/gitea.sh"
hash_id="$dir/hash-id"
TAB=$(printf '\t')
TYPES='QUESTION PLAN ANALYZE DELIVER MAINTAIN REPO LOG LEARN ERROR CHECK REPORT SKILLSET TOOLING MONITOR'
usage() {
echo 'usage: migrate-wiki.sh [--apply] [--key <候選鍵>]...' >&2
exit 2
}
apply=0
work=$(mktemp -d)
trap 'rm -rf "$work"' EXIT
: > "$work/keys"
while [ "$#" -gt 0 ]; do
case "$1" in
--apply) apply=1; shift ;;
--key)
[ "$#" -ge 2 ] && [ -n "$2" ] || usage
printf '%s\n' "$2" >> "$work/keys"; shift 2 ;;
*) usage ;;
esac
done
old_hash() { # 舊演算法:SHA-1 前 8 碼大寫;首碼落在 0-9ABC 就改成 H 加前 7 碼
if command -v sha1sum >/dev/null 2>&1; then
raw=$(printf '%s' "$1" | sha1sum | awk '{print $1}')
else
raw=$(printf '%s' "$1" | shasum -a 1 | awk '{print $1}')
fi
h=$(printf '%s' "$raw" | cut -c1-8 | tr a-f A-F)
case "$h" in [0-9ABC]*) h="H$(printf '%s' "$h" | cut -c1-7)" ;; esac
printf '%s\n' "$h"
}
rc=0
contents_repo=$(sh "$gitea" wiki-repo CONTENTS) || rc=$?
if [ "$rc" -ne 0 ]; then
echo '[jsc][gitea][ERR]:目錄頁的專用存取庫沒有設定。請先設 JSC_WIKI_REPO_CONTENTS 或 JSC_WIKI_REPO。' >&2
exit 2
fi
: > "$work/map" # 舊存取庫 \t 舊頁名 \t 新存取庫 \t 新頁名
: > "$work/orphans" # 存取庫 \t 頁名 \t 原因
: > "$work/pending" # 存取庫 \t 頁名 \t 型別 \t 舊 HASH
: > "$work/allpages" # 存取庫 \t 頁名(列得出來的每一頁,含不搬的)
: > "$work/refs" # 存取庫 \t 頁名 \t 被指到的舊頁名
: > "$work/manual" # 需人工確認的項目,一行一句
: > "$work/notes"
# ---- 第一輪:列出各型別的頁,分成目錄頁、待配對的內容頁、已經合規的頁 ----
for ty in $TYPES; do
rc=0
repo=$(sh "$gitea" wiki-repo "$ty" 2>/dev/null) || rc=$?
if [ "$rc" -ne 0 ]; then
printf '%s:沒有設定存取庫,略過。\n' "$ty" >> "$work/notes"
continue
fi
rc=0
pages=$(sh "$gitea" wiki-list "$repo" 2>/dev/null) || rc=$?
if [ "$rc" -ne 0 ]; then
printf '%s(%s):列不出 wiki 頁(結束碼 %s)。\n' "$ty" "$repo" "$rc" >> "$work/notes"
continue
fi
printf '%s\n' "$pages" > "$work/pagelist"
while IFS= read -r pg <&3; do
[ -n "$pg" ] || continue
# 先收進全頁清單,再過型別前綴。引用方掃描要看的是沒被搬的那些頁。
printf '%s\t%s\n' "$repo" "$pg" >> "$work/allpages"
case "$pg" in "${ty}_"*) ;; *) continue ;; esac
suffix=${pg#"${ty}_"}
if [ "$suffix" = CONTENTS ]; then
if [ "$repo" = "$contents_repo" ]; then
printf '%s:已經在目錄頁存取庫。\n' "$pg" >> "$work/notes"
else
printf '%s\t%s\t%s\t%s\n' "$repo" "$pg" "$contents_repo" "$pg" >> "$work/map"
fi
elif printf '%s' "$suffix" | grep -Eq '^[0-9A-F]{40}$'; then
printf '%s:已經是 40 碼頁名。\n' "$pg" >> "$work/notes"
elif printf '%s' "$suffix" | grep -Eq '^([0-9A-F]{8}|H[0-9A-F]{7})$'; then
printf '%s\t%s\t%s\t%s\n' "$repo" "$pg" "$ty" "$suffix" >> "$work/pending"
else
printf '%s\t%s\t%s\n' "$repo" "$pg" '頁名不符任何已知樣式' >> "$work/orphans"
fi
done 3< "$work/pagelist"
done
# ---- 第二輪:蒐集候選鍵 ----
# 目錄頁表格的欄位值,以及依序串成的組合鍵
while IFS="$TAB" read -r orepo opage nrepo npage <&3; do
[ -n "$opage" ] || continue
case "$opage" in *_CONTENTS) ;; *) continue ;; esac
sh "$gitea" wiki-get "$orepo" "$opage" 2>/dev/null > "$work/one" || continue
python3 - "$work/one" <<'PY' >> "$work/keys" || true
import re
import sys
WANT = ['存取庫', '主機', '工具', '帳號', '期間']
lines = open(sys.argv[1], encoding='utf-8').read().split('\n')
def cells(line):
s = line.strip()
if not s.startswith('|'):
return None
s = s[1:]
if s.endswith('|'):
s = s[:-1]
return [c.strip() for c in s.split('|')]
def is_sep(cs):
return bool(cs) and all(c and set(c) <= set('-: ') for c in cs)
def plain(v):
v = re.sub(r'\[\[([^\]|]*)\|([^\]]*)\]\]', r'\1', v)
v = re.sub(r'\[\[([^\]]*)\]\]', r'\1', v)
v = re.sub(r'\[([^\]]*)\]\([^)]*\)', r'\1', v)
return v.replace('`', '').strip()
idx = {}
seen_sep = False
out = []
for line in lines:
cs = cells(line)
if cs is None:
idx, seen_sep = {}, False
continue
if is_sep(cs):
seen_sep = True
continue
if not seen_sep:
# 分隔列之前的那一列是表頭,欄位位置從它認出來。
idx = {}
for w in WANT:
for i, h in enumerate(cs):
if w in h:
idx[w] = i
break
continue
if not idx:
continue
parts = []
for w in WANT:
i = idx.get(w)
if i is None or i >= len(cs):
continue
v = plain(cs[i])
if not v:
continue
out.append(v)
parts.append(v)
if len(parts) > 1:
out.append('/'.join(parts))
for v in out:
print(v)
PY
done 3< "$work/map"
# 內容頁的 H1 標題
while IFS="$TAB" read -r repo pg ty suffix <&3; do
[ -n "$pg" ] || continue
sh "$gitea" wiki-get "$repo" "$pg" 2>/dev/null > "$work/one" || continue
sed -n 's/^# \{1,\}//p' "$work/one" | head -n 3 >> "$work/keys"
done 3< "$work/pending"
sort -u "$work/keys" > "$work/keys.uniq"
# 候選鍵的舊 HASH 對照表
: > "$work/hashes"
while IFS= read -r k <&3; do
[ -n "$k" ] || continue
printf '%s\t%s\n' "$(old_hash "$k")" "$k" >> "$work/hashes"
done 3< "$work/keys.uniq"
# ---- 第三輪:正推配對 ----
while IFS="$TAB" read -r repo pg ty suffix <&3; do
[ -n "$pg" ] || continue
key=$(awk -F'\t' -v h="$suffix" '$1==h {print $2; exit}' "$work/hashes")
if [ -z "$key" ]; then
printf '%s\t%s\t%s\n' "$repo" "$pg" '找不到能正推出這個 HASH 的候選鍵' >> "$work/orphans"
continue
fi
printf '%s\t%s\t%s\t%s_%s\n' "$repo" "$pg" "$repo" "$ty" "$(sh "$hash_id" "$key")" >> "$work/map"
done 3< "$work/pending"
# ---- 第四輪:找出指向被搬頁、但自己沒被搬的引用方 ----
# 連結改寫只動被搬的那些頁自己的內容。沒被搬的頁裡那些 [[...]] 這支碰不到,
# 所以至少要把它們列出來交給人改,不能讓它們搬完就靜靜 404。
if [ -s "$work/map" ]; then
cut -f2 "$work/map" | sort -u > "$work/movednames"
cut -f1,2 "$work/map" | sort -u > "$work/movedkeys"
sort -u "$work/allpages" > "$work/allpages.uniq"
while IFS="$TAB" read -r repo pg <&3; do
[ -n "$pg" ] || continue
if grep -qxF "$repo$TAB$pg" "$work/movedkeys"; then continue; fi
sh "$gitea" wiki-get "$repo" "$pg" 2>/dev/null > "$work/one" || continue
# [[顯示文字|頁名]] 的頁名在豎線右邊,[[頁名]] 就是整段。兩種都收成頁名再比對。
targets=$(grep -oE '\[\[[^]]+\]\]' "$work/one" 2>/dev/null \
| sed -e 's/^\[\[//' -e 's/\]\]$//' -e 's/^.*|//' -e 's/^[[:space:]]*//' -e 's/[[:space:]]*$//' \
| sort -u | grep -xF -f "$work/movednames" 2>/dev/null || true)
for t in $targets; do
printf '%s\t%s\t%s\n' "$repo" "$pg" "$t" >> "$work/refs"
done
done 3< "$work/allpages.uniq"
fi
# ---- 報告 ----
echo '對照表(舊存取庫/舊頁名 → 新存取庫/新頁名)'
if [ -s "$work/map" ]; then
while IFS="$TAB" read -r orepo opage nrepo npage <&3; do
[ -n "$opage" ] || continue
printf ' %s/%s → %s/%s\n' "$orepo" "$opage" "$nrepo" "$npage"
done 3< "$work/map"
else
echo ' (沒有需要搬移的頁)'
fi
echo '孤兒頁(需人工處理)'
if [ -s "$work/orphans" ]; then
while IFS="$TAB" read -r repo pg why <&3; do
[ -n "$pg" ] || continue
printf ' %s/%s:%s\n' "$repo" "$pg" "$why"
done 3< "$work/orphans"
else
echo ' (無)'
fi
echo '引用被搬頁、但自己沒被搬(連結改寫碰不到,需人工改)'
if [ -s "$work/refs" ]; then
while IFS="$TAB" read -r repo pg target <&3; do
[ -n "$pg" ] || continue
printf ' %s/%s 指向 %s\n' "$repo" "$pg" "$target"
done 3< "$work/refs"
else
echo ' (無)'
fi
if [ -s "$work/notes" ]; then
echo '備註'
sed 's/^/ /' "$work/notes"
fi
if [ "$apply" -eq 0 ]; then
echo '(預覽模式,沒有寫入任何內容。加 --apply 才真的搬。)'
if [ -s "$work/orphans" ] || [ -s "$work/refs" ]; then exit 3; fi
exit 0
fi
# ---- 搬移第一步:把每一頁寫到新位置 ----
mkdir -p "$work/body"
: > "$work/done" # 舊存取庫 \t 舊頁名 \t 新存取庫 \t 新頁名 \t 序號
: > "$work/failed"
abort() { # $1=訊息。金鑰或 API 出問題就整個停下,把已經做完的部分照樣報出來。
printf '%s\n' "$1" >> "$work/failed"
echo '搬移中止'
sed 's/^/ /' "$work/failed"
if [ -s "$work/manual" ]; then
echo '需人工確認'
sed 's/^/ /' "$work/manual"
fi
exit 1
}
n=0
while IFS="$TAB" read -r orepo opage nrepo npage <&3; do
[ -n "$opage" ] || continue
n=$((n+1))
rc=0
sh "$gitea" wiki-get "$orepo" "$opage" > "$work/body/$n" 2>/dev/null || rc=$?
if [ "$rc" -ne 0 ]; then
printf '%s/%s:讀不回舊頁(結束碼 %s),這一頁不搬。\n' "$orepo" "$opage" "$rc" >> "$work/failed"
continue
fi
# 寫之前先讀目的地。只有 4 才准建新頁——部分搬完後重跑,或目的地那一頁已經由
# wiki-contents.sh 建好,直接寫就是拿舊庫那份蓋掉一個活著的頁。
rc=0
sh "$gitea" wiki-get "$nrepo" "$npage" >/dev/null 2>&1 || rc=$?
case "$rc" in
4) ;;
0)
printf '%s/%s 已經有內容,不覆蓋。請比對它與 %s/%s 之後自行決定。\n' \
"$nrepo" "$npage" "$orepo" "$opage" >> "$work/manual"
continue ;;
7)
abort "$(printf '讀 %s/%s 遇金鑰失效或權限不足(結束碼 7)。' "$nrepo" "$npage")" ;;
*)
abort "$(printf '讀 %s/%s 失敗(結束碼 %s)。' "$nrepo" "$npage" "$rc")" ;;
esac
rc=0
sh "$gitea" wiki-put "$nrepo" "$npage" "$work/body/$n" >/dev/null || rc=$?
if [ "$rc" -ne 0 ]; then
printf '%s/%s:寫不進 %s/%s(結束碼 %s)。\n' "$orepo" "$opage" "$nrepo" "$npage" "$rc" >> "$work/failed"
continue
fi
printf '%s\t%s\t%s\t%s\t%s\n' "$orepo" "$opage" "$nrepo" "$npage" "$n" >> "$work/done"
done 3< "$work/map"
# ---- 搬移第二步:查每一個目的地的絕對網址,改寫 [[...]] 連結 ----
: > "$work/urls" # 舊頁名 \t 新頁名 \t 絕對網址
while IFS="$TAB" read -r orepo opage nrepo npage idx <&3; do
[ -n "$opage" ] || continue
url=$(sh "$gitea" wiki-url "$nrepo" "$npage" 2>/dev/null) || continue
printf '%s\t%s\t%s\n' "$opage" "$npage" "$url" >> "$work/urls"
done 3< "$work/done"
while IFS="$TAB" read -r orepo opage nrepo npage idx <&3; do
[ -n "$opage" ] || continue
rc=0
python3 - "$work/body/$idx" "$work/urls" "$work/body/$idx.new" <<'PY' || rc=$?
import re
import sys
src, urlmap_path, out = sys.argv[1:4]
urlmap = {}
for line in open(urlmap_path, encoding='utf-8'):
parts = line.rstrip('\n').split('\t')
if len(parts) == 3:
urlmap[parts[0]] = (parts[1], parts[2])
text = open(src, encoding='utf-8').read()
def repl(m):
inner = m.group(1)
if '|' in inner:
label, target = inner.split('|', 1)
label, target = label.strip(), target.strip()
else:
target = inner.strip()
label = None
hit = urlmap.get(target)
if not hit:
return m.group(0)
newname, url = hit
# 原本沒有顯示文字的寫法,顯示的就是頁名本身;頁名換了就跟著顯示新頁名。
return '[%s](%s)' % (label if label is not None else newname, url)
new = re.sub(r'\[\[([^\]]+)\]\]', repl, text)
open(out, 'w', encoding='utf-8').write(new)
# 9 代表這一頁沒有需要改寫的連結,不必再寫一次。
raise SystemExit(0 if new != text else 9)
PY
if [ "$rc" -eq 9 ]; then continue; fi
if [ "$rc" -ne 0 ]; then
printf '%s/%s:連結改寫失敗。\n' "$nrepo" "$npage" >> "$work/failed"
continue
fi
if ! sh "$gitea" wiki-put "$nrepo" "$npage" "$work/body/$idx.new" >/dev/null; then
printf '%s/%s:連結改寫後寫不回去。\n' "$nrepo" "$npage" >> "$work/failed"
fi
done 3< "$work/done"
# ---- 搬移第三步:確認新頁讀得回來,才刪舊頁 ----
: > "$work/moved"
while IFS="$TAB" read -r orepo opage nrepo npage idx <&3; do
[ -n "$opage" ] || continue
if ! sh "$gitea" wiki-get "$nrepo" "$npage" >/dev/null 2>&1; then
printf '%s/%s:新頁讀不回來,舊頁保留不刪。\n' "$nrepo" "$npage" >> "$work/failed"
continue
fi
if ! sh "$gitea" wiki-delete "$orepo" "$opage" >/dev/null; then
printf '%s/%s:新頁已就位,舊頁刪不掉,請人工刪除。\n' "$orepo" "$opage" >> "$work/failed"
continue
fi
printf '%s/%s → %s/%s\n' "$orepo" "$opage" "$nrepo" "$npage" >> "$work/moved"
done 3< "$work/done"
echo '已搬移'
if [ -s "$work/moved" ]; then sed 's/^/ /' "$work/moved"; else echo ' (無)'; fi
if [ -s "$work/manual" ]; then
echo '需人工確認'
sed 's/^/ /' "$work/manual"
fi
if [ -s "$work/failed" ]; then
echo '搬移失敗'
sed 's/^/ /' "$work/failed"
exit 1
fi
if [ -s "$work/orphans" ] || [ -s "$work/refs" ] || [ -s "$work/manual" ]; then exit 3; fi
exit 0