如何用手继续

How to continue by hand

机器与分工

The machines and their roles

三台机器各司其职（cognition-os/08_RUNNING.md §2）。lang：分析与拟合（本项目所有 Sherpa/xsherpa 拟合都在 lang 上跑；SAS/HEASoft 通过 /u/Lang3/program/XMMLineMap/xmm_line_map/configs/env.sh 配置）。Virgo：SAS/ESAS 减数据（80 核；拟合环境是 Virgo 上的 ciao-4.18，需 SHERPARC=/media/data8t/huangrui/software/ciao418_sherpa.rc）。lang2：只做谱拟合（CPU \(\le\) 60 %，RAM \(\ge\) 10 GB）。规则：减数据在 Virgo，分析在 lang，从不在 Virgo 上拟合生产任务（复现检验除外，D022/D023）。

Three machines, three roles (cognition-os/08_RUNNING.md §2). lang: analysis and fits (every Sherpa/xsherpa production fit of this project runs on lang; SAS/HEASoft via /u/Lang3/program/XMMLineMap/xmm_line_map/configs/env.sh). Virgo: SAS/ESAS reduction (80 cores; the fit environment is Virgo's ciao-4.18, which needs SHERPARC=/media/data8t/huangrui/software/ciao418_sherpa.rc). lang2: spectral fits only (CPU \(\le\) 60 %, RAM \(\ge\) 10 GB). The rule: reduction on Virgo, analysis on lang, and production fits never run on Virgo (reproduction tests excepted, D022/D023).

来源：Source: cognition-os/08_RUNNING.md §2, cognition-os/00_AI_BRIEFING.md (T02), cognition-os/04_DECISION_LOG.md D022-D023

数据、代码与结果在哪里

Where the data, code and results live

内容What	路径Path
项目根（代码 + 认知 OS）Project root (code + cognition OS)	`/u/Lang3/program/DiffuseXrayCalibration`
数据产品（FITS/PHA/ARF/RMF，永不入 git）Data products (FITS/PHA/ARF/RMF, never committed)	`/u/Lang3/Data/DiffuseXrayCalibration`
考试集状态与审计Exam-set status and audit	`MosaicSelfCal/results/exam_v4_r240/status.csv` 与and `SUMMARY.md`
题目定义（固定天域 \(\times\) ObsID）Problem definitions (fixed cells \(\times\) ObsIDs)	`MosaicSelfCal/problems/v4/r240/{WCDFS,M31,LMC}/problems.csv`
指向角（用 RA_PNT，不用 RA_NOM）Pointing angles (RA_PNT, not RA_NOM)	`MosaicSelfCal/problems/v4/r240/theta_header.csv`
基线分数表Baseline score table	`MosaicSelfCal/results/cdfs_pipeline_comparison_v1.csv`
两个被检验的工具The two tools under test	`esas_data_reduction` （repo，固定版本）(repo, pinned version) 与and `xsherpa` （`releases/current` → v0.3.0-dev7）(`releases/current` → v0.3.0-dev7)
每个任务的记录The record of every task	`.agent/jobs/<日期-任务名>/` （每个任务一个目录，含 REPORT）(one directory per task, with its REPORT)

来源：Source: README.md, MosaicSelfCal/BENCHMARK.md, cognition-os/tool_iteration.md

可以复制的命令（只读，已在本机验证）

Copy-paste commands (read-only, verified on this machine)

以下每条命令都在 2026-10-19 的 lang 上实际运行过；输出贴在命令下方。PY 是 /home/huangrui/software/miniconda3/bin/python 的缩写。所有命令都从项目根 /u/Lang3/program/DiffuseXrayCalibration 出发。

Every command below was actually run on lang on 2026-10-19; its output is pasted under it. PY is short for /home/huangrui/software/miniconda3/bin/python. All commands start from the project root /u/Lang3/program/DiffuseXrayCalibration.

1. 看考试集的状态：多少 case 通过1. Read the exam-set status: how many cases pass

cd /u/Lang3/program/DiffuseXrayCalibration
PY=.agent/jobs/2026-10-19-handbook-build-dxc/figures/read_status.py
$PY   # prints the PASS/EXCLUDED counts from status.csv

输出（2026-10-19 实测）：

Output (measured 2026-10-19):

rows: 45
status: {'PASS': 40, 'EXCLUDED': 5}
obsids: 18
problems: ['LMC-v4-r240-001', 'M31-v4-r240-001', 'M31-v4-r240-002', 'M31-v4-r240-003',
           'WCDFS-v4-r240-001', 'WCDFS-v4-r240-002', 'WCDFS-v4-r240-003']
cameras: {'mos2': 25, 'mos1': 20}
fields: {'WCDFS': 15, 'LMC': 6, 'M31': 24}

来源：Source: MosaicSelfCal/results/exam_v4_r240/status.csv （45 行）与(45 rows) and SUMMARY.md （EXCLUDED=5, PASS=40, total=45）(EXCLUDED=5, PASS=40, total=45)

2. 读一个拟合 JSON：这是所有任务的记录格式2. Read one fit JSON: the record format of every task

cd /u/Lang3/program/DiffuseXrayCalibration
PY=.agent/jobs/2026-10-19-handbook-build-dxc/figures/read_fit_json.py
$PY .agent/jobs/2026-10-16-exam-v4-perobs-fit3/sol/results/fits_json/LMC-v4-r240-001_0137551401_f5e15_fixed_A.json

输出（2026-10-19 实测）：

Output (measured 2026-10-19):

problem_id: LMC-v4-r240-001
obsid: 0137551401
variant: f5e15 | sp_variant: A
fit_status: done
stat: 174.02861012052696
dof: 125
rstat: 1.392
free parameters: 9

来源：Source: .agent/jobs/2026-10-16-exam-v4-perobs-fit3/sol/results/fits_json/LMC-v4-r240-001_0137551401_f5e15_fixed_A.json

3. 考试设计的 --help：理解一个工具的入口3. The exam designer's --help: how to enter a tool

cd /u/Lang3/program/DiffuseXrayCalibration
/home/huangrui/software/miniconda3/bin/python -m MosaicSelfCal.benchmark.design_v4 --help

输出（2026-10-19 实测，节选）：

Output (measured 2026-10-19, excerpt):

usage: design_v4.py [-h] --field {WCDFS,LMC,M31} --out OUT

Exam v4 golden MOS design (D019/D020) from local archive metadata only. Same
geometry and balancing as design_v3 (kept frozen for v3 provenance); v4 adds
r=240 arcsec, a per-field scheduled-exposure floor per ObsID camera, and
excludes XMM mosaic-mode families (single-pointing rule, PI 2026-10-10).

options:
  -h, --help            show this help message and exit
  --field {WCDFS,LMC,M31}
  --out OUT

来源：Source: MosaicSelfCal/benchmark/design_v4.py （argparse 自述）(its own argparse text)

新来者必须遵守的规则

Rules a newcomer must keep

这些规则记录在 cognition-os/08_RUNNING.md §2 与 cognition-os/PITFALLS.md；违反任何一条都会毁掉几天的数据或拟合。前六条来自 T01–T06（信任表）。

These rules are recorded in cognition-os/08_RUNNING.md §2 and cognition-os/PITFALLS.md; breaking any one of them can destroy days of reduction or fitting. The first six come from T01–T06 (the trust table).

MOS1+MOS2 为主，pn 只做交叉检查（T01）：pn 有 out-of-time 事件、更难的 QPB、象限失效。MOS1+MOS2 is the basis; pn is a cross-check only (T01): pn has out-of-time events, a harder QPB, and quadrant failures.
减数据在 Virgo，分析在 lang；从不在 Virgo 上拟合（T02）。Reduction on Virgo, analysis on lang; never fit on Virgo (T02).
拟合永远 set_full_model，不用 set_source（T03）：用错会把归一化差 \(\sim14\times\)。skyarea 必须是 Python float，不能用 Const1D。Always set_full_model, never set_source (T03): the wrong call gives a \(\sim 14\times\) norm error. skyarea must be a Python float, not a Const1D.
QPB 用同一区域的 -back.pi（T04）：它不含 LHB/halo/CXB/SWCX/SP；绝不把别的区域的谱当背景。The QPB is the same region's -back.pi (T04): it holds no LHB/halo/CXB/SWCX/SP; never load another region's spectrum as background.
SP 模型是 rsp_flat(skyarea*SP) 的 bknpower（T05）：BreakE 冻在 3.2 keV，分阶段解冻；每数据集一个 SP 归一。The SP model is a bknpower through rsp_flat(skyarea*SP) (T05): BreakE frozen at 3.2 keV, staged thaw; one SP norm per dataset.
分组：净信噪比 \(\ge\) 6，RMF-FWHM 过采样 \(\le\) 6；扣减谱用 chi2gehrels；不用 WSTAT（T06）：Fitter.run() 会悄悄把统计量重置成 cstat，而 cstat 在扣减谱上崩溃。Grouping: net S/N \(\ge\) 6, RMF-FWHM oversampling \(\le\) 6; chi2gehrels on subtracted data; no WSTAT (T06): Fitter.run() silently resets the statistic to cstat, and cstat crashes on subtracted data.
重叠区域共享事件 → 配对重采样（T07）：所有配对统计都靠它。Overlapping regions share events → paired resampling (T07): every pair statistic here depends on it.
谱的面积永远是 BACKSCAL（PI 2026-10-13）：不要去研究 cheese 与 BACKSCAL 的差异。Spectrum area is always BACKSCAL (PI 2026-10-13): do not investigate cheese-vs-BACKSCAL gaps.
不要在 1.4–2.0 keV 用波段比（PITFALLS）：1.3 keV 仪器线是 MOS 特有的。No band ratios in 1.4–2.0 keV (PITFALLS): the 1.3 keV instrumental line is MOS-only.
conf 永远不是默认步骤（PI 2026-10-13）：先最佳拟合；确需 conf 时评估耗时（LMC 试点：一次 16 数据集的统计评估 0.13 s，全自由 conf 每参数 1–1.5 h）。conf is never a default step (PI 2026-10-13): best fit first; when conf is needed, estimate its cost (LMC pilot: one stat evaluation of 16 datasets = 0.13 s, but full-free conf = 1–1.5 h per parameter).
离轴角用指向（RA_PNT/DEC_PNT），不用档案目标位置（RT-M08）：两者可差 \(1.76'\)，16/40 case 换 bin。用 theta_header.csv。Off-axis angle from the pointing (RA_PNT/DEC_PNT), not the archive target (RT-M08): they differ by up to \(1.76'\), 16/40 cases change bin. Use theta_header.csv.
ESAS cheese 掩膜：1 = 可用（RT-M36）：读反会把移除量报成 48–92 %，实际是 8–52 %。ESAS cheese masks: 1 = usable (RT-M36): reading it backwards reports 48–92 % removed where the truth is 8–52 %.
任何跨 2012-12 的 MOS SP 归一不能合并（R20i/RT-M67）：每个观测带一个 epoch class。Never pool a MOS SP normalisation across 2012-12 (R20i/RT-M67): carry an epoch class per observation.

来源：Source: cognition-os/03_TRUST_TABLE.md T01–T07, cognition-os/PITFALLS.md, cognition-os/08_RUNNING.md §2

下一步：编号计划

The next steps: a numbered plan

计划顺序照 MosaicSelfCal/RESEARCH_PLAN.md 的 SP 路线图与 cognition-os/08_RUNNING.md §7。每一步给出：目标、输入、方法、成功/失败判据、时间估计、先读哪个文件。

The order follows the SP roadmap in MosaicSelfCal/RESEARCH_PLAN.md and cognition-os/08_RUNNING.md §7. Each step gives: goal, inputs, method, success/failure criteria, time estimate, and the file to read first.

第 1 步：R20o —— 一个每观测的 MOS1/MOS2 公共因子？Step 1: R20o — one per-observation MOS1/MOS2 common factor?

目标：检验"一个每观测的、同时缩放 sky 与 SP 的 MOS1/MOS2 因子 \(g_{\text{obs}}\)"能否解释两台相机的不一致（R20n 已排除逐成分的全局标度）。输入：15 个 exam 观测的联合拟合会话（.agent/jobs/2026-10-19-r20o-camera-common-factor/）。方法：模型 A（天空共享）/ B（\(g_{\text{obs}}\times\)MOS1 的 sky+SP）/ B′（只缩 sky）/ C（完全解开），比较 \(\chi^2\) 与 \(g_{\text{obs}}\)，看 A→C 的改善被 B 捕获多少；\(g\) 对 epoch/滤波/CCD 集/SP 份额的回归（LOFO）。 成功判据：B 把 A→C 的 \(\Delta\chi^2\) 捕获 \(\ge\) 大部分，且 \(g\) 跟某个协变量走 → 相机因子成为每个联合/边缘拟合需要的 nuisance（xsherpa 选项）。 失败判据：B 捕获不了 → 某个成分的形状错了（每相机的 SP 谱形状、热晕温度、MOS2 在 0.7–2 keV 的软响应——R20n 的残差正集中在这里）。时间：一个 sol 任务，数小时（已在运行，2026-10-19 10:35 EDT 启动）。先读：.agent/jobs/2026-10-19-r20o-camera-common-factor/REPORT.sol.md（完成后）。

Goal: test whether one per-observation MOS1/MOS2 factor \(g_{\text{obs}}\), shared by sky and SP, explains the camera disagreement (R20n already excluded per-component global scales). Inputs: the joint-fit sessions of the 15 exam observations (.agent/jobs/2026-10-19-r20o-camera-common-factor/). Method: models A (sky linked) / B (\(g_{\text{obs}}\times\) MOS1 sky+SP) / B′ (sky only) / C (unlinked); compare \(\chi^2\) and \(g_{\text{obs}}\); how much of the A→C gain B captures; regress \(g\) on epoch/filter/CCD set/SP fraction (LOFO). Success: B captures most of the A→C \(\Delta\chi^2\) and \(g\) follows a covariate → the camera factor becomes the nuisance every joint/edge fit needs (an xsherpa option). Failure: B captures little → a component shape is wrong (SP spectral shape per camera, hot-halo temperature, MOS2 soft response at 0.7–2 keV — where R20n's residuals peak). Time: one sol task, hours (running since 2026-10-19 10:35 EDT). Read first: .agent/jobs/2026-10-19-r20o-camera-common-factor/REPORT.sol.md (when it lands).

来源：Source: cognition-os/08_RUNNING.md §3 (R20o), §7 item 1, MosaicSelfCal/RESEARCH_PLAN.md R20n/R20o 行R20n/R20o rows

第 2 步：SAS 22.1 受影响臂重导出（等 Virgo 空闲）Step 2: re-derive the affected SAS 22.1 arms (when Virgo is free)

目标：把 SAS 22.1 后端（esas v1.8.0-dev.5，commit 431a054）从"代码已审"推进到"数据可用"。输入：dev.5 修复清单（ARF = arfgen 默认开关；QPB STAT_ERR/TUNIT 缺陷已修，tool_iteration 45）。方法：在 Virgo（load \(\le\) 48 时允许）重导出受影响的臂，然后做 D026 的 apple-to-apple 比较（同 ODF、同 CCF、同 GTI、同区域，逐阶段比较）。 成功判据：golden 通过 + 每阶段差异可归因 → PI 签字进入生产。 失败判据：差异不可归因 → 保持 SAS 20 生产（v1.7.9）。时间：数个 Virgo 小时；受数据量决定。先读：cognition-os/tool_iteration.md item 45，.agent/jobs/2026-10-17-esas-sas22-backend/。

Goal: move the SAS 22.1 backend (esas v1.8.0-dev.5, commit 431a054) from "code reviewed" to "usable for data". Inputs: the dev.5 fix list (ARF = arfgen default switches; the QPB STAT_ERR/TUNIT defects fixed, tool_iteration 45). Method: re-derive the affected arms on Virgo (allowed at load \(\le\) 48), then the D026 apple-to-apple comparison (same ODF, CCF, GTIs, regions, compared stage by stage). Success: golden passes + every difference attributed → PI sign-off for production. Failure: unattributable differences → stay on SAS 20 (v1.7.9). Time: several Virgo hours, set by the data volume. Read first: cognition-os/tool_iteration.md item 45, .agent/jobs/2026-10-17-esas-sas22-backend/.

来源：Source: cognition-os/08_RUNNING.md §3 (esas dev.5), §7 item 2, cognition-os/04_DECISION_LOG.md D026

第 3 步：R13e 全局最小值审计收尾Step 3: close the R13e global-minimum audit

目标：回答"生产/复现拟合多久不在统计量最小值上"，并给出 xsherpa 的多起点保护建议。输入：.agent/jobs/2026-10-18-r13e-global-minimum-audit/（audit 相位进行中，2026-10-19 10:33 EDT 无错误）。方法：等 REPORT 头部出现 COMPLETE，然后把一个修复后的 baseline 对 R13b 验证。 成功判据：修复后的 baseline 复现 R13b 的数 → 审计关闭，多起点保护进入 xsherpa 议程（tool_iteration 42）。 失败判据：baseline 修复引入新差异 → 找出退化参数组（SP/halo/CXB），修正审计设计。时间：审计本身数小时；验证一次 refit 约 1 h。先读：.agent/jobs/2026-10-18-r13e-global-minimum-audit/REPORT.sol.md（等 COMPLETE）。

Goal: answer "how often the production/recovery fit is not at the statistic minimum", and propose the multi-start safeguard for xsherpa. Inputs: .agent/jobs/2026-10-18-r13e-global-minimum-audit/ (audit phase running, no errors as of 2026-10-19 10:33 EDT). Method: wait for the REPORT header to say COMPLETE, then verify one repaired baseline against R13b. Success: the repaired baseline reproduces R13b → audit closed; the multi-start safeguard joins the xsherpa agenda (tool_iteration 42). Failure: the repair introduces new differences → identify the degenerate parameter group (SP/halo/CXB) and fix the audit design. Time: hours for the audit itself; one refit verification about 1 h. Read first: .agent/jobs/2026-10-18-r13e-global-minimum-audit/REPORT.sol.md (wait for COMPLETE).

来源：Source: cognition-os/08_RUNNING.md §3 (R13e), MosaicSelfCal/RESEARCH_PLAN.md R13e 行R13e row

第 4 步：把 R20 家族的 SP 水平、QPB 协方差、SP 模板合并成每观测 SP 产品Step 4: merge the R20 family's SP level, QPB covariance and SP template into one per-observation SP product

目标：给每个观测一个可引用的 SP 产品（水平 + 空间模板 + 误差），使第 3 步成像的 SP 扣除可以标准化。输入：R20b/c/d/e/f/g/h/i/j/k/l/m/n/o 的结论（见现状页的表）。方法：把已测的量合并——SP 水平用 tracer+LOO 的条件值与上限；模板用 ESAS flare 图（R18c）；QPB 协方差用 R20e/f 的结构函数与构造协方差做包络。 成功判据：每个 exam 观测的 SP 产品在自身误差内同时满足谱、空间、时间三个杠杆 → 进入工具（esas SP 成像模板 + xsherpa SP 率选项）。 失败判据：三杠杆矛盾 → SP 水平继续是条件值，成像 SP 扣除保持逐案标注。时间：一个分析任务，约 1 天。先读：MosaicSelfCal/RESEARCH_PLAN.md 的 SP 路线图一节。

Goal: give every observation a quotable SP product (level + spatial template + error), so that the step-3 image SP subtraction can be standardised. Inputs: the conclusions of R20b/c/d/e/f/g/h/i/j/k/l/m/n/o (see the table on the state page). Method: merge what is measured — the SP level from the tracer+LOO conditional values and upper limits; the template from the ESAS flare map (R18c); the QPB covariance as an envelope from the R20e/f structure function and the construction covariance. Success: every exam observation's SP product satisfies the spectral, spatial and time levers within its own errors → into the tools (ESAS SP image template + xsherpa SP-rate option). Failure: the three levers contradict → the SP level stays conditional and image SP subtraction stays per-case labelled. Time: one analysis task, about a day. Read first: the SP roadmap section of MosaicSelfCal/RESEARCH_PLAN.md.

来源：Source: MosaicSelfCal/RESEARCH_PLAN.md （SP 路线图）(the SP roadmap), cognition-os/08_RUNNING.md §7 item 1

第 5 步：验收——PI 三步目标照 `BENCHMARK.md` 的分数重新打分Step 5: acceptance — rescore the PI's three-step goal with the `BENCHMARK.md` scores

目标：检验修正是否真的改进了基线（CDFS P0 \(\tau/\bar{I}=0.246\)、M31 P0 \(\tau/\bar{I}=0.14\)，cdfs_pipeline_comparison_v1.csv）。输入：修正后的工具版本（固定版本号）在冻结考试集上的重跑。方法：SP 路线图的验收标准：同一指向的高/低 SP 时间桶给同一面亮度；M31/LMC/WCDFS 配对斜率归零（\(\chi^2/\text{dof}\sim1\)）；mosaic 看不到 vignetting。 成功判据：分数改进且配对区间不含 0；否则写"持平"。 失败判据：分数不动或恶化 → 回到诊断（哪一杠杆没被修正覆盖）。时间：一轮完整考试重跑 + 打分，约 1–2 天（含 Virgo 减数据排队）。先读：MosaicSelfCal/BENCHMARK.md（分数定义）。

Goal: test whether the fixes actually improve the baseline (CDFS P0 \(\tau/\bar{I}=0.246\), M31 P0 \(\tau/\bar{I}=0.14\), cdfs_pipeline_comparison_v1.csv). Inputs: reruns of the fixed tool versions (pinned) on the frozen exam set. Method: the SP roadmap's acceptance: high- and low-SP time bins of one pointing give the same surface brightness; M31/LMC/WCDFS pair slopes zero (\(\chi^2/\text{dof}\sim1\)); mosaics show no visible vignetting. Success: scores improve with a paired interval excluding 0; otherwise write "No significant difference". Failure: scores do not move or worsen → back to diagnosis (which lever the fix did not cover). Time: one full exam rerun + scoring, about 1–2 days (including the Virgo reduction queue). Read first: MosaicSelfCal/BENCHMARK.md (score definitions).

来源：Source: MosaicSelfCal/BENCHMARK.md, MosaicSelfCal/RESEARCH_PLAN.md （SP 路线图第 6 步）(SP roadmap step 6), MosaicSelfCal/results/cdfs_pipeline_comparison_v1.csv

Restricted