diff --git a/.planning/codex-rollout-history-merge/findings.md b/.planning/codex-rollout-history-merge/findings.md index 40ec1cc..2716acd 100644 --- a/.planning/codex-rollout-history-merge/findings.md +++ b/.planning/codex-rollout-history-merge/findings.md @@ -6,3 +6,12 @@ - codebase-memory 项目 `home-cc-web` 索引状态为 ready（8208 nodes / 19300 edges）。 - 工作区存在用户已有未提交改动：`lib/ccweb-mcp-server.js`、`scripts/ccweb-message-reply-unit.js`，本任务不触碰。 + +- 2026-09-21：db04e08 已删除全量 native 替换，但 merge 的数值 baseIndex 回退仍不可靠；messagesEquivalent 不支持 codexAppTurnId 与 native turnId 跨来源对齐。 +- 前端 prepend/reconcile 仍以相同下标跳过不同 ID，缓存也会按下标覆盖不同消息。 +- sessions/35394008-4f0c-43c0-a481-b622d9e6b884.json 本机暂未找到，不能假装完成迁移。 + +- 自动回传使用 buildCrossConversationReplyAutoRunText 的“子对话回传已返回……”包装，需与“来自……”投递包装分别精确识别；两者均不能作为普通用户消息。 +- response_item 中存在 AGENTS/environment 等用户角色上下文；有 event_msg.user_message 时以事件为权威，仅从匹配 response 补充 ID。 +- done 原来不携带持久化助手身份，结束后的本地 stream 与刷新消息无法去重；现已将最终消息及合并后下标通过 done 传递。 +- 新迁移能力通过已认证 WebSocket 在服务端同步检查/备份/写入；单文件提供 --migrate-session-history 入口，默认预览，--apply 写入。 diff --git a/.planning/codex-rollout-history-merge/progress.md b/.planning/codex-rollout-history-merge/progress.md index 16b8f86..3f4595a 100644 --- a/.planning/codex-rollout-history-merge/progress.md +++ b/.planning/codex-rollout-history-merge/progress.md @@ -5,3 +5,18 @@ - 2026-09-20：rollout 解析按 `task_started` / `turn_context` / 完成事件聚合同一 turn，保留 function/custom tool，过滤跨对话内部 user 输入。 - 2026-09-20：前端缓存、DOM、重连和实时消息按稳定键去重，普通用户消息把本地 ID 作为 `clientMessageId` 发给服务端。 - 2026-09-20：通过 `node --check`、完整 `npm run regression`、`history-recall`、`session-switch-race`、既有单测和 `git diff --check`。 + +- 2026-09-21：收紧 native 合并，仅首条稳定标识/完整时间戳和内容摘要边界可信；废止数值基线切 native，跨线程和分段 turn 保守回退。 +- 2026-09-21：新增结构化截断元数据、修正消息名额及基线覆盖；服务端新增 idle 状态的预览/备份/原子迁移入口，脚本默认预览。 +- 2026-09-21：用户确认目标旧会话在其他机器，本轮不操作远端会话数据。 +- 2026-09-21：历史服务端 9 项、rollout 10 项、前端专项行为回归已通过；历史召回和切会话专项通过。额外补齐流式转持久化的身份绑定，准备全量和浏览器验证。 + +- 2026-09-21：全量 npm run regression 通过；关联 ccweb-message-reply、failed-insert-card、ccweb-list-user-inputs、javascript-session-runtime 单测和语法检查通过。最终 auto-run 过滤后的 history-recall、session-switch-race 复跑通过。 +- 2026-09-21：使用 /tmp/ccweb-bun.7eHd2F/node_modules/@oven/bun-linux-x64-baseline/bin/bun（1.4.2）开始构建发布包；Firefox 隔离浏览器验证执行中。 + +- 2026-09-21：CentOS 7 baseline 发布包构建完成，MCP initialize、单文件 --migrate-session-history 入口、tar 结构均通过烟测；SHA-256 为 7c8fe9bcb1be8c860e10578a1fac925c51785f83dd6449cbfa57f09dcc370c9e。 +- 2026-09-21：无本机目标数据迁移；本机列表仍有其他 running 对话，按 AGENTS.md 不重启本机服务。用户本轮要求异机使用的验证、打包与推送。 + +- 2026-09-21：最终全量回归再次通过。Firefox 156.0 / geckodriver 0.37.1 真浏览器验证通过：68 条快照、483 条 native；加载、运行、刷新、真实 WebSocket 重连、done、done 后刷新六阶段保持顺序且无重复，historySource=merged。 +- 浏览器证据：/tmp/cc-web-history-browser-KnMvZ7/evidence.json；运行中截图 /tmp/cc-web-history-browser-KnMvZ7/refreshed.png 已目视核对。隔离进程已清理。 +- 本轮目标在异机，未迁移或重启生产会话；发布包提供 idle 安全迁移工具。准备提交全部源码、测试、计划及发布包并推送 main。 diff --git a/.planning/codex-rollout-history-merge/task_plan.md b/.planning/codex-rollout-history-merge/task_plan.md index 636140c..1a12bed 100644 --- a/.planning/codex-rollout-history-merge/task_plan.md +++ b/.planning/codex-rollout-history-merge/task_plan.md @@ -7,14 +7,14 @@ ## 合并契约 - 持久化快照中去掉 `ccwebPersistenceNotice` 后的消息保持原有顺序，是当前会话的权威尾部；native 只能补充快照之前的消息。 -- 快照压缩时保存 `historySnapshotBaseIndex`（native 逻辑消息索引）与 `historySnapshotCount`，旧快照缺字段时按稳定 ID/最长公共后缀推断，无法确认边界时保留快照。 +- 快照压缩时保存 `historySnapshotBaseIndex`（cc-web 逻辑消息索引）与 `historySnapshotCount`；只按首条稳定 ID 别名或完整时间戳和内容摘要确认 native 边界，禁止用数值基线直接切 native。 - 合并顺序固定为：native 更早消息 → cc-web 持久化消息 → 当前运行中的消息；相同稳定 ID 只保留 cc-web 消息，ID 冲突且内容不一致时以持久化消息为准并记录告警。 - native 缺失、解析异常、边界无法确认或合并结果不能证明快照尾部完整时，返回 cc-web 快照，不把未经确认的 native 消息插入当前视图。 - `session_info`、`resume_session_result`、`session_history_chunk` 均调用同一解析结果；分页只对该结果切片。 ## 稳定标识契约 -- 用户消息：优先 `clientMessageId`，兼容旧字段 `id`；缺失时生成并持久化 `client:`。 +- 消息已有 `id` 始终保留；新增用户消息保留 `clientMessageId`，缺失时生成并持久化 UUID。 - Codex App 助手消息：使用 `codexAppTurnKey`，其输入为 session/thread/turn 的稳定字段；旧消息缺失时从对应运行状态或时间戳生成一次并保存。 - native rollout 助手消息：使用 `nativeTurnKey`，由 threadId + turnId/turn_context 组成；同一 turn 跨刷新保持一致。 - 跨对话回传：保留 `replyToRequestId`（兼容 `crossConversation.replyRequestId`），同时生成 `reply:` 作为稳定消息 ID；来源元数据始终保留。 @@ -23,7 +23,7 @@ ## Turn 聚合契约 - 以 `turn_id`、`turn_context.turn_id`、`turn_context.id` 或 session/thread 上下文组成 turn key；缺失时使用相邻事件窗口的稳定 fallback key。 -- 同一 turn 内按 rollout 文件顺序合并文本片段、工具调用与工具结果；只有遇到下一条用户消息、turn 完成/失败事件或文件结束才 flush。 +- 同一 turn 内按 rollout 文件顺序合并文本片段、工具调用与工具结果；只有遇到新 turn、下一条真实用户消息、turn 完成/失败事件或文件结束才 flush。 - tool call/result 绑定同一调用 ID；空 turn 不产出气泡；跨对话内部 user 回传不作为普通 user 消息插入。 ## 前端契约 @@ -60,3 +60,15 @@ | 错误 | 尝试 | 处理 | |---|---|---| + + +## 2026-09-21 反馈修订 + +- [complete] 收紧历史边界并补齐解析、前端稳定标识 +- [complete] 补齐结构化快照元数据及安全迁移入口 +- [complete] 运行历史、重连回归与必要浏览器验证 +- [complete] 检查旧会话迁移条件及服务重启条件 +- [complete] 重建 CentOS 7 发布包并烟测 +- [in_progress] 提交全部修改并推送 + +修订约束：不以 historySnapshotBaseIndex 直接切 native；只确认快照首条的可靠边界，不要求尚未进入 rollout 的最近消息也匹配。ID 别名优先，完整时间戳与内容摘要次之；未确认则只保留快照。目标旧会话只在 idle 且原文件未变化时备份和原子迁移。用户要求不再审计，本轮仅做实现与必要回归。update_plan 工具当前未提供，以本计划和 CSV 同步跟踪。 diff --git a/dist-exe/cc-web-bun-linux-x64-baseline.tar.gz b/dist-exe/cc-web-bun-linux-x64-baseline.tar.gz index eba8bfd..8c28cf1 100644 Binary files a/dist-exe/cc-web-bun-linux-x64-baseline.tar.gz and b/dist-exe/cc-web-bun-linux-x64-baseline.tar.gz differ diff --git a/lib/codex-rollouts.js b/lib/codex-rollouts.js index 00ad285..15bf7be 100644 --- a/lib/codex-rollouts.js +++ b/lib/codex-rollouts.js @@ -22,23 +22,37 @@ function createCodexRolloutStore(deps) { return crypto.createHash('sha256').update(String(value || '')).digest('hex').slice(0, 24); } - function extractTurnKey(entry, payload, fallback) { - const context = payload?.turn_context || entry?.turn_context || {}; - const value = entry?.turn_id - || payload?.turn_id - || context?.turn_id - || context?.id - || null; - return value ? String(value) : fallback; + function getRolloutTurnId(entry) { + const payload = entry?.payload || {}; + const context = payload.turn_context || entry?.turn_context || {}; + const value = payload.turn_id + || payload.internal_chat_message_metadata_passthrough?.turn_id + || entry?.turn_id + || context.turn_id + || context.id + || (entry?.type === 'turn_context' ? payload.id : null); + return value ? String(value) : null; } function extractCcwebSourceConversation(text) { - const match = String(text || '').match(/^来自「([^」]+)」对话（ID:\s*([0-9a-fA-F-]{36})）的消息：/); - if (!match) return null; - return { title: match[1], id: match[2].toLowerCase() }; + const value = String(text || '').replace(/\r\n/g, '\n'); + const match = value.match(/^来自「([^」]+)」对话（ID:\s*([0-9a-fA-F-]{36})）的消息：/); + if (match) return { title: match[1], id: match[2].toLowerCase() }; + // 自动继续包装必须同时满足固定开头、请求标识、目标会话和正文结构。 + // 普通用户只是在正文中提到“子对话回传”不能被当作内部消息过滤。 + const returned = value.match(/^子对话回传已返回，但已返回不等于已完成。请先检查返回内容是否完整满足原始请求，再决定继续推进、补问目标对话或向用户汇报。\n\nrequestId：([^\n]+)\n目标对话：「([^」]+)」（ID: ([0-9a-fA-F-]{36}|未知)）\n\n原始请求：\n[\s\S]*?\n\n返回正文：\n/); + if (!returned) return null; + return { title: returned[2], id: returned[3] === '未知' ? null : returned[3].toLowerCase() }; } function parseCodexRolloutLines(lines) { + const entries = []; + for (const line of lines) { + try { + const entry = JSON.parse(line); + if (entry && typeof entry === 'object') entries.push(entry); + } catch {} + } const messages = []; const pendingToolCalls = new Map(); const meta = { @@ -53,10 +67,14 @@ function createCodexRolloutStore(deps) { }; const totalUsage = { inputTokens: 0, cachedInputTokens: 0, outputTokens: 0 }; let currentAssistant = null; + let currentTurnId = null; let currentTurnKey = null; + let currentSegment = null; let implicitTurnSequence = 0; - let sawRealUserMessage = false; - const fallbackUserMessages = []; + let lastUser = null; + const userMessagesById = new Map(); + const assistantsByTurn = new Map(); + const originalAssistantIds = new WeakMap(); function rememberSourceConversation(text) { if (meta.sourceConversationId) return; @@ -66,42 +84,190 @@ function createCodexRolloutStore(deps) { meta.sourceConversationTitle = sourceConversation.title; } - function ensureAssistant(ts, turnKey = null) { - // 没有 turn_id 的旧 rollout 仍需在每次新助手输出时生成新的隐式轮次键。 - // 否则不同轮次会共用同一个 nativeTurnKey，前端按稳定键去重时会误删后续回复。 - const resolvedTurnKey = turnKey || currentTurnKey || `${meta.threadId || 'native'}:implicit:${implicitTurnSequence += 1}`; - if (currentAssistant && currentTurnKey !== resolvedTurnKey) flushAssistant(); - currentTurnKey = resolvedTurnKey; - if (!currentAssistant) { - currentAssistant = { - role: 'assistant', - content: '', - toolCalls: [], - timestamp: ts || null, - nativeTurnKey: `native:${meta.threadId || 'unknown-thread'}:${currentTurnKey}`, - id: `native:${stableHash(`${meta.threadId || ''}:${currentTurnKey}`)}`, - }; - } else if (!currentAssistant.timestamp && ts) { - currentAssistant.timestamp = ts; + function messageIdentity(entry) { + const payload = entry.payload || {}; + const passthrough = payload.internal_chat_message_metadata_passthrough || {}; + const clientMessageId = payload.clientMessageId || payload.client_user_message_id + || passthrough.clientMessageId || passthrough.client_user_message_id || null; + const id = payload.id || payload.message_id || entry.id || entry.message_id || null; + return { id: id ? String(id) : null, clientMessageId: clientMessageId ? String(clientMessageId) : null }; + } + + // 有 user_message 事件时，它才是用户输入记录；response_item 中还混有 + // AGENTS、environment 等运行上下文，只有配对成功的记录才可补充身份字段。 + const userEventsByText = new Map(); + const responseUsers = []; + const responseForUserEvent = new Map(); + let scannedTurnId = null; + let hasUserEvents = false; + for (let index = 0; index < entries.length; index += 1) { + const entry = entries[index]; + scannedTurnId = getRolloutTurnId(entry) || scannedTurnId; + if (entry.type === 'event_msg' && entry.payload?.type === 'user_message') { + hasUserEvents = true; + const text = String(entry.payload.message || '').trim(); + const candidates = userEventsByText.get(text) || []; + candidates.push({ entry, index, turnId: scannedTurnId }); + userEventsByText.set(text, candidates); + } else if (entry.type === 'response_item' && entry.payload?.type === 'message' && entry.payload.role === 'user') { + responseUsers.push({ entry, index, turnId: scannedTurnId }); } - return currentAssistant; + if (entry.type === 'event_msg' && ['task_complete', 'task_completed', 'turn_complete', 'turn_completed', 'turn_failed', 'turn_aborted'].includes(entry.payload?.type)) { + scannedTurnId = null; + } + } + for (const response of responseUsers) { + const text = extractCodexMessageText(response.entry.payload.content).trim(); + let matched = null; + for (const candidate of userEventsByText.get(text) || []) { + if (responseForUserEvent.has(candidate.entry)) continue; + if (response.turnId && candidate.turnId && response.turnId !== candidate.turnId) continue; + if (!matched || Math.abs(candidate.index - response.index) < Math.abs(matched.index - response.index)) matched = candidate; + } + if (matched) responseForUserEvent.set(matched.entry, response.entry); + } + + function assignAssistantIdentity(assistant) { + const segmentSuffix = currentSegment ? `:segment:${currentSegment}` : ''; + assistant.turnId = currentTurnId; + assistant.nativeThreadId = meta.threadId; + assistant.nativeTurnKey = `native:${meta.threadId || 'unknown-thread'}:${currentTurnKey}${segmentSuffix}`; + assistant.id = originalAssistantIds.get(assistant) + || `history-assistant:${stableHash(`${meta.threadId || ''}:${currentTurnKey}${segmentSuffix}`)}`; + if (currentSegment) assistant.nativeTurnSegment = currentSegment; } function flushAssistant() { - if (!currentAssistant) return; - if ((currentAssistant.content || '').trim() || currentAssistant.toolCalls.length > 0) { + if (currentAssistant && ((currentAssistant.content || '').trim() || currentAssistant.toolCalls.length > 0)) { + if (!currentAssistant.turnId && !originalAssistantIds.has(currentAssistant)) { + const content = currentAssistant.content || JSON.stringify(currentAssistant.toolCalls); + currentAssistant.id = `history-assistant:${stableHash(`${meta.threadId || ''}:${currentAssistant.timestamp || ''}:${content}`)}`; + } messages.push(currentAssistant); } currentAssistant = null; - currentTurnKey = null; pendingToolCalls.clear(); } - for (const line of lines) { - const trimmed = line.trim(); - if (!trimmed) continue; - let entry; - try { entry = JSON.parse(trimmed); } catch { continue; } + function selectTurn(turnId) { + if (!turnId || turnId === currentTurnId) return; + const previousTurnKey = currentTurnKey; + if (currentTurnId) flushAssistant(); + currentTurnId = turnId; + currentTurnKey = turnId; + currentSegment = null; + // 老格式可能在首段输出后才提供 turn_context，补齐身份而不凭空拆出气泡。 + if (currentAssistant) { + assistantsByTurn.delete(previousTurnKey); + assistantsByTurn.set(currentTurnKey, [currentAssistant]); + assignAssistantIdentity(currentAssistant); + } + if (lastUser && !lastUser.assistantStarted && lastUser.turnKey === previousTurnKey) { + lastUser.turnKey = currentTurnKey; + lastUser.message.turnId = currentTurnId; + } + } + + function ensureAssistant(entry) { + selectTurn(getRolloutTurnId(entry)); + if (!currentTurnKey) currentTurnKey = `implicit:${implicitTurnSequence += 1}`; + if (!currentAssistant) { + currentAssistant = { role: 'assistant', content: '', toolCalls: [], timestamp: entry.timestamp || null }; + assignAssistantIdentity(currentAssistant); + const assistants = assistantsByTurn.get(currentTurnKey) || []; + assistants.push(currentAssistant); + assistantsByTurn.set(currentTurnKey, assistants); + } + if (lastUser) lastUser.assistantStarted = true; + return currentAssistant; + } + + function rememberAssistantMessageId(assistant, entry) { + const { id } = messageIdentity(entry); + if (!id) return; + if (!originalAssistantIds.has(assistant)) { + originalAssistantIds.set(assistant, id); + assistant.id = id; + } + if (!assistant.nativeMessageIds) assistant.nativeMessageIds = []; + if (!assistant.nativeMessageIds.includes(id)) assistant.nativeMessageIds.push(id); + } + + function appendUser(entry, text) { + text = String(text || '').trim(); + if (!text) return; + rememberSourceConversation(text); + // 内部回传只是 native 的输入包装，不能冒充用户，也不能打断正在聚合的助手。 + if (extractCcwebSourceConversation(text)) return; + const identity = messageIdentity(entry); + const turnId = getRolloutTurnId(entry); + const ts = entry.timestamp || null; + const known = (identity.id && userMessagesById.get(identity.id)) + || (identity.clientMessageId && userMessagesById.get(identity.clientMessageId)); + const existing = known; + if (existing) { + if (identity.clientMessageId) existing.message.clientMessageId = identity.clientMessageId; + if (identity.id) { + if (!existing.hasOriginalId) { + existing.message.id = identity.id; + existing.hasOriginalId = true; + } + userMessagesById.set(identity.id, existing); + } + if (identity.clientMessageId) userMessagesById.set(identity.clientMessageId, existing); + return; + } + + selectTurn(turnId); + const hadAssistant = (assistantsByTurn.get(currentTurnKey) || []).length > 0; + flushAssistant(); + if (!currentTurnId) { + currentTurnKey = `implicit:${implicitTurnSequence += 1}`; + currentSegment = null; + } + const id = identity.id || (identity.clientMessageId ? `client:${identity.clientMessageId}` : null) + || `history-user:${stableHash(`${meta.threadId || ''}:${currentTurnId || ''}:${ts || ''}:${text}`)}`; + if (hadAssistant && currentTurnId) { + // steer 是真实用户输入：保持前后邻接，同时避免同 turn 的两个助手段互相去重。 + for (const assistant of assistantsByTurn.get(currentTurnKey) || []) { + if (!assistant.nativeTurnSegment) assistant.nativeTurnSegment = 'initial'; + } + currentSegment = stableHash(id); + } + const message = { role: 'user', content: text, timestamp: ts, id, turnId: currentTurnId, nativeThreadId: meta.threadId }; + if (identity.clientMessageId) message.clientMessageId = identity.clientMessageId; + messages.push(message); + lastUser = { message, turnKey: currentTurnKey, assistantStarted: false, hasOriginalId: !!identity.id }; + if (identity.id) userMessagesById.set(identity.id, lastUser); + if (identity.clientMessageId) userMessagesById.set(identity.clientMessageId, lastUser); + if (!meta.title) meta.title = text.slice(0, 80).replace(/\n/g, ' '); + } + + function appendTool(entry, output = false) { + const payload = entry.payload || {}; + const assistant = ensureAssistant(entry); + const fallbackName = payload.type.startsWith('custom_') ? 'CustomToolCall' + : payload.type.startsWith('mcp_') ? 'McpToolCall' : 'FunctionCall'; + const toolUseId = payload.call_id || payload.id + || `native-tool:${stableHash(`${assistant.nativeTurnKey}:${payload.type}:${entry.timestamp || ''}:${JSON.stringify(payload)}`)}`; + let tool = pendingToolCalls.get(toolUseId); + if (!tool) { + tool = { name: payload.name || payload.tool || fallbackName, id: toolUseId, input: null, done: false }; + assistant.toolCalls.push(tool); + pendingToolCalls.set(toolUseId, tool); + } + if (!output) { + tool.name = payload.name || payload.tool || tool.name; + tool.input = sanitizeToolInput(tool.name, payload.input ?? payload.arguments ?? ''); + } + if (output || payload.status === 'completed' || payload.status === 'failed' || payload.result !== undefined) { + tool.done = true; + const result = payload.output ?? payload.result; + if (result !== undefined) tool.result = (typeof result === 'string' ? result : JSON.stringify(result)).slice(0, 2000); + } + } + + for (const entry of entries) { const ts = entry.timestamp || null; if (ts) meta.updatedAt = ts; @@ -113,21 +279,22 @@ function createCodexRolloutStore(deps) { continue; } - if (entry.type === 'event_msg' && ['task_started', 'turn_started'].includes(entry.payload?.type)) { - flushAssistant(); - currentTurnKey = extractTurnKey(entry, entry.payload, `${meta.threadId || 'native'}:turn:${implicitTurnSequence += 1}`); + if ((entry.type === 'event_msg' && ['task_started', 'turn_started'].includes(entry.payload?.type)) + || entry.type === 'turn_context') { + selectTurn(getRolloutTurnId(entry)); + if (!currentTurnKey) currentTurnKey = `implicit:${implicitTurnSequence += 1}`; continue; } - if (entry.type === 'turn_context') { - const turnKey = extractTurnKey(entry, entry.payload, currentTurnKey || `${meta.threadId || 'native'}:turn:${implicitTurnSequence += 1}`); - if (currentAssistant && currentTurnKey !== turnKey) flushAssistant(); - currentTurnKey = turnKey; - continue; - } - - if (entry.type === 'event_msg' && ['task_complete', 'turn_completed', 'turn_failed', 'turn_aborted'].includes(entry.payload?.type)) { - flushAssistant(); + if (entry.type === 'event_msg' && ['task_complete', 'task_completed', 'turn_complete', 'turn_completed', 'turn_failed', 'turn_aborted'].includes(entry.payload?.type)) { + const turnId = getRolloutTurnId(entry); + // 迟到的上一轮结束事件不能截断当前轮。 + if (!turnId || !currentTurnId || turnId === currentTurnId) { + flushAssistant(); + currentTurnId = null; + currentTurnKey = null; + currentSegment = null; + } continue; } @@ -147,119 +314,41 @@ function createCodexRolloutStore(deps) { } if (entry.type === 'event_msg' && entry.payload?.type === 'user_message') { - const text = String(entry.payload?.message || '').trim(); - if (text) { - sawRealUserMessage = true; - flushAssistant(); - const sourceConversation = extractCcwebSourceConversation(text); - rememberSourceConversation(text); - // cc-web 跨对话投递在 native rollout 中会留下内部 user 输入。 - // 目标会话已经保存了带来源的消息，这里不再伪造普通 user 气泡。 - if (sourceConversation) continue; - if (!meta.title) meta.title = text.slice(0, 80).replace(/\n/g, ' '); - messages.push({ - role: 'user', - content: text, - timestamp: ts, - id: `native-user:${stableHash(`${meta.threadId || ''}:${ts || ''}:${text}`)}`, - }); - } + const response = responseForUserEvent.get(entry); + const responseIdentity = response ? messageIdentity(response) : {}; + const eventIdentity = messageIdentity(entry); + const enrichedEntry = response ? { + ...entry, + payload: { + ...entry.payload, + id: eventIdentity.id || responseIdentity.id, + clientMessageId: eventIdentity.clientMessageId || responseIdentity.clientMessageId, + turn_id: getRolloutTurnId(entry) || getRolloutTurnId(response), + }, + } : entry; + appendUser(enrichedEntry, entry.payload?.message); continue; } if (entry.type !== 'response_item') continue; - const payload = entry.payload || {}; - switch (payload.type) { - case 'message': { - if (payload.role === 'assistant') { - const text = extractCodexMessageText(payload.content); - if (text.trim()) { - appendAssistantContent(ensureAssistant(ts, extractTurnKey(entry, payload, currentTurnKey)), text); - } - } else if (payload.role === 'user' && !sawRealUserMessage) { - const text = extractCodexMessageText(payload.content); - if (text.trim()) { - rememberSourceConversation(text); - if (!extractCcwebSourceConversation(text)) { - fallbackUserMessages.push({ - role: 'user', - content: text, - timestamp: ts, - id: `native-user:${stableHash(`${meta.threadId || ''}:${ts || ''}:${text}`)}`, - }); - } - } + if (payload.type === 'message') { + const text = extractCodexMessageText(payload.content); + if (payload.role === 'assistant' && text.trim()) { + const assistant = ensureAssistant(entry); + rememberAssistantMessageId(assistant, entry); + appendAssistantContent(assistant, text); + } else if (payload.role === 'user' && !hasUserEvents) { + appendUser(entry, text); } - break; - } - case 'function_call': { - const assistant = ensureAssistant(ts, extractTurnKey(entry, payload, currentTurnKey)); - const toolUseId = payload.call_id || payload.id || crypto.randomUUID(); - const tc = { - name: payload.name || 'FunctionCall', - id: toolUseId, - input: sanitizeToolInput(payload.name || 'FunctionCall', payload.arguments || ''), - done: false, - }; - assistant.toolCalls.push(tc); - pendingToolCalls.set(toolUseId, tc); - break; - } - case 'function_call_output': { - const assistant = ensureAssistant(ts, extractTurnKey(entry, payload, currentTurnKey)); - const toolUseId = payload.call_id || crypto.randomUUID(); - let tc = pendingToolCalls.get(toolUseId); - if (!tc) { - tc = { name: 'FunctionCall', id: toolUseId, input: null, done: false }; - assistant.toolCalls.push(tc); - pendingToolCalls.set(toolUseId, tc); - } - tc.done = true; - tc.result = (typeof payload.output === 'string' - ? payload.output - : JSON.stringify(payload.output || '')).slice(0, 2000); - break; - } - case 'custom_tool_call': { - const assistant = ensureAssistant(ts, extractTurnKey(entry, payload, currentTurnKey)); - const toolUseId = payload.call_id || payload.id || crypto.randomUUID(); - const tc = { - name: payload.name || 'CustomToolCall', - id: toolUseId, - input: sanitizeToolInput(payload.name || 'CustomToolCall', payload.input || payload.arguments || ''), - done: payload.status === 'completed', - }; - assistant.toolCalls.push(tc); - pendingToolCalls.set(toolUseId, tc); - break; - } - case 'custom_tool_call_output': { - const assistant = ensureAssistant(ts, extractTurnKey(entry, payload, currentTurnKey)); - const toolUseId = payload.call_id || crypto.randomUUID(); - let tc = pendingToolCalls.get(toolUseId); - if (!tc) { - tc = { name: 'CustomToolCall', id: toolUseId, input: null, done: false }; - assistant.toolCalls.push(tc); - pendingToolCalls.set(toolUseId, tc); - } - tc.done = true; - tc.result = (typeof payload.output === 'string' - ? payload.output - : JSON.stringify(payload.output || '')).slice(0, 2000); - break; - } - default: - break; + } else if (['function_call', 'custom_tool_call', 'mcp_tool_call', 'mcp_call'].includes(payload.type)) { + appendTool(entry); + } else if (['function_call_output', 'custom_tool_call_output', 'mcp_tool_call_output', 'mcp_call_output'].includes(payload.type)) { + appendTool(entry, true); } } flushAssistant(); - if (!sawRealUserMessage && fallbackUserMessages.length > 0) { - const fallback = fallbackUserMessages[0]; - if (!meta.title) meta.title = fallback.content.trim().slice(0, 80).replace(/\n/g, ' '); - return { meta, messages: fallbackUserMessages.concat(messages), totalUsage }; - } return { meta, messages, totalUsage }; } @@ -309,6 +398,7 @@ function createCodexRolloutStore(deps) { } return { + getRolloutTurnId, parseCodexRolloutLines, getCodexRolloutFiles, getImportedCodexThreadIds, diff --git a/public/app.js b/public/app.js index e97702f..da09387 100644 --- a/public/app.js +++ b/public/app.js @@ -1811,13 +1811,46 @@ || message.clientMessageId || message.replyToRequestId || message.crossConversation?.replyToRequestId + || message.crossConversation?.replyRequestId || message.codexAppTurnKey || message.nativeTurnKey; if (explicit) return String(explicit); + const turnId = message.turnId || message.codexAppTurnId; + if (turnId) return `turn:${message.role || ''}:${turnId}`; const content = typeof message.content === 'string' ? message.content : JSON.stringify(message.content || ''); return `message:${stableMessageHash(`${message.role || ''}\u001f${message.timestamp || ''}\u001f${content}`)}`; } + function getMessageRenderFingerprint(message) { + // 工具结果、附件和跨会话来源变化也必须刷新气泡，不能只比较正文。 + return stableMessageHash(JSON.stringify(message || {})); + } + + function mergeIndexedSessionMessages(snapshot, messages, incomingBase = 0) { + const existingMessages = Array.isArray(snapshot.messages) ? snapshot.messages : []; + const existingBase = Number.isFinite(Number(snapshot.historyBaseIndex)) + ? Number(snapshot.historyBaseIndex) + : Math.max(0, Number(snapshot.historyTotal || existingMessages.length) - existingMessages.length); + const entries = new Map(); + const positions = snapshot.historyMessagePositions || {}; + const upsert = (message, index) => { + const key = getStableMessageKey(message); + const previous = entries.get(key); + entries.set(key, { message, index, order: previous?.order ?? entries.size }); + }; + existingMessages.forEach((message, index) => { + const storedIndex = positions[getStableMessageKey(message)]; + upsert(message, Number.isFinite(storedIndex) ? storedIndex : existingBase + index); + }); + messages.forEach((message, index) => upsert(message, incomingBase + index)); + // 下标只决定位置。同下标的不同 ID 均保留，同 ID 则更新消息及位置。 + const sorted = Array.from(entries.values()).sort((a, b) => a.index - b.index || a.order - b.order); + snapshot.messages = sorted.map((entry) => entry.message); + snapshot.historyMessagePositions = Object.fromEntries(sorted.map((entry) => [getStableMessageKey(entry.message), entry.index])); + snapshot.historyBaseIndex = sorted[0]?.index ?? incomingBase; + snapshot.historyBuffered = snapshot.messages.length; + } + function findRenderedSessionMessageByKey(key) { const normalizedKey = String(key || '').trim(); if (!normalizedKey) return null; @@ -2493,10 +2526,10 @@ messageEl.dataset.sessionMessage = 'true'; messageEl.dataset.messageIndex = String(messageIndex); if (message) { + messageEl.__sessionMessage = message; const key = getStableMessageKey(message); if (key) messageEl.dataset.messageKey = key; - const rawContent = typeof message.content === 'string' ? message.content : JSON.stringify(message.content || ''); - messageEl.dataset.messageFingerprint = stableMessageHash(`${message.role || ''}\u001f${rawContent}`); + messageEl.dataset.messageFingerprint = getMessageRenderFingerprint(message); } if (messageEl.classList.contains('assistant')) { syncAssistantBranchButton(messageEl, messageIndex); @@ -2673,6 +2706,7 @@ || payload.taskTracking?.enabled === true, historyTotal, historyBaseIndex, + historyMessagePositions: payload.historyMessagePositions ? deepClone(payload.historyMessagePositions) : {}, historyBuffered, historyCursor, historyTruncated, @@ -6755,6 +6789,7 @@ // session_history_chunk 的 prepend 并发，造成消息视觉顺序错乱。 immediate: !!options.immediate || !!snapshot.historyPending, baseIndex: snapshot.historyBaseIndex || 0, + messagePositions: snapshot.historyMessagePositions, }); scheduleAdvancedSearchJump(); if (snapshot.isRunning && snapshot.sessionId === currentSessionId) { @@ -6765,6 +6800,7 @@ // 仅保留 streaming-msg 会导致用户气泡在第一次切换时消失。 reconcileRenderedSessionMessages(snapshot.messages || [], { baseIndex: snapshot.historyBaseIndex || 0, + messagePositions: snapshot.historyMessagePositions, preserveScroll: true, }); generatingSessionId = snapshot.sessionId; @@ -7047,36 +7083,8 @@ const bufferedSnapshot = getSessionHistoryBuffer(sessionId); const snapshot = cachedSnapshot || bufferedSnapshot; if (!snapshot) return; - const existingMessages = Array.isArray(snapshot.messages) ? snapshot.messages : []; - const existingBase = Number.isFinite(Number(snapshot.historyBaseIndex)) - ? Number(snapshot.historyBaseIndex) - : Math.max(0, Number(snapshot.historyTotal || existingMessages.length) - existingMessages.length); const incomingBase = Number.isFinite(Number(options.baseIndex)) ? Number(options.baseIndex) : 0; - const stableKey = (message) => typeof getStableMessageKey === 'function' - ? getStableMessageKey(message) - : String(message?.id || message?.messageId || '').trim(); - const indexed = new Map(); - const keyed = new Map(); - existingMessages.forEach((message, index) => { - const position = existingBase + index; - indexed.set(position, message); - const key = stableKey(message); - if (key) keyed.set(key, position); - }); - messages.forEach((message, index) => { - const key = stableKey(message); - if (key && keyed.has(key)) { - indexed.set(keyed.get(key), message); - return; - } - const position = incomingBase + index; - indexed.set(position, message); - if (key) keyed.set(key, position); - }); - const indexes = Array.from(indexed.keys()).sort((a, b) => a - b); - snapshot.messages = indexes.map((index) => indexed.get(index)); - snapshot.historyBaseIndex = indexes.length > 0 ? indexes[0] : incomingBase; - snapshot.historyBuffered = snapshot.messages.length; + mergeIndexedSessionMessages(snapshot, messages, incomingBase); if (Number.isFinite(Number(options.total))) snapshot.historyTotal = Math.max(0, Number(options.total)); if (Number.isFinite(Number(options.cursor))) snapshot.historyCursor = Math.max(0, Number(options.cursor)); snapshot.historyTruncated = snapshot.historyCursor > 0 || snapshot.historyBaseIndex > 0; @@ -7094,36 +7102,8 @@ function mergeHistoryChunkIntoSessionLoadSnapshot(snapshot, messages, options = {}) { if (!snapshot || !Array.isArray(messages) || messages.length === 0) return; - const existingMessages = Array.isArray(snapshot.messages) ? snapshot.messages : []; - const existingBase = Number.isFinite(Number(snapshot.historyBaseIndex)) - ? Number(snapshot.historyBaseIndex) - : Math.max(0, Number(snapshot.historyTotal || existingMessages.length) - existingMessages.length); const incomingBase = Number.isFinite(Number(options.baseIndex)) ? Number(options.baseIndex) : 0; - const stableKey = (message) => typeof getStableMessageKey === 'function' - ? getStableMessageKey(message) - : String(message?.id || message?.messageId || '').trim(); - const indexed = new Map(); - const keyed = new Map(); - existingMessages.forEach((message, index) => { - const position = existingBase + index; - indexed.set(position, message); - const key = stableKey(message); - if (key) keyed.set(key, position); - }); - messages.forEach((message, index) => { - const key = stableKey(message); - if (key && keyed.has(key)) { - indexed.set(keyed.get(key), message); - return; - } - const position = incomingBase + index; - indexed.set(position, message); - if (key) keyed.set(key, position); - }); - const indexes = Array.from(indexed.keys()).sort((a, b) => a - b); - snapshot.messages = indexes.map((index) => indexed.get(index)); - snapshot.historyBaseIndex = indexes.length > 0 ? indexes[0] : incomingBase; - snapshot.historyBuffered = snapshot.messages.length; + mergeIndexedSessionMessages(snapshot, messages, incomingBase); if (Number.isFinite(Number(options.total))) { snapshot.historyTotal = Math.max(0, Number(options.total)); } @@ -8379,10 +8359,7 @@ ? findRenderedSessionMessageByKey(getStableMessageKey(msg.message)) : null; if (existingElement) { - const rawContent = typeof msg.message.content === 'string' - ? msg.message.content - : JSON.stringify(msg.message.content || ''); - const fingerprint = stableMessageHash(`${msg.message.role || ''}\u001f${rawContent}`); + const fingerprint = getMessageRenderFingerprint(msg.message); if (existingElement.dataset.messageFingerprint !== fingerprint) { const messageIndex = Number(existingElement.dataset.messageIndex); const replacement = buildMsgElement( @@ -8533,7 +8510,11 @@ } break; } - finishGenerating(msg.sessionId, { keepRunning: msg.goalActive === true }); + finishGenerating(msg.sessionId, { + keepRunning: msg.goalActive === true, + assistantMessage: msg.assistantMessage, + assistantMessageIndex: msg.assistantMessageIndex, + }); break; case 'system_message': @@ -8625,6 +8606,7 @@ const toolsDiv = document.querySelector('#streaming-msg .msg-tools'); if (toolsDiv) toolsDiv.innerHTML = ''; } + bindStreamingAssistantMessage(msg.assistantMessageId, msg.turnId); pendingText = msg.text || ''; flushRender(); if (msg.toolCalls && msg.toolCalls.length > 0) { @@ -8650,19 +8632,22 @@ if (!isCurrentSessionEvent(msg)) break; clearPendingSessionResumeRequest(msg.sessionId || currentSessionId, msg.requestId); if (msg.sessionId === currentSessionId && Array.isArray(msg.messages) && msg.messages.length > 0) { - const renderedMessages = messagesDiv.querySelectorAll('[data-session-message="true"]').length; const historyTotal = Number.isFinite(Number(msg.historyTotal)) ? Math.max(0, Number(msg.historyTotal)) : msg.messages.length; - if (renderedMessages < historyTotal || msg.messages.some((message) => !findRenderedSessionMessageByKey(getStableMessageKey(message)))) { - reconcileRenderedSessionMessages(msg.messages, { - preserveScroll: false, - baseIndex: Number.isFinite(Number(msg.historyBaseIndex)) - ? Number(msg.historyBaseIndex) - : Math.max(0, historyTotal - msg.messages.length), - }); - loadedHistorySessionId = msg.sessionId; - } + const historyBaseIndex = Number.isFinite(Number(msg.historyBaseIndex)) + ? Number(msg.historyBaseIndex) + : Math.max(0, historyTotal - msg.messages.length); + mergeHistoryChunkIntoCachedSnapshot(msg.sessionId, msg.messages, { + baseIndex: historyBaseIndex, + total: historyTotal, + }); + // 重连时同 ID 的正文、工具及来源仍可能更新，数量相同也需要对账。 + reconcileRenderedSessionMessages(msg.messages, { + preserveScroll: false, + baseIndex: historyBaseIndex, + }); + loadedHistorySessionId = msg.sessionId; } setCurrentSessionRunningState(!!msg.isRunning); if (!msg.isRunning && currentSessionId && msg.sessionId === currentSessionId) { @@ -8896,6 +8881,44 @@ return true; } + function bindStreamingAssistantMessage(messageId, turnId) { + const stream = document.getElementById('streaming-msg'); + if (!stream) return; + const key = messageId || (turnId ? getStableMessageKey({ role: 'assistant', turnId }) : ''); + if (!key) return; + stream.dataset.messageId = String(key); + stream.dataset.messageKey = String(key); + if (turnId) stream.dataset.turnId = String(turnId); + const existing = findRenderedSessionMessageByKey(key); + if (existing && existing !== stream) { + // 快照与运行态属于同一条助手消息，保留正在接收工具和正文的节点。 + stream.__sessionMessage = existing.__sessionMessage; + stream.dataset.messageIndex = existing.dataset.messageIndex; + existing.remove(); + updateUserOutlinePanel(); + } + } + + function commitCompletedAssistantMessage(message, requestedIndex) { + if (!message || message.role !== 'assistant') return false; + const stream = document.getElementById('streaming-msg'); + const existing = findRenderedSessionMessageByKey(getStableMessageKey(message)); + const knownIndex = existing?.dataset.messageIndex ?? stream?.dataset.messageIndex; + const messageIndex = Number.isFinite(requestedIndex) + ? requestedIndex + : knownIndex !== undefined && Number.isFinite(Number(knownIndex)) + ? Number(knownIndex) + : currentSessionMessageCount; + stream?.remove(); + reconcileRenderedSessionMessages([message], { baseIndex: messageIndex, preserveScroll: false }); + currentSessionMessageCount = Math.max(currentSessionMessageCount, messageIndex + 1); + mergeHistoryChunkIntoCachedSnapshot(currentSessionId, [message], { + baseIndex: messageIndex, + total: currentSessionMessageCount, + }); + return true; + } + function finishGenerating(sessionId, options = {}) { if (sessionId && currentSessionId && sessionId !== currentSessionId) return; const keepRunning = options.keepRunning === true; @@ -8919,7 +8942,11 @@ if (typing) typing.remove(); const streamEl = document.getElementById('streaming-msg'); - if (streamEl) { + const committedAssistant = commitCompletedAssistantMessage( + options.assistantMessage || streamEl?.__sessionMessage, + options.assistantMessageIndex, + ); + if (streamEl && !committedAssistant) { // 若本轮出现过父目录，把末尾散落的 .tool-call 也一并收入同一父节点 if (hasGrouped) { const toolsDiv = streamEl.querySelector('.msg-tools'); @@ -9290,7 +9317,10 @@ || (hasGiteaPrompt ? { source: 'gitea', label: 'Gitea', repository: '', resourceKind: 'issue', number: null, title: '' } : null); const isGiteaMessage = role === 'user' && !!giteaSource; const visibleContent = isGiteaMessage ? getGiteaVisibleMessageContent(content) : content; - const resolvedMessageId = meta?.messageId || meta?.id || getStableMessageKey(meta) || createLocalId('message'); + const stableKey = Object.keys(meta).length > 0 + ? getStableMessageKey({ ...meta, role, content, attachments }) + : ''; + const resolvedMessageId = meta?.messageId || meta?.id || stableKey || createLocalId('message'); div.className = `msg ${role}${role === 'assistant' ? ' agent-' + currentAgent : ''}${isCrossConversation ? ' cross-conversation' : ''}${isCrossConversationReply ? ' cross-conversation-reply' : ''}${isGoalMessage ? ' goal-message' : ''}${isGiteaMessage ? ' gitea-message' : ''}`; if (role === 'user') { div.id = `hapi-message-${resolvedMessageId}`; @@ -9299,9 +9329,8 @@ } if (role !== 'system') { div.dataset.messageId = resolvedMessageId; - div.dataset.messageKey = getStableMessageKey(meta) || resolvedMessageId; - const rawContent = typeof content === 'string' ? content : JSON.stringify(content || ''); - div.dataset.messageFingerprint = stableMessageHash(`${role}\u001f${rawContent}`); + div.dataset.messageKey = stableKey || resolvedMessageId; + div.dataset.messageFingerprint = getMessageRenderFingerprint({ ...meta, role, content, attachments }); } if (role === 'system') { @@ -10916,6 +10945,10 @@ renderEpoch++; const epoch = renderEpoch; const baseIndex = Number.isFinite(Number(options.baseIndex)) ? Number(options.baseIndex) : 0; + const messageIndexAt = (index) => { + const storedIndex = options.messagePositions?.[getStableMessageKey(messages[index])]; + return Number.isFinite(storedIndex) ? storedIndex : baseIndex + index; + }; closedCollabAgentIds = collectClosedCollabAgentIds(messages); collabAgentStateCache = new Map(); collabAgentIdsByToolUseId = new Map(); @@ -10932,7 +10965,7 @@ } if (options.immediate) { const frag = document.createDocumentFragment(); - messages.forEach((message, index) => frag.appendChild(buildMsgElement(message, baseIndex + index))); + messages.forEach((message, index) => frag.appendChild(buildMsgElement(message, messageIndexAt(index)))); messagesDiv.appendChild(frag); updateUserOutlinePanel(); renderPendingNotes({ scroll: false }); @@ -10955,7 +10988,7 @@ // Render first batch immediately const frag0 = document.createDocumentFragment(); - for (let i = batches[0][0]; i < batches[0][1]; i++) frag0.appendChild(buildMsgElement(messages[i], baseIndex + i)); + for (let i = batches[0][0]; i < batches[0][1]; i++) frag0.appendChild(buildMsgElement(messages[i], messageIndexAt(i))); messagesDiv.appendChild(frag0); updateUserOutlinePanel(); renderPendingNotes({ scroll: false }); @@ -10969,57 +11002,22 @@ delay += 16; setTimeout(() => { if (renderEpoch !== epoch) return; // session switched, abort stale render - const prevHeight = messagesDiv.scrollHeight; - const prevScrollTop = messagesDiv.scrollTop; - const frag = document.createDocumentFragment(); - for (let i = start; i < end; i++) frag.appendChild(buildMsgElement(messages[i], baseIndex + i)); - messagesDiv.insertBefore(frag, historyLoadMore?.nextSibling || null); - updateUserOutlinePanel(); - // Compensate scrollTop so visible area stays unchanged - messagesDiv.scrollTop = prevScrollTop + (messagesDiv.scrollHeight - prevHeight); - updateScrollbar(); + const pending = []; + const messagePositions = {}; + for (let i = start; i < end; i++) { + const key = getStableMessageKey(messages[i]); + // 延迟批次只补缺失气泡，不能覆盖重连期间已收到的更新。 + if (findRenderedSessionMessageByKey(key)) continue; + pending.push(messages[i]); + messagePositions[key] = messageIndexAt(i); + } + prependHistoryMessages(pending, { messagePositions, preserveScroll: true }); }, delay); } } function prependHistoryMessages(messages, options = {}) { - if (!Array.isArray(messages) || messages.length === 0) return; - const baseIndex = Number.isFinite(Number(options.baseIndex)) ? Number(options.baseIndex) : 0; - const existingIndexes = new Set(Array.from( - messagesDiv.querySelectorAll('[data-session-message="true"][data-message-index]'), - ).map((element) => Number(element.dataset.messageIndex))); - const existingKeys = new Set(Array.from( - messagesDiv.querySelectorAll('[data-session-message="true"][data-message-key]'), - ).map((element) => String(element.dataset.messageKey || '').trim()).filter(Boolean)); - const uniqueMessages = []; - messages.forEach((message, index) => { - const messageIndex = baseIndex + index; - const key = getStableMessageKey(message); - if (existingIndexes.has(messageIndex) || (key && existingKeys.has(key))) return; - existingIndexes.add(messageIndex); - if (key) existingKeys.add(key); - uniqueMessages.push({ message, index }); - }); - if (uniqueMessages.length === 0) return; - collectClosedCollabAgentIds(uniqueMessages.map((entry) => entry.message)).forEach((id) => closedCollabAgentIds.add(id)); - const preserveScroll = options.preserveScroll !== false; - const skipScrollbar = options.skipScrollbar === true; - const welcome = messagesDiv.querySelector('.welcome-msg'); - if (welcome) welcome.remove(); - const frag = document.createDocumentFragment(); - uniqueMessages.forEach(({ message, index }) => frag.appendChild(buildMsgElement(message, baseIndex + index))); - if (!preserveScroll) { - messagesDiv.insertBefore(frag, historyLoadMore?.nextSibling || null); - updateUserOutlinePanel(); - if (!skipScrollbar) updateScrollbar(); - return; - } - const prevHeight = messagesDiv.scrollHeight; - const prevScrollTop = messagesDiv.scrollTop; - messagesDiv.insertBefore(frag, historyLoadMore?.nextSibling || null); - updateUserOutlinePanel(); - messagesDiv.scrollTop = prevScrollTop + (messagesDiv.scrollHeight - prevHeight); - if (!skipScrollbar) updateScrollbar(); + reconcileRenderedSessionMessages(messages, options); } function normalizeAskUserInput(input) { @@ -11709,49 +11707,72 @@ function reconcileRenderedSessionMessages(messages, options = {}) { if (!Array.isArray(messages) || messages.length === 0) return; const baseIndex = Number.isFinite(Number(options.baseIndex)) ? Number(options.baseIndex) : 0; - const existing = new Map(); - const existingByKey = new Map(); - messagesDiv.querySelectorAll('[data-session-message="true"][data-message-index]').forEach((element) => { - const index = Number(element.dataset.messageIndex); - if (Number.isFinite(index)) existing.set(index, element); - const key = String(element.dataset.messageKey || '').trim(); - if (key) existingByKey.set(key, element); - }); const beforeHeight = messagesDiv.scrollHeight; const beforeScrollTop = messagesDiv.scrollTop; const stream = document.getElementById('streaming-msg'); - let inserted = 0; + const records = []; + const existingByKey = new Map(); + let changed = false; + messagesDiv.querySelectorAll('[data-session-message="true"]').forEach((element) => { + if (element === stream) return; + const key = String(element.dataset.messageKey || element.dataset.messageId || '').trim(); + if (key && key === stream?.dataset.messageKey) { + stream.__sessionMessage = element.__sessionMessage; + stream.dataset.messageIndex = element.dataset.messageIndex; + element.remove(); + changed = true; + return; + } + if (key && existingByKey.has(key)) { + element.remove(); + changed = true; + return; + } + const record = { element, index: Number(element.dataset.messageIndex), order: records.length }; + records.push(record); + if (key) existingByKey.set(key, record); + }); + const anchor = stream || records[records.length - 1]?.element.nextSibling || null; + collectClosedCollabAgentIds(messages).forEach((id) => closedCollabAgentIds.add(id)); messages.forEach((message, offset) => { - const messageIndex = baseIndex + offset; const messageKey = getStableMessageKey(message); - const keyed = existingByKey.get(messageKey); - if (keyed) { - const rawContent = typeof message.content === 'string' ? message.content : JSON.stringify(message.content || ''); - const fingerprint = stableMessageHash(`${message.role || ''}\u001f${rawContent}`); - if (keyed.dataset.messageFingerprint !== fingerprint) { + const storedIndex = options.messagePositions?.[messageKey]; + const messageIndex = Number.isFinite(storedIndex) ? storedIndex : baseIndex + offset; + if (messageKey && messageKey === stream?.dataset.messageKey) { + stream.__sessionMessage = message; + stream.dataset.messageIndex = String(messageIndex); + return; + } + let record = existingByKey.get(messageKey); + if (record) { + if (record.element.dataset.messageFingerprint !== getMessageRenderFingerprint(message)) { const replacement = buildMsgElement(message, messageIndex); - keyed.replaceWith(replacement); - existingByKey.set(messageKey, replacement); - existing.set(messageIndex, replacement); + record.element.replaceWith(replacement); + record.element = replacement; + changed = true; + } + if (record.index !== messageIndex) { + markSessionMessageElement(record.element, messageIndex, message); + record.index = messageIndex; + changed = true; } return; } - if (existing.has(messageIndex)) return; - const element = buildMsgElement(message, messageIndex); - const next = Array.from(existing.entries()) - .filter(([index]) => index > messageIndex) - .sort(([left], [right]) => left - right)[0]?.[1]; - messagesDiv.insertBefore(element, next || stream || null); - existing.set(messageIndex, element); - if (messageKey) existingByKey.set(messageKey, element); - inserted += 1; + record = { element: buildMsgElement(message, messageIndex), index: messageIndex, order: records.length }; + records.push(record); + if (messageKey) existingByKey.set(messageKey, record); + changed = true; }); - if (inserted === 0) return; + if (!changed) return; + messagesDiv.querySelector('.welcome-msg')?.remove(); + // 重新排序仅移动历史节点，保留正在生成的 DOM 与流式工具状态。 + records.sort((a, b) => a.index - b.index || a.order - b.order); + records.forEach((record) => messagesDiv.insertBefore(record.element, anchor)); updateUserOutlinePanel(); if (options.preserveScroll !== false) { messagesDiv.scrollTop = beforeScrollTop + (messagesDiv.scrollHeight - beforeHeight); } - updateScrollbar(); + if (!options.skipScrollbar) updateScrollbar(); } // --- Custom Scrollbar --- diff --git a/scripts/build-single-exe.js b/scripts/build-single-exe.js index 326cd45..1e52b6f 100644 --- a/scripts/build-single-exe.js +++ b/scripts/build-single-exe.js @@ -114,6 +114,19 @@ ${runCommand} 如果不设置上述变量，默认命令名仍是 \`claude\` 和 \`codex\`。 +## 迁移旧会话历史 + +先启动新版服务，再在会话 idle 时预览迁移： + +\`\`\`bash +CC_WEB_PASSWORD='登录密码' ${runCommand} --migrate-session-history <会话ID> +CC_WEB_PASSWORD='登录密码' ${runCommand} --migrate-session-history <会话ID> --apply +\`\`\` + +默认连接本机 8002 端口，可用 \`PORT\` 或 \`CC_WEB_SERVER_URL=ws://127.0.0.1:8002/ws\` 指定服务。 +只有 \`--apply\` 执行写入；服务端拒绝迁移运行中的会话或无法确认历史边界的快照。 +原 JSON 保存在 \`sessions/_history-backups/\`，写入后重新加载；无需另装 Node.js。 + ## 目录说明 - \`public/\`：前端静态资源 diff --git a/scripts/codex-rollouts-unit.js b/scripts/codex-rollouts-unit.js new file mode 100644 index 0000000..cb99b89 --- /dev/null +++ b/scripts/codex-rollouts-unit.js @@ -0,0 +1,195 @@ +const assert = require('node:assert/strict'); +const { test } = require('node:test'); +const { createCodexRolloutStore } = require('../lib/codex-rollouts'); + +const store = createCodexRolloutStore({ + codexSessionsDir: '/tmp/cc-web-rollout-unit-unused', + sessionsDir: '/tmp/cc-web-rollout-unit-unused', + normalizeSession: (value) => value, + sanitizeToolInput: (_name, value) => value, +}); +const timestamp = '2026-09-21T10:00:00.000Z'; +const event = (type, extra = {}) => ({ type: 'event_msg', timestamp, payload: { type, ...extra } }); +const item = (type, extra = {}) => ({ type: 'response_item', timestamp, payload: { type, ...extra } }); +const message = (role, text, extra = {}) => item('message', { + role, content: [{ type: role === 'user' ? 'input_text' : 'output_text', text }], ...extra, +}); +const parse = (...entries) => store.parseCodexRolloutLines([ + { type: 'session_meta', payload: { id: 'thread-unit' } }, ...entries, +].map((entry) => JSON.stringify(entry))); + +test('turn ID 支持 metadata 和 turn_context，按明确优先级读取', () => { + assert.equal(store.getRolloutTurnId({ turn_id: 'entry', payload: { + turn_id: 'payload', internal_chat_message_metadata_passthrough: { turn_id: 'metadata' }, + } }), 'payload'); + assert.equal(store.getRolloutTurnId({ turn_id: 'entry', payload: { + internal_chat_message_metadata_passthrough: { turn_id: 'metadata' }, + } }), 'metadata'); + assert.equal(store.getRolloutTurnId({ turn_id: 'entry' }), 'entry'); + assert.equal(store.getRolloutTurnId({ payload: { turn_context: { id: 'nested-context' } } }), 'nested-context'); + assert.equal(store.getRolloutTurnId({ type: 'turn_context', payload: { id: 'context' } }), 'context'); +}); + +test('同一 metadata turn 的几十个文本、重复开始事件和各类工具结果只生成一个助手气泡', () => { + const entries = [event('task_started', { turn_id: 'turn-many' }), event('user_message', { message: '检查页面' })]; + for (let index = 0; index < 40; index += 1) { + entries.push(message('assistant', `片段 ${index}`, { + internal_chat_message_metadata_passthrough: { turn_id: 'turn-many' }, + })); + if (index === 20) entries.push(event('task_started', { turn_id: 'turn-many' })); + } + for (const type of ['function_call', 'custom_tool_call', 'mcp_tool_call']) { + entries.push(item(type, { call_id: type, name: `${type}_name`, arguments: '{}', turn_id: 'turn-many' })); + entries.push(item(`${type}_output`, { call_id: type, output: `${type} 完成`, turn_id: 'turn-many' })); + } + entries.push(event('turn_completed', { turn_id: 'turn-many' })); + const result = parse(...entries); + assert.deepEqual(result.messages.map((entry) => entry.role), ['user', 'assistant']); + const assistant = result.messages[1]; + assert.equal(assistant.content.split('\n\n').length, 40); + assert.equal(assistant.turnId, 'turn-many'); + assert.equal(assistant.nativeThreadId, 'thread-unit'); + assert.equal(assistant.nativeTurnKey, 'native:thread-unit:turn-many'); + assert.equal(assistant.toolCalls.length, 3); + assert(assistant.toolCalls.every((tool) => tool.done && tool.result.endsWith('完成'))); + assert.equal(parse(...entries).messages[1].id, assistant.id); +}); + +test('只有 response_item 用户消息的旧 rollout 保持用户和回复的原始位置', () => { + const result = parse( + message('user', '第一问'), message('assistant', '第一答'), + message('user', '第二问'), message('assistant', '第二答'), + ); + assert.deepEqual(result.messages.map((entry) => entry.content), ['第一问', '第一答', '第二问', '第二答']); + assert.notEqual(result.messages[1].id, result.messages[3].id); + assert.notEqual(result.messages[1].nativeTurnKey, result.messages[3].nativeTurnKey); +}); + +test('event_msg 与 response_item 用户双写只保留一条，并保留原消息 ID 和客户端 ID', () => { + for (const reversed of [false, true]) { + const pair = [ + event('user_message', { message: '同一问题', client_user_message_id: 'client-1' }), + message('user', '同一问题', { id: 'original-user', clientMessageId: 'client-1' }), + ]; + if (reversed) pair.reverse(); + const result = parse(event('task_started', { turn_id: 'turn-pair' }), ...pair, message('assistant', '回答', { id: 'original-assistant' })); + assert.equal(result.messages.length, 2); + assert.equal(result.messages[0].id, 'original-user'); + assert.equal(result.messages[0].clientMessageId, 'client-1'); + assert.equal(result.messages[1].id, 'original-assistant'); + assert.equal(result.messages[1].turnId, 'turn-pair'); + } +}); + +test('有权威事件时忽略 AGENTS 和 environment 注入，双写先后两种顺序保持原消息位置', () => { + for (const reversed of [false, true]) { + const pair = [event('user_message', { message: '检查页面' }), message('user', '检查页面', { id: 'original-user' })]; + if (reversed) pair.reverse(); + const result = parse( + message('user', '# AGENTS.md instructions for /home/cc-web'), + event('task_started', { turn_id: 'turn-real' }), + message('user', '运行上下文'), + ...pair, + message('assistant', '开始检查'), + item('function_call', { call_id: 'tool-one', name: 'read_file', arguments: '{}' }), + message('user', '系统注入'), + item('function_call_output', { call_id: 'tool-one', output: '内容' }), + message('assistant', '检查完成'), + event('task_complete', { turn_id: 'turn-real' }), + message('user', '下一轮上下文'), + event('task_started', { turn_id: 'turn-next' }), + event('user_message', { message: '下一问', id: 'next-user' }), + message('assistant', '下一答'), + ); + assert.deepEqual(result.messages.map((entry) => entry.content), ['检查页面', '开始检查\n\n检查完成', '下一问', '下一答']); + assert.equal(result.messages[0].id, 'original-user'); + assert.equal(result.messages[1].toolCalls.length, 1); + assert.equal(result.messages[1].toolCalls[0].result, '内容'); + assert.equal(result.meta.title, '检查页面'); + } +}); + +test('内部跨会话回传不生成用户气泡，也不会切开同一助手 turn', () => { + const internal = '来自「来源对话」对话（ID: 00000000-0000-0000-0000-000000000001）的消息：处理完毕'; + const result = parse( + event('task_started', { turn_id: 'turn-cross' }), event('user_message', { message: '真正的问题' }), message('user', '真正的问题'), + message('assistant', '前半'), + event('user_message', { message: internal }), message('user', internal), + message('assistant', '后半'), event('task_complete', { turn_id: 'turn-cross' }), + ); + assert.deepEqual(result.messages.map((entry) => entry.content), ['真正的问题', '前半\n\n后半']); + assert.equal(result.meta.sourceConversationId, '00000000-0000-0000-0000-000000000001'); + assert.equal(result.messages[1].nativeTurnSegment, undefined); +}); + +test('自动继续的子对话回传包装不生成用户气泡或拆轮，同时保留目标来源', () => { + const returned = '子对话回传已返回，但已返回不等于已完成。请先检查返回内容是否完整满足原始请求，再决定继续推进、补问目标对话或向用户汇报。\n\nrequestId：request-unit\n目标对话：「实现会话」（ID: 00000000-0000-0000-0000-000000000002）\n\n原始请求：\n检查并修复历史顺序\n\n返回正文：\n修复完成，测试通过。'; + const result = parse( + event('task_started', { turn_id: 'turn-return' }), event('user_message', { message: '请处理问题' }), + message('assistant', '等待实现反馈'), event('user_message', { message: returned }), + message('user', returned), message('assistant', '反馈已核对'), + ); + assert.deepEqual(result.messages.map((entry) => entry.content), ['请处理问题', '等待实现反馈\n\n反馈已核对']); + assert.equal(result.messages[1].nativeTurnSegment, undefined); + assert.equal(result.meta.sourceConversationId, '00000000-0000-0000-0000-000000000002'); + assert.equal(result.meta.sourceConversationTitle, '实现会话'); +}); + +test('普通用户提到子对话回传或不完整包装仍作为用户消息保留', () => { + const texts = [ + '请解释子对话回传已返回，但已返回不等于已完成。是什么意思。', + '子对话回传已返回，但已返回不等于已完成。请帮我确认这句话是否正确。', + '子对话回传已返回，但已返回不等于已完成。请先检查返回内容是否完整满足原始请求，再决定继续推进、补问目标对话或向用户汇报。', + ]; + const result = parse(...texts.map((text) => event('user_message', { message: text }))); + assert.deepEqual(result.messages.map((entry) => entry.content), texts); +}); + +test('turn_context 的新轮 ID 和明确完成事件正确划分助手轮次', () => { + const result = parse( + { type: 'turn_context', payload: { id: 'turn-one' } }, message('assistant', '第一轮'), + { type: 'turn_context', payload: { turn_id: 'turn-two' } }, message('assistant', '第二轮'), + event('turn_completed', { turn_id: 'turn-one' }), message('assistant', '第二轮继续'), + event('turn_completed', { turn_id: 'turn-two' }), + message('assistant', '第三轮', { internal_chat_message_metadata_passthrough: { turn_id: 'turn-three' } }), + ); + assert.deepEqual(result.messages.map((entry) => entry.content), ['第一轮', '第二轮\n\n第二轮继续', '第三轮']); + assert.deepEqual(result.messages.map((entry) => entry.turnId), ['turn-one', 'turn-two', 'turn-three']); + assert.equal(new Set(result.messages.map((entry) => entry.id)).size, 3); +}); + +test('同轮真实追加用户输入保留邻接关系，前后助手段有不同稳定键', () => { + const entries = [ + event('task_started', { turn_id: 'turn-steer' }), event('user_message', { message: '开始', id: 'user-start' }), + message('assistant', '当前进展'), + event('user_message', { message: '补充要求', id: 'user-steer' }), + message('user', '补充要求', { id: 'user-steer-copy' }), + message('assistant', '按补充要求继续'), + ]; + const result = parse(...entries); + assert.deepEqual(result.messages.map((entry) => entry.content), ['开始', '当前进展', '补充要求', '按补充要求继续']); + assert.equal(result.messages[1].nativeTurnSegment, 'initial'); + assert(result.messages[3].nativeTurnSegment); + assert.equal(result.messages[1].turnId, result.messages[3].turnId); + assert.notEqual(result.messages[1].id, result.messages[3].id); + assert.notEqual(result.messages[1].nativeTurnKey, result.messages[3].nativeTurnKey); + assert.deepEqual(parse(...entries).messages.map((entry) => entry.id), result.messages.map((entry) => entry.id)); +}); + +test('两个相同来源的重复文本仍作为真实用户输入保留', () => { + const result = parse( + event('task_started', { turn_id: 'turn-repeat' }), event('user_message', { message: '继续', id: 'first' }), + message('assistant', '进展'), event('user_message', { message: '继续', id: 'second' }), message('assistant', '完成'), + ); + assert.equal(result.messages.filter((entry) => entry.role === 'user').length, 2); +}); + +test('迟到的 turn_context 补充身份时不会把已开始的同轮回复拆开', () => { + const result = parse( + message('assistant', '前半'), { type: 'turn_context', payload: { id: 'turn-late' } }, + message('assistant', '后半'), + ); + assert.equal(result.messages.length, 1); + assert.equal(result.messages[0].content, '前半\n\n后半'); + assert.equal(result.messages[0].turnId, 'turn-late'); +}); diff --git a/scripts/history-browser-check.js b/scripts/history-browser-check.js new file mode 100644 index 0000000..00a4481 --- /dev/null +++ b/scripts/history-browser-check.js @@ -0,0 +1,297 @@ +#!/usr/bin/env node +'use strict'; + +// 使用隔离会话、原生 rollout 和 mock 运行时，在真实 Firefox 中验证历史恢复。 +const assert = require('node:assert/strict'); +const fs = require('node:fs'); +const http = require('node:http'); +const net = require('node:net'); +const os = require('node:os'); +const path = require('node:path'); +const { spawn } = require('node:child_process'); +const { WebSocket, WebSocketServer } = require('ws'); +const { createCodexRolloutStore } = require('../lib/codex-rollouts'); + +const REPO = path.resolve(__dirname, '..'); +const FIREFOX = process.env.CC_WEB_BROWSER_FIREFOX || '/snap/firefox/current/usr/lib/firefox/firefox'; +const GECKODRIVER = process.env.CC_WEB_BROWSER_GECKODRIVER || '/snap/firefox/current/usr/lib/firefox/geckodriver'; +const QUESTION = '没参考其他的页面么，WMSA51 是你这么写的么？'; +const ARCHITECTURE = '先按项目规范核对现有页面、任务票和负责会话，再把样式返工交回原实现负责人。'; +const BOUNDARY = '这不是单纯的样式问题，而是 T-003 的页面职责和流程边界做错了。'; +const STREAM = '当前正在生成的消息：正在核对页面职责与流程边界。'; +const SESSION_ID = '11111111-2222-4333-8444-555555555555'; +const THREAD_ID = '11111111-2222-4333-8444-666666666666'; +const PASSWORD = 'HistoryBrowserFixture!234'; + +const sleep = (ms) => new Promise((resolve) => setTimeout(resolve, ms)); +async function freePort() { + return new Promise((resolve, reject) => { + const server = net.createServer(); + server.on('error', reject); + server.listen(0, '127.0.0.1', () => { + const port = server.address().port; + server.close(() => resolve(port)); + }); + }); +} +async function waitFor(predicate, description, timeout = 12000) { + const start = Date.now(); + let lastError; + while (Date.now() - start < timeout) { + try { const result = await predicate(); if (result) return result; } catch (error) { lastError = error; } + await sleep(80); + } + throw new Error(`${description}${lastError ? `：${lastError.message}` : ''}`); +} +async function stopChild(child) { + if (!child || child.exitCode !== null || child.signalCode) return; + child.kill('SIGTERM'); + await Promise.race([new Promise((resolve) => child.once('exit', resolve)), sleep(1000)]); + if (child.exitCode === null && !child.signalCode) child.kill('SIGKILL'); +} + +class FirefoxDriver { + constructor(port) { this.port = port; this.sessionId = ''; } + async request(method, endpoint, body) { + const response = await fetch(`http://127.0.0.1:${this.port}${endpoint}`, { + method, headers: body === undefined ? {} : { 'content-type': 'application/json' }, + body: body === undefined ? undefined : JSON.stringify(body), + }); + const payload = await response.json(); + if (!response.ok || payload.value?.error) throw new Error(JSON.stringify(payload.value || payload)); + return payload.value; + } + async start() { + const value = await this.request('POST', '/session', { capabilities: { alwaysMatch: { + browserName: 'firefox', 'moz:firefoxOptions': { binary: FIREFOX, args: ['-headless'] }, + } } }); + this.sessionId = value.sessionId; + await this.request('POST', `/session/${this.sessionId}/window/rect`, { width: 1280, height: 1050 }); + return value.capabilities; + } + execute(script, args = []) { return this.request('POST', `/session/${this.sessionId}/execute/sync`, { script, args }); } + navigate(url) { return this.request('POST', `/session/${this.sessionId}/url`, { url }); } + refresh() { return this.request('POST', `/session/${this.sessionId}/refresh`, {}); } + async screenshot(file) { + const data = await this.request('GET', `/session/${this.sessionId}/screenshot`); + fs.writeFileSync(file, Buffer.from(data, 'base64')); + } + async close() { if (this.sessionId) await this.request('DELETE', `/session/${this.sessionId}`); } +} + +function createFixture(root) { + const home = path.join(root, 'fixture-home'); + const sessions = path.join(root, 'sessions'); + const nativeDir = path.join(home, '.codex', 'sessions', '2026', '09', '21'); + for (const folder of [sessions, nativeDir, path.join(root, 'config'), path.join(root, 'logs')]) fs.mkdirSync(folder, { recursive: true }); + const rows = [{ type: 'session_meta', payload: { id: THREAD_ID, cwd: root } }]; + function turn(index, user, textParts) { + const stamp = new Date(Date.UTC(2026, 8, 21, 0, 0, index)).toISOString(); + const turnId = `browser-turn-${index}`; + rows.push({ type: 'event_msg', timestamp: stamp, payload: { type: 'task_started', turn_id: turnId } }); + rows.push({ type: 'event_msg', timestamp: stamp, payload: { type: 'user_message', message: user, id: `browser-user-${index}` } }); + textParts.forEach((text) => rows.push({ type: 'response_item', timestamp: stamp, payload: { + type: 'message', role: 'assistant', internal_chat_message_metadata_passthrough: { turn_id: turnId }, + content: [{ type: 'output_text', text }], + } })); + rows.push({ type: 'event_msg', timestamp: stamp, payload: { type: 'task_complete', turn_id: turnId } }); + } + for (let index = 0; index < 72; index++) turn(index, `早期问题 ${index}`, [`早期回答 ${index}`]); + const store = createCodexRolloutStore({ codexSessionsDir: nativeDir, sessionsDir: sessions, normalizeSession: (v) => v, sanitizeToolInput: (_name, v) => v }); + const prefix = store.parseCodexRolloutLines(rows.map((row) => JSON.stringify(row))).messages; + const snapshot = prefix.slice(-64); + snapshot.push( + { id: 'snapshot-latest-user', clientMessageId: 'snapshot-latest-user', role: 'user', timestamp: '2026-09-21T01:00:00.000Z', content: QUESTION }, + { id: 'snapshot-architecture', role: 'assistant', timestamp: '2026-09-21T01:00:01.000Z', content: ARCHITECTURE }, + { id: 'snapshot-boundary', role: 'assistant', timestamp: '2026-09-21T01:00:02.000Z', content: BOUNDARY }, + { id: 'snapshot-child-reply', role: 'assistant', timestamp: '2026-09-21T01:00:03.000Z', content: '已按现有 WMSA51 页面完成职责核对。', crossConversation: { + reply: true, replyToRequestId: 'fixture-child-request', sourceTitle: '页面实现负责人', sourceSessionId: '77777777-2222-4333-8444-555555555555', + } }, + ); + turn(73, QUESTION, Array.from({ length: 48 }, (_, index) => `native 不应覆盖当前快照的助手片段 ${index}`)); + for (let index = 74; index < 242; index++) turn(index, `native 内部输入 ${index}`, [`native 内部回答 ${index}`]); + rows.push({ type: 'event_msg', timestamp: '2026-09-21T02:00:00.000Z', payload: { + type: 'user_message', message: 'native 内部输入尾部', turn_id: 'browser-native-tail', + } }); + const rollout = path.join(nativeDir, `rollout-2026-09-21T00-00-00-${THREAD_ID}.jsonl`); + fs.writeFileSync(rollout, rows.map((row) => JSON.stringify(row)).join('\n') + '\n'); + const notice = { + role: 'system', content: '历史消息过多，cc-web 本地快照只保留最近 68 条……', ccwebPersistenceNotice: true, + snapshotMessageCount: 68, snapshotFirstMessageId: snapshot[0].id, snapshotLastMessageId: snapshot.at(-1).id, + nativeThreadId: THREAD_ID, truncatedAt: '2026-09-21T01:00:04.000Z', + }; + fs.writeFileSync(path.join(sessions, `${SESSION_ID}.json`), JSON.stringify({ + id: SESSION_ID, title: '历史恢复浏览器夹具', agent: 'codexapp', codexAppThreadId: THREAD_ID, cwd: root, + created: '2026-09-21T00:00:00.000Z', updated: '2026-09-21T01:00:04.000Z', permissionMode: 'yolo', + historySnapshotBaseIndex: 80, historySnapshotCount: 68, messages: [notice, ...snapshot], + }, null, 2)); + // 只在隔离子进程中替换 homedir；不覆盖真实 HOME / CODEX_HOME，也不访问生产 rollout。 + const preload = path.join(root, 'isolated-home.cjs'); + fs.writeFileSync(preload, ` +const fs = require('node:fs'); +const Module = require('node:module'); +const originalLoader = Module._extensions['.js']; +require('node:os').homedir = () => ${JSON.stringify(home)}; +Module._extensions['.js'] = (module, filename) => { + if (filename !== ${JSON.stringify(path.join(REPO, 'server.js'))}) return originalLoader(module, filename); + const source = fs.readFileSync(filename, 'utf8'); + const original = "const CODEX_SESSIONS_DIR = path.join(process.env.HOME || process.env.USERPROFILE || '', '.codex', 'sessions');"; + if (!source.includes(original)) throw new Error('原生历史目录注入点已变更，请更新浏览器夹具'); + module._compile(source.replace(original, 'const CODEX_SESSIONS_DIR = ' + ${JSON.stringify(JSON.stringify(path.join(home, '.codex', 'sessions')))} + ';'), filename); +}; +`); + const mock = path.join(root, 'mock-codex.js'); + const completeFlag = path.join(root, 'complete-turn'); + const existingMock = fs.readFileSync(path.join(REPO, 'scripts/mock-codex-app-server.js'), 'utf8'); + const timerCode = 'thread.timer = setTimeout(() => completeTurn(thread, turnId, text), delay);'; + assert(existingMock.includes(timerCode), 'mock 运行时启动入口已变更'); + const browserMock = existingMock.replace(timerCode, ` + send({method:'item/agentMessage/delta',params:{threadId:thread.id,turnId,itemId:'browser-stream',delta:${JSON.stringify(STREAM)}}}); + thread.timer = setInterval(() => { + if (!require('fs').existsSync(${JSON.stringify(completeFlag)})) return; + clearInterval(thread.timer); + send({method:'turn/completed',params:{threadId:thread.id,turn:{id:turnId,status:'completed',items:[]}}}); + thread.activeTurnId = null; + thread.timer = null; + }, 80); + `).replace("path.join(__dirname, 'mock-codex.js')", JSON.stringify(path.join(REPO, 'scripts/mock-codex.js'))); + fs.writeFileSync(mock, browserMock, { mode: 0o755 }); + return { home, sessions, rollout, preload, mock, completeFlag, snapshotCount: snapshot.length, nativeCount: store.parseCodexRolloutLines(rows.map((row) => JSON.stringify(row))).messages.length }; +} + +async function startRelay(port, targetPort, evidence) { + const relay = http.createServer((request, response) => { + const upstream = http.request({ hostname: '127.0.0.1', port: targetPort, path: request.url, method: request.method, headers: request.headers }, (result) => { + response.writeHead(result.statusCode, result.headers); result.pipe(response); + }); + upstream.on('error', () => response.destroy()); request.pipe(upstream); + }); + const sockets = new Set(); + const wsServer = new WebSocketServer({ noServer: true }); + let connections = 0; + relay.on('upgrade', (request, socket, head) => wsServer.handleUpgrade(request, socket, head, (client) => { + connections++; + sockets.add(client); + const upstream = new WebSocket(`ws://127.0.0.1:${targetPort}/ws`); + const pending = []; + upstream.on('open', () => { pending.splice(0).forEach((data) => upstream.send(data)); }); + client.on('message', (data) => { if (upstream.readyState === WebSocket.OPEN) upstream.send(data); else pending.push(data); }); + upstream.on('message', (data) => { + try { + const msg = JSON.parse(String(data)); + if (['session_info', 'resume_session_result', 'session_history_chunk'].includes(msg.type)) evidence.protocol.push({ + type: msg.type, historySource: msg.historySource, historyTotal: msg.historyTotal, isRunning: msg.isRunning, + ids: msg.messages?.map((message) => message.id), + }); + } catch {} + if (client.readyState === WebSocket.OPEN) client.send(data, { binary: false }); + }); + client.on('close', () => { sockets.delete(client); upstream.terminate(); }); + upstream.on('close', () => { if (client.readyState === WebSocket.OPEN) client.close(); }); + upstream.on('error', () => client.terminate()); + client.on('error', () => upstream.terminate()); + })); + await new Promise((resolve) => relay.listen(port, '127.0.0.1', resolve)); + return { + connectionCount: () => connections, + disconnect: () => { for (const socket of sockets) socket.terminate(); }, + close: async () => { for (const socket of sockets) socket.terminate(); wsServer.close(); relay.closeAllConnections(); await new Promise((resolve) => relay.close(resolve)); }, + }; +} + +async function capture(driver, stage, root, requireStream = true) { + await waitFor(() => driver.execute(`return [...document.querySelectorAll('#messages .msg')].some(node => node.textContent.includes(arguments[0]))`, [requireStream ? STREAM : QUESTION]), `${stage} 消息未恢复`); + const result = await driver.execute(`return { + messages: [...document.querySelectorAll('#messages > .msg')].map(node => ({ + role: node.classList.contains('user') ? 'user' : node.classList.contains('assistant') ? 'assistant' : 'system', + id: node.dataset.messageId || '', key: node.dataset.messageKey || '', text: node.innerText, + source: node.querySelector('.cross-conversation-label')?.innerText || '', streaming: node.id === 'streaming-msg', + })), running: document.querySelector('#chat-runtime-state')?.textContent || '', + };`); + const find = (text) => result.messages.filter((message) => message.text.includes(text)); + for (const text of [QUESTION, ARCHITECTURE, BOUNDARY]) assert.equal(find(text).length, 1, `${stage}：消息重复或消失：${text}`); + const userIndex = result.messages.findIndex((message) => message.text.includes(QUESTION)); + assert.equal(result.messages[userIndex].role, 'user'); + assert(result.messages[userIndex + 1].text.includes(ARCHITECTURE), `${stage}：用户与对应回复失去邻接关系`); + assert(result.messages[userIndex + 2].text.includes(BOUNDARY)); + assert.equal(result.messages[userIndex + 3].role, 'assistant'); + assert.equal(result.messages[userIndex + 3].source, '来自「页面实现负责人」的回复'); + assert(!result.messages.some((message) => message.text.includes('native 不应覆盖') || message.text.includes('native 内部输入'))); + if (requireStream) assert.equal(find(STREAM).length, 1, `${stage}：当前输出重复或丢失`); + const keys = result.messages.map((message) => message.key).filter(Boolean); + assert.equal(new Set(keys).size, keys.length, `${stage}：稳定 ID 重复渲染`); + await driver.execute(`const box=document.querySelector('#messages');box.scrollTop=box.scrollHeight;`); + // 等待浏览器提交滚动与消息淡入，截图必须展示真实可见内容。 + await sleep(500); + result.visibleTail = await driver.execute(`return [...document.querySelectorAll('#messages > .msg')].slice(-6).map(node => { + const box=node.getBoundingClientRect(), style=getComputedStyle(node); + return {text:node.innerText, top:box.top, bottom:box.bottom, height:box.height, opacity:style.opacity, visibility:style.visibility, display:style.display}; + });`); + const visibleQuestion = result.visibleTail.find((message) => message.text.includes(QUESTION)); + assert(visibleQuestion && visibleQuestion.top >= 0 && visibleQuestion.height > 0 && Number(visibleQuestion.opacity) > 0.9, + `${stage}：目标问题没有实际显示在视口`); + await driver.screenshot(path.join(root, `${stage}.png`)); + return result; +} + +async function main() { + assert(fs.existsSync(FIREFOX) && fs.existsSync(GECKODRIVER), '需要本机 Firefox 和 geckodriver，可通过 CC_WEB_BROWSER_* 指定'); + const root = fs.mkdtempSync(path.join(os.tmpdir(), 'cc-web-history-browser-')); + const fixture = createFixture(root); + const evidence = { root, fixture, protocol: [], stages: {}, kind: '真实 server.js / Firefox / 隔离持久化与 native rollout / mock Codex App' }; + const [serverPort, relayPort, driverPort] = await Promise.all([freePort(), freePort(), freePort()]); + let server, gecko, relay, driver; + try { + server = spawn(process.execPath, ['-r', fixture.preload, path.join(REPO, 'server.js')], { + cwd: REPO, env: { ...process.env, PORT: String(serverPort), CC_WEB_PASSWORD: PASSWORD, + CC_WEB_CONFIG_DIR: path.join(root, 'config'), CC_WEB_SESSIONS_DIR: fixture.sessions, CC_WEB_LOGS_DIR: path.join(root, 'logs'), + CODEX_PATH: fixture.mock, CLAUDE_PATH: path.join(REPO, 'scripts/mock-claude.js'), CC_WEB_TASK_BOARD_ENABLED: '0', + }, stdio: ['ignore', 'pipe', 'pipe'], + }); + const serverLog = fs.createWriteStream(path.join(root, 'server.log')); server.stdout.pipe(serverLog); server.stderr.pipe(serverLog); + await waitFor(async () => (await fetch(`http://127.0.0.1:${serverPort}/`)).ok, '隔离服务启动失败'); + relay = await startRelay(relayPort, serverPort, evidence); + gecko = spawn(GECKODRIVER, ['--port', String(driverPort)], { stdio: ['ignore', 'pipe', 'pipe'] }); + const geckoLog = fs.createWriteStream(path.join(root, 'geckodriver.log')); gecko.stdout.pipe(geckoLog); gecko.stderr.pipe(geckoLog); + await waitFor(async () => (await fetch(`http://127.0.0.1:${driverPort}/status`)).ok, 'geckodriver 启动失败'); + driver = new FirefoxDriver(driverPort); + evidence.browser = await driver.start(); + await driver.navigate(`http://127.0.0.1:${relayPort}/`); + await waitFor(async () => { + const ready = await driver.execute(`return !document.querySelector('#app').hidden && !!document.querySelector('.session-item')`); + if (ready) return true; + await driver.execute(`document.querySelector('#login-password').value=arguments[0];document.querySelector('#login-form').requestSubmit();`, [PASSWORD]); + return false; + }, '登录或会话列表未就绪'); + await driver.execute(`document.querySelector('.session-item').click();`); + evidence.stages.loaded = await capture(driver, 'loaded', root, false); + await driver.execute(`const input=document.querySelector('#msg-input');input.value='继续核对页面职责与流程边界';input.dispatchEvent(new Event('input',{bubbles:true}));document.querySelector('#send-btn').click();`); + evidence.stages.running = await capture(driver, 'running', root); + await driver.refresh(); + evidence.stages.refreshed = await capture(driver, 'refreshed', root); + const previousCount = relay.connectionCount(); + relay.disconnect(); + await waitFor(() => relay.connectionCount() > previousCount, '真实 WebSocket 断线后没有重连'); + await waitFor(() => evidence.protocol.some((message) => message.type === 'resume_session_result'), '没有收到恢复会话结果'); + evidence.stages.reconnected = await capture(driver, 'reconnected', root); + fs.writeFileSync(fixture.completeFlag, 'complete\n'); + await waitFor(() => driver.execute(`return !document.querySelector('#streaming-msg')`), '完成事件后流式气泡未收束'); + evidence.stages.completed = await capture(driver, 'completed', root); + await driver.refresh(); + evidence.stages.completedRefreshed = await capture(driver, 'completed-refreshed', root); + assert(evidence.protocol.some((message) => message.historySource === 'merged'), '没有验证到服务端真实 native 前缀合并'); + evidence.ok = true; + console.log(JSON.stringify({ ok: true, browser: evidence.browser.browserVersion, root, stages: Object.keys(evidence.stages), nativeCount: fixture.nativeCount, snapshotCount: fixture.snapshotCount }, null, 2)); + } catch (error) { + evidence.ok = false; evidence.error = error.stack; + if (driver?.sessionId) await driver.screenshot(path.join(root, 'failure.png')).catch(() => {}); + throw new Error(`${error.message}\n证据：${root}`, { cause: error }); + } finally { + fs.writeFileSync(path.join(root, 'evidence.json'), JSON.stringify(evidence, null, 2)); + if (driver) await driver.close().catch(() => {}); + if (relay) await relay.close(); + await stopChild(gecko); await stopChild(server); + } +} + +main().catch((error) => { console.error(error); process.exitCode = 1; }); diff --git a/scripts/history-frontend-unit.js b/scripts/history-frontend-unit.js new file mode 100644 index 0000000..0c64ef6 --- /dev/null +++ b/scripts/history-frontend-unit.js @@ -0,0 +1,346 @@ +#!/usr/bin/env node +'use strict'; + +const assert = require('node:assert/strict'); +const fs = require('node:fs'); +const path = require('node:path'); + +const source = fs.readFileSync(path.join(__dirname, '..', 'public', 'app.js'), 'utf8'); + +function extractFunction(name) { + const start = source.indexOf(`function ${name}(`); + assert(start >= 0, `缺少前端函数 ${name}`); + let signatureEnd = start; + let parentheses = 0; + for (let i = source.indexOf('(', start); i < source.length; i += 1) { + if (source[i] === '(') parentheses += 1; + if (source[i] === ')' && --parentheses === 0) { signatureEnd = i; break; } + } + let depth = 0; + for (let i = source.indexOf('{', signatureEnd); i < source.length; i += 1) { + if (source[i] === '{') depth += 1; + if (source[i] === '}' && --depth === 0) return source.slice(start, i + 1); + } + throw new Error(`无法提取 ${name}`); +} + +// 仅模拟浏览器节点操作，消息身份、对账、缓存合并及重连分支均执行 app.js 原函数。 +class Element { + constructor() { + this.dataset = {}; + this.children = []; + this.parentNode = null; + this.scrollTop = 0; + this.id = ''; + this.classList = { contains: (name) => this.message?.role === name }; + } + get scrollHeight() { return this.children.length * 50; } + get nextSibling() { + if (!this.parentNode) return null; + return this.parentNode.children[this.parentNode.children.indexOf(this) + 1] || null; + } + set innerHTML(value) { + this.children.forEach((child) => { child.parentNode = null; }); + this.children = []; + } + insertBefore(node, anchor) { + if (node.fragment) { + [...node.children].forEach((child) => this.insertBefore(child, anchor)); + return node; + } + if (node === anchor) return node; + node.remove(); + const index = anchor ? this.children.indexOf(anchor) : this.children.length; + assert(index >= 0, '插入锚点必须仍在消息容器中'); + this.children.splice(index, 0, node); + node.parentNode = this; + return node; + } + appendChild(node) { return this.insertBefore(node, null); } + removeAttribute(name) { if (name === 'id') this.id = ''; } + remove() { + if (!this.parentNode) return; + const nodes = this.parentNode.children; + nodes.splice(nodes.indexOf(this), 1); + this.parentNode = null; + } + replaceWith(node) { + if (!this.parentNode) return; + this.parentNode.insertBefore(node, this); + this.remove(); + } + querySelectorAll(selector) { + return this.children.filter((child) => { + if (selector.includes('[data-session-message="true"]') && child.dataset.sessionMessage !== 'true') return false; + if (selector.includes('[data-message-key]') && !child.dataset.messageKey) return false; + return true; + }); + } + querySelector() { return null; } +} + +function createHarness() { + const names = [ + 'stableMessageHash', 'getStableMessageKey', 'getMessageRenderFingerprint', + 'mergeIndexedSessionMessages', 'dedupeSessionMessages', 'markSessionMessageElement', + 'mergeHistoryChunkIntoCachedSnapshot', 'mergeHistoryChunkIntoSessionLoadSnapshot', + 'findRenderedSessionMessageByKey', 'renderMessages', 'reconcileRenderedSessionMessages', + 'prependHistoryMessages', 'bindStreamingAssistantMessage', 'commitCompletedAssistantMessage', + 'finishGenerating', + ]; + const resumeStart = source.indexOf("case 'resume_session_result':"); + const resumeEnd = source.indexOf("case 'error':", resumeStart); + const doneStart = source.indexOf("case 'done':"); + const doneEnd = source.indexOf("case 'system_message':", doneStart); + const generatingStart = source.indexOf("case 'resume_generating':"); + const messagesDiv = new Element(); + const api = new Function('Element', 'messagesDiv', ` + let renderEpoch = 0; + let closedCollabAgentIds = new Set(); + let collabAgentStateCache = new Map(); + let collabAgentIdsByToolUseId = new Map(); + let closedCollabAgentIdsByToolUseId = new Map(); + let currentSessionId = 'running-session'; + let currentSessionMessageCount = 0; + let isGenerating = false; + let generatingSessionId = null; + let pendingText = ''; + let toolGroupCount = 0; + let hasGrouped = false; + const currentAgent = 'codexapp'; + const activeToolCalls = new Map(); + const activeTodoCallTargets = new Map(); + const window = { pendingContentBlocks: [] }; + const msgInput = { focus() {} }; + let loadedHistorySessionId = ''; + let currentCwd = '/tmp/history-test'; + const historyLoadMore = null; + const sessionCache = new Map(); + const buffers = new Map(); + const timers = []; + let outlineUpdates = 0; + let scrollbarUpdates = 0; + const document = { + getElementById(id) { return messagesDiv.children.find((node) => node.id === id) || null; }, + querySelector() { return null; }, + createDocumentFragment() { const node = new Element(); node.fragment = true; return node; }, + }; + function setTimeout(callback) { timers.push(callback); } + function collectClosedCollabAgentIds() { return new Set(); } + function clearUserMessageIndex() {} + function updateUserOutlinePanel() { outlineUpdates += 1; } + function updateScrollbar() { scrollbarUpdates += 1; } + function renderPendingNotes() {} + function scrollToBottom() {} + function syncAssistantBranchButton() {} + function isCurrentSessionEvent(msg) { return msg.sessionId === currentSessionId; } + function clearPendingSessionResumeRequest() {} + function setCurrentSessionRunningState() {} + function updateNoteModeUI() {} + function migratePendingNotesToSession() {} + function migrateQueuedMessagesToSession() {} + function scheduleQueuedMessageDrain() {} + function syncAssistantLastSectionButton() {} + function updateGenerationControls() {} + function isNearBottom() { return true; } + function flushRender() { + const stream = document.getElementById('streaming-msg'); + if (stream) stream.pendingText = pendingText; + } + function startGenerating() { + isGenerating = true; + const stream = new Element(); + stream.id = 'streaming-msg'; + stream.dataset.messageKey = 'local-stream'; + messagesDiv.appendChild(stream); + } + function getSessionHistoryBuffer(id) { return buffers.get(id) || null; } + function isCompleteSessionSnapshot(snapshot) { return snapshot.complete; } + function cacheSessionSnapshot(snapshot) { sessionCache.set(snapshot.sessionId, { snapshot }); } + function bufferSessionSnapshot(snapshot) { buffers.set(snapshot.sessionId, snapshot); } + ${names.map(extractFunction).join('\n')} + function buildMsgElement(message, index) { + const element = new Element(); + element.message = JSON.parse(JSON.stringify(message)); + markSessionMessageElement(element, index, message); + return element; + } + function resume(msg) { + switch (msg.type) { ${source.slice(resumeStart, resumeEnd)} } + } + function done(msg) { + switch (msg.type) { ${source.slice(doneStart, doneEnd)} } + } + function resumeGenerating(msg) { + switch (msg.type) { ${source.slice(generatingStart, resumeStart)} } + } + return { + ${names.join(',')}, resume, done, resumeGenerating, messagesDiv, sessionCache, buffers, + setCount(value) { currentSessionMessageCount = value; }, + count() { return currentSessionMessageCount; }, + flushTimers() { while (timers.length) timers.shift()(); }, + updates() { return { outlineUpdates, scrollbarUpdates }; }, + }; + `)(Element, messagesDiv); + api.ids = () => messagesDiv.children.map((node) => node.message?.id || node.id); + return api; +} + +function message(id, role = 'assistant', content = id) { + return { id, role, content, timestamp: '2026-09-21T00:00:00.000Z' }; +} + +{ + const api = createHarness(); + assert.equal(api.getStableMessageKey({ role: 'assistant', turnId: 'turn-1', content: '初稿' }), + api.getStableMessageKey({ role: 'assistant', turnId: 'turn-1', content: '完成' })); + assert.notEqual(api.getStableMessageKey({ role: 'user', turnId: 'turn-1' }), + api.getStableMessageKey({ role: 'assistant', turnId: 'turn-1' })); + assert.equal(api.getStableMessageKey({ id: '已保存', turnId: 'turn-1' }), '已保存'); + const before = message('reply'); + assert.notEqual(api.getMessageRenderFingerprint(before), api.getMessageRenderFingerprint({ ...before, toolCalls: [{ result: '完成' }] })); + assert.notEqual(api.getMessageRenderFingerprint(before), api.getMessageRenderFingerprint({ ...before, crossConversation: { sourceTitle: '子对话' } })); +} + +for (const cached of [false, true]) { + const api = createHarness(); + const snapshot = { sessionId: 'history', messages: [message('user', 'user'), message('assistant')], historyBaseIndex: 7, historyTotal: 10 }; + api.buffers.set('history', snapshot); + const merge = (messages, baseIndex) => cached + ? api.mergeHistoryChunkIntoCachedSnapshot('history', messages, { baseIndex, total: 10, cursor: 3 }) + : api.mergeHistoryChunkIntoSessionLoadSnapshot(snapshot, messages, { baseIndex, total: 10 }); + merge([message('older')], 7); + assert.deepEqual(snapshot.messages.map((item) => item.id), ['user', 'older', 'assistant'], '相同下标不能吞掉不同 ID'); + merge([message('older')], 7); + assert.equal(snapshot.messages.length, 3, '重复分页不能重复插入'); + assert.equal(snapshot.historyMessagePositions.assistant, 8, '碰撞后的再次分页不能挤动原下标'); + const updated = { ...message('assistant', 'assistant', '新正文'), toolCalls: [{ result: '工具结果' }], crossConversation: { reply: true, sourceTitle: '负责人' } }; + merge([updated], 3); + assert.deepEqual(snapshot.messages.map((item) => item.id), ['assistant', 'user', 'older'], '同 ID 的新位置必须生效'); + assert.deepEqual(snapshot.messages[0], updated, '同 ID 的正文、工具和来源必须更新'); + assert.equal(snapshot.historyBaseIndex, 3); +} + +{ + const api = createHarness(); + api.renderMessages([message('user', 'user'), message('assistant'), message('assistant', 'assistant', '最终回复')], { immediate: true, baseIndex: 7 }); + assert.deepEqual(api.ids(), ['user', 'assistant'], '初始渲染按稳定 ID 去重'); + assert.equal(api.messagesDiv.children[1].message.content, '最终回复'); + api.prependHistoryMessages([message('older')], { baseIndex: 7 }); + assert.deepEqual(api.ids(), ['user', 'older', 'assistant'], '同下标的分页消息不得丢失'); + api.prependHistoryMessages([message('older')], { baseIndex: 7 }); + assert.equal(api.messagesDiv.children.length, 3); + const updated = { ...message('assistant', 'assistant', '最终回复'), toolCalls: [{ id: 'tool', result: '已更新' }], crossConversation: { reply: true, sourceTitle: '负责人' } }; + const counts = api.updates(); + api.prependHistoryMessages([updated], { baseIndex: 3 }); + assert.deepEqual(api.ids(), ['assistant', 'user', 'older'], '同 ID 跨下标移动必须重排 DOM'); + assert.deepEqual(api.messagesDiv.children[0].message, updated, '正文相同时也更新工具和来源'); + assert.equal(api.messagesDiv.children[0].dataset.messageIndex, '3'); + assert(api.updates().outlineUpdates > counts.outlineUpdates, '只更新消息也需要刷新目录'); +} + +{ + const api = createHarness(); + const initial = Array.from({ length: 35 }, (_, index) => message(`batch-${index}`)); + api.renderMessages(initial); + const latest = { ...initial[3], content: '重连期间更新的内容', toolCalls: [{ result: '新结果' }] }; + api.reconcileRenderedSessionMessages([latest], { baseIndex: 3 }); + api.flushTimers(); + assert.deepEqual(api.ids(), initial.map((item) => item.id), '延迟渲染与重连对账交错不能重复插入'); + assert.equal(api.messagesDiv.children[3].message.content, latest.content, '延迟批次不能把更新恢复成旧内容'); +} + +{ + const api = createHarness(); + const user = message('question', 'user', '没参考其他的页面么，WMSA51 是你这么写的么？'); + const architecture = message('architecture', 'assistant', '页面职责和流程边界需要返工'); + const reply = { ...message('reply'), crossConversation: { reply: true, sourceTitle: '旧来源' } }; + api.renderMessages([architecture, reply], { immediate: true, baseIndex: 20 }); + const stream = new Element(); + stream.id = 'streaming-msg'; + stream.pendingText = '当前正在生成的消息'; + api.messagesDiv.appendChild(stream); + const refreshedReply = { ...reply, crossConversation: { reply: true, sourceTitle: '子对话负责人' }, toolCalls: [{ result: '验收完成' }] }; + const snapshot = { sessionId: 'running-session', messages: [architecture, reply], historyBaseIndex: 20, historyTotal: 23 }; + api.buffers.set('running-session', snapshot); + const resume = { + type: 'resume_session_result', sessionId: 'running-session', isRunning: true, + historyBaseIndex: 20, historyTotal: 23, messages: [user, architecture, refreshedReply], + }; + api.resume(resume); + api.resume(resume); + assert.deepEqual(api.ids(), ['question', 'architecture', 'reply', 'streaming-msg']); + assert.strictEqual(api.messagesDiv.children.at(-1), stream, '运行中重连必须保留原流式节点'); + assert.equal(stream.pendingText, '当前正在生成的消息'); + assert.equal(api.messagesDiv.children[2].message.crossConversation.sourceTitle, '子对话负责人'); + const changed = { ...refreshedReply, content: '回传内容更新', toolCalls: [{ result: '新结果' }] }; + api.resume({ ...resume, messages: [user, architecture, changed] }); + assert.equal(api.messagesDiv.children[2].message.content, '回传内容更新', '同数量同 ID 的重连也必须更新'); + assert.equal(snapshot.messages.find((item) => item.id === 'reply').toolCalls[0].result, '新结果'); + assert.equal(api.messagesDiv.children[0].message.role, 'user'); + assert.equal(api.messagesDiv.children[2].message.role, 'assistant'); +} + +{ + const api = createHarness(); + const question = message('question', 'user'); + const completed = { ...message('turn-reply'), toolCalls: [{ id: 'tool', result: '最终结果' }] }; + api.renderMessages([question], { immediate: true }); + api.setCount(1); + api.resumeGenerating({ type: 'resume_generating', sessionId: 'running-session', assistantMessageId: completed.id, turnId: 'turn-1', text: '生成中的回复' }); + const stream = api.messagesDiv.children.at(-1); + assert.equal(stream.dataset.messageKey, completed.id, '恢复流式状态必须绑定持久化助手 ID'); + assert.equal(stream.dataset.turnId, 'turn-1'); + api.done({ type: 'done', sessionId: 'running-session', assistantMessage: completed, assistantMessageIndex: 1 }); + assert.deepEqual(api.ids(), ['question', 'turn-reply']); + assert.deepEqual(api.messagesDiv.children[1].message, completed, '完成事件必须采用规范化助手消息'); + assert.equal(api.count(), 2); + api.done({ type: 'done', sessionId: 'running-session', assistantMessage: completed, assistantMessageIndex: 1 }); + assert.deepEqual(api.ids(), ['question', 'turn-reply'], '重复完成事件不能重复增加助手'); + assert.equal(api.count(), 2, '重复完成事件不能增加消息计数'); + api.resume({ type: 'resume_session_result', sessionId: 'running-session', isRunning: false, messages: [question, completed], historyBaseIndex: 0, historyTotal: 2 }); + assert.deepEqual(api.ids(), ['question', 'turn-reply'], '完成后的重连不得再次插入助手'); +} + +{ + const api = createHarness(); + const question = message('question', 'user'); + const completed = message('turn-reply', 'assistant', '快照中的最终回复'); + api.renderMessages([question, completed], { immediate: true }); + api.setCount(2); + api.resumeGenerating({ type: 'resume_generating', sessionId: 'running-session', assistantMessageId: completed.id, turnId: 'turn-2', text: '仍在生成' }); + const stream = api.messagesDiv.children.at(-1); + assert.deepEqual(api.ids(), ['question', 'streaming-msg'], '相同助手 ID 的历史气泡与 stream 只能显示一份'); + api.resume({ type: 'resume_session_result', sessionId: 'running-session', isRunning: true, messages: [question, completed], historyBaseIndex: 0, historyTotal: 2 }); + assert.deepEqual(api.ids(), ['question', 'streaming-msg'], '流式过程中重复快照不能再建立同 ID 历史气泡'); + assert.strictEqual(api.messagesDiv.children.at(-1), stream, '对账仍须保留运行中的原节点'); + api.resume({ type: 'resume_session_result', sessionId: 'running-session', isRunning: false, messages: [question, completed], historyBaseIndex: 0, historyTotal: 2 }); + assert.deepEqual(api.ids(), ['question', 'turn-reply'], '停止运行的恢复结果须把 stream 转成同 ID 最终助手'); + assert.equal(api.count(), 2); +} + +{ + const api = createHarness(); + const completed = message('already-saved'); + api.renderMessages([completed], { immediate: true }); + api.setCount(1); + api.resumeGenerating({ type: 'resume_generating', sessionId: 'running-session', text: '尚未绑定 ID 的流式输出' }); + assert.equal(api.messagesDiv.children.length, 2); + api.done({ type: 'done', sessionId: 'running-session', assistantMessage: { ...completed, content: '规范化最终回复' }, assistantMessageIndex: 0 }); + assert.deepEqual(api.ids(), ['already-saved'], '完成事件需移除与已保存消息重复的 stream'); + assert.equal(api.messagesDiv.children[0].message.content, '规范化最终回复'); + assert.equal(api.count(), 1); +} + +{ + const api = createHarness(); + const completed = message('without-stream'); + api.done({ type: 'done', sessionId: 'running-session', assistantMessage: completed, assistantMessageIndex: 3 }); + assert.deepEqual(api.ids(), ['without-stream'], '丢失流式节点时完成事件仍可补回最终消息'); + assert.equal(api.count(), 4); + api.done({ type: 'done', sessionId: 'running-session', assistantMessage: { ...completed, content: '最终内容更新' }, assistantMessageIndex: 3 }); + assert.equal(api.messagesDiv.children[0].message.content, '最终内容更新'); + assert.equal(api.count(), 4); +} + +console.log('前端历史单测通过：稳定 ID、分页、重连、流式身份绑定与完成去重'); diff --git a/scripts/migrate-session-history.js b/scripts/migrate-session-history.js new file mode 100644 index 0000000..248c578 --- /dev/null +++ b/scripts/migrate-session-history.js @@ -0,0 +1,37 @@ +#!/usr/bin/env node +'use strict'; + +// 默认预览；--apply 才会让服务端在 idle 状态下备份并原子迁移。 +// CC_WEB_PASSWORD=... node scripts/migrate-session-history.js <会话 ID> [--apply] +const WebSocket = require('ws'); + +const sessionId = process.argv.slice(2).find((arg) => !arg.startsWith('--')); +if (!sessionId || !/^[0-9a-f-]{36}$/i.test(sessionId)) { + console.error('用法：node scripts/migrate-session-history.js <会话 ID> [--apply]'); + process.exit(1); +} +const serverUrl = process.env.CC_WEB_SERVER_URL || `ws://127.0.0.1:${process.env.PORT || '8002'}/ws`; +const ws = new WebSocket(serverUrl); +const timer = setTimeout(() => finish({ ok: false, code: 'timeout' }), 15000); +let finished = false; +function finish(result) { + if (finished) return; + finished = true; + clearTimeout(timer); + console.log(JSON.stringify(result, null, 2)); + process.exitCode = result.ok ? 0 : 1; + ws.terminate(); +} +ws.on('open', () => ws.send(JSON.stringify({ type: 'auth', password: process.env.CC_WEB_PASSWORD || '' }))); +ws.on('error', (error) => finish({ ok: false, code: 'connection_failed', message: error.message })); +ws.on('close', () => { if (!finished) finish({ ok: false, code: 'connection_closed' }); }); +ws.on('message', (raw) => { + let msg; + try { msg = JSON.parse(String(raw)); } catch { return; } + if (msg.type === 'auth_result') { + if (!msg.success) return finish({ ok: false, code: 'authentication_failed' }); + ws.send(JSON.stringify({ type: 'migrate_session_history', sessionId, apply: process.argv.includes('--apply') })); + } else if (msg.type === 'migrate_session_history_result' || msg.type === 'error') { + finish(msg); + } +}); diff --git a/scripts/regression.js b/scripts/regression.js index f5184ed..b4b39ad 100644 --- a/scripts/regression.js +++ b/scripts/regression.js @@ -2818,6 +2818,7 @@ function assertSessionProjectSnapshotContract() { return new Date(b.updated || 0) - new Date(a.updated || 0); } ${normalizeGiteaSourceSource} + ${['stableMessageHash', 'getStableMessageKey', 'mergeIndexedSessionMessages'].map((name) => extractFunctionSource(frontendSource, name)).join('\n')} ${mergeSource} return { mergeSessionListSnapshot, @@ -3799,6 +3800,9 @@ function assertSessionRenderEpochRaceContract() { function scrollToBottom() {} function updateScrollbar() {} function buildMsgElement(message) { return { id: message.id }; } + function getStableMessageKey(message) { return message.id; } + function findRenderedSessionMessageByKey(key) { return messagesDiv.nodes.find((node) => node.id === key); } + function prependHistoryMessages(messages) { messagesDiv.insertBefore({ __fragment: true, nodes: messages.map(buildMsgElement) }); } ${beginSessionSwitchSource} ${renderMessagesSource} return { @@ -3929,6 +3933,7 @@ function assertSessionRequestIdRaceContract() { function bufferSessionSnapshot() {} function scheduleAdvancedSearchJump() {} function finalizeLoadedSession(sessionId, requestId) { finalized.push({ sessionId, requestId }); } + ${['stableMessageHash', 'getStableMessageKey', 'mergeIndexedSessionMessages'].map((name) => extractFunctionSource(frontendSource, name)).join('\n')} ${historyMergeSource} function handleHistoryMessage(msg) { switch (msg.type) { @@ -4174,6 +4179,7 @@ function assertRecoverCurrentHistoryMergeContract() { function scheduleAdvancedSearchJump() {} function cacheSessionSnapshot(snapshot) { cached.push(JSON.parse(JSON.stringify(snapshot))); } function finishSessionSwitch(sessionId, requestId) { finished.push({ sessionId, requestId }); } + ${['stableMessageHash', 'getStableMessageKey', 'mergeIndexedSessionMessages'].map((name) => extractFunctionSource(frontendSource, name)).join('\n')} ${historyMergeSource} ${finalizeLoadedSessionSource} function handleHistoryMessage(msg) { @@ -4273,6 +4279,7 @@ function assertPartialSessionSnapshotCacheContract() { ${bufferSource} ${getBufferSource} ${cacheSource} + ${['stableMessageHash', 'getStableMessageKey', 'mergeIndexedSessionMessages'].map((name) => extractFunctionSource(frontendSource, name)).join('\n')} ${mergeSource} ${dispositionSource} return { @@ -4562,6 +4569,11 @@ function assertServerSessionHistoryRequestIdContract() { } function assertHistoryRecallContract() { + for (const script of ['session-history-unit.js', 'codex-rollouts-unit.js', 'history-frontend-unit.js']) { + const result = spawnSync(process.execPath, [path.join(REPO_DIR, 'scripts', script)], { encoding: 'utf8', timeout: 60000 }); + assert(result.status === 0, `${script}: ${result.stderr || result.stdout || result.error}`); + } + const frontendSource = fs.readFileSync(PUBLIC_APP_PATH, 'utf8'); const serverSource = fs.readFileSync(SERVER_PATH, 'utf8'); const indexSource = fs.readFileSync(PUBLIC_INDEX_PATH, 'utf8'); @@ -4576,16 +4588,11 @@ function assertHistoryRecallContract() { assert(frontendSource.includes('function requestOlderHistory'), 'Frontend should request older history on demand'); assert(frontendSource.includes("type: 'load_history_page'"), 'Frontend should use the paged history WebSocket request'); assert(frontendSource.includes('mergeHistoryChunkIntoCachedSnapshot'), 'Frontend should merge history pages into the session cache'); - assert( - /const existingIndexes = new Set\([\s\S]*?data-message-index/.test(frontendSource), - 'Frontend should deduplicate history messages by stable message index' - ); assert(frontendSource.includes('function getStableMessageKey'), 'Frontend should use a stable message key for history reconciliation'); assert(frontendSource.includes('data-message-key'), 'Message DOM nodes should retain stable message keys'); - assert( - (frontendSource.match(/indexed\.set\(keyed\.get\(key\), message\)/g) || []).length >= 2, - 'History snapshot merges should update an existing stable ID in place' - ); + assert(!extractFunctionSource(frontendSource, 'reconcileRenderedSessionMessages').includes('if (existing.has(messageIndex)) return'), + 'Different IDs must not be dropped just because their indexes collide'); + assert(frontendSource.includes('function mergeIndexedSessionMessages'), 'History pages should share ID-based merging'); assert(serverSource.includes('function mergeNativeHistoryWithSnapshot'), 'Server should merge native history into the snapshot prefix'); assert(serverSource.includes('historySnapshotBaseIndex'), 'Persisted snapshots should record their logical history base index'); assert(frontendSource.includes('currentOutlineHistoryState'), @@ -4606,8 +4613,8 @@ function assertHistoryRecallContract() { /messagesDiv\.innerHTML = '';\s*if \(historyLoadMore\) messagesDiv\.appendChild\(historyLoadMore\)/.test(frontendSource), 'Message rerenders should preserve the in-content history control'); assert( - /messagesDiv\.insertBefore\(frag, historyLoadMore\?\.nextSibling \|\| null\)/.test(frontendSource), - 'Older message batches should be inserted below the in-content history control'); + extractFunctionSource(frontendSource, 'prependHistoryMessages').includes('reconcileRenderedSessionMessages'), + 'Older pages should use stable-ID reconciliation while preserving the in-content history control'); const historyHandlerStart = frontendSource.indexOf("case 'session_history_chunk':"); const historyHandlerEnd = frontendSource.indexOf("case 'session_message':", historyHandlerStart); const historyHandlerSource = historyHandlerStart >= 0 && historyHandlerEnd > historyHandlerStart @@ -4717,10 +4724,7 @@ function assertHistoryRecallContract() { assert(implicitAssistants.length === 2 && implicitAssistants[0].nativeTurnKey !== implicitAssistants[1].nativeTurnKey, 'Rollout parser should assign distinct stable keys to adjacent unmarked turns'); - assert( - /existingElement\.dataset\.messageFingerprint[\s\S]*?existingElement\.replaceWith\(replacement\)/.test(frontendSource), - 'Realtime session messages with an existing stable key should replace changed content instead of being dropped' - ); + assert(frontendSource.includes('getMessageRenderFingerprint'), 'History rendering should fingerprint tool results and source metadata too'); const historyMergeStart = serverSource.indexOf('function stableMessageHash'); const historyMergeEnd = serverSource.indexOf('function sessionHistoryCacheKey', historyMergeStart); @@ -4832,7 +4836,7 @@ function assertCodexAppGoalLifecycleContract() { assert( doneBlock.includes('snapshot.isRunning = msg.goalActive === true') - && doneBlock.includes('finishGenerating(msg.sessionId, { keepRunning: msg.goalActive === true })'), + && /finishGenerating\(msg.sessionId,\s*\{\s*keepRunning: msg.goalActive === true/.test(doneBlock), 'Frontend done handling should keep current and background Goal sessions running between turns' ); assert( @@ -4859,7 +4863,7 @@ function assertCodexAppGoalLifecycleContract() { ); assert( completeBlock.includes('const goalActive = isCodexAppGoalActive(sessionId)') - && completeBlock.includes("{ type: 'done', sessionId, costUsd: null, goalActive }") + && completeBlock.includes("type: 'done', sessionId, costUsd: null, goalActive") && completeBlock.includes('if (goalActive)') && completeBlock.includes('broadcastSessionList()'), 'Server turn completion should report active Goal state without sending final background completion' diff --git a/scripts/session-history-unit.js b/scripts/session-history-unit.js new file mode 100644 index 0000000..fa93c95 --- /dev/null +++ b/scripts/session-history-unit.js @@ -0,0 +1,167 @@ +#!/usr/bin/env node +'use strict'; + +const assert = require('node:assert/strict'); +const crypto = require('node:crypto'); +const fs = require('node:fs'); +const os = require('node:os'); +const path = require('node:path'); +const source = fs.readFileSync(path.join(__dirname, '..', 'server.js'), 'utf8'); +function extract(name) { + const start = source.indexOf(`function ${name}(`); + assert(start >= 0, `缺少 ${name}`); + return source.slice(start, source.indexOf('\nfunction ', start + 1)); +} +const pureSource = source.slice(source.indexOf('function stableMessageHash('), source.indexOf('function sessionHistoryCacheKey(')); +const api = new Function('crypto', `${pureSource}; return { ensureStableMessageId, mergeNativeHistoryWithSnapshot, mergeHistorySegments };`)(crypto); +let checks = 0; +function check(name, fn) { fn(); checks++; console.log(`通过：${name}`); } +const question = '没参考其他的页面么，WMSA51 是你这么写的么？'; +const notice = { role: 'system', ccwebPersistenceNotice: true, content: '旧截断提示' }; +const native = Array.from({ length: 483 }, (_, index) => ({ id: `native-${index}`, role: 'assistant', content: `原生历史 ${index}` })); +const snapshot = Array.from({ length: 68 }, (_, index) => ({ id: `ccweb-${index}`, role: index % 2 ? 'assistant' : 'user', content: `快照历史 ${index}` })); +snapshot[0] = { id: 'ccweb-turn', role: 'assistant', codexAppThreadId: 'thread', codexAppTurnId: 'turn-boundary', content: '权威快照首条' }; +native[415] = { id: 'native-turn', role: 'assistant', nativeThreadId: 'thread', turnId: 'turn-boundary', content: '原生首条片段' }; +snapshot[64] = { id: 'client-final', role: 'user', content: question }; +snapshot[65] = { id: 'architecture', role: 'assistant', content: '先按项目规范核对现有页面、任务票和负责会话。' }; +snapshot[66] = { id: 'reply', role: 'assistant', content: '子对话回传', replyToRequestId: 'request', crossConversation: { sourceTitle: '原实现负责人' } }; +snapshot[67] = { id: 'current-user', role: 'user', content: '继续返工' }; +const session = { codexAppThreadId: 'thread', messages: [notice, ...snapshot], historySnapshotBaseIndex: 900 }; +check('68 条快照与 483 条原生历史合并后保留权威尾部', () => { + const result = api.mergeNativeHistoryWithSnapshot(session, snapshot, native); + assert.equal(result.confirmed, true); + assert.equal(result.nativePrefixCount, 415); + assert.deepEqual(result.messages.slice(-68).map(m => m.id), snapshot.map(m => m.id)); + const index = result.messages.findIndex(m => m.content === question); + assert.equal(result.messages[index + 1].id, 'architecture'); + assert.equal(result.messages[index + 2].crossConversation.sourceTitle, '原实现负责人'); + assert.equal(result.messages.at(-1).id, 'current-user'); + assert(!result.messages.some(m => m.id === 'native-482')); +}); +check('错误数组基线和空快照都不能导入未经确认的 native', () => { + assert.equal(api.mergeNativeHistoryWithSnapshot({ historySnapshotBaseIndex: 10 }, [{ role: 'user', content: '不匹配' }], native).confirmed, false); + assert.equal(api.mergeNativeHistoryWithSnapshot({}, [], native).confirmed, false); +}); +check('不同来源 ID 可以按完整时间戳和内容摘要确认边界', () => { + const boundary = { id: 'local', role: 'user', content: '唯一问题', timestamp: '2026-09-20T10:00:00.000Z' }; + const nativeBoundary = { ...boundary, id: 'native', timestamp: '2026-09-20T18:00:00+08:00' }; + const result = api.mergeNativeHistoryWithSnapshot({}, [boundary, snapshot[65]], [native[0], nativeBoundary, native[2]]); + assert.equal(result.confirmed, true); + assert.deepEqual(result.messages.map(m => m.id), ['native-0', 'local', 'architecture']); + assert.equal(api.mergeNativeHistoryWithSnapshot({}, [boundary], [nativeBoundary, nativeBoundary]).confirmed, false); + assert.equal(api.mergeNativeHistoryWithSnapshot({}, [boundary], [{ ...nativeBoundary, timestamp: '' }]).confirmed, false); +}); +check('不同线程和分段 turn 不能错误匹配，已有 ID 不被改写', () => { + assert.equal(api.mergeNativeHistoryWithSnapshot({}, [snapshot[0]], [{ ...native[415], nativeThreadId: 'other' }]).confirmed, false); + assert.equal(api.mergeNativeHistoryWithSnapshot({}, [snapshot[0]], [{ ...native[415], nativeTurnSegment: 'initial' }]).confirmed, false); + const normalized = api.ensureStableMessageId(snapshot[0]); + assert.equal(normalized.id, snapshot[0].id); + assert.equal(normalized.turnId, 'turn-boundary'); + assert.equal(api.ensureStableMessageId({ role: 'assistant', turnId: 't' }).id, 'history:assistant::t'); +}); +check('重复回传按请求标识去重且以快照内容为准', () => { + const merged = api.mergeHistorySegments([{ ...snapshot[66], id: 'native-reply', content: '旧回传' }], [snapshot[66], snapshot[66]]); + assert.equal(merged.length, 1); + assert.equal(merged[0].content, '子对话回传'); +}); + +const persistApi = new Function('crypto', ` + ${pureSource} + ${extract('isPersistenceNoticeMessage')} + const SESSION_PERSIST_MAX_MESSAGES = 45; + function sanitizeMessageForPersist(message) { return JSON.parse(JSON.stringify(message)); } + function sanitizePersistValue(value) { return value; } + function normalizeSession(value) { return value; } + ${extract('sanitizeMessagesForPersist')} + ${extract('sanitizeSessionForPersist')} + return { sanitizeSessionForPersist }; +`)(crypto); +check('截断提示保存结构化字段，旧基线不覆盖新基线', () => { + const persisted = persistApi.sanitizeSessionForPersist(session, { maxMessages: 45 }); + const metadata = persisted.messages[0]; + assert.equal(metadata.ccwebPersistenceNotice, true); + assert.equal(metadata.snapshotMessageCount, 45); + assert.equal(metadata.snapshotFirstMessageId, snapshot[23].id); + assert.equal(metadata.snapshotLastMessageId, snapshot[67].id); + assert.equal(metadata.nativeThreadId, 'thread'); + assert(Number.isFinite(Date.parse(metadata.truncatedAt))); + assert.equal(persisted.historySnapshotBaseIndex, 923); + assert.equal(persisted.historySnapshotCount, 45); + const again = persistApi.sanitizeSessionForPersist(persisted, { maxMessages: 45 }); + assert.deepEqual(again, persisted); + assert.equal(api.mergeNativeHistoryWithSnapshot({ ...session, messages: [{ ...notice, snapshotFirstMessageId: 'bad' }, ...snapshot] }, snapshot, native).confirmed, false); +}); +check('结构化提示不占保留消息名额，新消息后仍刷新边界', () => { + const persisted = persistApi.sanitizeSessionForPersist({ ...session, messages: [notice, ...snapshot.slice(-45)] }, { maxMessages: 45 }); + assert.equal(persisted.messages.length, 46); + persisted.messages.push({ id: 'new-user', role: 'user', content: '运行中新消息' }); + const next = persistApi.sanitizeSessionForPersist(persisted, { maxMessages: 45 }); + assert.equal(next.messages[0].snapshotMessageCount, 45); + assert.equal(next.messages[0].snapshotLastMessageId, 'new-user'); + assert.equal(next.historySnapshotBaseIndex, persisted.historySnapshotBaseIndex + 1); +}); +check('三个历史出口使用同一 resolver，异常边界保持当前快照', () => { + const resolver = new Function('crypto', ` + ${pureSource} + ${extract('isPersistenceNoticeMessage')} + function loadNativeSessionHistory() { return { messages: ${JSON.stringify(native)} }; } + function plog() {} + ${extract('resolveSessionHistory')} + return resolveSessionHistory; + `)(crypto); + assert.equal(resolver(session).source, 'merged'); + const failed = resolver({ messages: [notice, { id: 'new', role: 'user', content: question }], historySnapshotBaseIndex: 45 }); + assert.equal(failed.available, false); + assert.equal(failed.messages[0].content, question); + for (const fn of ['buildSessionInfoPayload', 'handleLoadHistoryPage', 'handleResumeSession']) assert(extract(fn).includes('resolveSessionHistory')); +}); + +check('迁移只在 idle 执行并精确保留备份，写后可以重新加载', () => { + const dir = fs.mkdtempSync(path.join(os.tmpdir(), 'ccweb-history-unit-')); + try { + const id = '35394008-4f0c-43c0-a481-b622d9e6b884'; + const filePath = path.join(dir, `${id}.json`); + const original = JSON.stringify({ ...session, id }); + fs.writeFileSync(filePath, original); + const testApi = new Function('fs', 'path', 'crypto', 'SESSIONS_DIR', 'nativeMessages', ` + ${pureSource} + ${extract('isPersistenceNoticeMessage')} + let running = true; + let response; + const SESSION_LOAD_MAX_BYTES = 10000000; + const sessionHistoryCache = new Map(); + const sessionSearchIndex = { scheduleUpsert() {} }; + function scheduleUsageStatisticsUpsert() {} + function sanitizeId(id) { return id; } + function sessionPath(id) { return path.join(SESSIONS_DIR, id + '.json'); } + function isSessionRunning() { return running; } + function normalizeSession(value) { return value; } + function loadNativeSessionHistory() { return { messages: nativeMessages }; } + function textByteLength(value) { return Buffer.byteLength(value); } + function loadSession(id) { return JSON.parse(fs.readFileSync(sessionPath(id), 'utf8')); } + function sessionHistoryCacheKey(session) { return session.id; } + function wsSend(ws, result) { response = result; } + function attachClientRequestId(payload) { return payload; } + function findViewingSessionWs() { return null; } + function plog() {} + ${extract('writeFileAtomicSync')} + ${extract('handleMigrateSessionHistory')} + return { run(id, apply) { handleMigrateSessionHistory(null, { sessionId: id, apply }); return response; }, idle() { running = false; } }; + `)(fs, path, crypto, dir, native); + assert.equal(testApi.run(id, true).code, 'session_running'); + assert.equal(fs.readFileSync(filePath, 'utf8'), original); + testApi.idle(); + assert.equal(testApi.run(id, false).preview, true); + assert.equal(fs.readFileSync(filePath, 'utf8'), original); + const result = testApi.run(id, true); + assert.equal(result.ok, true); + assert.equal(result.changed, true); + assert.equal(fs.readFileSync(result.backupPath, 'utf8'), original); + const restored = JSON.parse(fs.readFileSync(filePath, 'utf8')); + assert.equal(restored.messages.length, 483); + assert(!restored.messages.some(m => m.ccwebPersistenceNotice)); + assert.deepEqual(restored.messages.slice(-68).map(m => m.id), snapshot.map(m => m.id)); + assert.equal(testApi.run(id, true).changed, false); + } finally { fs.rmSync(dir, { recursive: true, force: true }); } +}); +console.log(`历史服务端回归通过：${checks} 项`); diff --git a/server.js b/server.js index d508bda..5c37ab3 100644 --- a/server.js +++ b/server.js @@ -74,6 +74,11 @@ if (process.argv.includes('--ccweb-mcp-server')) { return; } +if (process.argv.includes('--migrate-session-history')) { + require('./scripts/migrate-session-history'); + return; +} + if (process.argv.includes('--codex-app-worker')) { require('./lib/codex-app-worker'); return; @@ -4438,91 +4443,121 @@ function messageReplyToRequestId(message) { function ensureStableMessageId(message) { if (!message || typeof message !== 'object' || Array.isArray(message)) return message; - if (String(message.id || '').trim()) return message; - const explicitId = String(message.clientMessageId || message.messageId || '').trim(); - const replyToRequestId = messageReplyToRequestId(message); + const turnId = String(message.turnId || message.codexAppTurnId || '').trim(); + const output = turnId && !message.turnId ? { ...message, turnId } : message; + if (String(output.id || '').trim()) return output; + const explicitId = String(output.clientMessageId || output.messageId || '').trim(); + const replyToRequestId = messageReplyToRequestId(output); + const threadId = output.nativeThreadId || output.codexAppThreadId || ''; const stableId = explicitId || (replyToRequestId ? `reply:${replyToRequestId}` : '') - || (message.codexAppTurnKey ? `codexapp:${message.codexAppTurnKey}` : '') - || (message.nativeTurnKey ? `native:${message.nativeTurnKey}` : ''); - if (stableId) return { ...message, id: stableId }; - const content = typeof message.content === 'string' ? message.content : JSON.stringify(message.content || ''); - const identity = [message.role || '', message.timestamp || message.createdAt || '', content].join('\u001f'); - return { ...message, id: `message:${stableMessageHash(identity)}` }; + || (turnId ? `history:${output.role || ''}:${threadId}:${turnId}` : '') + || (output.codexAppTurnKey ? `codexapp:${output.codexAppTurnKey}` : '') + || (output.nativeTurnKey ? `native:${output.nativeTurnKey}` : ''); + if (stableId) return { ...output, id: stableId }; + const content = typeof output.content === 'string' ? output.content : JSON.stringify(output.content || ''); + const identity = [output.role || '', output.timestamp || output.createdAt || '', content].join('\u001f'); + return { ...output, id: `message:${stableMessageHash(identity)}` }; } function normalizeHistoryMessage(message) { return ensureStableMessageId(message); } -function messagesEquivalent(left, right) { - if (!left || !right || left.role !== right.role) return false; - const leftId = String(left.id || '').trim(); - const rightId = String(right.id || '').trim(); - if (leftId && rightId && leftId === rightId) return true; - if (!leftId.startsWith('message:') || !rightId.startsWith('message:')) return false; - const leftContent = typeof left.content === 'string' ? left.content.trim() : JSON.stringify(left.content || ''); - const rightContent = typeof right.content === 'string' ? right.content.trim() : JSON.stringify(right.content || ''); - return !!leftContent - && leftContent === rightContent - && String(left.timestamp || '') === String(right.timestamp || ''); +function messageIdentityKeys(message, session = {}) { + if (!message || typeof message !== 'object') return []; + const role = message.role || ''; + const keys = [message.id, message.clientMessageId, message.messageId] + .map((value) => String(value || '').trim()).filter(Boolean) + .map((value) => `${role}:id:${value}`); + const replyId = messageReplyToRequestId(message); + if (replyId) keys.push(`${role}:reply:${replyId}`); + if (message.codexAppTurnKey) keys.push(`${role}:codexapp:${message.codexAppTurnKey}`); + if (message.nativeTurnKey) keys.push(`${role}:native:${message.nativeTurnKey}`); + const turnId = String(message.turnId || message.codexAppTurnId || '').trim(); + const threadId = message.nativeThreadId || message.codexAppThreadId || session.codexAppThreadId || session.codexThreadId || ''; + // 同 turn 的引导输入和分段回复是不同气泡，不能仅靠 turnId 将它们吞并。 + if (role === 'assistant' && turnId && !message.nativeTurnSegment) { + keys.push(`${role}:turn:${threadId}:${turnId}`); + } + return keys; } -function mergeHistorySegments(nativePrefix, snapshot) { +function messageContentKey(message) { + if (!message || !message.role) return ''; + const timestamp = String(message.timestamp || message.createdAt || '').trim(); + const time = Date.parse(timestamp); + const content = typeof message.content === 'string' ? message.content.trim() : JSON.stringify(message.content || ''); + if (!timestamp || !Number.isFinite(time) || !content) return ''; + return `${message.role}:${time}:${stableMessageHash(content)}`; +} + +function messagesEquivalent(left, right, session = {}) { + if (!left || !right || left.role !== right.role) return false; + const rightKeys = new Set(messageIdentityKeys(right, session)); + if (messageIdentityKeys(left, session).some((key) => rightKeys.has(key))) return true; + const contentKey = messageContentKey(left); + return !!contentKey && contentKey === messageContentKey(right); +} + +function mergeHistorySegments(nativePrefix, snapshot, session = {}) { const prefix = Array.isArray(nativePrefix) ? nativePrefix : []; const tail = Array.isArray(snapshot) ? snapshot : []; - const snapshotIds = new Set(tail.map((message) => String(message?.id || '').trim()).filter(Boolean)); + const snapshotKeys = new Set(tail.flatMap((message) => messageIdentityKeys(message, session))); const merged = []; - const prefixIds = new Set(); - for (const message of prefix) { - const id = String(message?.id || '').trim(); - if (id && snapshotIds.has(id)) continue; - if (id && prefixIds.has(id)) continue; - if (id) prefixIds.add(id); - merged.push(message); - } - const snapshotPositions = new Map(); - for (const message of tail) { - const id = String(message?.id || '').trim(); - if (id && snapshotPositions.has(id)) { - merged[snapshotPositions.get(id)] = message; - continue; + const positions = new Map(); + const earlier = prefix.filter((message) => !messageIdentityKeys(message, session).some((key) => snapshotKeys.has(key))); + for (const message of [...earlier, ...tail]) { + const keys = messageIdentityKeys(message, session); + const position = keys.map((key) => positions.get(key)).find((value) => value !== undefined); + if (position !== undefined) { + merged[position] = message; + keys.forEach((key) => positions.set(key, position)); + } else { + keys.forEach((key) => positions.set(key, merged.length)); + merged.push(message); } - if (id) snapshotPositions.set(id, merged.length); - merged.push(message); } return merged; } function mergeNativeHistoryWithSnapshot(session, persistedMessages, nativeMessages) { - const snapshot = (Array.isArray(persistedMessages) ? persistedMessages : []).map(normalizeHistoryMessage); + const persisted = (Array.isArray(persistedMessages) ? persistedMessages : []).map(normalizeHistoryMessage); + const snapshot = mergeHistorySegments([], persisted, session); const native = (Array.isArray(nativeMessages) ? nativeMessages : []).map(normalizeHistoryMessage); - if (snapshot.length === 0) return native.length > 0 ? { messages: mergeHistorySegments(native, []), confirmed: true } : { messages: snapshot, confirmed: true }; - if (native.length === 0) return { messages: snapshot, confirmed: false }; + const fallback = { messages: snapshot, confirmed: false }; + if (snapshot.length === 0 || native.length === 0) return fallback; - // 从尾部寻找快照与 native 的最长有序重叠，允许 native 中存在旧版本额外事件。 - let nativeCursor = native.length - 1; - let snapshotCursor = snapshot.length - 1; - while (nativeCursor >= 0 && snapshotCursor >= 0) { - if (messagesEquivalent(native[nativeCursor], snapshot[snapshotCursor])) { - nativeCursor -= 1; - snapshotCursor -= 1; - continue; - } - nativeCursor -= 1; - } - if (snapshotCursor < 0) { - const overlapStart = nativeCursor + 1; - return { messages: mergeHistorySegments(native.slice(0, overlapStart), snapshot), confirmed: true }; + const notice = (session?.messages || []).find((message) => message?.ccwebPersistenceNotice === true); + const threadId = session?.codexAppThreadId || session?.codexThreadId || session?.claudeSessionId || ''; + if (notice?.nativeThreadId && notice.nativeThreadId !== threadId) return fallback; + if (notice?.snapshotFirstMessageId && notice.snapshotFirstMessageId !== snapshot[0]?.id) return fallback; + if (notice?.snapshotMessageCount !== undefined) { + const count = Number(notice.snapshotMessageCount); + if (!Number.isSafeInteger(count) || count <= 0 || count > persisted.length) return fallback; + if (notice.snapshotLastMessageId && notice.snapshotLastMessageId !== persisted[count - 1]?.id) return fallback; } - const baseIndex = Number(session?.historySnapshotBaseIndex); - const hasSnapshotBoundary = Number.isSafeInteger(baseIndex) && baseIndex > 0; - if (hasSnapshotBoundary && native.length >= baseIndex) { - const prefix = native.slice(0, Math.min(baseIndex, native.length)); - return { messages: mergeHistorySegments(prefix, snapshot), confirmed: true }; + // 只定位快照首条的可靠边界。快照尾部可能包含尚未写入 rollout 的当前输入和回传。 + const firstKeys = new Set(messageIdentityKeys(snapshot[0], session)); + let matches = native.map((message, index) => ({ message, index })) + .filter(({ message }) => messageIdentityKeys(message, session).some((key) => firstKeys.has(key))); + if (matches.length === 0) { + const fingerprint = messageContentKey(snapshot[0]); + if (!fingerprint) return fallback; + matches = native.map((message, index) => ({ message, index })) + .filter(({ message }) => messageContentKey(message) === fingerprint); + if (matches.length !== 1) return fallback; + } else if (matches.length > 1) { + // 原生记录重复同一个 ID 可以取最早出现处；多个不同 ID 的别名命中不能猜测。 + if (new Set(matches.map(({ message }) => message.id)).size !== 1) return fallback; } - return { messages: snapshot, confirmed: false }; + const boundaryIndex = matches[0].index; + return { + messages: mergeHistorySegments(native.slice(0, boundaryIndex), snapshot, session), + confirmed: true, + nativePrefixCount: boundaryIndex, + }; } function sessionHistoryCacheKey(session) { @@ -4594,7 +4629,7 @@ function resolveSessionHistory(session) { const notice = persisted.some(isPersistenceNoticeMessage); if (!notice) { return { - messages: persistedMessages.map(normalizeHistoryMessage), + messages: mergeHistorySegments([], persistedMessages.map(normalizeHistoryMessage), session), source: 'snapshot', available: true, recoverable: false, @@ -4603,7 +4638,7 @@ function resolveSessionHistory(session) { const native = loadNativeSessionHistory(session); const merged = mergeNativeHistoryWithSnapshot(session, persistedMessages, native?.messages); - if (merged.confirmed && merged.messages.length >= persistedMessages.length) { + if (merged.confirmed) { return { messages: merged.messages, source: merged.messages.length > persistedMessages.length ? 'merged' : 'snapshot', @@ -4618,7 +4653,7 @@ function resolveSessionHistory(session) { nativeCount: Array.isArray(native.messages) ? native.messages.length : 0, }); } - return { messages: persistedMessages.map(normalizeHistoryMessage), source: 'snapshot', available: false, recoverable: false }; + return { messages: mergeHistorySegments([], persistedMessages.map(normalizeHistoryMessage), session), source: 'snapshot', available: false, recoverable: false }; } function normalizeAgent(agent) { @@ -5526,17 +5561,27 @@ function sanitizeMessageForPersist(message, limits = {}) { return output; } -function sanitizeMessagesForPersist(messages, limits = {}) { - const list = Array.isArray(messages) ? messages : []; +function sanitizeMessagesForPersist(messages, limits = {}, session = {}) { + const source = Array.isArray(messages) ? messages : []; + const previousNotice = source.find(isPersistenceNoticeMessage); + const list = source.filter((message) => !isPersistenceNoticeMessage(message)); const maxMessages = limits.maxMessages || SESSION_PERSIST_MAX_MESSAGES; const selected = list.length > maxMessages ? list.slice(-maxMessages) : list; const output = selected.map((message) => sanitizeMessageForPersist(ensureStableMessageId(message), limits)); - if (list.length > selected.length) { + if (previousNotice || list.length > selected.length) { + const truncatedAt = list.length > selected.length + ? new Date().toISOString() + : previousNotice.truncatedAt || previousNotice.timestamp || new Date().toISOString(); output.unshift({ role: 'system', - content: `历史消息过多，cc-web 本地快照只保留最近 ${selected.length} 条；点击顶部“查看更早消息”可从原始会话记录加载省略的 ${list.length - selected.length} 条旧消息。`, - timestamp: new Date().toISOString(), + content: `cc-web 本地快照保留最近 ${selected.length} 条消息；更早历史仅在确认与原始会话的边界后加载。`, + timestamp: truncatedAt, ccwebPersistenceNotice: true, + snapshotMessageCount: output.length, + snapshotFirstMessageId: output[0]?.id || null, + snapshotLastMessageId: output[output.length - 1]?.id || null, + nativeThreadId: session.codexAppThreadId || session.codexThreadId || session.claudeSessionId || previousNotice?.nativeThreadId || null, + truncatedAt, }); } return output; @@ -5571,29 +5616,10 @@ function sanitizeSessionForPersist(session, limits = {}) { for (const [key, value] of Object.entries(session || {})) { if (skipKeys.has(key)) continue; if (key === 'messages') { - output.messages = sanitizeMessagesForPersist(value, limits); - const sourceMessages = Array.isArray(value) ? value : []; - const persistedMessages = sourceMessages.filter((message) => !isPersistenceNoticeMessage(message)); - const existingBaseIndex = Number(session?.historySnapshotBaseIndex); - const existingCount = Number(session?.historySnapshotCount); - if (sourceMessages.length > (limits.maxMessages || SESSION_PERSIST_MAX_MESSAGES)) { - const maxPersistedMessages = limits.maxMessages || SESSION_PERSIST_MAX_MESSAGES; - const logicalDroppedCount = Math.max(0, persistedMessages.length - maxPersistedMessages); - output.historySnapshotBaseIndex = Number.isSafeInteger(existingBaseIndex) && existingBaseIndex > 0 - ? existingBaseIndex + logicalDroppedCount - : logicalDroppedCount; - output.historySnapshotCount = Math.max(0, output.messages.length - (output.messages.some(isPersistenceNoticeMessage) ? 1 : 0)); - } else if (Number.isSafeInteger(existingBaseIndex) && existingBaseIndex > 0) { - output.historySnapshotBaseIndex = existingBaseIndex; - output.historySnapshotCount = Number.isSafeInteger(existingCount) && existingCount > 0 - ? Math.max(existingCount, persistedMessages.length) - : persistedMessages.length; - } else { - output.historySnapshotBaseIndex = 0; - output.historySnapshotCount = persistedMessages.length; - } + output.messages = sanitizeMessagesForPersist(value, limits, session); continue; } + if (key === 'historySnapshotBaseIndex' || key === 'historySnapshotCount') continue; output[key] = sanitizePersistValue(value, { maxString: limits.topLevelMaxChars || 16 * 1024, maxDepth: 4, @@ -5602,6 +5628,13 @@ function sanitizeSessionForPersist(session, limits = {}) { }); } if (!Object.prototype.hasOwnProperty.call(output, 'messages')) output.messages = []; + const sourceMessages = (Array.isArray(session?.messages) ? session.messages : []).filter((message) => !isPersistenceNoticeMessage(message)); + const snapshotMessages = output.messages.filter((message) => !isPersistenceNoticeMessage(message)); + const existingBaseIndex = Number(session?.historySnapshotBaseIndex); + // 基线只表示 cc-web 的逻辑序列位置，不能拿它直接切割不同粒度的 native 数组。 + output.historySnapshotBaseIndex = (Number.isSafeInteger(existingBaseIndex) && existingBaseIndex > 0 ? existingBaseIndex : 0) + + Math.max(0, sourceMessages.length - snapshotMessages.length); + output.historySnapshotCount = snapshotMessages.length; return normalizeSession(output); } @@ -9762,6 +9795,9 @@ wss.on('connection', (ws, req) => { case 'load_history_page': handleLoadHistoryPage(ws, msg); break; + case 'migrate_session_history': + handleMigrateSessionHistory(ws, msg); + break; case 'search_sessions': handleSearchSessions(ws, msg); break; @@ -11313,6 +11349,61 @@ function handleLoadHistoryPage(ws, msg = {}) { }, msg)); } +function handleMigrateSessionHistory(ws, msg = {}) { + const sessionId = sanitizeId(msg.sessionId || ''); + const respond = (result) => wsSend(ws, attachClientRequestId({ + type: 'migrate_session_history_result', sessionId, ...result, + }, msg)); + // 检查、备份和写入在服务端同一同步调用内完成，避免运行任务保存覆盖迁移。 + if (isSessionRunning(sessionId)) return respond({ ok: false, code: 'session_running' }); + const filePath = sessionPath(sessionId); + if (!sessionId || !fs.existsSync(filePath)) return respond({ ok: false, code: 'session_not_found' }); + try { + const original = fs.readFileSync(filePath, 'utf8'); + const session = normalizeSession(JSON.parse(original)); + if (!session.messages.some(isPersistenceNoticeMessage)) { + return respond({ ok: true, changed: false, messageCount: session.messages.length }); + } + const snapshot = session.messages.filter((message) => !isPersistenceNoticeMessage(message)); + const native = loadNativeSessionHistory(session); + const merged = mergeNativeHistoryWithSnapshot(session, snapshot, native?.messages); + if (!merged.confirmed) return respond({ ok: false, code: 'history_boundary_unconfirmed' }); + const migrated = { + ...session, + messages: merged.messages, + historySnapshotBaseIndex: 0, + historySnapshotCount: merged.messages.length, + }; + // 迁移保留快照原文；不再走会立即截断的普通持久化路径。 + const json = JSON.stringify(migrated, null, 2); + if (textByteLength(json) > SESSION_LOAD_MAX_BYTES) return respond({ ok: false, code: 'history_migration_too_large' }); + const details = { snapshotCount: snapshot.length, messageCount: merged.messages.length, nativePrefixCount: merged.nativePrefixCount }; + if (msg.apply !== true) return respond({ ok: true, preview: true, ...details }); + if (isSessionRunning(sessionId) || fs.readFileSync(filePath, 'utf8') !== original) { + return respond({ ok: false, code: 'session_changed' }); + } + const backupDir = path.join(SESSIONS_DIR, '_history-backups'); + fs.mkdirSync(backupDir, { recursive: true }); + const backupPath = path.join(backupDir, `${sessionId}.${Date.now()}.${crypto.randomUUID()}.json`); + fs.writeFileSync(backupPath, original, { flag: 'wx', mode: 0o600 }); + writeFileAtomicSync(filePath, json); + const reloaded = loadSession(sessionId); + if (!reloaded || reloaded.messages.length !== migrated.messages.length) { + writeFileAtomicSync(filePath, original); + return respond({ ok: false, code: 'history_reload_failed', backupPath }); + } + sessionHistoryCache.delete(sessionHistoryCacheKey(session)); + sessionSearchIndex.scheduleUpsert(sessionId); + scheduleUsageStatisticsUpsert(sessionId); + respond({ ok: true, changed: true, backupPath, ...details }); + const viewingWs = findViewingSessionWs(sessionId); + if (viewingWs) wsSend(viewingWs, buildSessionInfoPayload(reloaded)); + } catch (error) { + plog('WARN', 'session_history_migration_failed', { sessionId, error: error?.message || String(error) }); + respond({ ok: false, code: 'history_migration_failed' }); + } +} + function attachActiveRuntimeToWs(ws, sessionId, source = {}) { if (activeProcesses.has(sessionId)) { const entry = activeProcesses.get(sessionId); @@ -11345,6 +11436,11 @@ function attachActiveRuntimeToWs(ws, sessionId, source = {}) { wsSend(ws, attachClientRequestId({ type: 'resume_generating', sessionId, + assistantMessageId: ensureStableMessageId({ + role: 'assistant', codexAppThreadId: entry.threadId, codexAppTurnId: entry.turnId, + codexAppTurnKey: codexAppTurnKey(sessionId, entry), + }).id, + turnId: entry.turnId || null, text: truncateTextValue(entry.fullText || '', SESSION_MESSAGE_CONTENT_MAX_CHARS), toolCalls: sanitizeToolCallsForPersist(entry.toolCalls || []), }, source)); @@ -14268,7 +14364,7 @@ function handleCodexAppTurnComplete(sessionId, options = {}) { } if (session && (assistantContent.trim() || assistantToolCalls.length > 0) && !hasCodexAppTurnMessage(session, turnKey)) { - const assistantMessage = { + const assistantMessage = ensureStableMessageId({ role: 'assistant', content: assistantContent, toolCalls: assistantToolCalls, @@ -14277,7 +14373,7 @@ function handleCodexAppTurnComplete(sessionId, options = {}) { codexAppThreadId: entry.threadId || null, codexAppTurnId: entry.turnId || null, interrupted: !!options.interrupted, - }; + }); const beforeUserMessage = options.beforeUserMessage; const beforeUserMessageIndex = beforeUserMessage ? session.messages.findIndex((message) => ( @@ -14355,7 +14451,14 @@ function handleCodexAppTurnComplete(sessionId, options = {}) { entry.errorSent = true; wsSend(entry.ws, { type: 'error', sessionId, message: completionError }); } - wsSend(entry.ws, { type: 'done', sessionId, costUsd: null, goalActive }); + const assistantMessage = session?.messages?.find((message) => message.codexAppTurnKey === turnKey); + const assistantMessageIndex = assistantMessage + ? resolveSessionHistory(session).messages.findIndex((message) => message.id === assistantMessage.id) + : -1; + wsSend(entry.ws, { + type: 'done', sessionId, costUsd: null, goalActive, + ...(assistantMessage ? { assistantMessage: sanitizeMessageForTransport(assistantMessage), assistantMessageIndex } : {}), + }); sendSessionList(entry.ws); return; } @@ -14546,6 +14649,7 @@ function handleCodexAppSteerMessage(ws, msg, options = {}) { } persistedUserMessage = { id: userMessageId, + ...(clientMessageId ? { clientMessageId } : {}), role: 'user', content: textValue, attachments: savedAttachments,