工具使用
This commit is contained in:
@@ -1,6 +1,6 @@
|
||||
# Command Code OpenAI Bridge
|
||||
|
||||
Command Code OpenAI Bridge 是一个仅在本机运行的 OpenAI Chat Completions 兼容服务。它把 OpenAI 客户端发送的文本对话转交给 Command Code CLI 执行,并将最终 assistant 回复以标准 Chat Completions 格式返回。
|
||||
Command Code OpenAI Bridge 是一个仅在本机运行的 OpenAI 兼容服务。它把 Chat Completions 文本对话与 function calling 决策转交给 Command Code CLI 执行,并提供 Responses API 文本子集。
|
||||
|
||||
默认 API 地址:`http://127.0.0.1:18000/v1`
|
||||
|
||||
@@ -12,7 +12,7 @@ Command Code OpenAI Bridge 是一个仅在本机运行的 OpenAI Chat Completion
|
||||
2. 保留消息的角色、顺序和完整文本,将全部会话历史序列化后通过 stdin 传给 Command Code。
|
||||
3. 在启动服务的终端中显示模型状态、turn、工具调用、工具结果、stderr、最终回答和耗时。
|
||||
4. 从 Command Code 的 NDJSON 结构化结果中提取 `finalText`。
|
||||
5. 只把最终回复和可用的 token usage 返回给 API 客户端。
|
||||
5. 把最终回复、外部 function tool calls 和可用的 token usage 返回给 API 客户端。
|
||||
|
||||
服务不保存跨请求会话。每次调用都使用 `--no-session`,对话历史由客户端维护并在请求中完整提供。
|
||||
|
||||
@@ -20,16 +20,17 @@ Command Code OpenAI Bridge 是一个仅在本机运行的 OpenAI Chat Completion
|
||||
|
||||
- 提供 `GET /health`、`GET /v1/models` 和 `POST /v1/chat/completions`。
|
||||
- 支持字符串形式的 `content` 和 OpenAI 文本 part 数组。
|
||||
- 接受 `stream: true`,但 Command Code 调用仍为非流式;完整结果生成后再一次性封装为 SSE 事件返回。
|
||||
- Chat Completions 支持标准 function tools、工具选择、assistant tool calls 和 tool 结果消息;工具由客户端执行。
|
||||
- 接受 `stream: true`;普通 Chat 实时转发文本 delta,function calling 在完整决策校验后发送工具调用 SSE。
|
||||
- 支持中文、Unicode、Markdown、代码块和较长文本。
|
||||
- 模型名称通过 YAML 配置映射到实际 Command Code 模型。
|
||||
- 请求内容通过 stdin 传输,不受命令行参数长度限制。
|
||||
- 同时只运行一个 Command Code 请求;忙碌时返回 HTTP 429。
|
||||
- 同时只运行一个 Command Code 请求;其余请求进入有限 FIFO 队列,队列已满时返回 HTTP 429。
|
||||
- 客户端断开、请求取消、总超时和 Ctrl+C 会终止当前子进程组。
|
||||
- 单次 Command Code 失败后,HTTP 服务可继续处理后续请求。
|
||||
- Authorization 请求头会被忽略,服务强制绑定 `127.0.0.1`。
|
||||
|
||||
当前只支持文本 Chat Completions。`stream: true` 是客户端兼容层,不会实时输出 Command Code 的生成过程。图片、音频、function calling、Responses API 和服务端会话均未实现。
|
||||
当前只接受文本消息和文本输入。普通 Chat SSE 实时转发 Command Code 文本 delta;function calling 会等待完整决策通过 JSON 与参数 Schema 校验后发送 `tool_calls`。Responses 只实现文本子集。图片、音频、文件输入、Computer Use、托管工具和服务端 Chat 会话均未实现。
|
||||
|
||||
## 技术实现
|
||||
|
||||
@@ -50,6 +51,7 @@ src/
|
||||
├── server.ts # HTTP 路由、单请求状态和错误处理
|
||||
├── openai.ts # 请求校验、消息转换和响应生成
|
||||
├── command-code.ts # CLI 检查、子进程管理和最终结果提取
|
||||
├── tool-calling.ts # 外部 function calling 决策解析与参数校验
|
||||
├── renderer.ts # Command Code 事件的终端显示
|
||||
└── config.ts # YAML 配置读取与校验
|
||||
|
||||
@@ -71,7 +73,7 @@ command-code status --json
|
||||
./scripts/start.sh
|
||||
```
|
||||
|
||||
默认配置位于 `config.yaml`,配置示例位于 `config.example.yaml`。常用配置包括监听端口、Command Code 可执行文件、工作目录、总超时、请求体上限、最大 turn 数、权限模式和模型映射。
|
||||
默认配置位于 `config.yaml`,配置示例位于 `config.example.yaml`。常用配置包括监听端口、Command Code 可执行文件、工作目录、总超时、请求体上限、等待队列上限、最大 turn 数、权限模式和模型映射。
|
||||
|
||||
调用示例:
|
||||
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
# Command Code OpenAI Bridge
|
||||
|
||||
这是一个只绑定本机地址的 OpenAI 兼容转发服务。它保留 Chat Completions 文本接口,并实现单并发的 Responses API 文本子集。每个生成请求启动一次独立的 Command Code headless agent,将完整消息历史通过 stdin 发送给 CLI,在当前终端显示运行过程,再把结构化事件和最终 `finalText` 转换成 OpenAI 风格的 JSON 或 SSE。
|
||||
这是一个只绑定本机地址的 OpenAI 兼容转发服务。它实现 Chat Completions 文本与 function calling,并提供 Responses API 文本子集。每个生成请求启动一次独立的 Command Code headless agent,将完整消息历史通过 stdin 发送给 CLI,在当前终端显示运行过程,再把结构化事件和最终 `finalText` 转换成 OpenAI 风格的 JSON 或 SSE。
|
||||
|
||||
默认地址:`http://127.0.0.1:18000/v1`
|
||||
|
||||
@@ -52,7 +52,8 @@ command-code-openai-bridge/
|
||||
│ ├── request-coordinator.ts
|
||||
│ ├── responses.ts
|
||||
│ ├── renderer.ts
|
||||
│ └── server.ts
|
||||
│ ├── server.ts
|
||||
│ └── tool-calling.ts
|
||||
├── scripts/
|
||||
│ ├── install.sh
|
||||
│ └── start.sh
|
||||
@@ -90,6 +91,7 @@ command_code_executable: command-code
|
||||
command_code_working_directory: .
|
||||
timeout_seconds: 1800
|
||||
max_request_bytes: 20971520
|
||||
max_queue_size: 8
|
||||
max_turns: 100
|
||||
stream_thinking: false
|
||||
response_store_directory: .command-code-openai-bridge/responses
|
||||
@@ -102,7 +104,7 @@ models:
|
||||
effort: max
|
||||
```
|
||||
|
||||
`command_code_working_directory` 决定 Command Code 能看到和操作的项目目录。`response_store_directory` 保存 `store: true` 的 Response;两个相对路径都以配置文件所在目录为基准。默认存储目录是隐藏目录 `.command-code-openai-bridge/responses`。`stream_thinking` 控制 Chat Completions 流是否把 Command Code 思考事件包装成 `<think>` 块;默认关闭。`models.<name>.effort` 会传给 Command Code 的 `--effort`,Responses 请求中的 `reasoning.effort` 优先。可用 effort 由对应底层模型决定。服务强制只监听 `127.0.0.1`。客户端只能选择配置中的模型名,不能注入额外 CLI 参数。
|
||||
`command_code_working_directory` 决定 Command Code 能看到和操作的项目目录。`response_store_directory` 保存 `store: true` 的 Response;两个相对路径都以配置文件所在目录为基准。默认存储目录是隐藏目录 `.command-code-openai-bridge/responses`。`max_queue_size` 是等待执行的生成请求上限,不包含当前运行中的请求;设为 `0` 可恢复忙碌时立即返回 429 的行为。`stream_thinking` 控制 Chat Completions 流是否把 Command Code 思考事件包装成 `<think>` 块;默认关闭。`models.<name>.effort` 会传给 Command Code 的 `--effort`,Responses 请求中的 `reasoning.effort` 优先。可用 effort 由对应底层模型决定。服务强制只监听 `127.0.0.1`。客户端只能选择配置中的模型名,不能注入额外 CLI 参数。
|
||||
|
||||
### Obsidian Copilot 流式输出
|
||||
|
||||
@@ -134,7 +136,7 @@ npm run build
|
||||
|
||||
空闲时按 Ctrl+C 停止。请求运行时第一次 Ctrl+C 取消当前 Command Code 子进程组并保留服务;请求结束后再按 Ctrl+C 停止服务。重启就是再次执行启动脚本。
|
||||
|
||||
客户端断开连接也会先给整个 Command Code 子进程组发送 SIGINT,随后按需升级为 SIGTERM 和 SIGKILL。总超时由 `timeout_seconds` 控制。
|
||||
运行中的客户端断开连接会先给整个 Command Code 子进程组发送 SIGINT,随后按需升级为 SIGTERM 和 SIGKILL;排队中的客户端断开连接只会从等待队列移除。总超时由 `timeout_seconds` 控制。
|
||||
|
||||
## API
|
||||
|
||||
@@ -146,15 +148,21 @@ npm run build
|
||||
- `DELETE /v1/responses/:response_id`
|
||||
- `GET /v1/responses/:response_id/input_items`
|
||||
|
||||
Bearer Token 会被忽略。所有生成接口只接受文本。图片、音频、文件、工具和其他未实现字段会返回 OpenAI 格式的 400,`code` 为 `unsupported_parameter`;服务不会静默忽略会改变行为的参数。
|
||||
`GET /health` 额外返回 `busy`、`queue_length` 和 `queue_capacity`,可用于观察当前执行槽和等待队列。
|
||||
|
||||
Bearer Token 会被忽略。所有生成接口只接受文本内容。图片、音频、文件输入和其他未实现字段会返回 OpenAI 格式的 400,`code` 为 `unsupported_parameter`;服务不会静默忽略会改变行为的参数。
|
||||
|
||||
### Chat Completions
|
||||
|
||||
支持字符串 `content`,也支持由 `{ "type": "text", "text": "..." }` 组成的数组。`stream` 省略或设为 `false` 时返回普通 JSON。`stream: true` 会立即建立 SSE 连接并发送 assistant role 块,随后在 Command Code 的 `text_delta` 到达时立即发送对应 SSE chunk,最后发送 `finish_reason: "stop"`、可选 usage 块和 `[DONE]`。`stream_thinking: true` 时,`thinking_delta` 会先作为 `<think>` 内容流发送;该兼容格式不是原生 OpenAI reasoning item。传入 `stream_options.include_usage: true` 时,结束前返回 usage 块。
|
||||
支持 `system`、`developer`、`user`、`assistant` 和 `tool` 消息。文本 `content` 可以是字符串,也可以是由 `{ "type": "text", "text": "..." }` 组成的数组;assistant 工具轮次还支持 `content: null`、`tool_calls`,tool 消息支持 `tool_call_id` 和可选 `name`。
|
||||
|
||||
Chat 接口接受标准 function tools、`tool_choice` 和 `parallel_tool_calls`。Bridge 把工具定义、完整消息历史和工具结果交给 Command Code 决定下一步,校验返回的工具名与 arguments JSON Schema;第一次不合格时在原请求总截止时间内执行一次修复。工具由 API 客户端执行,Bridge 不执行客户端工具,也不直接访问 Obsidian vault。该流程适用于 Obsidian Copilot 的 `localSearch`、`readNote`、`getFileTree`、`writeFile` 和 `editFile`,也适用于其他标准 function tools。协议流程参考 [OpenAI Function calling](https://developers.openai.com/api/docs/guides/function-calling)。
|
||||
|
||||
`stream` 省略或设为 `false` 时返回普通 JSON。普通文本请求的 `stream: true` 会立即建立 SSE 连接并发送 assistant role 块,随后在 Command Code 的 `text_delta` 到达时立即发送对应 SSE chunk,最后发送 `finish_reason: "stop"`、可选 usage 块和 `[DONE]`。工具模式需要先解析和校验完整决策:最终文本会在校验后作为 content chunk 发送;工具调用会作为 `delta.tool_calls` 发送,并以 `finish_reason: "tool_calls"` 结束。`stream_thinking: true` 只作用于普通文本请求;该兼容格式不是原生 OpenAI reasoning item。传入 `stream_options.include_usage: true` 时,结束前返回 usage 块。
|
||||
|
||||
为兼容 Obsidian Copilot 和 LangChain OpenAI-format 客户端,Chat 接口还接受 `temperature`、`max_tokens`、`max_completion_tokens`、`top_p`、`frequency_penalty`、`presence_penalty` 和 `n: 1`。这些采样和输出限制参数会按 OpenAI 取值范围严格校验。Command Code CLI 1.12.0 没有对应的 headless 参数,因此 Bridge 不会把它们伪装成已生效:服务终端会输出警告,HTTP 响应带 `X-Command-Code-Ignored-Parameters`。其他未实现字段仍返回 `400 unsupported_parameter`。
|
||||
|
||||
Chat Completions 和 Responses 共用 Command Code NDJSON 实时事件执行层。服务终端会在事件到达时立即显示状态、文本和工具调用。Chat Completions 会把每个 `text_delta` 直接写入 SSE,实现生成过程中的打字机效果。Bridge 提示 Command Code 在工具调用 turn 不输出面向用户的文本;Command Code 只有在 `turn_end` 才给出 `hadToolCalls`,因此上游若仍在工具 turn 输出文本,该文本已经发送,SSE 无法撤回。
|
||||
Chat Completions 和 Responses 共用 Command Code NDJSON 执行层。服务终端会在事件到达时立即显示状态、文本和 Command Code 自身的工具调用。普通文本 Chat 会把每个 `text_delta` 直接写入 SSE;外部 function calling 模式会缓存 `finalText`,避免内部 JSON 决策进入客户端正文。
|
||||
|
||||
### Responses
|
||||
|
||||
@@ -286,15 +294,17 @@ command-code \
|
||||
|
||||
随后通过子进程 stdin 写入 UTF-8 的完整请求历史。消息序列被放进一个 JSON 对象,角色、顺序、空消息、Unicode、Markdown、代码块和自定义分隔符均不会被简单文本分隔符破坏。Bridge 不总结、不删除、不截断消息。Chat Completions 不保存会话;Responses 只在 `store: true` 时保存协议对象,并在新请求中把响应链重新序列化给一个新的 `--no-session` 进程。
|
||||
|
||||
## 单并发和错误
|
||||
## 有限队列和错误
|
||||
|
||||
同一时间只允许一个 Command Code 实例。第二个请求直接返回 HTTP 429 和 `code: busy`。请求体超过 `max_request_bytes` 返回 413。未知模型和非文本内容返回 4xx。CLI 的详细错误留在服务终端,客户端只收到简洁的 OpenAI 格式错误。
|
||||
同一时间只运行一个 Command Code 实例,其余生成请求按到达顺序进入等待队列。默认最多等待 8 个请求,队列已满时返回 HTTP 429 和 `code: queue_full`。流式请求进入队列后会立即建立 SSE 连接;非流式请求保持等待。客户端在排队期间断开会立即移出队列。`timeout_seconds` 从请求取得执行位置后开始计算。
|
||||
|
||||
请求体超过 `max_request_bytes` 返回 413。未知模型和非文本内容返回 4xx。CLI 的详细错误留在服务终端,客户端只收到简洁的 OpenAI 格式错误。
|
||||
|
||||
一次 CLI 异常不会结束 HTTP 服务,后续请求仍可继续。
|
||||
|
||||
## 本机实际检查结果
|
||||
|
||||
检查日期:Chat Completions 原有检查为 2026-08-04;Responses 新增检查为 2026-08-05。没有编写测试用例,以下均为构建后运行真实 CLI 和真实 HTTP/SDK 客户端得到的端到端结果。
|
||||
检查日期:Chat Completions 原有检查为 2026-08-04;Responses 新增检查为 2026-08-05;function calling 协议检查为 2026-08-06。没有编写测试用例。原有条目来自真实 CLI 和真实 HTTP/SDK 客户端;有限队列改动完成了类型检查、生产构建和协调器运行时冒烟检查,尚未重新运行真实 CLI 并发检查。function calling 完成了内部协议冒烟检查、Obsidian Copilot 当前依赖 `@langchain/openai 1.2.2` 的双轮 wire compatibility 检查,以及真实 Command Code 与 OpenAI Node.js SDK 的双轮 HTTP/SSE 调用;尚未在 Obsidian UI 中运行完整检查。
|
||||
|
||||
- TypeScript 严格类型检查和生产构建通过。
|
||||
- npm 生产依赖审计:0 个已知漏洞。
|
||||
@@ -309,16 +319,16 @@ command-code \
|
||||
- OpenAI Node.js SDK 示例通过。
|
||||
- OpenAI Python SDK 示例通过。
|
||||
- OpenAI Node.js SDK 的 Chat `stream: true` 检查中,约 31 ms 收到 assistant role 块;最终文本按 10 个原始 delta 返回,随后收到 `stop`、真实 usage 和 `[DONE]`。
|
||||
- LangChain 双轮检查成功把 SSE `delta.tool_calls` 重建为 `AIMessage.tool_calls`,随后发送 assistant tool call 与带 `name` 的 tool result;Bridge Schema 接受实际请求,第二轮文本也被正确重建。
|
||||
- 真实 `deepseek/deepseek-v4-flash` 双轮检查中,第一轮在 `tool_choice: auto` 下返回 `localSearch`,arguments 通过 JSON Schema 校验并以 `finish_reason: tool_calls` 结束;第二轮消费模拟 vault 结果后返回 `Projects/独立决策记录.md` 并以 `stop` 结束。两次累计 usage 为输入 62,850、输出 394、合计 63,244 tokens。
|
||||
- 强制 `read_file` 的 Chat 两 turn 流只向客户端输出第二个无工具 turn 的 6 个文本 delta;终端实时显示工具状态、结果和最终文本,中间轮次没有污染客户端内容。
|
||||
- Chat 流式连接收到首块后断开会取消 CLI 并恢复空闲;流占用期间第二个生成请求返回 429 `busy`,取消后的普通 Chat 请求正常完成。
|
||||
- Obsidian Copilot/LangChain 风格的 `temperature: 0.1`、`max_tokens: 1000`、流式 usage 请求返回 HTTP 200 和正确 SSE;响应头列出两个未映射参数。可选 `top_p`、frequency/presence penalty、`max_completion_tokens` 和 `n: 1` 通过严格校验,`n: 2` 及未知 `tools` 仍返回 400。
|
||||
- 并发检查中第二个请求返回 HTTP 429 和 `code: busy`,没有启动第二个 CLI。
|
||||
- Chat 流式连接收到首块后断开会取消 CLI 并恢复空闲;取消后的普通 Chat 请求正常完成。
|
||||
- Obsidian Copilot/LangChain 风格的 `temperature: 0.1`、`max_tokens: 1000`、流式 usage 请求返回 HTTP 200 和正确 SSE;响应头列出两个未映射参数。可选 `top_p`、frequency/presence penalty、`max_completion_tokens` 和 `n: 1` 通过严格校验,`n: 2` 仍返回 400。
|
||||
- 超过 20 MiB 的请求体返回 HTTP 413,服务保持可用。
|
||||
- 客户端 1 秒超时断开后,当前子进程组被清理,`busy` 恢复为 false,没有发现残留 headless CLI。
|
||||
- 运行中按 Ctrl+C 后客户端收到 HTTP 499,服务保持运行;紧接着的真实请求返回 `中断后恢复成功`。
|
||||
- 原始交互模式的本机 TTY 探测确认 Ink TUI、ANSI、输入框和双 Ctrl+C 退出行为存在;该模式没有独立结构化最终结果通道。
|
||||
- OpenAI Node.js SDK 的 `responses.create()` 返回标准文本 Response;请求级 `reasoning.effort: "max"` 覆盖模型配置并实际传给 CLI。
|
||||
- Responses 运行中第二个生成请求返回 429 `busy`,第一个请求正常完成。
|
||||
- `store: true` 的 Response 可通过 GET 和 input items 查询;`previous_response_id` 成功重建历史并从 `链起点-42` 得到下一轮 `42`。
|
||||
- `json_schema` 合格输出通过 Ajv;不可满足的合法 Schema 顺序执行两次 CLI 后返回 `incomplete` 和 `structured_output_validation_failed`,usage 为两次实际用量之和。
|
||||
- OpenAI Node.js SDK 成功消费 Responses SSE;事件顺序、递增 sequence number、多个 text delta 和最终文本均正确。
|
||||
@@ -326,15 +336,16 @@ command-code \
|
||||
- `store: false` 的流式 Response 随后查询返回 404;删除已保存 Response 返回 deleted,随后查询返回 404。
|
||||
- `input_image` 返回 400 `unsupported_parameter`,并包含准确的参数路径。
|
||||
- 真实 `max_turns: 1` 工具请求返回 200 `incomplete/max_turns`;1 秒总超时返回 200 `incomplete/timeout`;主动取消返回 499 `cancelled`。
|
||||
- 取消后紧接着的旧 Chat Completions 请求返回 `Chat恢复成功`,证明共享单并发状态和错误恢复正常。
|
||||
- 取消后紧接着的旧 Chat Completions 请求返回 `Chat恢复成功`,证明共享执行状态和错误恢复正常。
|
||||
|
||||
## 已知限制
|
||||
|
||||
- 没有原始 Command Code TUI、颜色布局、动画和键盘交互。
|
||||
- headless 无法在服务终端进行批准、拒绝、选项选择或文字回答;`ask_user_question` 不能由等待中的 HTTP 客户端处理。
|
||||
- 终端事件渲染由本项目完成,格式接近日志,无法等同原始 TUI。
|
||||
- Chat Completions 会实时转发 Command Code 文本 delta;上游若在工具 turn 输出文本,已经发送的 SSE 内容无法撤回。Responses 的文本 delta 需要等到 `turn_end.hadToolCalls: false` 才发送,结构化输出还需要等待 Bridge 校验完成。
|
||||
- 只实现 Responses 文本子集,不实现图片、音频、文件、function calling、Computer Use、托管工具、原生 reasoning item、加密 reasoning 或隐藏思维过程。
|
||||
- 普通文本 Chat Completions 会实时转发 Command Code 文本 delta;外部 function calling 和 Responses 结构化输出需要等待 Bridge 校验完成。
|
||||
- Chat Completions 只实现 function tools,不实现图片、音频、文件输入、Computer Use 或托管工具。Responses 仍是文本子集,不实现 function calling、原生 reasoning item、加密 reasoning 或隐藏思维过程。
|
||||
- Chat function calling 是 Bridge 通过提示协议、JSON 解析、工具参数 Schema 校验和一次修复实现的兼容层;Command Code CLI 没有公开原生 function calling 输出接口,因此连续两次输出不合格时返回 HTTP 502 `invalid_tool_decision`。
|
||||
- Responses 的本地文件存储只供本 Bridge 使用,没有跨进程锁、队列或多实例一致性保证。项目本身仍严格单并发。
|
||||
- Responses Structured Outputs 是 Bridge 层约束,底层 Command Code 模型仍可能连续两次输出不合格 JSON;此时状态为 incomplete。
|
||||
- `usage` 使用 Command Code 最终结果提供的真实 input/output token;Responses 缺失时返回 `null`,Chat Completions 为兼容旧行为返回 0。
|
||||
|
||||
@@ -4,6 +4,7 @@ command_code_executable: command-code
|
||||
command_code_working_directory: .
|
||||
timeout_seconds: 1800
|
||||
max_request_bytes: 20971520
|
||||
max_queue_size: 8
|
||||
max_turns: 100
|
||||
stream_thinking: false
|
||||
response_store_directory: .command-code-openai-bridge/responses
|
||||
|
||||
@@ -4,6 +4,7 @@ command_code_executable: command-code
|
||||
command_code_working_directory: .
|
||||
timeout_seconds: 1800
|
||||
max_request_bytes: 20971520
|
||||
max_queue_size: 8
|
||||
max_turns: 100
|
||||
stream_thinking: true
|
||||
response_store_directory: .command-code-openai-bridge/responses
|
||||
|
||||
+46
-1
@@ -2,7 +2,7 @@ import { randomUUID } from "node:crypto";
|
||||
import type { ServerResponse } from "node:http";
|
||||
import type { FastifyReply } from "fastify";
|
||||
import { FinalTurnAccumulator } from "./final-turn.js";
|
||||
import type { CommandUsage } from "./openai.js";
|
||||
import type { ChatCompletionToolCall, CommandUsage } from "./openai.js";
|
||||
|
||||
export class ChatCompletionSseWriter {
|
||||
private readonly id = `chatcmpl-local-${randomUUID().replaceAll("-", "")}`;
|
||||
@@ -78,6 +78,51 @@ export class ChatCompletionSseWriter {
|
||||
this.writeDone();
|
||||
}
|
||||
|
||||
finishToolCalls(
|
||||
toolCalls: ChatCompletionToolCall[],
|
||||
usage?: CommandUsage,
|
||||
includeUsage = false,
|
||||
): void {
|
||||
this.closeThinking();
|
||||
for (const [index, toolCall] of toolCalls.entries()) {
|
||||
this.write({
|
||||
...this.base(),
|
||||
choices: [{
|
||||
index: 0,
|
||||
delta: {
|
||||
tool_calls: [{
|
||||
index,
|
||||
id: toolCall.id,
|
||||
type: toolCall.type,
|
||||
function: toolCall.function,
|
||||
}],
|
||||
},
|
||||
finish_reason: null,
|
||||
}],
|
||||
});
|
||||
}
|
||||
this.write({
|
||||
...this.base(),
|
||||
choices: [{ index: 0, delta: {}, finish_reason: "tool_calls" }],
|
||||
});
|
||||
|
||||
if (includeUsage) {
|
||||
const promptTokens = usage?.inputTokens ?? 0;
|
||||
const completionTokens = usage?.outputTokens ?? 0;
|
||||
this.write({
|
||||
...this.base(),
|
||||
choices: [],
|
||||
usage: {
|
||||
prompt_tokens: promptTokens,
|
||||
completion_tokens: completionTokens,
|
||||
total_tokens: promptTokens + completionTokens,
|
||||
},
|
||||
});
|
||||
}
|
||||
|
||||
this.writeDone();
|
||||
}
|
||||
|
||||
error(error: object): void {
|
||||
this.closeThinking();
|
||||
this.write(error);
|
||||
|
||||
@@ -15,6 +15,7 @@ const configSchema = z.object({
|
||||
command_code_working_directory: z.string().min(1).default("."),
|
||||
timeout_seconds: z.number().int().positive().default(1800),
|
||||
max_request_bytes: z.number().int().positive().default(20 * 1024 * 1024),
|
||||
max_queue_size: z.number().int().min(0).max(1000).default(8),
|
||||
max_turns: z.number().int().positive().default(100),
|
||||
stream_thinking: z.boolean().default(false),
|
||||
response_store_directory: z.string().min(1).default(".command-code-openai-bridge/responses"),
|
||||
|
||||
+221
-9
@@ -6,11 +6,80 @@ const textPartSchema = z.object({
|
||||
text: z.string(),
|
||||
}).strict();
|
||||
|
||||
const messageSchema = z.object({
|
||||
role: z.enum(["system", "user", "assistant"]),
|
||||
content: z.union([z.string(), z.array(textPartSchema)]),
|
||||
const messageContentSchema = z.union([z.string(), z.array(textPartSchema)]);
|
||||
const messageNameSchema = z.string().min(1).max(64);
|
||||
const functionNameSchema = z.string().min(1).max(64).regex(/^[A-Za-z0-9_-]+$/);
|
||||
|
||||
const assistantToolCallSchema = z.object({
|
||||
id: z.string().min(1).max(128),
|
||||
type: z.literal("function"),
|
||||
function: z.object({
|
||||
name: functionNameSchema,
|
||||
arguments: z.string(),
|
||||
}).strict(),
|
||||
}).strict();
|
||||
|
||||
const systemMessageSchema = z.object({
|
||||
role: z.literal("system"),
|
||||
content: messageContentSchema,
|
||||
name: messageNameSchema.optional(),
|
||||
}).strict();
|
||||
|
||||
const developerMessageSchema = z.object({
|
||||
role: z.literal("developer"),
|
||||
content: messageContentSchema,
|
||||
name: messageNameSchema.optional(),
|
||||
}).strict();
|
||||
|
||||
const userMessageSchema = z.object({
|
||||
role: z.literal("user"),
|
||||
content: messageContentSchema,
|
||||
name: messageNameSchema.optional(),
|
||||
}).strict();
|
||||
|
||||
const assistantMessageSchema = z.object({
|
||||
role: z.literal("assistant"),
|
||||
content: messageContentSchema.nullable(),
|
||||
name: messageNameSchema.optional(),
|
||||
tool_calls: z.array(assistantToolCallSchema).min(1).optional(),
|
||||
}).strict();
|
||||
|
||||
const toolMessageSchema = z.object({
|
||||
role: z.literal("tool"),
|
||||
content: messageContentSchema,
|
||||
tool_call_id: z.string().min(1).max(128),
|
||||
name: functionNameSchema.optional(),
|
||||
}).strict();
|
||||
|
||||
const messageSchema = z.discriminatedUnion("role", [
|
||||
systemMessageSchema,
|
||||
developerMessageSchema,
|
||||
userMessageSchema,
|
||||
assistantMessageSchema,
|
||||
toolMessageSchema,
|
||||
]);
|
||||
|
||||
const functionToolSchema = z.object({
|
||||
type: z.literal("function"),
|
||||
function: z.object({
|
||||
name: functionNameSchema,
|
||||
description: z.string().optional(),
|
||||
parameters: z.record(z.unknown()).optional().default({
|
||||
type: "object",
|
||||
properties: {},
|
||||
}),
|
||||
strict: z.boolean().optional(),
|
||||
}).strict(),
|
||||
}).strict();
|
||||
|
||||
const toolChoiceSchema = z.union([
|
||||
z.enum(["none", "auto", "required"]),
|
||||
z.object({
|
||||
type: z.literal("function"),
|
||||
function: z.object({ name: functionNameSchema }).strict(),
|
||||
}).strict(),
|
||||
]);
|
||||
|
||||
export const chatCompletionRequestSchema = z.object({
|
||||
model: z.string().min(1),
|
||||
messages: z.array(messageSchema).min(1),
|
||||
@@ -25,7 +94,41 @@ export const chatCompletionRequestSchema = z.object({
|
||||
frequency_penalty: z.number().min(-2).max(2).nullable().optional(),
|
||||
presence_penalty: z.number().min(-2).max(2).nullable().optional(),
|
||||
n: z.literal(1).nullable().optional(),
|
||||
}).strict();
|
||||
tools: z.array(functionToolSchema).min(1).max(128).optional(),
|
||||
tool_choice: toolChoiceSchema.optional(),
|
||||
parallel_tool_calls: z.boolean().optional(),
|
||||
}).strict().superRefine((request, context) => {
|
||||
const toolNames = new Set<string>();
|
||||
for (const [index, tool] of (request.tools ?? []).entries()) {
|
||||
if (toolNames.has(tool.function.name)) {
|
||||
context.addIssue({
|
||||
code: z.ZodIssueCode.custom,
|
||||
path: ["tools", index, "function", "name"],
|
||||
message: `Duplicate tool name '${tool.function.name}'`,
|
||||
});
|
||||
}
|
||||
toolNames.add(tool.function.name);
|
||||
}
|
||||
|
||||
if (request.tool_choice !== undefined && request.tools === undefined) {
|
||||
context.addIssue({
|
||||
code: z.ZodIssueCode.custom,
|
||||
path: ["tool_choice"],
|
||||
message: "tool_choice requires tools",
|
||||
});
|
||||
}
|
||||
|
||||
if (typeof request.tool_choice === "object") {
|
||||
const name = request.tool_choice.function.name;
|
||||
if (!toolNames.has(name)) {
|
||||
context.addIssue({
|
||||
code: z.ZodIssueCode.custom,
|
||||
path: ["tool_choice", "function", "name"],
|
||||
message: `Unknown forced tool '${name}'`,
|
||||
});
|
||||
}
|
||||
}
|
||||
});
|
||||
|
||||
export type ChatCompletionRequest = z.infer<typeof chatCompletionRequestSchema>;
|
||||
|
||||
@@ -34,6 +137,15 @@ export interface CommandUsage {
|
||||
outputTokens?: number;
|
||||
}
|
||||
|
||||
export interface ChatCompletionToolCall {
|
||||
id: string;
|
||||
type: "function";
|
||||
function: {
|
||||
name: string;
|
||||
arguments: string;
|
||||
};
|
||||
}
|
||||
|
||||
const ignoredCompatibilityParameters = [
|
||||
"temperature",
|
||||
"max_tokens",
|
||||
@@ -48,15 +160,46 @@ export function ignoredChatCompatibilityParameters(request: ChatCompletionReques
|
||||
}
|
||||
|
||||
export function normalizeMessages(request: ChatCompletionRequest) {
|
||||
return request.messages.map((message) => ({
|
||||
role: message.role,
|
||||
content: typeof message.content === "string"
|
||||
return request.messages.map((message) => {
|
||||
const content = message.content === null
|
||||
? null
|
||||
: typeof message.content === "string"
|
||||
? message.content
|
||||
: message.content.map((part) => part.text).join(""),
|
||||
}));
|
||||
: message.content.map((part) => part.text).join("");
|
||||
|
||||
if (message.role === "assistant") {
|
||||
return {
|
||||
role: message.role,
|
||||
content,
|
||||
...(message.name ? { name: message.name } : {}),
|
||||
...(message.tool_calls ? { tool_calls: message.tool_calls } : {}),
|
||||
};
|
||||
}
|
||||
if (message.role === "tool") {
|
||||
return {
|
||||
role: message.role,
|
||||
content,
|
||||
tool_call_id: message.tool_call_id,
|
||||
...(message.name ? { name: message.name } : {}),
|
||||
};
|
||||
}
|
||||
return {
|
||||
role: message.role,
|
||||
content,
|
||||
...(message.name ? { name: message.name } : {}),
|
||||
};
|
||||
});
|
||||
}
|
||||
|
||||
export function usesToolCalling(request: ChatCompletionRequest): boolean {
|
||||
return request.tools !== undefined || request.messages.some((message) => (
|
||||
message.role === "tool" || (message.role === "assistant" && message.tool_calls !== undefined)
|
||||
));
|
||||
}
|
||||
|
||||
export function buildCommandPrompt(request: ChatCompletionRequest): string {
|
||||
if (usesToolCalling(request)) return buildToolCommandPrompt(request);
|
||||
|
||||
const envelope = {
|
||||
protocol: "openai-chat-completions-history-v1",
|
||||
messages: normalizeMessages(request),
|
||||
@@ -74,6 +217,49 @@ export function buildCommandPrompt(request: ChatCompletionRequest): string {
|
||||
].join("\n\n");
|
||||
}
|
||||
|
||||
function buildToolCommandPrompt(request: ChatCompletionRequest): string {
|
||||
const effectiveToolChoice = request.tool_choice ?? (request.tools ? "auto" : "none");
|
||||
const envelope = {
|
||||
protocol: "openai-chat-completions-tools-v1",
|
||||
messages: normalizeMessages(request),
|
||||
tools: request.tools ?? [],
|
||||
tool_choice: effectiveToolChoice,
|
||||
parallel_tool_calls: request.parallel_tool_calls ?? true,
|
||||
};
|
||||
|
||||
return [
|
||||
"下面 JSON 对象是外部客户端提交的一次 OpenAI Chat Completions 工具调用任务。",
|
||||
"你只负责决定当前这一轮应该调用外部工具还是返回最终回答。外部工具由客户端执行,你不能模拟工具结果。",
|
||||
"不要使用 Command Code 自身的文件、终端、搜索、网络或其他工具替代外部 tools;只依据 messages、tools 和工具结果作出决定。",
|
||||
"严格执行 messages 中的 system、developer 和 user 指令,并结合 assistant.tool_calls 与 tool 消息理解完整历史。",
|
||||
"只返回以下两个 JSON 对象之一,禁止 Markdown 代码围栏、前后说明和额外字段:",
|
||||
"最终回答:{\"type\":\"final\",\"content\":\"面向用户的完整回答\"}",
|
||||
"调用工具:{\"type\":\"tool_calls\",\"calls\":[{\"name\":\"工具名称\",\"arguments\":{}}]}",
|
||||
"调用工具时,name 必须与 tools 中的名称完全一致,arguments 必须是符合该工具 parameters JSON Schema 的对象。",
|
||||
"tool_choice 为 none 时必须返回 final;为 required 或指定函数时必须返回 tool_calls。parallel_tool_calls 为 false 时 calls 只能有一项。",
|
||||
"需要工具提供信息或执行动作时返回 tool_calls 并立即结束;已有足够信息时返回 final。不要在工具调用轮次生成面向用户的正文。",
|
||||
"JSON 数据开始:",
|
||||
JSON.stringify(envelope),
|
||||
].join("\n\n");
|
||||
}
|
||||
|
||||
export function buildToolDecisionRepairPrompt(
|
||||
originalPrompt: string,
|
||||
invalidOutput: string,
|
||||
errors: string[],
|
||||
): string {
|
||||
return [
|
||||
"上一次输出不符合外部工具调用传输协议。保持原来的决策意图,只修复 JSON 结构、工具名称或参数。",
|
||||
"只返回原任务要求的 final 或 tool_calls JSON 对象,禁止 Markdown 代码围栏、前后说明和额外字段。",
|
||||
"校验错误:",
|
||||
errors.join("\n"),
|
||||
"上一次输出:",
|
||||
invalidOutput,
|
||||
"原始任务:",
|
||||
originalPrompt,
|
||||
].join("\n\n");
|
||||
}
|
||||
|
||||
export function completionResponse(model: string, content: string, usage?: CommandUsage) {
|
||||
const promptTokens = usage?.inputTokens ?? 0;
|
||||
const completionTokens = usage?.outputTokens ?? 0;
|
||||
@@ -96,6 +282,32 @@ export function completionResponse(model: string, content: string, usage?: Comma
|
||||
};
|
||||
}
|
||||
|
||||
export function toolCallsCompletionResponse(
|
||||
model: string,
|
||||
toolCalls: ChatCompletionToolCall[],
|
||||
usage?: CommandUsage,
|
||||
) {
|
||||
const promptTokens = usage?.inputTokens ?? 0;
|
||||
const completionTokens = usage?.outputTokens ?? 0;
|
||||
|
||||
return {
|
||||
id: `chatcmpl-local-${randomUUID().replaceAll("-", "")}`,
|
||||
object: "chat.completion",
|
||||
created: Math.floor(Date.now() / 1000),
|
||||
model,
|
||||
choices: [{
|
||||
index: 0,
|
||||
message: { role: "assistant", content: null, tool_calls: toolCalls },
|
||||
finish_reason: "tool_calls",
|
||||
}],
|
||||
usage: {
|
||||
prompt_tokens: promptTokens,
|
||||
completion_tokens: completionTokens,
|
||||
total_tokens: promptTokens + completionTokens,
|
||||
},
|
||||
};
|
||||
}
|
||||
|
||||
export function openAIError(
|
||||
message: string,
|
||||
code: string,
|
||||
|
||||
@@ -1,18 +1,73 @@
|
||||
export interface RequestLease {
|
||||
controller: AbortController;
|
||||
ready: Promise<boolean>;
|
||||
}
|
||||
|
||||
interface QueuedRequest {
|
||||
controller: AbortController;
|
||||
resolve: (ready: boolean) => void;
|
||||
}
|
||||
|
||||
export class RequestCoordinator {
|
||||
private activeController: AbortController | undefined;
|
||||
private readonly queue: QueuedRequest[] = [];
|
||||
|
||||
constructor(private readonly maxQueueSize: number) {}
|
||||
|
||||
get busy(): boolean {
|
||||
return this.activeController !== undefined;
|
||||
}
|
||||
|
||||
begin(): AbortController | undefined {
|
||||
if (this.activeController) return undefined;
|
||||
this.activeController = new AbortController();
|
||||
return this.activeController;
|
||||
get queueLength(): number {
|
||||
return this.queue.length;
|
||||
}
|
||||
|
||||
begin(): RequestLease | undefined {
|
||||
const controller = new AbortController();
|
||||
if (!this.activeController) {
|
||||
this.activeController = controller;
|
||||
return { controller, ready: Promise.resolve(true) };
|
||||
}
|
||||
if (this.queue.length >= this.maxQueueSize) return undefined;
|
||||
|
||||
let resolveReady: (ready: boolean) => void = () => undefined;
|
||||
const ready = new Promise<boolean>((resolve) => {
|
||||
resolveReady = resolve;
|
||||
});
|
||||
this.queue.push({ controller, resolve: resolveReady });
|
||||
return { controller, ready };
|
||||
}
|
||||
|
||||
cancel(controller: AbortController, reason = "request cancelled"): boolean {
|
||||
if (this.activeController === controller) {
|
||||
controller.abort(reason);
|
||||
return true;
|
||||
}
|
||||
|
||||
const index = this.queue.findIndex((entry) => entry.controller === controller);
|
||||
if (index === -1) return false;
|
||||
const [entry] = this.queue.splice(index, 1);
|
||||
if (!entry) return false;
|
||||
entry.controller.abort(reason);
|
||||
entry.resolve(false);
|
||||
return true;
|
||||
}
|
||||
|
||||
finish(controller: AbortController): void {
|
||||
if (this.activeController === controller) this.activeController = undefined;
|
||||
if (this.activeController !== controller) return;
|
||||
this.activeController = undefined;
|
||||
|
||||
while (this.queue.length > 0) {
|
||||
const next = this.queue.shift();
|
||||
if (!next) return;
|
||||
if (next.controller.signal.aborted) {
|
||||
next.resolve(false);
|
||||
continue;
|
||||
}
|
||||
this.activeController = next.controller;
|
||||
next.resolve(true);
|
||||
return;
|
||||
}
|
||||
}
|
||||
|
||||
abortActive(reason = "bridge interrupted"): boolean {
|
||||
|
||||
+19
-11
@@ -459,7 +459,6 @@ export async function registerResponseRoutes(
|
||||
});
|
||||
|
||||
app.post("/v1/responses", async (request, reply) => {
|
||||
if (coordinator.busy) return sendBusy(reply);
|
||||
if (!installation.installed) {
|
||||
return reply.status(503).send(openAIError(
|
||||
"Command Code CLI is not installed",
|
||||
@@ -494,12 +493,14 @@ export async function registerResponseRoutes(
|
||||
));
|
||||
}
|
||||
|
||||
const abortController = coordinator.begin();
|
||||
if (!abortController) return sendBusy(reply);
|
||||
const lease = coordinator.begin();
|
||||
if (!lease) return sendBusy(reply);
|
||||
const { controller: abortController } = lease;
|
||||
|
||||
let responseCompleted = false;
|
||||
let acquired = false;
|
||||
const cancelOnDisconnect = () => {
|
||||
if (!responseCompleted) abortController.abort("client disconnected");
|
||||
if (!responseCompleted) coordinator.cancel(abortController, "client disconnected");
|
||||
};
|
||||
request.raw.once("aborted", cancelOnDisconnect);
|
||||
reply.raw.once("close", cancelOnDisconnect);
|
||||
@@ -520,17 +521,23 @@ export async function registerResponseRoutes(
|
||||
|
||||
let writer: ResponsesSseWriter | undefined;
|
||||
try {
|
||||
if (parsed.stream) {
|
||||
writer = startSse(reply, messageId);
|
||||
writer.begin(initialResponse);
|
||||
}
|
||||
if (request.raw.aborted || reply.raw.destroyed) cancelOnDisconnect();
|
||||
acquired = await lease.ready;
|
||||
if (!acquired) {
|
||||
responseCompleted = true;
|
||||
return reply;
|
||||
}
|
||||
|
||||
const chain = parsed.previous_response_id
|
||||
? await store.loadChain(parsed.previous_response_id)
|
||||
: [];
|
||||
const prompt = buildResponsesPrompt(parsed, chain, inputItems);
|
||||
const deadline = Date.now() + config.timeout_seconds * 1000;
|
||||
|
||||
if (parsed.stream) {
|
||||
writer = startSse(reply, messageId);
|
||||
writer.begin(initialResponse);
|
||||
}
|
||||
|
||||
const execution = await executeResponse({
|
||||
config,
|
||||
cliModel: model.cli_model,
|
||||
@@ -609,6 +616,7 @@ export async function registerResponseRoutes(
|
||||
} finally {
|
||||
request.raw.off("aborted", cancelOnDisconnect);
|
||||
reply.raw.off("close", cancelOnDisconnect);
|
||||
if (!acquired) coordinator.cancel(abortController, "request ended before execution");
|
||||
coordinator.finish(abortController);
|
||||
}
|
||||
});
|
||||
@@ -1082,8 +1090,8 @@ function sendRouteError(reply: FastifyReply, error: unknown) {
|
||||
|
||||
function sendBusy(reply: FastifyReply) {
|
||||
return reply.status(429).send(openAIError(
|
||||
"Another Command Code request is already running",
|
||||
"busy",
|
||||
"Command Code request queue is full",
|
||||
"queue_full",
|
||||
"server_error",
|
||||
));
|
||||
}
|
||||
|
||||
+163
-25
@@ -6,17 +6,29 @@ import {
|
||||
CommandCodeError,
|
||||
inspectInstallation,
|
||||
runCommandCode,
|
||||
type CommandResult,
|
||||
type InstallationStatus,
|
||||
} from "./command-code.js";
|
||||
import {
|
||||
buildCommandPrompt,
|
||||
buildToolDecisionRepairPrompt,
|
||||
chatCompletionRequestSchema,
|
||||
completionResponse,
|
||||
ignoredChatCompatibilityParameters,
|
||||
openAIError,
|
||||
toolCallsCompletionResponse,
|
||||
usesToolCalling,
|
||||
type ChatCompletionRequest,
|
||||
type CommandUsage,
|
||||
} from "./openai.js";
|
||||
import { RequestCoordinator } from "./request-coordinator.js";
|
||||
import { registerResponseRoutes } from "./responses.js";
|
||||
import {
|
||||
parseToolDecision,
|
||||
ToolDecisionError,
|
||||
validateToolSchemas,
|
||||
type ToolDecision,
|
||||
} from "./tool-calling.js";
|
||||
|
||||
export interface BridgeServer {
|
||||
app: FastifyInstance;
|
||||
@@ -31,7 +43,7 @@ export async function createServer(config: BridgeConfig): Promise<BridgeServer>
|
||||
bodyLimit: config.max_request_bytes,
|
||||
requestTimeout: 0,
|
||||
});
|
||||
const coordinator = new RequestCoordinator();
|
||||
const coordinator = new RequestCoordinator(config.max_queue_size);
|
||||
|
||||
app.addHook("onRequest", async (request, reply) => {
|
||||
const requestedHeaders = request.headers["access-control-request-headers"];
|
||||
@@ -78,6 +90,8 @@ export async function createServer(config: BridgeConfig): Promise<BridgeServer>
|
||||
status: installation.installed && installation.authenticated ? "ok" : "degraded",
|
||||
command_code: installation,
|
||||
busy: coordinator.busy,
|
||||
queue_length: coordinator.queueLength,
|
||||
queue_capacity: config.max_queue_size,
|
||||
}));
|
||||
|
||||
app.get("/v1/models", async () => ({
|
||||
@@ -91,14 +105,6 @@ export async function createServer(config: BridgeConfig): Promise<BridgeServer>
|
||||
}));
|
||||
|
||||
app.post("/v1/chat/completions", async (request, reply) => {
|
||||
if (coordinator.busy) {
|
||||
return reply.status(429).send(openAIError(
|
||||
"Another Command Code request is already running",
|
||||
"busy",
|
||||
"server_error",
|
||||
));
|
||||
}
|
||||
|
||||
if (!installation.installed) {
|
||||
return reply.status(503).send(openAIError(
|
||||
"Command Code CLI is not installed",
|
||||
@@ -146,6 +152,16 @@ export async function createServer(config: BridgeConfig): Promise<BridgeServer>
|
||||
));
|
||||
}
|
||||
|
||||
const toolSchemaErrors = validateToolSchemas(parsed);
|
||||
if (toolSchemaErrors.length > 0) {
|
||||
return reply.status(400).send(openAIError(
|
||||
toolSchemaErrors.join("; "),
|
||||
"invalid_tool_schema",
|
||||
"invalid_request_error",
|
||||
"tools",
|
||||
));
|
||||
}
|
||||
|
||||
const ignoredParameters = ignoredChatCompatibilityParameters(parsed);
|
||||
if (ignoredParameters.length > 0) {
|
||||
const parameterList = ignoredParameters.join(",");
|
||||
@@ -155,18 +171,20 @@ export async function createServer(config: BridgeConfig): Promise<BridgeServer>
|
||||
);
|
||||
}
|
||||
|
||||
const abortController = coordinator.begin();
|
||||
if (!abortController) {
|
||||
const lease = coordinator.begin();
|
||||
if (!lease) {
|
||||
return reply.status(429).send(openAIError(
|
||||
"Another Command Code request is already running",
|
||||
"busy",
|
||||
"Command Code request queue is full",
|
||||
"queue_full",
|
||||
"server_error",
|
||||
));
|
||||
}
|
||||
const { controller: abortController } = lease;
|
||||
let responseCompleted = false;
|
||||
let acquired = false;
|
||||
|
||||
const cancelOnDisconnect = () => {
|
||||
if (!responseCompleted) abortController.abort("client disconnected");
|
||||
if (!responseCompleted) coordinator.cancel(abortController, "client disconnected");
|
||||
};
|
||||
request.raw.once("aborted", cancelOnDisconnect);
|
||||
reply.raw.once("close", cancelOnDisconnect);
|
||||
@@ -174,24 +192,52 @@ export async function createServer(config: BridgeConfig): Promise<BridgeServer>
|
||||
let writer: ChatCompletionSseWriter | undefined;
|
||||
try {
|
||||
if (parsed.stream) writer = startChatCompletionSse(reply, parsed.model, config.stream_thinking);
|
||||
const result = await runCommandCode(
|
||||
config,
|
||||
model.cli_model,
|
||||
model.effort,
|
||||
buildCommandPrompt(parsed),
|
||||
abortController.signal,
|
||||
writer ? { onEvent: (event) => writer?.commandEvent(event) } : {},
|
||||
);
|
||||
if (request.raw.aborted || reply.raw.destroyed) cancelOnDisconnect();
|
||||
acquired = await lease.ready;
|
||||
if (!acquired) {
|
||||
responseCompleted = true;
|
||||
return reply;
|
||||
}
|
||||
const toolMode = usesToolCalling(parsed);
|
||||
const execution = await executeChatTurn({
|
||||
config,
|
||||
cliModel: model.cli_model,
|
||||
effort: model.effort,
|
||||
request: parsed,
|
||||
signal: abortController.signal,
|
||||
deadline: Date.now() + config.timeout_seconds * 1000,
|
||||
toolMode,
|
||||
...(writer ? { writer } : {}),
|
||||
});
|
||||
const { result, decision } = execution;
|
||||
responseCompleted = true;
|
||||
|
||||
if (decision?.type === "tool_calls") {
|
||||
if (writer) {
|
||||
writer.finish(
|
||||
result.finalText,
|
||||
writer.finishToolCalls(
|
||||
decision.toolCalls,
|
||||
result.usage,
|
||||
parsed.stream_options?.include_usage,
|
||||
);
|
||||
return reply;
|
||||
}
|
||||
return reply.send(completionResponse(parsed.model, result.finalText, result.usage));
|
||||
return reply.send(toolCallsCompletionResponse(
|
||||
parsed.model,
|
||||
decision.toolCalls,
|
||||
result.usage,
|
||||
));
|
||||
}
|
||||
|
||||
const finalText = decision?.type === "final" ? decision.content : result.finalText;
|
||||
if (writer) {
|
||||
writer.finish(
|
||||
finalText,
|
||||
result.usage,
|
||||
parsed.stream_options?.include_usage,
|
||||
);
|
||||
return reply;
|
||||
}
|
||||
return reply.send(completionResponse(parsed.model, finalText, result.usage));
|
||||
} catch (error) {
|
||||
responseCompleted = true;
|
||||
if (abortController.signal.aborted) {
|
||||
@@ -213,6 +259,23 @@ export async function createServer(config: BridgeConfig): Promise<BridgeServer>
|
||||
return;
|
||||
}
|
||||
|
||||
if (error instanceof ToolDecisionError) {
|
||||
process.stderr.write(`\n工具调用决策连续两次校验失败:${error.errors.join("; ")}\n`);
|
||||
if (writer) {
|
||||
writer.error(openAIError(
|
||||
"Command Code returned an invalid tool decision",
|
||||
"invalid_tool_decision",
|
||||
"server_error",
|
||||
));
|
||||
return reply;
|
||||
}
|
||||
return reply.status(502).send(openAIError(
|
||||
"Command Code returned an invalid tool decision",
|
||||
"invalid_tool_decision",
|
||||
"server_error",
|
||||
));
|
||||
}
|
||||
|
||||
const code = error instanceof CommandCodeError ? error.exitCode : undefined;
|
||||
const status = code === 5 ? 429 : code === 10 ? 402 : 502;
|
||||
const errorCode = code === 5
|
||||
@@ -236,6 +299,7 @@ export async function createServer(config: BridgeConfig): Promise<BridgeServer>
|
||||
} finally {
|
||||
request.raw.off("aborted", cancelOnDisconnect);
|
||||
reply.raw.off("close", cancelOnDisconnect);
|
||||
if (!acquired) coordinator.cancel(abortController, "request ended before execution");
|
||||
coordinator.finish(abortController);
|
||||
}
|
||||
});
|
||||
@@ -248,3 +312,77 @@ export async function createServer(config: BridgeConfig): Promise<BridgeServer>
|
||||
abortActive: () => coordinator.abortActive(),
|
||||
};
|
||||
}
|
||||
|
||||
interface ChatExecution {
|
||||
result: CommandResult;
|
||||
decision?: ToolDecision;
|
||||
}
|
||||
|
||||
async function executeChatTurn(options: {
|
||||
config: BridgeConfig;
|
||||
cliModel: string;
|
||||
effort: string;
|
||||
request: ChatCompletionRequest;
|
||||
signal: AbortSignal;
|
||||
deadline: number;
|
||||
toolMode: boolean;
|
||||
writer?: ChatCompletionSseWriter;
|
||||
}): Promise<ChatExecution> {
|
||||
const originalPrompt = buildCommandPrompt(options.request);
|
||||
const first = await runCommandCode(
|
||||
options.config,
|
||||
options.cliModel,
|
||||
options.effort,
|
||||
originalPrompt,
|
||||
options.signal,
|
||||
{
|
||||
timeoutMs: options.deadline - Date.now(),
|
||||
...(!options.toolMode && options.writer
|
||||
? { onEvent: (event: Record<string, unknown>) => options.writer?.commandEvent(event) }
|
||||
: {}),
|
||||
},
|
||||
);
|
||||
if (!options.toolMode) return { result: first };
|
||||
|
||||
try {
|
||||
return {
|
||||
result: first,
|
||||
decision: parseToolDecision(first.finalText, options.request),
|
||||
};
|
||||
} catch (error) {
|
||||
if (!(error instanceof ToolDecisionError)) throw error;
|
||||
process.stderr.write(`\n工具调用决策校验失败,正在修复:${error.errors.join("; ")}\n`);
|
||||
|
||||
const repairPrompt = buildToolDecisionRepairPrompt(
|
||||
originalPrompt,
|
||||
first.finalText,
|
||||
error.errors,
|
||||
);
|
||||
const second = await runCommandCode(
|
||||
options.config,
|
||||
options.cliModel,
|
||||
options.effort,
|
||||
repairPrompt,
|
||||
options.signal,
|
||||
{ timeoutMs: options.deadline - Date.now() },
|
||||
);
|
||||
const usage = addUsage(first.usage, second.usage);
|
||||
const result: CommandResult = {
|
||||
finalText: second.finalText,
|
||||
durationMs: first.durationMs + second.durationMs,
|
||||
...(usage ? { usage } : {}),
|
||||
};
|
||||
return {
|
||||
result,
|
||||
decision: parseToolDecision(second.finalText, options.request),
|
||||
};
|
||||
}
|
||||
}
|
||||
|
||||
function addUsage(left?: CommandUsage, right?: CommandUsage): CommandUsage | undefined {
|
||||
if (!left && !right) return undefined;
|
||||
return {
|
||||
inputTokens: (left?.inputTokens ?? 0) + (right?.inputTokens ?? 0),
|
||||
outputTokens: (left?.outputTokens ?? 0) + (right?.outputTokens ?? 0),
|
||||
};
|
||||
}
|
||||
|
||||
@@ -0,0 +1,198 @@
|
||||
import { randomUUID } from "node:crypto";
|
||||
import { Ajv, type ErrorObject, type ValidateFunction } from "ajv";
|
||||
import type {
|
||||
ChatCompletionRequest,
|
||||
ChatCompletionToolCall,
|
||||
} from "./openai.js";
|
||||
|
||||
export type ToolDecision =
|
||||
| { type: "final"; content: string }
|
||||
| { type: "tool_calls"; toolCalls: ChatCompletionToolCall[] };
|
||||
|
||||
export class ToolDecisionError extends Error {
|
||||
constructor(readonly errors: string[]) {
|
||||
super(errors.join("; "));
|
||||
this.name = "ToolDecisionError";
|
||||
}
|
||||
}
|
||||
|
||||
export function validateToolSchemas(request: ChatCompletionRequest): string[] {
|
||||
const errors: string[] = [];
|
||||
for (const [index, tool] of (request.tools ?? []).entries()) {
|
||||
try {
|
||||
createValidator(tool.function.parameters);
|
||||
} catch (error) {
|
||||
errors.push(`tools.${index}.function.parameters: ${formatUnknownError(error)}`);
|
||||
}
|
||||
}
|
||||
return errors;
|
||||
}
|
||||
|
||||
export function parseToolDecision(
|
||||
output: string,
|
||||
request: ChatCompletionRequest,
|
||||
): ToolDecision {
|
||||
const value = parseJsonValue(output);
|
||||
if (!isRecord(value)) {
|
||||
throw new ToolDecisionError(["Output must be a JSON object"]);
|
||||
}
|
||||
|
||||
if (value.type === "final") return parseFinalDecision(value, request);
|
||||
if (value.type === "tool_calls") return parseCallsDecision(value, request);
|
||||
throw new ToolDecisionError(["type must be 'final' or 'tool_calls'"]);
|
||||
}
|
||||
|
||||
function parseFinalDecision(
|
||||
value: Record<string, unknown>,
|
||||
request: ChatCompletionRequest,
|
||||
): ToolDecision {
|
||||
const errors: string[] = [];
|
||||
const extraKeys = Object.keys(value).filter((key) => key !== "type" && key !== "content");
|
||||
if (extraKeys.length > 0) errors.push(`Unexpected final fields: ${extraKeys.join(", ")}`);
|
||||
if (typeof value.content !== "string") errors.push("final.content must be a string");
|
||||
|
||||
const toolChoice = effectiveToolChoice(request);
|
||||
if (toolChoice === "required" || typeof toolChoice === "object") {
|
||||
errors.push("tool_choice requires a tool call");
|
||||
}
|
||||
if (errors.length > 0) throw new ToolDecisionError(errors);
|
||||
return { type: "final", content: value.content as string };
|
||||
}
|
||||
|
||||
function parseCallsDecision(
|
||||
value: Record<string, unknown>,
|
||||
request: ChatCompletionRequest,
|
||||
): ToolDecision {
|
||||
const errors: string[] = [];
|
||||
const extraKeys = Object.keys(value).filter((key) => key !== "type" && key !== "calls");
|
||||
if (extraKeys.length > 0) errors.push(`Unexpected tool_calls fields: ${extraKeys.join(", ")}`);
|
||||
if (!Array.isArray(value.calls) || value.calls.length === 0) {
|
||||
errors.push("tool_calls.calls must be a non-empty array");
|
||||
throw new ToolDecisionError(errors);
|
||||
}
|
||||
|
||||
const toolChoice = effectiveToolChoice(request);
|
||||
if (toolChoice === "none") errors.push("tool_choice is none");
|
||||
if (request.parallel_tool_calls === false && value.calls.length > 1) {
|
||||
errors.push("parallel_tool_calls is false, so calls may contain only one item");
|
||||
}
|
||||
|
||||
const tools = new Map((request.tools ?? []).map((tool) => [tool.function.name, tool]));
|
||||
const toolCalls: ChatCompletionToolCall[] = [];
|
||||
for (const [index, rawCall] of value.calls.entries()) {
|
||||
if (!isRecord(rawCall)) {
|
||||
errors.push(`calls.${index} must be an object`);
|
||||
continue;
|
||||
}
|
||||
const callExtraKeys = Object.keys(rawCall).filter((key) => key !== "name" && key !== "arguments");
|
||||
if (callExtraKeys.length > 0) {
|
||||
errors.push(`calls.${index} has unexpected fields: ${callExtraKeys.join(", ")}`);
|
||||
}
|
||||
if (typeof rawCall.name !== "string") {
|
||||
errors.push(`calls.${index}.name must be a string`);
|
||||
continue;
|
||||
}
|
||||
|
||||
const tool = tools.get(rawCall.name);
|
||||
if (!tool) {
|
||||
errors.push(`calls.${index}.name references unknown tool '${rawCall.name}'`);
|
||||
continue;
|
||||
}
|
||||
if (typeof toolChoice === "object" && rawCall.name !== toolChoice.function.name) {
|
||||
errors.push(`calls.${index}.name must be forced tool '${toolChoice.function.name}'`);
|
||||
}
|
||||
|
||||
const args = normalizeArguments(rawCall.arguments, index, errors);
|
||||
if (args === undefined) continue;
|
||||
|
||||
let validator: ValidateFunction;
|
||||
try {
|
||||
validator = createValidator(tool.function.parameters);
|
||||
} catch (error) {
|
||||
errors.push(`Invalid schema for tool '${rawCall.name}': ${formatUnknownError(error)}`);
|
||||
continue;
|
||||
}
|
||||
if (!validator(args)) {
|
||||
errors.push(...formatAjvErrors(rawCall.name, validator.errors));
|
||||
continue;
|
||||
}
|
||||
|
||||
toolCalls.push({
|
||||
id: `call_${randomUUID().replaceAll("-", "")}`,
|
||||
type: "function",
|
||||
function: {
|
||||
name: rawCall.name,
|
||||
arguments: JSON.stringify(args),
|
||||
},
|
||||
});
|
||||
}
|
||||
|
||||
if (errors.length > 0) throw new ToolDecisionError(errors);
|
||||
return { type: "tool_calls", toolCalls };
|
||||
}
|
||||
|
||||
function parseJsonValue(output: string): unknown {
|
||||
const trimmed = output.trim();
|
||||
const candidates = [trimmed];
|
||||
const fenced = /^```(?:json)?\s*([\s\S]*?)\s*```$/i.exec(trimmed);
|
||||
if (fenced?.[1]) candidates.push(fenced[1]);
|
||||
const firstBrace = trimmed.indexOf("{");
|
||||
const lastBrace = trimmed.lastIndexOf("}");
|
||||
if (firstBrace >= 0 && lastBrace > firstBrace) {
|
||||
candidates.push(trimmed.slice(firstBrace, lastBrace + 1));
|
||||
}
|
||||
|
||||
let lastError: unknown;
|
||||
for (const candidate of [...new Set(candidates)]) {
|
||||
try {
|
||||
return JSON.parse(candidate);
|
||||
} catch (error) {
|
||||
lastError = error;
|
||||
}
|
||||
}
|
||||
throw new ToolDecisionError([`Invalid JSON: ${formatUnknownError(lastError)}`]);
|
||||
}
|
||||
|
||||
function normalizeArguments(
|
||||
value: unknown,
|
||||
index: number,
|
||||
errors: string[],
|
||||
): Record<string, unknown> | undefined {
|
||||
let parsed = value;
|
||||
if (typeof value === "string") {
|
||||
try {
|
||||
parsed = JSON.parse(value);
|
||||
} catch (error) {
|
||||
errors.push(`calls.${index}.arguments is invalid JSON: ${formatUnknownError(error)}`);
|
||||
return undefined;
|
||||
}
|
||||
}
|
||||
if (!isRecord(parsed)) {
|
||||
errors.push(`calls.${index}.arguments must be an object`);
|
||||
return undefined;
|
||||
}
|
||||
return parsed;
|
||||
}
|
||||
|
||||
function createValidator(schema: Record<string, unknown>): ValidateFunction {
|
||||
return new Ajv({ allErrors: true, strict: false }).compile(schema);
|
||||
}
|
||||
|
||||
function effectiveToolChoice(request: ChatCompletionRequest) {
|
||||
return request.tool_choice ?? (request.tools ? "auto" as const : "none" as const);
|
||||
}
|
||||
|
||||
function formatAjvErrors(toolName: string, errors: ErrorObject[] | null | undefined): string[] {
|
||||
if (!errors || errors.length === 0) return [`Arguments for '${toolName}' do not match its schema`];
|
||||
return errors.map((error) => (
|
||||
`Arguments for '${toolName}'${error.instancePath || "/"}: ${error.message ?? error.keyword}`
|
||||
));
|
||||
}
|
||||
|
||||
function isRecord(value: unknown): value is Record<string, unknown> {
|
||||
return typeof value === "object" && value !== null && !Array.isArray(value);
|
||||
}
|
||||
|
||||
function formatUnknownError(error: unknown): string {
|
||||
return error instanceof Error ? error.message : String(error);
|
||||
}
|
||||
Reference in New Issue
Block a user