支持responses

This commit is contained in:
Sirius
2026-08-05 16:43:18 +08:00
parent 15176a3238
commit 7ec6914872
15 changed files with 1560 additions and 101 deletions
+2 -1
View File
@@ -4,4 +4,5 @@ dist/
.DS_Store
.gitnexus
CLAUDE.md
.claude
.claude
.command-code-openai-bridge/
+1 -1
View File
@@ -90,7 +90,7 @@ curl http://127.0.0.1:18000/v1/chat/completions \
<!-- gitnexus:start -->
# GitNexus — Code Intelligence
This project is indexed by GitNexus as **command-code-openai-bridge** (95 symbols, 190 relationships, 9 execution flows). Use the GitNexus MCP tools to understand code, assess impact, and navigate safely.
This project is indexed by GitNexus as **command-code-openai-bridge** (106 symbols, 201 relationships, 9 execution flows). Use the GitNexus MCP tools to understand code, assess impact, and navigate safely.
> If any GitNexus tool warns the index is stale, run `npx gitnexus analyze` in terminal first.
+114 -9
View File
@@ -1,6 +1,6 @@
# Command Code OpenAI Bridge
这是一个只绑定本机地址的 OpenAI Chat Completions 兼容转发服务。它为每个 API 请求启动一次独立的 Command Code headless agent,将完整消息历史通过 stdin 发送给 CLI,在当前终端显示运行过程,只把官方结构化结果中的 `finalText` 返回给客户端
这是一个只绑定本机地址的 OpenAI 兼容转发服务。它保留 Chat Completions 文本接口,并实现单并发的 Responses API 文本子集。每个生成请求启动一次独立的 Command Code headless agent,将完整消息历史通过 stdin 发送给 CLI,在当前终端显示运行过程,再把结构化事件和最终 `finalText` 转换成 OpenAI 风格的 JSON 或 SSE
默认地址:`http://127.0.0.1:18000/v1`
@@ -49,6 +49,8 @@ command-code-openai-bridge/
│ ├── config.ts
│ ├── index.ts
│ ├── openai.ts
│ ├── request-coordinator.ts
│ ├── responses.ts
│ ├── renderer.ts
│ └── server.ts
├── scripts/
@@ -89,6 +91,7 @@ command_code_working_directory: .
timeout_seconds: 1800
max_request_bytes: 20971520
max_turns: 100
response_store_directory: .command-code-openai-bridge/responses
permission_mode: auto-accept
dangerously_skip_permissions: true
@@ -98,7 +101,7 @@ models:
effort: max
```
`command_code_working_directory` 决定 Command Code 能看到和操作的项目目录。相对路径以 `config.yaml` 所在目录为基准。`models.<name>.effort` 会传给 Command Code 的 `--effort`,可用值取决于对应模型。服务强制只监听 `127.0.0.1`。客户端只能选择配置中的模型名,不能注入额外 CLI 参数。
`command_code_working_directory` 决定 Command Code 能看到和操作的项目目录。`response_store_directory` 保存 `store: true` 的 Response;两个相对路径都以配置文件所在目录为基准。默认存储目录是隐藏目录 `.command-code-openai-bridge/responses``models.<name>.effort` 会传给 Command Code 的 `--effort`Responses 请求中的 `reasoning.effort` 优先。可用 effort 由对应底层模型决定。服务强制只监听 `127.0.0.1`。客户端只能选择配置中的模型名,不能注入额外 CLI 参数。
## 启动、停止与重启
@@ -124,10 +127,97 @@ npm run build
- `GET /health`
- `GET /v1/models`
- `POST /v1/chat/completions`
- `POST /v1/responses`
- `GET /v1/responses/:response_id`
- `DELETE /v1/responses/:response_id`
- `GET /v1/responses/:response_id/input_items`
Bearer Token 会被忽略。只接受文本消息。支持字符串 `content`,也支持由 `{ "type": "text", "text": "..." }` 组成的数组。图片、音频和其他 part 会返回 400
Bearer Token 会被忽略。所有生成接口只接受文本。图片、音频、文件、工具和其他未实现字段会返回 OpenAI 格式的 400,`code``unsupported_parameter`;服务不会静默忽略会改变行为的参数
`stream` 省略或设为 `false` 时返回普通 JSON。`stream: true` 时,服务仍会等待 Command Code 完整执行结束,再将最终文本一次性封装为 Chat Completions SSE 事件返回;它不提供实时生成过程。传入 `stream_options.include_usage: true` 时,结束前还会返回 usage 块。
### Chat Completions
支持字符串 `content`,也支持由 `{ "type": "text", "text": "..." }` 组成的数组。`stream` 省略或设为 `false` 时返回普通 JSON。`stream: true` 会立即建立 SSE 连接并发送 assistant role 块,随后发送最终回答轮次的原始文本 delta、`finish_reason: "stop"`、可选 usage 块和 `[DONE]`。传入 `stream_options.include_usage: true` 时,结束前返回 usage 块。
Chat Completions 和 Responses 共用 Command Code NDJSON 实时事件执行层。服务终端会在事件到达时立即显示状态、文本和工具调用。由于 Command Code 只有在 `turn_end` 才给出 `hadToolCalls`Bridge 会按 turn 缓冲客户端文本,丢弃工具轮次,在确认 `hadToolCalls: false` 后按原 delta 边界写入 Chat SSE。连接会立即建立,最终文本不会混入工具轮次内容,文本首包仍受最终 turn 完成时间约束。
### Responses
`POST /v1/responses` 支持这些字段:
- `model`
- `input`
- `instructions`
- `stream`
- `store`
- `previous_response_id`
- `metadata`
- `reasoning.effort`
- `text.format`
`input` 可以是字符串,也可以是 message item 数组。message 支持 `system``developer``user``assistant` 角色、字符串 content,以及 `input_text``output_text` part。
普通响应包含 `id``object`、时间、状态、错误、instructions、model、标准 assistant message output item、`output_text`、响应链 ID、effective reasoning effort、store、metadata、text format 和 usage。usage 只使用 Command Code 实际提供的 input/output token;没有 usage 时返回 `null`,不生成 token 明细。
`stream: true` 会立即建立 SSE 连接并发送:
1. `response.created`
2. `response.in_progress`
3. `response.output_item.added`
4. `response.content_part.added`
5. 一个或多个 `response.output_text.delta`
6. `response.output_text.done`
7. `response.content_part.done`
8. `response.output_item.done`
9. `response.completed``response.incomplete`
所有事件都包含递增的 `sequence_number`。连接使用 `no-cache, no-transform`,收到客户端断开后会终止 Command Code 子进程组。
Command Code 在 `turn_end` 之前不能保证当前 `text_delta` 属于最终回答,因为同一 turn 随后可能产生工具调用。Bridge 按 turn 缓冲文本,丢弃 `hadToolCalls: true` 的中间轮次,在 `hadToolCalls: false` 时立即按原 delta 边界发送。因此 SSE 连接、状态事件和最终 turn 输出是真实增量事件;文本 delta 会延迟到最终 turn 边界,不能承诺逐 token 的到达时延。结构化输出还会延迟到 JSON 校验完成,避免把随后需要修复的无效 JSON 发给客户端。
### 本地存储和响应链
`store` 默认是 `true`。完成、incomplete、failed 和 cancelled Response 会写入配置的本地目录;`store: false` 不写入当前 Response,因此它不能在后续作为 `previous_response_id` 使用。
收到 `previous_response_id` 时,Bridge 从本地文件沿链向前读取每个 Response,按原顺序重放历史 input 和 assistant output,再追加当前 input。可以从任意仍存在的旧 Response 创建分支。删除某个祖先后,依赖该祖先的链会返回 404。Response ID 使用固定本地格式并在拼接文件路径前校验。
input items 查询返回标准 `{ object: "list", data, first_id, last_id, has_more }`,支持 `after``limit``order`。删除成功返回 `{ id, object: "response", deleted: true }`
### 结构化输出
支持:
```json
{ "text": { "format": { "type": "text" } } }
```
```json
{ "text": { "format": { "type": "json_object" } } }
```
```json
{
"text": {
"format": {
"type": "json_schema",
"name": "result",
"strict": true,
"schema": { "type": "object" }
}
}
}
```
Bridge 把格式要求作为独立内部指令发送给 Command Code。最终文本必须能解析为 JSON;`json_object` 要求顶层对象,`json_schema` 使用 Ajv 校验。第一次失败后会在同一 AbortSignal 和原请求总截止时间内顺序执行一次修复,usage 是两次真实用量之和。第二次仍失败时返回 `status: "incomplete"``incomplete_details.reason: "structured_output_validation_failed"`
这是 Bridge 层的提示、解析、校验和单次修复约束,不具备底层模型原生 Responses Structured Outputs 的解码级保证。
官方协议依据:
- [Chat Completions streaming events](https://developers.openai.com/api/reference/resources/chat/subresources/completions/streaming-events)
- [Responses API reference](https://developers.openai.com/api/reference/resources/responses/methods/create)
- [Responses streaming events](https://developers.openai.com/api/reference/resources/responses/streaming-events)
- [List input items](https://developers.openai.com/api/reference/resources/responses/subresources/input_items/methods/list)
- [Structured Outputs](https://developers.openai.com/api/docs/guides/structured-outputs)
### curl
@@ -178,7 +268,7 @@ command-code \
--yolo
```
随后通过子进程 stdin 写入 UTF-8 的完整请求历史。消息序列被放进一个 JSON 对象,角色、顺序、空消息、Unicode、Markdown、代码块和自定义分隔符均不会被简单文本分隔符破坏。服务不总结、不删除、不截断消息,也不保存会话
随后通过子进程 stdin 写入 UTF-8 的完整请求历史。消息序列被放进一个 JSON 对象,角色、顺序、空消息、Unicode、Markdown、代码块和自定义分隔符均不会被简单文本分隔符破坏。Bridge 不总结、不删除、不截断消息。Chat Completions 不保存会话;Responses 只在 `store: true` 时保存协议对象,并在新请求中把响应链重新序列化给一个新的 `--no-session` 进程
## 单并发和错误
@@ -188,7 +278,7 @@ command-code \
## 本机实际检查结果
检查日期:2026-08-04。没有编写测试用例,以下均为构建后运行真实 CLI 和真实 HTTP 客户端得到的端到端结果。
检查日期:Chat Completions 原有检查为 2026-08-04Responses 新增检查为 2026-08-05。没有编写测试用例,以下均为构建后运行真实 CLI 和真实 HTTP/SDK 客户端得到的端到端结果。
- TypeScript 严格类型检查和生产构建通过。
- npm 生产依赖审计:0 个已知漏洞。
@@ -202,19 +292,34 @@ command-code \
- `--yolo` 下 shell_command 实际执行并返回 `shell-tool-ok``auto-accept` 下该工具确实被 headless 权限引擎拒绝。
- OpenAI Node.js SDK 示例通过。
- OpenAI Python SDK 示例通过。
- `stream: false` 返回普通 JSON`stream: true` 在完整结果生成后一次性返回兼容 SSE
- OpenAI Node.js SDK 的 Chat `stream: true` 检查中,约 31 ms 收到 assistant role 块;最终文本按 10 个原始 delta 返回,随后收到 `stop`、真实 usage 和 `[DONE]`
- 强制 `read_file` 的 Chat 两 turn 流只向客户端输出第二个无工具 turn 的 6 个文本 delta;终端实时显示工具状态、结果和最终文本,中间轮次没有污染客户端内容。
- Chat 流式连接收到首块后断开会取消 CLI 并恢复空闲;流占用期间第二个生成请求返回 429 `busy`,取消后的普通 Chat 请求正常完成。
- 并发检查中第二个请求返回 HTTP 429 和 `code: busy`,没有启动第二个 CLI。
- 超过 20 MiB 的请求体返回 HTTP 413,服务保持可用。
- 客户端 1 秒超时断开后,当前子进程组被清理,`busy` 恢复为 false,没有发现残留 headless CLI。
- 运行中按 Ctrl+C 后客户端收到 HTTP 499,服务保持运行;紧接着的真实请求返回 `中断后恢复成功`
- 原始交互模式的本机 TTY 探测确认 Ink TUI、ANSI、输入框和双 Ctrl+C 退出行为存在;该模式没有独立结构化最终结果通道。
- OpenAI Node.js SDK 的 `responses.create()` 返回标准文本 Response;请求级 `reasoning.effort: "max"` 覆盖模型配置并实际传给 CLI。
- Responses 运行中第二个生成请求返回 429 `busy`,第一个请求正常完成。
- `store: true` 的 Response 可通过 GET 和 input items 查询;`previous_response_id` 成功重建历史并从 `链起点-42` 得到下一轮 `42`
- `json_schema` 合格输出通过 Ajv;不可满足的合法 Schema 顺序执行两次 CLI 后返回 `incomplete``structured_output_validation_failed`usage 为两次实际用量之和。
- OpenAI Node.js SDK 成功消费 Responses SSE;事件顺序、递增 sequence number、多个 text delta 和最终文本均正确。
- 强制 `read_file` 的两 turn 请求只向 SSE 输出第二个无工具 turn 的 6 个文本 delta,中间工具轮次没有污染 `output_text`
- `store: false` 的流式 Response 随后查询返回 404;删除已保存 Response 返回 deleted,随后查询返回 404。
- `input_image` 返回 400 `unsupported_parameter`,并包含准确的参数路径。
- 真实 `max_turns: 1` 工具请求返回 200 `incomplete/max_turns`1 秒总超时返回 200 `incomplete/timeout`;主动取消返回 499 `cancelled`
- 取消后紧接着的旧 Chat Completions 请求返回 `Chat恢复成功`,证明共享单并发状态和错误恢复正常。
## 已知限制
- 没有原始 Command Code TUI、颜色布局、动画和键盘交互。
- headless 无法在服务终端进行批准、拒绝、选项选择或文字回答;`ask_user_question` 不能由等待中的 HTTP 客户端处理。
- 终端事件渲染由本项目完成,格式接近日志,无法等同原始 TUI。
- API 只实现 Chat Completions 文本范围;`stream: true` 是最终结果的 SSE 兼容封装,不是实时生成。不实现 Responses API、图片、音频、function calling 或服务端会话
- `usage` 使用 Command Code 最终结果提供的真实 input/output tokenCLI 未提供时返回 0
- Chat Completions 和 Responses 的文本 delta 都需要等到 `turn_end.hadToolCalls: false` 才发送;SSE 连接和初始事件会立即建立。Responses 结构化输出需要再等 Bridge 校验完成
- 只实现 Responses 文本子集,不实现图片、音频、文件、function calling、Computer Use、托管工具、原生 reasoning item、加密 reasoning 或隐藏思维过程
- Responses 的本地文件存储只供本 Bridge 使用,没有跨进程锁、队列或多实例一致性保证。项目本身仍严格单并发。
- Responses Structured Outputs 是 Bridge 层约束,底层 Command Code 模型仍可能连续两次输出不合格 JSON;此时状态为 incomplete。
- `usage` 使用 Command Code 最终结果提供的真实 input/output tokenResponses 缺失时返回 `null`Chat Completions 为兼容旧行为返回 0。
- 长文本受 HTTP 请求体上限和 Command Code 模型上下文上限共同限制,不会由桥接服务自行截断。
- v1.10.0 的大输入会让 `run_end.nextState` 重复完整提示词。实测约 96 KiB 中文输入时,CLI 退出前可能截断该大事件并丢掉紧随其后的 compact `result` 行。桥接服务会忽略冗余 `run_end`,优先使用 `result.finalText`;若退出码为 0 且 result 缺失,只使用最后一个已完整结束、无工具调用的结构化 turn 文本和真实 turn usage,不从 TUI 文本解析。
+1
View File
@@ -5,6 +5,7 @@ command_code_working_directory: .
timeout_seconds: 1800
max_request_bytes: 20971520
max_turns: 100
response_store_directory: .command-code-openai-bridge/responses
permission_mode: auto-accept
dangerously_skip_permissions: true
+1
View File
@@ -5,6 +5,7 @@ command_code_working_directory: .
timeout_seconds: 1800
max_request_bytes: 20971520
max_turns: 100
response_store_directory: .command-code-openai-bridge/responses
permission_mode: auto-accept
dangerously_skip_permissions: true
+1
View File
@@ -8,6 +8,7 @@
"name": "command-code-openai-bridge",
"version": "1.0.0",
"dependencies": {
"ajv": "^8.20.0",
"fastify": "^5.5.0",
"openai": "^5.19.1",
"yaml": "^2.8.1",
+2 -1
View File
@@ -3,7 +3,7 @@
"version": "1.0.0",
"private": true,
"type": "module",
"description": "Local OpenAI Chat Completions compatible bridge for Command Code CLI",
"description": "Local OpenAI Chat Completions and Responses compatible bridge for Command Code CLI",
"engines": {
"node": ">=22"
},
@@ -14,6 +14,7 @@
"start": "node dist/index.js --config config.yaml"
},
"dependencies": {
"ajv": "^8.20.0",
"fastify": "^5.5.0",
"openai": "^5.19.1",
"yaml": "^2.8.1",
+112
View File
@@ -0,0 +1,112 @@
import { randomUUID } from "node:crypto";
import type { ServerResponse } from "node:http";
import type { FastifyReply } from "fastify";
import { FinalTurnAccumulator } from "./final-turn.js";
import type { CommandUsage } from "./openai.js";
export class ChatCompletionSseWriter {
private readonly id = `chatcmpl-local-${randomUUID().replaceAll("-", "")}`;
private readonly created = Math.floor(Date.now() / 1000);
private readonly accumulator = new FinalTurnAccumulator();
private outputText = "";
constructor(
private readonly response: ServerResponse,
private readonly model: string,
) {}
begin(): void {
this.chunk({ role: "assistant", content: "" }, null);
}
commandEvent(event: Record<string, unknown>): void {
const chunks = this.accumulator.event(event);
if (chunks) this.addText(chunks);
}
finish(finalText: string, usage?: CommandUsage, includeUsage = false): void {
this.ensureFinalText(finalText);
this.chunk({}, "stop");
if (includeUsage) {
const promptTokens = usage?.inputTokens ?? 0;
const completionTokens = usage?.outputTokens ?? 0;
this.write({
...this.base(),
choices: [],
usage: {
prompt_tokens: promptTokens,
completion_tokens: completionTokens,
total_tokens: promptTokens + completionTokens,
},
});
}
this.writeDone();
}
error(error: object): void {
this.write(error);
this.writeDone();
}
private addText(chunks: string[]): void {
for (const delta of chunks) {
if (delta === "") continue;
this.outputText += delta;
this.chunk({ content: delta }, null);
}
}
private ensureFinalText(finalText: string): void {
if (this.outputText === finalText) return;
if (finalText.startsWith(this.outputText)) {
this.addText([finalText.slice(this.outputText.length)]);
return;
}
throw new Error("Command Code finalText does not match the final structured turn text");
}
private chunk(delta: Record<string, unknown>, finishReason: "stop" | null): void {
this.write({
...this.base(),
choices: [{ index: 0, delta, finish_reason: finishReason }],
});
}
private base() {
return {
id: this.id,
object: "chat.completion.chunk",
created: this.created,
model: this.model,
};
}
private write(value: object): void {
if (this.response.destroyed || this.response.writableEnded) return;
this.response.write(`data: ${JSON.stringify(value)}\n\n`);
}
private writeDone(): void {
if (this.response.destroyed || this.response.writableEnded) return;
this.response.end("data: [DONE]\n\n");
}
}
export function startChatCompletionSse(
reply: FastifyReply,
model: string,
): ChatCompletionSseWriter {
reply.hijack();
reply.raw.writeHead(200, {
"Content-Type": "text/event-stream; charset=utf-8",
"Cache-Control": "no-cache, no-transform",
Connection: "keep-alive",
"X-Accel-Buffering": "no",
});
reply.raw.flushHeaders();
const writer = new ChatCompletionSseWriter(reply.raw, model);
writer.begin();
return writer;
}
+35 -5
View File
@@ -19,6 +19,13 @@ export interface CommandResult {
usage?: CommandUsage;
}
export interface CommandRunOptions {
onEvent?: (event: Record<string, unknown>) => void;
timeoutMs?: number;
}
export type CommandFailureKind = "cancelled" | "max_turns" | "timeout" | "command_error";
export interface InstallationStatus {
installed: boolean;
authenticated: boolean;
@@ -29,6 +36,7 @@ export class CommandCodeError extends Error {
constructor(
message: string,
readonly exitCode?: number | null,
readonly kind: CommandFailureKind = "command_error",
) {
super(message);
this.name = "CommandCodeError";
@@ -70,7 +78,13 @@ export async function runCommandCode(
effort: string,
prompt: string,
signal: AbortSignal,
options: CommandRunOptions = {},
): Promise<CommandResult> {
const timeoutMs = options.timeoutMs ?? config.timeout_seconds * 1000;
if (timeoutMs <= 0) {
throw new CommandCodeError("Command Code request timed out", undefined, "timeout");
}
const args = [
"-p",
"--output-format", "json",
@@ -105,6 +119,7 @@ export async function runCommandCode(
}
let settled = false;
let timedOut = false;
let result: ResultFrame | undefined;
let parseError: Error | undefined;
let stderr = "";
@@ -128,9 +143,10 @@ export async function runCommandCode(
};
const timeout = setTimeout(() => {
renderer.fail(`超过 ${config.timeout_seconds} 秒总超时`);
timedOut = true;
renderer.fail(`超过 ${(timeoutMs / 1000).toFixed(1)} 秒总超时`);
cancelEscalation ??= terminateChild(child);
}, config.timeout_seconds * 1000);
}, timeoutMs);
timeout.unref();
signal.addEventListener("abort", onAbort, { once: true });
@@ -180,6 +196,7 @@ export async function runCommandCode(
lastMessageText = undefined;
}
renderer.event(event);
options.onEvent?.(event);
} else if (frame.type === "result") {
result = frame as unknown as ResultFrame;
}
@@ -192,16 +209,25 @@ export async function runCommandCode(
child.once("close", (code, closeSignal) => {
lines.close();
if (timedOut) {
finish(new CommandCodeError("Command Code request timed out", code, "timeout"));
return;
}
if (signal.aborted) {
renderer.fail("请求已取消");
finish(new CommandCodeError("Command Code request cancelled", code));
finish(new CommandCodeError("Command Code request cancelled", code, "cancelled"));
return;
}
if (code !== 0) {
const detail = stderr.trim().split("\n").slice(-3).join(" | ");
renderer.fail(`退出码 ${String(code)}${closeSignal ? `,信号 ${closeSignal}` : ""}${detail ? `${detail}` : ""}`);
finish(new CommandCodeError("Command Code request failed", code));
finish(new CommandCodeError(
"Command Code request failed",
code,
code === 8 ? "max_turns" : "command_error",
));
return;
}
@@ -224,7 +250,11 @@ export async function runCommandCode(
if (result.subtype !== "success") {
renderer.fail(`结果状态 ${result.subtype}`);
finish(new CommandCodeError("Command Code request failed", code));
finish(new CommandCodeError(
"Command Code request failed",
code,
result.subtype === "max_turns" ? "max_turns" : "command_error",
));
return;
}
+3
View File
@@ -16,6 +16,7 @@ const configSchema = z.object({
timeout_seconds: z.number().int().positive().default(1800),
max_request_bytes: z.number().int().positive().default(20 * 1024 * 1024),
max_turns: z.number().int().positive().default(100),
response_store_directory: z.string().min(1).default(".command-code-openai-bridge/responses"),
permission_mode: z.enum(["default", "standard", "plan", "auto-accept", "dont-ask"]).default("auto-accept"),
dangerously_skip_permissions: z.boolean().default(false),
models: z.record(z.string().min(1), modelSchema).refine(
@@ -26,6 +27,7 @@ const configSchema = z.object({
export type BridgeConfig = z.infer<typeof configSchema> & {
configDirectory: string;
resolvedResponseStoreDirectory: string;
resolvedWorkingDirectory: string;
};
@@ -38,6 +40,7 @@ export async function loadConfig(configPath: string): Promise<BridgeConfig> {
return {
...parsed,
configDirectory,
resolvedResponseStoreDirectory: path.resolve(configDirectory, parsed.response_store_directory),
resolvedWorkingDirectory: path.resolve(configDirectory, parsed.command_code_working_directory),
};
}
+43
View File
@@ -0,0 +1,43 @@
export class FinalTurnAccumulator {
private deltas: string[] = [];
private messageText: string | undefined;
finalChunks: string[] | undefined;
event(event: Record<string, unknown>): string[] | undefined {
if (event.type === "turn_start") {
this.deltas = [];
this.messageText = undefined;
return undefined;
}
if (event.type === "text_delta" && typeof event.delta === "string") {
this.deltas.push(event.delta);
return undefined;
}
if (event.type === "message_end") {
this.messageText = extractMessageText(event);
return undefined;
}
if (event.type !== "turn_end" || event.hadToolCalls !== false) return undefined;
const deltaText = this.deltas.join("");
const finalText = this.messageText ?? deltaText;
this.finalChunks = deltaText === finalText && this.deltas.length > 0
? [...this.deltas]
: [finalText];
return this.finalChunks;
}
}
function extractMessageText(event: Record<string, unknown>): string | undefined {
if (!Array.isArray(event.content)) return undefined;
const text = event.content
.filter(isRecord)
.filter((part) => part.type === "text" && typeof part.text === "string")
.map((part) => String(part.text))
.join("");
return text === "" ? undefined : text;
}
function isRecord(value: unknown): value is Record<string, unknown> {
return typeof value === "object" && value !== null && !Array.isArray(value);
}
+9 -57
View File
@@ -17,8 +17,8 @@ export const chatCompletionRequestSchema = z.object({
stream: z.boolean().optional().default(false),
stream_options: z.object({
include_usage: z.boolean().optional().default(false),
}).passthrough().optional(),
}).passthrough();
}).strict().optional(),
}).strict();
export type ChatCompletionRequest = z.infer<typeof chatCompletionRequestSchema>;
@@ -75,59 +75,11 @@ export function completionResponse(model: string, content: string, usage?: Comma
};
}
export function completionStreamResponse(
model: string,
content: string,
usage?: CommandUsage,
includeUsage = false,
): string {
const id = `chatcmpl-local-${randomUUID().replaceAll("-", "")}`;
const created = Math.floor(Date.now() / 1000);
const base = { id, object: "chat.completion.chunk", created, model };
const chunks: object[] = [
{
...base,
choices: [{
index: 0,
delta: { role: "assistant", content: "" },
finish_reason: null,
}],
},
{
...base,
choices: [{
index: 0,
delta: { content },
finish_reason: null,
}],
},
{
...base,
choices: [{
index: 0,
delta: {},
finish_reason: "stop",
}],
},
];
if (includeUsage) {
const promptTokens = usage?.inputTokens ?? 0;
const completionTokens = usage?.outputTokens ?? 0;
chunks.push({
...base,
choices: [],
usage: {
prompt_tokens: promptTokens,
completion_tokens: completionTokens,
total_tokens: promptTokens + completionTokens,
},
});
}
return `${chunks.map((chunk) => `data: ${JSON.stringify(chunk)}\n\n`).join("")}data: [DONE]\n\n`;
}
export function openAIError(message: string, code: string, type = "invalid_request_error") {
return { error: { message, type, code } };
export function openAIError(
message: string,
code: string,
type = "invalid_request_error",
param: string | null = null,
) {
return { error: { message, type, param, code } };
}
+23
View File
@@ -0,0 +1,23 @@
export class RequestCoordinator {
private activeController: AbortController | undefined;
get busy(): boolean {
return this.activeController !== undefined;
}
begin(): AbortController | undefined {
if (this.activeController) return undefined;
this.activeController = new AbortController();
return this.activeController;
}
finish(controller: AbortController): void {
if (this.activeController === controller) this.activeController = undefined;
}
abortActive(reason = "bridge interrupted"): boolean {
if (!this.activeController) return false;
this.activeController.abort(reason);
return true;
}
}
+1156
View File
File diff suppressed because it is too large Load Diff
+57 -27
View File
@@ -1,5 +1,6 @@
import Fastify, { type FastifyInstance } from "fastify";
import { ZodError } from "zod";
import { startChatCompletionSse, type ChatCompletionSseWriter } from "./chat-stream.js";
import type { BridgeConfig } from "./config.js";
import {
CommandCodeError,
@@ -11,13 +12,10 @@ import {
buildCommandPrompt,
chatCompletionRequestSchema,
completionResponse,
completionStreamResponse,
openAIError,
} from "./openai.js";
interface ActiveRequest {
abortController: AbortController;
}
import { RequestCoordinator } from "./request-coordinator.js";
import { registerResponseRoutes } from "./responses.js";
export interface BridgeServer {
app: FastifyInstance;
@@ -32,7 +30,7 @@ export async function createServer(config: BridgeConfig): Promise<BridgeServer>
bodyLimit: config.max_request_bytes,
requestTimeout: 0,
});
let active: ActiveRequest | undefined;
const coordinator = new RequestCoordinator();
app.setErrorHandler((error, _request, reply) => {
const fastifyError = error as Error & { code?: string; statusCode?: number };
@@ -56,7 +54,7 @@ export async function createServer(config: BridgeConfig): Promise<BridgeServer>
app.get("/health", async () => ({
status: installation.installed && installation.authenticated ? "ok" : "degraded",
command_code: installation,
busy: active !== undefined,
busy: coordinator.busy,
}));
app.get("/v1/models", async () => ({
@@ -70,7 +68,7 @@ export async function createServer(config: BridgeConfig): Promise<BridgeServer>
}));
app.post("/v1/chat/completions", async (request, reply) => {
if (active) {
if (coordinator.busy) {
return reply.status(429).send(openAIError(
"Another Command Code request is already running",
"busy",
@@ -98,6 +96,19 @@ export async function createServer(config: BridgeConfig): Promise<BridgeServer>
try {
parsed = chatCompletionRequestSchema.parse(request.body);
} catch (error) {
const unknown = error instanceof ZodError
? error.issues.find((issue) => issue.code === "unrecognized_keys")
: undefined;
if (unknown?.code === "unrecognized_keys") {
const key = unknown.keys[0] ?? "unknown";
const param = [...unknown.path, key].join(".");
return reply.status(400).send(openAIError(
`Unsupported parameter: '${param}'`,
"unsupported_parameter",
"invalid_request_error",
param,
));
}
const message = error instanceof ZodError
? error.issues.map((issue) => `${issue.path.join(".")}: ${issue.message}`).join("; ")
: "Invalid request body";
@@ -112,8 +123,14 @@ export async function createServer(config: BridgeConfig): Promise<BridgeServer>
));
}
const abortController = new AbortController();
active = { abortController };
const abortController = coordinator.begin();
if (!abortController) {
return reply.status(429).send(openAIError(
"Another Command Code request is already running",
"busy",
"server_error",
));
}
let responseCompleted = false;
const cancelOnDisconnect = () => {
@@ -122,31 +139,38 @@ export async function createServer(config: BridgeConfig): Promise<BridgeServer>
request.raw.once("aborted", cancelOnDisconnect);
reply.raw.once("close", cancelOnDisconnect);
let writer: ChatCompletionSseWriter | undefined;
try {
if (parsed.stream) writer = startChatCompletionSse(reply, parsed.model);
const result = await runCommandCode(
config,
model.cli_model,
model.effort,
buildCommandPrompt(parsed),
abortController.signal,
writer ? { onEvent: (event) => writer?.commandEvent(event) } : {},
);
responseCompleted = true;
if (parsed.stream) {
return reply
.type("text/event-stream; charset=utf-8")
.header("Cache-Control", "no-cache")
.header("Connection", "keep-alive")
.send(completionStreamResponse(
parsed.model,
result.finalText,
result.usage,
parsed.stream_options?.include_usage,
));
if (writer) {
writer.finish(
result.finalText,
result.usage,
parsed.stream_options?.include_usage,
);
return reply;
}
return reply.send(completionResponse(parsed.model, result.finalText, result.usage));
} catch (error) {
responseCompleted = true;
if (abortController.signal.aborted) {
if (writer) {
writer.error(openAIError(
"Command Code request cancelled",
"cancelled",
"server_error",
));
return reply;
}
if (!reply.raw.destroyed) {
return reply.status(499).send(openAIError(
"Command Code request cancelled",
@@ -164,6 +188,14 @@ export async function createServer(config: BridgeConfig): Promise<BridgeServer>
: code === 10
? "insufficient_credits"
: "command_code_error";
if (writer) {
writer.error(openAIError(
"Command Code request failed",
errorCode,
"server_error",
));
return reply;
}
return reply.status(status).send(openAIError(
"Command Code request failed",
errorCode,
@@ -172,17 +204,15 @@ export async function createServer(config: BridgeConfig): Promise<BridgeServer>
} finally {
request.raw.off("aborted", cancelOnDisconnect);
reply.raw.off("close", cancelOnDisconnect);
active = undefined;
coordinator.finish(abortController);
}
});
await registerResponseRoutes(app, { config, installation, coordinator });
return {
app,
installation,
abortActive: () => {
if (!active) return false;
active.abortController.abort("bridge interrupted");
return true;
},
abortActive: () => coordinator.abortActive(),
};
}