Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
23 changes: 7 additions & 16 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -21,7 +21,6 @@ The official command-line interface for [ZeroGPU](https://zerogpu.ai) — run fa
- [`chat`](#chat)
- [`chat_thinking`](#chat_thinking)
- [`summarize`](#summarize)
- [`generate_followups`](#generate_followups)
- [Classification](#classification)
- [`classify_iab`](#classify_iab)
- [`classify_iab_enriched`](#classify_iab_enriched)
Expand Down Expand Up @@ -135,11 +134,11 @@ zerogpu chat "Why does my API keep getting rate-limited?" -m gpt-oss-120b -r
# Multilingual reasoning
zerogpu chat "Explique la mise en cache en une phrase." -m qwen3-30b-a3b-fp8

# A 1M-token context, for whole repos and very long documents
# The platform's most capable model, and its priciest
zerogpu chat "$(cat ARCHITECTURE.md)" -m glm-5.2

# Coding and agentic work, at a fraction of the flagship price
zerogpu chat "Port this helper to async/await." -m deepseek-v4-flash
zerogpu chat "Port this helper to async/await." -m deepseek-v4-flash-0731
```

| Option | Description |
Expand All @@ -152,12 +151,12 @@ zerogpu chat "Port this helper to async/await." -m deepseek-v4-flash
|---|---|
| `LFM2.5-1.2B-Instruct` | Default. Fast edge chat. |
| `LFM2.5-1.2B-Thinking` | Compact reasoning model. |
| `gpt-oss-120b` | 117B MoE, 131K context, reasoning + function calling. |
| `qwen3-30b-a3b-fp8` | 30.5B MoE, 100+ languages, reasoning + function calling. |
| `glm-5.2` | 753B MoE, 1M context, reasoning + function calling. The platform's most capable model, and its priciest. |
| `deepseek-v4-flash` | 284B MoE (13B active), 1M context, coding and agentic workflows. |
| `gpt-oss-120b` | 120B MoE, 131K context, reasoning + function calling. |
| `qwen3-30b-a3b-fp8` | 30B MoE, 100+ languages, reasoning + function calling. |
| `glm-5.2` | 753B MoE, 262K context, reasoning + function calling. The platform's most capable model, and its priciest. |
| `deepseek-v4-flash-0731` | 284B MoE (13B active), 1M context, coding and agentic workflows. |

`qwen3-30b-a3b-fp8`, `glm-5.2`, and `deepseek-v4-flash` are served by the Chat Completions API rather than the Responses API; the CLI routes them automatically.
`qwen3-30b-a3b-fp8`, `glm-5.2`, and `deepseek-v4-flash-0731` are served by the Chat Completions API rather than the Responses API; the CLI routes them automatically.

#### `chat_thinking`

Expand All @@ -175,14 +174,6 @@ Summarize text with the **llama-3.1-8b-instruct-fast** model.
zerogpu summarize "Long article text goes here..."
```

#### `generate_followups`

Generate contextual follow-up questions using the ZeroGPU follow-up edge model.

```bash
zerogpu generate_followups "We just shipped a new pricing page focused on enterprise plans."
```

---

### Classification
Expand Down
2 changes: 1 addition & 1 deletion docs/ADDING_COMMANDS.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@ This guide explains how to add a new CLI command to the ZeroGPU CLI.
- `src/commands/` — one file per command, each exporting a `register<Name>Command(program)` function.
- `src/cli.ts` — wires every command into the root program.
- `src/lib/responses.ts` — shared `RESPONSES_ENDPOINT`, `ResponsesApiResponse`, and the `extractOutputText` / `extractReasoningText` helpers for `/v1/responses` calls.
- `src/lib/chatCompletions.ts` — the same for `/v1/chat/completions`, used by models the platform serves only there (currently `qwen3-30b-a3b-fp8`, `glm-5.2`, and `deepseek-v4-flash`), plus `toResponsesUsage` to normalize token counts for savings tracking.
- `src/lib/chatCompletions.ts` — the same for `/v1/chat/completions`, used by models the platform serves only there (currently `qwen3-30b-a3b-fp8`, `glm-5.2`, and `deepseek-v4-flash-0731`), plus `toResponsesUsage` to normalize token counts for savings tracking.
- `src/lib/auth.ts` — `getApiKey()` for authenticated requests.

## Steps
Expand Down
54 changes: 9 additions & 45 deletions docs/DOCUMENTATION.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,15 +5,14 @@
`zerogpu-cli` is the official command-line interface for [ZeroGPU](https://zerogpu.ai), a distributed / edge inference platform for small language models (SLMs) and nano language models. The CLI is a thin, OpenAI-compatible client around the ZeroGPU **Responses API** (`https://api.zerogpu.ai/v1/responses`) — and, for models served only there, the **Chat Completions API** (`https://api.zerogpu.ai/v1/chat/completions`) — that lets you call a curated set of edge-optimized models directly from your terminal for common NLP workloads:

- Conversational chat (`LFM2.5-1.2B-Instruct`, `LFM2.5-1.2B-Thinking`)
- Reasoning and tool-use chat (`gpt-oss-120b`, `qwen3-30b-a3b-fp8`, `glm-5.2`, `deepseek-v4-flash`)
- Reasoning and tool-use chat (`gpt-oss-120b`, `qwen3-30b-a3b-fp8`, `glm-5.2`, `deepseek-v4-flash-0731`)
- IAB content/audience classification (`zlm-v1-iab-classify-edge`, `zlm-v2-iab-classify-edge-enriched`)
- Domain-level IAB classification (`zlm-v1-iab-domain-classifier`)
- Zero-shot classification (`deberta-v3-small`)
- Structured / schema-driven classification and JSON extraction (`gliner2-base-v1`)
- Named-entity recognition with custom labels (`gliner2-base-v1`)
- PII extraction & redaction (`gliner-multi-pii-v1`)
- Text summarization (`llama-3.1-8b-instruct-fast`)
- Follow-up question generation (`zlm-v1-followup-questions-edge`)

It is written in TypeScript (ESM, Node ≥ 20), built on [`commander`](https://github.com/tj/commander.js), and ships a single executable: `zerogpu`.

Expand Down Expand Up @@ -102,8 +101,7 @@ The CLI exposes the following commands:
| [`redact_pii`](#411-redact_pii) | Mask PII in-line in the text |
| [`extract_json`](#412-extract_json) | Schema-driven structured JSON extraction |
| [`summarize`](#413-summarize) | Summarize text with `llama-3.1-8b-instruct-fast` |
| [`generate_followups`](#414-generate_followups) | Generate follow-up questions |
| [`classify_domain`](#415-classify_domain) | Domain-level IAB classification |
| [`classify_domain`](#414-classify_domain) | Domain-level IAB classification |

### Common exit codes
| Code | Meaning |
Expand Down Expand Up @@ -207,12 +205,12 @@ zerogpu chat <text> [-i <instructions>] [-m <model>] [-r]
|---|---|---|
| `LFM2.5-1.2B-Instruct` | Responses | Default. Fast edge chat. |
| `LFM2.5-1.2B-Thinking` | Responses | Compact reasoning model. |
| `gpt-oss-120b` | Responses | 117B MoE, 131K context, reasoning + function calling. |
| `qwen3-30b-a3b-fp8` | Chat Completions | 30.5B MoE, 100+ languages, reasoning + function calling. |
| `glm-5.2` | Chat Completions | 753B MoE, 1,048,576-token context, reasoning + function calling. The most capable model on the platform, and the most expensive by an order of magnitude. |
| `deepseek-v4-flash` | Chat Completions | 284B MoE (13B active), 1,048,576-token context, coding and agentic workflows. |
| `gpt-oss-120b` | Responses | 120B MoE, 131K context, reasoning + function calling. |
| `qwen3-30b-a3b-fp8` | Chat Completions | 30B MoE, 100+ languages, reasoning + function calling. |
| `glm-5.2` | Chat Completions | 753B MoE, 262,144-token context, reasoning + function calling. The most capable model on the platform, and the most expensive by an order of magnitude. |
| `deepseek-v4-flash-0731` | Chat Completions | 284B MoE (13B active), 1,048,576-token context, coding and agentic workflows. |

`qwen3-30b-a3b-fp8`, `glm-5.2`, and `deepseek-v4-flash` have no Responses endpoint, so the CLI posts them to `/v1/chat/completions` instead, mapping `--instructions` to a `system` message and normalizing `prompt_tokens` / `completion_tokens` back to Responses token names for savings tracking. This routing is transparent — the command and its output are identical either way.
`qwen3-30b-a3b-fp8`, `glm-5.2`, and `deepseek-v4-flash-0731` have no Responses endpoint, so the CLI posts them to `/v1/chat/completions` instead, mapping `--instructions` to a `system` message and normalizing `prompt_tokens` / `completion_tokens` back to Responses token names for savings tracking. This routing is transparent — the command and its output are identical either way.

**Example**
```bash
Expand Down Expand Up @@ -599,41 +597,7 @@ A single condensed summary string.

---

### 4.14 `generate_followups`

Generate contextual follow-up questions using `zlm-v1-followup-questions-edge`.

**Synopsis**
```
zerogpu generate_followups <text>
```

**Parameters**

| Name | Type | Required | Description |
|---|---|---|---|
| `text` (positional) | string | yes | Conversation turn / passage to generate follow-ups for. |

**Example**
```bash
zerogpu generate_followups \
"Solar panel adoption increased 35% in the US last year."
```

**Expected output (illustrative)**
```json
[
"Which states drove the largest share of the increase?",
"How does residential adoption compare to commercial?",
"What policy changes contributed to this growth?"
]
```

**Outcomes** — same as the common table.

---

### 4.15 `classify_domain`
### 4.14 `classify_domain`

Classify a domain name against the IAB taxonomy using `zlm-v1-iab-domain-classifier`. It infers the categories that characterize a site as a whole from the hostname alone — no crawl, no page text — which keeps payloads roughly 10x smaller than page-level classification. Use it for bidstream enrichment, allow/deny-list scoring, and inventory-level targeting; when you have the page text and need per-URL precision, use [`classify_iab`](#45-classify_iab) instead.

Expand Down Expand Up @@ -684,7 +648,7 @@ Content-Type: application/json
x-api-key: <ZEROGPU_API_KEY>
```

The exceptions are `summarize`, `chat --model qwen3-30b-a3b-fp8`, `chat --model glm-5.2`, and `chat --model deepseek-v4-flash`, whose models the ZeroGPU platform serves only through the OpenAI-compatible Chat Completions endpoint:
The exceptions are `summarize`, `chat --model qwen3-30b-a3b-fp8`, `chat --model glm-5.2`, and `chat --model deepseek-v4-flash-0731`, whose models the ZeroGPU platform serves only through the OpenAI-compatible Chat Completions endpoint:

```
POST https://api.zerogpu.ai/v1/chat/completions
Expand Down
4 changes: 2 additions & 2 deletions package-lock.json

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

2 changes: 1 addition & 1 deletion package.json
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
{
"name": "zerogpu-cli",
"version": "3.7.2",
"version": "3.8.0",
"description": "Command-line interface for ZeroGPU.",
"type": "module",
"bin": {
Expand Down
2 changes: 0 additions & 2 deletions src/cli.ts
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,6 @@ import { registerCostSavingsCommand } from "./commands/costSavings.js";
import { registerExtractEntitiesCommand } from "./commands/extractEntities.js";
import { registerExtractJsonCommand } from "./commands/extractJson.js";
import { registerExtractPiiCommand } from "./commands/extractPii.js";
import { registerGenerateFollowupsCommand } from "./commands/generateFollowups.js";
import { registerLoginCommand } from "./commands/login.js";
import { registerRedactPiiCommand } from "./commands/redactPii.js";
import { registerStatusCommand } from "./commands/status.js";
Expand Down Expand Up @@ -99,7 +98,6 @@ export function buildProgram(): Command {
registerClassifyDomainCommand(program);
registerClassifyStructuredCommand(program);
registerClassifyZeroShotCommand(program);
registerGenerateFollowupsCommand(program);
registerRedactPiiCommand(program);
registerExtractPiiCommand(program);
registerExtractEntitiesCommand(program);
Expand Down
4 changes: 2 additions & 2 deletions src/commands/chat.ts
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,7 @@ import { recordAndMaybeNotify } from "../lib/savings.js";
const DEFAULT_MODEL = "LFM2.5-1.2B-Instruct";

// Text-generation models `--model` accepts, and the API each one speaks.
// qwen3-30b-a3b-fp8, glm-5.2, and deepseek-v4-flash are Chat Completions only —
// qwen3-30b-a3b-fp8, glm-5.2, and deepseek-v4-flash-0731 are Chat Completions only —
// they have no Responses endpoint.
// Source: https://docs.zerogpu.ai/docs/text-generation
const CHAT_MODELS: Record<string, "responses" | "chat-completions"> = {
Expand All @@ -26,7 +26,7 @@ const CHAT_MODELS: Record<string, "responses" | "chat-completions"> = {
"gpt-oss-120b": "responses",
"qwen3-30b-a3b-fp8": "chat-completions",
"glm-5.2": "chat-completions",
"deepseek-v4-flash": "chat-completions",
"deepseek-v4-flash-0731": "chat-completions",
};

// Model ids are case-sensitive to the API but not to the person typing them.
Expand Down
73 changes: 0 additions & 73 deletions src/commands/generateFollowups.ts

This file was deleted.

10 changes: 4 additions & 6 deletions src/lib/savings.ts
Original file line number Diff line number Diff line change
Expand Up @@ -23,24 +23,22 @@ export const ZGPU_PRICING: Record<string, { in: number; out: number }> = {
"gpt-oss-120b": { in: 0.15, out: 0.6 },
"qwen3-30b-a3b-fp8": { in: 0.05, out: 0.3 },
"glm-5.2": { in: 1.1, out: 3.5 },
"deepseek-v4-flash": { in: 0.07, out: 0.14 },
"deepseek-v4-flash-0731": { in: 0.16, out: 0.38 },
"llama-3.1-8b-instruct-fast": { in: 0.02, out: 0.05 },
"zlm-v1-iab-classify-edge": { in: 0.02, out: 0.05 },
"zlm-v2-iab-classify-edge-enriched": { in: 0.02, out: 0.05 },
"zlm-v1-iab-domain-classifier": { in: 0.02, out: 0.05 },
"zlm-v1-followup-questions-edge": { in: 0.02, out: 0.05 },
"zlm-v1-signal-extract": { in: 0.02, out: 0.05 },
"gliner-multi-pii-v1": { in: 0.02, out: 0.05 },
"gliner2-base-v1": { in: 0.02, out: 0.05 },
"deberta-v3-small": { in: 0.02, out: 0.05 },
"LFM2.5-1.2B-Thinking": { in: 0.02, out: 0.05 },
"LFM2.5-1.2B-Instruct": { in: 0.02, out: 0.05 },
"zlm-v1-moderation-edge": { in: 0.02, out: 0.05 },
"t5-small": { in: 0.05, out: 0.4 },
"zlm-v1-multi-iab-classify": { in: 0.05, out: 0.4 },
// Embedding models bill input tokens only; there are no output tokens to
// charge, so `out: 0` is the real rate, not a placeholder.
"all-minilm-l6-v2": { in: 0.5, out: 0 },
"bge-small-en-v1.5": { in: 0.5, out: 0 },
"all-minilm-l6-v2": { in: 0.004, out: 0 },
"bge-small-en-v1.5": { in: 0.004, out: 0 },
};
// Conservative fallback for any model id not in the table above: the priciest
// published rate, so an unlisted model never overstates savings. Currently
Expand Down
17 changes: 5 additions & 12 deletions tests/savings.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -93,7 +93,7 @@ describe("computeCallSavings", () => {
"gpt-oss-120b",
"qwen3-30b-a3b-fp8",
"glm-5.2",
"deepseek-v4-flash",
"deepseek-v4-flash-0731",
"LFM2.5-1.2B-Instruct",
]) {
expect(unknown).toBeLessThanOrEqual(
Expand All @@ -110,31 +110,24 @@ describe("ZGPU_PRICING tracks the published model catalog", () => {
// below then force src/lib/savings.ts to change with it. Without this guard the
// CLI silently keeps a stale rate and misreports savings — which is exactly how
// gpt-oss-120b sat at $0.03/$0.10 long after it was repriced to $0.15/$0.60.
//
// One deliberate exception: zlm-v1-followup-questions-edge was removed from
// the published catalog when the model was deprecated, but `zerogpu
// generate-followups` still routes to it, so the CLI must still price it.
// Drop both entries together when that command goes.
const CATALOG: Record<string, { in: number; out: number }> = {
"gpt-oss-120b": { in: 0.15, out: 0.6 },
"qwen3-30b-a3b-fp8": { in: 0.05, out: 0.3 },
"glm-5.2": { in: 1.1, out: 3.5 },
"deepseek-v4-flash": { in: 0.07, out: 0.14 },
"deepseek-v4-flash-0731": { in: 0.16, out: 0.38 },
"llama-3.1-8b-instruct-fast": { in: 0.02, out: 0.05 },
"zlm-v2-iab-classify-edge-enriched": { in: 0.02, out: 0.05 },
"zlm-v1-iab-classify-edge": { in: 0.02, out: 0.05 },
"zlm-v1-iab-domain-classifier": { in: 0.02, out: 0.05 },
"zlm-v1-followup-questions-edge": { in: 0.02, out: 0.05 },
"zlm-v1-signal-extract": { in: 0.02, out: 0.05 },
"gliner-multi-pii-v1": { in: 0.02, out: 0.05 },
"gliner2-base-v1": { in: 0.02, out: 0.05 },
"deberta-v3-small": { in: 0.02, out: 0.05 },
"LFM2.5-1.2B-Thinking": { in: 0.02, out: 0.05 },
"LFM2.5-1.2B-Instruct": { in: 0.02, out: 0.05 },
"zlm-v1-moderation-edge": { in: 0.02, out: 0.05 },
"t5-small": { in: 0.05, out: 0.4 },
"zlm-v1-multi-iab-classify": { in: 0.05, out: 0.4 },
"all-minilm-l6-v2": { in: 0.5, out: 0 },
"bge-small-en-v1.5": { in: 0.5, out: 0 },
"all-minilm-l6-v2": { in: 0.004, out: 0 },
"bge-small-en-v1.5": { in: 0.004, out: 0 },
};

it("prices every catalog model at the published rate", () => {
Expand Down