Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
24 changes: 8 additions & 16 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -21,7 +21,6 @@ The official command-line interface for [ZeroGPU](https://zerogpu.ai) — run fa
- [`chat`](#chat)
- [`chat_thinking`](#chat_thinking)
- [`summarize`](#summarize)
- [`generate_followups`](#generate_followups)
- [Classification](#classification)
- [`classify_iab`](#classify_iab)
- [`classify_iab_enriched`](#classify_iab_enriched)
Expand Down Expand Up @@ -140,11 +139,11 @@ zerogpu chat "Why does my API keep getting rate-limited?" -m gpt-oss-120b -r
# Multilingual reasoning
zerogpu chat "Explique la mise en cache en une phrase." -m qwen3-30b-a3b-fp8

# A 1M-token context, for whole repos and very long documents
# A 262K-token context, for whole repos and very long documents
zerogpu chat "$(cat ARCHITECTURE.md)" -m glm-5.2

# Coding and agentic work, at a fraction of the flagship price
zerogpu chat "Port this helper to async/await." -m deepseek-v4-flash
zerogpu chat "Port this helper to async/await." -m deepseek-v4-flash-0731
```

| Option | Description |
Expand All @@ -157,12 +156,13 @@ zerogpu chat "Port this helper to async/await." -m deepseek-v4-flash
|---|---|
| `LFM2.5-1.2B-Instruct` | Default. Fast edge chat. |
| `LFM2.5-1.2B-Thinking` | Compact reasoning model. |
| `gpt-oss-120b` | 117B MoE, 131K context, reasoning + function calling. |
| `qwen3-30b-a3b-fp8` | 30.5B MoE, 100+ languages, reasoning + function calling. |
| `glm-5.2` | 753B MoE, 1M context, reasoning + function calling. The platform's most capable model, and its priciest. |
| `deepseek-v4-flash` | 284B MoE (13B active), 1M context, coding and agentic workflows. |
| `gpt-oss-120b` | 120B MoE, 131K context, reasoning + function calling. |
| `llama-guard-4-12b` | 12B dense, 164K context, brand safety + text moderation. |
| `qwen3-30b-a3b-fp8` | 30B MoE, 100+ languages, reasoning + function calling. |
| `glm-5.2` | 753B MoE, 262K context, reasoning + function calling. The platform's most capable model, and its priciest. |
| `deepseek-v4-flash-0731` | 284B MoE (13B active), 1M context, coding and agentic workflows. |

`qwen3-30b-a3b-fp8`, `glm-5.2`, and `deepseek-v4-flash` are served by the Chat Completions API rather than the Responses API; the CLI routes them automatically.
`qwen3-30b-a3b-fp8`, `glm-5.2`, and `deepseek-v4-flash-0731` are served by the Chat Completions API rather than the Responses API; the CLI routes them automatically.

#### `chat_thinking`

Expand All @@ -180,14 +180,6 @@ Summarize text with the **llama-3.1-8b-instruct-fast** model.
zerogpu summarize "Long article text goes here..."
```

#### `generate_followups`

Generate contextual follow-up questions using the ZeroGPU follow-up edge model.

```bash
zerogpu generate_followups "We just shipped a new pricing page focused on enterprise plans."
```

---

### Classification
Expand Down
2 changes: 1 addition & 1 deletion docs/ADDING_COMMANDS.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@ This guide explains how to add a new CLI command to the ZeroGPU CLI.
- `src/commands/` — one file per command, each exporting a `register<Name>Command(program)` function.
- `src/cli.ts` — wires every command into the root program.
- `src/lib/responses.ts` — shared `RESPONSES_ENDPOINT`, `ResponsesApiResponse`, and the `extractOutputText` / `extractReasoningText` helpers for `/v1/responses` calls.
- `src/lib/chatCompletions.ts` — the same for `/v1/chat/completions`, used by models the platform serves only there (currently `qwen3-30b-a3b-fp8`, `glm-5.2`, and `deepseek-v4-flash`), plus `toResponsesUsage` to normalize token counts for savings tracking.
- `src/lib/chatCompletions.ts` — the same for `/v1/chat/completions`, used by models the platform serves only there (currently `qwen3-30b-a3b-fp8`, `glm-5.2`, and `deepseek-v4-flash-0731`), plus `toResponsesUsage` to normalize token counts for savings tracking.
- `src/lib/auth.ts` — `getApiKey()` for authenticated requests.
- `src/lib/request.ts` — plumbing for the endpoint commands: `requireApiKey`, `resolveInput` (argument or stdin), `parseJsonObject`, `postJson`, and the print helpers.

Expand Down
81 changes: 23 additions & 58 deletions docs/DOCUMENTATION.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,15 +5,14 @@
`zerogpu-cli` is the official command-line interface for [ZeroGPU](https://zerogpu.ai), a distributed / edge inference platform for small language models (SLMs) and nano language models. The CLI is a thin, OpenAI-compatible client around the ZeroGPU **Responses API** (`https://api.zerogpu.ai/v1/responses`) — and, for models served only there, the **Chat Completions API** (`https://api.zerogpu.ai/v1/chat/completions`) — that lets you call a curated set of edge-optimized models directly from your terminal for common NLP workloads:

- Conversational chat (`LFM2.5-1.2B-Instruct`, `LFM2.5-1.2B-Thinking`)
- Reasoning and tool-use chat (`gpt-oss-120b`, `qwen3-30b-a3b-fp8`, `glm-5.2`, `deepseek-v4-flash`)
- Reasoning and tool-use chat (`gpt-oss-120b`, `llama-guard-4-12b`, `qwen3-30b-a3b-fp8`, `glm-5.2`, `deepseek-v4-flash-0731`)
- IAB content/audience classification (`zlm-v1-iab-classify-edge`, `zlm-v2-iab-classify-edge-enriched`)
- Domain-level IAB classification (`zlm-v1-iab-domain-classifier`)
- Zero-shot classification (`deberta-v3-small`)
- Structured / schema-driven classification and JSON extraction (`gliner2-base-v1`)
- Named-entity recognition with custom labels (`gliner2-base-v1`)
- PII extraction & redaction (`gliner-multi-pii-v1`)
- Text summarization (`llama-3.1-8b-instruct-fast`)
- Follow-up question generation (`zlm-v1-followup-questions-edge`)

It is written in TypeScript (ESM, Node ≥ 20), built on [`commander`](https://github.com/tj/commander.js), and ships a single executable: `zerogpu`.

Expand Down Expand Up @@ -102,14 +101,13 @@ The CLI exposes the following commands:
| [`redact_pii`](#411-redact_pii) | Mask PII in-line in the text |
| [`extract_json`](#412-extract_json) | Schema-driven structured JSON extraction |
| [`summarize`](#413-summarize) | Summarize text with `llama-3.1-8b-instruct-fast` |
| [`generate_followups`](#414-generate_followups) | Generate follow-up questions |
| [`classify_domain`](#415-classify_domain) | Domain-level IAB classification |
| [`responses`](#416-responses) | Call `/v1/responses` with any model |
| [`chat_completions`](#417-chat_completions) | Call `/v1/chat/completions` with any model |
| [`moderations`](#418-moderations) | Call `/v1/moderations` with any model |
| [`embeddings`](#419-embeddings) | Call `/v1/embeddings` with any model |
| [`classify_domain`](#414-classify_domain) | Domain-level IAB classification |
| [`responses`](#415-responses) | Call `/v1/responses` with any model |
| [`chat_completions`](#416-chat_completions) | Call `/v1/chat/completions` with any model |
| [`moderations`](#417-moderations) | Call `/v1/moderations` with any model |
| [`embeddings`](#418-embeddings) | Call `/v1/embeddings` with any model |

Commands 4.1–4.15 each wrap one task and pin its model. The endpoint commands, 4.16–4.19, take the model from `--model` and keep no model list, so a model the platform adds or renames works with them without a CLI release.
Commands 4.1–4.14 each wrap one task and pin its model. The endpoint commands, 4.15–4.18, take the model from `--model` and keep no model list, so a model the platform adds or renames works with them without a CLI release.

### Common exit codes
| Code | Meaning |
Expand Down Expand Up @@ -213,12 +211,13 @@ zerogpu chat <text> [-i <instructions>] [-m <model>] [-r]
|---|---|---|
| `LFM2.5-1.2B-Instruct` | Responses | Default. Fast edge chat. |
| `LFM2.5-1.2B-Thinking` | Responses | Compact reasoning model. |
| `gpt-oss-120b` | Responses | 117B MoE, 131K context, reasoning + function calling. |
| `qwen3-30b-a3b-fp8` | Chat Completions | 30.5B MoE, 100+ languages, reasoning + function calling. |
| `glm-5.2` | Chat Completions | 753B MoE, 1,048,576-token context, reasoning + function calling. The most capable model on the platform, and the most expensive by an order of magnitude. |
| `deepseek-v4-flash` | Chat Completions | 284B MoE (13B active), 1,048,576-token context, coding and agentic workflows. |
| `gpt-oss-120b` | Responses | 120B MoE, 131K context, reasoning + function calling. |
| `llama-guard-4-12b` | Responses | 12B dense, 163,840-token context, brand safety + text moderation. |
| `qwen3-30b-a3b-fp8` | Chat Completions | 30B MoE, 100+ languages, reasoning + function calling. |
| `glm-5.2` | Chat Completions | 753B MoE, 262,144-token context, reasoning + function calling. The most capable model on the platform, and the most expensive by an order of magnitude. |
| `deepseek-v4-flash-0731` | Chat Completions | 284B MoE (13B active), 1,048,576-token context, coding and agentic workflows. |

`qwen3-30b-a3b-fp8`, `glm-5.2`, and `deepseek-v4-flash` have no Responses endpoint, so the CLI posts them to `/v1/chat/completions` instead, mapping `--instructions` to a `system` message and normalizing `prompt_tokens` / `completion_tokens` back to Responses token names for savings tracking. This routing is transparent — the command and its output are identical either way.
`qwen3-30b-a3b-fp8`, `glm-5.2`, and `deepseek-v4-flash-0731` have no Responses endpoint, so the CLI posts them to `/v1/chat/completions` instead, mapping `--instructions` to a `system` message and normalizing `prompt_tokens` / `completion_tokens` back to Responses token names for savings tracking. This routing is transparent — the command and its output are identical either way.

**Example**
```bash
Expand Down Expand Up @@ -605,41 +604,7 @@ A single condensed summary string.

---

### 4.14 `generate_followups`

Generate contextual follow-up questions using `zlm-v1-followup-questions-edge`.

**Synopsis**
```
zerogpu generate_followups <text>
```

**Parameters**

| Name | Type | Required | Description |
|---|---|---|---|
| `text` (positional) | string | yes | Conversation turn / passage to generate follow-ups for. |

**Example**
```bash
zerogpu generate_followups \
"Solar panel adoption increased 35% in the US last year."
```

**Expected output (illustrative)**
```json
[
"Which states drove the largest share of the increase?",
"How does residential adoption compare to commercial?",
"What policy changes contributed to this growth?"
]
```

**Outcomes** — same as the common table.

---

### 4.15 `classify_domain`
### 4.14 `classify_domain`

Classify a domain name against the IAB taxonomy using `zlm-v1-iab-domain-classifier`. It infers the categories that characterize a site as a whole from the hostname alone — no crawl, no page text — which keeps payloads roughly 10x smaller than page-level classification. Use it for bidstream enrichment, allow/deny-list scoring, and inventory-level targeting; when you have the page text and need per-URL precision, use [`classify_iab`](#45-classify_iab) instead.

Expand Down Expand Up @@ -680,7 +645,7 @@ zerogpu classify_domain nytimes.com

---

### 4.16 `responses`
### 4.15 `responses`

Call the Responses API with any model. The model id is sent exactly as given — the CLI does not check it against a list, so the API decides whether it exists.

Expand Down Expand Up @@ -734,7 +699,7 @@ Email [PERSON] at [EMAIL] about invoice 12345.

---

### 4.17 `chat_completions`
### 4.16 `chat_completions`

Call the Chat Completions API with any model. Alias: `chat-completions`.

Expand Down Expand Up @@ -777,11 +742,11 @@ zerogpu chat_completions "The application is built with Python 3.11 and uses Pos
}
```

**Outcomes** — as for [`responses`](#416-responses), except a response with no message content exits `1` with `Response did not contain any message content.` + raw JSON dump.
**Outcomes** — as for [`responses`](#415-responses), except a response with no message content exits `1` with `Response did not contain any message content.` + raw JSON dump.

---

### 4.18 `moderations`
### 4.17 `moderations`

Call the Moderations API with any model and print the response.

Expand All @@ -805,11 +770,11 @@ zerogpu moderations "Screen this comment before we publish it." -m zlm-v1-modera

**Expected output** — the moderations response as JSON: `results[].flagged`, `results[].categories`, and `results[].category_scores`.

**Outcomes** — as for [`responses`](#416-responses); there is no content to extract, so a successful response is always printed.
**Outcomes** — as for [`responses`](#415-responses); there is no content to extract, so a successful response is always printed.

---

### 4.19 `embeddings`
### 4.18 `embeddings`

Call the Embeddings API with any model and print the response.

Expand All @@ -833,13 +798,13 @@ zerogpu embeddings "ZeroGPU runs small models at the edge." -m all-minilm-l6-v2

**Expected output** — the embeddings response as JSON: `data[].embedding` holds the vector and `usage` the input tokens.

**Outcomes** — as for [`moderations`](#418-moderations).
**Outcomes** — as for [`moderations`](#417-moderations).

---

## 5. Network & API Contract

The endpoint commands (4.16–4.19) POST to the endpoint they are named for — `/v1/responses`, `/v1/chat/completions`, `/v1/moderations`, `/v1/embeddings` — with the headers below. The rest of this section describes the task commands.
The endpoint commands (4.15–4.18) POST to the endpoint they are named for — `/v1/responses`, `/v1/chat/completions`, `/v1/moderations`, `/v1/embeddings` — with the headers below. The rest of this section describes the task commands.

All inference commands POST to:

Expand All @@ -849,7 +814,7 @@ Content-Type: application/json
x-api-key: <ZEROGPU_API_KEY>
```

The exceptions are `summarize`, `chat --model qwen3-30b-a3b-fp8`, `chat --model glm-5.2`, and `chat --model deepseek-v4-flash`, whose models the ZeroGPU platform serves only through the OpenAI-compatible Chat Completions endpoint:
The exceptions are `summarize`, `chat --model qwen3-30b-a3b-fp8`, `chat --model glm-5.2`, and `chat --model deepseek-v4-flash-0731`, whose models the ZeroGPU platform serves only through the OpenAI-compatible Chat Completions endpoint:

```
POST https://api.zerogpu.ai/v1/chat/completions
Expand Down
4 changes: 2 additions & 2 deletions package-lock.json

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

2 changes: 1 addition & 1 deletion package.json
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
{
"name": "zerogpu-cli",
"version": "3.8.0",
"version": "3.9.0",
"description": "Command-line interface for ZeroGPU.",
"type": "module",
"bin": {
Expand Down
2 changes: 0 additions & 2 deletions src/cli.ts
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,6 @@ import { registerEmbeddingsCommand } from "./commands/embeddings.js";
import { registerExtractEntitiesCommand } from "./commands/extractEntities.js";
import { registerExtractJsonCommand } from "./commands/extractJson.js";
import { registerExtractPiiCommand } from "./commands/extractPii.js";
import { registerGenerateFollowupsCommand } from "./commands/generateFollowups.js";
import { registerLoginCommand } from "./commands/login.js";
import { registerModerationsCommand } from "./commands/moderations.js";
import { registerRedactPiiCommand } from "./commands/redactPii.js";
Expand Down Expand Up @@ -103,7 +102,6 @@ export function buildProgram(): Command {
registerClassifyDomainCommand(program);
registerClassifyStructuredCommand(program);
registerClassifyZeroShotCommand(program);
registerGenerateFollowupsCommand(program);
registerRedactPiiCommand(program);
registerExtractPiiCommand(program);
registerExtractEntitiesCommand(program);
Expand Down
7 changes: 4 additions & 3 deletions src/commands/chat.ts
Original file line number Diff line number Diff line change
Expand Up @@ -17,16 +17,17 @@ import { recordAndMaybeNotify } from "../lib/savings.js";
const DEFAULT_MODEL = "LFM2.5-1.2B-Instruct";

// Text-generation models `--model` accepts, and the API each one speaks.
// qwen3-30b-a3b-fp8, glm-5.2, and deepseek-v4-flash are Chat Completions only —
// they have no Responses endpoint.
// qwen3-30b-a3b-fp8, glm-5.2, and deepseek-v4-flash-0731 are Chat Completions
// only — they have no Responses endpoint.
// Source: https://docs.zerogpu.ai/docs/text-generation
const CHAT_MODELS: Record<string, "responses" | "chat-completions"> = {
"LFM2.5-1.2B-Instruct": "responses",
"LFM2.5-1.2B-Thinking": "responses",
"gpt-oss-120b": "responses",
"llama-guard-4-12b": "responses",
"qwen3-30b-a3b-fp8": "chat-completions",
"glm-5.2": "chat-completions",
"deepseek-v4-flash": "chat-completions",
"deepseek-v4-flash-0731": "chat-completions",
};

// Model ids are case-sensitive to the API but not to the person typing them.
Expand Down
73 changes: 0 additions & 73 deletions src/commands/generateFollowups.ts

This file was deleted.

Loading