From 656823e6c9753d8a1c8f44e36354bc02812fc24a Mon Sep 17 00:00:00 2001 From: Szymon Kurcab Date: Fri, 18 Sep 2026 20:36:59 +0200 Subject: [PATCH] feat: list the models a provider accepts Typesafe documents GET /v1/models, and both official SDKs expose it. This client had no way to reach it, so the only way to find out which names the `model` field takes was to read the docs and hardcode one -- and an alias moves when a release ships, which is the thing a list is for. Client#models returns ModelCards: name, description, release_date. A provider says whether it has a list at all by declaring models_path, and OpenRouter does not -- its /models is the whole gateway catalogue under a different base than the alpha decisions endpoint, so it raises rather than composing a URL that means something else. This is the first request that is not a POST, so the transport contract grows an optional `method:` keyword alongside the existing three. A transport that does not declare it keeps being called exactly as before, and asking it to list models raises ConfigurationError rather than quietly POSTing to the list endpoint. Listing goes through the same retry loop and the same error classes as asking does. Co-Authored-By: Claude Opus 5 (1M context) --- CHANGELOG.md | 13 ++ README.md | 27 +++- lib/ruby_decision_model.rb | 1 + lib/ruby_decision_model/client.rb | 89 ++++++++-- lib/ruby_decision_model/model_card.rb | 8 + lib/ruby_decision_model/providers/base.rb | 14 ++ lib/ruby_decision_model/providers/typesafe.rb | 5 + test/models_test.rb | 152 ++++++++++++++++++ 8 files changed, 294 insertions(+), 15 deletions(-) create mode 100644 lib/ruby_decision_model/model_card.rb create mode 100644 test/models_test.rb diff --git a/CHANGELOG.md b/CHANGELOG.md index 846df76..bc76884 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -1,5 +1,18 @@ # Changelog +## Unreleased + +- Added `Client#models`, which lists the model names an account can send in + `model` as `ModelCard`s (`name`, `description`, `release_date`). Typesafe + publishes this at `GET /v1/models` and both official SDKs expose it; this + client had no way to reach it. +- Providers declare a `models_path`; `lists_models?` and `models_url` follow + from it. OpenRouter publishes no decision-model list, so `client.models` + raises `ConfigurationError` there rather than guessing a URL. +- The transport contract accepts an optional `method:` keyword (`:post` for + `ask`, `:get` for `models`). Transports that do not declare it are called + unchanged and cannot be asked for a GET. + ## 0.1.0 - 2026-09-18 Provider-neutral release. One `Client`, two providers behind it. diff --git a/README.md b/README.md index 79638d6..93d1099 100644 --- a/README.md +++ b/README.md @@ -68,6 +68,22 @@ Typesafe returns an `x-typesafe-request-id` header, exposed as `response.request_id` (nil on OpenRouter). Quote it when reporting a problem to Typesafe. +### Listing models + +```ruby +client = RubyDecisionModel::Client.new(provider: :typesafe) + +client.models +# => [#, ...] +``` + +`GET /v1/models` returns the names this account can send in `model`. The list +carries the aliases; a versioned id such as `jev-1.13.0` is accepted by `model` +whether or not it appears. OpenRouter publishes no equivalent list for +decisions, so `client.models` raises `ConfigurationError` there. + ### Options ```ruby @@ -104,9 +120,9 @@ is whatever the provider returned. ### Writing a provider Subclass `RubyDecisionModel::Providers::Base` and define `name`, `env_var`, -`default_base_url`, `endpoint_path`, `default_model`, and optionally `aliases` -and `reports_cost?`. Override `headers`, `request_body`, or `usage` when the -wire format differs. Pass an instance as `provider:`. +`default_base_url`, `endpoint_path`, `default_model`, and optionally `aliases`, +`reports_cost?`, and `models_path`. Override `headers`, `request_body`, or +`usage` when the wire format differs. Pass an instance as `provider:`. ## Questions and answers @@ -167,6 +183,11 @@ that accepts `url:`, `headers:`, `body:` and returns return is still accepted and treated as having no headers, which means no `Retry-After` support and a nil `request_id`. +A transport that also declares a `method:` keyword (or `**`) is told which verb +to use — `:post` for `ask`, `:get` for `models`. One that does not is called +exactly as before and always POSTs, so `client.models` raises +`ConfigurationError` rather than posting to the model list. + ## Errors | Error | Meaning | diff --git a/lib/ruby_decision_model.rb b/lib/ruby_decision_model.rb index fb00614..cedd739 100644 --- a/lib/ruby_decision_model.rb +++ b/lib/ruby_decision_model.rb @@ -7,6 +7,7 @@ require_relative "ruby_decision_model/errors" require_relative "ruby_decision_model/questions" require_relative "ruby_decision_model/answers" +require_relative "ruby_decision_model/model_card" require_relative "ruby_decision_model/response" require_relative "ruby_decision_model/providers" require_relative "ruby_decision_model/retry_policy" diff --git a/lib/ruby_decision_model/client.rb b/lib/ruby_decision_model/client.rb index 3457726..473e7a1 100644 --- a/lib/ruby_decision_model/client.rb +++ b/lib/ruby_decision_model/client.rb @@ -63,6 +63,24 @@ def ask(state:, questions:) handle_response(status, response_body, response_headers, questions) end + # The names this account may send in `model`, as ModelCards. Providers + # that do not publish a list raise ConfigurationError. + def models + unless @provider.lists_models? + raise ConfigurationError, "#{@provider.name} does not publish a model list" + end + + unless transport_accepts_method? + raise ConfigurationError, + "listing models needs a GET, and this transport takes no method: keyword" + end + + status, body, headers = perform_with_retry( + url: @provider.models_url, headers: @provider.headers, body: nil, method: :get + ) + handle_models_response(status, body, headers) + end + private def resolve_provider(provider, api_key:, base_url:) @@ -88,24 +106,69 @@ def resolve_provider(provider, api_key:, base_url:) end end + REQUEST_CLASSES = { post: Net::HTTP::Post, get: Net::HTTP::Get }.freeze + def default_transport - lambda do |url:, headers:, body:| + lambda do |url:, headers:, body:, method: :post| uri = URI.parse(url) http = Net::HTTP.new(uri.host, uri.port) http.use_ssl = uri.scheme == "https" http.open_timeout = @timeout http.read_timeout = @timeout - request = Net::HTTP::Post.new(uri.request_uri) + request = REQUEST_CLASSES.fetch(method).new(uri.request_uri) headers.each { |k, v| request[k] = v } - request.body = body + request.body = body unless body.nil? response = http.request(request) [response.code.to_i, response.body, response.each_header.to_h] end end - def perform_with_retry(url:, headers:, body:) + # The transport contract grew a `method:` keyword for the one endpoint + # that is not a POST. A transport written against the old contract is + # called the way it always was, and cannot be asked for a GET. + def transport_accepts_method? + return @transport_accepts_method unless @transport_accepts_method.nil? + + parameters = @transport.respond_to?(:parameters) ? @transport.parameters : @transport.method(:call).parameters + @transport_accepts_method = parameters.any? do |kind, name| + kind == :keyrest || (%i[key keyreq].include?(kind) && name == :method) + end + end + + def call_transport(url:, headers:, body:, method:) + if transport_accepts_method? + @transport.call(url: url, headers: headers, body: body, method: method) + else + @transport.call(url: url, headers: headers, body: body) + end + end + + def handle_models_response(status, response_body, response_headers) + unless (200..299).cover?(status) + return handle_response(status, response_body, response_headers, {}) + end + + parsed = parse_json(response_body) + raise InvalidResponse, "model list was not a JSON object" unless parsed.is_a?(Hash) + + listed = parsed["models"] + raise InvalidResponse, "model list had no models array" unless listed.is_a?(Array) + + listed.map { |entry| model_card(entry) } + end + + def model_card(entry) + raise InvalidResponse, "model list entry was not an object" unless entry.is_a?(Hash) + + name = entry["name"] + raise InvalidResponse, "model list entry had no name" unless name.is_a?(String) && !name.empty? + + ModelCard.new(name: name, description: entry["description"], release_date: entry["release_date"]) + end + + def perform_with_retry(url:, headers:, body:, method: :post) policy = @retry_policy started_at = @clock.call retries = 0 @@ -113,7 +176,7 @@ def perform_with_retry(url:, headers:, body:) loop do begin status, response_body, response_headers = normalize_transport_result( - @transport.call(url: url, headers: headers, body: body) + call_transport(url: url, headers: headers, body: body, method: method) ) rescue Error raise @@ -184,13 +247,7 @@ def handle_response(status, response_body, response_headers, questions) end def parse_success(response_body, response_headers, questions) - raise InvalidResponse, "response body was empty" if response_body.nil? || response_body.to_s.strip.empty? - - parsed = begin - JSON.parse(response_body.to_s) - rescue JSON::ParserError => e - raise InvalidResponse, "response body was not valid JSON: #{e.message}" - end + parsed = parse_json(response_body) raise InvalidResponse, "response body was not a JSON object" unless parsed.is_a?(Hash) @@ -242,6 +299,14 @@ def parse_success(response_body, response_headers, questions) ) end + def parse_json(response_body) + raise InvalidResponse, "response body was empty" if response_body.nil? || response_body.to_s.strip.empty? + + JSON.parse(response_body.to_s) + rescue JSON::ParserError => e + raise InvalidResponse, "response body was not valid JSON: #{e.message}" + end + def request_id_from(headers) return nil unless headers.is_a?(Hash) diff --git a/lib/ruby_decision_model/model_card.rb b/lib/ruby_decision_model/model_card.rb new file mode 100644 index 0000000..a6d558a --- /dev/null +++ b/lib/ruby_decision_model/model_card.rb @@ -0,0 +1,8 @@ +# frozen_string_literal: true + +module RubyDecisionModel + # One entry from a provider's model list: a name the `model` field accepts, + # what it is for, and when it shipped. An alias such as "jev-latest" is + # listed the same way a versioned id is. + ModelCard = Data.define(:name, :description, :release_date) +end diff --git a/lib/ruby_decision_model/providers/base.rb b/lib/ruby_decision_model/providers/base.rb index 85f7089..ab564ea 100644 --- a/lib/ruby_decision_model/providers/base.rb +++ b/lib/ruby_decision_model/providers/base.rb @@ -55,6 +55,20 @@ def url "#{base_url}#{endpoint_path}" end + # Where this provider lists the models its `model` field accepts, or + # nil when it does not offer a list. + def models_path + nil + end + + def lists_models? + !models_path.nil? + end + + def models_url + "#{base_url}#{models_path}" if lists_models? + end + def api_key? !(api_key.nil? || api_key.to_s.strip.empty?) end diff --git a/lib/ruby_decision_model/providers/typesafe.rb b/lib/ruby_decision_model/providers/typesafe.rb index c8a1ee7..521b2f1 100644 --- a/lib/ruby_decision_model/providers/typesafe.rb +++ b/lib/ruby_decision_model/providers/typesafe.rb @@ -26,6 +26,11 @@ def endpoint_path "/v1/systemone" end + # https://docs.typesafe.ai/models#listing-models + def models_path + "/v1/models" + end + def default_model "jev-latest" end diff --git a/test/models_test.rb b/test/models_test.rb new file mode 100644 index 0000000..85647a0 --- /dev/null +++ b/test/models_test.rb @@ -0,0 +1,152 @@ +# frozen_string_literal: true + +require "test_helper" + +# A transport that records the method it was asked for. +class MethodRecordingTransport + attr_reader :calls + + def initialize(responses) + @responses = responses + @calls = [] + end + + def call(url:, headers:, body:, method: :post) + @calls << { url: url, headers: headers, body: body, method: method } + response = @responses.length > 1 ? @responses.shift : @responses.first + raise response if response.is_a?(Exception) + + response + end +end + +class ModelsTest < Minitest::Test + def model_list + JSON.generate( + "models" => [ + { "name" => "jev-latest", "description" => "Most recent stable release", "release_date" => "2026-02-11" }, + { "name" => "jev-preview", "description" => "Most recent release", "release_date" => "2026-02-11" } + ] + ) + end + + def typesafe_client(transport) + RubyDecisionModel::Client.new(provider: :typesafe, api_key: "ts-key", transport: transport, sleeper: no_sleep, + retry: { max_retries: 0 }) + end + + def test_models_returns_typed_cards + client = typesafe_client(MethodRecordingTransport.new([[200, model_list]])) + + models = client.models + + assert_equal 2, models.size + assert_equal "jev-latest", models.first.name + assert_equal "Most recent stable release", models.first.description + assert_equal "2026-02-11", models.first.release_date + assert_instance_of RubyDecisionModel::ModelCard, models.first + end + + def test_models_gets_the_documented_endpoint + transport = MethodRecordingTransport.new([[200, model_list]]) + typesafe_client(transport).models + + call = transport.calls.first + assert_equal "https://api.typesafe.ai/v1/models", call[:url] + assert_equal :get, call[:method] + assert_nil call[:body] + assert_equal "Bearer ts-key", call[:headers]["Authorization"] + end + + def test_asking_still_posts + transport = MethodRecordingTransport.new([[200, JSON.generate( + "answers" => { "q" => { "type" => "noul", "noul" => 0.5 } }, "usage" => {} + )]]) + typesafe_client(transport).ask(state: {}, questions: { "q" => RubyDecisionModel::Questions.noul("Urgent?") }) + + assert_equal :post, transport.calls.first[:method] + end + + def test_a_provider_without_a_model_list_says_so + client = RubyDecisionModel::Client.new(provider: :open_router, api_key: "or-key", + transport: MethodRecordingTransport.new([])) + + error = assert_raises(RubyDecisionModel::ConfigurationError) { client.models } + assert_match(/does not publish a model list/, error.message) + end + + def test_a_transport_that_cannot_get_says_so + # The old three-keyword contract has no way to ask for anything but a POST. + client = typesafe_client(FakeTransport.new([[200, model_list]])) + + error = assert_raises(RubyDecisionModel::ConfigurationError) { client.models } + assert_match(/no method: keyword/, error.message) + end + + def test_errors_use_the_same_classes_as_asking + client = typesafe_client(MethodRecordingTransport.new([[401, "nope", {}]])) + + assert_raises(RubyDecisionModel::Unauthorized) { client.models } + end + + def test_a_model_list_that_is_not_an_object_is_rejected + client = typesafe_client(MethodRecordingTransport.new([[200, "[]"]])) + + error = assert_raises(RubyDecisionModel::InvalidResponse) { client.models } + assert_match(/not a JSON object/, error.message) + end + + def test_a_missing_models_array_is_rejected + client = typesafe_client(MethodRecordingTransport.new([[200, "{}"]])) + + error = assert_raises(RubyDecisionModel::InvalidResponse) { client.models } + assert_match(/no models array/, error.message) + end + + def test_an_entry_without_a_name_is_rejected + body = JSON.generate("models" => [{ "description" => "nameless" }]) + client = typesafe_client(MethodRecordingTransport.new([[200, body]])) + + error = assert_raises(RubyDecisionModel::InvalidResponse) { client.models } + assert_match(/no name/, error.message) + end + + def test_an_entry_may_omit_description_and_release_date + body = JSON.generate("models" => [{ "name" => "jev-1.13.0" }]) + client = typesafe_client(MethodRecordingTransport.new([[200, body]])) + + card = client.models.first + assert_equal "jev-1.13.0", card.name + assert_nil card.description + assert_nil card.release_date + end + + def test_an_empty_list_is_an_empty_array + client = typesafe_client(MethodRecordingTransport.new([[200, JSON.generate("models" => [])]])) + + assert_empty client.models + end + + def test_listing_is_retried_like_asking + transport = MethodRecordingTransport.new([[503, "{}"], [200, model_list]]) + client = RubyDecisionModel::Client.new(provider: :typesafe, api_key: "k", transport: transport, + sleeper: no_sleep, random: -> { 0.0 }) + + assert_equal 2, client.models.size + assert_equal 2, transport.calls.length + assert_equal [:get, :get], transport.calls.map { |c| c[:method] } + end + + # --- the provider's own view of it --- + + def test_provider_reports_whether_it_lists_models + assert_predicate RubyDecisionModel::Providers::Typesafe.new, :lists_models? + refute_predicate RubyDecisionModel::Providers::OpenRouter.new, :lists_models? + assert_nil RubyDecisionModel::Providers::OpenRouter.new.models_url + end + + def test_models_url_follows_a_base_url_override + provider = RubyDecisionModel::Providers::Typesafe.new(base_url: "https://staging.typesafe.example") + assert_equal "https://staging.typesafe.example/v1/models", provider.models_url + end +end