Feat/configurable timeout - #76
Open
geoffhancock wants to merge 3 commits into
Open
geoffhancock wants to merge 3 commits into
geoffhancock wants to merge 3 commits into
Conversation
…ical_csv get_historical_csv passed include_imputed_marker as the sixth positional argument to get_historical_pandas, whose sixth parameter is include_meta. As a result, get_historical_csv(..., include_imputed_marker=True) wrote a CSV containing a meta column and no imputed_data_used column. Pass all defaulted parameters by keyword in the internal call chain (get_historical_csv -> get_historical_pandas -> get_historical_jsons) so argument order differences between signatures cannot misroute a flag. Public signatures are unchanged. Adds a regression test asserting the CSV contains imputed_data_used and not meta when include_imputed_marker=True. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…eout Add a keyword-only `timeout` to WattTimeBase.__init__ (default (10, 60), matching the existing login/register literals) and pass it at all three HTTP call sites. The data request previously used a scalar 60, so its default connect timeout drops from 60 s to 10 s; default read timeouts are unchanged. Add a keyword-only `chunk_size` to get_historical_jsons/_pandas/_csv, forwarded to _get_chunks, which now accepts None (30 days) and rejects sizes of 5 minutes or less -- the per-chunk trim -- instead of looping forever or producing inverted chunks. When a request fails on a timeout, append a hint to the RuntimeError naming both knobs. Once the session's retries are exhausted, requests raises ConnectionError -> MaxRetryError -> ReadTimeoutError rather than ReadTimeout, so _is_timeout walks the exception chain. Motivation: in August 2026, 30-day historical pulls for some regions took over 180 s server-side while the same pulls at 10 days took ~11 s. Users hit an opaque read timeout after ~4 minutes with no supported lever; the only workaround was monkeypatching _get_chunks. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Add a short "Tuning requests that time out" note under the historical data example showing both options and that the timeout is per attempt. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
This was referenced Sep 18, 2026
geoffhancock
marked this pull request as ready for review
September 18, 2026 19:12
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Expose request timeout and historical chunk size, and point timeout errors at both
Why
The client hardcodes a 60 s read timeout and splits historical pulls into 30-day requests. Neither is configurable. When the API is slow to answer a large span, a user waits ~4 minutes (60 s x 4 attempts from the session's retry policy) and then gets
RuntimeError: API Request Failed: ... Read timed outwith no indication of why and nothing to change. The only workaround has been to monkeypatch the private_get_chunks.This happened in August 2026: 30-day
co2_moerpulls for some regions took over 180 s server-side while the same query at 10 days took ~11 s. The backend cause is being addressed separately; this PR is about not leaving SDK users stranded the next time something similar happens. This PR adds knobs; a follow-on PR will propose a more graceful retry strategy.What changes
timeouton the constructor (WattTimeBase.__init__, default(10, 60)), applied at all three HTTP call sites. Accepts the standardrequestsforms: a number, a(connect, read)tuple, orNone.chunk_sizeonget_historical_jsons/_pandas/_csv(defaultNone= 30 days, unchanged)._get_chunksnow acceptsNoneand rejects sizes of 5 minutes or less.requestsraisesConnectionError -> MaxRetryError -> ReadTimeoutErrorrather thanReadTimeout, so a small_is_timeouthelper walks the exception chain.Both new parameters are keyword-only, and the internal call chain (
get_historical_csv -> _pandas -> _jsons) passes every defaulted argument by keyword -- the same convention as theinclude_imputed_markermisrouting fix, so argument-order differences between signatures cannot silently route a value to the wrong parameter.Implications
60(connect = read = 60); it now shares the(10, 60)default with login and register. The default read timeout of 60 s is unchanged at all three sites._get_chunksraisesValueErrorforchunk_size <= 5 min. Before, zero or negative never terminated and 5 minutes produced an inverted chunk. No caller passed such values.4 x read_timeout + 6 sto surface. Documented on the parameter.chunk_sizemeans more requests and, withinclude_meta=True, moremetarows -- one per response, as today.timeout/chunk_sizeis the final commit on its own; drop it if you would rather not have that section.Stacked on #75: the first commit here is #75's and drops out once it merges. Both PRs touch the
_csv -> _pandas -> _jsonscall chain, so this ordering avoids a conflict.Tests
Nine mocked tests, no network: the timeout reaches all three call sites; tuple/scalar/None accepted; the new parameters are keyword-only and existing positional calls still work; the hint appears for both the bare and the retry-wrapped timeout (with
__cause__preserved) and not for other errors;chunk_sizeis honoured by the chunker, forwarded from_pandas, and controls the request count; the validation floor.Follow-up (separate PR #77 )
These knobs let a user react, but they still have to hit the error first. PR #77 proposes a design concept to make retry policy failure-class-aware so the common cases need no intervention: split the span and retry smaller on read timeout / 504 instead of re-issuing the identical request four times; keep backoff for 5xx; refresh the token on 401; fail fast on other 4xx. It builds on the
chunk_sizeplumbing here, which is why this lands first.🤖 Generated with the help of Claude Code