Feat/updated since and Fix/for empty responses

[geoffhancock]

Add updated_since support, and fix a KeyError on empty historical responses

Stacked on #76: this PR's base branch is feat/configurable-timeout, so the diff below is only this PR's own change. Once #76 merges, GitHub retargets this to main. It is independent of #77 -- the two touch different code and merge cleanly in either order.

Why there is a bug fix in a feature PR. updated_since makes the empty response routine rather than exceptional: "nothing has been revised since X" is the normal answer most of the time, and a client that crashes on it is unusable for incremental sync. So handling the empty case was a prerequisite of the feature, not an extra. Doing it properly turned out to fix a KeyError that get_historical_pandas raises on origin/main today for reasons that have nothing to do with updated_since -- including start == end, the exact span #74 just made reachable.

In other words: the feature forced us to look at a code path that was already broken. The fix is one guard, described next; the feature follows.

The empty-response crash

/v3/historical answers 200 with "data": [] -- never an error -- when a request matches no points. pd.json_normalize turns that into a 0x0 frame with no columns, and the next line indexes df["point_time"]. That is reachable three ways today, only one of which involves updated_since. Verified against the live API, then against origin/main itself:

===== origin/main (5ec3a8f) =====
  RAISES zero-length span (start == end): KeyError: 'point_time'
  RAISES range before data_start (2010): KeyError: 'point_time'
  RAISES range in the future (+30d):     KeyError: 'point_time'

===== this branch =====
  OK  shape=(0, 2) cols=['point_time', 'value']   (all three)

The zero-length case is the pointed one. #74 fixed the IndexError in _get_chunks for start == end and covered it with a get_historical_forecast_pandas test, which passes because the forecast endpoint returns the run generated at that instant. /v3/historical returns "data": [] for the same span, so get_historical_pandas(start=X, end=X) currently trades #74's IndexError for a KeyError. This closes that gap.

The guard is keyed on df.empty, so one piece of code covers both the feature's normal case and the three pre-existing ones. updated_since appears inside it only to decide whether last_updated belongs in the expected column list -- which is why the two changes are the same change, and why splitting them would mean writing the same guard twice.

Why updated_since

WattTime revises historical data after it is first published. Today the only way to pick up those revisions is to re-pull the whole range and diff it locally, which means re-fetching a year of five-minute data to find the handful of points that actually changed. The /v3/historical endpoint already supports an updated_since query parameter for exactly this; the client just did not expose it.

What changes

updated_since on get_historical_jsons, get_historical_pandas and get_historical_csv, accepting a str or datetime and normalised to UTC. Keyword-only, like chunk_size in #76.

• The filter is inclusive (last_updated >= updated_since). Verified empirically against the live API rather than assumed, and stated in the docstrings so callers can page through revisions without off-by-one guessing.
• last_updated comes back as a parsed UTC column in get_historical_pandas when updated_since is passed, since the API adds that field to each data point.
• An empty result is a normal outcome. Nothing may have been revised in the window. get_historical_pandas now returns a typed empty DataFrame carrying the expected columns instead of raising KeyError: 'point_time', so downstream code can rely on the schema either way.
• get_historical_csv appends _updated-since-<timestamp> to the filename. A filtered CSV is a partial dataset and should not be mistakable for a full pull sitting in the same directory.
• _parse_date is extracted from _parse_dates, which now wraps it. No behaviour change; updated_since is a single date and needed the same normalisation.

Implications

• point_time is now parsed with utc=True in get_historical_pandas, making explicit what was previously inferred. Measured on a normal call against both versions: datetime64[ns, UTC] either way, so this is a no-op for existing callers and simply guarantees the dtype when a response is empty or offsets are mixed.
• updated_since is keyword-only, so no positional call is affected and the positional contract of all three signatures is unchanged.
• Passing nothing behaves exactly as before: no updated_since parameter is sent, no last_updated column appears, and the CSV filename is unchanged.

Tests

Four tests against the live API, matching the style of the surrounding TestWattTimeHistorical cases.

For the crash: test_get_historical_pandas_empty_without_updated_since covers all three empty-response spans -- zero-length, before data_start, and future -- with no filter set, asserting an empty frame with point_time and value present, last_updated absent, and a datetime dtype. Each subtest raises KeyError on origin/main.

For the feature: a filtered pull returns rows carrying a parsed last_updated column; a filter set in the future returns an empty DataFrame with the expected columns; and the CSV path writes a file with the _updated-since- suffix. The existing suite passes unchanged.

The README changes are the final commit on its own; drop it if you would rather not have them.

🤖 Generated with Claude Code