You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Design question: why not use model metadata for contextWindow + built-in compaction? #246
Has no contextWindow metadata — It doesn't track each model's max context length (unlike pi, where models.json contains contextWindow per model)
Has no proactive compaction — It relies entirely on the API returning a context-length error, then does a reactive auto-handoff (anchor-based truncation, retrying once)
Uses handoff for truncation, not summarization — When context overflows, older tape entries before the anchor are simply dropped, not summarized
Current behavior
AgentSettings:
- max_tokens: 16384 (output limit only, not context window)
On context overflow:
- is_context_length_error() regex-matches the API error
- tape.handoff() creates a new anchor → older messages are silently dropped
- MAX_AUTO_HANDOFF_RETRIES = 1
Question
Is this by design? Specifically:
Model metadata: any-llm-sdk doesn't appear to expose contextWindow for providers/models. Is there a plan to add this? Or is the philosophy that Bub should be model-agnostic and not rely on hardcoded limits?
Built-in compaction: Would the project consider built-in summarization-based compaction (like pi's /compact), or is the anchor-based handoff considered sufficient? The current approach drops history rather than preserving it in compressed form.
Prevention vs recovery: Is the preference for "try first, recover on error" over "track context usage proactively"? The latter would allow Bub to trim/compact before hitting the API limit.
I'm happy to contribute if there's alignment on a direction. Thanks!
Activity