Improve the HEMS tutorial for production use, asynchronous ingestion, cleanup, and interrupted-run recovery. - #222
Open
nhoening wants to merge 14 commits into
Open
Improve the HEMS tutorial for production use, asynchronous ingestion, cleanup, and interrupted-run recovery.#222nhoening wants to merge 14 commits into
nhoening wants to merge 14 commits into
Conversation
…be admin: account-admin suffices - improve docs and function names as all assets are in account Signed-off-by: Nicolas Höning <nicolas@seita.nl>
… on circumventing rate limiting as well Signed-off-by: Nicolas Höning <nicolas@seita.nl>
Signed-off-by: Nicolas Höning <nicolas@seita.nl>
Signed-off-by: Nicolas Höning <nicolas@seita.nl>
… tests Signed-off-by: Nicolas Höning <nicolas@seita.nl>
…not the strucure Signed-off-by: Nicolas Höning <nicolas@seita.nl>
Signed-off-by: Nicolas Höning <nicolas@seita.nl>
Coverage Report for CI Build 32086241636Coverage increased (+0.08%) to 96.658%Details
Uncovered ChangesNo uncovered changes found. Coverage RegressionsNo coverage regressions found. Coverage Stats
💛 - Coveralls |
Signed-off-by: Nicolas Höning <nicolas@seita.nl>
…; add some helpful prints in the startup Signed-off-by: Nicolas Höning <nicolas@seita.nl>
HEMS cleanup relies on depth zero to select only top-level assets, but the client discarded that value as falsy. This could broaden cleanup queries and make nested name collisions unsafe. Preserve zero-valued asset query parameters and expose the server's source and event-time filters for sensor data deletion. Add regression coverage for both request payloads. Signed-off-by: Mohamed Belhsan Hmida <mohamedbelhsanhmida@gmail.com>
Interrupted asset creation, partial data wipes, and legacy site names could leave users with unsafe or destructive recovery paths. Global name lookups could also select assets outside the intended community or account. Make structure repair idempotent, preserve existing IDs, and offer explicit keep, rename, recreate, continue-wipe, and exit choices where applicable. Validate stored workflow state, scope asset and sensor resolution, retain canonical ingestion jobs, and document the recovery behavior. Signed-off-by: Mohamed Belhsan Hmida <mohamedbelhsanhmida@gmail.com>
FlexMeasures 0.33 returns ingestion IDs under job_id and expands asset and sensor queries to include roots or descendants. The HEMS recovery flow treated these valid responses as missing or ambiguous, preventing clean runs and incomplete-structure repair. Accept both job field names, deduplicate repeated roots by ID, and constrain sensor matches to exact owners while retaining genuine ambiguity checks. Align deletion confirmation and recovery documentation, remove the obsolete global lookup helper, and add regression coverage. Signed-off-by: Mohamed Belhsan Hmida <mohamedbelhsanhmida@gmail.com>
BelhsanHmida
approved these changes
Aug 18, 2026
BelhsanHmida
left a comment
BelhsanHmida
left a comment
Contributor
There was a problem hiding this comment.
I completed a final review. looks good to me now. i pushed some follow-ups
The follow-up fixes make cleanup respect depth=0, add filtered sensor-data deletion, provide explicit and safe recovery choices for incomplete setups and interrupted wipes, preserve existing IDs when requested, scope asset and sensor lookups correctly, and support both FlexMeasures 0.33 job_id responses and newer job responses.
I manually verified the complete workflow on FlexMeasures 0.33, including resume, wipe recovery, incomplete-structure repair, legacy-name choices, scoped lookup, recreation cancellation, and cleanup. The full automated suite also passes: 220 tests, Black, Flake8, compilation, and git diff --check.
I found no remaining blockers. From my review, the PR is ready.
Building A asset graph:
Signed-off-by: Mohamed Belhsan Hmida <mohamedbelhsanhmida@gmail.com>
The pre-commit workflow reordered the HEMS workflow test imports, causing CI to fail because the generated formatting diff was not committed. Apply the deterministic import order so the repository remains unchanged when the full pre-commit suite runs. Signed-off-by: Mohamed Belhsan Hmida <mohamedbelhsanhmida@gmail.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Improve the HEMS tutorial for production use, asynchronous ingestion, cleanup, and interrupted-run recovery.
Production issues addressed
Verification
Remaining limitation
Report generation still requires a FlexMeasures CLI connected to the server database. It should be separated from the API-driven setup until reporting APIs are available.