Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 2 additions & 1 deletion CLAUDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -106,7 +106,8 @@ itself (`__main__.py`).
`/set?state=open|closed|partial`, `/set?night=1`), `/photo/<name>.jpg` (the CC0 photos from
`backend/tests/assets`, for object sensors), `/display.jpg?text=12:05&style=lcd|led` (a drawn
seven-segment display) and `/counter.jpg?value=89939.5&digits=7&decimals=3` (a drawn mechanical
counter; `.5` = the last wheel half way to the next digit).
counter; `.5` = the last wheel half way to the next digit; with `&camera=a`,
`/set?wider=400&camera=a` makes it come back wider, like a camera that changed its size).
- **The Home Assistant side:** any local MQTT broker (e.g. Mosquitto on 1883) and
`VISIONSTATE_MQTT_HOST=127.0.0.1`: discovery configs appear under `homeassistant/…`, values under
`visionstate/<slug>/…`.
Expand Down
16 changes: 11 additions & 5 deletions docs/ARCHITECTURE.md
Original file line number Diff line number Diff line change
Expand Up @@ -58,6 +58,7 @@ API as `request.app.state.runtime`.
| `ha_events.py` | Home Assistant WebSocket: state changes of trigger entities, entity registry |
| `lights.py` | A sensor's light: switched on for checks and while someone looks, shared by both |
| `backbones.py`, `detectors.py`, `readers.py` | The three AI models and their registries (`*.json`): embedding backbone (state sensors), object detector, number reader (and the wheel reader for mechanical counters, `readers.WheelReader`; trained with `tools/wheelreader`) |
| `change_limit.py` | Counters: how much the value may go up since the last accepted reading — what it learns, time not seen (pure logic; the engine keeps a `Watch` per sensor) |
| `faces.py` | Faces in person boxes (registry `faces.json`, never bundled): finding them (YuNet), the quality gate, aligning, evening out the light (`even_light`), embedding, telling who it is |
| `classifier.py` | The small per-sensor classifier ("head") on top of the backbone |
| `teach.py` | Object sensors: comparing detected boxes with the boxes the user taught |
Expand Down Expand Up @@ -107,7 +108,7 @@ sensor loop (checks._sensor_loop) woken by: its interval, a trigger ent
└─ by kind:
state sensor → checks._run_states backbone embedding → head → probabilities
object sensor → objects._run_objects detector → taught boxes (teaching) → faces → tracks (logic)
reading sensor → reading._run_reading reader → parse → plausibility → debounce
reading sensor → reading._run_reading reader → parse → plausibility (change_limit) → debounce
│
├─ publish (publishing._send → mqtt.MqttBridge.publish) state, attributes, image
└─ history: a Prediction row + frame, when the value changed or the frame goes to review
Expand All @@ -120,13 +121,18 @@ error is kept in `LiveState.error` and shown in the UI.

- **SQLite** (`<data>/visionstate.db`): `sensor` (+ `state`), `sample` (+ `sample_label`,
`embedding` cache), `prediction` (the history and the review queue), `reading_stat`,
`model_info` (per sensor), `setting` (global settings as JSON), `person` (+ `face`,
`reading_watch` (a counter's limit between readings), `model_info` (per sensor), `setting` (global settings as JSON), `person` (+ `face`,
`face_embedding` cache: the people taught by face, shared by every sensor).
- **Faces** (`<data>/faces/`): crops of the people's taught faces — in the app's own storage, not
the media folder (see `storage.Storage.face_path`).
- **Files** (`<media>/`): `samples/<sensor>/` (training images; for object sensors the crops of
taught boxes), `history/<sensor>/`, `thumbs/`; `<data>/heads/` (trained heads), `<data>/models/`
(downloaded models, left out of backups).
- **Files**: `<data>/samples/<sensor>/` (training images; for object sensors the crops of taught
boxes — next to the database that lists them, so a backup holds both), `<data>/thumbs/sample/`,
`<data>/heads/` (trained heads), `<data>/models/` (downloaded models; models and thumbnails are
left out of backups); `<media>/history/<sensor>/` and `<media>/thumbs/history/`. Versions before
the move kept training images in `<media>/samples/`: they are read there until
`engine.HistoryMixin.move_samples` has copied them (`settings.SAMPLES_MOVE`). A file is removed
only after the row that lists it (`db.Database.after_commit`), and an image that is gone anyway
is left out of training and of the taught boxes instead of stopping the sensor.
- **Object sensors' taught boxes** are `sample` rows with `object_label`, `detected`, `box` and
`score`; own labels live in `sensor.objects["custom"]`. Readings and object events are
`prediction` rows (see `db.py`).
Expand Down
8 changes: 4 additions & 4 deletions docs/FACE_RECOGNITION_PLAN.md
Original file line number Diff line number Diff line change
Expand Up @@ -177,10 +177,10 @@ Design rules that follow:
added to the person's faces by itself (that drifts: one wrong match teaches the next one).
Unknown faces are not stored as templates; they only live in history frames, under the normal
history limits (`history_days`, `history_max_gb`).
3. **Faces are kept where only the app can read them.** Training images live in `/media`, which
Home Assistant's media browser shows to every user of the house; taught face crops go to the
app's own `/data` instead. That puts them in Home Assistant backups — say so in DOCS.md, and that
backups should be encrypted.
3. **Faces are kept where only the app can read them.** Training images lived in `/media` then,
which Home Assistant's media browser shows to every user of the house; taught face crops go to
the app's own `/data` instead (training images followed later, see SCOPE.md §11). That puts
them in Home Assistant backups — say so in DOCS.md, and that backups should be encrypted.
4. **Embeddings are biometric data too.** Published attacks rebuild a recognisable face from an
ArcFace-style template, so the cached embeddings are deleted with the person and never exported
(an import computes them again).
Expand Down
2 changes: 1 addition & 1 deletion docs/RELEASING.md
Original file line number Diff line number Diff line change
Expand Up @@ -76,7 +76,7 @@ all of it and tell the maintainer what was checked and what was found.
keep working, their entities keep their IDs, the log has no warnings or errors;
- restart Home Assistant, reboot the host and restart the MQTT broker: the app and its entities
come back by themselves;
- make a backup of the app and restore it: the sensors are back;
- make a backup of the app and restore it: the sensors are back, with their training images;
- **soak for 1 hour** with the sensors checking: the app's memory (Supervisor app stats) and its
data on disk level off instead of growing, the log stays quiet;
- open the app in Home Assistant at desktop width and on a phone (Ingress);
Expand Down
Loading
Loading