Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
18 changes: 9 additions & 9 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -22,8 +22,8 @@ jobs:
matrix:
python-version: ["3.9", "3.10", "3.11", "3.12", "3.13", "3.14"]
steps:
- uses: actions/checkout@v7
- uses: actions/setup-python@v7
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
- uses: actions/setup-python@5fda3b95a4ea91299a34e894583c3862153e4b97 # v7.0.0
with:
python-version: ${{ matrix.python-version }}
cache: pip
Expand All @@ -40,8 +40,8 @@ jobs:
name: ESP32 Arduino examples
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v7
- uses: actions/setup-python@v7
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
- uses: actions/setup-python@5fda3b95a4ea91299a34e894583c3862153e4b97 # v7.0.0
with:
python-version: "3.14"
cache: pip
Expand All @@ -54,8 +54,8 @@ jobs:
name: Arduino native tests
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v7
- uses: actions/setup-python@v7
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
- uses: actions/setup-python@5fda3b95a4ea91299a34e894583c3862153e4b97 # v7.0.0
with:
python-version: "3.14"
- run: python -m pip install platformio
Expand All @@ -67,8 +67,8 @@ jobs:
env:
SOURCE_DATE_EPOCH: "1700000000"
steps:
- uses: actions/checkout@v7
- uses: actions/setup-python@v7
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
- uses: actions/setup-python@5fda3b95a4ea91299a34e894583c3862153e4b97 # v7.0.0
with:
python-version: "3.14"
cache: pip
Expand All @@ -79,7 +79,7 @@ jobs:
- run: python scripts/build_checksums.py
- run: python scripts/verify_reproducible.py
- run: sh -n packaging/linux/install.sh packaging/linux/uninstall.sh
- uses: actions/upload-artifact@v7
- uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
with:
name: packages
path: |
Expand Down
8 changes: 4 additions & 4 deletions .github/workflows/codeql.yml
Original file line number Diff line number Diff line change
Expand Up @@ -18,16 +18,16 @@ jobs:
matrix:
language: ["python", "cpp"]
steps:
- uses: actions/checkout@v7
- uses: github/codeql-action/init@v4
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
- uses: github/codeql-action/init@e4fba868fa4b1b91e1fdab776edc8cfbe6e9fb81 # v4.37.3
with:
languages: ${{ matrix.language }}
- if: matrix.language == 'cpp'
uses: actions/setup-python@v7
uses: actions/setup-python@5fda3b95a4ea91299a34e894583c3862153e4b97 # v7.0.0
with:
python-version: "3.14"
- if: matrix.language == 'cpp'
run: |
python -m pip install platformio
platformio run -e esp32dev
- uses: github/codeql-action/analyze@v4
- uses: github/codeql-action/analyze@e4fba868fa4b1b91e1fdab776edc8cfbe6e9fb81 # v4.37.3
8 changes: 5 additions & 3 deletions .github/workflows/release.yml
Original file line number Diff line number Diff line change
Expand Up @@ -13,20 +13,22 @@ jobs:
env:
SOURCE_DATE_EPOCH: "1700000000"
steps:
- uses: actions/checkout@v7
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
with:
fetch-depth: 0
- uses: actions/setup-python@v7
- uses: actions/setup-python@5fda3b95a4ea91299a34e894583c3862153e4b97 # v7.0.0
with:
python-version: "3.14"
cache: pip
- name: Verify, test, and build
run: |
python -m pip install -e "./python[dev]" build
python -m pip install -e "./python[dev]" build platformio
python -m pytest python
python -m ruff check python benchmarks scripts
python -m mypy python/src
python scripts/check_docs.py
platformio test -e native
platformio run -e esp32dev
python -m build python
python scripts/build_release.py --expected-version "${GITHUB_REF_NAME#v}"
python scripts/build_linux_bundle.py --expected-version "${GITHUB_REF_NAME#v}"
Expand Down
62 changes: 61 additions & 1 deletion CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,65 @@ All notable changes are documented here. OpenNet follows

## [Unreleased]

## [0.2.0] - 2026-07-29

### Added

- Optional synchronous or asynchronous server authorization before DATA handlers
and acknowledgements.
- TLS state and verified peer-certificate access through the Python `Peer` API.
- Fail-closed authorization behavior, a denial counter, mutual-TLS integration
coverage, and test-only certificate fixtures.
- A practical certificate-to-topic authorization guide and v0.2.0 release notes.

### Changed

- Release verification now includes the native Arduino tests and ESP32 build.
- Current install examples and package metadata target v0.2.0.
- Project positioning now emphasizes direct embedded-to-host communication, and
the reliable-stream boundary explicitly documents why raw UDP is unsupported.
- Arduino receive handling now moves the validated body buffer into the callback
message instead of allocating and copying a second payload vector.
- ACK semantics now explicitly mean validated transport acceptance before
handler completion; delivery guarantees and timeout/overload policies are
documented without changing ONP/1 bytes.

### Fixed

- Python PING no longer cycles forever when DATA arrives before PONG.
- A full Python application queue no longer blocks ACK and control processing;
overflow now sends ERROR and closes explicitly.
- Python message-ID wrap skips IDs with pending ACK futures.
- Python JSON decoding rejects non-standard `NaN` and infinity constants.
- Python client writes now have a configurable timeout.
- Python servers ACK accepted DATA before bounded handler dispatch, so slow
handlers no longer block PING or turn completed side effects into ACK timeout
ambiguity.
- Python servers bound idle/header/body/TLS/write/handler waits and reject a full
handler set explicitly.
- Python and Arduino now share malformed UTF-8/JSON conformance vectors, enforce
strict text/control encoding, and bound JSON nesting to 32.
- Arduino polling has a configurable byte budget, partial writes have a deadline
and cooperative yield, generic sends enforce typed invariants, and ACK tracking
uses bounded pending/history tables.
- Arduino applications can provide a fixed receive buffer and function-pointer
message view to avoid OpenNet receive-path heap allocation.
- The CLI rejects oversized files before reading them, logs payload metadata
rather than values by default, offers a shared topic-prefix allowlist, and
labels its benchmark as sequential acknowledged round trips.
- A Python server at its connection limit now sends a best-effort ERROR before
closing the rejected connection.
- GitHub Actions dependencies are pinned to immutable upstream commits with
readable release-version comments.
- Deterministic fault tests cover concurrent sends, every truncation point of a
valid frame, fragmented ACK delivery, and a reproducible 24-byte-header
mutation corpus.

### Compatibility

- ONP/1 framing and typed-value encodings are unchanged.
- The Python server additions are optional and backward compatible.

## [0.1.1] - 2026-07-29

### Added
Expand Down Expand Up @@ -38,6 +97,7 @@ All notable changes are documented here. OpenNet follows
- Arduino IDE, PlatformIO, and Python examples.
- Cross-platform tests, CI, packaging, and contributor documentation.

[Unreleased]: https://github.com/devkyato/OpenNet/compare/v0.1.1...HEAD
[Unreleased]: https://github.com/devkyato/OpenNet/compare/v0.2.0...HEAD
[0.2.0]: https://github.com/devkyato/OpenNet/compare/v0.1.1...v0.2.0
[0.1.1]: https://github.com/devkyato/OpenNet/compare/v0.1.0...v0.1.1
[0.1.0]: https://github.com/devkyato/OpenNet/releases/tag/v0.1.0
49 changes: 37 additions & 12 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,19 +7,34 @@
[![License: MIT](https://img.shields.io/badge/license-MIT-4a638f.svg)](LICENSE)
[![Protocol](https://img.shields.io/badge/protocol-ONP%2F1-4a638f.svg)](docs/protocol.md)

OpenNet is a dependency-free typed messaging protocol for direct communication
between ESP32 devices, Raspberry Pi computers, and backend services. It sends
OpenNet is a focused, dependency-free messaging protocol I built for direct
links between embedded devices and a Python host—especially an ESP32 talking to
a Raspberry Pi or backend service. It is not a general cross-platform
communication layer or a replacement for MQTT, HTTP, or WebSocket. It sends
JSON, text, signed integers, finite doubles, booleans, null, and arbitrary binary
data over TCP/TLS or another reliable ordered stream.

Version 0.1.1 is an alpha release for real projects and learning. It has a
Version 0.2.0 is an alpha release for real projects and learning. It has a
documented wire format, bounded resource use, retries and duplicate suppression,
native Arduino protocol tests, Python tests across 3.9–3.14, reproducible release
packages, measured local performance, and security guidance. It does not claim
that a radio or internet route exists when the underlying hardware/network is
unavailable.
packages, measured local performance, and an optional topic-authorization hook.
It does not claim that a radio or internet route exists when the underlying
hardware or network is unavailable.

## Why OpenNet
## What I wanted from OpenNet

I kept coming back to one idea: a sensor message should have the same bytes and
the same meaning whether it travels through an ESP32 Wi-Fi client, a serial
stream, or a Python TLS connection. I thought about reliability on that point
too, which is why retries keep the same message ID and receivers can suppress
recent duplicates without pretending the message is durably stored.

Why make ONP/1 when other protocols already exist? I wanted the framing, typing,
ACK behavior, limits, and cross-language test vectors to stay small enough to
read end to end. MQTT is the stronger choice for brokered fleets and retained
messages; HTTP and WebSocket are stronger when web ecosystems matter. ONP/1 is
for the narrower space where one embedded peer and one host need a direct,
inspectable contract without a broker.

- One small API and one frame format across ESP32 and Python.
- Typed values without a JSON dependency for every scalar or binary payload.
Expand All @@ -30,6 +45,8 @@ unavailable.
SPP on compatible ESP32 hardware.
- Strict control frames, CRC-32 corruption detection, payload limits, bounded
queues, connection limits, and partial transport-write handling.
- Optional Python server authorization using the topic and verified TLS peer
certificate before a handler runs or an ACK is sent.
- CLI tools to serve, send, ping, and benchmark.
- Arduino IDE, PlatformIO/VS Code, Python, Linux/systemd, and security tutorials.

Expand All @@ -39,23 +56,23 @@ Python wheel from the release page:

```sh
python -m pip install \
https://github.com/devkyato/OpenNet/releases/download/v0.1.1/opennet_protocol-0.1.1-py3-none-any.whl
https://github.com/devkyato/OpenNet/releases/download/v0.2.0/opennet_protocol-0.2.0-py3-none-any.whl
opennet --version
```

For an isolated command:

```sh
pipx install \
https://github.com/devkyato/OpenNet/releases/download/v0.1.1/opennet_protocol-0.1.1-py3-none-any.whl
https://github.com/devkyato/OpenNet/releases/download/v0.2.0/opennet_protocol-0.2.0-py3-none-any.whl
```

For Raspberry Pi/Linux, extract `OpenNet-linux-0.1.1.tar.gz` and run
For Raspberry Pi/Linux, extract `OpenNet-linux-0.2.0.tar.gz` and run
`sudo ./install.sh`. It creates a hardened, unprivileged systemd service with a
loopback-only default. For Arduino IDE, install `OpenNet-0.1.1.zip` through
loopback-only default. For Arduino IDE, install `OpenNet-0.2.0.zip` through
**Sketch > Include Library > Add .ZIP Library**.

The Python distribution is installable with pip, but v0.1.1 is distributed from
The Python distribution is installable with pip, but v0.2.0 is distributed from
GitHub Releases rather than the public PyPI index.

## Five-minute local test
Expand Down Expand Up @@ -99,14 +116,22 @@ device identity matters—outside isolated local testing.

- [Measured TCP/TLS and 20-client load results](docs/benchmarks.md)
- [Comparison with raw TCP, MQTT, WebSocket, and HTTP](docs/comparison.md)
- [Why ONP/1 exists and what it deliberately does not do](docs/design-rationale.md)
- [Architecture, ACK retry, and defensive boundaries](docs/architecture.md)
- [Certificate-to-topic authorization](docs/authorization.md)
- [Actual GitHub adoption counters and analytics limits](docs/adoption-analytics.md)
- [Transport support, including Bluetooth boundaries](docs/transports.md)
- [Arduino memory model and payload sizing](docs/embedded-memory.md)

Measured loopback results are software baselines, not invented Wi-Fi or hardware
claims. Hardware field reports are welcome through the
[compatibility process](docs/compatibility.md).

ACKs mean validated transport acceptance, not successful application side
effects. Duplicate suppression is bounded and connection-local; applications
must use durable operation IDs when replay after reconnect could repeat an
unsafe action.

## Learn and build

- [Tutorials](docs/tutorials.md)
Expand Down
4 changes: 3 additions & 1 deletion SECURITY.md
Original file line number Diff line number Diff line change
Expand Up @@ -22,7 +22,9 @@ coordinate disclosure after a fix is available.
- Plain TCP provides no confidentiality, peer identity, or replay protection.
- Use TLS with certificate validation across untrusted networks.
- Do not use `setInsecure()` in production.
- Applications must authorize topics and validate payloads.
- Applications must authorize topics and validate payloads. Python servers can
use the optional pre-handler `authorizer`; other deployments must enforce an
equivalent application policy.
- The 16 MiB frame limit reduces memory-exhaustion risk; deployments should choose
a smaller application limit appropriate for their hardware.

Expand Down
Loading
Loading