Enhance testing and error handling; update documentation and examples

This commit is contained in:
pdxlocations
2026-08-01 01:14:46 -07:00
parent d242a82d1d
commit dbaa3b38b8
21 changed files with 228 additions and 61 deletions
+3
View File
@@ -25,6 +25,9 @@ jobs:
- name: Install build tools - name: Install build tools
run: python -m pip install --upgrade build twine run: python -m pip install --upgrade build twine
- name: Run tests
run: python -m unittest discover -s tests
- name: Build wheel and source distribution - name: Build wheel and source distribution
run: python -m build run: python -m build
+46 -19
View File
@@ -22,7 +22,7 @@ three wires, and regular Python.
- Send binary data and positions - Send binary data and positions
- Read user, position, signal, battery, and hop information - Read user, position, signal, battery, and hop information
- Preserve unknown protobuf fields for forward compatibility - Preserve unknown protobuf fields for forward compatibility
- Recover from serial noise and skip newer messages it cannot decode - Recover from serial noise and optionally report and skip undecodable frames
- Run a curses dashboard with live stats, nodes, and messages - Run a curses dashboard with live stats, nodes, and messages
- Work on MicroPython without third-party runtime dependencies - Work on MicroPython without third-party runtime dependencies
@@ -59,16 +59,21 @@ You need:
- A separate Meshtastic-compatible radio running Meshtastic firmware - A separate Meshtastic-compatible radio running Meshtastic firmware
- Three jumper wires for TX, RX, and GND - Three jumper wires for TX, RX, and GND
- A data-capable USB cable for the XIAO - A data-capable USB cable for the XIAO
- Python 3 on your Mac, Linux computer, or Windows PC - Python 3.8 or newer on your Mac, Linux computer, or Windows PC
Configure the Meshtastic radio's serial interface for: For the UART-through-microcontroller setups, configure the Meshtastic radio's serial
module for:
```text ```text
Enabled: yes Enabled: yes
Mode: PROTO Mode: PROTO
Baud: 115200 Baud: 115200 (must match the MicroPython UART)
``` ```
Meshtastic's serial-module default is 38400 baud; these examples deliberately use
115200. Set both ends to the same value. A direct USB connection to the radio does not
use this external-UART setup.
The radio's UART pin names depend on its model. Consult that board's pinout before The radio's UART pin names depend on its model. Consult that board's pinout before
connecting wires. connecting wires.
@@ -168,8 +173,14 @@ mpremote connect /dev/cu.usbmodem2101 fs cp examples/xiao_rp2040.py :main.py
mpremote connect /dev/cu.usbmodem2101 reset mpremote connect /dev/cu.usbmodem2101 reset
``` ```
To stop or replace an automatically running script, connect the XIAO, press `Ctrl-C` To replace an automatically running script, upload a different `main.py`; `mpremote`
in `mpremote repl`, and upload a different `main.py`. stops the running program before filesystem commands. To disable automatic startup
without replacing it:
```sh
mpremote connect /dev/cu.usbmodem2101 rm :main.py
mpremote connect /dev/cu.usbmodem2101 reset
```
## Curses dashboard through the XIAO ## Curses dashboard through the XIAO
@@ -178,10 +189,17 @@ It displays live connection stats, known nodes, battery levels, SNR, hops, last-
times, and incoming messages. The XIAO runs a small bridge between the radio UART and times, and incoming messages. The XIAO runs a small bridge between the radio UART and
the dashboard over USB. the dashboard over USB.
The header shows `Config: complete` after a matching configuration handshake. Some
radios do not return the matching completion ID; once local and node data are usable,
the dashboard shows `Config: ready` instead.
The dashboard needs a small bridge program on the XIAO. The regular The dashboard needs a small bridge program on the XIAO. The regular
`xiao_rp2040.py` example prints human-readable logs; it does not expose structured data `xiao_rp2040.py` example prints human-readable logs; it does not expose structured data
to desktop applications. to desktop applications.
Complete the XIAO quick-start installation through step 4 first, so the `micromesh`
package is present on the board.
### 1. Install the desktop dependencies ### 1. Install the desktop dependencies
```sh ```sh
@@ -214,7 +232,8 @@ Only one application can own a serial port. Close `mpremote`, Arduino Serial Mon
screen, minicom, and other serial tools before launching the dashboard. screen, minicom, and other serial tools before launching the dashboard.
The message field is always active. Type a message and press Enter—there is no separate The message field is always active. Type a message and press Enter—there is no separate
compose mode. compose mode. `message queued` means the radio accepted the packet for transmission;
it is not a delivery receipt.
| Key | Action | | Key | Action |
| --- | --- | | --- | --- |
@@ -296,9 +315,11 @@ Send a **new** text from another Meshtastic node while MicroMesh is running.
### A frame is skipped or reports a decode error ### A frame is skipped or reports a decode error
Meshtastic's protobuf schema evolves. MicroMesh reports and skips newer messages that Meshtastic's protobuf schema evolves. Unknown fields are retained automatically. When
its compact schema does not yet understand. Text and other supported packets continue an `on_error` callback is configured—as it is in the board examples—MicroMesh reports
processing. Update the installed files after pulling a newer MicroMesh version: and skips a frame if it is malformed or conflicts with the compact schema; later
frames continue processing. Without that callback, `poll()` raises the decoding error.
Update the installed files after pulling a newer MicroMesh version:
```sh ```sh
mpremote connect /dev/cu.usbmodem2101 mip install package.json mpremote connect /dev/cu.usbmodem2101 mip install package.json
@@ -324,14 +345,19 @@ def received(packet):
if packet.WhichOneof("payload_variant") != "decoded": if packet.WhichOneof("payload_variant") != "decoded":
return return
if packet.decoded.portnum == PortNum.TEXT_MESSAGE_APP: if packet.decoded.portnum == PortNum.TEXT_MESSAGE_APP:
print("from !%08x: %s" % ( try:
packet.from_, text = packet.decoded.payload.decode("utf-8")
packet.decoded.payload.decode("utf-8"), except UnicodeError:
)) text = repr(packet.decoded.payload)
print("from !%08x: %s" % (packet.from_, text))
def decode_error(error, payload):
print("skipping undecodable frame:", error, "(%d bytes)" % len(payload))
uart = UART(0, 115200, tx=Pin(0), rx=Pin(1), timeout=0, rxbuf=1024) uart = UART(0, 115200, tx=Pin(0), rx=Pin(1), timeout=0, rxbuf=1024)
mesh = SerialInterface(uart, on_packet=received) mesh = SerialInterface(uart, on_packet=received, on_error=decode_error)
mesh.connect() mesh.connect()
while True: while True:
@@ -355,8 +381,9 @@ mesh.sendPosition(45.5152, -122.6784, altitude=15)
mesh.sendData(b"custom", portNum=PortNum.PRIVATE_APP) mesh.sendData(b"custom", portNum=PortNum.PRIVATE_APP)
``` ```
Payloads may be at most 233 bytes. `poll()` is non-blocking when the UART uses Data payloads may be at most 233 bytes, including UTF-8 encoded text bytes rather than
`timeout=0`, so call it frequently from your main loop. characters. `poll()` is non-blocking when the UART uses `timeout=0`, so call it
frequently from your main loop.
### Inspect connection state ### Inspect connection state
@@ -437,8 +464,8 @@ python -m pip install -e .
python -m unittest discover -s tests python -m unittest discover -s tests
``` ```
The test suite covers encoding, decoding, framing, serial recovery, sending, and The test suite covers encoding, decoding, field presence, framing, serial recovery,
configuration state. sending, input validation, and configuration state.
## Releasing to PyPI ## Releasing to PyPI
+7 -4
View File
@@ -1,8 +1,9 @@
# Board examples # Board examples
These examples connect a MicroPython board to the hardware UART exposed by a These examples connect a MicroPython board to the hardware UART exposed by a
Meshtastic radio. The radio must have its serial module enabled in `PROTO` mode at Meshtastic radio. The radio must have its serial module enabled in `PROTO` mode. The
115200 baud. examples choose 115200 baud, so configure the radio to match; Meshtastic's serial
module defaults to 38400 baud.
| Board | Example | MicroPython UART | TX pin | RX pin | | Board | Example | MicroPython UART | TX pin | RX pin |
| --- | --- | --- | --- | --- | | --- | --- | --- | --- | --- |
@@ -21,9 +22,11 @@ python -m pip install -e . pyserial
python examples/curses_dashboard.py /dev/cu.usbmodem123456 python examples/curses_dashboard.py /dev/cu.usbmodem123456
``` ```
For a radio wired to a XIAO RP2040, first install the bridge as the XIAO's boot script: For a radio wired to a XIAO RP2040, install MicroMesh and then install the bridge as
the XIAO's boot script:
```sh ```sh
mpremote connect /dev/cu.usbmodem2101 mip install package.json
mpremote connect /dev/cu.usbmodem2101 fs cp examples/xiao_dashboard_bridge.py :main.py mpremote connect /dev/cu.usbmodem2101 fs cp examples/xiao_dashboard_bridge.py :main.py
mpremote connect /dev/cu.usbmodem2101 reset mpremote connect /dev/cu.usbmodem2101 reset
python examples/curses_dashboard.py --xiao-bridge /dev/cu.usbmodem2101 python examples/curses_dashboard.py --xiao-bridge /dev/cu.usbmodem2101
@@ -37,7 +40,7 @@ Use `mpremote connect list`, `python -m serial.tools.list_ports`, or your operat
system's device list to find the radio port. The message field is always active: type system's device list to find the radio port. The message field is always active: type
and press Enter to send. Use Escape to clear it, F5 to refresh, and F10 to quit. and press Enter to send. Use Escape to clear it, F5 to refresh, and F10 to quit.
`Ctrl-R` and `Ctrl-Q` are also supported. On Windows, install `windows-curses` and use `Ctrl-R` and `Ctrl-Q` are also supported. On Windows, install `windows-curses` and use
a port such as `COM4`. a port such as `COM4`. A `message queued` status is not a delivery receipt.
## Wiring ## Wiring
+33 -7
View File
@@ -21,6 +21,7 @@ from micromesh import PortNum, SerialInterface
EVENT_PREFIX = "MMEVT " EVENT_PREFIX = "MMEVT "
COMMAND_PREFIX = "MMCMD " COMMAND_PREFIX = "MMCMD "
MAX_BRIDGE_BUFFER = 8192
class PySerialUART: class PySerialUART:
@@ -85,7 +86,10 @@ class XiaoBridge:
command = {"type": kind} command = {"type": kind}
command.update(values) command.update(values)
line = COMMAND_PREFIX + json.dumps(command, separators=(",", ":")) + "\n" line = COMMAND_PREFIX + json.dumps(command, separators=(",", ":")) + "\n"
self.port.write(line.encode("utf-8")) data = line.encode("utf-8")
written = self.port.write(data)
if written is not None and written != len(data):
raise OSError("short serial write")
def connect(self): def connect(self):
self._command("refresh") self._command("refresh")
@@ -101,6 +105,11 @@ class XiaoBridge:
waiting = self.port.in_waiting waiting = self.port.in_waiting
if waiting: if waiting:
self._buffer.extend(self.port.read(waiting)) self._buffer.extend(self.port.read(waiting))
if len(self._buffer) > MAX_BRIDGE_BUFFER and b"\n" not in self._buffer:
self._buffer = bytearray()
self.state.decode_errors += 1
self.state.status = "discarded oversized bridge line"
return
while True: while True:
newline = self._buffer.find(b"\n") newline = self._buffer.find(b"\n")
if newline < 0: if newline < 0:
@@ -112,8 +121,11 @@ class XiaoBridge:
if marker < 0: if marker < 0:
continue continue
try: try:
self._event(json.loads(line[marker + len(EVENT_PREFIX):])) event = json.loads(line[marker + len(EVENT_PREFIX):])
except (ValueError, TypeError) as error: if not isinstance(event, dict):
raise ValueError("event is not an object")
self._event(event)
except (KeyError, ValueError, TypeError) as error:
self.state.decode_errors += 1 self.state.decode_errors += 1
self.state.status = "bad bridge event: %s" % error self.state.status = "bad bridge event: %s" % error
@@ -206,6 +218,17 @@ def node_values(info):
return node_id(info.num), name, short, snr, battery, hops, heard return node_id(info.num), name, short, snr, battery, hops, heard
def config_status(mesh, state):
"""Describe useful progress even when the completion ID never matches."""
if mesh.config_complete:
return "complete"
if mesh.my_info is not None and mesh.nodes:
return "ready"
if state.frames or mesh.my_info is not None or mesh.nodes:
return "receiving"
return "waiting"
def add_line(screen, row, text, style=0): def add_line(screen, row, text, style=0):
height, width = screen.getmaxyx() height, width = screen.getmaxyx()
if 0 <= row < height and width > 1: if 0 <= row < height and width > 1:
@@ -225,13 +248,13 @@ def draw(screen, mesh, state):
add_line(screen, 0, " MicroMesh dashboard ", title_style) add_line(screen, 0, " MicroMesh dashboard ", title_style)
local = node_id(mesh.my_info.my_node_num) if mesh.my_info else "unknown" local = node_id(mesh.my_info.my_node_num) if mesh.my_info else "unknown"
connected = "yes" if mesh.config_complete else "waiting" config = config_status(mesh, state)
uptime = int(time.monotonic() - state.started) uptime = int(time.monotonic() - state.started)
add_line( add_line(
screen, screen,
1, 1,
"Local: %s Config: %s Uptime: %ds Nodes: %d" % ( "Local: %s Config: %s Uptime: %ds Nodes: %d" % (
local, connected, uptime, len(mesh.nodes) local, config, uptime, len(mesh.nodes)
), ),
) )
add_line( add_line(
@@ -348,10 +371,10 @@ def run(screen, port_name, baudrate, xiao_bridge):
state.status = "configuration requested" state.status = "configuration requested"
elif key in (10, 13, curses.KEY_ENTER): elif key in (10, 13, curses.KEY_ENTER):
text = state.draft.strip() text = state.draft.strip()
state.draft = ""
if text: if text:
try: try:
mesh.sendText(text) mesh.sendText(text)
state.draft = ""
state.text_sent += 1 state.text_sent += 1
state.add_message("local", "all", text) state.add_message("local", "all", text)
state.status = "message queued" state.status = "message queued"
@@ -372,7 +395,10 @@ def run(screen, port_name, baudrate, xiao_bridge):
def main(): def main():
parser = argparse.ArgumentParser(description="MicroMesh curses dashboard") parser = argparse.ArgumentParser(description="MicroMesh curses dashboard")
parser.add_argument("port", help="Meshtastic serial port, such as /dev/cu.usbmodem1234") parser.add_argument("port", help="Meshtastic serial port, such as /dev/cu.usbmodem1234")
parser.add_argument("--baud", type=int, default=115200, help="UART baud rate (default: 115200)") parser.add_argument(
"--baud", type=int, default=115200,
help="UART baud rate (default: 115200)",
)
parser.add_argument( parser.add_argument(
"--xiao-bridge", "--xiao-bridge",
action="store_true", action="store_true",
+10 -4
View File
@@ -18,13 +18,19 @@ def on_packet(packet):
if packet.WhichOneof("payload_variant") != "decoded": if packet.WhichOneof("payload_variant") != "decoded":
return return
if packet.decoded.portnum == PortNum.TEXT_MESSAGE_APP: if packet.decoded.portnum == PortNum.TEXT_MESSAGE_APP:
print("from !%08x: %s" % ( try:
packet.from_, packet.decoded.payload.decode("utf-8", "replace") text = packet.decoded.payload.decode("utf-8")
)) except UnicodeError:
text = repr(packet.decoded.payload)
print("from !%08x: %s" % (packet.from_, text))
def on_error(error, payload):
print("skipping undecodable radio frame:", error, "(%d bytes)" % len(payload))
uart = UART(2, baudrate=115200, tx=Pin(17), rx=Pin(16), timeout=0, rxbuf=1024) uart = UART(2, baudrate=115200, tx=Pin(17), rx=Pin(16), timeout=0, rxbuf=1024)
mesh = SerialInterface(uart, on_packet=on_packet, on_log=print) mesh = SerialInterface(uart, on_packet=on_packet, on_log=print, on_error=on_error)
mesh.connect() mesh.connect()
hello_sent = False hello_sent = False
+10 -4
View File
@@ -16,13 +16,19 @@ def on_packet(packet):
if packet.WhichOneof("payload_variant") != "decoded": if packet.WhichOneof("payload_variant") != "decoded":
return return
if packet.decoded.portnum == PortNum.TEXT_MESSAGE_APP: if packet.decoded.portnum == PortNum.TEXT_MESSAGE_APP:
print("from !%08x: %s" % ( try:
packet.from_, packet.decoded.payload.decode("utf-8", "replace") text = packet.decoded.payload.decode("utf-8")
)) except UnicodeError:
text = repr(packet.decoded.payload)
print("from !%08x: %s" % (packet.from_, text))
def on_error(error, payload):
print("skipping undecodable radio frame:", error, "(%d bytes)" % len(payload))
uart = UART(0, 115200, tx=Pin(0), rx=Pin(1), timeout=0, rxbuf=1024) uart = UART(0, 115200, tx=Pin(0), rx=Pin(1), timeout=0, rxbuf=1024)
mesh = SerialInterface(uart, on_packet=on_packet, on_log=print) mesh = SerialInterface(uart, on_packet=on_packet, on_log=print, on_error=on_error)
mesh.connect() mesh.connect()
hello_sent = False hello_sent = False
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
+20 -6
View File
@@ -34,14 +34,21 @@ def node_num(value):
value = value.strip() value = value.strip()
if value.startswith("!"): if value.startswith("!"):
value = value[1:] value = value[1:]
return int(value, 16) value = int(value, 16)
return int(value) else:
value = int(value)
if not 0 <= value <= 0xFFFFFFFF:
raise ValueError("node number must fit in 32 bits")
return value
class SerialInterface: class SerialInterface:
"""A non-blocking Meshtastic client around a ``machine.UART``-like object. """A non-blocking Meshtastic client around a ``machine.UART``-like object.
``poll()`` should be called regularly. Callbacks receive ``FromRadio`` objects. ``poll()`` should be called regularly. ``on_receive`` receives each
``FromRadio`` message, ``on_packet`` receives its ``MeshPacket`` payload,
``on_log`` receives a text line, and ``on_error`` receives an exception and
the undecodable frame payload.
""" """
def __init__(self, uart, on_receive=None, on_packet=None, on_log=None, def __init__(self, uart, on_receive=None, on_packet=None, on_log=None,
@@ -102,6 +109,8 @@ class SerialInterface:
def sendText(self, text, destinationId=BROADCAST_ADDR, wantAck=False, def sendText(self, text, destinationId=BROADCAST_ADDR, wantAck=False,
wantResponse=False, channelIndex=0, hopLimit=None): wantResponse=False, channelIndex=0, hopLimit=None):
if not isinstance(text, str):
raise TypeError("text must be a string")
return self.sendData(text.encode("utf-8"), destinationId, return self.sendData(text.encode("utf-8"), destinationId,
PortNum.TEXT_MESSAGE_APP, wantAck, wantResponse, PortNum.TEXT_MESSAGE_APP, wantAck, wantResponse,
channelIndex, hopLimit) channelIndex, hopLimit)
@@ -109,10 +118,15 @@ class SerialInterface:
def sendPosition(self, latitude, longitude, altitude=0, def sendPosition(self, latitude, longitude, altitude=0,
destinationId=BROADCAST_ADDR, wantAck=False, destinationId=BROADCAST_ADDR, wantAck=False,
channelIndex=0, hopLimit=None): channelIndex=0, hopLimit=None):
latitude = float(latitude)
longitude = float(longitude)
if not -90 <= latitude <= 90:
raise ValueError("latitude must be between -90 and 90")
if not -180 <= longitude <= 180:
raise ValueError("longitude must be between -180 and 180")
position = Position(latitude_i=int(latitude * 10000000), position = Position(latitude_i=int(latitude * 10000000),
longitude_i=int(longitude * 10000000)) longitude_i=int(longitude * 10000000),
if altitude: altitude=int(altitude))
position.altitude = int(altitude)
return self.sendData(position, destinationId, PortNum.POSITION_APP, return self.sendData(position, destinationId, PortNum.POSITION_APP,
wantAck, False, channelIndex, hopLimit) wantAck, False, channelIndex, hopLimit)
+18 -7
View File
@@ -17,7 +17,8 @@ class Position(ProtoMessage):
altitude_hae=Field(9, "sint", optional=True), altitude_hae=Field(9, "sint", optional=True),
altitude_geoidal_separation=Field(10, "sint", optional=True), altitude_geoidal_separation=Field(10, "sint", optional=True),
PDOP=Field(11), HDOP=Field(12), VDOP=Field(13), gps_accuracy=Field(14), PDOP=Field(11), HDOP=Field(12), VDOP=Field(13), gps_accuracy=Field(14),
ground_speed=Field(15, optional=True), ground_track=Field(16, optional=True), fix_quality=Field(17), ground_speed=Field(15, optional=True), ground_track=Field(16, optional=True),
fix_quality=Field(17),
fix_type=Field(18), sats_in_view=Field(19), sensor_id=Field(20), fix_type=Field(18), sats_in_view=Field(19), sensor_id=Field(20),
next_update=Field(21), seq_number=Field(22), precision_bits=Field(23), next_update=Field(21), seq_number=Field(22), precision_bits=Field(23),
) )
@@ -55,6 +56,12 @@ class Routing(ProtoMessage):
DUTY_CYCLE_LIMIT = 9 DUTY_CYCLE_LIMIT = 9
BAD_REQUEST = 32 BAD_REQUEST = 32
NOT_AUTHORIZED = 33 NOT_AUTHORIZED = 33
PKI_FAILED = 34
PKI_UNKNOWN_PUBKEY = 35
ADMIN_BAD_SESSION_KEY = 36
ADMIN_PUBLIC_KEY_UNAUTHORIZED = 37
RATE_LIMIT_EXCEEDED = 38
PKI_SEND_FAIL_PUBLIC_KEY = 39
FIELDS = fields( FIELDS = fields(
route_request=Field(1, "message", RouteDiscovery), route_request=Field(1, "message", RouteDiscovery),
@@ -68,6 +75,7 @@ class Data(ProtoMessage):
portnum=Field(1), payload=Field(2, "bytes"), want_response=Field(3, "bool"), portnum=Field(1), payload=Field(2, "bytes"), want_response=Field(3, "bool"),
dest=Field(4, "fixed32"), source=Field(5, "fixed32"), request_id=Field(6, "fixed32"), dest=Field(4, "fixed32"), source=Field(5, "fixed32"), request_id=Field(6, "fixed32"),
reply_id=Field(7, "fixed32"), emoji=Field(8, "fixed32"), bitfield=Field(9, optional=True), reply_id=Field(7, "fixed32"), emoji=Field(8, "fixed32"), bitfield=Field(9, optional=True),
xeddsa_signature=Field(10, "bytes"),
) )
@@ -91,11 +99,12 @@ class MeshPacket(ProtoMessage):
"encrypted": Field(5, "bytes"), "id": Field(6, "fixed32"), "encrypted": Field(5, "bytes"), "id": Field(6, "fixed32"),
"rx_time": Field(7, "fixed32", optional=True), "rx_snr": Field(8, "float"), "rx_time": Field(7, "fixed32", optional=True), "rx_snr": Field(8, "float"),
"hop_limit": Field(9), "want_ack": Field(10, "bool"), "hop_limit": Field(9), "want_ack": Field(10, "bool"),
"priority": Field(11), "rx_rssi": Field(12, "int", optional=True), "delayed": Field(13), "priority": Field(11), "rx_rssi": Field(12, "int", optional=True),
"delayed": Field(13),
"via_mqtt": Field(14, "bool"), "hop_start": Field(15), "via_mqtt": Field(14, "bool"), "hop_start": Field(15),
"public_key": Field(16, "bytes"), "pki_encrypted": Field(17, "bool"), "public_key": Field(16, "bytes"), "pki_encrypted": Field(17, "bool"),
"next_hop": Field(18), "relay_node": Field(19), "tx_after": Field(20), "next_hop": Field(18), "relay_node": Field(19), "tx_after": Field(20),
"transport_mechanism": Field(21), "transport_mechanism": Field(21), "xeddsa_signed": Field(22, "bool"),
} }
) )
ONEOFS = {"payload_variant": ("decoded", "encrypted")} ONEOFS = {"payload_variant": ("decoded", "encrypted")}
@@ -103,8 +112,9 @@ class MeshPacket(ProtoMessage):
class DeviceMetrics(ProtoMessage): class DeviceMetrics(ProtoMessage):
FIELDS = fields( FIELDS = fields(
battery_level=Field(1), voltage=Field(2, "float"), channel_utilization=Field(3, "float"), battery_level=Field(1, optional=True), voltage=Field(2, "float", optional=True),
air_util_tx=Field(4, "float"), uptime_seconds=Field(5), channel_utilization=Field(3, "float", optional=True),
air_util_tx=Field(4, "float", optional=True), uptime_seconds=Field(5, optional=True),
) )
@@ -113,9 +123,10 @@ class NodeInfo(ProtoMessage):
num=Field(1), user=Field(2, "message", User), position=Field(3, "message", Position), num=Field(1), user=Field(2, "message", User), position=Field(3, "message", Position),
snr=Field(4, "float"), last_heard=Field(5, "fixed32"), snr=Field(4, "float"), last_heard=Field(5, "fixed32"),
device_metrics=Field(6, "message", DeviceMetrics), channel=Field(7), device_metrics=Field(6, "message", DeviceMetrics), channel=Field(7),
via_mqtt=Field(8, "bool"), hops_away=Field(9, optional=True), is_favorite=Field(10, "bool"), via_mqtt=Field(8, "bool"), hops_away=Field(9, optional=True),
is_favorite=Field(10, "bool"),
is_ignored=Field(11, "bool"), is_key_manually_verified=Field(12, "bool"), is_ignored=Field(11, "bool"), is_key_manually_verified=Field(12, "bool"),
is_muted=Field(13, "bool"), is_muted=Field(13, "bool"), has_xeddsa_signed=Field(14, "bool"),
) )
+28 -7
View File
@@ -24,6 +24,8 @@ def encode_varint(value):
value = int(value) value = int(value)
if value < 0: if value < 0:
value &= 0xFFFFFFFFFFFFFFFF value &= 0xFFFFFFFFFFFFFFFF
if value > 0xFFFFFFFFFFFFFFFF:
raise ValueError("varint exceeds 64 bits")
out = bytearray() out = bytearray()
while value > 0x7F: while value > 0x7F:
out.append((value & 0x7F) | 0x80) out.append((value & 0x7F) | 0x80)
@@ -39,6 +41,8 @@ def decode_varint(buf, offset=0):
while offset < length and shift < 70: while offset < length and shift < 70:
byte = buf[offset] byte = buf[offset]
offset += 1 offset += 1
if shift == 63 and byte > 1:
raise DecodeError("invalid 64-bit varint")
value |= (byte & 0x7F) << shift value |= (byte & 0x7F) << shift
if not byte & 0x80: if not byte & 0x80:
return value, offset return value, offset
@@ -65,7 +69,8 @@ def _skip(buf, offset, wire):
class Field: class Field:
def __init__(self, number, kind="uint", message=None, repeated=False, packed=False, optional=False): def __init__(self, number, kind="uint", message=None, repeated=False,
packed=False, optional=False):
self.number = number self.number = number
self.kind = kind self.kind = kind
self.message = message self.message = message
@@ -153,9 +158,13 @@ class ProtoMessage:
self._values[name] = value self._values[name] = value
def HasField(self, name): def HasField(self, name):
if name.endswith("_") and name[:-1] in type(self).FIELDS:
name = name[:-1]
return name in self._values return name in self._values
def ClearField(self, name): def ClearField(self, name):
if name.endswith("_") and name[:-1] in type(self).FIELDS:
name = name[:-1]
self._values.pop(name, None) self._values.pop(name, None)
def WhichOneof(self, name): def WhichOneof(self, name):
@@ -174,26 +183,30 @@ class ProtoMessage:
continue continue
value = self._values[name] value = self._values[name]
in_oneof = any(name in members for members in type(self).ONEOFS.values()) in_oneof = any(name in members for members in type(self).ONEOFS.values())
if not in_oneof and not field.optional and not field.repeated and field.kind != "message": implicit_scalar = (
not in_oneof and not field.optional and not field.repeated
and field.kind != "message"
)
if implicit_scalar:
if value in (0, False, b"", "", 0.0): if value in (0, False, b"", "", 0.0):
continue continue
values = value if field.repeated else (value,) values = value if field.repeated else (value,)
if field.packed and values: if field.packed and values:
body = bytearray() body = bytearray()
for item in values: for item in values:
body.extend(self._encode_scalar(field, item, include_tag=False)) body.extend(self._encode_scalar(field, item))
out.extend(encode_varint((field.number << 3) | BYTES)) out.extend(encode_varint((field.number << 3) | BYTES))
out.extend(encode_varint(len(body))) out.extend(encode_varint(len(body)))
out.extend(body) out.extend(body)
else: else:
for item in values: for item in values:
out.extend(encode_varint((field.number << 3) | field.wire)) out.extend(encode_varint((field.number << 3) | field.wire))
out.extend(self._encode_scalar(field, item, include_tag=False)) out.extend(self._encode_scalar(field, item))
for raw in self._unknown: for raw in self._unknown:
out.extend(raw) out.extend(raw)
return bytes(out) return bytes(out)
def _encode_scalar(self, field, value, include_tag=False): def _encode_scalar(self, field, value):
kind = field.kind kind = field.kind
if kind == "message": if kind == "message":
data = value.SerializeToString() data = value.SerializeToString()
@@ -225,6 +238,8 @@ class ProtoMessage:
start = offset start = offset
tag, offset = decode_varint(data, offset) tag, offset = decode_varint(data, offset)
number, wire = tag >> 3, tag & 7 number, wire = tag >> 3, tag & 7
if number == 0:
raise DecodeError("invalid protobuf field number: 0")
entry = by_number.get(number) entry = by_number.get(number)
if entry is None: if entry is None:
offset = _skip(data, offset, wire) offset = _skip(data, offset, wire)
@@ -238,7 +253,10 @@ class ProtoMessage:
raise DecodeError("truncated length-delimited field") raise DecodeError("truncated length-delimited field")
raw = data[offset:end] raw = data[offset:end]
offset = end offset = end
if field.packed and field.wire != BYTES: if field.wire != BYTES:
if not field.packed:
self._unknown.append(data[start:offset])
continue
values = self._values.setdefault(name, []) values = self._values.setdefault(name, [])
inner = 0 inner = 0
while inner < len(raw): while inner < len(raw):
@@ -303,7 +321,10 @@ class ProtoMessage:
if isinstance(value, ProtoMessage): if isinstance(value, ProtoMessage):
value = value.to_dict() value = value.to_dict()
elif isinstance(value, list): elif isinstance(value, list):
value = [item.to_dict() if isinstance(item, ProtoMessage) else item for item in value] value = [
item.to_dict() if isinstance(item, ProtoMessage) else item
for item in value
]
result[name] = value result[name] = value
return result return result
+2
View File
@@ -13,6 +13,8 @@ license-files = ["LICENSE"]
authors = [{name = "MicroMesh contributors"}] authors = [{name = "MicroMesh contributors"}]
keywords = ["meshtastic", "micropython", "lora", "protobuf"] keywords = ["meshtastic", "micropython", "lora", "protobuf"]
classifiers = [ classifiers = [
"Development Status :: 3 - Alpha",
"Operating System :: OS Independent",
"Programming Language :: Python :: 3", "Programming Language :: Python :: 3",
"Programming Language :: Python :: Implementation :: MicroPython", "Programming Language :: Python :: Implementation :: MicroPython",
"Topic :: Communications :: Ham Radio", "Topic :: Communications :: Ham Radio",
Binary file not shown.
Binary file not shown.
+51 -3
View File
@@ -1,6 +1,9 @@
import unittest import unittest
from micromesh import Data, FromRadio, MeshPacket, PortNum, Position, SerialInterface, ToRadio from micromesh import (
Data, DeviceMetrics, FromRadio, MeshPacket, PortNum, Position,
SerialInterface, ToRadio, node_num,
)
from micromesh.protobuf import DecodeError, decode_varint, encode_varint from micromesh.protobuf import DecodeError, decode_varint, encode_varint
from micromesh.stream import StreamParser, frame from micromesh.stream import StreamParser, frame
@@ -30,10 +33,17 @@ class MicroMeshTests(unittest.TestCase):
encoded = encode_varint(value) encoded = encode_varint(value)
self.assertEqual(decode_varint(encoded), (value, len(encoded))) self.assertEqual(decode_varint(encoded), (value, len(encoded)))
def test_varint_rejects_values_wider_than_64_bits(self):
with self.assertRaises(ValueError):
encode_varint(1 << 64)
with self.assertRaises(DecodeError):
decode_varint(b"\xff" * 9 + b"\x02")
def test_known_data_wire_bytes(self): def test_known_data_wire_bytes(self):
message = Data(portnum=PortNum.TEXT_MESSAGE_APP, payload=b"hello", want_response=True) message = Data(portnum=PortNum.TEXT_MESSAGE_APP, payload=b"hello", want_response=True)
self.assertEqual(message.SerializeToString(), b"\x08\x01\x12\x05hello\x18\x01") self.assertEqual(message.SerializeToString(), b"\x08\x01\x12\x05hello\x18\x01")
self.assertEqual(Data().ParseFromString(message.SerializeToString()).to_dict(), message.to_dict()) decoded = Data().ParseFromString(message.SerializeToString())
self.assertEqual(decoded.to_dict(), message.to_dict())
def test_generated_style_imports(self): def test_generated_style_imports(self):
from micromesh import mesh_pb2, portnums_pb2 from micromesh import mesh_pb2, portnums_pb2
@@ -41,10 +51,17 @@ class MicroMeshTests(unittest.TestCase):
self.assertEqual(portnums_pb2.PortNum.TEXT_MESSAGE_APP, 1) self.assertEqual(portnums_pb2.PortNum.TEXT_MESSAGE_APP, 1)
def test_proto3_scalar_defaults_are_omitted(self): def test_proto3_scalar_defaults_are_omitted(self):
self.assertEqual(Data(portnum=0, payload=b"", want_response=False).SerializeToString(), b"") empty = Data(portnum=0, payload=b"", want_response=False)
self.assertEqual(empty.SerializeToString(), b"")
self.assertEqual(ToRadio(disconnect=False).SerializeToString(), b"\x20\x00") self.assertEqual(ToRadio(disconnect=False).SerializeToString(), b"\x20\x00")
self.assertEqual(Position(latitude_i=0).SerializeToString(), b"\x0d\x00\x00\x00\x00") self.assertEqual(Position(latitude_i=0).SerializeToString(), b"\x0d\x00\x00\x00\x00")
def test_optional_device_metric_zero_is_preserved(self):
metrics = DeviceMetrics(battery_level=0, voltage=0.0)
decoded = DeviceMetrics().ParseFromString(metrics.SerializeToString())
self.assertTrue(decoded.HasField("battery_level"))
self.assertTrue(decoded.HasField("voltage"))
def test_mesh_packet_fixed_fields_and_nested_message(self): def test_mesh_packet_fixed_fields_and_nested_message(self):
packet = MeshPacket(**{"from": 0x12345678, "to": 0xFFFFFFFF, "id": 42}) packet = MeshPacket(**{"from": 0x12345678, "to": 0xFFFFFFFF, "id": 42})
packet.decoded.portnum = PortNum.TEXT_MESSAGE_APP packet.decoded.portnum = PortNum.TEXT_MESSAGE_APP
@@ -55,18 +72,45 @@ class MicroMeshTests(unittest.TestCase):
self.assertEqual(decoded.id, 42) self.assertEqual(decoded.id, 42)
self.assertEqual(decoded.decoded.payload, b"hi") self.assertEqual(decoded.decoded.payload, b"hi")
def test_current_signature_fields_round_trip(self):
data = Data(xeddsa_signature=b"signature")
packet = MeshPacket(decoded=data, xeddsa_signed=True)
decoded = MeshPacket().ParseFromString(packet.SerializeToString())
self.assertEqual(decoded.decoded.xeddsa_signature, b"signature")
self.assertTrue(decoded.xeddsa_signed)
def test_signed_position(self): def test_signed_position(self):
original = Position(latitude_i=-1220000000, longitude_i=455000000, altitude=-12) original = Position(latitude_i=-1220000000, longitude_i=455000000, altitude=-12)
decoded = Position().ParseFromString(original.SerializeToString()) decoded = Position().ParseFromString(original.SerializeToString())
self.assertEqual(decoded.latitude_i, -1220000000) self.assertEqual(decoded.latitude_i, -1220000000)
self.assertEqual(decoded.altitude, -12) self.assertEqual(decoded.altitude, -12)
def test_send_position_keeps_zero_altitude_and_validates_coordinates(self):
uart = FakeUART()
packet = SerialInterface(uart).sendPosition(0, 0, altitude=0)
self.assertTrue(Position().ParseFromString(packet.decoded.payload).HasField("altitude"))
with self.assertRaises(ValueError):
SerialInterface(FakeUART()).sendPosition(91, 0)
def test_node_numbers_must_fit_fixed32(self):
self.assertEqual(node_num("!ffffffff"), 0xFFFFFFFF)
with self.assertRaises(ValueError):
node_num(-1)
with self.assertRaises(ValueError):
node_num(1 << 32)
def test_unknown_fields_are_preserved(self): def test_unknown_fields_are_preserved(self):
raw = b"\x08\x01" + encode_varint(99 << 3) + b"\x07" raw = b"\x08\x01" + encode_varint(99 << 3) + b"\x07"
parsed = Data().ParseFromString(raw) parsed = Data().ParseFromString(raw)
self.assertEqual(parsed.portnum, 1) self.assertEqual(parsed.portnum, 1)
self.assertEqual(parsed.SerializeToString(), raw) self.assertEqual(parsed.SerializeToString(), raw)
def test_known_field_with_wrong_wire_type_is_preserved_as_unknown(self):
raw = b"\x0a\x01x"
parsed = Data().ParseFromString(raw)
self.assertFalse(parsed.HasField("portnum"))
self.assertEqual(parsed.SerializeToString(), raw)
def test_stream_parser_handles_chunks_logs_and_resync(self): def test_stream_parser_handles_chunks_logs_and_resync(self):
logs = [] logs = []
parser = StreamParser(on_log=logs.append) parser = StreamParser(on_log=logs.append)
@@ -132,6 +176,10 @@ class MicroMeshTests(unittest.TestCase):
with self.assertRaises(DecodeError): with self.assertRaises(DecodeError):
Data().ParseFromString(b"\x12\x05no") Data().ParseFromString(b"\x12\x05no")
def test_field_number_zero_raises(self):
with self.assertRaises(DecodeError):
Data().ParseFromString(b"\x00\x00")
if __name__ == "__main__": if __name__ == "__main__":
unittest.main() unittest.main()