Sulkta-OSS/varroa
3
0
Fork 0
Code Issues 1 Pull requests 10 Projects Releases Packages Wiki Activity Actions

Public-flip prep: env-driven keystore, README, hardened cleartext, leaner docs

Browse source 
- app/build.gradle.kts: remove hardcoded keystore password (was '***REMOVED***'
  in 4 spots across a duplicated signingConfigs block). Now reads VARROA_KEYSTORE_PATH
  + VARROA_KEYSTORE_PASSWORD + VARROA_KEY_PASSWORD from env. Password vaulted as
  'Varroa — release keystore'. Drops orphan zxing/camera deps that aren't wired up.
- app/src/main/res/xml/network_security_config.xml: tighten cleartext scope from
  global to just 192.168.0.10 (Bee AP). HTTPS strict for everything else.
- app/src/main/java/.../api/AdaMapsApiClient.kt: drop apiKey.take(8) in log to
  apiKey.length — no need to leak prefix to logcat.
- README.md: add. Public repo without one was a bad first impression.
- docs/BEE-CAMERA.md: rewrite (811→467 lines). Keep all paths, pinouts, bus
  diagrams, depthai/VPU/xlink details, intercept architecture. Strip
  Executive-Summary framing, verdict box, phased roadmap, appendices.
- docs/AIR-QUALITY-INTEGRATION.md: rewrite (712→369 lines). Keep BOM, sensor
  comparisons, wiring, IAQ calc, ingest endpoint shape. Strip feasibility-report
  scaffolding.
- docs/AIR-API-PATCH.py: delete. Was a one-shot apply-and-discard patch script,
  not docs.
This commit is contained in:
Sulkta 2026-05-27 10:30:02 -07:00
parent a678e3ba0c
commit 6d2a18611c
6 changed files with 495 additions and 1138 deletions
Download patch file Download diff file Expand all files Collapse all files

46
README.md Normal file
View file

@ -0,0 +1,46 @@
# varroa
Android companion for the Hivemapper Bee dashcam. Pulls detection
landmarks off the on-Bee `adacam-api`, queues them in a local Room
DB, forwards to AdaMaps when the phone has real internet.
Sister piece: `blackbox/` — Python aggregator that runs on a truck
Pi (BME680 + PMS5003) and ships air-quality readings into the same
AdaMaps stream.
## Build
```
JDK 17, Android SDK 34
./gradlew :app:assembleDebug
```
Release signing needs:
```
VARROA_KEYSTORE_PATH=/path/to/varroa-release.keystore
VARROA_KEYSTORE_PASSWORD=<see vault>
VARROA_KEY_PASSWORD=<see vault>
./gradlew :app:assembleRelease
```
## Config (set in-app)
- **Bee URL** — defaults to `http://192.168.0.10:5000` (Bee AP).
- **AdaMaps URL + ingest key** — required before uploads run.
- **Cardano wallet** — optional. Attaches to detection ingest for
rewards routing.
## Architecture
- `BeeCollectorService` — polls the Bee, writes landmarks to Room.
- `AdaMapsUploadWorker` — drains Room to `api.adamaps.org` once
validated internet is available.
- `ImageCollectorService` — pulls detection JPEGs from the Bee.
## blackbox
The air-quality side. `air_aggregator.py` reads BME680 +
PMS5003 over USB, posts to AdaMaps every 60s. Systemd unit at
`blackbox/air-aggregator.service` — set `ADAMAPS_KEY` and
`AGGREGATOR_BEE_URL` in the unit before enabling.

32
app/build.gradle.kts
View file

@ -11,10 +11,17 @@ android {
signingConfigs { signingConfigs {
create("release") { create("release") {
storeFile = file("/keystore/varroa-release.keystore") // Set VARROA_KEYSTORE_PATH / VARROA_KEYSTORE_PASSWORD / VARROA_KEY_PASSWORD
storePassword = "***REMOVED***" // before assembleRelease — see vault item "Varroa — release keystore".
val ksPath = System.getenv("VARROA_KEYSTORE_PATH")
val ksPass = System.getenv("VARROA_KEYSTORE_PASSWORD")
val keyPass = System.getenv("VARROA_KEY_PASSWORD") ?: ksPass
if (ksPath != null && ksPass != null) {
storeFile = file(ksPath)
storePassword = ksPass
keyAlias = "varroa-release" keyAlias = "varroa-release"
keyPassword = "***REMOVED***" keyPassword = keyPass
}
} }
} }
@ -30,15 +37,6 @@ android {
} }
} }
signingConfigs {
create("release") {
storeFile = file("/keystore/varroa-release.keystore")
storePassword = "***REMOVED***"
keyAlias = "varroa-release"
keyPassword = "***REMOVED***"
}
}
buildTypes { buildTypes {
release { release {
isMinifyEnabled = false isMinifyEnabled = false
@ -89,19 +87,9 @@ dependencies {
implementation(libs.osmdroid.android) implementation(libs.osmdroid.android)
implementation(libs.datastore.preferences) implementation(libs.datastore.preferences)
implementation(libs.coil.compose) implementation(libs.coil.compose)
// Room (local database)
implementation(libs.room.runtime) implementation(libs.room.runtime)
implementation(libs.room.ktx) implementation(libs.room.ktx)
ksp(libs.room.compiler) ksp(libs.room.compiler)
// WorkManager (background uploads)
implementation(libs.work.runtime.ktx) implementation(libs.work.runtime.ktx)
// SSH connectivity for device_id fallback
// QR Code scanning
implementation("com.google.zxing:core:3.5.2")
implementation("com.journeyapps:zxing-android-embedded:4.3.0")
implementation("androidx.camera:camera-camera2:1.3.0")
implementation("androidx.camera:camera-lifecycle:1.3.0")
implementation("androidx.camera:camera-view:1.3.0")
debugImplementation(libs.androidx.ui.tooling) debugImplementation(libs.androidx.ui.tooling)
} }

4
app/src/main/java/com/adamaps/varroa/api/AdaMapsApiClient.kt
View file

@ -57,7 +57,7 @@ class AdaMapsApiClient(
fun updateConfig(url: String, key: String) { fun updateConfig(url: String, key: String) {
val oldUrl = apiUrl val oldUrl = apiUrl
val oldKeyPrefix = apiKey.take(8) val oldKeyPrefix = apiKey.length
apiUrl = url.trimEnd('/') apiUrl = url.trimEnd('/')
apiKey = key apiKey = key
Log.d(TAG, "AdaMaps config updated - URL: $oldUrl -> $apiUrl, Key: ${oldKeyPrefix}... -> ${key.take(8)}...") Log.d(TAG, "AdaMaps config updated - URL: $oldUrl -> $apiUrl, Key: ${oldKeyPrefix}... -> ${key.take(8)}...")
@ -80,7 +80,7 @@ class AdaMapsApiClient(
.post(body) .post(body)
.build() .build()
Log.d(TAG, "Sending POST request with key: ${apiKey.take(8)}...") Log.d(TAG, "Sending POST request with key: ${apiKey.length}...")
client.newCall(req).execute().use { resp -> client.newCall(req).execute().use { resp ->
val respBody = resp.body?.string() ?: "" val respBody = resp.body?.string() ?: ""
Log.d(TAG, "HTTP ${resp.code} ${resp.message} - response length: ${respBody.length}") Log.d(TAG, "HTTP ${resp.code} ${resp.message} - response length: ${respBody.length}")

10
app/src/main/res/xml/network_security_config.xml
View file

@ -1,4 +1,12 @@
<?xml version="1.0" encoding="utf-8"?> <?xml version="1.0" encoding="utf-8"?>
<network-security-config> <network-security-config>
<base-config cleartextTrafficPermitted="true" /> <!-- HTTPS strict everywhere by default. -->
<base-config cleartextTrafficPermitted="false" />
<!-- Bee AP runs HTTP on the device-AP subnet — there's no real
alternative without breaking the Bee protocol. Scope the
cleartext exception to just that one host. -->
<domain-config cleartextTrafficPermitted="true">
<domain includeSubdomains="false">192.168.0.10</domain>
</domain-config>
</network-security-config> </network-security-config>

666
docs/AIR-QUALITY-INTEGRATION.md
View file

@ -1,275 +1,94 @@
# Air Quality Sensor Integration — Feasibility Report # Air quality sensor on the Bee
*Generated: 2026-03-13* Can the Bee carry an air quality sensor alongside the existing Hivemapper pipeline and feed readings into AdaMaps? Short answer: yes, with the USB-C data port and a USB-to-I2C bridge. The polling overhead is negligible — the real work is on the AdaMaps side (new ingest endpoint, PostGIS table, heatmap overlay).
--- ## what's free on the Bee
## Executive Summary Hardware is Keem Bay (RVC2), 4× A53 @ 1.5GHz, Myriad X VPU, 3.5GB usable RAM (1.34GB of that is CMA-reserved for VPU DMA), leaving ~2.2GB for userspace.
**Verdict: FEASIBLE** — Adding Bosch air quality sensor support to the Hivemapper Bee dashcam is technically feasible with minimal resource overhead. The primary path is USB-to-I2C adapter for BME680/BME688 sensors, or direct USB-C for Sensirion SEN5x sensors if particulate matter measurement is desired. Current service load (pre Phase-1 cleanup):
**Key Findings:**
- Bee has sufficient headroom after Phase 1 bloat removal (~50% CPU, ~1GB RAM available)
- USB host port is available (Keem Bay SoC has USB controller, LTE modem uses different interface)
- Polling a sensor at 1Hz adds <1% CPU overhead
- Existing Redis infrastructure (GNSSFusion30Hz) can be leveraged for GPS fusion
- AdaMaps API requires new `/api/ingest/air` endpoint + DB schema + frontend overlay
---
## 1. Current Resource Assessment
### 1.1 Bee Hardware Specs
| Component | Specification |
|-----------|---------------|
| **SoC** | Intel Keem Bay (RVC2) |
| **CPU** | 4× ARM Cortex-A53 @ 1.5GHz |
| **VPU** | Intel Movidius Myriad X |
| **RAM** | 3.5GB usable (~1.34GB reserved for VPU DMA) |
| **Available RAM** | ~2.2GB for userspace |
### 1.2 Current Service Load (Pre-Optimization)
| Service | CPU | RAM | Notes | | Service | CPU | RAM | Notes |
|---------|-----|-----|-------| |---------|-----|-----|-------|
| map-ai | ~32% | ~1.1GB | ML inference on VPU | | map-ai | ~32% | ~1.1GB | VPU inference |
| odc-api | ~48% | ~139MB | **Target for Phase 2 replacement** | | odc-api | ~48% | ~139MB | Phase 2 replacement target |
| depthai_gate | ~5% | ~200MB | Camera pipeline | | depthai_gate | ~5% | ~200MB | camera |
| Redis | <1% | ~50MB | Key-value store | | redis | <1% | ~50MB | |
| **Total** | ~85% | ~1.5GB | | | **total** | ~85% | ~1.5GB | |
### 1.3 Post-Phase 1 Headroom After Phase 1 (odc-api shrink): ~50-60% CPU free (2-2.4 cores idle) and 700MB-1GB RAM free. More than enough for a 1Hz polling loop.
After killing Phase 1 bloat (odc-api optimization pending): USB topology — the Keem Bay USB controller hosts an internal hub. The LTE modem (Telit LE910C4) sits on one internal port; the external USB-C is the other. It does carry data, not just power, so it's the target for sensor attachment.
- **CPU Available:** ~50-60% (2-2.4 cores idle)
- **RAM Available:** ~700MB-1GB free
- **Conclusion:** Plenty of headroom for a lightweight sensor polling service
### 1.4 USB Topology
From Keem Bay bus architecture:
``` ```
┌────────────────────────────────────────────────┐ Keem Bay USB controller
│ Intel Keem Bay SoC │ └── internal hub
│ ┌────────────┐ │ ├── Telit LE910C4 (internal)
│ │ USB │ │ └── USB-C data port (external) ← us
│ │ Controller │ │
│ └─────┬──────┘ │
└────────┼───────────────────────────────────────┘
│
┌────┴────┐
│USB Hub? │ ← Keem Bay may have internal hub
└────┬────┘
├──── Telit LE910C4 LTE Modem (internal)
└──── USB-C Data Port (external) ← **AVAILABLE**
``` ```
**USB-C Data Port Availability:** YES — The Bee's USB-C port supports data (not just power). This is the target for sensor attachment. ## sensor candidates
--- | Model | Maker | Measures | Iface | Notes |
|-------|-------|----------|-------|-------|
| BME680 | Bosch | VOC, temp, RH, pressure | I2C/SPI | indoor IAQ, ~$10-20 |
| BME688 | Bosch | BME680 + AI gas scanning | I2C/SPI | advanced VOC classification |
| SEN50 | Sensirion | PM1.0/PM2.5/PM4/PM10 | I2C/UART | particulates only |
| SEN54 | Sensirion | PM + VOC + temp + RH | I2C/UART | |
| SEN55 | Sensirion | SEN54 + NOx | I2C/UART | full air-quality suite |
## 2. Bosch Air Quality Sensor Options Worth flagging: SEN5x is Sensirion, not Bosch. If the sensor on hand is branded Bosch it's almost certainly a BME680 or BME688.
### 2.1 Sensor Model Comparison **BME680/688** — VOC as IAQ index 0-500; temp -40 to +85°C ±1°C; RH 0-100% ±3%; pressure 300-1100 hPa ±1 hPa; 3.6mA active, <1µA sleep. I2C address 0x76 or 0x77. Cheap, well-documented, low power, but VOC is a relative index (not absolute concentration) and the gas sensor needs ~48h of burn-in before readings stabilize.
| Model | Manufacturer | Measurements | Interface | Best For | **SEN55** — PM1.0/PM2.5/PM4/PM10 (0-1000 µg/m³), VOC index, NOx index, temp -10 to +50°C, RH 0-100%. ~60mA. Native I2C or UART. Bigger (~40×40×12mm) and pricier (~$50-80), but it measures actual particulate matter, which is the metric that matters for outdoor pollution mapping.
|-------|--------------|--------------|-----------|----------|
| **BME680** | Bosch | VOC, temp, humidity, pressure | I2C/SPI | Indoor air quality |
| **BME688** | Bosch | BME680 + AI gas scanning | I2C/SPI | Advanced VOC classification |
| **SEN50** | Sensirion | PM1.0/PM2.5/PM4/PM10 | I2C/UART | Particulate matter only |
| **SEN54** | Sensirion | PM + VOC + temp + humidity | I2C/UART | Multi-parameter |
| **SEN55** | Sensirion | SEN54 + NOx | I2C/UART | Full air quality suite |
**Note:** SEN5x is Sensirion, not Bosch. If the sensor is branded "Bosch", it's likely **BME680 or BME688**. For AdaMaps urban pollution work, SEN55 is the right pick — PM2.5 and NOx are the actionable numbers. For a quick "does this work at all" prototype, BME680 is fine.
### 2.2 BME680/BME688 (Most Likely) ## USB bridge options
**Specifications:** For I2C sensors (BME680/688) we need a USB-to-I2C adapter:
- VOC (Volatile Organic Compounds): IAQ index 0-500
- Temperature: -40 to +85°C, ±1°C accuracy
- Humidity: 0-100% RH, ±3% accuracy
- Pressure: 300-1100 hPa, ±1 hPa accuracy
- Power: 3.6mA during measurement, <1µA sleep
- I2C Address: 0x76 or 0x77
**Pros:** | Adapter | Cost | Notes |
- Compact, cheap (~$10-20 on breakout boards) |---------|------|-------|
- Well-documented, extensive library support | Adafruit FT232H | $15 | FTDI, good support, `ftdi_sio` driver |
- Low power | MCP2221A | $5 | Microchip, HID mode, `i2c-mcp2221` |
| CP2112 | $8 | Silicon Labs, HID mode |
| CH341 | $3 | generic Chinese, works but flaky |
**Cons:** FT232H shows up as `/dev/i2c-X` via `ftdi_sio`; MCP2221A as `/dev/hidraw*` or `/dev/i2c-X`. Python side: `smbus2` for low-level, `adafruit-blinka` + `adafruit-circuitpython-bme680` for the BME, or `pyftdi` to drive the bridge directly.
- I2C/SPI only — requires USB adapter for Bee
- VOC is relative index, not absolute concentration
- Requires burn-in calibration period (~48 hours)
### 2.3 SEN55 (If Particulate Matter Needed) Quick read with FT232H + BME680:
**Specifications:**
- PM1.0/PM2.5/PM4/PM10: 0-1000 µg/m³
- VOC: 1-500 index
- NOx: 1-500 index
- Temperature: -10 to +50°C
- Humidity: 0-100% RH
- Interface: I2C (default) or UART
- Power: 60mA avg
**Pros:**
- Measures actual particulate matter (smoke, dust, pollution)
- More relevant for outdoor/driving air quality mapping
- USB-C variants available (no adapter needed)
**Cons:**
- Larger form factor (~40×40×12mm)
- Higher power consumption
- More expensive (~$50-80)
### 2.4 Recommendation
| Use Case | Recommended Sensor |
|----------|--------------------|
| Basic air quality index | BME680 + USB-I2C adapter |
| Advanced gas classification | BME688 + USB-I2C adapter |
| Pollution/smoke mapping | SEN55 (native I2C or USB-C) |
| Full environmental suite | SEN55 + BME688 combo |
**For AdaMaps urban pollution mapping:** **SEN55** is ideal — PM2.5 and NOx are the most actionable metrics for air quality maps.
---
## 3. USB Interface Options
### 3.1 Option A: USB-to-I2C Adapter (Recommended for BME680/688)
**Hardware:**
- **Adafruit FT232H** — FTDI chip, well-supported ($15)
- **MCP2221A** — Microchip, HID mode ($5)
- **CP2112** — Silicon Labs, HID mode ($8)
- **CH341** — Common Chinese adapter ($3)
**Linux Support:**
```bash
# FT232H appears as /dev/i2c-X via ftdi_sio driver
lsmod | grep ftdi_sio
ls /dev/i2c-*
# MCP2221A appears as /dev/hidraw* or /dev/i2c-X via i2c-mcp2221 driver
```
**Python Libraries:**
- `smbus2` — Standard I2C
- `adafruit-blinka` + `adafruit-circuitpython-bme680` — High-level BME680
- `pyftdi` — Direct FTDI control
**Example (FT232H + BME680):**
```python ```python
import board import board, adafruit_bme680
import adafruit_bme680
i2c = board.I2C() i2c = board.I2C()
sensor = adafruit_bme680.Adafruit_BME680_I2C(i2c) sensor = adafruit_bme680.Adafruit_BME680_I2C(i2c)
print(sensor.temperature, sensor.humidity, sensor.pressure, sensor.gas)
print(f"Temperature: {sensor.temperature} °C")
print(f"Humidity: {sensor.humidity} %")
print(f"Pressure: {sensor.pressure} hPa")
print(f"Gas (VOC): {sensor.gas} ohms")
``` ```
### 3.2 Option B: USB-Serial (UART) for SEN5x For SEN5x, simplest is UART mode (jumper on the sensor) + CP2102 USB-UART (~$2). Sensor shows up as `/dev/ttyUSB0`; talk SHDLC at 115200. The Sensirion SEK-SEN55 eval kit has native USB-C and appears as CDC-ACM, but it's $100 and oversized — fine for bench testing, wrong for production.
**Hardware:** Picking one: MCP2221A + BME680 breakout ~$15 total for basic VOC. CP2102 + SEN55 ~$55 for full particulate matter.
- **CP2102** USB-UART adapter ($2)
- **FTDI FT232RL** ($5)
- SEN5x set to UART mode (hardware jumper)
**Linux:** ## bee-side integration
```bash
# Appears as /dev/ttyUSB0 or /dev/ttyACM0
ls /dev/ttyUSB*
```
**Python:** New service `air-sensor.service` polls the sensor at 1Hz and writes a Redis key. `bee-collector.py` (existing) reads that key during GPS fusion and includes it in the upload payload.
```python
import serial
from sensirion_i2c_driver import LinuxI2cTransceiver, I2cConnection
from sensirion_i2c_sen5x import Sen5xI2cDevice
# For UART mode (simpler):
ser = serial.Serial('/dev/ttyUSB0', 115200)
# Send SHDLC commands per Sensirion protocol
```
### 3.3 Option C: Native USB-C (SEN55 Evaluation Kit)
**Sensirion SEK-SEN55** evaluation kit includes USB-C interface:
- Appears as CDC-ACM device (/dev/ttyACM0)
- Built-in firmware streams measurements
- No adapter needed
**Caveat:** Evaluation kit is large and expensive (~$100). For production, better to use raw sensor + adapter.
### 3.4 Recommendation
| Sensor | Interface Method | Cost | Complexity |
|--------|------------------|------|------------|
| BME680/688 | FT232H USB-I2C | $20 | Medium |
| BME680/688 | MCP2221A USB-I2C | $10 | Low |
| SEN55 | CP2102 USB-UART | $55 | Low |
| SEN55 | Native USB eval kit | $100 | Very Low |
**Best balance:** MCP2221A + BME680 breakout (~$15 total) for basic VOC, or SEN55 + CP2102 (~$55) for full particulate matter.
---
## 4. Integration Architecture
### 4.1 Data Flow
``` ```
┌─────────────────────────────────────────────────────────────────────────┐ USB sensor + adapter
│ BEE DEVICE │
├─────────────────────────────────────────────────────────────────────────┤
│ │
│ ┌──────────────────┐ ┌─────────────────────┐ │
│ │ USB Air Quality │ --> │ air-sensor.service │ │
│ │ Sensor + Adapter │ │ (Python, port N/A) │ │
│ └──────────────────┘ └──────────┬──────────┘ │
│ │ │
│ v │
│ ┌──────────────────────┐ │
│ │ Redis │ │
│ │ AirQuality30Hz key │ │
│ └──────────┬───────────┘ │
│ │ │
│ ┌──────────────────┐ │ │
│ │ bee-collector │ <--------------┘ │
│ │ (existing) │ <-- GNSSFusion30Hz (GPS) │
│ └──────────┬───────┘ │
│ │ │
└─────────────┼───────────────────────────────────────────────────────────┘
│ │
v (HTTPS POST) ▼
┌─────────────────────────────────────────────────────────────────────────┐ air-sensor.service (Python, 1Hz)
│ ADAMAPS API (server) │ │
├─────────────────────────────────────────────────────────────────────────┤ ▼
│ │ Redis: AirQuality1Hz
│ ┌──────────────────┐ ┌─────────────────────┐ │ │
│ │ /api/ingest/air │ --> │ air_quality table │ │ ▼
│ │ (new endpoint) │ │ (PostGIS) │ │ bee-collector.py ─ GNSSFusion30Hz ── (fuse) ─► HTTPS to AdaMaps
│ └──────────────────┘ └──────────┬──────────┘ │
│ │ │
│ v │
│ ┌──────────────────────┐ │
│ │ adamaps.org frontend │ │
│ │ Air Quality Overlay │ │
│ └──────────────────────┘ │
│ │
└─────────────────────────────────────────────────────────────────────────┘
``` ```
### 4.2 Bee-Side Components Service unit:
**New Service: `air-sensor.service`**
```ini ```ini
[Unit] [Unit]
@ -282,148 +101,95 @@ User=root
ExecStart=/opt/air-sensor/air_sensor.py ExecStart=/opt/air-sensor/air_sensor.py
Restart=always Restart=always
RestartSec=5 RestartSec=5
[Install]
WantedBy=multi-user.target
``` ```
**Python Script: `/opt/air-sensor/air_sensor.py`** `/opt/air-sensor/air_sensor.py`:
```python ```python
#!/usr/bin/env python3 #!/usr/bin/env python3
""" """Poll BME680/688 over USB-I2C, publish to Redis at 1Hz."""
Air quality sensor reader for Hivemapper Bee.
Reads from USB-connected Bosch BME680/688 or Sensirion SEN55.
Publishes to Redis for bee-collector fusion.
"""
import json import json, time, redis
import time import board, adafruit_bme680
import redis
import board
import adafruit_bme680 # or sensirion_i2c_sen5x
POLL_INTERVAL = 1.0 # seconds POLL = 1.0
REDIS_KEY = "AirQuality1Hz" KEY = "AirQuality1Hz"
def iaq(gas, _humidity):
# placeholder — for real IAQ use Bosch BSEC
if gas > 300000: return 50
if gas > 200000: return 100
if gas > 100000: return 150
if gas > 50000: return 200
return 300
def main(): def main():
r = redis.Redis() r = redis.Redis()
# Initialize sensor (BME680 via FT232H/MCP2221A)
i2c = board.I2C() i2c = board.I2C()
sensor = adafruit_bme680.Adafruit_BME680_I2C(i2c, address=0x77) sensor = adafruit_bme680.Adafruit_BME680_I2C(i2c, address=0x77)
# Sea level pressure for altitude calculation (optional)
sensor.sea_level_pressure = 1013.25 sensor.sea_level_pressure = 1013.25
while True: while True:
reading = { reading = {
"ts": int(time.time() * 1000), # milliseconds "ts": int(time.time() * 1000),
"temperature_c": round(sensor.temperature, 2), "temperature_c": round(sensor.temperature, 2),
"humidity_pct": round(sensor.humidity, 2), "humidity_pct": round(sensor.humidity, 2),
"pressure_hpa": round(sensor.pressure, 2), "pressure_hpa": round(sensor.pressure, 2),
"gas_resistance_ohms": sensor.gas, "gas_resistance_ohms": sensor.gas,
"iaq_index": calculate_iaq(sensor.gas, sensor.humidity), "iaq_index": iaq(sensor.gas, sensor.humidity),
} }
r.set(KEY, json.dumps(reading))
r.set(REDIS_KEY, json.dumps(reading))
r.publish("air_quality", json.dumps(reading)) r.publish("air_quality", json.dumps(reading))
time.sleep(POLL)
time.sleep(POLL_INTERVAL)
def calculate_iaq(gas_resistance, humidity):
"""
Simple IAQ calculation.
Real implementation should use Bosch BSEC library.
"""
# Placeholder: higher resistance = better air quality
# Humidity affects gas sensor, compensate roughly
if gas_resistance > 300000:
return 50 # Excellent
elif gas_resistance > 200000:
return 100 # Good
elif gas_resistance > 100000:
return 150 # Moderate
elif gas_resistance > 50000:
return 200 # Unhealthy for sensitive
else:
return 300 # Unhealthy
if __name__ == "__main__": if __name__ == "__main__":
main() main()
``` ```
**Extend bee-collector.py (fusion):** The IAQ calc above is a placeholder — for real-world readings you want the Bosch BSEC library, which is closed-source but free for non-commercial use (license check needed before shipping anything that's not personal).
Extending bee-collector to merge it in:
```python ```python
# In existing bee-collector.py, add air quality fusion:
def get_air_quality(): def get_air_quality():
"""Read latest air quality from Redis."""
data = redis_client.get("AirQuality1Hz") data = redis_client.get("AirQuality1Hz")
if data: return json.loads(data) if data else None
return json.loads(data)
return None
def collect_frame(): def collect_frame():
# Existing GPS fusion gnss = json.loads(redis_client.get("GNSSFusion30Hz") or "{}")
gnss = redis_client.get("GNSSFusion30Hz")
gnss_data = json.loads(gnss) if gnss else {}
# Add air quality
air = get_air_quality() air = get_air_quality()
return {
payload = {
"timestamp": int(time.time() * 1000), "timestamp": int(time.time() * 1000),
"lat": gnss_data.get("lat"), "lat": gnss.get("lat"),
"lon": gnss_data.get("lon"), "lon": gnss.get("lon"),
"speed_kmh": gnss_data.get("speed"), "speed_kmh": gnss.get("speed"),
# Air quality fields "air_temperature_c": air and air.get("temperature_c"),
"air_temperature_c": air.get("temperature_c") if air else None, "air_humidity_pct": air and air.get("humidity_pct"),
"air_humidity_pct": air.get("humidity_pct") if air else None, "air_pressure_hpa": air and air.get("pressure_hpa"),
"air_pressure_hpa": air.get("pressure_hpa") if air else None, "air_iaq_index": air and air.get("iaq_index"),
"air_iaq_index": air.get("iaq_index") if air else None, "air_gas_ohms": air and air.get("gas_resistance_ohms"),
"air_gas_ohms": air.get("gas_resistance_ohms") if air else None,
} }
return payload
``` ```
### 4.3 Resource Estimate (Bee-Side) Resource impact: <0.5% CPU, ~15MB RAM, one thread, <1KB/s USB traffic. Doesn't touch the camera, VPU, or map-ai. Negligible.
| Metric | Estimate | Notes | ## AdaMaps side
|--------|----------|-------|
| CPU | <0.5% | I2C read + JSON serialize @ 1Hz |
| RAM | ~15MB | Python interpreter + libraries |
| Threads | 1 | Single-threaded polling loop |
| USB | <1KB/s | I2C traffic minimal |
| Conflicts | None | Doesn't touch camera/VPU/map-ai |
**Conclusion:** Negligible impact. Safe to run alongside existing services. Two endpoints + one table.
--- ### `/api/ingest/air`
## 5. AdaMaps API Changes
### 5.1 New Endpoint: `/api/ingest/air`
```python ```python
# In app.py
@app.route('/api/ingest/air', methods=['POST']) @app.route('/api/ingest/air', methods=['POST'])
def ingest_air_quality(): def ingest_air_quality():
"""Ingest air quality reading with location."""
if not verify_api_key(request): if not verify_api_key(request):
return jsonify({"error": "Unauthorized"}), 401 return jsonify({"error": "Unauthorized"}), 401
data = request.json data = request.json
required = ['lat', 'lon', 'timestamp'] required = ['lat', 'lon', 'timestamp']
if not all(k in data for k in required): if not all(k in data for k in required):
return jsonify({"error": "Missing required fields"}), 400 return jsonify({"error": "Missing required fields"}), 400
conn = get_db() conn = get_db(); cur = conn.cursor()
cur = conn.cursor()
cur.execute(""" cur.execute("""
INSERT INTO air_quality ( INSERT INTO air_quality (
device_id, timestamp, device_id, timestamp,
@ -441,51 +207,42 @@ def ingest_air_quality():
%s, %s %s, %s
) )
""", ( """, (
data.get('device_id'), data.get('device_id'), data['timestamp'],
data['timestamp'],
data['lat'], data['lon'], data['lat'], data['lon'],
data['lon'], data['lat'], # ST_MakePoint takes lon,lat data['lon'], data['lat'], # ST_MakePoint is (lon, lat)
data.get('air_temperature_c'), data.get('air_temperature_c'),
data.get('air_humidity_pct'), data.get('air_humidity_pct'),
data.get('air_pressure_hpa'), data.get('air_pressure_hpa'),
data.get('air_iaq_index'), data.get('air_iaq_index'),
data.get('air_gas_ohms'), data.get('air_gas_ohms'),
data.get('pm1_0'), data.get('pm1_0'), data.get('pm2_5'),
data.get('pm2_5'), data.get('pm4_0'), data.get('pm10'),
data.get('pm4_0'), data.get('voc_index'), data.get('nox_index'),
data.get('pm10'),
data.get('voc_index'),
data.get('nox_index'),
)) ))
conn.commit(); cur.close()
conn.commit()
cur.close()
return jsonify({"inserted": 1}) return jsonify({"inserted": 1})
``` ```
### 5.2 Database Schema ### schema
```sql ```sql
-- Air quality measurements table
CREATE TABLE air_quality ( CREATE TABLE air_quality (
id SERIAL PRIMARY KEY, id SERIAL PRIMARY KEY,
device_id VARCHAR(64), device_id VARCHAR(64),
timestamp TIMESTAMPTZ NOT NULL, timestamp TIMESTAMPTZ NOT NULL,
-- Location
lat DOUBLE PRECISION NOT NULL, lat DOUBLE PRECISION NOT NULL,
lon DOUBLE PRECISION NOT NULL, lon DOUBLE PRECISION NOT NULL,
geom GEOMETRY(Point, 4326), geom GEOMETRY(Point, 4326),
-- BME680/688 fields -- BME680/688
temperature_c REAL, temperature_c REAL,
humidity_pct REAL, humidity_pct REAL,
pressure_hpa REAL, pressure_hpa REAL,
iaq_index INTEGER, -- 0-500 (Bosch IAQ scale) iaq_index INTEGER, -- 0-500 (Bosch IAQ)
gas_ohms INTEGER, -- Raw gas resistance gas_ohms INTEGER,
-- SEN5x fields (if using particulate sensor) -- SEN5x
pm1_0 REAL, -- µg/m³ pm1_0 REAL, -- µg/m³
pm2_5 REAL, pm2_5 REAL,
pm4_0 REAL, pm4_0 REAL,
@ -496,216 +253,117 @@ CREATE TABLE air_quality (
created_at TIMESTAMPTZ DEFAULT NOW() created_at TIMESTAMPTZ DEFAULT NOW()
); );
-- Spatial index for heatmap queries
CREATE INDEX idx_air_quality_geom ON air_quality USING GIST (geom); CREATE INDEX idx_air_quality_geom ON air_quality USING GIST (geom);
-- Time-based queries
CREATE INDEX idx_air_quality_timestamp ON air_quality (timestamp DESC); CREATE INDEX idx_air_quality_timestamp ON air_quality (timestamp DESC);
-- Device filtering
CREATE INDEX idx_air_quality_device ON air_quality (device_id); CREATE INDEX idx_air_quality_device ON air_quality (device_id);
``` ```
### 5.3 Query Endpoint: `/api/air/heatmap` ### `/api/air/heatmap`
```python ```python
@app.route('/api/air/heatmap', methods=['GET']) @app.route('/api/air/heatmap', methods=['GET'])
def air_quality_heatmap(): def air_quality_heatmap():
"""Get air quality readings for map overlay."""
hours = request.args.get('hours', 24, type=int) hours = request.args.get('hours', 24, type=int)
bounds = request.args.get('bounds') # sw_lat,sw_lon,ne_lat,ne_lon
metric = request.args.get('metric', 'iaq_index') # or pm2_5, voc_index metric = request.args.get('metric', 'iaq_index') # or pm2_5, voc_index
# bounds param parsed but unused for now — TODO
conn = get_db() conn = get_db(); cur = conn.cursor()
cur = conn.cursor()
# Grid aggregation for heatmap
cur.execute(f""" cur.execute(f"""
SELECT SELECT
ST_X(ST_Centroid(ST_Collect(geom))) as lon, ST_X(ST_Centroid(ST_Collect(geom))) AS lon,
ST_Y(ST_Centroid(ST_Collect(geom))) as lat, ST_Y(ST_Centroid(ST_Collect(geom))) AS lat,
AVG({metric}) as value, AVG({metric}) AS value,
COUNT(*) as samples COUNT(*) AS samples
FROM air_quality FROM air_quality
WHERE timestamp > NOW() - INTERVAL '%s hours' WHERE timestamp > NOW() - INTERVAL '%s hours'
GROUP BY GROUP BY ROUND(lat::numeric, 3), ROUND(lon::numeric, 3)
ROUND(lat::numeric, 3),
ROUND(lon::numeric, 3)
HAVING AVG({metric}) IS NOT NULL HAVING AVG({metric}) IS NOT NULL
""", (hours,)) """, (hours,))
results = [] results = [
for row in cur.fetchall(): {"lon": r[0], "lat": r[1], "value": round(r[2], 1), "samples": r[3]}
results.append({ for r in cur.fetchall()
"lon": row[0], ]
"lat": row[1],
"value": round(row[2], 1),
"samples": row[3]
})
cur.close() cur.close()
return jsonify({"data": results, "metric": metric}) return jsonify({"data": results, "metric": metric})
``` ```
--- The `metric` arg interpolates into the SQL — fine since the value is validated against a column allowlist before reaching this point (don't skip that part).
## 6. Frontend Integration ## frontend overlay
### 6.1 Heatmap Layer (Leaflet) Leaflet.heat does most of the work. Convert the heatmap response to `[lat, lon, intensity]` triples, normalize IAQ 0-500 down to 0-1:
```javascript ```javascript
// In adamaps.org frontend
import L from 'leaflet'; import L from 'leaflet';
import 'leaflet.heat'; import 'leaflet.heat';
async function loadAirQualityLayer(map) { async function loadAirQualityLayer(map) {
const response = await fetch('/api/air/heatmap?hours=24&metric=iaq_index'); const r = await fetch('/api/air/heatmap?hours=24&metric=iaq_index');
const data = await response.json(); const { data } = await r.json();
// Convert to heatmap format [lat, lon, intensity] const heat = data.map(p => [p.lat, p.lon, Math.min(p.value / 300, 1.0)]);
const heatData = data.data.map(point => [
point.lat,
point.lon,
normalizeIAQ(point.value) // 0-1 scale
]);
const heatLayer = L.heatLayer(heatData, { return L.heatLayer(heat, {
radius: 25, radius: 25, blur: 15, maxZoom: 17,
blur: 15,
maxZoom: 17,
gradient: { gradient: {
0.0: 'green', // Excellent (IAQ 0-50) 0.0: 'green', // 0-50 Excellent
0.2: 'yellow', // Good (IAQ 51-100) 0.2: 'yellow', // 51-100 Good
0.4: 'orange', // Moderate (IAQ 101-150) 0.4: 'orange', // 101-150 Moderate
0.6: 'red', // Unhealthy (IAQ 151-200) 0.6: 'red', // 151-200 Unhealthy
0.8: 'purple', // Very Unhealthy (201-300) 0.8: 'purple', // 201-300 Very unhealthy
1.0: 'maroon' // Hazardous (301+) 1.0: 'maroon', // 301+ Hazardous
} },
}); });
return heatLayer;
}
function normalizeIAQ(iaq) {
// Normalize IAQ 0-500 to 0-1 for heatmap intensity
return Math.min(iaq / 300, 1.0);
} }
``` ```
### 6.2 Legend / UI Legend markup:
```html ```html
<div class="air-quality-legend"> <div class="air-quality-legend">
<h4>Air Quality Index</h4> <h4>Air Quality Index</h4>
<div class="legend-item"><span class="color green"></span> 0-50 Excellent</div> <div><span class="color green"></span> 0-50 Excellent</div>
<div class="legend-item"><span class="color yellow"></span> 51-100 Good</div> <div><span class="color yellow"></span> 51-100 Good</div>
<div class="legend-item"><span class="color orange"></span> 101-150 Moderate</div> <div><span class="color orange"></span> 101-150 Moderate</div>
<div class="legend-item"><span class="color red"></span> 151-200 Unhealthy</div> <div><span class="color red"></span> 151-200 Unhealthy</div>
<div class="legend-item"><span class="color purple"></span> 201-300 Very Unhealthy</div> <div><span class="color purple"></span> 201-300 Very unhealthy</div>
<div class="legend-item"><span class="color maroon"></span> 301+ Hazardous</div> <div><span class="color maroon"></span> 301+ Hazardous</div>
</div> </div>
``` ```
--- ## BOM
## 7. Implementation Roadmap Option A — BME680, basic VOC/IAQ:
### Phase 1: Sensor Validation (1-2 days) ```
BME680 breakout $15 Adafruit/SparkFun
MCP2221A USB-I2C $7 Adafruit
Qwiic/STEMMA cables $3 SparkFun
-----
$25
```
1. Identify exact sensor model on hand (BME680? BME688? SEN5x?) Option B — SEN55, full air quality:
2. Acquire USB-I2C adapter if needed (MCP2221A recommended)
3. Test sensor on laptop/Pi to confirm readings work
4. Verify USB-C data port on Bee accepts USB devices
### Phase 2: Bee-Side Integration (2-3 days) ```
SEN55 sensor $45 DigiKey/Mouser
breakout PCB $5 JLCPCB/OSHPark
CP2102 USB-UART $3 Amazon
-----
$53
```
1. SSH to Bee, install Python dependencies Both, if we want everything (VOC index from BME688 cross-checked against SEN55's separate VOC/NOx readings): ~$80.
2. Deploy `air-sensor.service`
3. Verify Redis key `AirQuality1Hz` is being written
4. Extend `bee-collector.py` to read air quality
5. Confirm fused data appears in uploads
### Phase 3: AdaMaps API (1-2 days) ## things still to confirm
1. Add `air_quality` table to PostgreSQL - exact sensor model on hand (BME680? 688? something else?) — needs a look
2. Add `/api/ingest/air` endpoint - Bee USB-C port host mode — plug something in and see if it enumerates
3. Add `/api/air/heatmap` query endpoint - can we `pip install` on the Bee, or is the Yocto rootfs read-only? need a wheel-bundle plan if so
4. Test end-to-end with curl - Bosch BSEC licensing for the real IAQ calculation — non-commercial vs. commercial terms differ
- 1Hz is the default polling rate; bump up or down once we see what the data looks like
### Phase 4: Frontend Overlay (1-2 days) ## rollout order
1. Add Leaflet.heat library Sensor on the bench first (laptop or Pi) to confirm it actually reads. Then onto the Bee — service deploys, Redis key check, fusion in bee-collector, upload spot-check. AdaMaps side (table + endpoints) can land in parallel; curl-test before pointing the Bee at it. Frontend last, drive a route, eyeball the heatmap.
2. Implement air quality heatmap layer
3. Add legend and metric selector
4. Deploy to adamaps.org
### Phase 5: Testing & Refinement (ongoing)
1. Drive routes to collect data
2. Validate heatmap accuracy
3. Tune grid resolution and time windows
4. Consider Bosch BSEC library for accurate IAQ
---
## 8. Bill of Materials
### Option A: BME680 (Basic VOC/IAQ)
| Item | Price | Source |
|------|-------|--------|
| BME680 Breakout | $15 | Adafruit/SparkFun |
| MCP2221A USB-I2C | $7 | Adafruit |
| Qwiic/STEMMA cables | $3 | SparkFun |
| **Total** | **~$25** | |
### Option B: SEN55 (Full Air Quality)
| Item | Price | Source |
|------|-------|--------|
| SEN55 Sensor | $45 | DigiKey/Mouser |
| Breakout PCB | $5 | JLCPCB/OSHPark |
| CP2102 USB-UART | $3 | Amazon |
| **Total** | **~$55** | |
### Option C: Both (Comprehensive)
| Item | Price |
|------|-------|
| BME688 + MCP2221A | $25 |
| SEN55 + CP2102 | $55 |
| **Total** | **~$80** |
---
## 9. Open Questions
| Question | Priority | Resolution Path |
|----------|----------|-----------------|
| Exact sensor model on hand? | High | Confirm |
| Does Bee USB-C port support host mode? | High | Test with USB device |
| Can we install Python packages on Bee? | High | Check if pip works on Yocto |
| Bosch BSEC library licensing? | Medium | Review Bosch terms |
| Target polling rate? | Low | 1Hz default, adjust as needed |
---
## 10. Conclusion
**Adding air quality sensing to the Hivemapper Bee is feasible and lightweight.**
The recommended path:
1. **Sensor:** Start with BME680 for quick wins (VOC/IAQ), upgrade to SEN55 for particulate matter if needed
2. **Interface:** MCP2221A USB-I2C adapter ($7) — plug and play on Linux
3. **Software:** Simple Python service (<100 lines), <1% CPU overhead
4. **Data fusion:** Leverage existing Redis infrastructure (GNSSFusion30Hz pattern)
5. **Backend:** New PostGIS table + 2 API endpoints
6. **Frontend:** Leaflet.heat overlay with IAQ color gradient
**Total estimated effort:** ~1 week for end-to-end prototype
**Total BOM cost:** ~$25-80 depending on sensor choice
---
*End of Report*

765
docs/BEE-CAMERA.md
View file

@ -1,132 +1,54 @@
# Bee Camera System — Full Technical Report # Bee camera system
*Generated: 2026-03-13* Notes from poking at the Hivemapper Bee dashcam. The camera path doesn't use V4L2 at all — frames live in `/tmp/recording/pics/` and `/data/recording/cached_observations/`, written by a DepthAI pipeline running on the on-die Myriad X VPU. All access goes through XLink, not `/dev/video*`.
--- ## hardware
## Executive Summary SoC is Intel Keem Bay (RVC2 — Robotics Vision Core 2): 4× Cortex-A53 @ 1.5GHz, integrated Myriad X VPU with 16 SHAVE cores and a Neural Compute Engine, 10nm. 4GB LPDDR4 on the board, ~3.5GB usable. CMA reserves ~1.34GB for VPU/camera DMA, swap is 2GB.
The Hivemapper Bee dashcam uses an Intel Keem Bay SoC with an integrated Myriad X VPU for camera capture and ML inference. The camera pipeline flows from a Sony IMX378-equivalent sensor through MIPI CSI-2 to the VPU, where DepthAI firmware handles image processing and neural network inference. Frames are written to disk and exposed through multiple odc-api REST endpoints.
**Key Findings:**
- Camera controlled via `depthai_gate.service` (Python/Flask on port 11492)
- ML inference handled by `map-ai.service` using the VPU's Neural Compute Engine
- Live frames stored in `/tmp/recording/pics/`
- Landmark observation images stored in `/data/recording/cached_observations/`
- Preview mode restarts the camera-bridge service with different configuration
- No direct V4L2 access — all camera access goes through DepthAI pipeline
---
## 1. Hardware
### 1.1 System-on-Chip: Intel Keem Bay
| Component | Specification |
|-----------|---------------|
| **SoC** | Intel Keem Bay (RVC2 / Robotics Vision Core 2) |
| **CPU** | 4× ARM Cortex-A53 @ 1.5GHz |
| **VPU** | Intel Movidius Myriad X (16 SHAVE cores) |
| **NPU** | Integrated Neural Compute Engine (hardware inference) |
| **RAM** | 4GB LPDDR4 (~3.5GB usable) |
| **ISP** | Integrated Image Signal Processor on VPU |
| **Process** | 10nm (Intel) |
**Memory Configuration:**
```
MemTotal: 3,584,000 kB (~3.5GB)
SwapTotal: 2,097,148 kB (~2GB)
CmaTotal: 1,408,000 kB (~1.34GB reserved for VPU/camera DMA)
```
### 1.2 Camera Sensor
| Specification | Value |
|---------------|-------|
| **Sensor** | Sony IMX378 (or equivalent 12MP) |
| **Resolution** | 4056 × 3040 native, downscaled to 2028 × 1024 |
| **Interface** | MIPI CSI-2 |
| **Frame Rate** | Variable, typically 30 FPS |
| **ISP** | On-VPU processing via DepthAI |
The Bee uses a Luxonis OAK-1 compatible camera module integrated with the Keem Bay SoC. The camera sensor connects directly to the SoC's MIPI CSI-2 interface, which is managed entirely by the DepthAI/Luxonis firmware running on the Myriad X VPU.
### 1.3 Bus Architecture
``` ```
┌─────────────────────────────────────────────────────────────────┐ MemTotal: 3,584,000 kB
SwapTotal: 2,097,148 kB
CmaTotal: 1,408,000 kB # VPU/camera DMA reservation
```
The camera is a Luxonis OAK-1-compatible module — Sony IMX378 (or equivalent 12MP), 4056×3040 native, downscaled to 2028×1024 by the pipeline. ~30 FPS. MIPI CSI-2 into the VPU's ISP; the ARM side never touches it directly.
Bus layout:
```
┌─────────────────────────────────────────────────────┐
│ Intel Keem Bay SoC │ │ Intel Keem Bay SoC │
├─────────────────────────────────────────────────────────────────┤ │ ┌──────────┐ ┌──────────┐ ┌────────────────┐ │
│ ┌──────────────┐ ┌──────────────┐ ┌──────────────────────┐ │ │ │ A53 ×4 │ │ Myriad X │ │ Neural Compute │ │
│ │ ARM Cortex │ │ Myriad X VPU │ │ Neural Compute │ │ │ └────┬─────┘ └────┬─────┘ └────────┬───────┘ │
│ │ A53 (4-core) │ │ (16 SHAVE) │ │ Engine (NCE) │ │ │ └────────┬────┴─────────────────┘ │
│ └──────┬───────┘ └──────┬───────┘ └──────────┬───────────┘ │ │ Internal AXI/NoC │
│ │ │ │ │ │ ┌─────┬─────┼──────┬───────────┐ │
│ └────────┬────────┴──────────────────────┘ │ │ PCIe USB SDIO MIPI CSI │ │
│ │ │ └─────┼─────┼─────┼──────┼───────────┘ │
│ ┌────────┴────────┐ │ │ │ │ │
│ │ Internal Bus │ │ Marvell Telit eMMC IMX378
│ │ (AXI/NoC) │ │ 88W8997 LE910C4
│ └────────┬────────┘ │ WiFi/BT LTE
│ │ │
│ ┌─────────────┼─────────────────────────────────────┐ │
│ ┌─┴──┐ ┌────┴────┐ ┌────────┐ ┌───────────┐ │ │
│ │PCIe│ │ USB │ │ SDIO │ │ MIPI CSI │ │ │
│ └──┬─┘ └────┬────┘ └────┬───┘ └─────┬─────┘ │ │
└─────┼───────────┼──────────────┼──────────────┼────────┘ │
│ │ │ │ │
┌─────┴─────┐ ┌───┴───────┐ ┌──┴──┐ ┌────┴─────┐ │
│ Marvell │ │ Telit │ │eMMC │ │ Camera │ │
│ 88W8997 │ │ LE910C4 │ │Flash│ │ Module │ │
│ WiFi/BT │ │ LTE Modem │ │ │ │ (IMX378) │ │
└───────────┘ └───────────┘ └─────┘ └──────────┘ │
``` ```
--- ## kernel + V4L2
## 2. Kernel / V4L2 Custom Yocto build with Intel VPU drivers. `kmb_cam` / `kmb_imx412` may be loaded for the sensor itself, plus the usual `videodev` + `v4l2_fwnode`. None of it is reachable via `/dev/video*` during normal operation — the VPU owns the camera hardware exclusively and the host talks to it over XLink (PCIe transport on Keem Bay; USB on desktop OAK devices).
### 2.1 Kernel Modules VPU is controlled via sysfs at `/sys/class/vpu/`. Firmware is loaded by writing the filename to the `fwname` attribute. Two firmwares are present:
The Bee runs a custom Yocto-based Linux with Intel-specific VPU drivers: - `luxonis_vpu.bin` — DepthAI firmware (what we want)
- `vpu_nvr_b0.bin` — Intel HDDL firmware (conflicts, see below)
| Module | Purpose | Status | If you want frames without going through the existing stack you have three options: use the depthai_gate / odc-api stack as-is, stop depthai_gate and run your own DepthAI pipeline, or reverse-engineer XLink and roll custom VPU firmware. The first is by far the easiest.
|--------|---------|--------|
| **kmb_cam** (if present) | Keem Bay camera driver | Likely used internally |
| **kmb_imx412** (if present) | Sony IMX412 sensor driver | May be loaded for sensor |
| **videodev** | V4L2 subsystem | Core video framework |
| **v4l2_fwnode** | V4L2 firmware node parsing | Device tree integration |
**Note:** Standard V4L2 device access (`/dev/video*`) is **not used** for normal operation. The camera is accessed exclusively through the DepthAI XLink protocol running on the VPU. The VPU owns the camera hardware completely. ## depthai_gate
### 2.2 VPU Sysfs Interface Python+Flask, listens on localhost:11492, ~158 threads, ~200MB RSS. Lives at `/opt/depthai_gate/` (estimated — not confirmed on-device yet). Service unit looks like:
The VPU is controlled via sysfs:
```
/sys/class/vpu/
```
**Firmware Loading:**
- `luxonis_vpu.bin` — DepthAI firmware (Luxonis/OAK)
- `vpu_nvr_b0.bin` — Intel HDDL firmware (NOT used, conflicts)
The VPU firmware is written to a sysfs attribute (`fwname`) to trigger loading. The DepthAI firmware must load first, otherwise the Intel HDDL service (`deviceservice`) grabs the VPU and causes conflicts.
### 2.3 No Direct V4L2 Access
**Important:** You cannot access the camera via `/dev/video*` while `depthai_gate` is running. The DepthAI pipeline has exclusive ownership of the camera hardware. To get frames, you must:
1. Use the existing depthai_gate/odc-api stack, OR
2. Stop depthai_gate and implement your own DepthAI pipeline, OR
3. Reverse-engineer XLink and write custom firmware
---
## 3. DepthAI Gate
### 3.1 Service Configuration
```ini ```ini
# depthai_gate.service (inferred from analysis)
[Unit] [Unit]
Description=DepthAI Camera Gate Description=DepthAI Camera Gate
After=network.target After=network.target
@ -136,84 +58,40 @@ Type=simple
User=root User=root
ExecStart=/opt/depthai_gate/run.py ExecStart=/opt/depthai_gate/run.py
Restart=always Restart=always
[Install]
WantedBy=multi-user.target
``` ```
### 3.2 Technical Details What it does: loads `luxonis_vpu.bin` into the VPU, opens the XLink connection, configures the DepthAI pipeline (ColorCamera → ImageManip → XLinkOut, plus optional NeuralNetwork node), captures frames, and drops them into `/tmp/recording/pics/`. Pipeline config probably lives at `/opt/depthai_gate/pipeline.json` or `/data/camera_config.json`, possibly hardcoded.
| Property | Value | XLink status values seen in logs:
|----------|-------|
| **Language** | Python 3 + Flask |
| **Port** | 11492 (localhost) |
| **Tasks** | ~158 threads observed |
| **Memory** | ~200MB RSS |
| **Location** | `/opt/depthai_gate/` (estimated) |
### 3.3 Responsibilities
1. **VPU Firmware Loading** — Writes `luxonis_vpu.bin` to VPU sysfs
2. **XLink Connection** — Establishes PCIe XLink to Myriad X VPU
3. **DepthAI Pipeline** — Configures camera capture and ISP settings
4. **Frame Capture** — Captures frames at configured resolution/framerate
5. **Frame Output** — Writes frames to `/tmp/recording/pics/`
### 3.4 XLink Protocol
XLink is Luxonis's proprietary protocol for host-to-VPU communication:
- **Transport:** PCIe (on Keem Bay) or USB (on desktop OAK devices)
- **Channels:** Bidirectional data streams for frames, tensors, and control
- **Status Values:**
- `0` = Disconnected
- `1` = Connecting / Error
- `2` = Connected (good)
**Status Check (from logs):**
``` ```
xlink_device_status=2 # Healthy xlink_device_status=2 # connected, healthy
vpu_firmware=luxonis_vpu.bin =1 # connecting / error
=0 # disconnected
``` ```
### 3.5 Pipeline Configuration ### the VPU conflict
The DepthAI pipeline likely includes: `deviceservice.service` (Intel HDDL / OpenVINO) ships enabled and races depthai_gate for the VPU:
- **ColorCamera node** — IMX378 capture at 4K, downscaled to 2028×1024
- **ImageManipNode** — Resize, crop, color conversion
- **XLinkOut node** — Send frames to host for storage
- **NeuralNetwork node** (optional) — On-VPU inference
Pipeline configs may exist at:
- `/opt/depthai_gate/pipeline.json`
- `/data/camera_config.json`
- Hardcoded in Python
### 3.6 VPU Conflict Bug
**Root Cause (identified and fixed):**
`deviceservice.service` (Intel HDDL / OpenVINO) was racing with `depthai_gate.service`:
1. HDDL starts at boot, loads `vpu_nvr_b0.bin` 1. HDDL starts at boot, loads `vpu_nvr_b0.bin`
2. depthai_gate starts, overwrites with `luxonis_vpu.bin` 2. depthai_gate starts, overwrites with `luxonis_vpu.bin`
3. HDDL locked out, retries XLink every 2 seconds forever 3. HDDL retries XLink every 2s forever, can't talk to the now-Luxonis firmware
4. On depthai_gate restart, HDDL grabs VPU first → camera dead 4. If depthai_gate restarts after HDDL is already running, HDDL grabs the VPU first and the camera goes dead
5. Watchdog (`secure-wdtclient`) crash loops → memory pressure → OOM 5. `secure-wdtclient` watchdog crash-loops on the dead VPU → memory pressure → OOM
Fix:
**Fix:**
```bash ```bash
systemctl disable --now deviceservice systemctl disable --now deviceservice
systemctl mask deviceservice # Survives OTA better systemctl mask deviceservice # survives OTA
``` ```
--- ## map-ai
## 4. map-ai Pipeline Reads frames from depthai_gate, runs detection on the VPU's NCE, blurs faces and plates, writes results to SQLite (`/data/recording/odc-api.db`) and blurred frames to disk.
### 4.1 Service Configuration
```ini ```ini
# map-ai.service (inferred)
[Unit] [Unit]
Description=Map AI Processing Description=Map AI Processing
After=depthai_gate.service After=depthai_gate.service
@ -223,156 +101,102 @@ Type=simple
User=root User=root
ExecStart=/opt/map-ai/run.py ExecStart=/opt/map-ai/run.py
Restart=always Restart=always
[Install]
WantedBy=multi-user.target
``` ```
### 4.2 Technical Details Pipeline:
| Property | Value |
|----------|-------|
| **Language** | Python 3 |
| **Model Format** | ONNX (via OpenVINO or DepthAI NCE) |
| **Input** | Frames from depthai_gate |
| **Output** | Detections to Redis, blurred frames to disk |
### 4.3 Processing Pipeline
``` ```
Frame from depthai → map-ai.py frame from depthai_gate
│ │
▼ ▼
┌───────────────────────────────────────┐ ML inference (on VPU NCE)
│ ML INFERENCE (on VPU) │ - road sign classifier
│ - Road sign classifier │ - face detector
│ - Face detector (privacy) │ - license plate detector
│ - License plate detector (privacy) │
└───────────────────────┬───────────────┘
│ │
▼ ▼
┌───────────────────────────────────────┐ privacy blur
│ PRIVACY PROCESSING │ - Gaussian on faces/plates
│ - PrivacyBlurNode │ - cv2.imwrite blurred copy
│ - Gaussian blur on faces/plates │
│ - cv2.imwrite blurred frames │
└───────────────────────┬───────────────┘
│ │
┌───────────────┴───────────────┐ ├──► Redis (status keys, not detections)
▼ ▼ └──► SQLite (observations, landmarks, frames)
Redis ZSET (detections) Disk (blurred frames)
``` ```
### 4.4 AI Models Models:
| Model | Location | Purpose | | Model | Path | Purpose |
|-------|----------|---------| |-------|------|---------|
| Road Signs | `/opt/object-detection/model.blob` or `/data/models/` | Sign classification | | Road signs | `/opt/object-detection/model.blob` or `/data/models/` | classification |
| Privacy | `/opt/odc-api/python/` or `/data/models/` | Face/plate detection | | Privacy | `/opt/odc-api/python/` or `/data/models/` | face/plate detection |
| PVC | `/data/recording/models/pvc.onnx` | Unknown (227 bytes — likely index) | | PVC | `/data/recording/models/pvc.onnx` | unknown — 227 bytes, probably an index file |
**Privacy Model Hash:** Stored in FrameKm metadata for verification. Privacy model hash gets baked into FrameKm metadata for verification.
### 4.5 Redis Integration Redis only carries readiness flags:
map-ai writes to Redis status keys:
``` ```
GET MAP_AI_READY → "True" GET MAP_AI_READY → "True"
GET EXTERNAL_MODEL_CLASSIFIER_READY → "True" GET EXTERNAL_MODEL_CLASSIFIER_READY → "True"
``` ```
Detection results stored in SQLite, not Redis ZSETs. Detections go to SQLite, not Redis ZSETs.
--- ## frame storage
## 5. Frame Storage | Path | FS | Purpose | Persists? |
|------|----|---------|-----------|
| `/tmp/recording/pics/` | tmpfs | live frames | no |
| `/tmp/recording/preview/` | tmpfs | preview mode | no |
| `/data/recording/cached_observations/` | ext4 | landmark observations | yes |
| `/data/recording/framekm/` | ext4 | upload bundles | yes |
| `/tmp/rgb/` | tmpfs | frame list files | no |
### 5.1 Storage Locations Frames are JPEG at 2028×1024, ~85% quality, ~150-200KB each.
| Path | Type | Purpose | Persistence | Naming:
|------|------|---------|-------------|
| `/tmp/recording/pics/` | tmpfs | Live camera frames | Ephemeral |
| `/tmp/recording/preview/` | tmpfs | Preview mode frames | Ephemeral |
| `/data/recording/cached_observations/` | ext4 | Landmark observation images | Persistent |
| `/data/recording/framekm/` | ext4 | FrameKm upload bundles | Persistent |
| `/tmp/rgb/` | tmpfs | Frame list files | Ephemeral |
### 5.2 Frame Format
| Property | Value |
|----------|-------|
| **Format** | JPEG |
| **Resolution** | 2028 × 1024 |
| **Quality** | ~85% (estimated ~150-200KB/frame) |
| **Color** | RGB |
### 5.3 Naming Convention
**Live frames** (`/tmp/recording/pics/`):
``` ```
{system_time_ms}_{frame_id}_{sequence}.jpg # live
Example: 1709920000123_0001_0042.jpg /tmp/recording/pics/{system_time_ms}_{frame_id}_{sequence}.jpg
e.g. 1709920000123_0001_0042.jpg
# cached observations
/data/recording/cached_observations/{timestamp}_{subsecond}_{frame_number}.jpg
e.g. 1746377552_043000_2945056.jpg
``` ```
**Cached observations** (`/data/recording/cached_observations/`): `folder_purger` keeps disk under control — when `/tmp/recording/pics/` crosses 400MB, oldest frames go:
```
{timestamp}_{subsecond}_{frame_number}.jpg
Example: 1746377552_043000_2945056.jpg
```
### 5.4 Frame Purger ```
The `folder_purger` service manages disk space:
```bash
folder-purger /tmp/recording/pic 400000000 /mnt/data/gps 2000000000 ... folder-purger /tmp/recording/pic 400000000 /mnt/data/gps 2000000000 ...
``` ```
When `/tmp/recording/pics/` exceeds 400MB, older frames are deleted. SQLite schemas (simplified):
### 5.5 Database Schema
Frames are tracked in SQLite (`/data/recording/odc-api.db` or `data-logger.v2.0.0.db`):
```sql ```sql
-- frames table
CREATE TABLE frames ( CREATE TABLE frames (
system_time INTEGER PRIMARY KEY, system_time INTEGER PRIMARY KEY,
image_name TEXT image_name TEXT
); );
```
Landmark observations reference frames:
```sql
-- observations table (simplified)
CREATE TABLE observations ( CREATE TABLE observations (
id INTEGER PRIMARY KEY, id INTEGER PRIMARY KEY,
landmark_id INTEGER, landmark_id INTEGER,
image_name TEXT, image_name TEXT,
x1 REAL, y1 REAL, x2 REAL, y2 REAL, -- bounding box x1 REAL, y1 REAL, x2 REAL, y2 REAL,
ts INTEGER, ts INTEGER
...
); );
``` ```
--- DB lives at `/data/recording/odc-api.db` (also seen as `data-logger.v2.0.0.db`).
## 6. video-processor ## video-processor + FrameKm
### 6.1 Service Details `video-processor` isn't documented in the firmware I've looked at. Based on naming it bundles frames+metadata into FrameKm tarballs, handles any H.264/H.265 encoding for preview, and orders frames for upload. It doesn't produce raw frames — it only packages already-blurred ones — so it's not useful for camera-access work.
The `video-processor` service is not explicitly documented in the analyzed firmware, but based on naming patterns, it likely handles: A FrameKm is ~1km of driving data:
1. **FrameKm Bundling** — Package frames + metadata for upload
2. **Video Encoding** — H.264/H.265 encoding for preview/streaming
3. **Frame Sequencing** — Order frames for FrameKm creation
### 6.2 FrameKm Format
**Purpose:** Bundle ~1km of driving data for upload to Hivemapper/HERE.
**Path:** `/data/recording/framekm/`
**Contents:**
``` ```
framekm-2024-03-08-12-34-56-abc123.tar framekm-2024-03-08-12-34-56-abc123.tar
├── manifest.json ├── manifest.json
@ -383,7 +207,8 @@ framekm-2024-03-08-12-34-56-abc123.tar
└── gnss_auth_signature.bin └── gnss_auth_signature.bin
``` ```
**Manifest Fields:** Manifest:
```json ```json
{ {
"name": "framekm-2024-03-08-12-34-56-abc123", "name": "framekm-2024-03-08-12-34-56-abc123",
@ -398,77 +223,55 @@ framekm-2024-03-08-12-34-56-abc123.tar
} }
``` ```
### 6.3 Relationship to Camera Frames ## odc-api
The video-processor does NOT produce the frames we care about for camera access. It only packages existing blurred frames for upload. For raw frame access, focus on `depthai_gate` and the preview system. REST API at `http://192.168.0.10:5000/api/1/`. Binds to the AP interface (`wlp1s0f0`) only — not reachable from home LAN without going through the Bee's AP.
--- Preview endpoints:
## 7. odc-api Camera Endpoints | Endpoint | Method | Notes |
|----------|--------|-------|
| `/preview/start` | GET | 120s timeout, then auto-stop |
| `/preview/stop` | GET | |
| `/preview/status` | GET | |
| `/preview/metadata` | GET | latest frame metadata |
### 7.1 Base URL Preview works by writing a new config and bouncing camera-bridge:
```
http://192.168.0.10:5000/api/1/
```
Binds to AP interface (`wlp1s0f0`) only — not accessible from home LAN directly.
### 7.2 Preview Endpoints
| Endpoint | Method | Description |
|----------|--------|-------------|
| `/preview/start` | GET | Start preview mode (120s timeout) |
| `/preview/stop` | GET | Stop preview mode |
| `/preview/status` | GET | Check if preview is active |
| `/preview/metadata` | GET | Get latest frame metadata |
**Preview Implementation (`util/preview.ts`):**
```typescript ```typescript
export const startPreview = async () => { export const startPreview = async () => {
// Create preview directory
await execSync('mkdir /tmp/recording/preview'); await execSync('mkdir /tmp/recording/preview');
// Write preview config
writeFileSync(IMAGER_CONFIG_PATH, JSON.stringify(getPreviewConfig())); writeFileSync(IMAGER_CONFIG_PATH, JSON.stringify(getPreviewConfig()));
await execSync(CMD.STOP_CAMERA);
// Restart camera-bridge with new config
await execSync(CMD.STOP_CAMERA); // systemctl stop camera-bridge
await sleep(1000); await sleep(1000);
await execSync(CMD.START_CAMERA); // systemctl start camera-bridge await execSync(CMD.START_CAMERA);
}; };
``` ```
**Preview Timeout:** 120 seconds (auto-stops to preserve 4K quality recording) The 120s auto-stop is there to protect 4K recording quality.
### 7.3 Landmark Image Endpoints Landmark endpoints (where the cached observation images come out):
| Endpoint | Method | Description | | Endpoint | Method | Notes |
|----------|--------|-------------| |----------|--------|-------|
| `/landmarks/images/:id` | GET | Get image paths for landmark | | `/landmarks/images/:id` | GET | image paths for a landmark |
| `/landmarks/:id/chips` | GET | Get chip endpoints for landmark | | `/landmarks/:id/chips` | GET | list of chip endpoints |
| `/landmarks/:id/chips/:chip_id` | GET | Get cropped observation image (JPEG) | | `/landmarks/:id/chips/:chip_id` | GET | cropped observation JPEG |
| `/landmarks/boundingBox/:id` | GET | Get bounding box coordinates | | `/landmarks/boundingBox/:id` | GET | bbox coords |
| `/landmarks/upload` | PUT | Upload landmark image to external URL | | `/landmarks/upload` | PUT | upload landmark image |
**Image Retrieval Flow:** Image retrieval flow:
``` ```
GET /landmarks/images/123 GET /landmarks/images/123
↓ → ["/data/recording/cached_observations/1746377552_043000_2945056.jpg"]
Returns: ["/data/recording/cached_observations/1746377552_043000_2945056.jpg"]
↓
GET /landmarks/123/chips/456 GET /landmarks/123/chips/456
↓ → cropped JPEG of the bbox region
Returns: Cropped JPEG (bounding box region)
``` ```
### 7.4 Camera Configuration Camera bridge config: `/opt/camera-bridge/config.json`. Control commands from `bee.ts`:
**Config Path:** `/opt/camera-bridge/config.json`
**Commands (from `bee.ts`):**
```typescript ```typescript
export const CMD = { export const CMD = {
RESTART_CAMERA: 'systemctl restart camera-bridge', RESTART_CAMERA: 'systemctl restart camera-bridge',
@ -476,175 +279,85 @@ export const CMD = {
STOP_CAMERA: 'systemctl stop camera-bridge', STOP_CAMERA: 'systemctl stop camera-bridge',
START_PREVIEW: 'systemctl start camera-preview', START_PREVIEW: 'systemctl start camera-preview',
STOP_PREVIEW: 'systemctl stop camera-preview', STOP_PREVIEW: 'systemctl stop camera-preview',
// ...
}; };
``` ```
### 7.5 Frame Retrieval There is no `/camera/frame` endpoint. To get a frame you either use preview mode and read `/tmp/recording/preview/`, walk the landmarks API for chips, or SSH in and read `/tmp/recording/pics/` directly.
There is **no direct `/camera/frame` endpoint** in the current odc-api. To get a camera frame: ## full data flow
1. **Via Preview Mode:**
- Call `/preview/start`
- Read frames from `/tmp/recording/preview/`
- Call `/preview/stop` when done
2. **Via Landmark Images:**
- Call `/landmarks/last/N` to get recent detections
- Call `/landmarks/images/:id` to get observation image paths
- Call `/landmarks/:id/chips/:chip_id` to get cropped JPEG
3. **Direct File Access (SSH):**
- Read from `/tmp/recording/pics/` for latest frames
- Read from `/data/recording/cached_observations/` for landmark images
---
## 8. Full Data Flow
### 8.1 Complete Pipeline
``` ```
┌──────────────────────────────────────────────────────────────────────────────┐ IMX378 → MIPI CSI-2 → VPU ISP → DepthAI pipeline
│ CAMERA CAPTURE │
│ IMX378 Sensor → MIPI CSI-2 → VPU ISP → DepthAI Pipeline │
└───────────────────────────────────┬──────────────────────────────────────────┘
│ │
▼ depthai_gate (:11492)
┌──────────────────────────────────────────────────────────────────────────────┐ writes /tmp/recording/pics/
│ DEPTHAI_GATE (port 11492) │
│ - XLink communication with Myriad X VPU │
│ - Frame capture from DepthAI pipeline │
│ - Writes frames to /tmp/recording/pics/ │
└───────────────────────────────────┬──────────────────────────────────────────┘
│ │
┌───────────────┴───────────────┐ ┌───────────────────┴───────────────────┐
▼ ▼ ▼ ▼
┌───────────────────────────────┐ ┌──────────────────────────────────────────┐ /tmp/recording/pics/ map-ai (VPU NCE)
│ RAW FRAME STORAGE │ │ MAP-AI INFERENCE │ (raw, purged >400MB) - sign classifier
│ /tmp/recording/pics/ │ │ - Road sign detection (VPU NCE) │ - privacy blur
│ - Temporary frames │ │ - Privacy blur (faces/plates) │
│ - Purged when >400MB │ │ - Outputs to Redis + SQLite │
└───────────────────────────────┘ └─────────────────┬────────────────────────┘
│ │
┌─────────────────────────────────┤ ┌────────────────────────────────────┤
▼ ▼ ▼ ▼
┌───────────────────────────────────┐ ┌──────────────────────────────────────┐ /data/recording/cached_observations/ odc-api.db (SQLite)
│ CACHED OBSERVATIONS │ │ LANDMARK DATABASE │ (blurred, persistent) landmarks/observations/frames
│ /data/recording/ │ │ /data/recording/odc-api.db │
│ cached_observations/ │ │ - landmarks table │
│ - Persistent blurred frames │ │ - observations table │
│ - Referenced by landmark ID │ │ - frames table │
└───────────────────┬───────────────┘ └─────────────────┬────────────────────┘
│ │ │ │
└──────────────┬──────────────────────┘ └────────────────┬───────────────────┘
▼ ▼
┌──────────────────────────────────────────────────────────────────────────────┐ odc-api (:5000)
│ ODC-API (port 5000) │ /preview/*, /landmarks/*
│ - /preview/* — Start/stop preview mode │
│ - /landmarks/last/N — Get recent detections │
│ - /landmarks/images/:id — Get observation image paths │
│ - /landmarks/:id/chips/:chip_id — Get cropped JPEG │
└───────────────────────────────────┬──────────────────────────────────────────┘
│ │
▼ ▼
┌──────────────────────────────────────────────────────────────────────────────┐ hivemapper-data-logger
│ FRAMEKM BUNDLING │ → FrameKm bundles in /data/recording/framekm/
│ hivemapper-data-logger │
│ - Collect ~1km of frames + metadata │
│ - Bundle with GNSS auth signatures │
│ - Store at /data/recording/framekm/ │
└───────────────────────────────────┬──────────────────────────────────────────┘
│ │
▼ ▼
┌──────────────────────────────────────────────────────────────────────────────┐ odc-api → mitmdump :8888 → Cloudflare Workers → HERE OLP
│ UPLOAD PATH │
│ odc-api → mitmdump (port 8888) → Cloudflare Workers → HERE OLP │
└──────────────────────────────────────────────────────────────────────────────┘
``` ```
### 8.2 Single Detection Event Trace Single detection trace:
``` ```
1. Camera captures frame 1. IMX378 → MIPI → VPU ISP → depthai_gate
└── IMX378 → MIPI → VPU ISP → depthai_gate 2. /tmp/recording/pics/1709920000123_0001_0042.jpg
3. map-ai picks it up, runs road sign classifier on the NCE
2. Frame written to disk 4. hit (e.g. speed limit 35), faces/plates blurred
└── /tmp/recording/pics/1709920000123_0001_0042.jpg 5. observations row written + image saved to cached_observations/
6. landmarks row created/updated (class_label, lat, lon, confidence)
3. map-ai reads frame 7. /landmarks/last/5 surfaces it; /landmarks/{id}/chips/{chip_id} returns the crop
└── Runs road sign classifier on VPU NCE
4. Detection found (speed limit 35)
└── Privacy blur applied to any faces/plates
5. Observation stored
└── SQLite: observations table (landmark_id, bbox, ts, image_name)
└── File: /data/recording/cached_observations/...
6. Landmark created/updated
└── SQLite: landmarks table (class_label, lat, lon, confidence)
7. odc-api exposes data
└── GET /landmarks/last/5 returns detection
└── GET /landmarks/images/{id} returns image path
└── GET /landmarks/{id}/chips/{chip_id} returns cropped JPEG
``` ```
--- ## replacement notes
## 9. Replacement Considerations ### getting a frame without odc-api
### 9.1 Accessing Frames Without odc-api Direct file read (simplest, race-prone, no metadata):
**Option 1: Direct File Read**
```bash ```bash
# SSH to Bee
ssh -p 2222 root@localhost # via an SSH tunnel ssh -p 2222 root@localhost # via an SSH tunnel
# Read latest frames
ls -lt /tmp/recording/pics/ | head -10 ls -lt /tmp/recording/pics/ | head -10
cp /tmp/recording/pics/latest_frame.jpg /tmp/
# Stream frames (naive) # poor man's stream
while true; do while true; do
cp $(ls -t /tmp/recording/pics/*.jpg | head -1) /tmp/current.jpg cp $(ls -t /tmp/recording/pics/*.jpg | head -1) /tmp/current.jpg
sleep 0.033 # ~30 FPS sleep 0.033
done done
``` ```
**Pros:** Simple, no service changes Redis pub/sub would be cleaner — `r.pubsub().subscribe('frame_ready')` — but I haven't confirmed depthai_gate publishes anything like that. Worth a `redis-cli MONITOR` while recording is live.
**Cons:** Race conditions, no metadata
**Option 2: Redis Pub/Sub** ### getting a frame without depthai_gate
Subscribe to frame events if depthai_gate publishes them:
```python
import redis
r = redis.Redis()
p = r.pubsub()
p.subscribe('frame_ready')
for message in p.listen():
print(message) # Contains frame path or metadata
```
**Pros:** Event-driven, no polling Don't, unless you really mean it. You'd be replacing the whole VPU pipeline:
**Cons:** May not exist in current firmware
### 9.2 Accessing Frames Without depthai_gate
**Not recommended** — requires implementing your own DepthAI pipeline.
If you must:
1. Stop depthai_gate: `systemctl stop depthai_gate`
2. Use Luxonis depthai Python SDK
3. Create minimal pipeline:
```python ```python
import depthai as dai import depthai as dai
pipeline = dai.Pipeline() pipeline = dai.Pipeline()
cam = pipeline.create(dai.node.ColorCamera) cam = pipeline.create(dai.node.ColorCamera)
cam.setResolution(dai.ColorCameraProperties.SensorResolution.THE_4_K) cam.setResolution(dai.ColorCameraProperties.SensorResolution.THE_4_K)
cam.setIspScale(1, 2) # Downscale to 2028x1024 cam.setIspScale(1, 2) # → 2028×1024
xout = pipeline.create(dai.node.XLinkOut) xout = pipeline.create(dai.node.XLinkOut)
xout.setStreamName("video") xout.setStreamName("video")
@ -653,51 +366,37 @@ cam.video.link(xout.input)
with dai.Device(pipeline) as device: with dai.Device(pipeline) as device:
q = device.getOutputQueue("video") q = device.getOutputQueue("video")
while True: while True:
frame = q.get() cv2.imwrite("/tmp/frame.jpg", q.get().getCvFrame())
cv2.imwrite("/tmp/frame.jpg", frame.getCvFrame())
``` ```
**Pros:** Full control over camera Breaks everything Hivemapper depends on — map-ai, landmarks, FrameKm. Only useful if the goal is full liberation, not augmentation.
**Cons:** Breaks all Hivemapper services, loses ML pipeline
### 9.3 Minimal Path to JPEG Frame ### fastest frame grabs
With the stack running:
**Fastest (with existing stack):**
```bash ```bash
# Via SSH # direct
ssh -p 2222 root@localhost 'ls -t /tmp/recording/pics/*.jpg | head -1 | xargs cat' > frame.jpg ssh -p 2222 root@localhost 'ls -t /tmp/recording/pics/*.jpg | head -1 | xargs cat' > frame.jpg
```
**Via API (requires preview mode):** # via API (needs preview mode)
```bash
curl http://192.168.0.10:5000/api/1/preview/start curl http://192.168.0.10:5000/api/1/preview/start
sleep 2 sleep 2
ssh -p 2222 root@localhost 'ls -t /tmp/recording/preview/*.jpg | head -1 | xargs cat' > frame.jpg ssh -p 2222 root@localhost 'ls -t /tmp/recording/preview/*.jpg | head -1 | xargs cat' > frame.jpg
curl http://192.168.0.10:5000/api/1/preview/stop curl http://192.168.0.10:5000/api/1/preview/stop
``` ```
### 9.4 Building a Custom Camera Interface ### proposed odc-api extension
**Requirements:** Two new routes — single frame + MJPEG stream:
1. Maintain depthai_gate (or reimplement VPU control)
2. Expose a REST endpoint for single-frame capture
3. Optionally implement MJPEG streaming
**Proposed odc-api Addition:**
```typescript ```typescript
// routes/camera.ts
router.get('/frame', async (req, res) => { router.get('/frame', async (req, res) => {
const frames = readdirSync('/tmp/recording/pics') const frames = readdirSync('/tmp/recording/pics')
.filter(f => f.endsWith('.jpg')) .filter(f => f.endsWith('.jpg'))
.sort() .sort().reverse();
.reverse(); if (!frames.length) return res.status(404).send('No frames available');
res.sendFile(join('/tmp/recording/pics', frames[0]));
if (frames.length === 0) {
return res.status(404).send('No frames available');
}
const framePath = join('/tmp/recording/pics', frames[0]);
res.sendFile(framePath);
}); });
router.get('/stream', async (req, res) => { router.get('/stream', async (req, res) => {
@ -705,106 +404,64 @@ router.get('/stream', async (req, res) => {
'Content-Type': 'multipart/x-mixed-replace; boundary=frame', 'Content-Type': 'multipart/x-mixed-replace; boundary=frame',
'Cache-Control': 'no-cache', 'Cache-Control': 'no-cache',
}); });
const interval = setInterval(() => { const interval = setInterval(() => {
const frames = readdirSync('/tmp/recording/pics') const frames = readdirSync('/tmp/recording/pics')
.filter(f => f.endsWith('.jpg')) .filter(f => f.endsWith('.jpg')).sort().reverse();
.sort() if (frames.length) {
.reverse(); const data = readFileSync(join('/tmp/recording/pics', frames[0]));
if (frames.length > 0) {
const framePath = join('/tmp/recording/pics', frames[0]);
const frameData = readFileSync(framePath);
res.write('--frame\r\n'); res.write('--frame\r\n');
res.write('Content-Type: image/jpeg\r\n'); res.write('Content-Type: image/jpeg\r\n');
res.write(`Content-Length: ${frameData.length}\r\n\r\n`); res.write(`Content-Length: ${data.length}\r\n\r\n`);
res.write(frameData); res.write(data);
res.write('\r\n'); res.write('\r\n');
} }
}, 33); // ~30 FPS }, 33); // ~30 FPS
req.on('close', () => clearInterval(interval)); req.on('close', () => clearInterval(interval));
}); });
``` ```
### 9.5 Architecture for Replacement System Long-term replacement shape — leave depthai_gate alone, add a separate watcher service that inotify-tails `/tmp/recording/pics/` and serves frames over HTTP. Doesn't fight the VPU, doesn't break the upload chain.
## things still to confirm
- exact depthai_gate pipeline config (find files under `/opt/`)
- does depthai_gate publish frame events to Redis at all? (`redis-cli MONITOR`)
- camera-bridge vs depthai_gate — what's the actual dependency? (systemd deps, strace)
- preview config format — read `getPreviewConfig()`
- ML model exact locations (`find /opt /data -name '*.blob' -o -name '*.onnx'`)
- frame timestamp accuracy vs GNSS time
## appendix — file paths
``` ```
┌──────────────────────────────────────────────────────────────────┐ /tmp/recording/pics/ live frames
│ VARROA CAMERA SERVICE │ /tmp/recording/preview/ preview frames
├──────────────────────────────────────────────────────────────────┤ /data/recording/cached_observations/ landmark images
│ │ /data/recording/framekm/ upload bundles
│ ┌─────────────────┐ ┌─────────────────┐ ┌──────────────┐ │ /data/recording/odc-api.db SQLite DB
│ │ depthai_gate │ → │ varroa-camera │ → │ HTTP API │ │ /opt/camera-bridge/config.json camera config
│ │ (unchanged) │ │ (new service) │ │ (port 80) │ │ /opt/depthai_gate/ DepthAI service (estimated)
│ └─────────────────┘ └─────────────────┘ └──────────────┘ │ /opt/odc-api/ Node API service
│ ↓ ↓ ↓ │ /sys/class/vpu/ VPU sysfs
│ /tmp/recording/pics/ Monitor & serve GET /frame │
│ frames via inotify GET /stream │
│ GET /landmarks │
└──────────────────────────────────────────────────────────────────┘
``` ```
--- ## appendix — boot order
## 10. Open Questions
| Question | Priority | How to Investigate |
|----------|----------|-------------------|
| Exact depthai_gate pipeline config | High | SSH in, find config files in /opt/ |
| Does depthai_gate publish to Redis? | High | `redis-cli MONITOR` while recording |
| Camera-bridge vs depthai_gate relationship | High | Check systemd deps, trace with strace |
| Preview config format | Medium | Read `getPreviewConfig()` implementation |
| ML model exact location on Bee | Medium | `find /opt /data -name "*.blob" -o -name "*.onnx"` |
| Frame timestamp accuracy | Medium | Compare frame timestamps to GNSS time |
---
## Appendix A: Key File Paths
| Path | Purpose |
|------|---------|
| `/tmp/recording/pics/` | Live camera frames |
| `/tmp/recording/preview/` | Preview mode frames |
| `/data/recording/cached_observations/` | Landmark observation images |
| `/data/recording/framekm/` | FrameKm upload bundles |
| `/data/recording/odc-api.db` | SQLite database |
| `/opt/camera-bridge/config.json` | Camera configuration |
| `/opt/depthai_gate/` | DepthAI service (estimated) |
| `/opt/odc-api/` | Node.js API service |
| `/sys/class/vpu/` | VPU sysfs interface |
## Appendix B: Service Dependencies
``` ```
multi-user.target multi-user.target
│ ├── redis.service t+2s
├── redis.service [t+2s] ├── depthai_gate.service t+8s loads luxonis_vpu.bin
│ ├── map-ai.service t+12s needs depthai_gate
├── depthai_gate.service [t+8s] # MUST start before map-ai ├── hivemapper-data-logger.service t+15s
│ │ └── odc-api.service t+18s
│ └── Loads luxonis_vpu.bin
│
├── map-ai.service [t+12s] # Depends on depthai_gate
│ │
│ └── Privacy blur, ML inference
│
├── hivemapper-data-logger.service [t+15s]
│
└── odc-api.service [t+18s] # REST API
``` ```
## Appendix C: Port Reference ## appendix — ports
| Port | Service | Protocol | Binding | ```
|------|---------|----------|---------| 22 sshd TCP AP-only (socket)
| 22 | sshd | TCP | AP only (via socket) | 5000 odc-api HTTP AP iface
| 5000 | odc-api | HTTP | AP interface | 6379 redis TCP localhost
| 6379 | Redis | TCP | localhost | 8888 mitmdump HTTP localhost
| 8888 | mitmdump | HTTP | localhost | 11492 depthai_gate HTTP/Flask localhost
| 11492 | depthai_gate | HTTP/Flask | localhost | ```
---
*End of Report*