810 lines
32 KiB
Markdown
810 lines
32 KiB
Markdown
# Bee Camera System — Full Technical Report
|
||
|
||
*Generated: 2026-03-13*
|
||
|
||
---
|
||
|
||
## Executive Summary
|
||
|
||
The Hivemapper Bee dashcam uses an Intel Keem Bay SoC with an integrated Myriad X VPU for camera capture and ML inference. The camera pipeline flows from a Sony IMX378-equivalent sensor through MIPI CSI-2 to the VPU, where DepthAI firmware handles image processing and neural network inference. Frames are written to disk and exposed through multiple odc-api REST endpoints.
|
||
|
||
**Key Findings:**
|
||
- Camera controlled via `depthai_gate.service` (Python/Flask on port 11492)
|
||
- ML inference handled by `map-ai.service` using the VPU's Neural Compute Engine
|
||
- Live frames stored in `/tmp/recording/pics/`
|
||
- Landmark observation images stored in `/data/recording/cached_observations/`
|
||
- Preview mode restarts the camera-bridge service with different configuration
|
||
- No direct V4L2 access — all camera access goes through DepthAI pipeline
|
||
|
||
---
|
||
|
||
## 1. Hardware
|
||
|
||
### 1.1 System-on-Chip: Intel Keem Bay
|
||
|
||
| Component | Specification |
|
||
|-----------|---------------|
|
||
| **SoC** | Intel Keem Bay (RVC2 / Robotics Vision Core 2) |
|
||
| **CPU** | 4× ARM Cortex-A53 @ 1.5GHz |
|
||
| **VPU** | Intel Movidius Myriad X (16 SHAVE cores) |
|
||
| **NPU** | Integrated Neural Compute Engine (hardware inference) |
|
||
| **RAM** | 4GB LPDDR4 (~3.5GB usable) |
|
||
| **ISP** | Integrated Image Signal Processor on VPU |
|
||
| **Process** | 10nm (Intel) |
|
||
|
||
**Memory Configuration:**
|
||
```
|
||
MemTotal: 3,584,000 kB (~3.5GB)
|
||
SwapTotal: 2,097,148 kB (~2GB)
|
||
CmaTotal: 1,408,000 kB (~1.34GB reserved for VPU/camera DMA)
|
||
```
|
||
|
||
### 1.2 Camera Sensor
|
||
|
||
| Specification | Value |
|
||
|---------------|-------|
|
||
| **Sensor** | Sony IMX378 (or equivalent 12MP) |
|
||
| **Resolution** | 4056 × 3040 native, downscaled to 2028 × 1024 |
|
||
| **Interface** | MIPI CSI-2 |
|
||
| **Frame Rate** | Variable, typically 30 FPS |
|
||
| **ISP** | On-VPU processing via DepthAI |
|
||
|
||
The Bee uses a Luxonis OAK-1 compatible camera module integrated with the Keem Bay SoC. The camera sensor connects directly to the SoC's MIPI CSI-2 interface, which is managed entirely by the DepthAI/Luxonis firmware running on the Myriad X VPU.
|
||
|
||
### 1.3 Bus Architecture
|
||
|
||
```
|
||
┌─────────────────────────────────────────────────────────────────┐
|
||
│ Intel Keem Bay SoC │
|
||
├─────────────────────────────────────────────────────────────────┤
|
||
│ ┌──────────────┐ ┌──────────────┐ ┌──────────────────────┐ │
|
||
│ │ ARM Cortex │ │ Myriad X VPU │ │ Neural Compute │ │
|
||
│ │ A53 (4-core) │ │ (16 SHAVE) │ │ Engine (NCE) │ │
|
||
│ └──────┬───────┘ └──────┬───────┘ └──────────┬───────────┘ │
|
||
│ │ │ │ │
|
||
│ └────────┬────────┴──────────────────────┘ │
|
||
│ │ │
|
||
│ ┌────────┴────────┐ │
|
||
│ │ Internal Bus │ │
|
||
│ │ (AXI/NoC) │ │
|
||
│ └────────┬────────┘ │
|
||
│ │ │
|
||
│ ┌─────────────┼─────────────────────────────────────┐ │
|
||
│ ┌─┴──┐ ┌────┴────┐ ┌────────┐ ┌───────────┐ │ │
|
||
│ │PCIe│ │ USB │ │ SDIO │ │ MIPI CSI │ │ │
|
||
│ └──┬─┘ └────┬────┘ └────┬───┘ └─────┬─────┘ │ │
|
||
└─────┼───────────┼──────────────┼──────────────┼────────┘ │
|
||
│ │ │ │ │
|
||
┌─────┴─────┐ ┌───┴───────┐ ┌──┴──┐ ┌────┴─────┐ │
|
||
│ Marvell │ │ Telit │ │eMMC │ │ Camera │ │
|
||
│ 88W8997 │ │ LE910C4 │ │Flash│ │ Module │ │
|
||
│ WiFi/BT │ │ LTE Modem │ │ │ │ (IMX378) │ │
|
||
└───────────┘ └───────────┘ └─────┘ └──────────┘ │
|
||
```
|
||
|
||
---
|
||
|
||
## 2. Kernel / V4L2
|
||
|
||
### 2.1 Kernel Modules
|
||
|
||
The Bee runs a custom Yocto-based Linux with Intel-specific VPU drivers:
|
||
|
||
| Module | Purpose | Status |
|
||
|--------|---------|--------|
|
||
| **kmb_cam** (if present) | Keem Bay camera driver | Likely used internally |
|
||
| **kmb_imx412** (if present) | Sony IMX412 sensor driver | May be loaded for sensor |
|
||
| **videodev** | V4L2 subsystem | Core video framework |
|
||
| **v4l2_fwnode** | V4L2 firmware node parsing | Device tree integration |
|
||
|
||
**Note:** Standard V4L2 device access (`/dev/video*`) is **not used** for normal operation. The camera is accessed exclusively through the DepthAI XLink protocol running on the VPU. The VPU owns the camera hardware completely.
|
||
|
||
### 2.2 VPU Sysfs Interface
|
||
|
||
The VPU is controlled via sysfs:
|
||
```
|
||
/sys/class/vpu/
|
||
```
|
||
|
||
**Firmware Loading:**
|
||
- `luxonis_vpu.bin` — DepthAI firmware (Luxonis/OAK)
|
||
- `vpu_nvr_b0.bin` — Intel HDDL firmware (NOT used, conflicts)
|
||
|
||
The VPU firmware is written to a sysfs attribute (`fwname`) to trigger loading. The DepthAI firmware must load first, otherwise the Intel HDDL service (`deviceservice`) grabs the VPU and causes conflicts.
|
||
|
||
### 2.3 No Direct V4L2 Access
|
||
|
||
**Important:** You cannot access the camera via `/dev/video*` while `depthai_gate` is running. The DepthAI pipeline has exclusive ownership of the camera hardware. To get frames, you must:
|
||
1. Use the existing depthai_gate/odc-api stack, OR
|
||
2. Stop depthai_gate and implement your own DepthAI pipeline, OR
|
||
3. Reverse-engineer XLink and write custom firmware
|
||
|
||
---
|
||
|
||
## 3. DepthAI Gate
|
||
|
||
### 3.1 Service Configuration
|
||
|
||
```ini
|
||
# depthai_gate.service (inferred from analysis)
|
||
[Unit]
|
||
Description=DepthAI Camera Gate
|
||
After=network.target
|
||
|
||
[Service]
|
||
Type=simple
|
||
User=root
|
||
ExecStart=/opt/depthai_gate/run.py
|
||
Restart=always
|
||
|
||
[Install]
|
||
WantedBy=multi-user.target
|
||
```
|
||
|
||
### 3.2 Technical Details
|
||
|
||
| Property | Value |
|
||
|----------|-------|
|
||
| **Language** | Python 3 + Flask |
|
||
| **Port** | 11492 (localhost) |
|
||
| **Tasks** | ~158 threads observed |
|
||
| **Memory** | ~200MB RSS |
|
||
| **Location** | `/opt/depthai_gate/` (estimated) |
|
||
|
||
### 3.3 Responsibilities
|
||
|
||
1. **VPU Firmware Loading** — Writes `luxonis_vpu.bin` to VPU sysfs
|
||
2. **XLink Connection** — Establishes PCIe XLink to Myriad X VPU
|
||
3. **DepthAI Pipeline** — Configures camera capture and ISP settings
|
||
4. **Frame Capture** — Captures frames at configured resolution/framerate
|
||
5. **Frame Output** — Writes frames to `/tmp/recording/pics/`
|
||
|
||
### 3.4 XLink Protocol
|
||
|
||
XLink is Luxonis's proprietary protocol for host-to-VPU communication:
|
||
- **Transport:** PCIe (on Keem Bay) or USB (on desktop OAK devices)
|
||
- **Channels:** Bidirectional data streams for frames, tensors, and control
|
||
- **Status Values:**
|
||
- `0` = Disconnected
|
||
- `1` = Connecting / Error
|
||
- `2` = Connected (good)
|
||
|
||
**Status Check (from logs):**
|
||
```
|
||
xlink_device_status=2 # Healthy
|
||
vpu_firmware=luxonis_vpu.bin
|
||
```
|
||
|
||
### 3.5 Pipeline Configuration
|
||
|
||
The DepthAI pipeline likely includes:
|
||
- **ColorCamera node** — IMX378 capture at 4K, downscaled to 2028×1024
|
||
- **ImageManipNode** — Resize, crop, color conversion
|
||
- **XLinkOut node** — Send frames to host for storage
|
||
- **NeuralNetwork node** (optional) — On-VPU inference
|
||
|
||
Pipeline configs may exist at:
|
||
- `/opt/depthai_gate/pipeline.json`
|
||
- `/data/camera_config.json`
|
||
- Hardcoded in Python
|
||
|
||
### 3.6 VPU Conflict Bug
|
||
|
||
**Root Cause (identified and fixed):**
|
||
|
||
`deviceservice.service` (Intel HDDL / OpenVINO) was racing with `depthai_gate.service`:
|
||
|
||
1. HDDL starts at boot, loads `vpu_nvr_b0.bin`
|
||
2. depthai_gate starts, overwrites with `luxonis_vpu.bin`
|
||
3. HDDL locked out, retries XLink every 2 seconds forever
|
||
4. On depthai_gate restart, HDDL grabs VPU first → camera dead
|
||
5. Watchdog (`secure-wdtclient`) crash loops → memory pressure → OOM
|
||
|
||
**Fix:**
|
||
```bash
|
||
systemctl disable --now deviceservice
|
||
systemctl mask deviceservice # Survives OTA better
|
||
```
|
||
|
||
---
|
||
|
||
## 4. map-ai Pipeline
|
||
|
||
### 4.1 Service Configuration
|
||
|
||
```ini
|
||
# map-ai.service (inferred)
|
||
[Unit]
|
||
Description=Map AI Processing
|
||
After=depthai_gate.service
|
||
|
||
[Service]
|
||
Type=simple
|
||
User=root
|
||
ExecStart=/opt/map-ai/run.py
|
||
Restart=always
|
||
|
||
[Install]
|
||
WantedBy=multi-user.target
|
||
```
|
||
|
||
### 4.2 Technical Details
|
||
|
||
| Property | Value |
|
||
|----------|-------|
|
||
| **Language** | Python 3 |
|
||
| **Model Format** | ONNX (via OpenVINO or DepthAI NCE) |
|
||
| **Input** | Frames from depthai_gate |
|
||
| **Output** | Detections to Redis, blurred frames to disk |
|
||
|
||
### 4.3 Processing Pipeline
|
||
|
||
```
|
||
Frame from depthai → map-ai.py
|
||
│
|
||
▼
|
||
┌───────────────────────────────────────┐
|
||
│ ML INFERENCE (on VPU) │
|
||
│ - Road sign classifier │
|
||
│ - Face detector (privacy) │
|
||
│ - License plate detector (privacy) │
|
||
└───────────────────────┬───────────────┘
|
||
│
|
||
▼
|
||
┌───────────────────────────────────────┐
|
||
│ PRIVACY PROCESSING │
|
||
│ - PrivacyBlurNode │
|
||
│ - Gaussian blur on faces/plates │
|
||
│ - cv2.imwrite blurred frames │
|
||
└───────────────────────┬───────────────┘
|
||
│
|
||
┌───────────────┴───────────────┐
|
||
▼ ▼
|
||
Redis ZSET (detections) Disk (blurred frames)
|
||
```
|
||
|
||
### 4.4 AI Models
|
||
|
||
| Model | Location | Purpose |
|
||
|-------|----------|---------|
|
||
| Road Signs | `/opt/object-detection/model.blob` or `/data/models/` | Sign classification |
|
||
| Privacy | `/opt/odc-api/python/` or `/data/models/` | Face/plate detection |
|
||
| PVC | `/data/recording/models/pvc.onnx` | Unknown (227 bytes — likely index) |
|
||
|
||
**Privacy Model Hash:** Stored in FrameKm metadata for verification.
|
||
|
||
### 4.5 Redis Integration
|
||
|
||
map-ai writes to Redis status keys:
|
||
```
|
||
GET MAP_AI_READY → "True"
|
||
GET EXTERNAL_MODEL_CLASSIFIER_READY → "True"
|
||
```
|
||
|
||
Detection results stored in SQLite, not Redis ZSETs.
|
||
|
||
---
|
||
|
||
## 5. Frame Storage
|
||
|
||
### 5.1 Storage Locations
|
||
|
||
| Path | Type | Purpose | Persistence |
|
||
|------|------|---------|-------------|
|
||
| `/tmp/recording/pics/` | tmpfs | Live camera frames | Ephemeral |
|
||
| `/tmp/recording/preview/` | tmpfs | Preview mode frames | Ephemeral |
|
||
| `/data/recording/cached_observations/` | ext4 | Landmark observation images | Persistent |
|
||
| `/data/recording/framekm/` | ext4 | FrameKm upload bundles | Persistent |
|
||
| `/tmp/rgb/` | tmpfs | Frame list files | Ephemeral |
|
||
|
||
### 5.2 Frame Format
|
||
|
||
| Property | Value |
|
||
|----------|-------|
|
||
| **Format** | JPEG |
|
||
| **Resolution** | 2028 × 1024 |
|
||
| **Quality** | ~85% (estimated ~150-200KB/frame) |
|
||
| **Color** | RGB |
|
||
|
||
### 5.3 Naming Convention
|
||
|
||
**Live frames** (`/tmp/recording/pics/`):
|
||
```
|
||
{system_time_ms}_{frame_id}_{sequence}.jpg
|
||
Example: 1709920000123_0001_0042.jpg
|
||
```
|
||
|
||
**Cached observations** (`/data/recording/cached_observations/`):
|
||
```
|
||
{timestamp}_{subsecond}_{frame_number}.jpg
|
||
Example: 1746377552_043000_2945056.jpg
|
||
```
|
||
|
||
### 5.4 Frame Purger
|
||
|
||
The `folder_purger` service manages disk space:
|
||
```bash
|
||
folder-purger /tmp/recording/pic 400000000 /mnt/data/gps 2000000000 ...
|
||
```
|
||
|
||
When `/tmp/recording/pics/` exceeds 400MB, older frames are deleted.
|
||
|
||
### 5.5 Database Schema
|
||
|
||
Frames are tracked in SQLite (`/data/recording/odc-api.db` or `data-logger.v2.0.0.db`):
|
||
|
||
```sql
|
||
-- frames table
|
||
CREATE TABLE frames (
|
||
system_time INTEGER PRIMARY KEY,
|
||
image_name TEXT
|
||
);
|
||
```
|
||
|
||
Landmark observations reference frames:
|
||
```sql
|
||
-- observations table (simplified)
|
||
CREATE TABLE observations (
|
||
id INTEGER PRIMARY KEY,
|
||
landmark_id INTEGER,
|
||
image_name TEXT,
|
||
x1 REAL, y1 REAL, x2 REAL, y2 REAL, -- bounding box
|
||
ts INTEGER,
|
||
...
|
||
);
|
||
```
|
||
|
||
---
|
||
|
||
## 6. video-processor
|
||
|
||
### 6.1 Service Details
|
||
|
||
The `video-processor` service is not explicitly documented in the analyzed firmware, but based on naming patterns, it likely handles:
|
||
|
||
1. **FrameKm Bundling** — Package frames + metadata for upload
|
||
2. **Video Encoding** — H.264/H.265 encoding for preview/streaming
|
||
3. **Frame Sequencing** — Order frames for FrameKm creation
|
||
|
||
### 6.2 FrameKm Format
|
||
|
||
**Purpose:** Bundle ~1km of driving data for upload to Hivemapper/HERE.
|
||
|
||
**Path:** `/data/recording/framekm/`
|
||
|
||
**Contents:**
|
||
```
|
||
framekm-2024-03-08-12-34-56-abc123.tar
|
||
├── manifest.json
|
||
├── frame_0001.jpg
|
||
├── frame_0002.jpg
|
||
├── ...
|
||
├── gnss_auth_buffer.bin
|
||
└── gnss_auth_signature.bin
|
||
```
|
||
|
||
**Manifest Fields:**
|
||
```json
|
||
{
|
||
"name": "framekm-2024-03-08-12-34-56-abc123",
|
||
"numFrames": 150,
|
||
"deviceId": "fvhL2I-iCT",
|
||
"firmwareVersion": "0.0.1",
|
||
"privacyModelHash": "sha256:abc123...",
|
||
"gnssAuthBuffer": "base64...",
|
||
"gnssAuthSignature": "base64...",
|
||
"gnssAuthPublicKey": "base64...",
|
||
"createdAt": 1709920000000
|
||
}
|
||
```
|
||
|
||
### 6.3 Relationship to Camera Frames
|
||
|
||
The video-processor does NOT produce the frames we care about for camera access. It only packages existing blurred frames for upload. For raw frame access, focus on `depthai_gate` and the preview system.
|
||
|
||
---
|
||
|
||
## 7. odc-api Camera Endpoints
|
||
|
||
### 7.1 Base URL
|
||
|
||
```
|
||
http://192.168.0.10:5000/api/1/
|
||
```
|
||
|
||
Binds to AP interface (`wlp1s0f0`) only — not accessible from home LAN directly.
|
||
|
||
### 7.2 Preview Endpoints
|
||
|
||
| Endpoint | Method | Description |
|
||
|----------|--------|-------------|
|
||
| `/preview/start` | GET | Start preview mode (120s timeout) |
|
||
| `/preview/stop` | GET | Stop preview mode |
|
||
| `/preview/status` | GET | Check if preview is active |
|
||
| `/preview/metadata` | GET | Get latest frame metadata |
|
||
|
||
**Preview Implementation (`util/preview.ts`):**
|
||
|
||
```typescript
|
||
export const startPreview = async () => {
|
||
// Create preview directory
|
||
await execSync('mkdir /tmp/recording/preview');
|
||
|
||
// Write preview config
|
||
writeFileSync(IMAGER_CONFIG_PATH, JSON.stringify(getPreviewConfig()));
|
||
|
||
// Restart camera-bridge with new config
|
||
await execSync(CMD.STOP_CAMERA); // systemctl stop camera-bridge
|
||
await sleep(1000);
|
||
await execSync(CMD.START_CAMERA); // systemctl start camera-bridge
|
||
};
|
||
```
|
||
|
||
**Preview Timeout:** 120 seconds (auto-stops to preserve 4K quality recording)
|
||
|
||
### 7.3 Landmark Image Endpoints
|
||
|
||
| Endpoint | Method | Description |
|
||
|----------|--------|-------------|
|
||
| `/landmarks/images/:id` | GET | Get image paths for landmark |
|
||
| `/landmarks/:id/chips` | GET | Get chip endpoints for landmark |
|
||
| `/landmarks/:id/chips/:chip_id` | GET | Get cropped observation image (JPEG) |
|
||
| `/landmarks/boundingBox/:id` | GET | Get bounding box coordinates |
|
||
| `/landmarks/upload` | PUT | Upload landmark image to external URL |
|
||
|
||
**Image Retrieval Flow:**
|
||
|
||
```
|
||
GET /landmarks/images/123
|
||
↓
|
||
Returns: ["/data/recording/cached_observations/1746377552_043000_2945056.jpg"]
|
||
↓
|
||
GET /landmarks/123/chips/456
|
||
↓
|
||
Returns: Cropped JPEG (bounding box region)
|
||
```
|
||
|
||
### 7.4 Camera Configuration
|
||
|
||
**Config Path:** `/opt/camera-bridge/config.json`
|
||
|
||
**Commands (from `bee.ts`):**
|
||
```typescript
|
||
export const CMD = {
|
||
RESTART_CAMERA: 'systemctl restart camera-bridge',
|
||
START_CAMERA: 'systemctl start camera-bridge',
|
||
STOP_CAMERA: 'systemctl stop camera-bridge',
|
||
START_PREVIEW: 'systemctl start camera-preview',
|
||
STOP_PREVIEW: 'systemctl stop camera-preview',
|
||
// ...
|
||
};
|
||
```
|
||
|
||
### 7.5 Frame Retrieval
|
||
|
||
There is **no direct `/camera/frame` endpoint** in the current odc-api. To get a camera frame:
|
||
|
||
1. **Via Preview Mode:**
|
||
- Call `/preview/start`
|
||
- Read frames from `/tmp/recording/preview/`
|
||
- Call `/preview/stop` when done
|
||
|
||
2. **Via Landmark Images:**
|
||
- Call `/landmarks/last/N` to get recent detections
|
||
- Call `/landmarks/images/:id` to get observation image paths
|
||
- Call `/landmarks/:id/chips/:chip_id` to get cropped JPEG
|
||
|
||
3. **Direct File Access (SSH):**
|
||
- Read from `/tmp/recording/pics/` for latest frames
|
||
- Read from `/data/recording/cached_observations/` for landmark images
|
||
|
||
---
|
||
|
||
## 8. Full Data Flow
|
||
|
||
### 8.1 Complete Pipeline
|
||
|
||
```
|
||
┌──────────────────────────────────────────────────────────────────────────────┐
|
||
│ CAMERA CAPTURE │
|
||
│ IMX378 Sensor → MIPI CSI-2 → VPU ISP → DepthAI Pipeline │
|
||
└───────────────────────────────────┬──────────────────────────────────────────┘
|
||
│
|
||
▼
|
||
┌──────────────────────────────────────────────────────────────────────────────┐
|
||
│ DEPTHAI_GATE (port 11492) │
|
||
│ - XLink communication with Myriad X VPU │
|
||
│ - Frame capture from DepthAI pipeline │
|
||
│ - Writes frames to /tmp/recording/pics/ │
|
||
└───────────────────────────────────┬──────────────────────────────────────────┘
|
||
│
|
||
┌───────────────┴───────────────┐
|
||
▼ ▼
|
||
┌───────────────────────────────┐ ┌──────────────────────────────────────────┐
|
||
│ RAW FRAME STORAGE │ │ MAP-AI INFERENCE │
|
||
│ /tmp/recording/pics/ │ │ - Road sign detection (VPU NCE) │
|
||
│ - Temporary frames │ │ - Privacy blur (faces/plates) │
|
||
│ - Purged when >400MB │ │ - Outputs to Redis + SQLite │
|
||
└───────────────────────────────┘ └─────────────────┬────────────────────────┘
|
||
│
|
||
┌─────────────────────────────────┤
|
||
▼ ▼
|
||
┌───────────────────────────────────┐ ┌──────────────────────────────────────┐
|
||
│ CACHED OBSERVATIONS │ │ LANDMARK DATABASE │
|
||
│ /data/recording/ │ │ /data/recording/odc-api.db │
|
||
│ cached_observations/ │ │ - landmarks table │
|
||
│ - Persistent blurred frames │ │ - observations table │
|
||
│ - Referenced by landmark ID │ │ - frames table │
|
||
└───────────────────┬───────────────┘ └─────────────────┬────────────────────┘
|
||
│ │
|
||
└──────────────┬──────────────────────┘
|
||
▼
|
||
┌──────────────────────────────────────────────────────────────────────────────┐
|
||
│ ODC-API (port 5000) │
|
||
│ - /preview/* — Start/stop preview mode │
|
||
│ - /landmarks/last/N — Get recent detections │
|
||
│ - /landmarks/images/:id — Get observation image paths │
|
||
│ - /landmarks/:id/chips/:chip_id — Get cropped JPEG │
|
||
└───────────────────────────────────┬──────────────────────────────────────────┘
|
||
│
|
||
▼
|
||
┌──────────────────────────────────────────────────────────────────────────────┐
|
||
│ FRAMEKM BUNDLING │
|
||
│ hivemapper-data-logger │
|
||
│ - Collect ~1km of frames + metadata │
|
||
│ - Bundle with GNSS auth signatures │
|
||
│ - Store at /data/recording/framekm/ │
|
||
└───────────────────────────────────┬──────────────────────────────────────────┘
|
||
│
|
||
▼
|
||
┌──────────────────────────────────────────────────────────────────────────────┐
|
||
│ UPLOAD PATH │
|
||
│ odc-api → mitmdump (port 8888) → Cloudflare Workers → HERE OLP │
|
||
└──────────────────────────────────────────────────────────────────────────────┘
|
||
```
|
||
|
||
### 8.2 Single Detection Event Trace
|
||
|
||
```
|
||
1. Camera captures frame
|
||
└── IMX378 → MIPI → VPU ISP → depthai_gate
|
||
|
||
2. Frame written to disk
|
||
└── /tmp/recording/pics/1709920000123_0001_0042.jpg
|
||
|
||
3. map-ai reads frame
|
||
└── Runs road sign classifier on VPU NCE
|
||
|
||
4. Detection found (speed limit 35)
|
||
└── Privacy blur applied to any faces/plates
|
||
|
||
5. Observation stored
|
||
└── SQLite: observations table (landmark_id, bbox, ts, image_name)
|
||
└── File: /data/recording/cached_observations/...
|
||
|
||
6. Landmark created/updated
|
||
└── SQLite: landmarks table (class_label, lat, lon, confidence)
|
||
|
||
7. odc-api exposes data
|
||
└── GET /landmarks/last/5 returns detection
|
||
└── GET /landmarks/images/{id} returns image path
|
||
└── GET /landmarks/{id}/chips/{chip_id} returns cropped JPEG
|
||
```
|
||
|
||
---
|
||
|
||
## 9. Replacement Considerations
|
||
|
||
### 9.1 Accessing Frames Without odc-api
|
||
|
||
**Option 1: Direct File Read**
|
||
```bash
|
||
# SSH to Bee
|
||
ssh -p 2222 root@localhost # via Lucy tunnel
|
||
|
||
# Read latest frames
|
||
ls -lt /tmp/recording/pics/ | head -10
|
||
cp /tmp/recording/pics/latest_frame.jpg /tmp/
|
||
|
||
# Stream frames (naive)
|
||
while true; do
|
||
cp $(ls -t /tmp/recording/pics/*.jpg | head -1) /tmp/current.jpg
|
||
sleep 0.033 # ~30 FPS
|
||
done
|
||
```
|
||
|
||
**Pros:** Simple, no service changes
|
||
**Cons:** Race conditions, no metadata
|
||
|
||
**Option 2: Redis Pub/Sub**
|
||
Subscribe to frame events if depthai_gate publishes them:
|
||
```python
|
||
import redis
|
||
r = redis.Redis()
|
||
p = r.pubsub()
|
||
p.subscribe('frame_ready')
|
||
for message in p.listen():
|
||
print(message) # Contains frame path or metadata
|
||
```
|
||
|
||
**Pros:** Event-driven, no polling
|
||
**Cons:** May not exist in current firmware
|
||
|
||
### 9.2 Accessing Frames Without depthai_gate
|
||
|
||
**Not recommended** — requires implementing your own DepthAI pipeline.
|
||
|
||
If you must:
|
||
1. Stop depthai_gate: `systemctl stop depthai_gate`
|
||
2. Use Luxonis depthai Python SDK
|
||
3. Create minimal pipeline:
|
||
```python
|
||
import depthai as dai
|
||
|
||
pipeline = dai.Pipeline()
|
||
cam = pipeline.create(dai.node.ColorCamera)
|
||
cam.setResolution(dai.ColorCameraProperties.SensorResolution.THE_4_K)
|
||
cam.setIspScale(1, 2) # Downscale to 2028x1024
|
||
|
||
xout = pipeline.create(dai.node.XLinkOut)
|
||
xout.setStreamName("video")
|
||
cam.video.link(xout.input)
|
||
|
||
with dai.Device(pipeline) as device:
|
||
q = device.getOutputQueue("video")
|
||
while True:
|
||
frame = q.get()
|
||
cv2.imwrite("/tmp/frame.jpg", frame.getCvFrame())
|
||
```
|
||
|
||
**Pros:** Full control over camera
|
||
**Cons:** Breaks all Hivemapper services, loses ML pipeline
|
||
|
||
### 9.3 Minimal Path to JPEG Frame
|
||
|
||
**Fastest (with existing stack):**
|
||
```bash
|
||
# Via SSH
|
||
ssh -p 2222 root@localhost 'ls -t /tmp/recording/pics/*.jpg | head -1 | xargs cat' > frame.jpg
|
||
```
|
||
|
||
**Via API (requires preview mode):**
|
||
```bash
|
||
curl http://192.168.0.10:5000/api/1/preview/start
|
||
sleep 2
|
||
ssh -p 2222 root@localhost 'ls -t /tmp/recording/preview/*.jpg | head -1 | xargs cat' > frame.jpg
|
||
curl http://192.168.0.10:5000/api/1/preview/stop
|
||
```
|
||
|
||
### 9.4 Building a Custom Camera Interface
|
||
|
||
**Requirements:**
|
||
1. Maintain depthai_gate (or reimplement VPU control)
|
||
2. Expose a REST endpoint for single-frame capture
|
||
3. Optionally implement MJPEG streaming
|
||
|
||
**Proposed odc-api Addition:**
|
||
```typescript
|
||
// routes/camera.ts
|
||
router.get('/frame', async (req, res) => {
|
||
const frames = readdirSync('/tmp/recording/pics')
|
||
.filter(f => f.endsWith('.jpg'))
|
||
.sort()
|
||
.reverse();
|
||
|
||
if (frames.length === 0) {
|
||
return res.status(404).send('No frames available');
|
||
}
|
||
|
||
const framePath = join('/tmp/recording/pics', frames[0]);
|
||
res.sendFile(framePath);
|
||
});
|
||
|
||
router.get('/stream', async (req, res) => {
|
||
res.writeHead(200, {
|
||
'Content-Type': 'multipart/x-mixed-replace; boundary=frame',
|
||
'Cache-Control': 'no-cache',
|
||
});
|
||
|
||
const interval = setInterval(() => {
|
||
const frames = readdirSync('/tmp/recording/pics')
|
||
.filter(f => f.endsWith('.jpg'))
|
||
.sort()
|
||
.reverse();
|
||
|
||
if (frames.length > 0) {
|
||
const framePath = join('/tmp/recording/pics', frames[0]);
|
||
const frameData = readFileSync(framePath);
|
||
|
||
res.write('--frame\r\n');
|
||
res.write('Content-Type: image/jpeg\r\n');
|
||
res.write(`Content-Length: ${frameData.length}\r\n\r\n`);
|
||
res.write(frameData);
|
||
res.write('\r\n');
|
||
}
|
||
}, 33); // ~30 FPS
|
||
|
||
req.on('close', () => clearInterval(interval));
|
||
});
|
||
```
|
||
|
||
### 9.5 Architecture for Replacement System
|
||
|
||
```
|
||
┌──────────────────────────────────────────────────────────────────┐
|
||
│ VARROA CAMERA SERVICE │
|
||
├──────────────────────────────────────────────────────────────────┤
|
||
│ │
|
||
│ ┌─────────────────┐ ┌─────────────────┐ ┌──────────────┐ │
|
||
│ │ depthai_gate │ → │ varroa-camera │ → │ HTTP API │ │
|
||
│ │ (unchanged) │ │ (new service) │ │ (port 80) │ │
|
||
│ └─────────────────┘ └─────────────────┘ └──────────────┘ │
|
||
│ ↓ ↓ ↓ │
|
||
│ /tmp/recording/pics/ Monitor & serve GET /frame │
|
||
│ frames via inotify GET /stream │
|
||
│ GET /landmarks │
|
||
└──────────────────────────────────────────────────────────────────┘
|
||
```
|
||
|
||
---
|
||
|
||
## 10. Open Questions
|
||
|
||
| Question | Priority | How to Investigate |
|
||
|----------|----------|-------------------|
|
||
| Exact depthai_gate pipeline config | High | SSH in, find config files in /opt/ |
|
||
| Does depthai_gate publish to Redis? | High | `redis-cli MONITOR` while recording |
|
||
| Camera-bridge vs depthai_gate relationship | High | Check systemd deps, trace with strace |
|
||
| Preview config format | Medium | Read `getPreviewConfig()` implementation |
|
||
| ML model exact location on Bee | Medium | `find /opt /data -name "*.blob" -o -name "*.onnx"` |
|
||
| Frame timestamp accuracy | Medium | Compare frame timestamps to GNSS time |
|
||
|
||
---
|
||
|
||
## Appendix A: Key File Paths
|
||
|
||
| Path | Purpose |
|
||
|------|---------|
|
||
| `/tmp/recording/pics/` | Live camera frames |
|
||
| `/tmp/recording/preview/` | Preview mode frames |
|
||
| `/data/recording/cached_observations/` | Landmark observation images |
|
||
| `/data/recording/framekm/` | FrameKm upload bundles |
|
||
| `/data/recording/odc-api.db` | SQLite database |
|
||
| `/opt/camera-bridge/config.json` | Camera configuration |
|
||
| `/opt/depthai_gate/` | DepthAI service (estimated) |
|
||
| `/opt/odc-api/` | Node.js API service |
|
||
| `/sys/class/vpu/` | VPU sysfs interface |
|
||
|
||
## Appendix B: Service Dependencies
|
||
|
||
```
|
||
multi-user.target
|
||
│
|
||
├── redis.service [t+2s]
|
||
│
|
||
├── depthai_gate.service [t+8s] # MUST start before map-ai
|
||
│ │
|
||
│ └── Loads luxonis_vpu.bin
|
||
│
|
||
├── map-ai.service [t+12s] # Depends on depthai_gate
|
||
│ │
|
||
│ └── Privacy blur, ML inference
|
||
│
|
||
├── hivemapper-data-logger.service [t+15s]
|
||
│
|
||
└── odc-api.service [t+18s] # REST API
|
||
```
|
||
|
||
## Appendix C: Port Reference
|
||
|
||
| Port | Service | Protocol | Binding |
|
||
|------|---------|----------|---------|
|
||
| 22 | sshd | TCP | AP only (via socket) |
|
||
| 5000 | odc-api | HTTP | AP interface |
|
||
| 6379 | Redis | TCP | localhost |
|
||
| 8888 | mitmdump | HTTP | localhost |
|
||
| 11492 | depthai_gate | HTTP/Flask | localhost |
|
||
|
||
---
|
||
|
||
*End of Report*
|