Sulkta-Coop/adacam
2
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

add ADAMAPS-TECHNICAL.md from research sprint

Browse source
This commit is contained in:
kayos 2026-03-22 11:09:44 -07:00
parent 35b0694b53
commit cacdce1f34
1 changed files with 528 additions and 0 deletions
Download patch file Download diff file Expand all files Collapse all files

528
docs/ADAMAPS-TECHNICAL.md Normal file
View file

@ -0,0 +1,528 @@
# ADAMaps Technical Documentation
*Last updated: 2026-03-22*
---
## 1. Overview
ADAMaps is a community-owned road sign mapping network built on:
- **Hivemapper Bee dashcams** (liberated from Hivemapper's proprietary ecosystem)
- **PostGIS** for geospatial data storage
- **Flask API** on Rackham (public VPS) for data ingest and serving
- **MAP token** (Cardano native asset) for contributor rewards
- **Agent Training API** for AI-based sign classification
The system has three main code repositories:
1. `Sulkta-Coop/adamaps` — Backend API, database schema, web frontend, deployment scripts
2. `Sulkta-Coop/adacam-api` — On-device API running on liberated Bee dashcams
3. `Sulkta-Coop/adacam` — Liberation scripts, firmware recovery, security research
---
## 2. API Architecture
### 2.1 Main API (Rackham)
**Location:** Deployed on Rackham VPS (142.44.213.229)
**Framework:** Flask (Python)
**WSGI:** Gunicorn (2 workers)
**Port:** 5001 (internal), reverse-proxied via ISPConfig to `api.adamaps.org`
#### Core Endpoints
| Method | Path | Auth | Description |
|--------|------|------|-------------|
| GET | `/api/health` | None | Health check, returns `{"status": "ok", "node": "rackham"}` |
| GET | `/api/stats` | None | Total detections, devices, images, signs counts |
| GET | `/api/detections` | None | GeoJSON-style detection list (`?limit=N`) |
| GET | `/api/signs` | None | Clustered sign inventory (`?verified=true`, `?limit=N`) |
| POST | `/api/ingest` | `X-AdaMaps-Key` | Ingest detection batch from devices |
| POST | `/api/images` | `X-AdaMaps-Key` | Upload detection images (multipart) |
| GET | `/api/images/<filename>` | None | Serve stored images |
#### Agent Training Endpoints
| Method | Path | Auth | Description |
|--------|------|------|-------------|
| GET | `/api/agent/info` | None | API discovery, sign types, reward structure |
| GET | `/api/agent/task` | `X-Agent-Wallet` | Get task (image) to label |
| POST | `/api/agent/submit` | `X-Agent-Wallet` | Submit label for task |
| GET | `/api/agent/stats` | `X-Agent-Wallet` | Agent performance stats |
| GET | `/api/agent/leaderboard` | None | Top agents by earnings |
| GET | `/.well-known/agent-training.json` | None | Discovery endpoint |
**Note:** Agent API is gated by `AGENT_API_LIVE = False`. When image pipeline is ready, flip to `True` to enable task generation.
#### Authentication
**Device Ingest:**
```
Header: X-AdaMaps-Key: adamaps-ingest-2026
```
**Agent Operations:**
```
Header: X-Agent-Wallet: addr1qxy... (Cardano wallet address)
```
No API key required for agents — the wallet IS the identity.
### 2.2 On-Device API (adacam-api)
Runs on each liberated Hivemapper Bee dashcam.
**Framework:** Flask
**Port:** 5000
**Data sources:** Redis (GPS/IMU from camera pipeline), SQLite (local detection storage)
#### Endpoints
| Method | Path | Auth | Description |
|--------|------|------|-------------|
| GET | `/pair` | None | Pairing info for Varroa app |
| GET | `/api/1/info` | None | Device identity |
| GET | `/api/1/landmarks/last/<N>` | None | Last N detections |
| POST | `/api/1/landmarks` | Token | Ingest new detection |
| GET | `/api/1/gnssConcise/latestValid` | None | Current GPS fix |
| GET | `/api/1/status` | None | Device status |
| GET | `/api/1/recording/frames/latest` | None | Latest frame path |
| GET | `/api/1/wifi/status` | None | WiFi client status |
| POST | `/api/1/wifi/connect` | Token | Connect to WiFi |
| GET | `/api/1/ssh/status` | None | SSH daemon status |
| POST | `/api/1/ssh/toggle` | Token | Enable/disable SSH |
**Config file:** `/data/adacam/config.json`
```json
{
"device_id": "auto-generated UUID",
"adamaps_key": "adamaps-ingest-2026",
"adamaps_api": "https://api.adamaps.org",
"ap_interface": "wlp1s0f0"
}
```
---
## 3. Database Schema
### 3.1 PostGIS Database (adamaps-postgres)
**Image:** `postgis/postgis:16-3.4`
**Container:** `adamaps-postgres`
**Credentials:** `adamaps:adamaps2026`
**Database:** `adamaps`
#### Tables
**`detections`** — Raw detection observations
```sql
CREATE TABLE detections (
id SERIAL PRIMARY KEY,
device_id TEXT NOT NULL,
detected_at TIMESTAMPTZ NOT NULL DEFAULT NOW(),
lat DOUBLE PRECISION NOT NULL,
lon DOUBLE PRECISION NOT NULL,
geom GEOMETRY(Point, 4326) GENERATED ALWAYS AS
(ST_SetSRID(ST_MakePoint(lon, lat), 4326)) STORED,
sign_type TEXT,
confidence DOUBLE PRECISION,
speed_mph DOUBLE PRECISION,
heading DOUBLE PRECISION,
image_path TEXT,
wallet_address TEXT,
raw_json JSONB
);
-- Indexes
CREATE INDEX detections_geom_idx ON detections USING GIST(geom);
CREATE INDEX detections_device_idx ON detections(device_id);
CREATE INDEX detections_time_idx ON detections(detected_at DESC);
```
**`signs`** — Clustered, deduplicated sign inventory (created by clustering job)
```sql
CREATE TABLE signs (
id SERIAL PRIMARY KEY,
lat DOUBLE PRECISION,
lon DOUBLE PRECISION,
sign_type TEXT,
heading TEXT,
observation_count INTEGER,
avg_confidence DOUBLE PRECISION,
first_seen TIMESTAMPTZ,
last_seen TIMESTAMPTZ,
device_count INTEGER,
verified BOOLEAN DEFAULT FALSE
);
```
**`agent_stats`** — Agent performance tracking
```sql
CREATE TABLE agent_stats (
wallet VARCHAR(128) PRIMARY KEY,
tasks_completed INTEGER DEFAULT 0,
consensus_agreements INTEGER DEFAULT 0,
total_earned DOUBLE PRECISION DEFAULT 0,
pending_rewards DOUBLE PRECISION DEFAULT 0,
created_at TIMESTAMP DEFAULT NOW(),
updated_at TIMESTAMP DEFAULT NOW()
);
```
**`training_tasks`** — Image labeling tasks for agents
```sql
CREATE TABLE training_tasks (
task_id VARCHAR(64) PRIMARY KEY,
detection_id INTEGER NOT NULL,
image_url VARCHAR(512),
ai_guess VARCHAR(128),
ai_confidence DOUBLE PRECISION,
status VARCHAR(32) DEFAULT 'pending', -- pending | consensus | disputed
final_label VARCHAR(128),
labels_received INTEGER DEFAULT 0,
created_at TIMESTAMP DEFAULT NOW(),
expires_at TIMESTAMP,
consensus_at TIMESTAMP
);
```
**`task_labels`** — Individual agent label submissions
```sql
CREATE TABLE task_labels (
id SERIAL PRIMARY KEY,
task_id VARCHAR(64) NOT NULL,
wallet VARCHAR(128) NOT NULL,
label VARCHAR(128) NOT NULL,
confidence DOUBLE PRECISION,
reasoning TEXT,
reward_earned DOUBLE PRECISION DEFAULT 0,
submitted_at TIMESTAMP DEFAULT NOW(),
UNIQUE(task_id, wallet)
);
```
### 3.2 On-Device SQLite (adacam-api)
**Path:** `/data/adacam/adacam.db`
```sql
CREATE TABLE landmarks (
id INTEGER PRIMARY KEY,
class_label TEXT,
class_label_confidence REAL,
overall_confidence REAL,
lat REAL, lon REAL, alt REAL,
azimuth REAL,
width INTEGER, height INTEGER,
ts INTEGER,
forwarded INTEGER DEFAULT 0
);
CREATE TABLE forward_queue (
id INTEGER PRIMARY KEY AUTOINCREMENT,
payload TEXT,
created_at INTEGER DEFAULT (strftime('%s','now'))
);
```
---
## 4. Data Ingest Flow
### 4.1 Detection Ingest
**Endpoint:** `POST /api/ingest`
**Headers:**
```
X-AdaMaps-Key: adamaps-ingest-2026
Content-Type: application/json
```
**Payload formats accepted:**
Array of detections:
```json
[{
"device_id": "fvhL2I-iCT",
"lat": 33.8677,
"lon": -118.3776,
"class_label": "stop_sign",
"overall_confidence": 0.94,
"ts": 1710345600000,
"wallet_address": "addr1..."
}]
```
Or wrapped format:
```json
{
"device_id": "fvhL2I-iCT",
"detections": [...]
}
```
**Response:**
```json
{"inserted": 5, "device_id": "fvhL2I-iCT"}
```
### 4.2 Image Upload
**Endpoint:** `POST /api/images`
**Headers:**
```
X-AdaMaps-Key: adamaps-ingest-2026
Content-Type: multipart/form-data
```
**Form fields:**
- `detection_id` (required): ID of detection this image belongs to
- `device_id` (optional): Device identifier
- `image` (file): JPEG/PNG image file
**Response:**
```json
{
"status": "ok",
"filename": "fvhL2I-iCT_12345_a1b2c3d4.jpg",
"url": "https://api.adamaps.org/api/images/fvhL2I-iCT_12345_a1b2c3d4.jpg"
}
```
**Storage:** `/opt/adamaps/images/` on Rackham
### 4.3 Device → AdaMaps Flow
```
┌──────────────────┐ Redis ┌──────────────────┐
│ Camera Pipeline │───GNSSFusion30Hz──▶│ adacam-api │
│ (Bee onboard) │ │ (Flask:5000) │
└──────────────────┘ └────────┬─────────┘
│
SQLite + Forward Queue
│
▼
┌──────────────────────────┐
│ AdaMaps API (Rackham) │
│ https://api.adamaps.org │
│ POST /api/ingest │
└──────────────────────────┘
```
**Offline resilience:** adacam-api queues detections in SQLite's `forward_queue` table if Rackham is unreachable. Background thread retries every 30 seconds.
---
## 5. Infrastructure
### 5.1 Rackham VPS
**Hostname:** rackham.hazedhosting.com
**Public IP:** 142.44.213.229
**VPN IP:** 192.168.254.1 (WireGuard)
**Provider:** OVH (via HazedHosting reseller)
**SSH:** Port 22, key-based (cobb@, sudo password: `T3mLHfzb`)
**Docker containers:**
- `adamaps-api` — Flask API (port 5001 internal)
- `adamaps-postgres` — PostGIS database (internal only, accessible via VPN at 192.168.254.112)
**Reverse proxy:** ISPConfig vhost routes `api.adamaps.org` → `127.0.0.1:5001`
**Image storage:** `/opt/adamaps/images/`
### 5.2 Lucy (Unraid)
**Role:** Development, database (some configs point here)
**IP:** 192.168.0.5 (LAN), 192.168.254.112 (VPN)
**Services:**
- Gitea: `http://192.168.0.5:3001`
- Dev nginx: `http://192.168.0.5:9999` (artifact serving)
- Docker Compose stack (local testing)
### 5.3 VPN Topology
WireGuard mesh connecting infrastructure:
```
┌─────────────────┐ ┌─────────────────┐
│ Lucy │◄───────►│ Rackham │
│ 192.168.254.112│ VPN │ 192.168.254.1 │
│ (Unraid/home) │ │ (OVH VPS) │
└────────┬────────┘ └─────────────────┘
│
│ VPN (when device online)
│
┌────────▼────────┐
│ Bee Dashcam │
│ 192.168.254.x │
│ (mobile, LTE) │
└─────────────────┘
```
**Postgres access:** Rackham API connects to `192.168.254.112:5432` (Lucy's Postgres via VPN)
### 5.4 Bee Dashcam (AdaCam)
**Hardware:**
- Intel Keem Bay SoC (Movidius MV0212)
- Yocto Linux, dm-verity rootfs (verity disabled at runtime)
- eMMC storage (11 partitions, A/B slots)
- LTE modem (LE910C4-NF)
- Dual WiFi (AP + client interfaces)
**Key paths:**
- `/data` — Persistent storage (survives firmware flash)
- `/data/adacam/` — Config, SQLite database
- `/opt/adacam/` — adacam-api installation
- `/etc` — Overlay from `/data/overlay/current/`
**Services (post-liberation):**
- `adacam-api.service` — Flask API (systemd)
- `bee-tunnel.service` — SSH tunnel to Lucy
---
## 6. Current State
### 6.1 Deployed & Working
| Component | Status | Notes |
|-----------|--------|-------|
| AdaMaps API (Rackham) | ✅ Running | Core endpoints functional |
| PostGIS database | ✅ Running | On Lucy, accessible via VPN |
| Detection ingest | ✅ Working | `/api/ingest` accepts device data |
| Image upload | ✅ Working | `/api/images` stores to disk |
| Agent Training API | ⏸️ Code ready | `AGENT_API_LIVE = False`, awaiting image pipeline |
| Bee Unit 1 | ✅ Liberated | SSH working, tunnel to Lucy |
| Bee Unit 2 | ⚠️ Locked out | SSH key-only, recovery artifact v5 pending test |
| MAP Token | ❌ Not minted | Tokenomics designed, wallets not created |
| DAO/Governance | ❌ Not deployed | Agora governor planned |
### 6.2 Known Issues
1. **Unit 2 SSH lockout** — Old liberate.sh hardened sshd_config before writing keys. Recovery artifact `adacam-ssh-fix-v5.mender` is built and waiting for testing.
2. **Agent API disabled** — `AGENT_API_LIVE = False` because image upload pipeline isn't consistently flowing yet. Need Bee devices actively uploading images with detections.
3. **No clustering job** — The `signs` table is created by a clustering script that hasn't been deployed yet. Detections go in, but no deduplication/consensus runs.
4. **VPN dependency** — Rackham API currently connects to Lucy's Postgres via VPN. If tunnel drops, API returns 503. Should migrate to Postgres running locally on Rackham for production.
### 6.3 API Endpoints Summary
**Live:**
- `GET /api/health`
- `GET /api/stats`
- `GET /api/detections`
- `GET /api/signs` (404 if clustering not run)
- `POST /api/ingest`
- `POST /api/images`
- `GET /api/images/<filename>`
- `GET /api/agent/info`
- `GET /api/agent/leaderboard`
**Gated (returns "coming_soon"):**
- `GET /api/agent/task`
- `POST /api/agent/submit`
- `GET /api/agent/stats` (works but no data)
---
## 7. Sign Types & Classification
The system recognizes these sign types:
```
stop-sign, speed-limit, yield, one-way, no-parking,
crosswalk, school-zone, construction, street-name,
highway-sign, traffic-light, turn-restriction,
regulatory-sign, warning-sign, guide-sign,
not-a-sign, unknown
```
**Consensus mechanism:**
- 3+ agents must agree on a label
- Agreeing agents earn base + consensus bonus
- Disagreeing agents earn nothing
- If no consensus after 10 labels → human review queue
**Rewards:**
```json
{
"base_label": 0.5,
"consensus_bonus": 1.0,
"new_type_discovery": 2.0,
"false_positive_id": 0.25
}
```
---
## 8. Appendix
### 8.1 Environment Variables
**AdaMaps API (Rackham):**
```
DB_HOST=192.168.254.112
API_KEY=adamaps-ingest-2026
FLASK_ENV=production
```
**adacam-api (Bee device):**
```
ADAMAPS_KEY=adamaps-ingest-2026
ADAMAPS_API=https://api.adamaps.org
```
### 8.2 Docker Compose (Rackham)
```yaml
services:
adamaps-postgres:
image: postgis/postgis:16-3.4
container_name: adamaps-postgres
environment:
POSTGRES_DB: adamaps
POSTGRES_USER: adamaps
POSTGRES_PASSWORD: adamaps2026
volumes:
- adamaps-pgdata:/var/lib/postgresql/data
- ./init.sql:/docker-entrypoint-initdb.d/init.sql:ro
adamaps-api:
build: ./api
container_name: adamaps-api
environment:
DB_URL: postgresql://adamaps:adamaps2026@adamaps-postgres:5432/adamaps
API_KEY: adamaps-ingest-2026
ports:
- "127.0.0.1:5001:5000"
volumes:
- /opt/adamaps/images:/images
```
### 8.3 Gitea Access
```
Base URL: http://192.168.0.5:3001/api/v1
Token: 33a9eb57b58c262f4434c12028bc3a30b1ff7021
Organization: Sulkta-Coop
Repos: adamaps, adacam, adacam-api
```
### 8.4 SSH Keys (authorized for Bee devices)
```
ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIK87jxvlXvo60pxwdtyJsXeFsb4KsAiFx4FnyXz81kh7 cobb@adacam
ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIOQxwJU91TCxds34P18D3xRbu7rxlrgTUoml/H8nxeDK kayos@openclaw
```
---
*Document generated from Gitea repos: adamaps, adacam, adacam-api*