Kayos 46201c731f Phase 1 — Foundation

Mirror NPE's dependency-free spine in Rust:

* exceptions   — NetworkError + ParsingError + ContentUnavailable
                 + ExtractionError tree, with reqwest/serde_json conversions
* localization — Localization + ContentCountry, default (en, GB)
* downloader/  — Downloader trait, Request builder, Response,
                 reqwest blocking default impl
* page         — continuation-token carrier
* image        — Image + ImageSet + ResolutionLevel
                 (HEIGHT_UNKNOWN/WIDTH_UNKNOWN = -1)
* metainfo     — title/content/url/url_text grab-bag
* service      — StreamingService trait + LinkType + ServiceInfo
* newpipe      — process-global Downloader / Localization /
                 ContentCountry singleton

Foundational invariants nailed down (per SPEC §3):
* HTTP non-2xx returns Ok(Response); only 429 throws NetworkError::Recaptcha
* Response header keys lowercase-normalized
* Request.add_header PARITY with NPE bug (silent overwrite);
  append_header is our clean addition
* default Localization is en-GB
* No cookie jar in the default downloader

Tests: 7 unit + 7 live smoke against httpbin.org (gated on
'online-tests' feature). All green.

2026-05-24 16:32:36 -07:00

1.5 KiB

Raw Blame History

strawcore

Rust port of NewPipeExtractor (v0.26.2), YouTube-only. Plugs into Straw via UniFFI.

Why this exists

rustypipe regex-parses YouTube's player.js and reimplements the signature deobfuscator in Rust. Every YT player rotation breaks it. NPE embeds Mozilla Rhino and executes the JS function live — resilient by design, and that's the architecture we're mirroring.

The rustypipe-backed Straw build (vc=15..17) also routed playback through iOS-progressive URLs, which hit a server-side ~917 KiB end-byte cap. NPE uses the Android client + po_token → DASH manifest path, which doesn't see the cap. Same fix, different layer.

See memory/npe-audit-2026-05-24/SPEC.md in the workspace repo for the full plan.

Status

Phase	Subsystem	Status
1	Foundation (downloader + service spine)	in progress
2	JS engine (rquickjs + ress)	pending
3	InnerTube + itag table	pending
4	Stream extractor + DASH	pending
5	PoTokenProvider trait + Android JNI bridge	pending
6	Search + Channel + Playlist + Kiosks	pending
7	UniFFI surface swap	pending
8	Delete rustypipe everywhere	pending

Build + test

cargo build
cargo test --lib                          # offline unit tests
cargo test --features online-tests        # full smoke incl. live httpbin.org

License

GPL-3.0-or-later. NPE is GPL-3.0; this port inherits.