Sulkta-OSS/strawcore
2
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

Phase 1 — Foundation

Browse source 
Mirror NPE's dependency-free spine in Rust:

* exceptions   — NetworkError + ParsingError + ContentUnavailable
                 + ExtractionError tree, with reqwest/serde_json conversions
* localization — Localization + ContentCountry, default (en, GB)
* downloader/  — Downloader trait, Request builder, Response,
                 reqwest blocking default impl
* page         — continuation-token carrier
* image        — Image + ImageSet + ResolutionLevel
                 (HEIGHT_UNKNOWN/WIDTH_UNKNOWN = -1)
* metainfo     — title/content/url/url_text grab-bag
* service      — StreamingService trait + LinkType + ServiceInfo
* newpipe      — process-global Downloader / Localization /
                 ContentCountry singleton

Foundational invariants nailed down (per SPEC §3):
* HTTP non-2xx returns Ok(Response); only 429 throws NetworkError::Recaptcha
* Response header keys lowercase-normalized
* Request.add_header PARITY with NPE bug (silent overwrite);
  append_header is our clean addition
* default Localization is en-GB
* No cookie jar in the default downloader

Tests: 7 unit + 7 live smoke against httpbin.org (gated on
'online-tests' feature). All green.
This commit is contained in:
Kayos 2026-05-24 16:32:36 -07:00
parent f44b46fab5
commit 46201c731f
16 changed files with 2689 additions and 1 deletions
Download patch file Download diff file Expand all files Collapse all files

35
README.md
View file

@ -1,3 +1,36 @@
# strawcore
Rust port of NewPipeExtractor (YT-only). Plugs into Straw via UniFFI.
Rust port of [NewPipeExtractor](https://github.com/TeamNewPipe/NewPipeExtractor) (v0.26.2), YouTube-only. Plugs into [Straw](http://192.168.0.5:3001/Sulkta-Coop/straw) via UniFFI.
## Why this exists
`rustypipe` regex-parses YouTube's `player.js` and reimplements the signature deobfuscator in Rust. Every YT player rotation breaks it. NPE embeds Mozilla Rhino and executes the JS function live — resilient by design, and that's the architecture we're mirroring.
The rustypipe-backed Straw build (vc=15..17) also routed playback through iOS-progressive URLs, which hit a server-side ~917 KiB end-byte cap. NPE uses the Android client + po_token → DASH manifest path, which doesn't see the cap. Same fix, different layer.
See `memory/npe-audit-2026-05-24/SPEC.md` in the workspace repo for the full plan.
## Status
| Phase | Subsystem | Status |
|---|---|---|
| 1 | Foundation (downloader + service spine) | **in progress** |
| 2 | JS engine (rquickjs + ress) | pending |
| 3 | InnerTube + itag table | pending |
| 4 | Stream extractor + DASH | pending |
| 5 | PoTokenProvider trait + Android JNI bridge | pending |
| 6 | Search + Channel + Playlist + Kiosks | pending |
| 7 | UniFFI surface swap | pending |
| 8 | Delete rustypipe everywhere | pending |
## Build + test
```bash
cargo build
cargo test --lib # offline unit tests
cargo test --features online-tests # full smoke incl. live httpbin.org
```
## License
GPL-3.0-or-later. NPE is GPL-3.0; this port inherits.