Sulkta-Coop/strawcore
2
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions
Rust port of NewPipeExtractor (YT-only). Plugs into Straw via UniFFI.
4 commits 1 branch 0 tags 211 KiB
Find a file
Kayos 3014410cba Phase 3 — InnerTube + itag 
Port the YT client matrix + request envelope + itag lookup table.

src/youtube/
  * constants.rs       — ClientsConstants.java verbatim. All six live
                         clients (WEB, WEB_EMBEDDED_PLAYER,
                         WEB_MUSIC_ANALYTICS, ANDROID, IOS, plus the
                         WEB_REMIX values for completeness). Base URLs
                         + prettyPrint=false suffix.
  * client_request.rs  — ClientInfo / DeviceInfo / InnertubeClientRequestInfo
                         + the 5 factory constructors NPE exposes
                         (ofWebClient, ofWebEmbeddedPlayer, ofCharts,
                         ofAndroid, ofIos). build_envelope() emits the
                         InnerTube JSON in NPE's exact insertion order;
                         build_desktop_envelope() is the WEB-fast-path
                         used by search/browse/next/resolve_url/comments.
  * itag.rs            — 57-entry itag table (14 progressive + 10 audio +
                         33 video-only). MediaFormat enum + ItagType
                         enum + ItagItem struct + lookup().
  * parsing.rs         — consent toggle + cookie generator (SOCS=CAE= /
                         SOCS=CAISAiAD), WEB client-version cache + sw.js
                         scrape, WEB/mobile header builders (mobile
                         deliberately strips X-YouTube-Client-Name +
                         Origin/Referer + Cookie per audit Track A §6.2),
                         android/ios UA templates, visitor_data bootstrap
                         POST to /youtubei/v1/visitor_id.

PARITY notes flagged in code:
  * androidSdkVersion=36 + osVersion=16 but Android-15 in UA — NPE-intentional
  * mobile clients send NO X-YouTube-Client-* headers
  * audit doc says "53 entries" but tallies + NPE source = 57 ItagItems

Tests: 62 lib unit pass (up from 43 in Phase 2). All Phase 1 + Phase 2
smoke still green. Live InnerTube POSTs (visitor_data bootstrap +
/player) deferred to Phase 4 integration.
2026-05-24 16:57:47 -07:00
src Phase 3 — InnerTube + itag 2026-05-24 16:57:47 -07:00
tests Phase 2 — JS deobfuscator (rquickjs + ress) 2026-05-24 16:53:19 -07:00
.gitignore Initial commit 2026-05-24 16:26:57 -07:00
Cargo.lock Phase 2 — JS deobfuscator (rquickjs + ress) 2026-05-24 16:53:19 -07:00
Cargo.toml Phase 2 — JS deobfuscator (rquickjs + ress) 2026-05-24 16:53:19 -07:00
LICENSE Initial commit 2026-05-24 16:26:57 -07:00
README.md Phase 1 — Foundation 2026-05-24 16:32:36 -07:00

README.md

strawcore

Rust port of NewPipeExtractor (v0.26.2), YouTube-only. Plugs into Straw via UniFFI.

Why this exists

rustypipe regex-parses YouTube's player.js and reimplements the signature deobfuscator in Rust. Every YT player rotation breaks it. NPE embeds Mozilla Rhino and executes the JS function live — resilient by design, and that's the architecture we're mirroring.

The rustypipe-backed Straw build (vc=15..17) also routed playback through iOS-progressive URLs, which hit a server-side ~917 KiB end-byte cap. NPE uses the Android client + po_token → DASH manifest path, which doesn't see the cap. Same fix, different layer.

See memory/npe-audit-2026-05-24/SPEC.md in the workspace repo for the full plan.

Status

Phase Subsystem Status
1 Foundation (downloader + service spine) in progress
2 JS engine (rquickjs + ress) pending
3 InnerTube + itag table pending
4 Stream extractor + DASH pending
5 PoTokenProvider trait + Android JNI bridge pending
6 Search + Channel + Playlist + Kiosks pending
7 UniFFI surface swap pending
8 Delete rustypipe everywhere pending

Build + test

cargo build
cargo test --lib                          # offline unit tests
cargo test --features online-tests        # full smoke incl. live httpbin.org

License

GPL-3.0-or-later. NPE is GPL-3.0; this port inherits.