Rust port of NewPipeExtractor (YT-only). Plugs into Straw via UniFFI.
Kayos
f79d8fb109
Pulls in the read-side extractor surfaces Straw needs at app open
(search bar) + on detail screens (channel + playlist).
src/youtube/linkhandler/
* mod.rs — ACCEPTED_HOSTS allowlist (youtube.com /
youtube-nocookie.com / youtu.be / m.youtube.com /
music.youtube.com); 27 Invidious mirror hosts
intentionally dropped (SPEC §6.6).
* stream.rs — extract_video_id() handles /watch?v= / youtu.be/ /
/embed/ / /shorts/ / /v/ / /live/ / attribution_link;
strict 11-char [A-Za-z0-9_-] validation.
* channel.rs — ChannelIdentifier enum (DirectId / Handle / Custom /
LegacyUser). Resolution to UC… id lands in
youtube/channel.rs.
* playlist.rs — extracts ?list=<PLid> from /playlist and /watch URLs.
* search.rs — SearchFilter enum + params() opaque base64 strings +
uses_music_endpoint() routing flag.
src/youtube/search_extractor.rs
* search(query, filter) → SearchInfo { query, corrected_query,
videos, continuation_token }
* Walks twoColumnSearchResultsRenderer → sectionListRenderer →
itemSectionRenderer → videoRenderer (+ shelfRenderer recursion).
* Parses YT duration strings, view-count abbreviations ('1.5M views'),
publishedTimeText, ownerBadges verified flag, badge LIVE flag.
* Music-search filters route to WEB_REMIX — flagged as not-yet-impl.
src/youtube/suggestion_extractor.rs
* suggestions(query) → Vec<String> via the suggestqueries-clients6
endpoint; handles both XSSI-prefixed and bare JSON responses.
src/youtube/channel.rs
* resolve_handle_to_channel_id() via /youtubei/v1/navigation/resolve_url
* channel_info(ChannelIdentifier) → ChannelInfo {
name, description, avatars, banners, subscriber_count, verified,
recent_videos, videos_continuation
}
* Parses both c4TabbedHeaderRenderer (most common) and the newer
pageHeaderRenderer flavor.
* subscriber_count parser handles K/M/B suffixes.
src/youtube/playlist_extractor.rs
* playlist_info(playlist_id) → PlaylistInfo with first-page video
list + continuation_token. Browses with browseId='VL<id>'.
* Walks playlistMetadataRenderer + playlistSidebarRenderer + the
playlistVideoListRenderer.contents[] for video items.
Tests: 121 lib unit pass (+44 since Phase 5). All previous phase smoke
tests still green.
What's left:
* Phase 6 kiosks (Trending etc) — minor, deferred
* Phase 7 — UniFFI surface swap into Straw (Straw repo work)
* Phase 8 — delete rustypipe (Straw repo work)
|
||
|---|---|---|
| src | ||
| tests | ||
| .gitignore | ||
| Cargo.lock | ||
| Cargo.toml | ||
| LICENSE | ||
| README.md | ||
strawcore
Rust port of NewPipeExtractor (v0.26.2), YouTube-only. Plugs into Straw via UniFFI.
Why this exists
rustypipe regex-parses YouTube's player.js and reimplements the signature deobfuscator in Rust. Every YT player rotation breaks it. NPE embeds Mozilla Rhino and executes the JS function live — resilient by design, and that's the architecture we're mirroring.
The rustypipe-backed Straw build (vc=15..17) also routed playback through iOS-progressive URLs, which hit a server-side ~917 KiB end-byte cap. NPE uses the Android client + po_token → DASH manifest path, which doesn't see the cap. Same fix, different layer.
See memory/npe-audit-2026-05-24/SPEC.md in the workspace repo for the full plan.
Status
| Phase | Subsystem | Status |
|---|---|---|
| 1 | Foundation (downloader + service spine) | in progress |
| 2 | JS engine (rquickjs + ress) | pending |
| 3 | InnerTube + itag table | pending |
| 4 | Stream extractor + DASH | pending |
| 5 | PoTokenProvider trait + Android JNI bridge | pending |
| 6 | Search + Channel + Playlist + Kiosks | pending |
| 7 | UniFFI surface swap | pending |
| 8 | Delete rustypipe everywhere | pending |
Build + test
cargo build
cargo test --lib # offline unit tests
cargo test --features online-tests # full smoke incl. live httpbin.org
License
GPL-3.0-or-later. NPE is GPL-3.0; this port inherits.