The channel Videos tab and search both parsed the page-1 continuation
token but had no way to fetch the next page. Add the next-page fetchers:
- youtube/continuation.rs (new): shared ContinuationPage + the
appendContinuationItemsAction/continuationItems unwrap + trailing-token
scan, parameterised on the container key — onResponseReceivedActions
for browse/channel, onResponseReceivedCommands for search (matches NPE).
- channel.rs: channel_videos_continuation() POSTs browse with
{continuation: token}; reuse the page-1 rich-grid item parser (factored
out as parse_rich_grid_item, now also accepting a bare
lockupViewModel/videoRenderer cell so a layout variant can't silently
return an empty page while the token keeps advancing).
- search_extractor.rs: search_continuation() POSTs search with
{continuation: token}; walk itemSectionRenderer contents via the
existing extract_item_into.
Empty/last page yields a None token so callers stop. New unit tests over
realistic continuation fixtures (bare-lockup, shelf-in-continuation,
last-page, empty body).
Round-2 cruft audit punch list — mechanical deletes, no behavior change.
Whole modules deleted (no wrapper consumer):
* youtube/playlist_extractor.rs (297 LOC) — full playlist extraction
* youtube/linkhandler/playlist.rs (81 LOC) — playlist URL parser
* youtube/suggestion_extractor.rs (91 LOC) — search-as-you-type
* tests/stream_phase4_offline.rs (186 LOC) — tautological test
Dead pub fns + enum variants + constants:
* WEB_REMIX_* constants (3) + WEB_MUSIC_ANALYTICS_* constants (3)
* InnertubeClientRequestInfo::of_web_music_analytics_charts_client
factory + its charts_client_omits_platform_and_screen test
* SearchFilter::Music{Songs,Videos,Albums,Playlists,Artists} variants
(5 of 9 cases) + uses_music_endpoint helper + the search_extractor
'music search not implemented' reject branch
* Two #[allow(dead_code)] _suppress_unused stub fns and the imports
they were keeping alive (std::sync::Arc in js/extractor.rs,
NetworkError in stream_extractor.rs)
Renamed:
* search_extractor::test_helpers -> renderer_helpers. Mis-named:
it's production code called from channel.rs, not a test fixture.
potoken/ kept and documented as the designed Phase-5 extension point
for YouTube bot-detection — wrapper's Android side hasn't registered
a real provider yet, but the trait + global slot stay so when YT
forces po_token universally the integration is one Kotlin patch away,
not a Rust-side rewrite.
~580 LOC removed from production. Wrapper does not need to change.
Mirrors NPE PoTokenProvider.java + PoTokenResult.java; defines the
host-injection surface for BotGuard attestation. The Rust crate stays
out of the BotGuard business — embedders (Straw on Android, future
Sulkta CLI via Browserless, etc.) supply their own impl.
src/youtube/potoken/mod.rs
* PoTokenResult { player_request_po_token, streaming_data_po_token,
visitor_data } + ::new + ::single constructors
* PoTokenError (Unavailable, MintFailed) — FIX vs NPE: split 'declined'
(Ok(None)) from 'errored' (Err) so callers can react differently
* trait PoTokenProvider with 4 client-scoped methods; default impl
returns Ok(None) so embedders can override just what they support
* set_po_token_provider / clear_po_token_provider / po_token_provider
static registration via RwLock<Option<Arc<dyn PoTokenProvider>>>
src/youtube/potoken/noop.rs
* NoopPoTokenProvider — safe default
src/youtube/stream_extractor.rs
* resolve_po_token via options-first-then-provider helper
(options_or_provider)
* Android branch: pulls player_request_po_token + visitor_data into
/player body, streams streaming_data_po_token through to URL &pot=
* iOS branch: same shape, gated on fetch_ios_client AND non-empty
provider result
Kotlin side (PoTokenWebView lift into Straw via UniFFI's foreign-trait
bridge) is separate work — strawcore just owns the contract.
Tests: 77 lib unit pass (+4 since Phase 4) + 7 Phase 2 offline + 7
Phase 4 offline = 91 green.
Port NewPipeExtractor's JS pipeline: player.js fetch + cache, sig and
nsig function extraction, deobfuscation, sticky-error caching.
src/youtube/js/
* runtime.rs — rquickjs wrapper (mirrors utils/JavaScript.java)
compile_or_throw + run(snippet, name, parameter)
* lexer.rs — match_to_closing_brace via the `ress` JS scanner
(NPE's lexer is derived from the same crate
upstream)
* extractor.rs — iframe_api → embed page fallback for player.js
URL, regex-driven hash extraction, clean-and-fetch
* signature.rs — 6 sig fn name regexes (front-most-recent),
deobf-function-body via lexer w/ regex fallback,
helper-object + global-string-array extraction,
signatureTimestamp, snippet assembler
* nsig.rs — 8 nsig fn name regexes (incl. array-indirection),
body via lexer w/ regex fallback, fixupFunction
early-return strip
* player_manager.rs — orchestrator + sticky-error cache mirroring
YoutubeJavaScriptPlayerManager
PORT DEVIATIONS from NPE (each flagged in code):
* dropped the 6th sig fn name regex (used Java backref \2; Rust's
`regex` crate is backtracking-free, so we substitute a loose form
that NPE itself half-broke per audit Track B §2.1)
* dropped the Java atomic group `(?>...)` from helper-object regex —
Rust's NFA is already linear-time
* nsig fixup substitutes `(?:"undefined"|'undefined')` for the
\1 backref; harmless loosening
* sig and nsig assembled snippets prepend `var` — QuickJS rejects
bare-assignment to undeclared identifiers; NPE relied on Rhino's
non-strict mode
Tests:
* 43 lib unit tests (up from 7 in Phase 1)
* 7 Phase 2 offline integration tests against a hand-crafted
minified synthetic player.js — exercises the full sig pipeline
(build_deobfuscator → runtime::run) and nsig fixup_function
* 7 Phase 1 live smoke tests still green
57/57 total green.
