The crate had zero logging — every failure, fallback, cache reset, and
empty-parse was silent by construction, which made the 2026 bot-wall outage
nearly undiagnosable. Add `log = "0.4"` and 52 statements across the extractor
following a one-INFO-per-outcome / WARN-on-every-failure / DEBUG-for-internals
policy, so a future break shows up in a log grep instead of a debugger.
Highlights: nsig->throttled-URL fallback (aggregated one WARN per extraction,
never per-format, via a threaded UrlProcessStats counter); the nsig
identity-output trap now returns DeobfError::NsigIdentity and is NOT cached
(previously it cached and permanently throttled); player.js build/eval/install/
StillBad lifecycle; the ANDROID->VISIONOS cascade outcome; visitorData decline +
reset; HTTP 429 bot-flag; channel/search layout-change empties; per-request
DEBUG (query stripped). Plus a `BotDetected` error variant (Display byte-
identical to the old string) so the wall is greppable.
No secrets: only videoId/channel ids, error Displays (already URL/token-scrubbed
at the exceptions.rs choke points), query-stripped endpoints/player.js URLs,
counts, and lengths are logged — never token/poToken/visitorData/signature
values or response bodies.
- nsig name resolution: route direct-vs-array on capture-group PARTICIPATION,
not declared-group count (regexes 6/7 carry an optional array group) — a
direct-call player.js shape no longer hard-fails NsigArrayLookupFailed.
- nsig fixup_function: also strip the `if(typeof X==="undefined")return a;`
guard on the `function name(a){...}` body shape (previously required a
leading `;`), so the regex-fallback shape no longer silently yields an
identity result that gets cached as a permanent throttle.
- player_manager: never hold the state lock across the network; serialize
player.js downloads behind a fetch gate; add a per-artifact failure memo
(5-min cooldown) so a persistently-broken player.js costs one refetch per
open instead of a ~20-25 x 1.7MB storm. Transient-rotation self-heal
preserved (eval failure -> invalidate + refetch, no memo).
- errors no longer leak watch URLs / video ids: From<reqwest::Error> uses
without_url(); Recaptcha + link errors reduce to scheme+host (paths like
/embed/<id> and /shorts/<id> carried the id in the path).
- revive tests/ after the strawcore-core rename; js_phase2_offline green.
133 lib + 7 integration tests, clippy -D clean.
Second pass through the cruft inventory. All deletes — no behavior change.
Method enum + downloader trait:
* Method::Head / Put / Delete dropped (no caller). Match arms in
request.rs::as_str() and default_impl.rs::execute() collapse to
just Get / Post.
* Request::head builder dropped.
* Downloader trait's get / get_localized / head / post default
methods dropped. Every caller went through execute() directly
anyway; the convenience wrappers carried 4 dead arms each.
Parsing module:
* bootstrap_visitor_data — pub fn, no caller.
* discover_web_client_version + CACHED_WEB_CLIENT_VERSION +
reset_web_client_version_cache — entire sw.js live-version
discovery pipeline, never wired up by any caller. The cache was
never populated, so web_client_version() always returned the
hardcoded constant. Collapsed to just returning the constant.
* Drops once_cell::Lazy, parking_lot::RwLock around the version
cache (consent flag still uses RwLock), Regex import, serde_json::Value
import, downloader/exceptions/Request/InnertubeClientRequestInfo
imports — all only kept alive by the deleted code.
stream_helper:
* get_web_embedded_player_response — pub fn, no caller.
js/player_manager + extractor:
* player_manager::player_hash — pub fn, no caller. Was only kept
alive by its own definition.
* extractor::extract_player_hash — pub fn, only called by the now-
dead player_hash. Test removed alongside.
Stale comments:
* itag.rs:1 header claimed 53 entries; ITAG_TABLE has 57 and the
test at line 179 already asserts it.
* js/mod.rs:12-13 claimed the submodules were 'crate-private
plumbing' but they're declared pub mod. Tightened the comment to
explain the integration-test dependency that keeps them public.
Net delete: ~170 LOC of dead surface across 9 files.
Port NewPipeExtractor's JS pipeline: player.js fetch + cache, sig and
nsig function extraction, deobfuscation, sticky-error caching.
src/youtube/js/
* runtime.rs — rquickjs wrapper (mirrors utils/JavaScript.java)
compile_or_throw + run(snippet, name, parameter)
* lexer.rs — match_to_closing_brace via the `ress` JS scanner
(NPE's lexer is derived from the same crate
upstream)
* extractor.rs — iframe_api → embed page fallback for player.js
URL, regex-driven hash extraction, clean-and-fetch
* signature.rs — 6 sig fn name regexes (front-most-recent),
deobf-function-body via lexer w/ regex fallback,
helper-object + global-string-array extraction,
signatureTimestamp, snippet assembler
* nsig.rs — 8 nsig fn name regexes (incl. array-indirection),
body via lexer w/ regex fallback, fixupFunction
early-return strip
* player_manager.rs — orchestrator + sticky-error cache mirroring
YoutubeJavaScriptPlayerManager
PORT DEVIATIONS from NPE (each flagged in code):
* dropped the 6th sig fn name regex (used Java backref \2; Rust's
`regex` crate is backtracking-free, so we substitute a loose form
that NPE itself half-broke per audit Track B §2.1)
* dropped the Java atomic group `(?>...)` from helper-object regex —
Rust's NFA is already linear-time
* nsig fixup substitutes `(?:"undefined"|'undefined')` for the
\1 backref; harmless loosening
* sig and nsig assembled snippets prepend `var` — QuickJS rejects
bare-assignment to undeclared identifiers; NPE relied on Rhino's
non-strict mode
Tests:
* 43 lib unit tests (up from 7 in Phase 1)
* 7 Phase 2 offline integration tests against a hand-crafted
minified synthetic player.js — exercises the full sig pipeline
(build_deobfuscator → runtime::run) and nsig fixup_function
* 7 Phase 1 live smoke tests still green
57/57 total green.
