The crate had zero logging — every failure, fallback, cache reset, and
empty-parse was silent by construction, which made the 2026 bot-wall outage
nearly undiagnosable. Add `log = "0.4"` and 52 statements across the extractor
following a one-INFO-per-outcome / WARN-on-every-failure / DEBUG-for-internals
policy, so a future break shows up in a log grep instead of a debugger.
Highlights: nsig->throttled-URL fallback (aggregated one WARN per extraction,
never per-format, via a threaded UrlProcessStats counter); the nsig
identity-output trap now returns DeobfError::NsigIdentity and is NOT cached
(previously it cached and permanently throttled); player.js build/eval/install/
StillBad lifecycle; the ANDROID->VISIONOS cascade outcome; visitorData decline +
reset; HTTP 429 bot-flag; channel/search layout-change empties; per-request
DEBUG (query stripped). Plus a `BotDetected` error variant (Display byte-
identical to the old string) so the wall is greppable.
No secrets: only videoId/channel ids, error Displays (already URL/token-scrubbed
at the exceptions.rs choke points), query-stripped endpoints/player.js URLs,
counts, and lengths are logged — never token/poToken/visitorData/signature
values or response bodies.
Straw's wrapper crate already owns the name 'strawcore' (and that name
is baked into the Android .so file + Kotlin's System.loadLibrary call).
Renaming this extractor crate to 'strawcore-core' resolves the cargo
package-name collision so both can live in the same workspace dep tree.
The repository keeps the name strawcore.
Port NewPipeExtractor's JS pipeline: player.js fetch + cache, sig and
nsig function extraction, deobfuscation, sticky-error caching.
src/youtube/js/
* runtime.rs — rquickjs wrapper (mirrors utils/JavaScript.java)
compile_or_throw + run(snippet, name, parameter)
* lexer.rs — match_to_closing_brace via the `ress` JS scanner
(NPE's lexer is derived from the same crate
upstream)
* extractor.rs — iframe_api → embed page fallback for player.js
URL, regex-driven hash extraction, clean-and-fetch
* signature.rs — 6 sig fn name regexes (front-most-recent),
deobf-function-body via lexer w/ regex fallback,
helper-object + global-string-array extraction,
signatureTimestamp, snippet assembler
* nsig.rs — 8 nsig fn name regexes (incl. array-indirection),
body via lexer w/ regex fallback, fixupFunction
early-return strip
* player_manager.rs — orchestrator + sticky-error cache mirroring
YoutubeJavaScriptPlayerManager
PORT DEVIATIONS from NPE (each flagged in code):
* dropped the 6th sig fn name regex (used Java backref \2; Rust's
`regex` crate is backtracking-free, so we substitute a loose form
that NPE itself half-broke per audit Track B §2.1)
* dropped the Java atomic group `(?>...)` from helper-object regex —
Rust's NFA is already linear-time
* nsig fixup substitutes `(?:"undefined"|'undefined')` for the
\1 backref; harmless loosening
* sig and nsig assembled snippets prepend `var` — QuickJS rejects
bare-assignment to undeclared identifiers; NPE relied on Rhino's
non-strict mode
Tests:
* 43 lib unit tests (up from 7 in Phase 1)
* 7 Phase 2 offline integration tests against a hand-crafted
minified synthetic player.js — exercises the full sig pipeline
(build_deobfuscator → runtime::run) and nsig fixup_function
* 7 Phase 1 live smoke tests still green
57/57 total green.
