strawcore/src/youtube/js/lexer.rs
Sulkta 6ad0c52aaf Phase 2 — JS deobfuscator (rquickjs + ress)
Port NewPipeExtractor's JS pipeline: player.js fetch + cache, sig and
nsig function extraction, deobfuscation, sticky-error caching.

src/youtube/js/
  * runtime.rs        — rquickjs wrapper (mirrors utils/JavaScript.java)
                        compile_or_throw + run(snippet, name, parameter)
  * lexer.rs          — match_to_closing_brace via the `ress` JS scanner
                        (NPE's lexer is derived from the same crate
                        upstream)
  * extractor.rs      — iframe_api → embed page fallback for player.js
                        URL, regex-driven hash extraction, clean-and-fetch
  * signature.rs      — 6 sig fn name regexes (front-most-recent),
                        deobf-function-body via lexer w/ regex fallback,
                        helper-object + global-string-array extraction,
                        signatureTimestamp, snippet assembler
  * nsig.rs           — 8 nsig fn name regexes (incl. array-indirection),
                        body via lexer w/ regex fallback, fixupFunction
                        early-return strip
  * player_manager.rs — orchestrator + sticky-error cache mirroring
                        YoutubeJavaScriptPlayerManager

PORT DEVIATIONS from NPE (each flagged in code):
  * dropped the 6th sig fn name regex (used Java backref \2; Rust's
    `regex` crate is backtracking-free, so we substitute a loose form
    that NPE itself half-broke per audit Track B §2.1)
  * dropped the Java atomic group `(?>...)` from helper-object regex —
    Rust's NFA is already linear-time
  * nsig fixup substitutes `(?:"undefined"|'undefined')` for the
    \1 backref; harmless loosening
  * sig and nsig assembled snippets prepend `var` — QuickJS rejects
    bare-assignment to undeclared identifiers; NPE relied on Rhino's
    non-strict mode

Tests:
  * 43 lib unit tests (up from 7 in Phase 1)
  * 7 Phase 2 offline integration tests against a hand-crafted
    minified synthetic player.js — exercises the full sig pipeline
    (build_deobfuscator → runtime::run) and nsig fixup_function
  * 7 Phase 1 live smoke tests still green

57/57 total green.
2026-05-24 16:53:19 -07:00

114 lines
4 KiB
Rust

// JS lexer helpers. Mirrors NPE's utils/jsextractor/JavaScriptExtractor.java
// + Lexer.java + EcmaScriptTokenStream.java.
//
// NPE's lexer is vendored from Rhino 1.7.14 to handle regex-vs-division
// disambiguation. The `ress` crate is the direct Rust analog — same shape,
// pure-rust scanner. We delegate to it.
//
// Public surface mirrors NPE's static `matchToClosingBrace(src, start)`:
// given a substring `start` (e.g. `"xyz=function"`), find its first
// occurrence in `src`, then walk forward through tokens balancing braces
// (skipping braces inside strings, regex literals, comments) until the
// matching `}` of the function body is consumed. Returns the slice from
// the first `{` of the body through that closing `}`, inclusive.
use ress::tokens::{Punct, Token};
use ress::Scanner;
use crate::youtube::js::DeobfError;
/// Returns the substring of `src` starting immediately after `start` and
/// ending at the matching `}` of the first function body that follows
/// (inclusive). Everything between the anchor and the first `{` (e.g.
/// `(a)` for `xyz=function`) is included.
///
/// Mirrors `JavaScriptExtractor.matchToClosingBrace(src, start)` from NPE.
pub fn match_to_closing_brace(src: &str, start: &str) -> Result<String, DeobfError> {
let prefix_idx = src
.find(start)
.ok_or_else(|| DeobfError::SigBodyParseFailed(format!("anchor not found: {start}")))?;
let scan_from = prefix_idx + start.len();
let scanner = Scanner::new(&src[scan_from..]);
let mut depth = 0i32;
let mut saw_open = false;
let mut last_brace: Option<usize> = None;
for item in scanner {
let item = item.map_err(|e| {
DeobfError::SigBodyParseFailed(format!("lexer error at byte {}: {e}", scan_from))
})?;
match item.token {
Token::Punct(Punct::OpenBrace) => {
saw_open = true;
depth += 1;
}
Token::Punct(Punct::CloseBrace) => {
depth -= 1;
if depth == 0 && saw_open {
last_brace = Some(item.span.end);
break;
}
}
_ => {}
}
}
let last = last_brace.ok_or_else(|| {
DeobfError::SigBodyParseFailed("no matching closing brace found".into())
})?;
Ok(src[scan_from..scan_from + last].to_string())
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn balances_simple_function() {
let src = "var a=1;xyz=function(b){return b;};var c=2;";
let body = match_to_closing_brace(src, "xyz=function").unwrap();
assert_eq!(body, "(b){return b;}");
}
#[test]
fn skips_braces_inside_strings() {
let src = r#"xyz=function(a){var x="}}";return a+x;}"#;
let body = match_to_closing_brace(src, "xyz=function").unwrap();
assert_eq!(body, r#"(a){var x="}}";return a+x;}"#);
}
#[test]
fn skips_braces_inside_regex() {
let src = r#"xyz=function(a){var re=/}{/;return a.replace(re,"");}"#;
let body = match_to_closing_brace(src, "xyz=function").unwrap();
assert!(body.starts_with("(a){"));
assert!(body.ends_with("}"));
}
#[test]
fn handles_nested_blocks() {
let src = r#"
xyz=function(a){
if (a.length > 3) {
a = a.split("");
for (var i=0; i<a.length; i++) { a[i] = a[i]; }
return a.join("");
}
return a;
};
"#;
let body = match_to_closing_brace(src, "xyz=function").unwrap();
let opens = body.matches('{').count();
let closes = body.matches('}').count();
assert_eq!(opens, closes, "balanced");
assert!(body.contains("(a)"));
assert!(body.ends_with("}"));
}
#[test]
fn anchor_not_found_returns_err() {
let err = match_to_closing_brace("var a=1;", "qqq=function").unwrap_err();
assert!(matches!(err, DeobfError::SigBodyParseFailed(_)));
}
}