Commit graph

61 commits

Author SHA1 Message Date
1211f601df deobfuscate: silence dead_code on Deobfuscator::has_sig/has_nsig
Public API methods exposed for downstream consumers (e.g. straw can
call deobf.has_sig() to skip cipher streams without observing an Err).
The internal short-circuit uses the struct field directly so the
methods register as unused at compile time.
2026-05-24 12:21:55 -07:00
8126cc0da5 audit-fix sprint: all 13 findings (CRIT/HIGH/MED/LOW)
CRIT-1: ExtractionError::Deobfuscation is now switchable.
        Deobfuscator gains has_sig()/has_nsig() — deobfuscate_sig/_nsig
        short-circuit with a recognisable error class so cipher streams
        on the wrong client fall through to the next client in the chain
        instead of killing the whole call.

CRIT-2: Soft-failed DeobfData now caches with a 1-hour retry instead of
        living for 24h. Re-extraction kicks in automatically once YT
        rotates back to a player.js shape we recognise — no more
        wall-clock-day-of-poisoned-cache.

HIGH-1: Reporter now emits a Level::WRN `extract_deobf_soft_fail` report
        on partial extraction. straw / torttube get an artefact when
        sig/nsig regex starts missing.

HIGH-2: player_client_order branches on opts.auth. With botguard
        + authed-cookie users, Desktop is now position 2 (where their
        cookie maps to an OAuth session) instead of position 4.

HIGH-3: Android dropped from the default order. needs_po_token doesn't
        flag Android, so requests were firing unsigned and tripping
        YT's bot-check rejection — which is also not switchable.
        Re-add when a real po_token strategy lands.

MED-1: Comment in needs_deobf softened — the iOS/Android-no-deobf
        property is a current YT behaviour, not a permanent protocol.

MED-2: Cargo.toml workspace pin bumped 0.11.4 → 0.11.5 so it matches
        the package version (avoids future 0.12.x bump surprises).

MED-3: Smoke test fixture uses an isolated per-process scratch dir
        instead of the repo root, avoiding cache-race with
        tests/youtube.rs (which uses CARGO_MANIFEST_DIR and could
        wipe OAuth tokens).

LOW-1: Misleading "dead-code fallback" comment in extract_fns replaced
        with the actual behaviour description.

LOW-2: get_deobf_data uses read-then-write — concurrent player calls
        on warm cache no longer serialise on the write lock.

LOW-3: Smoke test catches IpBan via exact UnavailabilityReason match
        instead of substring "Sign in/IpBan/bot" — a real regression
        won't silently pass anymore.

LOW-4: TV smoke test now asserts !audio_streams.is_empty() too,
        matching iOS / default-order tests.

LOW-5: needs_deobf comment notes YT's historical n= experiments on
        Android — sets expectation for future review passes.
2026-05-24 12:20:14 -07:00
bda0fea193 deobfuscate: soft-fail sig_fn/nsig_fn extraction
When YouTube rotates player.js to a shape our six sig/nsig regex
patterns don't recognise (eg. c2f7551f, May 2026), the whole player
path used to die at extract_fns even for clients that don't need the
sig fn at all (iOS, Android, Tv all get pre-signed stream URLs).

Now sig_fn / nsig_fn extraction is best-effort. Only the signature
timestamp is required — every `needs_deobf` client needs sts in
the request payload, but the actual deobfuscation functions are only
consumed by map_url when a stream URL carries `&s=` or `&n=`.
On failure we log a warning and store an empty string; Deobfuscator
then skips the JS eval, and any deobfuscate_sig/deobfuscate_nsig
call will fail loudly with "sig fn unavailable" instead of crashing
the player.

Keeps the Tv fallback alive even when sig deobf regex breaks.
2026-05-24 11:53:12 -07:00
ThetaDev
f0477ea3a9
test: add sig deobf test case 2025-04-23 21:29:51 +02:00
ThetaDev
d675987654
fix: deobfuscator: handle 1-char long global variables, find nsig fn (player 6450230e) 2025-04-23 17:22:22 +02:00
ThetaDev
ea80717f69
fix: handle music playlist/album not found 2025-03-26 02:35:03 +01:00
ThetaDev
939a7aea61
fix: deobfuscator: handle global functions as well 2025-03-26 02:12:18 +01:00
ThetaDev
189ba81a42
fix: extractor: small simplification 2025-03-26 01:38:12 +01:00
ThetaDev
ac44e95a88
fix: extractor: global variable extraction fixed 2025-03-26 01:20:35 +01:00
ThetaDev
4ce6746be5
fix: extract deobf data with global strings variable 2025-03-24 01:12:01 +01:00
ThetaDev
1d755b76bf
feat: add RustyPipe::version_botguard fn, detect rustypipe-botguard in current dir, add botguard version to report 2025-02-09 01:52:09 +01:00
ThetaDev
50ab1f7a5d
fix: retry updating deobf data after a RustyPipe update 2025-02-05 11:55:05 +01:00
ThetaDev
eda16e3787
fix: extracting nsig fn when outside variable starts with $ 2025-02-05 10:15:52 +01:00
ThetaDev
75c3746890
fix: switch to rquickjs crate for deobfuscator 2025-01-13 03:30:07 +01:00
ThetaDev
162959ca45
fix: remove leading zero-width-space from comments, ensure space after links 2024-12-18 19:31:24 +01:00
ThetaDev
8cadbc1a4c
fix: deobf function extraction, allow $ in variable names 2024-12-16 01:31:04 +01:00
ThetaDev
80147413ee
fix: nsig fn extra variable extraction 2024-12-13 22:36:01 +01:00
ThetaDev
f5437aa127
fix: deobfuscation function extraction 2024-12-13 03:55:06 +01:00
ThetaDev
d36ba595da
fix: extraction error message 2024-08-08 15:10:55 +02:00
ThetaDev
c6bd03fb70
fix: add var to deobf fn assignment 2024-08-06 14:01:38 +02:00
ThetaDev
3c83e11e75
fix: nsig fn extraction 2024-07-31 21:46:32 +02:00
ThetaDev
8152ce6b08
fix: improve deobfuscator (support multiple nsig name matches, error if mapping all streams fails)
Since YouTube keeps changing the nsig function signature and a generic regex may match at multiple places, I changed the extraction logic to search for multiple matches if necessary and test the extracted deobfuscation functions.

I also found out that if the deobfuscation fails for all streams, fetching the player still returns a successful result with no streams, suggesting that the video is not available. So I changed the mapper to throw an ExtractionError if no streams are mapped successfully.
2024-07-29 14:45:52 +02:00
ThetaDev
fb7af3b966
fix: make nsig_fn regex more generic 2024-07-27 03:23:09 +02:00
ThetaDev
dd0565ba98
fix!: extracting nsig function, remove field throttled from Video/Audio stream model 2024-07-15 20:41:20 +02:00
ThetaDev
eecabffd18
tests: remove tokio_test::block_on 2024-03-16 19:21:30 +01:00
ThetaDev
339231924b
fix: update dictionary, fix parsing playlist dates in am and no 2024-02-13 18:38:58 +01:00
ThetaDev
1d1dcd622f feat: add tracing 2023-09-22 03:22:38 +02:00
ThetaDev
ac25490435 chore: fix clippy lints 2023-08-30 22:05:18 +02:00
ThetaDev
22e298ff98 fix: a/b test 8: parsing view count for tracks 2023-08-23 11:28:38 +02:00
ThetaDev
57628d1392 fix: adjust deobf helper object name regex 2023-08-16 01:52:39 +02:00
ThetaDev
5736d53c99 feat: add error reporting for deobf data extraction 2023-08-03 17:42:31 +02:00
ThetaDev
1bab2ef301 fix: deobfuscator not extracting array_str 2023-06-28 17:57:18 +02:00
ThetaDev
cbeb14f3fd fix: add pedantic lints 2023-05-13 02:40:26 +02:00
ThetaDev
6ab7b2415a refactor: make DeobfError private 2023-05-08 01:51:27 +02:00
ThetaDev
7bcff13eb8 fix: use fixed testfiles location 2023-03-25 23:18:49 +01:00
ThetaDev
ee5e82f6bb fix: instantiate deobfuscator in mapresponse fn
Reason: quickjs context is not Send
2023-02-08 12:29:08 +01:00
ThetaDev
79446bed1c fix: keep JS context when deobfuscating 2023-02-04 16:15:35 +01:00
ThetaDev
331a13568a fix: v1.67 clippy lints (inline format strings) 2023-01-27 21:19:08 +01:00
ThetaDev
06aa677ef7 tests: use tokio_test::block_on() instead of test macro 2023-01-27 20:47:37 +01:00
ThetaDev
f94d8db4d0 feat: add logging for all operations
fix: music_artist: fetch visitor data only once
2023-01-27 19:49:16 +01:00
ThetaDev
92a358a079 refactor: use fancy-regex only for backtracking 2023-01-23 15:33:05 +01:00
ThetaDev
64d009615e fix: use path_macro in tests/codegen for cross-platform paths 2022-11-30 10:04:16 +01:00
ThetaDev
b5f6b7a174 refactor: convert _or_bail macros to ok_or functions 2022-10-18 19:09:16 +02:00
ThetaDev
4dfdb47cae fix: don't use a specific letter for the nsig function name regex 2022-10-10 22:25:20 +02:00
ThetaDev
a3e3269fb3 feat: add custom error types, remove anyhow 2022-10-08 14:36:37 +02:00
ThetaDev
fe5468313a fix: remove dead code 2022-09-27 21:43:12 +02:00
ThetaDev
05f609e247 fix: clippy warnings 2022-09-15 16:05:06 +02:00
ThetaDev
b52fd7349b refactored client API (query builder)
use VecLogError for player streams
2022-09-14 23:55:44 +02:00
ThetaDev
dda2211e04 refactored cache 2022-09-14 00:04:51 +02:00
ThetaDev
346406c1c8 feat: add timeago parser, playlist_cont 2022-09-01 20:16:00 +02:00