Commit graph

4 commits

Author SHA1 Message Date
Sulkta
5e0ec08928 perf(extract): borrow instead of clone + parallelize channel browse
Four allocation/latency wins in the hot extraction paths. Behavior
preserved — all 120 in-crate tests green, clippy clean.

- stream_extractor: borrow streamingData out of the owned android + ios
  player responses instead of deep-cloning the largest subtree of each
  response. A shared Value::Null static backs the absent case.
- stream_extractor: merge_formats returns borrowed &Value format objects
  rather than cloning each one (~20-40 per video) — they are only read
  (.get()) downstream.
- channel: fetch the Home and Videos tabs concurrently via thread::scope
  so a channel open costs one round-trip of latency, not two. Home stays
  mandatory; the Videos tab stays best-effort (now also resilient to a
  panicked worker thread).
- downloader: build the response body with String::from_utf8 (in-place on
  the valid-UTF-8 common case) instead of from_utf8_lossy, which always
  copies; the lossy fallback for invalid bytes is preserved.
2026-06-21 05:40:11 -07:00
Sulkta
f917e4adbd Drop dead Method variants, Downloader default fns, parsing/stream_helper unused, suppress_unused leftovers, stale comments
Second pass through the cruft inventory. All deletes — no behavior change.

Method enum + downloader trait:
  * Method::Head / Put / Delete dropped (no caller). Match arms in
    request.rs::as_str() and default_impl.rs::execute() collapse to
    just Get / Post.
  * Request::head builder dropped.
  * Downloader trait's get / get_localized / head / post default
    methods dropped. Every caller went through execute() directly
    anyway; the convenience wrappers carried 4 dead arms each.

Parsing module:
  * bootstrap_visitor_data — pub fn, no caller.
  * discover_web_client_version + CACHED_WEB_CLIENT_VERSION +
    reset_web_client_version_cache — entire sw.js live-version
    discovery pipeline, never wired up by any caller. The cache was
    never populated, so web_client_version() always returned the
    hardcoded constant. Collapsed to just returning the constant.
  * Drops once_cell::Lazy, parking_lot::RwLock around the version
    cache (consent flag still uses RwLock), Regex import, serde_json::Value
    import, downloader/exceptions/Request/InnertubeClientRequestInfo
    imports — all only kept alive by the deleted code.

stream_helper:
  * get_web_embedded_player_response — pub fn, no caller.

js/player_manager + extractor:
  * player_manager::player_hash — pub fn, no caller. Was only kept
    alive by its own definition.
  * extractor::extract_player_hash — pub fn, only called by the now-
    dead player_hash. Test removed alongside.

Stale comments:
  * itag.rs:1 header claimed 53 entries; ITAG_TABLE has 57 and the
    test at line 179 already asserts it.
  * js/mod.rs:12-13 claimed the submodules were 'crate-private
    plumbing' but they're declared pub mod. Tightened the comment to
    explain the integration-test dependency that keeps them public.

Net delete: ~170 LOC of dead surface across 9 files.
2026-05-26 22:33:00 -07:00
Sulkta
0c75d64707 Cap attribution_link recursion + Downloader response body size
Two adversarial bugs surfaced by the round-2 audit on this crate.

extract_video_id recursion (linkhandler/stream.rs)
  /attribution_link?u=<inner> recursed on the inner URL with no depth
  guard. The comment claimed 'only one level deep' but the call was
  plain recursion — a pasted URL whose u= param decodes to another
  /attribution_link would recurse until the JVM stack blew. Wrap the
  recursion in extract_video_id_inner with an explicit depth counter
  capped at MAX_ATTRIBUTION_DEPTH = 1.

ReqwestDownloader body cap (downloader/default_impl.rs)
  resp.text() read the entire response body into a String with no
  upper bound. Player.js is ~1.5 MB, watch HTML ~3 MB, channel
  responses well under 1 MB. A hostile redirect target (or compromised
  host) could blast multi-GB and OOM-kill the Android process — there
  is no headroom on a 1 GB JVM heap ceiling.

  Cap at 32 MB. Two-stage check: bail fast on a known Content-Length
  that exceeds the cap, and use Read::take(MAX+1) on the stream so we
  detect overrun rather than silently truncate. Switched the final
  decode to from_utf8_lossy so a single mojibake byte doesn't drop the
  whole response (same fix shape as the wrapper's read_capped_body).
2026-05-26 22:02:40 -07:00
Sulkta
2a2367a3d0 Phase 1 — Foundation
Mirror NPE's dependency-free spine in Rust:

* exceptions   — NetworkError + ParsingError + ContentUnavailable
                 + ExtractionError tree, with reqwest/serde_json conversions
* localization — Localization + ContentCountry, default (en, GB)
* downloader/  — Downloader trait, Request builder, Response,
                 reqwest blocking default impl
* page         — continuation-token carrier
* image        — Image + ImageSet + ResolutionLevel
                 (HEIGHT_UNKNOWN/WIDTH_UNKNOWN = -1)
* metainfo     — title/content/url/url_text grab-bag
* service      — StreamingService trait + LinkType + ServiceInfo
* newpipe      — process-global Downloader / Localization /
                 ContentCountry singleton

Foundational invariants nailed down (per SPEC §3):
* HTTP non-2xx returns Ok(Response); only 429 throws NetworkError::Recaptcha
* Response header keys lowercase-normalized
* Request.add_header PARITY with NPE bug (silent overwrite);
  append_header is our clean addition
* default Localization is en-GB
* No cookie jar in the default downloader

Tests: 7 unit + 7 live smoke against httpbin.org (gated on
'online-tests' feature). All green.
2026-05-24 16:32:36 -07:00