Sulkta-Coop/strawcore
2
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

Drop dead Method variants, Downloader default fns, parsing/stream_helper unused, suppress_unused leftovers, stale comments

Browse source 
Second pass through the cruft inventory. All deletes — no behavior change.

Method enum + downloader trait:
  * Method::Head / Put / Delete dropped (no caller). Match arms in
    request.rs::as_str() and default_impl.rs::execute() collapse to
    just Get / Post.
  * Request::head builder dropped.
  * Downloader trait's get / get_localized / head / post default
    methods dropped. Every caller went through execute() directly
    anyway; the convenience wrappers carried 4 dead arms each.

Parsing module:
  * bootstrap_visitor_data — pub fn, no caller.
  * discover_web_client_version + CACHED_WEB_CLIENT_VERSION +
    reset_web_client_version_cache — entire sw.js live-version
    discovery pipeline, never wired up by any caller. The cache was
    never populated, so web_client_version() always returned the
    hardcoded constant. Collapsed to just returning the constant.
  * Drops once_cell::Lazy, parking_lot::RwLock around the version
    cache (consent flag still uses RwLock), Regex import, serde_json::Value
    import, downloader/exceptions/Request/InnertubeClientRequestInfo
    imports — all only kept alive by the deleted code.

stream_helper:
  * get_web_embedded_player_response — pub fn, no caller.

js/player_manager + extractor:
  * player_manager::player_hash — pub fn, no caller. Was only kept
    alive by its own definition.
  * extractor::extract_player_hash — pub fn, only called by the now-
    dead player_hash. Test removed alongside.

Stale comments:
  * itag.rs:1 header claimed 53 entries; ITAG_TABLE has 57 and the
    test at line 179 already asserts it.
  * js/mod.rs:12-13 claimed the submodules were 'crate-private
    plumbing' but they're declared pub mod. Tightened the comment to
    explain the integration-test dependency that keeps them public.

Net delete: ~170 LOC of dead surface across 9 files.
This commit is contained in:
Kayos 2026-05-26 22:33:00 -07:00
parent 59d2ee07be
commit 56afa423fb
9 changed files with 18 additions and 189 deletions
Download patch file Download diff file Expand all files Collapse all files

3
src/downloader/default_impl.rs
View file

@ -54,10 +54,7 @@ impl Downloader for ReqwestDownloader {
fn execute(&self, request: Request) -> Result<Response, NetworkError> {
let method = match request.method() {
Method::Get => reqwest::Method::GET,
Method::Head => reqwest::Method::HEAD,
Method::Post => reqwest::Method::POST,
Method::Put => reqwest::Method::PUT,
Method::Delete => reqwest::Method::DELETE,
};
let mut builder = self.client.request(method, request.url());

21
src/downloader/mod.rs
View file

@ -17,7 +17,6 @@ pub mod request;
pub mod response;
use crate::exceptions::NetworkError;
use crate::localization::Localization;
pub use default_impl::ReqwestDownloader;
pub use request::{Request, RequestBuilder};
@ -25,24 +24,4 @@ pub use response::Response;
pub trait Downloader: Send + Sync {
fn execute(&self, request: Request) -> Result<Response, NetworkError>;
fn get(&self, url: &str) -> Result<Response, NetworkError> {
self.execute(Request::get(url).build())
}
fn get_localized(
&self,
url: &str,
localization: Localization,
) -> Result<Response, NetworkError> {
self.execute(Request::get(url).localization(Some(localization)).build())
}
fn head(&self, url: &str) -> Result<Response, NetworkError> {
self.execute(Request::head(url).build())
}
fn post(&self, url: &str, body: Vec<u8>) -> Result<Response, NetworkError> {
self.execute(Request::post(url, body).build())
}
}

10
src/downloader/request.rs
View file

@ -13,20 +13,14 @@ pub type Headers = BTreeMap<String, Vec<String>>;
#[derive(Clone, Debug, Eq, PartialEq)]
pub enum Method {
Get,
Head,
Post,
Put,
Delete,
}
impl Method {
pub fn as_str(&self) -> &'static str {
match self {
Method::Get => "GET",
Method::Head => "HEAD",
Method::Post => "POST",
Method::Put => "PUT",
Method::Delete => "DELETE",
}
}
}
@ -46,10 +40,6 @@ impl Request {
RequestBuilder::new(Method::Get, url)
}
pub fn head(url: impl Into<String>) -> RequestBuilder {
RequestBuilder::new(Method::Head, url)
}
pub fn post(url: impl Into<String>, body: Vec<u8>) -> RequestBuilder {
RequestBuilder::new(Method::Post, url).body(Some(body))
}

3
src/youtube/itag.rs
View file

@ -1,6 +1,5 @@
// itag → MediaFormat table. Mirrors NPE ItagItem.java:28-101 — the
// hard-coded array of 53 entries (14 combined-AV + 10 audio + 33
// video-only).
// hard-coded array (currently 57 entries, see ITAG_TABLE).
//
// Codec column ("AV1", "VP9") is derived from response mimeType at extract
// time, NOT stored here — matches NPE's source comment ItagItem.java:26.

14
src/youtube/js/extractor.rs
View file

@ -126,15 +126,6 @@ fn download_javascript_code(downloader: &dyn Downloader, url: &str) -> Result<St
Ok(resp.response_body().to_string())
}
/// Extracts the 8-char player hash from a URL like
/// `https://www.youtube.com/s/player/<hash>/player_ias.vflset/.../base.js`.
/// Used for rotation detection.
pub fn extract_player_hash(url: &str) -> Option<String> {
static RE: Lazy<Regex> =
Lazy::new(|| Regex::new(r"/s/player/([A-Za-z0-9]{8})/").unwrap());
RE.captures(url).and_then(|c| c.get(1)).map(|m| m.as_str().to_string())
}
#[cfg(test)]
mod tests {
use super::*;
@ -181,9 +172,4 @@ mod tests {
assert_eq!(out, "https://www.youtube.com/s/player/x/base.js");
}
#[test]
fn player_hash_extracted_from_url() {
let url = "https://www.youtube.com/s/player/c2f7551f/player_ias.vflset/en_GB/base.js";
assert_eq!(extract_player_hash(url).as_deref(), Some("c2f7551f"));
}
}

10
src/youtube/js/mod.rs
View file

@ -1,7 +1,7 @@
// JS deobfuscator subsystem — mirrors NPE's player.js / sig / nsig pipeline.
//
// Public surface is the `player_manager` module (mirrors NPE's
// YoutubeJavaScriptPlayerManager — the sole public class in the subsystem):
// Production callers go through `player_manager` (mirrors NPE's
// YoutubeJavaScriptPlayerManager):
// * signature_timestamp(video_id)
// * deobfuscate_signature(video_id, obfuscated)
// * url_with_throttling_parameter_deobfuscated(video_id, url)
@ -9,8 +9,10 @@
// * clear_all_caches()
// * clear_throttling_parameters_cache()
//
// Everything else (runtime / lexer / extractor / signature / nsig) is
// crate-private plumbing.
// The other submodules (runtime / lexer / extractor / signature / nsig) are
// kept `pub` because `tests/js_phase2_offline.rs` exercises them directly
// from outside the crate. If the integration test gets folded into the
// inline `#[cfg(test)]` blocks, these can drop to `pub(crate)`.
pub mod extractor;
pub mod lexer;

8
src/youtube/js/player_manager.rs
View file

@ -194,14 +194,6 @@ impl PlayerManager {
self.inner.lock().player_url.clone()
}
pub fn player_hash(&self) -> Option<String> {
self.inner
.lock()
.player_url
.as_deref()
.and_then(extractor::extract_player_hash)
}
fn ensure_player_code(state: &mut ManagerState, video_id: &str) -> Result<(), DeobfError> {
if state.player_code.is_some() {
return Ok(());

117
src/youtube/parsing.rs
View file

@ -1,31 +1,18 @@
// YoutubeParsingHelper-shaped helpers — mirrors NPE
// services/youtube/YoutubeParsingHelper.java.
//
// Currently implements:
// YoutubeParsingHelper-shaped helpers:
// * consent toggle + cookie generator (set_consent_accepted, consent_cookie)
// * client-version cache + sw.js fetch fallback (get_web_client_version)
// * visitor-data bootstrap via /youtubei/v1/visitor_id
// * client/origin/referer header builder
// * WEB client-version constant (web_client_version)
// * client/origin/referer header builder (youtube_post_headers / mobile_post_headers)
// * mobile user-agent string builders
//
// PoToken integration lands in Phase 5. po_token / DroidGuard / BotGuard
// machinery is host-provided (PoTokenProvider trait).
// PoToken integration lives in youtube::potoken — host-provided via the
// PoTokenProvider trait.
use once_cell::sync::Lazy;
use parking_lot::RwLock;
use regex::Regex;
use serde_json::Value;
use crate::downloader::request::Request;
use crate::exceptions::ParsingError;
use crate::localization::{ContentCountry, Localization};
use crate::newpipe::NewPipe;
use crate::youtube::client_request::{
build_envelope, InnertubeClientRequestInfo,
};
use crate::youtube::constants::*;
static CONSENT_ACCEPTED: Lazy<RwLock<bool>> = Lazy::new(|| RwLock::new(false));
static CACHED_WEB_CLIENT_VERSION: Lazy<RwLock<Option<String>>> = Lazy::new(|| RwLock::new(None));
pub fn set_consent_accepted(accepted: bool) {
*CONSENT_ACCEPTED.write() = accepted;
@ -45,53 +32,14 @@ pub fn consent_cookie() -> &'static str {
}
}
/// Returns the cached WEB client version. Falls back to the hardcoded
/// constant if no live extraction has run.
/// Returns the WEB client version. The sw.js-based live-version discovery
/// (NPE pattern) was ported but never wired up by any caller, so we just
/// ship the hardcoded constant. If/when live version probing matters
/// again, restore `discover_web_client_version` from git history.
pub fn web_client_version() -> String {
if let Some(v) = CACHED_WEB_CLIENT_VERSION.read().as_ref() {
return v.clone();
}
WEB_HARDCODED_CLIENT_VERSION.to_string()
}
pub fn reset_web_client_version_cache() {
*CACHED_WEB_CLIENT_VERSION.write() = None;
}
static SW_JS_VERSION_RE: Lazy<Regex> = Lazy::new(|| {
Regex::new(r#"INNERTUBE_CONTEXT_CLIENT_VERSION":\s*"([^"]+)""#).unwrap()
});
/// Fetches sw.js + extracts the live WEB client version. Caches the
/// result. Returns the cached value if already known.
pub fn discover_web_client_version() -> Result<String, ParsingError> {
if let Some(v) = CACHED_WEB_CLIENT_VERSION.read().as_ref() {
return Ok(v.clone());
}
let downloader = NewPipe::downloader()
.ok_or_else(|| ParsingError::Invalid("downloader not initialized".into()))?;
let req = Request::get("https://www.youtube.com/sw.js")
.add_header("Origin", "https://www.youtube.com")
.add_header("Referer", "https://www.youtube.com")
.build();
let resp = downloader
.execute(req)
.map_err(|e| ParsingError::Invalid(format!("sw.js fetch: {e}")))?;
if resp.response_code() != 200 {
return Err(ParsingError::Invalid(format!(
"sw.js HTTP {}",
resp.response_code()
)));
}
let version = SW_JS_VERSION_RE
.captures(resp.response_body())
.and_then(|c| c.get(1))
.map(|m| m.as_str().to_string())
.ok_or_else(|| ParsingError::RegexMiss("INNERTUBE_CONTEXT_CLIENT_VERSION".into()))?;
*CACHED_WEB_CLIENT_VERSION.write() = Some(version.clone());
Ok(version)
}
/// Headers for a WEB-flavor POST (JSON content-type, client headers,
/// origin/referer, consent cookie).
pub fn youtube_post_headers() -> Vec<(String, String)> {
@ -129,48 +77,6 @@ pub fn ios_user_agent(country: &ContentCountry) -> String {
)
}
/// Bootstraps a visitor_data token via `/youtubei/v1/visitor_id`. Returns
/// the value of `responseContext.visitorData` from the response.
pub fn bootstrap_visitor_data(
info: &InnertubeClientRequestInfo,
localization: &Localization,
content_country: &ContentCountry,
use_gapis_endpoint: bool,
) -> Result<String, ParsingError> {
let downloader = NewPipe::downloader()
.ok_or_else(|| ParsingError::Invalid("downloader not initialized".into()))?;
let envelope = build_envelope(info, localization, content_country, None);
let body = serde_json::to_vec(&envelope)?;
let base = if use_gapis_endpoint {
YOUTUBEI_V1_GAPIS_URL
} else {
YOUTUBEI_V1_URL
};
let url = format!("{base}visitor_id{DISABLE_PRETTY_PRINT_PARAM}");
let mut req_builder = Request::post(&url, body);
for (k, v) in youtube_post_headers() {
req_builder = req_builder.add_header(&k, &v);
}
let resp = downloader
.execute(req_builder.build())
.map_err(|e| ParsingError::Invalid(format!("visitor_id POST: {e}")))?;
if resp.response_code() != 200 {
return Err(ParsingError::Invalid(format!(
"visitor_id HTTP {}",
resp.response_code()
)));
}
let parsed: Value = serde_json::from_str(resp.response_body())?;
parsed
.get("responseContext")
.and_then(|rc| rc.get("visitorData"))
.and_then(|v| v.as_str())
.map(|s| s.to_string())
.ok_or_else(|| ParsingError::MissingField("responseContext.visitorData".into()))
}
#[cfg(test)]
mod tests {
use super::*;
@ -185,8 +91,7 @@ mod tests {
}
#[test]
fn web_client_version_falls_back_to_hardcoded() {
reset_web_client_version_cache();
fn web_client_version_returns_hardcoded() {
assert_eq!(web_client_version(), WEB_HARDCODED_CLIENT_VERSION);
}

21
src/youtube/stream_helper.rs
View file

@ -91,27 +91,6 @@ pub fn get_web_metadata_player_response(
post_youtube(&url, &Value::Object(body), youtube_post_headers())
}
/// WEB_EMBEDDED_PLAYER /player call. Carries embedUrl + signatureTimestamp.
pub fn get_web_embedded_player_response(
video_id: &str,
localization: &Localization,
content_country: &ContentCountry,
signature_timestamp: i32,
po_token: Option<&str>,
) -> Result<Value, ExtractionError> {
let info = InnertubeClientRequestInfo::of_web_embedded_player_client();
let embed_url = format!("https://www.youtube.com/embed/{video_id}");
let env = build_envelope(&info, localization, content_country, Some(&embed_url));
let mut body = envelope_to_body(env);
add_player_body_fields(&mut body, video_id, &generate_content_playback_nonce());
add_playback_context(&mut body, signature_timestamp, &embed_url);
if let Some(token) = po_token {
add_service_integrity_dimensions(&mut body, token);
}
let url = format!("{YOUTUBEI_V1_URL}player{DISABLE_PRETTY_PRINT_PARAM}");
post_youtube(&url, &Value::Object(body), youtube_post_headers())
}
/// ANDROID full /player call. Hits the gapis endpoint with the mobile
/// header set. Caller must supply (cpn, po_token) — they are paired with
/// the URLs the response will return; mixing them with iOS values returns