strawcore

Sulkta-OSS/strawcore

Fork 0

Commit graph

Author	SHA1	Message	Date
Cobb	2452f1785d	npe-sync: visitorData + visionOS(101) + xtags audio typing (best-effort, additive) Some checks failed gitleaks / scan (push) Failing after 1s Details Brings the port up to current NewPipe Extractor coverage: - visitorData: fetched + attached to the anonymous reel/web/ios/visionos InnerTube contexts (upstream "always pass a valid visitorData"). Best-effort — a failure returns None and extraction proceeds exactly as before. Cached process-wide with a 15-min TTL, and invalidated when the primary android response is rejected (decoy/playability) so a bad session self-heals. - visionOS client (101): a 4th /player fetch, ON by default, recovering >360p streams for SABR-only / made-for-kids videos where android/ios cap at 360p. Formats merged android-first (android wins itag ties); a failure changes nothing. HLS + DASH manifest branches now skip empty urls (an empty visionOS url no longer shadows a real ios/android one). - xtags: derive audio track type (original/dubbed/descriptive/secondary) from the format xtags protobuf, falling back to the audioIsDefault heuristic when absent. Hand-rolled protobuf walker, fuzzed (4M inputs, no panic); 32-bit length guarded via try_from. FFI surface unchanged (AudioStream gains an additive track_type the wrapper may later surface). 155 lib + 7 integration tests, clippy -D clean.	2026-07-29 07:07:42 -07:00
Sulkta	589e7446c3	Phase 4 (partial) — stream value types + InnerTube /player helpers Lands the data shapes + the HTTP layer for stream extraction. The extractor orchestrator + DASH manifest creator are deferred to the next session — the parsing logic is dense enough to want a focused pass. src/stream/ * mod.rs — StreamInfo + StreamInfoItem (full + 'card' shapes) mirroring NPE StreamInfo.java + StreamInfoItem.java * delivery.rs — DeliveryMethod (Progressive/Dash/Hls/Torrent) * audio.rs — AudioStream (itag, format, url, bitrate, codec, audio_track_id, content_length, etc.) * video.rs — VideoStream (itag, format, url, resolution, fps, bandwidth, codec, video_only flag) * subtitles.rs — SubtitlesStream (url, lang, auto_generated, mime) src/youtube/stream_helper.rs * generate_content_playback_nonce() — 16-char LCG-shuffled cpn * get_web_metadata_player_response (microformat + thumbnails only) * get_web_embedded_player_response (embed-url + signatureTimestamp) * get_android_player_response (full Android /player + poToken) * get_android_reel_player_response (no-poToken fallback) * get_ios_player_response (iOS — flagged with 917 KiB cap warning in the doc comment) Per-helper headers + URL shapes match audit Track C §2.7 verbatim: Android/iOS hit gapis endpoint with mobile UA; WEB family hits www.youtube.com with the WEB headers. Tests: 64 lib unit pass (up from 62 in Phase 3). Next session: full stream_extractor.rs orchestrator + dash_manifest/ creator + Phase 4 done-when smoke (extract NCS Spektrem).	2026-05-24 17:01:03 -07:00

Author

SHA1

Message

Date

Cobb

2452f1785d

npe-sync: visitorData + visionOS(101) + xtags audio typing (best-effort, additive)

gitleaks / scan (push) Failing after 1s

Details

Brings the port up to current NewPipe Extractor coverage:
- visitorData: fetched + attached to the anonymous reel/web/ios/visionos
  InnerTube contexts (upstream "always pass a valid visitorData"). Best-effort
  — a failure returns None and extraction proceeds exactly as before. Cached
  process-wide with a 15-min TTL, and invalidated when the primary android
  response is rejected (decoy/playability) so a bad session self-heals.
- visionOS client (101): a 4th /player fetch, ON by default, recovering >360p
  streams for SABR-only / made-for-kids videos where android/ios cap at 360p.
  Formats merged android-first (android wins itag ties); a failure changes
  nothing. HLS + DASH manifest branches now skip empty urls (an empty visionOS
  url no longer shadows a real ios/android one).
- xtags: derive audio track type (original/dubbed/descriptive/secondary) from
  the format xtags protobuf, falling back to the audioIsDefault heuristic when
  absent. Hand-rolled protobuf walker, fuzzed (4M inputs, no panic); 32-bit
  length guarded via try_from.

FFI surface unchanged (AudioStream gains an additive track_type the wrapper
may later surface). 155 lib + 7 integration tests, clippy -D clean.

2026-07-29 07:07:42 -07:00

Sulkta

589e7446c3

Phase 4 (partial) — stream value types + InnerTube /player helpers

Lands the data shapes + the HTTP layer for stream extraction. The
extractor orchestrator + DASH manifest creator are deferred to the
next session — the parsing logic is dense enough to want a focused
pass.

src/stream/
  * mod.rs       — StreamInfo + StreamInfoItem (full + 'card' shapes)
                   mirroring NPE StreamInfo.java + StreamInfoItem.java
  * delivery.rs  — DeliveryMethod (Progressive/Dash/Hls/Torrent)
  * audio.rs     — AudioStream (itag, format, url, bitrate, codec,
                   audio_track_id, content_length, etc.)
  * video.rs     — VideoStream (itag, format, url, resolution, fps,
                   bandwidth, codec, video_only flag)
  * subtitles.rs — SubtitlesStream (url, lang, auto_generated, mime)

src/youtube/stream_helper.rs
  * generate_content_playback_nonce() — 16-char LCG-shuffled cpn
  * get_web_metadata_player_response       (microformat + thumbnails only)
  * get_web_embedded_player_response       (embed-url + signatureTimestamp)
  * get_android_player_response            (full Android /player + poToken)
  * get_android_reel_player_response       (no-poToken fallback)
  * get_ios_player_response                (iOS — flagged with 917 KiB cap
                                            warning in the doc comment)

Per-helper headers + URL shapes match audit Track C §2.7 verbatim:
Android/iOS hit gapis endpoint with mobile UA; WEB family hits
www.youtube.com with the WEB headers.

Tests: 64 lib unit pass (up from 62 in Phase 3).

Next session: full stream_extractor.rs orchestrator + dash_manifest/
creator + Phase 4 done-when smoke (extract NCS Spektrem).

2026-05-24 17:01:03 -07:00

2 commits