Commit graph

9 commits

Author SHA1 Message Date
e4a8751942 reliability + security fix sprint (vc=91)
Straw full-audit fix batch (2026-07-04). Confirmed HIGH/MED findings:

- Updater self-brick: drop the fdroid.sulkta.com leaf+E7 SPKI pins (LE
  rotates the leaf every ~90d + intermediates from a pool → a routine
  renewal was guaranteed to miss both pins and silently kill the only
  in-app update path). System-CA validation + the install-time APK
  signature gate remain. Also distinguish "index unreachable" from "up to
  date", add a 30s callTimeout + bounded index read, guard the API-26
  NotificationChannel, and setOnlyAlertOnce.
- Request POST_NOTIFICATIONS at runtime so A13+ update/media notifications
  actually post (was declared but never requested → silent no-op).
- isDebuggable=false on the shipped debug variant (closes ADB run-as dump
  of watch/search history + subs; no package-id cutover).
- Crash fixes: distinctBy(url) on moreFromChannel + related (duplicate
  LazyColumn key); back-handling driven by live nav depth so it survives
  moveTaskToBack on A12+; PlaylistsStore per-store caps + off-main
  hydration (hostile import ANR); SponsorBlock staleness fence via
  NowPlaying.refreshIfCurrent.
- CacheCap.nearest() snaps up to the nearest finite cap (defaults no longer
  resolve to Unlimited/100k on fresh installs).
- RYD/SB FFI shims wrap the uniffi call (swallow a core panic per contract).
- Rust wrapper: release panic="unwind" (was "abort", which defeated UniFFI
  catch_unwind → any core panic = whole-app SIGABRT) + a 60s wall-clock
  timeout on every blocking extractor call (unblocks the caller, bounds
  thread pileup). Pairs with the strawcore-core reliability fix.
2026-07-04 07:09:54 -07:00
Sulkta
4affa1e739 vc=84 — rename launcher to 'Straw' + move stream picker into Rust
Two changes:

1. Launcher name is now just 'Straw', not 'Straw debug' — past the
   debug-branding phase. Kept the .debug applicationId suffix (package
   stays com.sulkta.straw.debug) on purpose so fdroid updates install in
   place and the in-app auto-updater keeps working; dropping the suffix
   would change the package id and force a reinstall that wipes everyone's
   subs/history. That's a separate, deliberate release-track cutover.

2. Stream-selection logic moved out of Kotlin (resolveStreamPlayback) into
   the Rust strawcore wrapper as resolve_playback(StreamInfo, max_height)
   -> ResolvedStreams. The app keeps a thin shim that supplies the
   resolution cap (Settings.maxResolution) and attaches SponsorBlock
   segments. Byte-for-byte behavior parity with the old Kotlin picker:
   highest-bitrate stream at/under the cap, lowest-height fallback when
   nothing fits, first-element-wins on ties (matching Kotlin
   maxByOrNull/minByOrNull, not Rust's last-on-ties max_by_key), and
   isNotBlank() handling for the DASH/HLS URLs. First step of moving all
   backend logic to Rust; the picking lives at the FFI boundary because it
   depends on an app setting, keeping strawcore-core a pure extractor.

Wrapper cargo check + clippy clean (no new warnings); FFI surface adds
ResolvedStreams + resolvePlayback, bindings regen at build.
2026-06-21 11:41:50 -07:00
Sulkta
a636977512 vc=82 — subscription-feed enrichment via lightweight stream_metadata
enrich_feed_item now calls the new strawcore stream_metadata() path (Android
/player + videoDetails read only) instead of the full stream_info. The full
path ran the JS sig/nsig deobf, an extra WEB /player metadata round-trip, the
iOS client, and stream/manifest/caption extraction — then kept only view_count
+ duration_seconds. Those two come from the same videoDetails the lightweight
path reads (populate_microformat never touches them), so the values are
identical; the feed just stops paying for the discarded work — ~one heavy
round-trip dropped per enriched item per refresh.

FFI surface (enrichFeedItem -> EnrichedFeedMetadata) unchanged. Needs
strawcore 30f24d2 (pushed first; CI clones strawcore main).
2026-06-21 06:56:06 -07:00
Sulkta
738401b0ed vc=40: loop round 2/5 — round-1 misses + new HIGHs from round-5 audits
Three parallel Opus round-5 audits ran on vc=39. The big finds were
regressions on vc=39's own fixes — the retry-init wiring was incomplete,
the URL-fence pattern missed CancellationException swallowing, and the
SettingsStore idempotency branch was dead code. Real round-5 work.

HIGH
  R5-1  Rust `ensure_initialized` was only called from `init_logging`
        — extractor entry points never invoked it, so the 5s-backoff
        retry from vc=39 was unreachable. A cold-boot init failure
        still bricked extraction for the whole process. Now every
        `search()` / `stream_info()` / `channel_info()` calls
        ensure_initialized at entry; cheap when INITIALIZED is true.
  R5-2  runtime.rs in-progress race: prior shape stamped
        LAST_ATTEMPT_MS at the *start* of an attempt, then let
        concurrent callers short-circuit via the backoff check
        and proceed to call into the extractor before init
        completed. New: lock FIRST (mutex IS the in-progress
        queue), re-check INITIALIZED under lock, only THEN check
        backoff (and only stamp it on failure).
  R5-3  `runCatching` in VideoDetailViewModel + SubscriptionFeed
        swallows CancellationException — when a job was cancelled
        mid-suspend inside channelInfo/RYD/SB, the inner cancellation
        was eaten, the lambda returned its default, the job carried
        past the runCatching to its terminal write, and the URL fence
        let it through because same-URL races can't be distinguished
        by string equality. New util.runCatchingCancellable
        re-throws CancellationException; all coroutine-body
        runCatching sites in the affected VMs migrated.
  R5-4  SearchViewModel.submit post-network fence only guarded
        `_ui.update`. SearchCache.record + pool rebuild proceeded
        for a cancelled query → ghost cache entries for queries
        the user abandoned mid-stroke. Now: re-check the query
        AFTER the IO suspend and before the cache write.
  R5-5  ChannelViewModel.load / VideoDetailViewModel.load now gate
        the extractor entry on isAllowedYtUrl(channelUrl/streamUrl).
        Prior shape only gated recordWatch persistence — extractor
        invocation for poisoned uploaderUrl still happened.

MED
  R5-6  SettingsStore.set{MaxResolution,ThemeMode}: vc=39 used
        `updateAndGet { r } == r` which is unconditionally true
        (lambda ignores prior) — the in-memory equality check was
        dead code. SP-side check still gated the disk write so the
        feature worked, but the dead branch was a comment-vs-code
        liar. Rewrote with explicit before-capture + equality
        gate.
  R5-7  SponsorBlockSkipLoop polled `controller.currentPosition`
        every 150ms even when paused — paused-overnight playback
        ate ~24k binder calls/hour. Now: when `!isPlaying`, sleep
        1s and continue.
  R5-8  StrawApp.appScope had no CoroutineExceptionHandler — an
        uncaught throwable in a top-level launch could crash on
        cold start even with SupervisorJob (top-level failure
        still propagates to default handler). Added handler that
        logs via strawLogW.
  R5-9  YtUrl.isAllowedYtUrl now requires http/https scheme
        (schemeless `//host/...` URLs no longer pass) and strips
        a single trailing dot from host (RFC FQDN form). Defense
        in depth.

LOW
  R5-10 NowPlaying.set() removed — non-CAS setter footgun
        alongside the race-free claim()/CAS path. No external
        callers (grep clean). Doc-comment updated.

Deferred (later round / different scope):
  - PlaylistsStore URL canonicalization (round-5 MED-1 — needs a
    shared YT-id-extract util; not blocking).
  - Release R8 + Nav rememberSaveable (still deferred).
  - LazyColumn keys + collectAsStateWithLifecycle (cosmetic).
  - SponsorBlockSkipLoop currentMediaItem binding (round-5
    deferred).
2026-05-25 15:12:30 -07:00
Sulkta
27818caf59 vc=37 (rust): scrub PII from strawcore info-logs
CVE round-2 HIGH-2: android_logger is configured at info-level in
release builds, so log::info!('strawcore::search query={}', query)
emits the user's actual search query to logcat. LogDump.scrubLine's
regex only catches googlevideo URLs + signed params — bare search
text rides through into a Settings → Export Logs share-sheet
attachment intact. Same for channel_info / stream_info URLs.

Replaced the value-bearing logs with shape-only (query_len /
input_len). The shape is enough to debug 'why did the search
return empty?' without the privacy hit.
2026-05-25 14:11:00 -07:00
Sulkta
fb89b22685 Phase 7 — strawcore wrapper now bridges to Sulkta-OSS/strawcore-core
Replaces the rustypipe-backed extraction with calls into the new
NPE-port crate. The UniFFI surface Kotlin sees is unchanged:

  suspend fun search(query: String): List<SearchItem>
  suspend fun streamInfo(input: String): StreamInfo
  suspend fun channelInfo(input: String): ChannelInfo
  fun initLogging()  // also wires the strawcore-core Downloader
  fun helloFromRust(name: String): String

rust/strawcore/
  * Cargo.toml      — dropped rustypipe + rquickjs-sys direct dep;
                      added strawcore-core path dep (../../../strawcore)
  * src/error.rs    — From<strawcore_core::ExtractionError>, mapping
                      ContentUnavailable variants to typed
                      StrawcoreError cases (AgeRestricted, GeoRestricted,
                      Private, RequiresLogin) instead of bucketing all
                      to Extractor
  * src/runtime.rs  — Once-guarded ReqwestDownloader init via
                      NewPipe::init_full
  * src/search.rs   — search() spawn_blocks core search_extractor::search
                      against SearchFilter::Videos
  * src/stream.rs   — stream_info() resolves URL → video_id via
                      strawcore_core::linkhandler::stream, then
                      spawn_blocks core stream_extractor::stream_info,
                      then maps StreamInfo → wrapper DTOs (combined/
                      video_only/audio_only/dash/hls)
  * src/channel.rs  — channel_info() parses input via
                      strawcore_core::linkhandler::channel (handle /
                      custom-url / legacy-user resolution lives in
                      core), then spawn_blocks core channel::channel_info

Build verified: wrapper compiles linking strawcore-core, uniffi-bindgen
generates Kotlin bindings with the same suspend fun + data class
surface Kotlin already consumes. Android NDK cross-compile + APK + on-
device smoke pending (needs build-host container).

This commits onto rollback/vc18-back-to-NPE — the existing Kotlin code
still calls NewPipeExtractor directly. Switching the Kotlin side to
consume the rust wrapper is a separate cutover.
2026-05-24 17:29:23 -07:00
Sulkta
cc3585549a v0.1.0-X (vc=12): revert to NewPipeExtractor for working playback
Phase U (rustypipe Rust extractor) rolled back. Symptom: black screen
on play, root cause: rustypipe 0.11.4's JS deobfuscator can't parse
current YouTube player.js (YT changed the obfuscation pattern, no
upstream rustypipe release since June 2025). Switching clients
(Web → TV → Android/Ios) didn't help — the deobfuscator init fires
universally.

Kept in place for the future:
- rust/strawcore/ Cargo workspace + UniFFI scaffolding
- build-host runtime install (rustup + 4 Android targets +
  cargo-ndk + NDK r27c)
- The U-2..U-5 commits in history (re-runnable when rustypipe is
  fixed or we fork it).

Restored from commit 9c4b3f831 (v0.1.0-T):
- NewPipe.init() in StrawApp.onCreate
- libs.newpipe.extractor + libs.squareup.okhttp deps
- NewPipeDownloader.kt + Thumbnails.kt
- ViewModels (Search/VideoDetail/Player/Channel/SubscriptionFeed) on
  NewPipeExtractor calls
- VideoDetailScreen Download dialog using NewPipe's StreamInfo

Future-direction memo: openclaw-workspace/memory/project_rustypipe_fork.md
— fork plan + revival path for the Rust extractor when we're ready
to maintain it.

Verified working in the Android emulator: dQw4w9WgXcQ plays, ExoPlayer
reports state=PLAYING(3), position advancing, video surface rendering.
2026-05-24 09:54:59 -07:00
Sulkta
04b0a8dbe9 v0.1.0-W2 (vc=11): fix playback — TV+Ios YT clients + visible play errors
Black-screen-on-play bug: rustypipe's default player() uses YT's Web
client, which serves stream URLs that are session/UA-locked. ExoPlayer
fetching with a different UA gets a silent 403 from googlevideo and
renders a black surface.

Fix: pin stream_info() to player_from_clients(id, [Tv, Ios]) — the
TVHTML5 + iOS Innertube clients serve direct-play URLs that work in
any HTTP player. Same trick NewPipe uses. No Apple/iOS code involved
— it's just the API client identifier rustypipe sends to YT.

Also added a Player.Listener in PlayerScreen that Toasts any
PlaybackException (codeName + message) so future stream failures don't
look like silent black screens. Logs to logcat 'StrawPlayer' too.
2026-05-24 09:33:34 -07:00
Sulkta
89b9837758 v0.1.0-V (vc=9): U-3 — streamInfo via rustypipe drives VideoDetail+Player
stream_info(url) UniFFI suspend fn replaces NewPipeExtractor's
StreamInfo.getInfo() for both VideoDetailViewModel and PlayerViewModel.
One Rust round-trip drives the detail screen render AND the player's
resolve(). The VideoDetailUiState.info field cached on detail load is
reused by the Download dialog so we don't refetch.

Deferred to U-3.5:
- like_count (rustypipe's player() doesn't surface engagement data;
  a separate query is needed)
- related (player() doesn't include 'up next'; comes from a separate
  endpoint). Kotlin gets empty list for now — RelatedRow handles it.

Type quirks vs my initial guesses (caught by cargo check):
- details.duration is u32, not Option<u32>
- channel is split into channel_id + channel_name, not a struct
- like_count doesn't exist at this query depth
- VideoFormat::Webm (lowercase mb), VideoCodec::Avc1 (not H264)
- video_only is a separate vec (video_only_streams), not a bool flag
2026-05-24 08:52:43 -07:00