Commit graph

5 commits

Author SHA1 Message Date
ecc54aaf38 vc=41: loop round 3/5 — round-2 misses caught + duplicate-entry zip guard
Three Opus round-6 audits found important regressions on the vc=39/40
fixes plus a fresh hostile-zip attack surface.

HIGH
  R6-1  SearchViewModel.submit fence-by-query swallowed valid
        results. `onQueryChange` mutates _ui.value.query without
        cancelling inFlight (for reactive cache filtering as the
        user types). The vc=40 string-equality fence treated a
        still-valid result as stale just because the user kept
        typing after submitting. Re-fenced by Job identity via
        ensureActive() — only a fresh submit (which calls
        inFlight?.cancel()) invalidates us, mere typing doesn't.
  R6-2  SettingsImport.run wrapped `runInner` (suspend) in plain
        runCatching, swallowing CancellationException and
        surfacing it as a Result.failure that produced a misleading
        "import failed" banner on a user-back abort. Migrated to
        runCatchingCancellable — exactly what round 5 added the
        util for; this call site was missed.
  R6-3  VideoDetail/ChannelViewModel.load early-return on bad URL
        didn't cancel inFlight. An in-flight prior load could
        resolve past the suspension point, see its fence pass
        (loadedUrl unchanged because we didn't update it on the
        rejected call), and clobber the "Unsupported URL" error
        banner the user is looking at. Now: inFlight?.cancel() +
        inFlight = null before the early return.
  R6-4  Rust ensure_initialized mutex contention pathology. The
        vc=40 mutex-first ordering correctly serialized init but
        blocked N concurrent tokio workers for the full duration
        of a slow ReqwestDownloader::new() (e.g. ~7s on a 6s DNS
        timeout). On a 4-core phone that froze the app for 7s.
        New: backoff check FIRST (lock-free), then try_lock — if
        someone else is initializing, return immediately. Caller
        gets one DownloaderMissing they can retry past, instead
        of multi-second UI lockup.

MED
  R6-5  Hostile zip with duplicate `newpipe.db` entries could
        masquerade a benign first DB past any pre-validation and
        ship the second malicious one (ZipInputStream walks in
        order; second write overwrites). Now: reject the archive
        when either `newpipe.db` or `preferences.json` appears
        twice.
  R6-6  importPlaylists outer + inner queries had no LIMIT — a
        crafted DB with 10M playlist rows or 10M items could walk
        unbounded cursors into memory even under the 256 MB DB
        size cap. LIMIT 256 / LIMIT 5000 now match the discipline
        on the other import paths.
  R6-7  SponsorBlock pickActiveSegment had a `posSec < endSec -
        0.05` exclusion that combined with the 150ms polling
        cadence missed short (<200ms) filler/reminder segments
        entirely. Dropped — the rate-limit work made it
        unnecessary.
  R6-8  SettingsImport.importSettings `applied++` was outside the
        `want != have` branch — inflated the import-summary count
        to "12 settings applied" when only 2 changed.

Deferred:
  - PlaylistsStore URL canonicalization (still deferred — needs
    shared YT-id-extract util, not blocking)
  - SQLite header magic validation (low-value defense-in-depth)
  - SP write reorder serialization (bigger refactor)
  - updateAvatar coalesced persists (12 parallel = wasted CPU
    but not visible)
  - Proguard rules staging (paired with R8 enable, both deferred)
  - DASH/HLS maxResolution cap via TrackSelectionParameters
    (real bug but needs careful Media3 wiring)
2026-05-25 15:25:25 -07:00
da48109a4d vc=40: loop round 2/5 — round-1 misses + new HIGHs from round-5 audits
Three parallel Opus round-5 audits ran on vc=39. The big finds were
regressions on vc=39's own fixes — the retry-init wiring was incomplete,
the URL-fence pattern missed CancellationException swallowing, and the
SettingsStore idempotency branch was dead code. Real round-5 work.

HIGH
  R5-1  Rust `ensure_initialized` was only called from `init_logging`
        — extractor entry points never invoked it, so the 5s-backoff
        retry from vc=39 was unreachable. A cold-boot init failure
        still bricked extraction for the whole process. Now every
        `search()` / `stream_info()` / `channel_info()` calls
        ensure_initialized at entry; cheap when INITIALIZED is true.
  R5-2  runtime.rs in-progress race: prior shape stamped
        LAST_ATTEMPT_MS at the *start* of an attempt, then let
        concurrent callers short-circuit via the backoff check
        and proceed to call into the extractor before init
        completed. New: lock FIRST (mutex IS the in-progress
        queue), re-check INITIALIZED under lock, only THEN check
        backoff (and only stamp it on failure).
  R5-3  `runCatching` in VideoDetailViewModel + SubscriptionFeed
        swallows CancellationException — when a job was cancelled
        mid-suspend inside channelInfo/RYD/SB, the inner cancellation
        was eaten, the lambda returned its default, the job carried
        past the runCatching to its terminal write, and the URL fence
        let it through because same-URL races can't be distinguished
        by string equality. New util.runCatchingCancellable
        re-throws CancellationException; all coroutine-body
        runCatching sites in the affected VMs migrated.
  R5-4  SearchViewModel.submit post-network fence only guarded
        `_ui.update`. SearchCache.record + pool rebuild proceeded
        for a cancelled query → ghost cache entries for queries
        the user abandoned mid-stroke. Now: re-check the query
        AFTER the IO suspend and before the cache write.
  R5-5  ChannelViewModel.load / VideoDetailViewModel.load now gate
        the extractor entry on isAllowedYtUrl(channelUrl/streamUrl).
        Prior shape only gated recordWatch persistence — extractor
        invocation for poisoned uploaderUrl still happened.

MED
  R5-6  SettingsStore.set{MaxResolution,ThemeMode}: vc=39 used
        `updateAndGet { r } == r` which is unconditionally true
        (lambda ignores prior) — the in-memory equality check was
        dead code. SP-side check still gated the disk write so the
        feature worked, but the dead branch was a comment-vs-code
        liar. Rewrote with explicit before-capture + equality
        gate.
  R5-7  SponsorBlockSkipLoop polled `controller.currentPosition`
        every 150ms even when paused — paused-overnight playback
        ate ~24k binder calls/hour. Now: when `!isPlaying`, sleep
        1s and continue.
  R5-8  StrawApp.appScope had no CoroutineExceptionHandler — an
        uncaught throwable in a top-level launch could crash on
        cold start even with SupervisorJob (top-level failure
        still propagates to default handler). Added handler that
        logs via strawLogW.
  R5-9  YtUrl.isAllowedYtUrl now requires http/https scheme
        (schemeless `//host/...` URLs no longer pass) and strips
        a single trailing dot from host (RFC FQDN form). Defense
        in depth.

LOW
  R5-10 NowPlaying.set() removed — non-CAS setter footgun
        alongside the race-free claim()/CAS path. No external
        callers (grep clean). Doc-comment updated.

Deferred (later round / different scope):
  - PlaylistsStore URL canonicalization (round-5 MED-1 — needs a
    shared YT-id-extract util; not blocking).
  - Release R8 + Nav rememberSaveable (still deferred).
  - LazyColumn keys + collectAsStateWithLifecycle (cosmetic).
  - SponsorBlockSkipLoop currentMediaItem binding (round-5
    deferred).
2026-05-25 15:12:30 -07:00
b8325d1726 vc=39: loop round 1/5 — 9 HIGH + 7 MED from 3 Opus round-4 audits
Three parallel Opus max-effort audits ran on vc=38. No new CRITs (the
LogDump + VM-error-scrub chain held), but real new HIGHs across VMs
that weren't touched in rounds 1-3 + the Rust runtime's brittle
one-shot init.

HIGH
  R4-1  Rust runtime::ensure_initialized was one-shot via Once.
        First-call failure (cold-boot in airplane mode, transient
        DNS/SELinux denial on first TLS init) consumed the Once slot
        and bricked the extractor for the rest of the process —
        every subsequent search/streamInfo/channelInfo returned
        DownloaderMissing forever. Replaced with AtomicBool + 5s
        backoff retry; success closes the door, failure retries on
        the next call.
  R4-2  VideoDetailViewModel.load tracked no inFlight Job.
        Activity-scoped VM is reused; tap video A → quickly tap a
        related-video B → both loads race, slower-finisher wins.
        A's resolved payload (different itags, different SB
        segments, wrong title chip) could render on the B detail
        page; recordWatch logged B while the player streamed A.
        Now: inFlight?.cancel() at top, fenced terminal writes with
        loadedUrl-stable guard. Same shape applied to
        ChannelViewModel (had no in-flight tracking at all).
  R4-3  `_ui.value = _ui.value.copy(...)` lost-write patterns
        survived round-3's pass in SearchViewModel + VideoDetail +
        Channel. Migrated all to `_ui.update {}` — same atomicity
        regression class round 3 was supposed to close. Submit/load
        terminal writes also now fence against late-arrivals.
  R4-4  HistoryStore.recordAllWatches reported `size_after -
        size_before` to SettingsImport — at a saturated store the
        post-state size equals the pre-state size even when 20
        fresh imports landed and 20 older entries got truncated.
        User saw "0 watch history imported" when 30 actually
        landed. Now: recordAllWatches/recordAllSearches return an
        AtomicInteger-counted actual-fresh-added count from inside
        the CAS lambda; SettingsImport plumbs through to the report.
  R4-5  SubscriptionFeedViewModel.refresh() filtered to stale-only
        — user-initiated tap of Refresh was a silent no-op when
        every channel had been refreshed in the last 28min.
        Split: refresh() forces fan-out across every sub;
        refreshIfStale() keeps the TTL filter. Both share
        refreshInternal(force: Bool).
  R4-6  SettingsImport.importPlaylists called create() + addItem()
        in a loop — both write SP, and addItem walks every playlist
        linearly per insert. A NewPipe export with 100 playlists ×
        100 items = ~10k SP commits + O(N²) work. New
        PlaylistsStore.importPlaylist mints a single Playlist with
        pre-attached items, one CAS, one SP write per playlist.
  R4-7  VideoDetailViewModel auto-called channelInfo(uploaderUrl)
        on every load — no allowlist gate. An extractor-emitted
        non-YT uploaderUrl (poisoned related/moreFromChannel)
        would have triggered an arbitrary-host network call.
  R4-8  Similar shape: VideoDetailViewModel.recordWatch persisted
        whatever URL was passed to load() — extractor-emitted non-YT
        URLs would have survived in Recent Watches past process
        death. Same import-time URL allowlist now gates both.
  CVE-1 The reCAPTCHA error path embedded the full google.com/sorry/
        URL into the user-visible banner. That URL carries
        `continue=<full-signed-googlevideo-url>` — and LogDump's
        scrub only matches googlevideo.com hosts. Now: strip the
        `continue=` param in Rust before propagating; UI shows a
        tappable challenge URL that still solves the rate-limit
        when the user opens it.

MED
  R4-9  SettingsStore.setMaxResolution/setThemeMode/setCacheEnabled
        were not atomic vs toggle()'s updateAndGet pattern. Now
        CAS-safe + idempotent (no SP write when the value is
        already what's stored).
  R4-10 SponsorBlockClient.fetch built the URL via string concat
        with un-percent-encoded JSON-shaped categories list.
        Switched to HttpUrl.Builder().addQueryParameter() — okhttp
        does the right escaping. SB happens to accept the raw form
        today; this guards future user-typed categories.
  R4-11 strawHttpClient() synchronized on the interned
        STRAW_USER_AGENT string literal — any unrelated code that
        happened to lock the same literal could contend. Replaced
        with lazy(SYNCHRONIZED) — same one-shot init, no shared
        global lock.
  R4-12 DownloadsScreen.queryDownloads ran on the main coroutine
        every 1-5s. DownloadManager.query is a ContentResolver IPC
        + SQLite cursor walk; on devices with hundreds of historical
        downloads it stuttered. withContext(Dispatchers.IO).
  R4-13 Co-located the YT host allowlist (was inline in
        SettingsImport) into util/YtUrl.kt — VideoDetailViewModel
        now imports the same function. Future host changes are
        one edit.

Deferred to round 2-5:
  R4-MED — Nav.kt has no rememberSaveable / Parcelize on Screen
        sealed types. Process-death loses entire back stack.
        Needs Parcelize plugin add + listSaver — bigger refactor.
  R4-HIGH — Release isMinifyEnabled = false / no R8. Needs
        comprehensive keep-rules for UniFFI + kotlinx-serialization
        before flipping safely. Holding for a dedicated round.
  R4-MED — LazyColumn key= missing in 5 list sites; quick win
        but cosmetic, won't slip into post-round-5 ship.
  R4-MED — collectAsStateWithLifecycle bulk-replace.
  R4-MED — SponsorBlock skip-loop should bind segments to
        controller.currentMediaItem to avoid one-tick misapply on
        track changes.
2026-05-25 14:56:38 -07:00
467a5f10fa Phase 7 — strawcore wrapper now bridges to Sulkta-Coop/strawcore-core
Replaces the rustypipe-backed extraction with calls into the new
NPE-port crate. The UniFFI surface Kotlin sees is unchanged:

  suspend fun search(query: String): List<SearchItem>
  suspend fun streamInfo(input: String): StreamInfo
  suspend fun channelInfo(input: String): ChannelInfo
  fun initLogging()  // also wires the strawcore-core Downloader
  fun helloFromRust(name: String): String

rust/strawcore/
  * Cargo.toml      — dropped rustypipe + rquickjs-sys direct dep;
                      added strawcore-core path dep (../../../strawcore)
  * src/error.rs    — From<strawcore_core::ExtractionError>, mapping
                      ContentUnavailable variants to typed
                      StrawcoreError cases (AgeRestricted, GeoRestricted,
                      Private, RequiresLogin) instead of bucketing all
                      to Extractor
  * src/runtime.rs  — Once-guarded ReqwestDownloader init via
                      NewPipe::init_full
  * src/search.rs   — search() spawn_blocks core search_extractor::search
                      against SearchFilter::Videos
  * src/stream.rs   — stream_info() resolves URL → video_id via
                      strawcore_core::linkhandler::stream, then
                      spawn_blocks core stream_extractor::stream_info,
                      then maps StreamInfo → wrapper DTOs (combined/
                      video_only/audio_only/dash/hls)
  * src/channel.rs  — channel_info() parses input via
                      strawcore_core::linkhandler::channel (handle /
                      custom-url / legacy-user resolution lives in
                      core), then spawn_blocks core channel::channel_info

Build verified: wrapper compiles linking strawcore-core, uniffi-bindgen
generates Kotlin bindings with the same suspend fun + data class
surface Kotlin already consumes. Android NDK cross-compile + APK + on-
device smoke pending (needs crafting-table container).

This commits onto rollback/vc18-back-to-NPE — the existing Kotlin code
still calls NewPipeExtractor directly. Switching the Kotlin side to
consume the rust wrapper is a separate cutover.
2026-05-24 17:29:23 -07:00
7ff5ac79e5 v0.1.0-U (vc=8): Phase U-1 + U-2 — Rust core + rustypipe search
NewPipeExtractor (Java) → strawcore (Rust) migration begins. Phase U:
- U-1: Rust toolchain + UniFFI smoke test
- U-2: rustypipe search via uniffi suspend fun, SearchViewModel swapped

What landed:
- rust/strawcore — UniFFI-exported Rust crate using proc-macros.
  Builds for arm64-v8a + armeabi-v7a + x86 + x86_64 via cargo-ndk.
  Tokio multi-thread runtime singleton drives rustypipe's async API.
- strawApp/build.gradle.kts — cargoBuildHost + cargoBuild + uniffiBindgen
  Gradle Exec tasks chained into the Android build. Generated Kotlin
  bindings land in src/main/java/uniffi/strawcore/ (gitignored).
- SearchViewModel.kt — calls uniffi.strawcore.search(query) directly.
  NewPipeExtractor still in deps for VideoDetail/Player/Channel paths;
  those move to Rust in U-3 / U-4.
- Build chain quirks beat:
  * cargo absolute path in Exec tasks (PATH wasn't propagating)
  * uniffi-bindgen needs UNSTRIPPED host .so — separate cargoBuildHost
    builds a debug-profile host lib to read metadata from
  * rustypipe rustls-tls-webpki-roots avoids the openssl-sys
    cross-compile tarpit
  * rquickjs-sys 'bindgen' feature opted in (no prebuilt Android
    bindings ship; crafting-table has libclang 14)
- crafting-table runtime install (until Dockerfile catches up):
  rustup + 4 Android targets + cargo-ndk + NDK r27c. Persists in
  /caches/cargo + /caches/android-sdk via the volume mount.

APK size: 22MB (U-1) → 37MB (U-2). libstrawcore.so 3-5MB per ABI carries
rustypipe + reqwest + tokio + rustls + rquickjs. NewPipeExtractor still
in for now (still drives detail + player + channel + feed), so the
Java half is doubled up. U-5 removes it.
2026-05-24 08:36:50 -07:00