v0.1.0-U (vc=8): Phase U-1 + U-2 — Rust core + rustypipe search

NewPipeExtractor (Java) → strawcore (Rust) migration begins. Phase U:
- U-1: Rust toolchain + UniFFI smoke test
- U-2: rustypipe search via uniffi suspend fun, SearchViewModel swapped

What landed:
- rust/strawcore — UniFFI-exported Rust crate using proc-macros.
  Builds for arm64-v8a + armeabi-v7a + x86 + x86_64 via cargo-ndk.
  Tokio multi-thread runtime singleton drives rustypipe's async API.
- strawApp/build.gradle.kts — cargoBuildHost + cargoBuild + uniffiBindgen
  Gradle Exec tasks chained into the Android build. Generated Kotlin
  bindings land in src/main/java/uniffi/strawcore/ (gitignored).
- SearchViewModel.kt — calls uniffi.strawcore.search(query) directly.
  NewPipeExtractor still in deps for VideoDetail/Player/Channel paths;
  those move to Rust in U-3 / U-4.
- Build chain quirks beat:
  * cargo absolute path in Exec tasks (PATH wasn't propagating)
  * uniffi-bindgen needs UNSTRIPPED host .so — separate cargoBuildHost
    builds a debug-profile host lib to read metadata from
  * rustypipe rustls-tls-webpki-roots avoids the openssl-sys
    cross-compile tarpit
  * rquickjs-sys 'bindgen' feature opted in (no prebuilt Android
    bindings ship; crafting-table has libclang 14)
- crafting-table runtime install (until Dockerfile catches up):
  rustup + 4 Android targets + cargo-ndk + NDK r27c. Persists in
  /caches/cargo + /caches/android-sdk via the volume mount.

APK size: 22MB (U-1) → 37MB (U-2). libstrawcore.so 3-5MB per ABI carries
rustypipe + reqwest + tokio + rustls + rquickjs. NewPipeExtractor still
in for now (still drives detail + player + channel + feed), so the
Java half is doubled up. U-5 removes it.
This commit is contained in:
Kayos 2026-05-24 08:36:50 -07:00
parent 9550b207ab
commit 7ff5ac79e5
14 changed files with 432 additions and 29 deletions

30
rust/Cargo.toml Normal file
View file

@ -0,0 +1,30 @@
# Straw Rust core — workspace.
# Hosts the JNI/UniFFI-exported library that replaces NewPipeExtractor.
#
# Builds via cargo-ndk for the four Android ABIs (arm64-v8a, armeabi-v7a,
# x86, x86_64). The Mozilla rust-android-gradle plugin wires this into the
# strawApp Gradle build so `./gradlew assembleDebug` produces a single APK
# with the .so files packed inside.
[workspace]
resolver = "2"
members = ["strawcore"]
[workspace.package]
edition = "2021"
license = "GPL-3.0-or-later"
authors = ["Sulkta-Coop"]
repository = "http://192.168.0.5:3001/Sulkta-Coop/straw"
[profile.release]
# Strip debug info, run thin LTO. APK size matters more than build time here.
strip = true
lto = "thin"
codegen-units = 1
panic = "abort"
opt-level = "z"
[profile.dev]
# Keep debug builds fast — we're rebuilding constantly during U-1..U-5.
opt-level = 0
debug = 1

56
rust/README.md Normal file
View file

@ -0,0 +1,56 @@
# `rust/` — strawcore: Rust YouTube core for Straw
Phase **U-** of the Straw build. Goal: replace the Java NewPipeExtractor
dependency with a Rust core (rustypipe + tokio + reqwest), exposed to the
Kotlin/Compose UI via [UniFFI](https://mozilla.github.io/uniffi-rs/).
Compose UI stays in Kotlin — only the YouTube/Innertube fetching layer
moves to Rust.
## Phases
| Phase | What |
|---|---|
| **U-1** | Toolchain + UniFFI smoke test (`hello_from_rust`) round-tripping through JNA. No real APIs yet. |
| **U-2** | rustypipe search. `SearchViewModel` calls the Rust core. |
| **U-3** | rustypipe streamInfo + streams. `VideoDetailViewModel` + `PlayerViewModel` use it. |
| **U-4** | rustypipe channel + tabs. `ChannelViewModel` + `SubscriptionFeedViewModel`. |
| **U-5** | Rip NewPipeExtractor Java dep. Measure APK + cold-fetch latency before/after. |
| **U-6** *(stretch)* | SponsorBlock + RYD HTTP through reqwest + tokio in the same lib. |
## Build chain
```
crafting-table
├── rustup stable (target add: aarch64-linux-android, armv7-linux-androideabi,
│ x86_64-linux-android, i686-linux-android)
├── cargo-ndk (cross-compile helper)
├── android-sdk (ANDROID_HOME, sdkmanager, build-tools, platforms)
└── android-ndk (ANDROID_NDK_HOME, r27c LTS at /caches/android-sdk/ndk/...)
Gradle (strawApp/build.gradle.kts)
├── cargoBuild Exec task → cargo ndk -t <abi>... -o jniLibs/ build --release
├── uniffiBindgen Exec task → cargo run --bin uniffi-bindgen ... --library libstrawcore.so
└── source-set wiring generated Kotlin lands in strawApp/src/main/java/uniffi/strawcore/
Runtime (StrawApp.onCreate)
├── System.loadLibrary("strawcore")
└── uniffi.strawcore.initLogging()
```
## Why UniFFI (and not raw JNI / JNA bindings)
- Hand-written JNI: tedious, error-prone, every type change is two files
(Kotlin + Rust) that must stay in sync.
- Raw JNA: better, but you still hand-write the Kotlin side and worry about
string ownership.
- UniFFI: write Rust, annotate with `#[uniffi::export]`, get a Kotlin shim
generated. Strings, structs, enums, Result types, `async` functions all
cross the boundary transparently. The runtime is JNA under the hood.
## When in doubt
- `cargo check -p strawcore --target aarch64-linux-android` — fast iteration.
- `cargo run --bin uniffi-bindgen -- generate ...` — regenerate Kotlin bindings.
- `adb logcat -s strawcore` — Rust `log::info!()` lands here.
- `aapt dump badging strawApp/build/outputs/apk/debug/strawApp-debug.apk`
inspect what ABIs/native-libs the APK carries.

46
rust/strawcore/Cargo.toml Normal file
View file

@ -0,0 +1,46 @@
[package]
name = "strawcore"
version = "0.1.0"
edition.workspace = true
license.workspace = true
authors.workspace = true
repository.workspace = true
[lib]
# cdylib — the .so that Android loads via System.loadLibrary("strawcore").
# staticlib — kept in case we ever want to link statically for a benchmark.
crate-type = ["cdylib", "staticlib"]
[dependencies]
# UniFFI generates the Kotlin bindings + the JNI glue. proc-macro mode
# (no .udl file) — annotate Rust fns directly with #[uniffi::export].
# `tokio` feature wires `#[uniffi::export(async_runtime = "tokio")]` so async
# fns surface as suspend fun on the Kotlin side.
uniffi = { version = "0.28", features = ["cli", "tokio"] }
# Tokio multi-thread runtime — rustypipe is async-first.
tokio = { version = "1", features = ["rt-multi-thread", "macros", "sync"] }
# rustypipe — the actual YouTube Innertube client. Phase U-2 wires search.
# Force rustls + webpki-roots so we don't pull openssl-sys (cross-compiling
# system OpenSSL to four Android ABIs is a tarpit; rustls is pure-Rust).
rustypipe = { version = "0.11", default-features = false, features = ["rustls-tls-webpki-roots"] }
# rquickjs-sys (transitive dep of rustypipe for YT signature decryption JS)
# doesn't ship prebuilt Android bindings. Direct-depend with `bindgen` feature
# so it generates them at build time. Crafting-table has libclang preinstalled.
rquickjs-sys = { version = "0.9", default-features = false, features = ["bindgen"] }
# Error glue.
thiserror = "1"
# Single-threaded init for the runtime + extractor singletons.
once_cell = "1"
# Android log integration — `log::info!()` ends up in `adb logcat -s strawcore`.
log = "0.4"
android_logger = { version = "0.14", default-features = false }
[build-dependencies]
uniffi = { version = "0.28", features = ["build"] }
[[bin]]
# Generates Kotlin bindings from the compiled .so:
# cargo run --bin uniffi-bindgen generate --library target/.../libstrawcore.so \
# --language kotlin --out-dir ../../strawApp/src/main/kotlin
name = "uniffi-bindgen"
path = "src/uniffi-bindgen.rs"

5
rust/strawcore/build.rs Normal file
View file

@ -0,0 +1,5 @@
// No-op build script. We're in proc-macro mode (no .udl file). The
// #[uniffi::export] / uniffi::setup_scaffolding!() macros register
// themselves at compile time.
fn main() {}

View file

@ -0,0 +1,29 @@
// Strawcore error type — anything that can fail in our Rust core surfaces
// as one of these variants. UniFFI maps it to a Kotlin sealed class
// `StrawcoreException` so Kotlin can pattern-match on the kind.
use thiserror::Error;
#[derive(Debug, Error, uniffi::Error)]
pub enum StrawcoreError {
#[error("network: {msg}")]
Network { msg: String },
#[error("extractor: {msg}")]
Extractor { msg: String },
#[error("not found: {what}")]
NotFound { what: String },
#[error("unsupported: {detail}")]
Unsupported { detail: String },
}
impl From<rustypipe::error::Error> for StrawcoreError {
fn from(e: rustypipe::error::Error) -> Self {
// Bucket every rustypipe error into Extractor for now. Phase U-3+
// can pull apart specific cases (e.g. ageRestricted → NotFound,
// network timeouts → Network) when we have a tighter UI for it.
StrawcoreError::Extractor { msg: e.to_string() }
}
}

46
rust/strawcore/src/lib.rs Normal file
View file

@ -0,0 +1,46 @@
// strawcore — Rust core for the Straw Android app.
//
// Phase U-1: hello-world smoke test. One exported function, returns a
// String so we know JNI + UniFFI marshalling works end-to-end before we
// pull in rustypipe.
//
// Phase U-2+ adds real APIs (search, stream_info, channel_info).
use std::sync::Once;
mod error;
mod runtime;
mod search;
#[allow(unused_imports)]
use runtime::block_on;
// Re-exports so UniFFI sees the types at the crate root for macro discovery.
pub use error::StrawcoreError;
pub use search::SearchItem;
/// Initialize Android logging once. The Kotlin side calls this from
/// StrawApp.onCreate() so anything emitted via `log::info!()` shows up in
/// `adb logcat -s strawcore`.
#[uniffi::export]
pub fn init_logging() {
static ONCE: Once = Once::new();
ONCE.call_once(|| {
android_logger::init_once(
android_logger::Config::default()
.with_max_level(log::LevelFilter::Info)
.with_tag("strawcore"),
);
log::info!("strawcore initialized");
});
}
/// Smoke-test entry point — round-trip a string through JNI so we know
/// the bridge is working before pulling in rustypipe.
#[uniffi::export]
pub fn hello_from_rust(name: String) -> String {
log::info!("hello_from_rust called with name={}", name);
format!("hello {} from rust 🦀 (strawcore v{})", name, env!("CARGO_PKG_VERSION"))
}
uniffi::setup_scaffolding!();

View file

@ -0,0 +1,21 @@
// Tokio runtime singleton — rustypipe is async-first, so every exported
// function that touches the network needs to drive a runtime. Building one
// per call is wasteful; we build one shared multi-thread runtime at first
// use and `block_on` against it.
use once_cell::sync::Lazy;
use tokio::runtime::Runtime;
static RT: Lazy<Runtime> = Lazy::new(|| {
tokio::runtime::Builder::new_multi_thread()
.worker_threads(2) // Mobile — we don't need many. Most blocks are I/O.
.enable_all()
.thread_name("strawcore-tokio")
.build()
.expect("strawcore: failed to build tokio runtime")
});
#[allow(dead_code)]
pub fn block_on<F: std::future::Future>(fut: F) -> F::Output {
RT.block_on(fut)
}

View file

@ -0,0 +1,65 @@
// Phase U-2 — search via rustypipe, exposed to Kotlin as a suspend fun.
//
// `SearchItem` mirrors the existing Kotlin `StreamItem` field for field so
// `SearchViewModel` can swap NewPipeExtractor for this with a one-line
// change. We map only Video items from rustypipe's result (it also returns
// channels and playlists which we don't need for the search list yet).
use crate::error::StrawcoreError;
use rustypipe::client::RustyPipe;
use rustypipe::model::YouTubeItem;
#[derive(Debug, Clone, uniffi::Record)]
pub struct SearchItem {
pub url: String,
pub title: String,
pub uploader: String,
pub uploader_url: Option<String>,
pub thumbnail: Option<String>,
/// Duration in seconds. 0 = live/unknown.
pub duration_seconds: i64,
/// Reported view count. 0 = unknown.
pub view_count: i64,
}
fn yt_video_url(id: &str) -> String {
format!("https://www.youtube.com/watch?v={}", id)
}
fn yt_channel_url(id: &str) -> String {
format!("https://www.youtube.com/channel/{}", id)
}
#[uniffi::export(async_runtime = "tokio")]
pub async fn search(query: String) -> Result<Vec<SearchItem>, StrawcoreError> {
log::info!("strawcore::search query={}", query);
let rp = RustyPipe::new();
let results = rp.query().search(query).await?;
let items: Vec<SearchItem> = results
.items
.items
.into_iter()
.filter_map(|item| match item {
YouTubeItem::Video(v) => Some(SearchItem {
url: yt_video_url(&v.id),
title: v.name,
uploader: v
.channel
.as_ref()
.map(|c| c.name.clone())
.unwrap_or_default(),
uploader_url: v.channel.as_ref().map(|c| yt_channel_url(&c.id)),
thumbnail: v.thumbnail.first().map(|t| t.url.clone()),
duration_seconds: v.duration.unwrap_or(0) as i64,
view_count: v.view_count.unwrap_or(0) as i64,
}),
// Channels + playlists are dropped at U-2; future phases can
// surface them as a "channels you might like" row.
_ => None,
})
.collect();
log::info!("strawcore::search returned {} videos", items.len());
Ok(items)
}

View file

@ -0,0 +1,6 @@
// Tiny entry point so `cargo run --bin uniffi-bindgen` works — the actual
// logic lives in the uniffi crate, this just delegates to its CLI.
fn main() {
uniffi::uniffi_bindgen_main()
}