v0.1.0-U (vc=8): Phase U-1 + U-2 — Rust core + rustypipe search
NewPipeExtractor (Java) → strawcore (Rust) migration begins. Phase U:
- U-1: Rust toolchain + UniFFI smoke test
- U-2: rustypipe search via uniffi suspend fun, SearchViewModel swapped
What landed:
- rust/strawcore — UniFFI-exported Rust crate using proc-macros.
Builds for arm64-v8a + armeabi-v7a + x86 + x86_64 via cargo-ndk.
Tokio multi-thread runtime singleton drives rustypipe's async API.
- strawApp/build.gradle.kts — cargoBuildHost + cargoBuild + uniffiBindgen
Gradle Exec tasks chained into the Android build. Generated Kotlin
bindings land in src/main/java/uniffi/strawcore/ (gitignored).
- SearchViewModel.kt — calls uniffi.strawcore.search(query) directly.
NewPipeExtractor still in deps for VideoDetail/Player/Channel paths;
those move to Rust in U-3 / U-4.
- Build chain quirks beat:
* cargo absolute path in Exec tasks (PATH wasn't propagating)
* uniffi-bindgen needs UNSTRIPPED host .so — separate cargoBuildHost
builds a debug-profile host lib to read metadata from
* rustypipe rustls-tls-webpki-roots avoids the openssl-sys
cross-compile tarpit
* rquickjs-sys 'bindgen' feature opted in (no prebuilt Android
bindings ship; crafting-table has libclang 14)
- crafting-table runtime install (until Dockerfile catches up):
rustup + 4 Android targets + cargo-ndk + NDK r27c. Persists in
/caches/cargo + /caches/android-sdk via the volume mount.
APK size: 22MB (U-1) → 37MB (U-2). libstrawcore.so 3-5MB per ABI carries
rustypipe + reqwest + tokio + rustls + rquickjs. NewPipeExtractor still
in for now (still drives detail + player + channel + feed), so the
Java half is doubled up. U-5 removes it.
This commit is contained in:
parent
9550b207ab
commit
7ff5ac79e5
14 changed files with 432 additions and 29 deletions
30
rust/Cargo.toml
Normal file
30
rust/Cargo.toml
Normal file
|
|
@ -0,0 +1,30 @@
|
|||
# Straw Rust core — workspace.
|
||||
# Hosts the JNI/UniFFI-exported library that replaces NewPipeExtractor.
|
||||
#
|
||||
# Builds via cargo-ndk for the four Android ABIs (arm64-v8a, armeabi-v7a,
|
||||
# x86, x86_64). The Mozilla rust-android-gradle plugin wires this into the
|
||||
# strawApp Gradle build so `./gradlew assembleDebug` produces a single APK
|
||||
# with the .so files packed inside.
|
||||
|
||||
[workspace]
|
||||
resolver = "2"
|
||||
members = ["strawcore"]
|
||||
|
||||
[workspace.package]
|
||||
edition = "2021"
|
||||
license = "GPL-3.0-or-later"
|
||||
authors = ["Sulkta-Coop"]
|
||||
repository = "http://192.168.0.5:3001/Sulkta-Coop/straw"
|
||||
|
||||
[profile.release]
|
||||
# Strip debug info, run thin LTO. APK size matters more than build time here.
|
||||
strip = true
|
||||
lto = "thin"
|
||||
codegen-units = 1
|
||||
panic = "abort"
|
||||
opt-level = "z"
|
||||
|
||||
[profile.dev]
|
||||
# Keep debug builds fast — we're rebuilding constantly during U-1..U-5.
|
||||
opt-level = 0
|
||||
debug = 1
|
||||
56
rust/README.md
Normal file
56
rust/README.md
Normal file
|
|
@ -0,0 +1,56 @@
|
|||
# `rust/` — strawcore: Rust YouTube core for Straw
|
||||
|
||||
Phase **U-** of the Straw build. Goal: replace the Java NewPipeExtractor
|
||||
dependency with a Rust core (rustypipe + tokio + reqwest), exposed to the
|
||||
Kotlin/Compose UI via [UniFFI](https://mozilla.github.io/uniffi-rs/).
|
||||
Compose UI stays in Kotlin — only the YouTube/Innertube fetching layer
|
||||
moves to Rust.
|
||||
|
||||
## Phases
|
||||
|
||||
| Phase | What |
|
||||
|---|---|
|
||||
| **U-1** | Toolchain + UniFFI smoke test (`hello_from_rust`) round-tripping through JNA. No real APIs yet. |
|
||||
| **U-2** | rustypipe search. `SearchViewModel` calls the Rust core. |
|
||||
| **U-3** | rustypipe streamInfo + streams. `VideoDetailViewModel` + `PlayerViewModel` use it. |
|
||||
| **U-4** | rustypipe channel + tabs. `ChannelViewModel` + `SubscriptionFeedViewModel`. |
|
||||
| **U-5** | Rip NewPipeExtractor Java dep. Measure APK + cold-fetch latency before/after. |
|
||||
| **U-6** *(stretch)* | SponsorBlock + RYD HTTP through reqwest + tokio in the same lib. |
|
||||
|
||||
## Build chain
|
||||
|
||||
```
|
||||
crafting-table
|
||||
├── rustup stable (target add: aarch64-linux-android, armv7-linux-androideabi,
|
||||
│ x86_64-linux-android, i686-linux-android)
|
||||
├── cargo-ndk (cross-compile helper)
|
||||
├── android-sdk (ANDROID_HOME, sdkmanager, build-tools, platforms)
|
||||
└── android-ndk (ANDROID_NDK_HOME, r27c LTS at /caches/android-sdk/ndk/...)
|
||||
|
||||
Gradle (strawApp/build.gradle.kts)
|
||||
├── cargoBuild Exec task → cargo ndk -t <abi>... -o jniLibs/ build --release
|
||||
├── uniffiBindgen Exec task → cargo run --bin uniffi-bindgen ... --library libstrawcore.so
|
||||
└── source-set wiring generated Kotlin lands in strawApp/src/main/java/uniffi/strawcore/
|
||||
|
||||
Runtime (StrawApp.onCreate)
|
||||
├── System.loadLibrary("strawcore")
|
||||
└── uniffi.strawcore.initLogging()
|
||||
```
|
||||
|
||||
## Why UniFFI (and not raw JNI / JNA bindings)
|
||||
|
||||
- Hand-written JNI: tedious, error-prone, every type change is two files
|
||||
(Kotlin + Rust) that must stay in sync.
|
||||
- Raw JNA: better, but you still hand-write the Kotlin side and worry about
|
||||
string ownership.
|
||||
- UniFFI: write Rust, annotate with `#[uniffi::export]`, get a Kotlin shim
|
||||
generated. Strings, structs, enums, Result types, `async` functions all
|
||||
cross the boundary transparently. The runtime is JNA under the hood.
|
||||
|
||||
## When in doubt
|
||||
|
||||
- `cargo check -p strawcore --target aarch64-linux-android` — fast iteration.
|
||||
- `cargo run --bin uniffi-bindgen -- generate ...` — regenerate Kotlin bindings.
|
||||
- `adb logcat -s strawcore` — Rust `log::info!()` lands here.
|
||||
- `aapt dump badging strawApp/build/outputs/apk/debug/strawApp-debug.apk` —
|
||||
inspect what ABIs/native-libs the APK carries.
|
||||
46
rust/strawcore/Cargo.toml
Normal file
46
rust/strawcore/Cargo.toml
Normal file
|
|
@ -0,0 +1,46 @@
|
|||
[package]
|
||||
name = "strawcore"
|
||||
version = "0.1.0"
|
||||
edition.workspace = true
|
||||
license.workspace = true
|
||||
authors.workspace = true
|
||||
repository.workspace = true
|
||||
|
||||
[lib]
|
||||
# cdylib — the .so that Android loads via System.loadLibrary("strawcore").
|
||||
# staticlib — kept in case we ever want to link statically for a benchmark.
|
||||
crate-type = ["cdylib", "staticlib"]
|
||||
|
||||
[dependencies]
|
||||
# UniFFI generates the Kotlin bindings + the JNI glue. proc-macro mode
|
||||
# (no .udl file) — annotate Rust fns directly with #[uniffi::export].
|
||||
# `tokio` feature wires `#[uniffi::export(async_runtime = "tokio")]` so async
|
||||
# fns surface as suspend fun on the Kotlin side.
|
||||
uniffi = { version = "0.28", features = ["cli", "tokio"] }
|
||||
# Tokio multi-thread runtime — rustypipe is async-first.
|
||||
tokio = { version = "1", features = ["rt-multi-thread", "macros", "sync"] }
|
||||
# rustypipe — the actual YouTube Innertube client. Phase U-2 wires search.
|
||||
# Force rustls + webpki-roots so we don't pull openssl-sys (cross-compiling
|
||||
# system OpenSSL to four Android ABIs is a tarpit; rustls is pure-Rust).
|
||||
rustypipe = { version = "0.11", default-features = false, features = ["rustls-tls-webpki-roots"] }
|
||||
# rquickjs-sys (transitive dep of rustypipe for YT signature decryption JS)
|
||||
# doesn't ship prebuilt Android bindings. Direct-depend with `bindgen` feature
|
||||
# so it generates them at build time. Crafting-table has libclang preinstalled.
|
||||
rquickjs-sys = { version = "0.9", default-features = false, features = ["bindgen"] }
|
||||
# Error glue.
|
||||
thiserror = "1"
|
||||
# Single-threaded init for the runtime + extractor singletons.
|
||||
once_cell = "1"
|
||||
# Android log integration — `log::info!()` ends up in `adb logcat -s strawcore`.
|
||||
log = "0.4"
|
||||
android_logger = { version = "0.14", default-features = false }
|
||||
|
||||
[build-dependencies]
|
||||
uniffi = { version = "0.28", features = ["build"] }
|
||||
|
||||
[[bin]]
|
||||
# Generates Kotlin bindings from the compiled .so:
|
||||
# cargo run --bin uniffi-bindgen generate --library target/.../libstrawcore.so \
|
||||
# --language kotlin --out-dir ../../strawApp/src/main/kotlin
|
||||
name = "uniffi-bindgen"
|
||||
path = "src/uniffi-bindgen.rs"
|
||||
5
rust/strawcore/build.rs
Normal file
5
rust/strawcore/build.rs
Normal file
|
|
@ -0,0 +1,5 @@
|
|||
// No-op build script. We're in proc-macro mode (no .udl file). The
|
||||
// #[uniffi::export] / uniffi::setup_scaffolding!() macros register
|
||||
// themselves at compile time.
|
||||
|
||||
fn main() {}
|
||||
29
rust/strawcore/src/error.rs
Normal file
29
rust/strawcore/src/error.rs
Normal file
|
|
@ -0,0 +1,29 @@
|
|||
// Strawcore error type — anything that can fail in our Rust core surfaces
|
||||
// as one of these variants. UniFFI maps it to a Kotlin sealed class
|
||||
// `StrawcoreException` so Kotlin can pattern-match on the kind.
|
||||
|
||||
use thiserror::Error;
|
||||
|
||||
#[derive(Debug, Error, uniffi::Error)]
|
||||
pub enum StrawcoreError {
|
||||
#[error("network: {msg}")]
|
||||
Network { msg: String },
|
||||
|
||||
#[error("extractor: {msg}")]
|
||||
Extractor { msg: String },
|
||||
|
||||
#[error("not found: {what}")]
|
||||
NotFound { what: String },
|
||||
|
||||
#[error("unsupported: {detail}")]
|
||||
Unsupported { detail: String },
|
||||
}
|
||||
|
||||
impl From<rustypipe::error::Error> for StrawcoreError {
|
||||
fn from(e: rustypipe::error::Error) -> Self {
|
||||
// Bucket every rustypipe error into Extractor for now. Phase U-3+
|
||||
// can pull apart specific cases (e.g. ageRestricted → NotFound,
|
||||
// network timeouts → Network) when we have a tighter UI for it.
|
||||
StrawcoreError::Extractor { msg: e.to_string() }
|
||||
}
|
||||
}
|
||||
46
rust/strawcore/src/lib.rs
Normal file
46
rust/strawcore/src/lib.rs
Normal file
|
|
@ -0,0 +1,46 @@
|
|||
// strawcore — Rust core for the Straw Android app.
|
||||
//
|
||||
// Phase U-1: hello-world smoke test. One exported function, returns a
|
||||
// String so we know JNI + UniFFI marshalling works end-to-end before we
|
||||
// pull in rustypipe.
|
||||
//
|
||||
// Phase U-2+ adds real APIs (search, stream_info, channel_info).
|
||||
|
||||
use std::sync::Once;
|
||||
|
||||
mod error;
|
||||
mod runtime;
|
||||
mod search;
|
||||
|
||||
#[allow(unused_imports)]
|
||||
use runtime::block_on;
|
||||
|
||||
// Re-exports so UniFFI sees the types at the crate root for macro discovery.
|
||||
pub use error::StrawcoreError;
|
||||
pub use search::SearchItem;
|
||||
|
||||
/// Initialize Android logging once. The Kotlin side calls this from
|
||||
/// StrawApp.onCreate() so anything emitted via `log::info!()` shows up in
|
||||
/// `adb logcat -s strawcore`.
|
||||
#[uniffi::export]
|
||||
pub fn init_logging() {
|
||||
static ONCE: Once = Once::new();
|
||||
ONCE.call_once(|| {
|
||||
android_logger::init_once(
|
||||
android_logger::Config::default()
|
||||
.with_max_level(log::LevelFilter::Info)
|
||||
.with_tag("strawcore"),
|
||||
);
|
||||
log::info!("strawcore initialized");
|
||||
});
|
||||
}
|
||||
|
||||
/// Smoke-test entry point — round-trip a string through JNI so we know
|
||||
/// the bridge is working before pulling in rustypipe.
|
||||
#[uniffi::export]
|
||||
pub fn hello_from_rust(name: String) -> String {
|
||||
log::info!("hello_from_rust called with name={}", name);
|
||||
format!("hello {} from rust 🦀 (strawcore v{})", name, env!("CARGO_PKG_VERSION"))
|
||||
}
|
||||
|
||||
uniffi::setup_scaffolding!();
|
||||
21
rust/strawcore/src/runtime.rs
Normal file
21
rust/strawcore/src/runtime.rs
Normal file
|
|
@ -0,0 +1,21 @@
|
|||
// Tokio runtime singleton — rustypipe is async-first, so every exported
|
||||
// function that touches the network needs to drive a runtime. Building one
|
||||
// per call is wasteful; we build one shared multi-thread runtime at first
|
||||
// use and `block_on` against it.
|
||||
|
||||
use once_cell::sync::Lazy;
|
||||
use tokio::runtime::Runtime;
|
||||
|
||||
static RT: Lazy<Runtime> = Lazy::new(|| {
|
||||
tokio::runtime::Builder::new_multi_thread()
|
||||
.worker_threads(2) // Mobile — we don't need many. Most blocks are I/O.
|
||||
.enable_all()
|
||||
.thread_name("strawcore-tokio")
|
||||
.build()
|
||||
.expect("strawcore: failed to build tokio runtime")
|
||||
});
|
||||
|
||||
#[allow(dead_code)]
|
||||
pub fn block_on<F: std::future::Future>(fut: F) -> F::Output {
|
||||
RT.block_on(fut)
|
||||
}
|
||||
65
rust/strawcore/src/search.rs
Normal file
65
rust/strawcore/src/search.rs
Normal file
|
|
@ -0,0 +1,65 @@
|
|||
// Phase U-2 — search via rustypipe, exposed to Kotlin as a suspend fun.
|
||||
//
|
||||
// `SearchItem` mirrors the existing Kotlin `StreamItem` field for field so
|
||||
// `SearchViewModel` can swap NewPipeExtractor for this with a one-line
|
||||
// change. We map only Video items from rustypipe's result (it also returns
|
||||
// channels and playlists which we don't need for the search list yet).
|
||||
|
||||
use crate::error::StrawcoreError;
|
||||
use rustypipe::client::RustyPipe;
|
||||
use rustypipe::model::YouTubeItem;
|
||||
|
||||
#[derive(Debug, Clone, uniffi::Record)]
|
||||
pub struct SearchItem {
|
||||
pub url: String,
|
||||
pub title: String,
|
||||
pub uploader: String,
|
||||
pub uploader_url: Option<String>,
|
||||
pub thumbnail: Option<String>,
|
||||
/// Duration in seconds. 0 = live/unknown.
|
||||
pub duration_seconds: i64,
|
||||
/// Reported view count. 0 = unknown.
|
||||
pub view_count: i64,
|
||||
}
|
||||
|
||||
fn yt_video_url(id: &str) -> String {
|
||||
format!("https://www.youtube.com/watch?v={}", id)
|
||||
}
|
||||
|
||||
fn yt_channel_url(id: &str) -> String {
|
||||
format!("https://www.youtube.com/channel/{}", id)
|
||||
}
|
||||
|
||||
#[uniffi::export(async_runtime = "tokio")]
|
||||
pub async fn search(query: String) -> Result<Vec<SearchItem>, StrawcoreError> {
|
||||
log::info!("strawcore::search query={}", query);
|
||||
let rp = RustyPipe::new();
|
||||
let results = rp.query().search(query).await?;
|
||||
|
||||
let items: Vec<SearchItem> = results
|
||||
.items
|
||||
.items
|
||||
.into_iter()
|
||||
.filter_map(|item| match item {
|
||||
YouTubeItem::Video(v) => Some(SearchItem {
|
||||
url: yt_video_url(&v.id),
|
||||
title: v.name,
|
||||
uploader: v
|
||||
.channel
|
||||
.as_ref()
|
||||
.map(|c| c.name.clone())
|
||||
.unwrap_or_default(),
|
||||
uploader_url: v.channel.as_ref().map(|c| yt_channel_url(&c.id)),
|
||||
thumbnail: v.thumbnail.first().map(|t| t.url.clone()),
|
||||
duration_seconds: v.duration.unwrap_or(0) as i64,
|
||||
view_count: v.view_count.unwrap_or(0) as i64,
|
||||
}),
|
||||
// Channels + playlists are dropped at U-2; future phases can
|
||||
// surface them as a "channels you might like" row.
|
||||
_ => None,
|
||||
})
|
||||
.collect();
|
||||
|
||||
log::info!("strawcore::search returned {} videos", items.len());
|
||||
Ok(items)
|
||||
}
|
||||
6
rust/strawcore/src/uniffi-bindgen.rs
Normal file
6
rust/strawcore/src/uniffi-bindgen.rs
Normal file
|
|
@ -0,0 +1,6 @@
|
|||
// Tiny entry point so `cargo run --bin uniffi-bindgen` works — the actual
|
||||
// logic lives in the uniffi crate, this just delegates to its CLI.
|
||||
|
||||
fn main() {
|
||||
uniffi::uniffi_bindgen_main()
|
||||
}
|
||||
Loading…
Add table
Add a link
Reference in a new issue