docs: rewrite README for public release

2026-06-28 12:06:47 -07:00 · 2026-06-28 12:06:47 -07:00 · 3740ec8f75
commit 3740ec8f75
parent c4c1d9436a
1 changed files with 108 additions and 6 deletions
--- a/README.md
+++ b/README.md
@ -1,17 +1,119 @@
 # strawcore

-Rust port of [NewPipeExtractor](https://github.com/TeamNewPipe/NewPipeExtractor) (v0.26.2), YouTube-only. Plugs into [Straw](https://git.sulkta.com/Sulkta-OSS/straw) via UniFFI.
+A Rust library for extracting YouTube stream URLs, metadata, search results, and
+channel listings — without the official Data API and without an API key. It is a
+Rust port of [NewPipeExtractor](https://github.com/TeamNewPipe/NewPipeExtractor)
+(tracking v0.26.2), scoped to YouTube.

-`rustypipe` regex-parses YouTube's `player.js` and reimplements the signature deobfuscator in Rust — every YT player rotation breaks it. NPE embeds a JS engine and executes the function live, which survives rotations. strawcore mirrors that architecture on QuickJS via rquickjs.
+strawcore speaks YouTube's internal **InnerTube** endpoints directly and resolves
+the playback-URL signature the same way NewPipeExtractor does: by running
+YouTube's own `player.js` in an embedded JavaScript engine. The crate ships as a
+plain Rust `rlib`, so it can be embedded in a desktop app, a server, a CLI, or
+bridged into a mobile app over FFI.

-## Build + test
+## Why an embedded JS engine
+
+Most lightweight extractors regex-scrape `player.js` and re-implement the
+signature/`n`-parameter deobfuscator in their host language. That re-implementation
+breaks every time YouTube rotates the player. NewPipeExtractor instead embeds a JS
+engine and *executes* the deobfuscation function live, which survives rotations.
+strawcore mirrors that architecture using [QuickJS](https://bellard.org/quickjs/)
+via [`rquickjs`](https://crates.io/crates/rquickjs), so the same resilience carries
+over to Rust.
+
+## Status
+
+Early (`0.1`). YouTube only. The public surface covers:
+
+- **Streams** — resolve a video ID to its audio/video stream URLs, formats, and metadata.
+- **Search** — query with an optional content-type filter, plus continuation (pagination).
+- **Channels** — resolve a channel ID / `@handle` / legacy URL to channel info and its video list, plus continuation.
+- **Localization** — preferred language + content country are honored on InnerTube requests.
+
+## Installation
+
+Not yet published to crates.io. Add it as a git dependency:
+
+```toml
+[dependencies]
+strawcore-core = { git = "https://git.sulkta.com/Sulkta-OSS/strawcore" }
+```
+
+The library crate is imported as `strawcore_core`.
+
+## Quick start
+
+You provide an HTTP client by implementing the `Downloader` trait, or use the
+bundled `reqwest`-based default. Install it once into the process-global service
+singleton, then call the extractors.
+
+```rust
+use std::sync::Arc;
+use strawcore_core::downloader::ReqwestDownloader;
+use strawcore_core::youtube::{stream_extractor, search_extractor, channel};
+use strawcore_core::youtube::linkhandler::search::SearchFilter;
+use strawcore_core::youtube::linkhandler::channel::ChannelIdentifier;
+use strawcore_core::NewPipe;
+
+fn main() -> Result<(), Box<dyn std::error::Error>> {
+    // Install a downloader (here, the bundled reqwest client).
+    NewPipe::init(Arc::new(ReqwestDownloader::new()?));
+
+    // Resolve a video's streams + metadata.
+    let info = stream_extractor::stream_info("dQw4w9WgXcQ")?;
+    println!("{} — {} streams", info.name, info.video_streams.len());
+
+    // Search for videos.
+    let results = search_extractor::search("rust programming", SearchFilter::Videos)?;
+    println!("{} results", results.videos.len());
+
+    // Look up a channel by @handle.
+    let ch = channel::channel_info(ChannelIdentifier::Handle("@rustlang".into()))?;
+    println!("channel: {}", ch.name);
+
+    Ok(())
+}
+```
+
+### Bring your own HTTP client
+
+`Downloader` is a single-method trait (`execute(Request) -> Result<Response, _>`).
+Implement it over whatever HTTP stack you already have and pass your instance to
+`NewPipe::init`, instead of the bundled `ReqwestDownloader`.
+
+### PoTokens
+
+Some YouTube requests can require a proof-of-origin token (`po_token`), which is
+minted by YouTube's BotGuard challenge and needs a real browser/WebView to solve.
+strawcore does not mint these itself; it exposes a `PoTokenProvider` trait so an
+embedder can supply one (for example by driving a headless browser). The default
+provider declines, which is fine for the code paths that do not require a token.
+
+## Build and test

 ```bash
 cargo build
-cargo test --lib                          # offline unit tests
-cargo test --features online-tests        # full smoke incl. live httpbin.org
+cargo test --lib                       # offline unit tests
+cargo test --features online-tests     # also runs tests that hit the network
 ```

+Network-dependent integration tests are gated behind the `online-tests` feature
+so offline and CI runs stay green.
+
 ## License

-GPL-3.0-or-later. NPE is GPL-3.0; this port inherits.
+[GPL-3.0-or-later](LICENSE). NewPipeExtractor is GPL-3.0-licensed; this port
+inherits that license.
+
+## Acknowledgements
+
+strawcore is a port of [NewPipeExtractor](https://github.com/TeamNewPipe/NewPipeExtractor)
+by the NewPipe team, and follows its extraction architecture closely. All credit
+for the original design and the hard-won InnerTube/player reverse-engineering goes
+to that project.
+
+## Contributing
+
+Issues and pull requests are welcome. Please keep changes focused, run
+`cargo fmt`, `cargo clippy`, and `cargo test --lib` before opening a PR, and add
+tests for new extraction logic where practical.