Cobb 932e267c77 channel: fetch and parse a channel's Live tab

channel_live(channel_id) fetches the first page of a channel's Live tab
with one browse POST (the Videos tab request with the Live tab params)
and channel_live_continuation(token) the pages after it with one POST
each. Each item keeps its title, URL and thumbnails and gets a status:
live now (with its "N watching" line), scheduled (its "N waiting" line,
its start as worded, and the start in Unix seconds when the request was
en-GB), or past (duration, views and "Streamed ..." age). Items come in
the order the site gives.

The status is read the same way in every language measured (en-GB,
en-US, de, fr): the LIVE badge style, then a duration-shaped badge
(hours may carry digit grouping: "20,772", "20.772", or a narrow
no-break space in fr), then any other badge. Only a selected tab whose
URL ends in /streams is read, because a channel without a Live tab
answers with its Home tab.

Crossed: four channel.rs helpers become pub(crate) so the new module can
reuse the browse requests and the lockup parse; nothing else in
channel.rs changes. Nothing existing is removed or changed in behaviour.

2026-10-23 09:41:06 -07:00

4.7 KiB

Raw Permalink Blame History

strawcore

A Rust library for extracting YouTube stream URLs, metadata, search results, and channel listings — without the official Data API and without an API key. It is a Rust port of NewPipeExtractor (tracking v0.26.2), scoped to YouTube.

strawcore speaks YouTube's internal InnerTube endpoints directly and resolves the playback-URL signature the same way NewPipeExtractor does: by running YouTube's own player.js in an embedded JavaScript engine. The crate ships as a plain Rust rlib, so it can be embedded in a desktop app, a server, a CLI, or bridged into a mobile app over FFI.

Why an embedded JS engine

Most lightweight extractors regex-scrape player.js and re-implement the signature/n-parameter deobfuscator in their host language. That re-implementation breaks every time YouTube rotates the player. NewPipeExtractor instead embeds a JS engine and executes the deobfuscation function live, which survives rotations. strawcore mirrors that architecture using QuickJS via rquickjs, so the same resilience carries over to Rust.

Status

Early (0.1). YouTube only. The public surface covers:

Streams — resolve a video ID to its audio/video stream URLs, formats, and metadata.
Search — query with an optional content-type filter, plus continuation (pagination).
Channels — resolve a channel ID / @handle / legacy URL to channel info and its video list, plus continuation.
Live tab: a channel's streams that are live now, scheduled or over, plus continuation, and whether a channel has a Live tab at all.
Localization — preferred language + content country are honored on InnerTube requests.

Installation

Not yet published to crates.io. Add it as a git dependency:

[dependencies]
strawcore-core = { git = "https://git.sulkta.com/Sulkta-OSS/strawcore" }

The library crate is imported as strawcore_core.

Quick start

You provide an HTTP client by implementing the Downloader trait, or use the bundled reqwest-based default. Install it once into the process-global service singleton, then call the extractors.

use std::sync::Arc;
use strawcore_core::downloader::ReqwestDownloader;
use strawcore_core::youtube::{stream_extractor, search_extractor, channel};
use strawcore_core::youtube::linkhandler::search::SearchFilter;
use strawcore_core::youtube::linkhandler::channel::ChannelIdentifier;
use strawcore_core::NewPipe;

fn main() -> Result<(), Box<dyn std::error::Error>> {
    // Install a downloader (here, the bundled reqwest client).
    NewPipe::init(Arc::new(ReqwestDownloader::new()?));

    // Resolve a video's streams + metadata.
    let info = stream_extractor::stream_info("dQw4w9WgXcQ")?;
    println!("{} — {} streams", info.name, info.video_streams.len());

    // Search for videos.
    let results = search_extractor::search("rust programming", SearchFilter::Videos)?;
    println!("{} results", results.videos.len());

    // Look up a channel by @handle.
    let ch = channel::channel_info(ChannelIdentifier::Handle("@rustlang".into()))?;
    println!("channel: {}", ch.name);

    Ok(())
}

Bring your own HTTP client

Downloader is a single-method trait (execute(Request) -> Result<Response, _>). Implement it over whatever HTTP stack you already have and pass your instance to NewPipe::init, instead of the bundled ReqwestDownloader.

PoTokens

Some YouTube requests can require a proof-of-origin token (po_token), which is minted by YouTube's BotGuard challenge and needs a real browser/WebView to solve. strawcore does not mint these itself; it exposes a PoTokenProvider trait so an embedder can supply one (for example by driving a headless browser). The default provider declines, which is fine for the code paths that do not require a token.

Build and test

cargo build
cargo test --lib                       # offline unit tests
cargo test --features online-tests     # also runs tests that hit the network

Network-dependent integration tests are gated behind the online-tests feature so offline and CI runs stay green.

License

GPL-3.0-or-later. NewPipeExtractor is GPL-3.0-licensed; this port inherits that license.

Acknowledgements

strawcore is a port of NewPipeExtractor by the NewPipe team, and follows its extraction architecture closely. All credit for the original design and the hard-won InnerTube/player reverse-engineering goes to that project.

Contributing

Issues and pull requests are welcome. Please keep changes focused, run cargo fmt, cargo clippy, and cargo test --lib before opening a PR, and add tests for new extraction logic where practical.

4.7 KiB Raw Permalink Blame History