M0 docs — lock three-tier resolve + upstream-contribution posture
The design calls for rustypipe primary, yt-dlp fallback, and rip-to-temp as the last-resort path when streams die mid-play. README expanded to spell out all three tiers + adds the 'fight YouTube alongside the FOSS ecosystem' framing. MILESTONES M1 rewritten to cover all three tiers. New file docs/upstream.md tracks every PR we file against rustypipe / NPE / yt-dlp with honest outcomes. Opens empty; fills as M1+ surface real bugs to fix.
This commit is contained in:
Sulkta
parent
053a4bbcc5
commit
5abca77814
3 changed files with 82 additions and 16 deletions
43
README.md
43
README.md
|
|
@ -11,32 +11,57 @@ required account-linking for the upstream addon.
|
|||
```
|
||||
Kodi (LibreELEC, RPi)
|
||||
└── plugin.video.torttube [Python addon — UI, browse, settings]
|
||||
└── torttube-sidecar [Rust binary — Innertube extraction,
|
||||
SponsorBlock fetch, JSON-over-stdio]
|
||||
├── rustypipe [YouTube Innertube client + sig decode]
|
||||
└── sponsorblock [REST API client, SHA-256 prefix lookup]
|
||||
└── torttube-sidecar [Rust binary — JSON-over-stdio]
|
||||
├── rustypipe [Tier 1: native Rust Innertube]
|
||||
├── yt-dlp subprocess [Tier 2: fallback resolve]
|
||||
├── yt-dlp rip-to-temp [Tier 3: download to /storage/.kodi/temp,
|
||||
│ play local file when streams die mid-play]
|
||||
└── sponsorblock [REST client, SHA-256 prefix lookup]
|
||||
```
|
||||
|
||||
Kodi addons are Python — the engine layer (n-param sig decoding, Innertube,
|
||||
SponsorBlock hashing) lives in a Rust sidecar so we get a single maintained
|
||||
extraction surface and clean aarch64/armv7 cross-compiles.
|
||||
|
||||
**Three-tier resolve** because YouTube actively fights every extractor:
|
||||
|
||||
1. **rustypipe (Rust)** — preferred. Fast, in-process, no Python dep on the RPi.
|
||||
2. **yt-dlp subprocess** — fallback when rustypipe sig-decoding falls behind YouTube's deobfuscator changes. yt-dlp updates weekly; we shell out, parse `-j` JSON.
|
||||
3. **Rip-to-temp** — last resort when stream URLs 403 mid-playback (poToken expiry, cookie session mismatch). yt-dlp downloads to `/storage/.kodi/temp/torttube/<id>.<ext>`, Kodi plays the local file. Temp dir has size cap + age cleanup.
|
||||
|
||||
## Status
|
||||
|
||||
M0 scaffold. Nothing playable yet — see [MILESTONES.md](MILESTONES.md).
|
||||
|
||||
## Upstream contributions
|
||||
## Upstream — we fight with the FOSS extractor ecosystem, not next to it
|
||||
|
||||
NewPipeExtractor (Java) is the canonical reference for Innertube behaviour.
|
||||
When we hit a corner case RustyPipe doesn't cover we look at NPE for the
|
||||
fix, then either port it to RustyPipe (Rust PR) or fix NPE directly (Java
|
||||
PR). Either way upstream lands a real improvement.
|
||||
YouTube's anti-scraping changes hit every extractor: NewPipe, yt-dlp, Invidious,
|
||||
rustypipe. Every fix we make in our sidecar gets evaluated for "is this
|
||||
upstreamable?" — if yes, the fix lands at the upstream project, not just here.
|
||||
|
||||
Active lanes:
|
||||
|
||||
- **rustypipe** (Rust, codeberg.org/ThetaDev/rustypipe) — maintenance has slowed.
|
||||
Open PR #77 "Some fixes" is unmerged as of 2026-05-23. We will either help land
|
||||
it (review + ping maintainer) or fork to `your own fork` if upstream
|
||||
stays quiet. Forking is the worst case, not the first move.
|
||||
- **NewPipeExtractor** (Java, github.com/TeamNewPipe/NewPipeExtractor) — actively
|
||||
maintained, 177 open issues. We use it as the reference implementation for
|
||||
Innertube behaviour. PRs to NPE land in Rust here via rustypipe, and vice
|
||||
versa.
|
||||
- **yt-dlp** (Python, github.com/yt-dlp/yt-dlp) — the gold standard. We're more
|
||||
consumers than contributors here, but if our rip-to-temp tier surfaces a
|
||||
specific extractor bug we file it.
|
||||
|
||||
Issues we're watching:
|
||||
- [NPE #1339](https://github.com/TeamNewPipe/NewPipeExtractor/issues/1339) — n-parameter deobfuscation
|
||||
- [NPE #1444](https://github.com/TeamNewPipe/NewPipeExtractor/issues/1444) — distinguish unavailable vs unextractable
|
||||
- [NPE #1360](https://github.com/TeamNewPipe/NewPipeExtractor/issues/1360) — refactor link handlers (help wanted)
|
||||
- [NPE #1357](https://github.com/TeamNewPipe/NewPipeExtractor/issues/1357) — JDoc checks in PR pipeline (good first issue)
|
||||
- [rustypipe PR #77](https://codeberg.org/ThetaDev/rustypipe/pulls/77) — open as of 2026-05-23, unmerged
|
||||
|
||||
Contribution log lives at [docs/upstream.md](docs/upstream.md) — every PR we
|
||||
file lands there with its outcome.
|
||||
|
||||
## License
|
||||
|
||||
|
|
|
|||
Reference in a new issue