Sulkta-Coop/torttube
2
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

M0 docs — lock three-tier resolve + upstream-contribution posture

Browse source 
Cobb wants rustypipe primary, yt-dlp fallback, and rip-to-temp as the
last-resort path when streams die mid-play. README expanded to spell
out all three tiers + adds the 'fight YouTube alongside the FOSS
ecosystem' framing. MILESTONES M1 rewritten to cover all three tiers.

New file docs/upstream.md tracks every PR we file against rustypipe /
NPE / yt-dlp with honest outcomes. Opens empty; fills as M1+ surface
real bugs to fix.
This commit is contained in:
Kayos 2026-05-23 08:20:50 -07:00
parent 238dfb8391
commit 76bd1d970e
3 changed files with 82 additions and 16 deletions
Download patch file Download diff file Expand all files Collapse all files

22
MILESTONES.md
View file

@ -7,11 +7,13 @@
- [x] GPL-3.0 license headers - [x] GPL-3.0 license headers
- [ ] crafting-table build target produces a static aarch64 sidecar binary - [ ] crafting-table build target produces a static aarch64 sidecar binary
## M1 — sidecar resolve ## M1 — sidecar resolve (three-tier)
- [ ] reads `{"op":"resolve","id":"<yt-id>"}` from stdin - [ ] reads `{"op":"resolve","id":"<yt-id>"}` from stdin
- [ ] calls rustypipe → returns `{"streams":[…],"title":"…","duration_s":N}` - [ ] **Tier 1:** rustypipe → `{"streams":[…],"title":"…","duration_s":N,"source":"rustypipe"}`
- [ ] handles age-restricted + region-restricted as typed errors, not panics - [ ] **Tier 2:** on Tier-1 failure, shell out to `yt-dlp -j <url>` → same JSON shape with `"source":"yt-dlp"`
- [ ] **Tier 3:** new op `{"op":"rip","id":"<yt-id>","dest":"/storage/.kodi/temp/torttube/<id>.<ext>"}` invoked by addon on Tier-1+2 stream failures or 403-mid-play; yt-dlp downloads file, sidecar returns local path
- [ ] typed errors for age-restricted / region-restricted / private (not panics)
- [ ] sig decoding verified against a known-good video - [ ] sig decoding verified against a known-good video
- [ ] DASH manifest URL or per-itag direct stream URL — whichever inputstream.adaptive prefers - [ ] DASH manifest URL or per-itag direct stream URL — whichever inputstream.adaptive prefers
@ -49,10 +51,16 @@
- [ ] `addon.zip` ships with platform detect via `xbmc.getCondVisibility('system.platform.linux.raspberrypi')` - [ ] `addon.zip` ships with platform detect via `xbmc.getCondVisibility('system.platform.linux.raspberrypi')`
- [ ] one-shot install path documented for LibreELEC `/storage/.kodi/` - [ ] one-shot install path documented for LibreELEC `/storage/.kodi/`
## Upstream PR targets (parallel, opportunistic) ## Upstream PR work (parallel lane — every bug evaluated for "fix it upstream?")
Pick one as we hit it organically — don't gate milestones on these. This isn't a separate milestone, it's a posture. Every sidecar bug we
diagnose: ask "is this rustypipe / NPE / yt-dlp's bug?" — if yes, fix lands
upstream too. Log every filed PR in [docs/upstream.md](docs/upstream.md).
- NPE #1357 (JDoc) — smallest, just to land a credible PR Opening shortlist:
- rustypipe PR #77 — review + help land
- NPE #1357 (JDoc) — smallest credible PR, opens the relationship
- NPE #1444 (typed errors) — clean signal/contract change - NPE #1444 (typed errors) — clean signal/contract change
- rustypipe — file issues + PRs on codeberg as we hit gaps - NPE #1360 (refactor link handlers, "help wanted")
- rustypipe ↔ NPE port — anything one project has that the other lacks

43
README.md
View file

@ -11,32 +11,57 @@ required account-linking for the upstream addon.
``` ```
Kodi (LibreELEC, RPi) Kodi (LibreELEC, RPi)
└── plugin.video.torttube [Python addon — UI, browse, settings] └── plugin.video.torttube [Python addon — UI, browse, settings]
└── torttube-sidecar [Rust binary — Innertube extraction, └── torttube-sidecar [Rust binary — JSON-over-stdio]
SponsorBlock fetch, JSON-over-stdio] ├── rustypipe [Tier 1: native Rust Innertube]
├── rustypipe [YouTube Innertube client + sig decode] ├── yt-dlp subprocess [Tier 2: fallback resolve]
└── sponsorblock [REST API client, SHA-256 prefix lookup] ├── yt-dlp rip-to-temp [Tier 3: download to /storage/.kodi/temp,
│ play local file when streams die mid-play]
└── sponsorblock [REST client, SHA-256 prefix lookup]
``` ```
Kodi addons are Python — the engine layer (n-param sig decoding, Innertube, Kodi addons are Python — the engine layer (n-param sig decoding, Innertube,
SponsorBlock hashing) lives in a Rust sidecar so we get a single maintained SponsorBlock hashing) lives in a Rust sidecar so we get a single maintained
extraction surface and clean aarch64/armv7 cross-compiles. extraction surface and clean aarch64/armv7 cross-compiles.
**Three-tier resolve** because YouTube actively fights every extractor:
1. **rustypipe (Rust)** — preferred. Fast, in-process, no Python dep on the RPi.
2. **yt-dlp subprocess** — fallback when rustypipe sig-decoding falls behind YouTube's deobfuscator changes. yt-dlp updates weekly; we shell out, parse `-j` JSON.
3. **Rip-to-temp** — last resort when stream URLs 403 mid-playback (poToken expiry, cookie session mismatch). yt-dlp downloads to `/storage/.kodi/temp/torttube/<id>.<ext>`, Kodi plays the local file. Temp dir has size cap + age cleanup.
## Status ## Status
M0 scaffold. Nothing playable yet — see [MILESTONES.md](MILESTONES.md). M0 scaffold. Nothing playable yet — see [MILESTONES.md](MILESTONES.md).
## Upstream contributions ## Upstream — we fight with the FOSS extractor ecosystem, not next to it
NewPipeExtractor (Java) is the canonical reference for Innertube behaviour. YouTube's anti-scraping changes hit every extractor: NewPipe, yt-dlp, Invidious,
When we hit a corner case RustyPipe doesn't cover we look at NPE for the rustypipe. Every fix we make in our sidecar gets evaluated for "is this
fix, then either port it to RustyPipe (Rust PR) or fix NPE directly (Java upstreamable?" — if yes, the fix lands at the upstream project, not just here.
PR). Either way upstream lands a real improvement.
Active lanes:
- **rustypipe** (Rust, codeberg.org/ThetaDev/rustypipe) — maintenance has slowed.
Open PR #77 "Some fixes" is unmerged as of 2026-05-23. We will either help land
it (review + ping maintainer) or fork to `Sulkta-Coop/rustypipe` if upstream
stays quiet. Forking is the worst case, not the first move.
- **NewPipeExtractor** (Java, github.com/TeamNewPipe/NewPipeExtractor) — actively
maintained, 177 open issues. We use it as the reference implementation for
Innertube behaviour. PRs to NPE land in Rust here via rustypipe, and vice
versa.
- **yt-dlp** (Python, github.com/yt-dlp/yt-dlp) — the gold standard. We're more
consumers than contributors here, but if our rip-to-temp tier surfaces a
specific extractor bug we file it.
Issues we're watching: Issues we're watching:
- [NPE #1339](https://github.com/TeamNewPipe/NewPipeExtractor/issues/1339) — n-parameter deobfuscation - [NPE #1339](https://github.com/TeamNewPipe/NewPipeExtractor/issues/1339) — n-parameter deobfuscation
- [NPE #1444](https://github.com/TeamNewPipe/NewPipeExtractor/issues/1444) — distinguish unavailable vs unextractable - [NPE #1444](https://github.com/TeamNewPipe/NewPipeExtractor/issues/1444) — distinguish unavailable vs unextractable
- [NPE #1360](https://github.com/TeamNewPipe/NewPipeExtractor/issues/1360) — refactor link handlers (help wanted) - [NPE #1360](https://github.com/TeamNewPipe/NewPipeExtractor/issues/1360) — refactor link handlers (help wanted)
- [NPE #1357](https://github.com/TeamNewPipe/NewPipeExtractor/issues/1357) — JDoc checks in PR pipeline (good first issue) - [NPE #1357](https://github.com/TeamNewPipe/NewPipeExtractor/issues/1357) — JDoc checks in PR pipeline (good first issue)
- [rustypipe PR #77](https://codeberg.org/ThetaDev/rustypipe/pulls/77) — open as of 2026-05-23, unmerged
Contribution log lives at [docs/upstream.md](docs/upstream.md) — every PR we
file lands there with its outcome.
## License ## License

33
docs/upstream.md Normal file
View file

@ -0,0 +1,33 @@
# Upstream contributions log
One row per PR/issue we file. Outcomes recorded honestly — merged, closed,
abandoned, whatever happened.
## Active
_(none yet — opens with M1 development)_
## Filed
| Date | Project | PR/Issue | Title | Status | Outcome |
|------|---------|----------|-------|--------|---------|
| _empty_ | | | | | |
## Watching (not ours, but relevant to torttube)
| Project | PR/Issue | Title | Why we care |
|---------|----------|-------|-------------|
| rustypipe | [PR #77](https://codeberg.org/ThetaDev/rustypipe/pulls/77) | "Some fixes" | Open 2026-05-23, unmerged. If maintainer stays quiet we may need to help land it or fork. |
| NPE | [#1339](https://github.com/TeamNewPipe/NewPipeExtractor/issues/1339) | n-parameter deobfuscation broken | Core to playback. Fix here = fix in our Tier-1. |
| NPE | [#1444](https://github.com/TeamNewPipe/NewPipeExtractor/issues/1444) | Distinguish unavailable vs unextractable | Clean typed-error PR target. |
| NPE | [#1360](https://github.com/TeamNewPipe/NewPipeExtractor/issues/1360) | Refactor link handlers | "help wanted" label. |
| NPE | [#1357](https://github.com/TeamNewPipe/NewPipeExtractor/issues/1357) | JDoc checks in PR pipeline | "good first issue" — smallest credible first PR. |
## Posture
- Every sidecar bug we trace ends with one question: **is the root cause
in rustypipe / NPE / yt-dlp?** If yes, fix it there first; pull the fix
into our sidecar via dep bump or backport.
- We are not pretending to be on the rustypipe / NPE / yt-dlp maintainer
level. We're a downstream consumer that does its share.
- A merged PR > a forked patch. A fork is the last resort, not the first move.