M0 docs — lock three-tier resolve + upstream-contribution posture
Cobb wants rustypipe primary, yt-dlp fallback, and rip-to-temp as the last-resort path when streams die mid-play. README expanded to spell out all three tiers + adds the 'fight YouTube alongside the FOSS ecosystem' framing. MILESTONES M1 rewritten to cover all three tiers. New file docs/upstream.md tracks every PR we file against rustypipe / NPE / yt-dlp with honest outcomes. Opens empty; fills as M1+ surface real bugs to fix.
This commit is contained in:
parent
238dfb8391
commit
76bd1d970e
3 changed files with 82 additions and 16 deletions
|
|
@ -7,11 +7,13 @@
|
|||
- [x] GPL-3.0 license headers
|
||||
- [ ] crafting-table build target produces a static aarch64 sidecar binary
|
||||
|
||||
## M1 — sidecar resolve
|
||||
## M1 — sidecar resolve (three-tier)
|
||||
|
||||
- [ ] reads `{"op":"resolve","id":"<yt-id>"}` from stdin
|
||||
- [ ] calls rustypipe → returns `{"streams":[…],"title":"…","duration_s":N}`
|
||||
- [ ] handles age-restricted + region-restricted as typed errors, not panics
|
||||
- [ ] **Tier 1:** rustypipe → `{"streams":[…],"title":"…","duration_s":N,"source":"rustypipe"}`
|
||||
- [ ] **Tier 2:** on Tier-1 failure, shell out to `yt-dlp -j <url>` → same JSON shape with `"source":"yt-dlp"`
|
||||
- [ ] **Tier 3:** new op `{"op":"rip","id":"<yt-id>","dest":"/storage/.kodi/temp/torttube/<id>.<ext>"}` invoked by addon on Tier-1+2 stream failures or 403-mid-play; yt-dlp downloads file, sidecar returns local path
|
||||
- [ ] typed errors for age-restricted / region-restricted / private (not panics)
|
||||
- [ ] sig decoding verified against a known-good video
|
||||
- [ ] DASH manifest URL or per-itag direct stream URL — whichever inputstream.adaptive prefers
|
||||
|
||||
|
|
@ -49,10 +51,16 @@
|
|||
- [ ] `addon.zip` ships with platform detect via `xbmc.getCondVisibility('system.platform.linux.raspberrypi')`
|
||||
- [ ] one-shot install path documented for LibreELEC `/storage/.kodi/`
|
||||
|
||||
## Upstream PR targets (parallel, opportunistic)
|
||||
## Upstream PR work (parallel lane — every bug evaluated for "fix it upstream?")
|
||||
|
||||
Pick one as we hit it organically — don't gate milestones on these.
|
||||
This isn't a separate milestone, it's a posture. Every sidecar bug we
|
||||
diagnose: ask "is this rustypipe / NPE / yt-dlp's bug?" — if yes, fix lands
|
||||
upstream too. Log every filed PR in [docs/upstream.md](docs/upstream.md).
|
||||
|
||||
- NPE #1357 (JDoc) — smallest, just to land a credible PR
|
||||
Opening shortlist:
|
||||
|
||||
- rustypipe PR #77 — review + help land
|
||||
- NPE #1357 (JDoc) — smallest credible PR, opens the relationship
|
||||
- NPE #1444 (typed errors) — clean signal/contract change
|
||||
- rustypipe — file issues + PRs on codeberg as we hit gaps
|
||||
- NPE #1360 (refactor link handlers, "help wanted")
|
||||
- rustypipe ↔ NPE port — anything one project has that the other lacks
|
||||
|
|
|
|||
43
README.md
43
README.md
|
|
@ -11,32 +11,57 @@ required account-linking for the upstream addon.
|
|||
```
|
||||
Kodi (LibreELEC, RPi)
|
||||
└── plugin.video.torttube [Python addon — UI, browse, settings]
|
||||
└── torttube-sidecar [Rust binary — Innertube extraction,
|
||||
SponsorBlock fetch, JSON-over-stdio]
|
||||
├── rustypipe [YouTube Innertube client + sig decode]
|
||||
└── sponsorblock [REST API client, SHA-256 prefix lookup]
|
||||
└── torttube-sidecar [Rust binary — JSON-over-stdio]
|
||||
├── rustypipe [Tier 1: native Rust Innertube]
|
||||
├── yt-dlp subprocess [Tier 2: fallback resolve]
|
||||
├── yt-dlp rip-to-temp [Tier 3: download to /storage/.kodi/temp,
|
||||
│ play local file when streams die mid-play]
|
||||
└── sponsorblock [REST client, SHA-256 prefix lookup]
|
||||
```
|
||||
|
||||
Kodi addons are Python — the engine layer (n-param sig decoding, Innertube,
|
||||
SponsorBlock hashing) lives in a Rust sidecar so we get a single maintained
|
||||
extraction surface and clean aarch64/armv7 cross-compiles.
|
||||
|
||||
**Three-tier resolve** because YouTube actively fights every extractor:
|
||||
|
||||
1. **rustypipe (Rust)** — preferred. Fast, in-process, no Python dep on the RPi.
|
||||
2. **yt-dlp subprocess** — fallback when rustypipe sig-decoding falls behind YouTube's deobfuscator changes. yt-dlp updates weekly; we shell out, parse `-j` JSON.
|
||||
3. **Rip-to-temp** — last resort when stream URLs 403 mid-playback (poToken expiry, cookie session mismatch). yt-dlp downloads to `/storage/.kodi/temp/torttube/<id>.<ext>`, Kodi plays the local file. Temp dir has size cap + age cleanup.
|
||||
|
||||
## Status
|
||||
|
||||
M0 scaffold. Nothing playable yet — see [MILESTONES.md](MILESTONES.md).
|
||||
|
||||
## Upstream contributions
|
||||
## Upstream — we fight with the FOSS extractor ecosystem, not next to it
|
||||
|
||||
NewPipeExtractor (Java) is the canonical reference for Innertube behaviour.
|
||||
When we hit a corner case RustyPipe doesn't cover we look at NPE for the
|
||||
fix, then either port it to RustyPipe (Rust PR) or fix NPE directly (Java
|
||||
PR). Either way upstream lands a real improvement.
|
||||
YouTube's anti-scraping changes hit every extractor: NewPipe, yt-dlp, Invidious,
|
||||
rustypipe. Every fix we make in our sidecar gets evaluated for "is this
|
||||
upstreamable?" — if yes, the fix lands at the upstream project, not just here.
|
||||
|
||||
Active lanes:
|
||||
|
||||
- **rustypipe** (Rust, codeberg.org/ThetaDev/rustypipe) — maintenance has slowed.
|
||||
Open PR #77 "Some fixes" is unmerged as of 2026-05-23. We will either help land
|
||||
it (review + ping maintainer) or fork to `Sulkta-Coop/rustypipe` if upstream
|
||||
stays quiet. Forking is the worst case, not the first move.
|
||||
- **NewPipeExtractor** (Java, github.com/TeamNewPipe/NewPipeExtractor) — actively
|
||||
maintained, 177 open issues. We use it as the reference implementation for
|
||||
Innertube behaviour. PRs to NPE land in Rust here via rustypipe, and vice
|
||||
versa.
|
||||
- **yt-dlp** (Python, github.com/yt-dlp/yt-dlp) — the gold standard. We're more
|
||||
consumers than contributors here, but if our rip-to-temp tier surfaces a
|
||||
specific extractor bug we file it.
|
||||
|
||||
Issues we're watching:
|
||||
- [NPE #1339](https://github.com/TeamNewPipe/NewPipeExtractor/issues/1339) — n-parameter deobfuscation
|
||||
- [NPE #1444](https://github.com/TeamNewPipe/NewPipeExtractor/issues/1444) — distinguish unavailable vs unextractable
|
||||
- [NPE #1360](https://github.com/TeamNewPipe/NewPipeExtractor/issues/1360) — refactor link handlers (help wanted)
|
||||
- [NPE #1357](https://github.com/TeamNewPipe/NewPipeExtractor/issues/1357) — JDoc checks in PR pipeline (good first issue)
|
||||
- [rustypipe PR #77](https://codeberg.org/ThetaDev/rustypipe/pulls/77) — open as of 2026-05-23, unmerged
|
||||
|
||||
Contribution log lives at [docs/upstream.md](docs/upstream.md) — every PR we
|
||||
file lands there with its outcome.
|
||||
|
||||
## License
|
||||
|
||||
|
|
|
|||
33
docs/upstream.md
Normal file
33
docs/upstream.md
Normal file
|
|
@ -0,0 +1,33 @@
|
|||
# Upstream contributions log
|
||||
|
||||
One row per PR/issue we file. Outcomes recorded honestly — merged, closed,
|
||||
abandoned, whatever happened.
|
||||
|
||||
## Active
|
||||
|
||||
_(none yet — opens with M1 development)_
|
||||
|
||||
## Filed
|
||||
|
||||
| Date | Project | PR/Issue | Title | Status | Outcome |
|
||||
|------|---------|----------|-------|--------|---------|
|
||||
| _empty_ | | | | | |
|
||||
|
||||
## Watching (not ours, but relevant to torttube)
|
||||
|
||||
| Project | PR/Issue | Title | Why we care |
|
||||
|---------|----------|-------|-------------|
|
||||
| rustypipe | [PR #77](https://codeberg.org/ThetaDev/rustypipe/pulls/77) | "Some fixes" | Open 2026-05-23, unmerged. If maintainer stays quiet we may need to help land it or fork. |
|
||||
| NPE | [#1339](https://github.com/TeamNewPipe/NewPipeExtractor/issues/1339) | n-parameter deobfuscation broken | Core to playback. Fix here = fix in our Tier-1. |
|
||||
| NPE | [#1444](https://github.com/TeamNewPipe/NewPipeExtractor/issues/1444) | Distinguish unavailable vs unextractable | Clean typed-error PR target. |
|
||||
| NPE | [#1360](https://github.com/TeamNewPipe/NewPipeExtractor/issues/1360) | Refactor link handlers | "help wanted" label. |
|
||||
| NPE | [#1357](https://github.com/TeamNewPipe/NewPipeExtractor/issues/1357) | JDoc checks in PR pipeline | "good first issue" — smallest credible first PR. |
|
||||
|
||||
## Posture
|
||||
|
||||
- Every sidecar bug we trace ends with one question: **is the root cause
|
||||
in rustypipe / NPE / yt-dlp?** If yes, fix it there first; pull the fix
|
||||
into our sidecar via dep bump or backport.
|
||||
- We are not pretending to be on the rustypipe / NPE / yt-dlp maintainer
|
||||
level. We're a downstream consumer that does its share.
|
||||
- A merged PR > a forked patch. A fork is the last resort, not the first move.
|
||||
Loading…
Add table
Add a link
Reference in a new issue