From 0a289fea3ab95ef937fb5331dd9835fb9c6ce7ab Mon Sep 17 00:00:00 2001 From: Kayos Date: Sat, 23 May 2026 11:46:56 -0700 Subject: [PATCH] =?UTF-8?q?M7=20=E2=80=94=20DASH=20partial:=20MPD=20serves?= =?UTF-8?q?=20OK,=20segment=20alignment=20still=20WIP?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Big strides today: - Sidecar resolve_dash op works (verified live on Pi) - MPD builder generates valid MPEG-DASH on-demand manifest with H.264 720p/1080p video reps + best AAC audio rep - ThreadingHTTPServer serves the MPD over the LAN IP (not 127.0.0.1 — curl in Kodi 20's inputstream.adaptive can't open that) - inputstream.adaptive PARSES our manifest cleanly: 'Successfully parsed manifest file (Periods: 1, Streams in first period: 2)' - Segment GETs work once we set stream_headers with User-Agent + Origin + Referer (otherwise googlevideo 403s the audio segments) Remaining issue: - Audio drifts -25s → -44s behind video within seconds of playback start. inputstream.adaptive needs explicit SegmentTimeline timing derived from each rep's sidx box to stay aligned. Plugin.video.youtube does this; we'd need to fetch+parse sidx ourselves or fork their MPD-builder. Documented as M7-blocking + upstream PR candidate. Default remains the stable yt-dlp progressive 360p path. DASH is behind dash_enabled setting OR a dash.on marker file in addon_data. Toggle on via: ssh 'touch /storage/.kodi/userdata/addon_data/plugin.video.torttube/dash.on' Toggle off: ssh 'rm /storage/.kodi/userdata/addon_data/plugin.video.torttube/dash.on' Addon v0.0.10. docs/upstream.md has the full segment-timing analysis. --- MILESTONES.md | 35 +++++++---- addon/plugin.video.torttube/addon.xml | 2 +- addon/plugin.video.torttube/main.py | 86 ++++++++++++++++++++++----- docs/upstream.md | 25 ++++++-- 4 files changed, 117 insertions(+), 31 deletions(-) diff --git a/MILESTONES.md b/MILESTONES.md index ac5f6ac..14991e1 100644 --- a/MILESTONES.md +++ b/MILESTONES.md @@ -86,21 +86,34 @@ progressive) is deprecated by YouTube; higher quality needs DASH manifest generation. Code path exists, gated behind `TORTTUBE_DASH=1`. -## M7 — DASH / HD playback [WIP] +## M7 — DASH / HD playback [WIP — close on segment timing] - [x] sidecar `resolve_dash` op returns rustypipe's full `video_only_streams` + `audio_streams` arrays (16+ representations) - [x] addon `_build_dash_mpd` constructs valid on-demand MPD with H.264 - video reps from 360p → 1080p + best AAC audio rep (XML-escaped URLs, - proper `SegmentBase indexRange` + `Initialization range`) -- [ ] **serve MPD over localhost HTTP** — inputstream.adaptive's libcurl - can't open `file://` URLs. ThreadingHTTPServer + Player monitor - lifecycle is sketched but caused Kodi's "two concurrent busydialogs" - fatal during rapid retries. Needs more work on lifecycle + retry - backoff before re-enabling. -- [ ] handle session-cookie / poToken inheritance — googlevideo URLs may - need the same client signature across MPD + segment fetches -- [ ] graceful fallback to progressive if MPD load fails mid-playback + video reps (filtered 720p-1080p so inputstream.adaptive's conservative + chooser doesn't land below HD) + best AAC audio rep, XML-escaped URLs, + proper `SegmentBase indexRange` + `Initialization range` +- [x] **serve MPD over localhost HTTP** — ThreadingHTTPServer binds to LAN IP + (not 127.0.0.1 — that fails curl auth in Kodi 20). Lifecycle: server + stays up until SponsorBlockMonitor exits, then `server.shutdown()` in + finally block. **Verified live: inputstream.adaptive parses the MPD + cleanly**. +- [x] `inputstream.adaptive.stream_headers` set with `User-Agent`, `Origin`, + and `Referer` matching what rustypipe / yt-dlp use when minting the + URL — fixes the 403 Forbidden from googlevideo on segment GETs +- [ ] **segment timing alignment** — audio drifts -25 → -44s behind video + within seconds. Need explicit `` per-segment timing + derived from each representation's sidx box, OR `presentationTimeOffset`. + Plugin.video.youtube derives these from sidx — we'd need to fetch + parse + that. See `docs/upstream.md` for the upstream PR target. +- [x] graceful fallback to progressive — if the DASH path returns no MPD + bytes (rustypipe error, no H.264 reps, etc.) the addon falls through + to yt-dlp progressive 360p +- [x] **gated behind `dash_enabled` setting** (default off) and a + `dash.on` marker file in addon_data (workaround for Kodi's settings + cache, useful for ad-hoc testing). Stable 360p path remains the + default until segment timing is solved. ## Upstream PR work (parallel lane — every bug evaluated for "fix it upstream?") diff --git a/addon/plugin.video.torttube/addon.xml b/addon/plugin.video.torttube/addon.xml index e4f8846..87167b2 100644 --- a/addon/plugin.video.torttube/addon.xml +++ b/addon/plugin.video.torttube/addon.xml @@ -1,7 +1,7 @@ diff --git a/addon/plugin.video.torttube/main.py b/addon/plugin.video.torttube/main.py index a963c49..e2e1ad0 100644 --- a/addon/plugin.video.torttube/main.py +++ b/addon/plugin.video.torttube/main.py @@ -22,7 +22,7 @@ import http.server import json import os import re -import socketserver +import socket import subprocess import sys import threading @@ -112,6 +112,20 @@ def _resolved_listitem(stream_url: str, title: str | None) -> xbmcgui.ListItem: if ".mpd" in stream_url: li.setProperty("inputstream", "inputstream.adaptive") li.setProperty("inputstream.adaptive.manifest_type", "mpd") + # googlevideo rejects segment GETs that don't carry an Origin/Referer + # from www.youtube.com — 403 Forbidden otherwise. Same Mozilla UA + # rustypipe / yt-dlp use when minting the URL. + ytdl_ua = ( + "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 " + "(KHTML, like Gecko) Chrome/120.0.0.0 Safari/537.36" + ) + seg_headers = ( + f"User-Agent={ytdl_ua}" + "&Origin=https://www.youtube.com" + "&Referer=https://www.youtube.com/" + ) + li.setProperty("inputstream.adaptive.stream_headers", seg_headers) + li.setProperty("inputstream.adaptive.manifest_headers", seg_headers) elif ".m3u8" in stream_url: li.setProperty("inputstream", "inputstream.adaptive") li.setProperty("inputstream.adaptive.manifest_type", "hls") @@ -138,24 +152,40 @@ class _MpdHandler(http.server.BaseHTTPRequestHandler): return -def _start_mpd_server(mpd_bytes: bytes) -> tuple[str, http.server.HTTPServer]: - """Spin up a one-shot localhost HTTP server that serves `mpd_bytes` at any - path. Returns (url, server). Caller is responsible for `server.shutdown()` - once playback ends.""" +def _lan_ip() -> str: + """Detect this host's LAN IP by opening a UDP socket toward an external + address (no packets actually sent — just lets the kernel pick the source IP). + plugin.video.youtube uses this same trick because inputstream.adaptive's + libcurl in Kodi 20 has trouble fetching from `127.0.0.1` reliably.""" + s = socket.socket(socket.AF_INET, socket.SOCK_DGRAM) + try: + s.connect(("8.8.8.8", 80)) + return s.getsockname()[0] + except Exception: + return "127.0.0.1" + finally: + s.close() + + +def _start_mpd_server(mpd_bytes: bytes) -> tuple[str, http.server.HTTPServer]: + """Spin up a one-shot HTTP server that serves `mpd_bytes` at any path. + Binds to the LAN IP so inputstream.adaptive can fetch via the same code + path it uses for real network URLs. Returns (url, server) — caller is + responsible for `server.shutdown()` once playback ends.""" - # Per-server handler subclass so the closure captures the bytes for THIS - # video without leaking state to a concurrent invocation. handler_cls = type( "_MpdHandlerInstance", (_MpdHandler,), {"mpd_bytes": mpd_bytes}, ) - # Port 0 → kernel picks free port. ThreadingHTTPServer so multiple GETs - # (the manifest + range requests, if any) don't serialize. - server = http.server.ThreadingHTTPServer(("127.0.0.1", 0), handler_cls) + lan_ip = _lan_ip() + # Bind to all interfaces so the LAN-IP URL also routes through (otherwise + # connect-by-IP on the same host can return 'connection refused' on some + # kernel/firewall configs). Port 0 → kernel picks free port. + server = http.server.ThreadingHTTPServer(("0.0.0.0", 0), handler_cls) threading.Thread(target=server.serve_forever, daemon=True).start() port = server.server_address[1] - url = f"http://127.0.0.1:{port}/manifest.mpd" + url = f"http://{lan_ip}:{port}/manifest.mpd" _log(f"MPD HTTP server up on {url}") return url, server @@ -189,16 +219,30 @@ def _build_dash_mpd( if duration_s <= 0: return None - # Filter video: H.264 only (avc1.*), height <= 1080 (RPi 4 H.264 ceiling). + # Filter video: H.264 only (avc1.*), 720p <= height <= 1080p. + # Floor at 720p so inputstream.adaptive's conservative-start chooser doesn't + # land us on a low-quality rep first. Ceiling at 1080p because that's the + # RPi 4's H.264 hardware-decode sweet spot. h264 = [ s for s in video_streams if "avc1" in (s.get("codec") or s.get("mime", "")) - and (s.get("height") or 0) <= 1080 + and 720 <= (s.get("height") or 0) <= 1080 and s.get("init_range") and s.get("index_range") and s.get("url") ] + if not h264: + # Fallback: drop the 720p floor if the video has no HD streams at all. + h264 = [ + s + for s in video_streams + if "avc1" in (s.get("codec") or s.get("mime", "")) + and (s.get("height") or 0) <= 1080 + and s.get("init_range") + and s.get("index_range") + and s.get("url") + ] if not h264: return None @@ -314,14 +358,26 @@ def _play(yt_id: str) -> None: mpd_bytes: bytes | None = None dash_resp: dict[str, Any] = {} + # DASH path: read setting first; fall back to env-var; OR honor a magic file + # at /storage/.kodi/userdata/addon_data/plugin.video.torttube/dash.on as a + # last-ditch trigger that doesn't depend on Kodi's settings cache. dash_enabled = False try: dash_enabled = ADDON.getSettingBool("dash_enabled") except Exception: - # Setting might not exist on older configs; treat as off. pass - if dash_enabled or os.environ.get("TORTTUBE_DASH") == "1": + env_dash = os.environ.get("TORTTUBE_DASH") == "1" + addon_data = "" + try: + import xbmcvfs + addon_data = xbmcvfs.translatePath("special://profile/addon_data/plugin.video.torttube/") + except Exception: + pass + file_dash = bool(addon_data) and os.path.isfile(os.path.join(addon_data, "dash.on")) + _log(f"play id={yt_id} dash_enabled={dash_enabled} env_dash={env_dash} file_dash={file_dash}") + if dash_enabled or env_dash or file_dash: mpd_bytes, dash_resp = _try_dash(yt_id) + _log(f"_try_dash returned mpd_bytes={'<%d bytes>' % len(mpd_bytes) if mpd_bytes else None}") if mpd_bytes: details = dash_resp.get("details") or {} title = details.get("name") diff --git a/docs/upstream.md b/docs/upstream.md index ac3d228..e085255 100644 --- a/docs/upstream.md +++ b/docs/upstream.md @@ -19,10 +19,27 @@ _(none yet — opens with M1 development)_ ([kodi addon, xbmc/inputstream.adaptive on github](https://github.com/xbmc/inputstream.adaptive)) Hit during torttube M7 DASH work 2026-05-23. Loading an MPD from a local filesystem path via `file:///storage/.kodi/temp/torttube/.mpd` fails - with `CURLOpen returned an error, download failed`. Common workaround - is a localhost HTTP server (plugin.video.youtube does this). Worth - filing an enhancement request to either accept `file://` or document - the recommended pattern. + with `CURLOpen returned an error, download failed`. **Workaround + confirmed**: bind a localhost ThreadingHTTPServer on the LAN IP (not + 127.0.0.1 — that also fails in some configs) and pass `http://:` + to setResolvedUrl. plugin.video.youtube uses this pattern via a long-lived + service addon. Worth filing an enhancement to either accept `file://` or + document the LAN-IP HTTP-server pattern in the inputstream.adaptive docs. +- **DASH segment timing for googlevideo SegmentBase URLs** — Hit 2026-05-23. + My MPD with one Representation per video/audio (using SegmentBase with + indexRange to the sidx box of the static MP4) parses cleanly and segments + fetch correctly once the `User-Agent=Mozilla/...&Origin=https://www.youtube.com&Referer=https://www.youtube.com/` + headers are set via `inputstream.adaptive.stream_headers`. BUT audio drifts + badly behind video (-25s growing to -44s within seconds of playback start). + Hypothesis: inputstream.adaptive needs explicit per-segment timing (via + `` entries) or `presentationTimeOffset` to + align separated audio + video streams correctly. plugin.video.youtube + derives these by parsing the sidx box of each representation. Possible + upstream PRs: (a) inputstream.adaptive should auto-derive segment timing + from sidx when SegmentBase + indexRange is present, OR (b) document the + requirement for SegmentTimeline on separated A/V. For torttube we'll need + to either parse sidx ourselves (extra HTTP HEAD + binary parse) or fork + plugin.video.youtube's MPD-builder. - **Kodi — "Logic error due to two concurrent busydialogs" fatal** Reproduced during rapid back-to-back `Player.Open` calls while a previous play's BusyDialog was still dismissing. Log message itself