Sulkta-OSS/straw
2
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

Add support of other delivery methods than progressive HTTP (in the player only)

Browse source 
Detailed changes:

- External players:

  - Add a message instruction about stream selection;
  - Add a message when there is no stream available for external players;
  - Return now HLS, DASH and SmoothStreaming URL contents, in addition to progressive HTTP ones.

- Player:

  - Support DASH, HLS and SmoothStreaming streams for videos, whether they are content URLs or the manifests themselves, in addition to progressive HTTP ones;
  - Use a custom HttpDataSource to play YouTube contents, based of ExoPlayer's default one, which allows better spoofing of official clients (custom user-agent and headers (depending of the client used), use of range and rn (set dynamically by the DataSource) parameters);
  - Fetch YouTube progressive contents as DASH streams, like official clients, support fully playback of livestreams which have ended recently and OTF streams;
  - Use ExoPlayer's default retries count for contents on non-fatal errors (instead of Integer.MAX_VALUE for non-live contents and 5 for live contents).

- Download dialog:

  - Add message about support of progressive HTTP streams only for downloading;
  - Remove several duplicated code and update relevant usages;
  - Support downloading of contents with an unknown media format.

- ListHelper:

  - Catch NumberFormatException when trying to compare two video streams between them.

- Tests:

  - Update ListHelperTest and StreamItemAdapterTest to fix breaking changes in the extractor.

- Other places:

  - Fixes deprecation of changes made in the extractor;
  - Improve some code related to the files changed.

- Issues fixed and/or improved with the changes:

  - Seeking of PeerTube HLS streams (the duration shown was the one from the stream duration and not the one parsed, incomplete because HLS streams are fragmented MP4s with multiple sidx boxes, for which seeking is not supported by ExoPlayer) (the app now uses the HLS manifest returned for each quality, in the master playlist (not fetched and computed by the extractor));
  - Crash when loading PeerTube streams with a separated audio;
  - Lack of some streams on some YouTube videos (OTF streams);
  - Loading times of YouTube streams, after a quality change or a playback start;
  - View count of YouTube ended livestreams interpreted as watching count (this type of streams is not interpreted anymore as livestreams);
  - Watchable time of YouTube ended livestreams;
  - Playback of SoundCloud HLS-only tracks (which cannot be downloaded anymore because the workaround which was used is being removed by SoundCloud, so it has been removed from the extractor).
This commit is contained in:
AudricV 2022-06-16 11:13:19 +02:00
parent a59660f421
commit 210834fbe9
No known key found for this signature in database
GPG key ID: DA92EC7905614198
27 changed files with 2417 additions and 539 deletions
Download patch file Download diff file Expand all files Collapse all files

187
app/src/test/java/org/schabi/newpipe/util/ListHelperTest.java
View file

@ -13,38 +13,41 @@ import static org.junit.Assert.assertEquals;
import static org.junit.Assert.assertFalse;
import static org.junit.Assert.assertTrue;
import androidx.annotation.NonNull;
import androidx.annotation.Nullable;
public class ListHelperTest {
private static final String BEST_RESOLUTION_KEY = "best_resolution";
private static final List<AudioStream> AUDIO_STREAMS_TEST_LIST = Arrays.asList(
new AudioStream("", MediaFormat.M4A, /**/ 128),
new AudioStream("", MediaFormat.WEBMA, /**/ 192),
new AudioStream("", MediaFormat.MP3, /**/ 64),
new AudioStream("", MediaFormat.WEBMA, /**/ 192),
new AudioStream("", MediaFormat.M4A, /**/ 128),
new AudioStream("", MediaFormat.MP3, /**/ 128),
new AudioStream("", MediaFormat.WEBMA, /**/ 64),
new AudioStream("", MediaFormat.M4A, /**/ 320),
new AudioStream("", MediaFormat.MP3, /**/ 192),
new AudioStream("", MediaFormat.WEBMA, /**/ 320));
generateAudioStream("m4a-128-1", MediaFormat.M4A, 128),
generateAudioStream("webma-192", MediaFormat.WEBMA, 192),
generateAudioStream("mp3-64", MediaFormat.MP3, 64),
generateAudioStream("webma-192", MediaFormat.WEBMA, 192),
generateAudioStream("m4a-128-2", MediaFormat.M4A, 128),
generateAudioStream("mp3-128", MediaFormat.MP3, 128),
generateAudioStream("webma-64", MediaFormat.WEBMA, 64),
generateAudioStream("m4a-320", MediaFormat.M4A, 320),
generateAudioStream("mp3-192", MediaFormat.MP3, 192),
generateAudioStream("webma-320", MediaFormat.WEBMA, 320));
private static final List<VideoStream> VIDEO_STREAMS_TEST_LIST = Arrays.asList(
new VideoStream("", MediaFormat.MPEG_4, /**/ "720p"),
new VideoStream("", MediaFormat.v3GPP, /**/ "240p"),
new VideoStream("", MediaFormat.WEBM, /**/ "480p"),
new VideoStream("", MediaFormat.v3GPP, /**/ "144p"),
new VideoStream("", MediaFormat.MPEG_4, /**/ "360p"),
new VideoStream("", MediaFormat.WEBM, /**/ "360p"));
generateVideoStream("mpeg_4-720", MediaFormat.MPEG_4, "720p", false),
generateVideoStream("v3gpp-240", MediaFormat.v3GPP, "240p", false),
generateVideoStream("webm-480", MediaFormat.WEBM, "480p", false),
generateVideoStream("v3gpp-144", MediaFormat.v3GPP, "144p", false),
generateVideoStream("mpeg_4-360", MediaFormat.MPEG_4, "360p", false),
generateVideoStream("webm-360", MediaFormat.WEBM, "360p", false));
private static final List<VideoStream> VIDEO_ONLY_STREAMS_TEST_LIST = Arrays.asList(
new VideoStream("", MediaFormat.MPEG_4, /**/ "720p", true),
new VideoStream("", MediaFormat.MPEG_4, /**/ "720p", true),
new VideoStream("", MediaFormat.MPEG_4, /**/ "2160p", true),
new VideoStream("", MediaFormat.MPEG_4, /**/ "1440p60", true),
new VideoStream("", MediaFormat.WEBM, /**/ "720p60", true),
new VideoStream("", MediaFormat.MPEG_4, /**/ "2160p60", true),
new VideoStream("", MediaFormat.MPEG_4, /**/ "720p60", true),
new VideoStream("", MediaFormat.MPEG_4, /**/ "1080p", true),
new VideoStream("", MediaFormat.MPEG_4, /**/ "1080p60", true));
generateVideoStream("mpeg_4-720-1", MediaFormat.MPEG_4, "720p", true),
generateVideoStream("mpeg_4-720-2", MediaFormat.MPEG_4, "720p", true),
generateVideoStream("mpeg_4-2160", MediaFormat.MPEG_4, "2160p", true),
generateVideoStream("mpeg_4-1440_60", MediaFormat.MPEG_4, "1440p60", true),
generateVideoStream("webm-720_60", MediaFormat.WEBM, "720p60", true),
generateVideoStream("mpeg_4-2160_60", MediaFormat.MPEG_4, "2160p60", true),
generateVideoStream("mpeg_4-720_60", MediaFormat.MPEG_4, "720p60", true),
generateVideoStream("mpeg_4-1080", MediaFormat.MPEG_4, "1080p", true),
generateVideoStream("mpeg_4-1080_60", MediaFormat.MPEG_4, "1080p60", true));
@Test
public void getSortedStreamVideosListTest() {
@ -56,7 +59,8 @@ public class ListHelperTest {
assertEquals(expected.size(), result.size());
for (int i = 0; i < result.size(); i++) {
assertEquals(expected.get(i), result.get(i).resolution);
assertEquals(result.get(i).getResolution(), expected.get(i));
assertEquals(expected.get(i), result.get(i).getResolution());
}
////////////////////
@ -69,7 +73,7 @@ public class ListHelperTest {
"720p", "480p", "360p", "240p", "144p");
assertEquals(expected.size(), result.size());
for (int i = 0; i < result.size(); i++) {
assertEquals(expected.get(i), result.get(i).resolution);
assertEquals(expected.get(i), result.get(i).getResolution());
}
}
@ -83,8 +87,8 @@ public class ListHelperTest {
assertEquals(expected.size(), result.size());
for (int i = 0; i < result.size(); i++) {
assertEquals(expected.get(i), result.get(i).resolution);
assertTrue(result.get(i).isVideoOnly);
assertEquals(expected.get(i), result.get(i).getResolution());
assertTrue(result.get(i).isVideoOnly());
}
//////////////////////////////////////////////////////////
@ -96,8 +100,8 @@ public class ListHelperTest {
expected = Arrays.asList("720p", "480p", "360p", "240p", "144p");
assertEquals(expected.size(), result.size());
for (int i = 0; i < result.size(); i++) {
assertEquals(expected.get(i), result.get(i).resolution);
assertFalse(result.get(i).isVideoOnly);
assertEquals(expected.get(i), result.get(i).getResolution());
assertFalse(result.get(i).isVideoOnly());
}
/////////////////////////////////////////////////////////////////
@ -113,10 +117,9 @@ public class ListHelperTest {
assertEquals(expected.size(), result.size());
for (int i = 0; i < result.size(); i++) {
assertEquals(expected.get(i), result.get(i).resolution);
assertEquals(
expectedVideoOnly.contains(result.get(i).resolution),
result.get(i).isVideoOnly);
assertEquals(expected.get(i), result.get(i).getResolution());
assertEquals(expectedVideoOnly.contains(result.get(i).getResolution()),
result.get(i).isVideoOnly());
}
}
@ -132,66 +135,66 @@ public class ListHelperTest {
"1080p60", "1080p", "720p60", "720p", "480p", "360p", "240p", "144p");
assertEquals(expected.size(), result.size());
for (int i = 0; i < result.size(); i++) {
assertEquals(expected.get(i), result.get(i).resolution);
assertEquals(expected.get(i), result.get(i).getResolution());
}
}
@Test
public void getDefaultResolutionTest() {
final List<VideoStream> testList = Arrays.asList(
new VideoStream("", MediaFormat.MPEG_4, /**/ "720p"),
new VideoStream("", MediaFormat.v3GPP, /**/ "240p"),
new VideoStream("", MediaFormat.WEBM, /**/ "480p"),
new VideoStream("", MediaFormat.WEBM, /**/ "240p"),
new VideoStream("", MediaFormat.MPEG_4, /**/ "240p"),
new VideoStream("", MediaFormat.WEBM, /**/ "144p"),
new VideoStream("", MediaFormat.MPEG_4, /**/ "360p"),
new VideoStream("", MediaFormat.WEBM, /**/ "360p"));
generateVideoStream("mpeg_4-720", MediaFormat.MPEG_4, "720p", false),
generateVideoStream("v3gpp-240", MediaFormat.v3GPP, "240p", false),
generateVideoStream("webm-480", MediaFormat.WEBM, "480p", false),
generateVideoStream("webm-240", MediaFormat.WEBM, "240p", false),
generateVideoStream("mpeg_4-240", MediaFormat.MPEG_4, "240p", false),
generateVideoStream("webm-144", MediaFormat.WEBM, "144p", false),
generateVideoStream("mpeg_4-360", MediaFormat.MPEG_4, "360p", false),
generateVideoStream("webm-360", MediaFormat.WEBM, "360p", false));
VideoStream result = testList.get(ListHelper.getDefaultResolutionIndex(
"720p", BEST_RESOLUTION_KEY, MediaFormat.MPEG_4, testList));
assertEquals("720p", result.resolution);
assertEquals("720p", result.getResolution());
assertEquals(MediaFormat.MPEG_4, result.getFormat());
// Have resolution and the format
result = testList.get(ListHelper.getDefaultResolutionIndex(
"480p", BEST_RESOLUTION_KEY, MediaFormat.WEBM, testList));
assertEquals("480p", result.resolution);
assertEquals("480p", result.getResolution());
assertEquals(MediaFormat.WEBM, result.getFormat());
// Have resolution but not the format
result = testList.get(ListHelper.getDefaultResolutionIndex(
"480p", BEST_RESOLUTION_KEY, MediaFormat.MPEG_4, testList));
assertEquals("480p", result.resolution);
assertEquals("480p", result.getResolution());
assertEquals(MediaFormat.WEBM, result.getFormat());
// Have resolution and the format
result = testList.get(ListHelper.getDefaultResolutionIndex(
"240p", BEST_RESOLUTION_KEY, MediaFormat.WEBM, testList));
assertEquals("240p", result.resolution);
assertEquals("240p", result.getResolution());
assertEquals(MediaFormat.WEBM, result.getFormat());
// The best resolution
result = testList.get(ListHelper.getDefaultResolutionIndex(
BEST_RESOLUTION_KEY, BEST_RESOLUTION_KEY, MediaFormat.WEBM, testList));
assertEquals("720p", result.resolution);
assertEquals("720p", result.getResolution());
assertEquals(MediaFormat.MPEG_4, result.getFormat());
// Doesn't have the 60fps variant and format
result = testList.get(ListHelper.getDefaultResolutionIndex(
"720p60", BEST_RESOLUTION_KEY, MediaFormat.WEBM, testList));
assertEquals("720p", result.resolution);
assertEquals("720p", result.getResolution());
assertEquals(MediaFormat.MPEG_4, result.getFormat());
// Doesn't have the 60fps variant
result = testList.get(ListHelper.getDefaultResolutionIndex(
"480p60", BEST_RESOLUTION_KEY, MediaFormat.WEBM, testList));
assertEquals("480p", result.resolution);
assertEquals("480p", result.getResolution());
assertEquals(MediaFormat.WEBM, result.getFormat());
// Doesn't have the resolution, will return the best one
result = testList.get(ListHelper.getDefaultResolutionIndex(
"2160p60", BEST_RESOLUTION_KEY, MediaFormat.WEBM, testList));
assertEquals("720p", result.resolution);
assertEquals("720p", result.getResolution());
assertEquals(MediaFormat.MPEG_4, result.getFormat());
}
@ -221,8 +224,8 @@ public class ListHelperTest {
////////////////////////////////////////
List<AudioStream> testList = Arrays.asList(
new AudioStream("", MediaFormat.M4A, /**/ 128),
new AudioStream("", MediaFormat.WEBMA, /**/ 192));
generateAudioStream("m4a-128", MediaFormat.M4A, 128),
generateAudioStream("webma-192", MediaFormat.WEBMA, 192));
// List doesn't contains this format
// It should fallback to the highest bitrate audio no matter what format it is
AudioStream stream = testList.get(ListHelper.getHighestQualityAudioIndex(
@ -235,13 +238,13 @@ public class ListHelperTest {
//////////////////////////////////////////////////////
testList = new ArrayList<>(Arrays.asList(
new AudioStream("", MediaFormat.WEBMA, /**/ 192),
new AudioStream("", MediaFormat.M4A, /**/ 192),
new AudioStream("", MediaFormat.WEBMA, /**/ 192),
new AudioStream("", MediaFormat.M4A, /**/ 192),
new AudioStream("", MediaFormat.WEBMA, /**/ 192),
new AudioStream("", MediaFormat.M4A, /**/ 192),
new AudioStream("", MediaFormat.WEBMA, /**/ 192)));
generateAudioStream("webma-192-1", MediaFormat.WEBMA, 192),
generateAudioStream("m4a-192-1", MediaFormat.M4A, 192),
generateAudioStream("webma-192-2", MediaFormat.WEBMA, 192),
generateAudioStream("m4a-192-2", MediaFormat.M4A, 192),
generateAudioStream("webma-192-3", MediaFormat.WEBMA, 192),
generateAudioStream("m4a-192-3", MediaFormat.M4A, 192),
generateAudioStream("webma-192-4", MediaFormat.WEBMA, 192)));
// List doesn't contains this format, it should fallback to the highest bitrate audio and
// the highest quality format.
stream = testList.get(ListHelper.getHighestQualityAudioIndex(MediaFormat.MP3, testList));
@ -250,7 +253,7 @@ public class ListHelperTest {
// Adding a new format and bitrate. Adding another stream will have no impact since
// it's not a preferred format.
testList.add(new AudioStream("", MediaFormat.WEBMA, /**/ 192));
testList.add(generateAudioStream("webma-192-5", MediaFormat.WEBMA, 192));
stream = testList.get(ListHelper.getHighestQualityAudioIndex(MediaFormat.MP3, testList));
assertEquals(192, stream.getAverageBitrate());
assertEquals(MediaFormat.M4A, stream.getFormat());
@ -288,8 +291,8 @@ public class ListHelperTest {
////////////////////////////////////////
List<AudioStream> testList = new ArrayList<>(Arrays.asList(
new AudioStream("", MediaFormat.M4A, /**/ 128),
new AudioStream("", MediaFormat.WEBMA, /**/ 192)));
generateAudioStream("m4a-128", MediaFormat.M4A, 128),
generateAudioStream("webma-192-1", MediaFormat.WEBMA, 192)));
// List doesn't contains this format
// It should fallback to the most compact audio no matter what format it is.
AudioStream stream = testList.get(ListHelper.getMostCompactAudioIndex(
@ -298,7 +301,7 @@ public class ListHelperTest {
assertEquals(MediaFormat.M4A, stream.getFormat());
// WEBMA is more compact than M4A
testList.add(new AudioStream("", MediaFormat.WEBMA, /**/ 128));
testList.add(generateAudioStream("webma-192-2", MediaFormat.WEBMA, 128));
stream = testList.get(ListHelper.getMostCompactAudioIndex(MediaFormat.MP3, testList));
assertEquals(128, stream.getAverageBitrate());
assertEquals(MediaFormat.WEBMA, stream.getFormat());
@ -308,12 +311,12 @@ public class ListHelperTest {
//////////////////////////////////////////////////////
testList = new ArrayList<>(Arrays.asList(
new AudioStream("", MediaFormat.WEBMA, /**/ 192),
new AudioStream("", MediaFormat.M4A, /**/ 192),
new AudioStream("", MediaFormat.WEBMA, /**/ 256),
new AudioStream("", MediaFormat.M4A, /**/ 192),
new AudioStream("", MediaFormat.WEBMA, /**/ 192),
new AudioStream("", MediaFormat.M4A, /**/ 192)));
generateAudioStream("webma-192-1", MediaFormat.WEBMA, 192),
generateAudioStream("m4a-192-1", MediaFormat.M4A, 192),
generateAudioStream("webma-256", MediaFormat.WEBMA, 256),
generateAudioStream("m4a-192-2", MediaFormat.M4A, 192),
generateAudioStream("webma-192-2", MediaFormat.WEBMA, 192),
generateAudioStream("m4a-192-3", MediaFormat.M4A, 192)));
// List doesn't contain this format
// It should fallback to the most compact audio no matter what format it is.
stream = testList.get(ListHelper.getMostCompactAudioIndex(MediaFormat.MP3, testList));
@ -335,14 +338,14 @@ public class ListHelperTest {
@Test
public void getVideoDefaultStreamIndexCombinations() {
final List<VideoStream> testList = Arrays.asList(
new VideoStream("", MediaFormat.MPEG_4, /**/ "1080p"),
new VideoStream("", MediaFormat.MPEG_4, /**/ "720p60"),
new VideoStream("", MediaFormat.MPEG_4, /**/ "720p"),
new VideoStream("", MediaFormat.WEBM, /**/ "480p"),
new VideoStream("", MediaFormat.MPEG_4, /**/ "360p"),
new VideoStream("", MediaFormat.WEBM, /**/ "360p"),
new VideoStream("", MediaFormat.v3GPP, /**/ "240p60"),
new VideoStream("", MediaFormat.WEBM, /**/ "144p"));
generateVideoStream("mpeg_4-1080", MediaFormat.MPEG_4, "1080p", false),
generateVideoStream("mpeg_4-720_60", MediaFormat.MPEG_4, "720p60", false),
generateVideoStream("mpeg_4-720", MediaFormat.MPEG_4, "720p", false),
generateVideoStream("webm-480", MediaFormat.WEBM, "480p", false),
generateVideoStream("mpeg_4-360", MediaFormat.MPEG_4, "360p", false),
generateVideoStream("webm-360", MediaFormat.WEBM, "360p", false),
generateVideoStream("v3gpp-240_60", MediaFormat.v3GPP, "240p60", false),
generateVideoStream("webm-144", MediaFormat.WEBM, "144p", false));
// exact matches
assertEquals(1, ListHelper.getVideoStreamIndex("720p60", MediaFormat.MPEG_4, testList));
@ -375,4 +378,30 @@ public class ListHelperTest {
// Can't find a match
assertEquals(-1, ListHelper.getVideoStreamIndex("100p", null, testList));
}
@NonNull
private static AudioStream generateAudioStream(@NonNull final String id,
@Nullable final MediaFormat mediaFormat,
final int averageBitrate) {
return new AudioStream.Builder()
.setId(id)
.setContent("", true)
.setMediaFormat(mediaFormat)
.setAverageBitrate(averageBitrate)
.build();
}
@NonNull
private static VideoStream generateVideoStream(@NonNull final String id,
@Nullable final MediaFormat mediaFormat,
@NonNull final String resolution,
final boolean isVideoOnly) {
return new VideoStream.Builder()
.setId(id)
.setContent("", true)
.setIsVideoOnly(isVideoOnly)
.setResolution(resolution)
.setMediaFormat(mediaFormat)
.build();
}
}