Commit Graph

149 Commits

Author SHA1 Message Date
Pierre Ynard e9705c553c youtube.lua: &fmt=hd URL parameter to leverage "adaptive" formats
This provides experimental support to play again all available video
resolutions, by combining audio and video "adaptive" elementary streams
using an input slave. Not in use by default; use at your own risks.

Again, by allowing to select resolutions lower than 360p, this also
provides mitigation against the throttling issue.

Ref #10237, #27227
2023-02-14 16:42:07 +00:00
Pierre Ynard b88da3715b youtube.lua: split stream URL parsing into separate function 2023-02-14 16:42:07 +00:00
Pierre Ynard 94d5e1aa65 youtube.lua: &fmt={audio,video} URL parameters for "adaptive" formats
This extends the old &fmt=[itag] URL syntax that we still supported all
along to force format selection, and allows choosing from "adaptive"
elementary streams without knowledge of itag specifics and in accordance
with normal resolution preferences.

This also allows playing only the audio part of music videos and
skipping the download of the video part entirely, greatly reducing
bitrate and providing mitigation against the throttling issue.

Ref #10237, #27227
2023-02-14 16:42:07 +00:00
Pierre Ynard aa2032bb2e youtube.lua: generic support for YouTube "adaptive" formats
Formats listed under that label are audio-only or video-only elementary
streams, but offer choice encompassing the full array of supported
resolutions, qualities and codecs; whereas classic multiplexed formats
have long been dwindling down to only two formats now, 720p and 360p, or
even 360p only for some content.

For now, these "adaptive" formats are only used if explicitly requested
by itag number.

Ref #10237
2023-02-14 16:42:07 +00:00
Pierre Ynard b916319907 youtube.lua: split stream format selection into separate function 2023-02-14 16:42:07 +00:00
Pierre Ynard 7d9722f206 youtube.lua: lazy initialization for fetching descrambling javascript
This makes more sense and will simplify code flow.
2023-02-14 16:42:07 +00:00
Pierre Ynard b5c12e5ed1 youtube.lua: update comments about fmt URL parameter
Link to old Wikipedia page revision where itag parameter formats were
actually still described.
2023-02-14 16:42:07 +00:00
Pierre Ynard 12b739ef68 youtube.lua: declare proper local scope for working variables 2023-02-14 16:42:07 +00:00
Pierre Ynard 91aac3a2fa youtube.lua: disable broken call in dead code 2023-02-14 16:42:07 +00:00
Pierre Ynard 9bbdd152cb youtube.lua: disable broken "n" descrambling for now
All it achieves at the moment is wasting CPU time, slowing down playback
startup, and distracting users with extraneous warnings.

Ref #27227
2023-02-14 16:42:07 +00:00
Pierre Ynard 5bab61310f youtube.lua: support YouTube Shorts URLs 2023-02-14 16:42:07 +00:00
Pierre Ynard a0b3c43815 youtube.lua: update for player config definition site change
Fixes #27529
2022-11-19 07:19:13 +00:00
Pierre Ynard 90dcfa0e78 youtube.lua: bump up severity of "n" parameter throttling warning
Due to the severely increased complexity of "n" descrambling code, a
quick fix is unfortunately not foreseeable. For now, let users know more
clearly what's going on and what's to expect or not.

Ref #27227
2022-09-01 20:09:08 +00:00
Pierre Ynard b735ef2cde youtube.lua: don't silently fail on "n" descrambling script parsing
The descrambling script section was updated from a simple and linear
chain of calls, to a complex execution tree with conditional branches.
Failure to recognize and parse this call structure (or lack thereof)
resulted in a silent no-op. Add a check to properly report an error.

Ref #27227
2022-09-01 20:09:08 +00:00
Pierre Ynard 1494441075 youtube.lua: fix crash on unexpected "n" descrambling transformation code
It was possible, when encountering different code from what was expected
for some known transformations, to void the code parsing pointer instead
of advancing it, resulting in a subsequent crash of the script and total
playback failure. Add a fallback and check, to prevent and gracefully
deal with this, and still allow playback, even if throttled, in case of
descrambling failure.
2022-09-01 20:09:08 +00:00
Marvin Scholz 6995265a57 share: lua: remove $Id$ magic comments
These were used by SVN, we are no longer using that though.
2022-07-29 06:39:56 +00:00
Pierre Ynard 3df28fc51e youtube.lua: update "n" descrambling function name parsing
The descrambling function is now called through an intermediate array
variable. This change has also added two extra ways to recover the
function name. Add support to parse and resolve any of them.

Fixes #26574
2022-02-06 08:24:28 +00:00
Pierre Ynard 6b5bc195a9 youtube.lua: exponential form integers in "n" descrambling input data
They now appear; add support for parsing them.

Fixes #26317
2021-11-25 11:33:31 +00:00
Pierre Ynard 84ba044df1 youtube.lua: document an "n" descrambling transformation 2021-11-25 11:33:31 +00:00
Pierre Ynard 7f6614cbaf youtube.lua: add "n" descrambling split compound transformations
A new variant of compound transformation has the Base64 alphabet
generation and the compounding itself as two separate data array
elements, contrary to what was observed so far. Add support for those.

Fixes #26285
2021-11-16 08:18:00 +00:00
Pierre Ynard fd2e937f28 youtube.lua: "n" descrambling transformations with two extra arguments
A new standalone compound transformation, taking its Base64 alphabet
as extra input argument, has revealed itself. We support parsing and
passing this one more argument from the script section.

Technically this last argument can be a function or rather the result
of its call, but with no argument, we know what's always returned, and
don't need to treat it as a function. This is less clean but simpler and
will do for now.
2021-11-16 08:18:00 +00:00
Pierre Ynard 174e12b3bc youtube.lua: simplify "n" descrambling compound transformation
Newly observed transformations reveal that the uncertain character
code variable used as constant offset, really isn't one and is simply
supposed to be the alphabet's length. Thus even more so, it is a no-op
on the alphabet's algebraic modulo group, and probably just an artifact
of how modulo of negative numbers is handled in javascript. Simplify it
away.
2021-11-16 08:18:00 +00:00
Pierre Ynard 03e6957832 youtube.lua: descramble "n" video URL parameter by parsing javascript
User agents are apparently now expected to do this; failure to do so
results in the video file data transfer getting throttled down to rates
such as 80 kB/s, 60 kB/s or 40 kB/s, below playback rate, and usually
resulting in a video that hangs upon loading or every few seconds, and
is impossible to play. This behavior seems to have first appeared in
June, but been fully rolled out only last week.

Just like with URL signatures, we interoperate with YouTube by
fulfilling what's apparently expected from us, using the same approach
as so far: we parse the descrambling rules from the javascript code, and
apply them.

Fixes #26174
2021-10-20 19:15:53 +02:00
Pierre Ynard f3963e68c1 youtube.lua: retry fetching descrambling javascript asset once
This should help against transient errors, and parsing of the javascript
URL isn't the part that's most likely to break.
2021-10-20 19:15:51 +02:00
Pierre Ynard 4cfa8b65ef youtube.lua: factor out descrambling javascript fetching
We'll be descrambling the "n" parameter in addition to the URL signature
using this same javascript web asset, so we want to be able to share and
reuse it.
2021-10-20 19:15:50 +02:00
Pierre Ynard 095f093093 youtube.lua: rename signature descrambling function
Use a more specific name as this isn't the only parameter anymore that
we'll be descrambling by parsing and emulating javascript.
2021-10-20 19:15:48 +02:00
Pierre Ynard 8473b3bf3b youtube.lua: rework error handling for signature descrambling 2021-10-20 19:15:46 +02:00
Pierre Ynard a178691270 youtube.lua: fix up signature descrambling function name extraction
Javascript variables can contain other, special characters, also %a
depends on the locale.
2021-10-20 19:15:44 +02:00
Pierre Ynard c7b4efcf75 youtube.lua: remove fallback to retired alternate video info API
After tightening access restrictions to it, the get_video_info YouTube
API was completely retired around July 2021, with an HTTP 410 Gone code.
All this fallback achieves anymore is poor UX.
2021-10-20 19:15:43 +02:00
Pierre Ynard b2c32b5eda youtube.lua: add extra "out of use" comment 2021-10-20 19:15:41 +02:00
Kung Lao c05eb00c25 youtube.lua - js_descramble function name return fix: the string pattern match now returns 2 or 3 char length function names. Issue ref: #25988, #25995 (closed) 2021-09-04 07:57:17 +00:00
Pierre Ynard f21c063ea3 youtube.lua: disable cookies if redirected to consent page
In the past few days, YouTube has started redirecting requests for video
pages to a cookie consent and preference prompt, on a whole separate
page and domain; which prevents playback. We are not interested in
YouTube cookies, nor in consenting to them on behalf of our users, so
this just retries with cookies disabled. YouTube gets the hint and
simply redirects back to the original video page.

Fixes #25616
2021-04-03 12:02:34 +02:00
Pierre Ynard 6e0f4a28c6 youtube.lua: retry alternate API fallback using either possible parameter
This improves the interoperability of the fallback mechanism and makes
the script more resilient as a whole.
2020-11-12 07:22:04 +01:00
Pierre Ynard 8ea54417ba youtube.lua: look for javascript descrambling URL on any line
This URL is forwarded to the fallback API and the whole point of that is
in case the main stream configuration line can't be found and parsed,
so don't look for it only on that line. The URL can indeed be found in
several other places on the page.
2020-11-12 06:21:27 +01:00
Pierre Ynard 2d0f63a61b youtube.lua: forward javascript descrambling URL to alternate API
This renders the fallback to the alternate video info API - which
doesn't provide the javascript URL itself - functional with many more
videos, and makes the script as a whole more resilient to future
failures.
2020-11-12 05:58:28 +01:00
Pierre Ynard 95cf2a33e9 youtube.lua: support music.youtube.com URLs
Simply redirect to the main domain

Fixes #25241
2020-11-12 05:53:40 +01:00
Pierre Ynard 77f19929db youtube.lua: make gaming.youtube.com subdomain support more generic
And mark it as out of use: gaming.youtube.com URLs now get redirected to
the main domain.
2020-11-12 05:42:59 +01:00
Pierre Ynard 7b7eac1461 youtube.lua: fix escaped '\r' in metadata from alternate API 2020-11-12 03:38:58 +01:00
Pierre Ynard 61b51a0dfd youtube.lua: always use long line workaround with new HTML code layout
This is required for the main configuration line, and possibly another
line before it. Until more is known, it seems more prudent to enable the
workaround unconditionally for now, than to try and guess what should
work correctly.
2020-11-04 19:22:10 +01:00
Pierre Ynard c20877a8b4 youtube.lua: split long line workaround into reusable helper 2020-11-04 19:17:23 +01:00
Pierre Ynard 671afe05eb youtube.lua: support new HTML code layout
This new layout is apparently getting phased in. As major differences,
line splitting is more erratic (with overall 10 times fewer lines for a
slightly bigger HTML document), and the main stream configuration isn't
subjected to double JSON string encoding anymore.
2020-11-03 01:20:21 +01:00
Pierre Ynard 611ba98b65 youtube.lua: stricter check for applying peek() long line workaround
Required with a new HTML code layout apparently getting phased in
2020-11-03 01:02:03 +01:00
Pierre Ynard f3d206499b youtube.lua: more robust meta tag metadata parsing
Required for a new HTML code layout apparently getting phased in
2020-11-03 00:41:48 +01:00
Pierre Ynard ff8edf3493 youtube.lua: stop trying to double XML-decode title metadata
This attribute isn't double-encoded now, and so attempting this is not
just pointless but can theoretically lead to wrong output.
2020-11-03 00:26:39 +01:00
Pierre Ynard 083059cd88 youtube.lua: fix escaped characters issue in artist metadata
The double encoding of \u0026 wasn't handled.
2020-11-03 00:14:45 +01:00
Pierre Ynard 43a5da94af youtube.lua: new signature descrambling javascript URL parameter name
Apparently the old parameter has been replaced by a new one, and is now
getting phased out. The signature descrambling javascript URL is still
available in several other places in the HTML page either way.

Fixes #25223
2020-10-26 08:27:28 +01:00
Pierre Ynard b42890f04c youtube.lua: return title as name field from alternate API too
The name is used as fallback if the title is unset, but not conversely:
so setting the item title instead can have interesting side effects.
This was an odd one; like in most lua playlist scripts we really want to
set a name here.

Fixes #25124
2020-09-19 08:18:13 +02:00
Pierre Ynard e3378ac3a8 youtube.lua: work around VLC API limitations on very long lines
The main configuration line is such a very long line, and has been
growing longer recently, frequently hitting the VLC core limit at
200 kB. This caused readline() to fail to return any data, and stop
parsing of the web page short, preventing playback as the stream URLs
were in that line that was never returned.

Instead this relies on peek() and sized read() calls to parse and
recover that line by hand. This effectively bumps things up to up to
1 MB of usable configuration data.

Fixes #24957
2020-09-08 06:30:54 +02:00
Pierre Ynard 9ddd1b30f1 youtube.lua: fix parsing of empty description field 2020-09-07 20:07:02 +02:00
Pierre Ynard ff152eafc3 youtube.lua: fix escaped '\r' in description metadata
The description field sometimes somehow even contains mixed LF and CRLF.
2020-09-07 20:05:27 +02:00