- Add `StreamlinkWarning` and `StreamlinkDeprecationWarning`
and replace `FutureWarning`s
- Don't include the warning's origin in the warning logger if it's a
subclass of `StreamlinkWarning`
- Update tests
Replace the logic of the `Streamlink.{get,set}_option()` methods with
a new `StreamlinkOptions` subclass which uses mapped option
getters+setters that apply the custom option logic instead.
This improves the runtime of said methods, as option mappers are
looked up in Python dictionaries in constant time, so option names
don't have to be checked one after another.
Also add deprecation messages when
- getting the deprecated `https-proxy` option
- setting the deprecated `{hls,dash}-segment-attempts` options
- setting the deprecated `{hls,dash}-segment-threads` options
- setting the deprecated `{hls,dash}-segment-timeout` options
- setting the deprecated `{hls,dash,http-stream}-timeout` options
The `parse_{cookies,headers,query_params}` methods were added when the
subclass of `requests.Session` was implemented in order to support
setting cookies, headers and query parameters via `k1=v1;k2=v2` strings
(in addition to key-value dicts) via the session API and via the CLI:
- 936e66dd90
- c6e54fd57a
Since these methods implement logic purely for the `Streamlink` session
interface and are not meant to be called by any plugin or stream
implementations which use the session's `HTTPSession` instance, they
should be removed. Cookies, headers and query string parameters should
be set directly on their respective `HTTPSession` attributes:
- `cookies`: instance of `requests.cookies.RequestsCookieJar`
- `headers`: instance of `requests.structures.CaseInsensitiveDict`
- `params`: instance of `dict`
Also, at least in regards to HTTP headers, the `key=value` syntax
does not reflect the syntax of raw HTTP requests/responses or interfaces
of other tools like cURL, etc., so having these methods on the
`HTTPSession` class makes it unnecessarily confusing. The method names
themselves are also confusing, as they suggest that the input gets
parsed and that some result gets returned, which is wrong.
This commit therefore moves the `k1=v1;k2=v2` string logic from the
`http_session` module to the `session` module where it belongs and it
also simplifies the option setter.
This changes the way how the Streamlink session and other objects like
the plugin cache and logger are stored on each plugin.
Previously, those objects were set as class attributes on every `Plugin`
class via `Plugin.bind()` when loading plugins via the session's
`load_plugins()` method that gets called on initialization.
This meant that whenever a new Streamlink session was initialized,
references to it (including a dict of every loaded plugin) were set
on each `Plugin` class as a class attribute, and Python's garbage
collector could not get rid of this memory when deleting the session
instance that was created last.
Removing `Plugin.bind()`, passing the session via the `Plugin.__init__`
constructor, and setting the cache, logger, etc. on `Plugin` instances
instead (only one gets initialized by `streamlink_cli`), removes those
static references that prevent the garbage collector to work.
Since the plugin "module" name now doesn't get set via `Plugin.bind()`
anymore, it derives its name via `self.__class__.__module__` on its own,
which means a change of the return type of `Streamlink.resolve_url()`
is necessary in order to pass the plugin name to `streamlink_cli`,
so that it can load config files and initialize plugin arguments, etc.
Breaking changes:
- Remove `Plugin.bind()`
- Pass the `session` instance via the Plugin constructor and set the
`module`, `cache` and `logger` on the plugin instance instead.
Derive `module` from the actual module name.
- Change the return type of `Session.resolve_url()` and include the
resolved plugin name in the returned tuple
Other changes:
- Remove `pluginclass.bind()` call from `Session.load_plugins()` and
use the loader's module name directly on the `Session.plugins` dict
- Remove initialization check from `Plugin` cookie methods
- Update streamlink_cli.main module according to breaking changes
- Update tests respectively
- Add explicit plugin initialization test
- Update tests with plugin constructors and custom plugin names
- Move testplugin override module, so that it shares the same module
name as the main testplugin module. Rel `Session.load_plugins()`
- Refactor most session tests and replace unneeded `resolve_url()`
wrappers in favor of calling `session.streams()`
- Remove `user_input_requester` from `Plugin.bind()`, don't set it in
`Session.load_plugins()` and remove `Plugin._user_input_requester`
- Make `Plugin.input_ask{,_password}` read input requester from session
- Move `UserInputRequester` to the new `streamlink.user_input` module,
turn it into an abstract base class, and update imports
- Fix docstring of `Session.set_option()`
- Rewrite tests
- Import urllib3 directly instead of importing from requests.packages
- Remove unneeded urllib3.contrib.pyopenssl.DEFAULT_SSL_CIPHER_LIST
override, as it's not used on py3
- Update ipv4/ipv6 tests, add http-disable-dh tests
- Replace collection.OrderedDict with builtins.dict where possible:
Python 3.7+ ensures the correct order in builtins.dict objects and is
no longer an implementation detail of cpython.
- Fix OrderedDict type annotation in streamlink.utils.cache.LRUCache
- Add unit test for streamlink.utils.cache.LRUCache
- drop RTMP stream implementation
- drop RTMP plugin
- drop RTMPDump dependency
- remove stream.streamprocess and utils.{rtmp,swf}
- remove "rtmp" from default stream types list
- remove "rtmp" from player-passthrough options list
- remove all `--rtmp*` CLI args and `rtmp-*` session options
- remove all `--subprocess-*` CLI args and `subprocess` session options,
as these were used only by StreamProcess which is no longer needed
- update tests
- update docs and CLI argument help texts
- drop HDS and AkamaiHD stream implementations
- drop HDS and AkamaiHD plugins
- drop streamlink.packages.flashmedia
- remove stream.flvconcat, stream.playlist and plugins.common_swf
- remove "hds" from default stream types list
- remove all `--hds-*` CLI arguments and `hds-*` session options
- remove unneeded flashmedia license text files
- update tests
- update docs and CLI argument help texts
Breaking change:
Instead of resolving a plugin instance in `Streamlink.resolve_url(url)`
from the provided input URL (which can be redirected), resolve the
plugin class and the resulting URL, and cache the tuple. Also affects
`Streamlink.resolve_url_no_redirect(url)`.
The main reason for this change is streamlink_cli and how the plugin
options get set. The plugin options need to be set on the class before
it gets initialized, so that the instance can access the options in its
constructor method. This also fixes any kind of state stored on the
plugin instance.
- resolve and cache a tuple of the plugin class and resulting URL
- initialize plugin in streamlink_cli after applying its options
- remove plugin variable from global scope in streamlink_cli
- fix, rewrite and add tests
This reverts commit ccdd84bf76
- resolve merge conflicts
- keep stream=False parameter in HLSStreamWriter.fetch_map
- keep updated help text of --twitch-low-latency
- bump websocket-client requirement to >=1.2.1,<2.0
- avoid circular import by importing HTTPSession in session directly
- implement a common WebsocketClient based on the WebSocketApp API of
websocket-client, so that plugins don't have to re-implement basic
boilerplate code
- automatically apply User-Agent header
- automatically apply proxy settings
- set websocket log level (globally)
- add tests
Apply the https scheme to all input URLs if the scheme is missing:
- session:
- Session.resolve_url()
- session option `http-proxy` (`--http-proxy` CLI argument)
- plugins.akamaihd: `akamaihd://URL`
- plugins.dash: `dash://URL` or `URL.mpd`
- plugins.hds: `hds://URL` or `URL.f4m`
- plugins.hls: `hls://URL` or `URL.m3u8`
- plugins.http: `httpstream://URL`
Regular http URLs (non-https/TLS) will need to be set explicitly now.
Also
- update/fix test_session
- refactor/fix test_dash
- refactor/fix test_stream
Previously, update_scheme only added schemes on scheme-less URLs and
kept target URLs which included a scheme intact.
This is confusing for plugin implementers and it has already been used
incorrectly a lot of times when trying to force the HTTPS protocol on
unencrypted HTTP URLs. Since the method is called update_scheme, it
should do exactly that, in all cases.
However, always updating the target scheme will break those calls of
update_scheme where the scheme was previously only added if needed, eg.
on scheme-less URLs, partial URLs or URL paths.
An example of this are scheme-less input URLs passed to Streamlink's
`Session.resolve_url()`, which has to implicitly set the HTTP scheme.
Always overriding the scheme would turn explicit HTTPS schemes to HTTP,
which can break certain sites/plugins. The various stream-protocol
plugins have the same issue and have to implicitly set a scheme as well.
This change therefore introduces the `force=True` keyword, so that URL
schemes can be set implicitly again in the methods and plugins mentioned
above by setting force to False.
Another effect of forcing the scheme override by default is the pattern
currently used in many plugins, which often update (partial) URLs by
calling `update_scheme(self.url, target)`. Since `self.url` depends on
the user's input and since the input URL's scheme is implicitly set to
HTTP by the Session's URL resolver, target URLs can be downgraded from
HTTPS to HTTP, which is not desired. This is not a problem in cases
where the server will redirect, but it can be a problem where the server
returns an error instead.
These plugins will therefore need to be fixed so that they either set
the URL schemes implicitly or so that they force the according scheme.
Another solution (which will be done eventually) is the change of the
implicit input URL scheme from HTTP to HTTPS in `Session.resolve_url()`
and the various stream-protocol plugins, which will turn all scheme-less
input URLs to HTTPS by default, unless the user explicitly sets an HTTP
URL.
- always update target URL scheme with current URL scheme
- add force=True keyword and do not override scheme if force is False
- set force=False on all implicitly updated URL schemes
- `Session.resolve_url`
- `plugins.{akamaihd,hds,hls,http}`
- parametrize tests, re-order/re-format assertions and add new ones
- move remaining methods from `streamlink.utils` to `streamlink.utils.*`
- streamlink.utils.cache.LRUCache
- streamlink.utils.data.search_dict
- streamlink.utils.module.load_module
- streamlink.utils.rtmp.{escape_librtmp,rtmpparse}
- streamlink.utils.swf.swfdecompress
- fix imports in streamlink, streamlink_cli and tests
- move utils tests into new `tests.utils` package
- split up `test_utils` into individual test modules
- move and rename `test_utils_*`
- move and rename `test_localization`
- fix up some tests
- always stream data from unencrypted segment downloads
- never stream data from cached initialization sections, as the
response content can only be consumed once when stream=True
- update --twitch-low-latency plugin argument and its description
Having multiple session options and CLI arguments for different stream
types which are already covered by generic options/arguments is not only
redundant, but also confusing.
A distinction between different stream types does only make sense when
multiple different stream types are available and the user needs to
explicitly set different values for each of them, but since it's not
always clear which stream type is returned when a stream is selected
via the "best" stream name synonym for example, this makes it even more
confusing. And it's redundant as well, since only one stream can be
selected anyway.
- deprecate `{dash,hds,hls}-segment-attempts`
in favor of `stream-segment-attempts`
- deprecate `{dash,hds,hls}-segment-threads`
in favor of `stream-segment-threads`
- deprecate `{dash,hds,hls}-segment-timeout`
in favor of `stream-segment-timeout`
- deprecate `{dash,hds,hls,rtmp,http-stream}-timeout`
in favor of `stream-timeout` (dash/http-stream were never used)
- fix `HTTPStream` and use `stream-timeout` instead of `http-timeout`
- suppress deprecated CLI arguments
- update help texts of generic CLI arguments
- fix docs and add entry to deprecations page
Resolve deprecated plugins which are using the old
`can_handle_url(url)` and `priority(url)` methods.
Since the `Plugin` class doesn't implement the `priority` classmethod
with the NORMAL_PRIORITY return value anymore, the NORMAL_PRIORITY
gets set by the resolver if the method is missing on plugins which
implement `can_handle_url` and don't have the `matchers` list.
- drop `Plugin.can_handle_url` and `Plugin.priority`
- introduce `Plugin.matchers`
- implement `pluginmatcher` decorator
- automatically match url (on change) and store result
- update `Session.resolve_url`
- add and fix related tests
- wrap deprecated config and plugin paths in `DeprecatedPath`,
subclassed from pathlib.Path
- log info messages when loading deprecated configs or plugins
- return success from `Streamlink.load_plugins(path)` to be able to
log plugin loading messages in the main cli module
- move HLS filtering logic into `HLSStreamWriter` and `HLSStreamReader`
- remove `ignore_names` arg from `SegmentedStreamWriter`
- remove `stream.hls_filtered`
- add `hls-segment-ignore-names` as session option
- fix tests, but keep test_hls_filtered.py
- add test for `hls-segment-ignore-names`
- refactor plugins which were using the old FilteredHLS classes
- reword help text of `--hls-segment-ignore-names`
Refactor `utils.load_module`:
- move it to the top and add it to `__all__`
- move imports and loader_details to the global scope
Refactor `session`:
- replace plugin imports via `imp` with `utils.load_module`
- fix incorrect plugin module names
"streamlink.plugin.{name}" -> "streamlink.plugins.{name}"
- use `log.exception()` to print plugin load failures
- remove unused `print_small_exception`
Add / fix tests:
- remove session dependency from test_plugins
- add proper assertions for plugin modules, methods and their signatures
- add tests for invalid and for overridden plugins
The change is introducing two new optional command line options:
`--ffmpeg-fout`: The output file format when muxing with ffmpeg
'--ffmpeg-no-start-at-zero': When used with ffmpeg and copyts, disables automatically shifting input timestamps so they start at zero
Both options defaults are keeping the predefined values, hence backwards compatibility is enforced.
Co-Authored-By: beardypig <beardypig@protonmail.com>
Co-authored-by: Luigi Capriotti <l.capriotti@tiscali.it>
and remove compat key name translations in the Session options.
Removes the following CLI arguments:
- `--rtmpdump-proxy` (use `--rtmp-proxy` instead)
- `-c` / `--cmdline` (use `--subprocess-cmdline` instead)
- `-e` / `--errorlog` (use `--subprocess-errorlog` instead)
- `--erorlog-path` (use `--subprocess-errorlog-path` instead)
Keeps `--rtmpdump` as a fallback for `--rtmp-rtmpdump`, as the default
streamlinkrc config file of the Windows installer is still using the
old parameter name.
This removes the old plugin-specific arguments:
- `--funimation-mux-subtitles`
- `--pluzz-mux-subtitles`
- `--rtve-mux-subtitles`
- `--svtplay-mux-subtitles`
- `--vimeo-mux-subtitles`
and replaces it with a single global `--mux-subtitles` argument.
Also adds `mux-subtitles` to the Session options.
bastimeyer