- move remaining methods from `streamlink.utils` to `streamlink.utils.*`
- streamlink.utils.cache.LRUCache
- streamlink.utils.data.search_dict
- streamlink.utils.module.load_module
- streamlink.utils.rtmp.{escape_librtmp,rtmpparse}
- streamlink.utils.swf.swfdecompress
- fix imports in streamlink, streamlink_cli and tests
- move utils tests into new `tests.utils` package
- split up `test_utils` into individual test modules
- move and rename `test_utils_*`
- move and rename `test_localization`
- fix up some tests
- Add `Plugin.get_metadata()` which returns a dict of all metadata.
- Add `metadata` property to list of all resolved streams, but merge
all metadata properties with the specific stream output in order to
not introduce a breaking change in the JSON output format.
- turn ask() and askpass() into regular instance methods
- use correct output stream in ask() and askpass()
- return None in ask() and askpass() if not a tty or on caught exception
- remove args/kwargs from msg(), exit(), ask() and askpass()
- merge multiple objects and keywords in msg_json()
- support the same plugin metadata variables in `--output`, `--record`
and `--record-and-pipe` like they already exist in `--title`:
`{url}`, `{author}`, `{category}`/`{game}` and `{title}`
- substitute unsupported file name characters on specific OS-types
- add `--fs-safe-rules` CLI argument for overriding substitution logic
- add new Formatter class with caching and parameter translation
- create formatter instance in `handle_stream` and pass it to output
methods where player titles or file names need to be formatted
- rewrite test_cli_main tests
Co-Authored-By: Ian Cameron <1661072+mkbloke@users.noreply.github.com>
- always stream data from unencrypted segment downloads
- never stream data from cached initialization sections, as the
response content can only be consumed once when stream=True
- update --twitch-low-latency plugin argument and its description
Having multiple session options and CLI arguments for different stream
types which are already covered by generic options/arguments is not only
redundant, but also confusing.
A distinction between different stream types does only make sense when
multiple different stream types are available and the user needs to
explicitly set different values for each of them, but since it's not
always clear which stream type is returned when a stream is selected
via the "best" stream name synonym for example, this makes it even more
confusing. And it's redundant as well, since only one stream can be
selected anyway.
- deprecate `{dash,hds,hls}-segment-attempts`
in favor of `stream-segment-attempts`
- deprecate `{dash,hds,hls}-segment-threads`
in favor of `stream-segment-threads`
- deprecate `{dash,hds,hls}-segment-timeout`
in favor of `stream-segment-timeout`
- deprecate `{dash,hds,hls,rtmp,http-stream}-timeout`
in favor of `stream-timeout` (dash/http-stream were never used)
- fix `HTTPStream` and use `stream-timeout` instead of `http-timeout`
- suppress deprecated CLI arguments
- update help texts of generic CLI arguments
- fix docs and add entry to deprecations page
- Don't log when loading a config from a deprecated path when using the
`--config` argument
- Only load the first existing plugin-specific config file
- Keep config file loading order
- Add tests for setup_config_args
- wrap deprecated config and plugin paths in `DeprecatedPath`,
subclassed from pathlib.Path
- log info messages when loading deprecated configs or plugins
- return success from `Streamlink.load_plugins(path)` to be able to
log plugin loading messages in the main cli module
- use pathlib.Path instead of strings
- rename PLUGINS_DIR to PLUGIN_DIRS and turn into list
- call expanduser() for all `--plugin-dirs` dirs
- reformat CLI docs
- fix tests
- refactor streamlink_cli.main.{setup_logging,setup_console}
and split into setup_logger_and_console and setup_signals
- add LOG_DIR to streamlink_cli.constants
- pass filename to logger.basicConfig
- re-use write stream of logger in ConsoleOutput
- fix escaped chars for percent-formatted argparse help strings in docs
- add tests
- always exit streamlink_cli.main.main at the same function call, namely
log_current_arguments
- patch out dummy functions in context manager
- slice debug log mock calls due to common test exit function
- properly reset mocks
- rename streamlink_cli.main.check_root to log_root_warning
- add test for streamlink_cli.main.log_root_warning
- split into two classes with an abstract base class
- remove name parameter from class and generate proper names
- remove mode parameter from open() as it's always binary mode
- change type of NamedPipe.path from str to pathlib.Path
- add tests
This adds a second layer of plugin argument groups, so that the docs can
show menu items for each plugin which makes finding them is easier.
- make the doc's argparse extension read argparse groups recursively
- override argparse.ArgumentParser.format_help and show nested groups
- fix/update tests
The change is introducing two new optional command line options:
`--ffmpeg-fout`: The output file format when muxing with ffmpeg
'--ffmpeg-no-start-at-zero': When used with ffmpeg and copyts, disables automatically shifting input timestamps so they start at zero
Both options defaults are keeping the predefined values, hence backwards compatibility is enforced.
Co-Authored-By: beardypig <beardypig@protonmail.com>
Co-authored-by: Luigi Capriotti <l.capriotti@tiscali.it>
and remove compat key name translations in the Session options.
Removes the following CLI arguments:
- `--rtmpdump-proxy` (use `--rtmp-proxy` instead)
- `-c` / `--cmdline` (use `--subprocess-cmdline` instead)
- `-e` / `--errorlog` (use `--subprocess-errorlog` instead)
- `--erorlog-path` (use `--subprocess-errorlog-path` instead)
Keeps `--rtmpdump` as a fallback for `--rtmp-rtmpdump`, as the default
streamlinkrc config file of the Windows installer is still using the
old parameter name.
This removes the old plugin-specific arguments:
- `--funimation-mux-subtitles`
- `--pluzz-mux-subtitles`
- `--rtve-mux-subtitles`
- `--svtplay-mux-subtitles`
- `--vimeo-mux-subtitles`
and replaces it with a single global `--mux-subtitles` argument.
Also adds `mux-subtitles` to the Session options.
- Add `is_global` parameter to `streamlink.options.Argument`
Global plugin arguments reference Streamlink's global arguments by
name and cannot receive any other parameters, like `required`, etc.
- In `streamlink_cli.main.setup_plugin_args`, find the global argument
references of each plugin, set the available default values and add
the plugin to the argument's `plugins` list, which will be read by
the docs generator
- In `streamlink_cli.main.setup_plugin_options`, set the provided
global argument values on the plugin's `options` dictionary
- Refactor docs/ext_argparse
- Don't override argparse.ArgumentParser when importing the parser
- Add list of supported plugins to the parameter description if the
`plugins` list is set on the argparse action object
- Add/improve/fix tests
This enables the output of the input-stream's manifest/master URL via
`streamlink --stream-url URL` / `streamlink --json URL`
in addition to the already supported output when selecting a stream via
`streamlink --stream-url URL STREAM` / `streamlink --json URL STREAM`
This is primarily helpful for HLS streams with a variant playlist.
- Format time via datetime.strftime instead of time.strftime
This avoids needing to format the logrecord's msecs separately
- Change datefmt of trace level logging to "%H:%M:%S.%f"
- Remove redundant StringFormatter.format code
- Fix StringFormatter.usesTime() being called twice during format
- Calculate usesTime value only once (depends on format, not record)
- Add tests
- remove old Logger and LoggerModule classes
- remove old {,Compat}LogRecord classes
- remove deprecated and unneeded methods from StreamlinkLogger
- remove usage of old Logger from Session
- remove session reference from ConsoleOutput
As arguments can be loaded from a config file these need to be loaded as
soon as possible otherwise some options will not have the desired
effect. For example, `quiet` can be specified in `.streamlinkrc`,
however it will not take effect. With this change the default config
args are loaded earlier, however plugin specific config files are
still loaded later.
Options that will take effect from the config file now include:
- `loglevel`
- `json`
- `stream_url`
- `subprocess-cmdline`
- `quiet`
- `stdout`
- `output`
- `record-and-pipe`
- `plugins-dir`
The plugin specific config option behaviour remains unchanged, and
`loglevel` is the only option that is affected.
E126 continuation line over-indented for hanging indent
E127 continuation line over-indented for visual indent
E128 continuation line under-indented for visual indent
These checks can take a long time in the case of a typo in the url, for example.
This sets the return code to 130 so that external scripts can handle it
correctly.
* Add a record option
Add a record option for recording a stream while also playing it.
* Fix recording added in #920 by @grazzolini
- Remove `-r` as short option for `--rtmp-rtmpdump`
- Add options `--record` (`-r`) and `--record-and-pipe` (`-R`) as suggested by @bastimeyer in discussion on #920
- Add info message that the download started as suggested by @back-to in discussion on #920
- Add tests for `streamlink_cli.main.create_output` where these arguments are used
* cli: disable progress output for -o when no tty is available
Changes to the behaviour of the progress bar when using -o.
If there is no tty available for output (eg. when piping, or running a
subprocess), then no progress bar is output. This is because the
progress bar rewrites the console, which is only possible on interactive
terminals not for a pipe.
If there is no tty available for input and the file already exists an
error is given instead of a prompt to the user.
* Updated tests
- removed versionchange references as Streamlink does not use it.
- removed old code that was deprecated in Livestreamer
- removed DeprecatedWarning for 2.6 as it is already mentioned in
`setup.py`
a14f170a04/setup.py (L82)
- removed unittest2
- Flake8 for some lines/files that I changed.
* Largely untested, introduces custom title attributes pulled from site APIs
Included is support for {title}, {category}/{game}, and {author}
for Twitch and Youtube
Only tested (briefly) for Python 3 + Linux.
Committing to test on other platforms (Windows)
* bug-fix 1
* bug-fix 2
changing to list2cmdline turns out to not be necessary, since subprocess.call has already disabled the ability to break out by using `shell=False`
also broke launching on windows.
* bug-fix 3
move this to the pi3 area
* allow user to escape $ with \$
in case the user wants to use format codes.
comes at the expense of allowing streamers to insert format codes
that only appear for streamlink users, but is not a security risk
* API for plugins to request input from the user
* console: handle lack of TTY for user input
* New fatal plugin error, for unrecoverable errors
* tests: remove deprecated calls
* stream.dash: parser for dash manifest files
* stream.dash: stream player for dash with plugin to support dash:// prefixed urls
* cli.main: make sure that streams are closed on errors
* stream.dash: fix some parsing bugs
* stream.dash: tidy up the segment number generation
* plugins.dash: wip segment timeline
* stream.dash: update to segment timeline parsing
* stream.dash: py3 support
* stream.dash: raise an error for DRM protected streams
* stream.dash: fixes for timescaling and some segment templates
* docs: add DASHStream to docs with other Stream classes
* dash: fix for video only stream
* plugins.dash: fix bug where all URLs were matched
* stream.dash: fix issue with manifest reload
* plugin.dash: add tests and fix a couple of bugs found in testing
* stream.dash: add some tests to cover the DASHStream classes
* WIP: audio only streams
* add some debugging for threads and remove the thread joins
* dash: startNumber should default to 1
* dash: follow redirects to get the base url
* dash: fix bool parser, and segment template parser
* dash: fixed some issues...
...with some segment templates, as well as improving the presentation
delay handling
* dash: add a back-off for checking for manifest changes
* dash: fix broken tests
* dash: incomplete support for Segment@r
* dash: fixed audio/video sync issue
Added a `copyts` option to the FFMPEG muxer class so that the timestamps
given in the source files are maintained, this appears to fix the a/v
sync issues.
NB. The timestamp can get weird, but that's how it is :)
* dash: support for Time _and_ Number in segment timeline
* tests: add some dash parser tests + a little refactor
* tests: add dash to built in plugins
* tests: more coverage of dash_parser
Added a new module for tests, `freezegun`, for mocking time.
* dash: fix for missing publishTime
* dash: update available_at times to be datetime
This should fix any timezone or leap-second issues, etc.
* fixed timing issue for 1tv.ru
* dash: fix availability timeline for segment timeline
* dash: flake8 tweaks
* dash: add a few debug logging messages
* logging: refactor to use python logging module
The logging output remains identical to before. The API has changed,
instead of passing a Logger instance around, a module level `log`
instance is used. This is the standard way to do logging in most Python
applications and module.
There are a number of advantages to doing this:
1. more Pythonic
2. easier to add logging message from utils and other methods without
passing session or loggers around
3. if required, log messages from other Python modules can be enabled
4. the log destination could be changed to (or duplicated to)
other places, syslog for example
5. when other applications include streamlink they can get the
streamlink logging message
* tests: refactor deprecated warnings for Session.logger
* logging: added Logger compatibility class
* fix issue in argparse docs
* fix rebase issue
* de-flaked
* plugins: PluginArguments API
Allow plugins to specify arguments within the Plugin class.
This makes it easier to add and test plugin specific arguments, as well
as enabling sideloaded plugins to support command line arguments.
* update docs build to work with the new plugin arguments
* fix argument name override for funimation and ustream
* add some more tests for options/arguments
* rebase with abweb plugin
* switch to RuntimeException for py2 compat
* fix bug where required arguments were not prompted
* bug fixes for option names in plugins, should match existing names
* restore "normal" exception messages
* missed a RecursionError -> RuntimeError
* updated the api slightly with better (I think) argument name
added some docs for the Argument and Arguments classes
* options/args: normalise option names to replace - with _, and ensure the defaults are set
* normalise the plugin module name, replacing _ with -
* style issues pointed out by @back-to
* plugin.btsports: update to use PluginArguments
* plugins.bbciplayer: add hd argument
* plugins.twitch: move --twitch-oauth-authenticate back to argparser
Moved the option back to argparser as it's a special case.
* test: fixed typo
New option added to enable HD streams for some live channels (BBC One,
BBC Two): `--bbciplayer-hd`. Note that not all programmes are
broadcast in HD.
Sped up the stream enumeration by ignoring duplicate manifest entries,
reduced number of stream looks from ~70 to ~8 on live streams.
* [ABweb] New plugin for BIS Livestreams of french AB Groupe
> AB Groupe
> The AB is a French business group in the field of broadcasting.
> https://en.wikipedia.org/wiki/AB_Groupe
Requires a login and a subscription.
Username/Password is only required for the first login every 24h,
it will cache the required cookies and force a relog every 24h.
```
$ streamlink
"http://www.abweb.com/BIS-TV-online/bistvo-tele-universal.aspx?chn=ab1"
best
[cli][info] Found matching plugin abweb for URL
http://www.abweb.com/BIS-TV-online/bistvo-tele-universal.aspx?chn=ab1
[plugin.abweb][info] Attempting to authenticate using cached cookies
[cli][info] Available streams: 75k (worst), 240p, 360p, 432p, 720p
(best)
[cli][info] Opening stream: 720p (hls)
[cli][info] Starting player: /usr/bin/vlc
[cli][info] Player closed
[cli][info] Stream ended
[cli][info] Closing currently open stream...
```
http://www.abweb.com/BIS-TV-Online/http://www.abweb.com/BIS-TV-Online/bistvo-tele-universal.aspx
* [ABweb] set self._expires cache correctly
* [ABweb] changes for site update
A generated m3u8 url expires after 24h
- removed --jtv and --gomtv commands
- removed websites that are dead
- removed websites that redirectes to a complete different website than the streamlink Plugin
if someone still wants to use the new website,
open a new Plugin Request the old websites are already broken for a long time.
| file | domain | status |
| --- | --- | --- |
| apac.py | a-pac.tv | redirected to different website |
| bambuser.py | bambuser.com | Bambuser community has ceased its services. |
| connectcast.py | connectcast.tv | 401 login - dead? |
| daisuki.py | daisuki.net | offline and redirected to different website |
| disney_de.py | disneychannel.de | redirected to different website - new website/domain |
| dmcloud.py | api.dmcloud.net | offline |
| dmcloud_embed.py | - | dmcloud is offline |
| gomexp.py | gomexp.com | redirected to different website |
| streamupcom.py | streamup.com | stay tuned since 2016 - dead |
| wattv.py | wat.tv | New website already covered by TF1 Plugin |
* E265 - Fix spacing after comment hash for block comments.
* W293 - Remove trailing whitespace on blank line.
* E305 - Add missing 2 blank lines after end of function or class.
* E123 - Align closing bracket to match opening bracket.
* E202 - Remove extraneous whitespace before bracket.
* Remaining fixes.
You can ignore segment names that might corrupt or lag your output
- this will freeze the player until there is a valid segment name
- this will not add the segment to an output file,
so it won't be damaged and can be edited.
Note: The `--hls-timeout` must be increased, to a time that is longer
than the ignored break.
Example:
`streamlink URL best --hls-segment-ignore-names 000,001,002
--hls-timeout 1200`
This will ignore every segment that ends with 000.ts, 001.ts and 002.ts
Maybe not exactly what was requested, but this won't break any third
party applications
Example:
```
[cli][debug] OS: Linux-4.14.4-1-ARCH-x86_64-with-arch
[cli][debug] Python: 3.6.3
[cli][debug] Streamlink: 0.9.0
[cli][debug] Requests(2.18.4), Socks(1.6.7), Websocket(0.44.0)
```
Fixed https://github.com/streamlink/streamlink/issues/1323
The deprecation warning has been removed from the
--http-{cookies,headers,query-params} options. The options are not
listed in the help or in the docs, but they will continue to work.
They should be considered unsupported.
* [zatto] New plugin for zattoo.com / tvonline.ewe.de / nettv.netcologne.com
* [zattoo] Better debug/error messages and use channel as cid if no cid.
streamlink/streamlink#804
* Fixed travis 20 character limit error and removed tabs from other plugins
/home/travis/build/streamlink/streamlink/docs/plugin_matrix.rst:265: (ERROR/3) Malformed table.
Text in column margin in table line 254.
* [zattoo] --zattoo-purge-credentials
You can reset your session with --zattoo-purge-credentials
Add two new arguments to support authentication:
- `--ufctv-username`: The username used to register with ufc.tv.
- `--ufctv-password`: A ufc.tv account password to use with
--ufctv-username.
As logging in will soon become a requirement, this PR lays the ground
work for that change. The login is not currently enforced, when it is
the plugin will need to be updated again.
bastimeyer