Commit Graph

191 Commits

Author SHA1 Message Date
bastimeyer 45e515eb5a session/plugin: fix DeprecationWarning stacklevel 2023-03-24 09:41:23 -07:00
bastimeyer 5e6f03c3cd chore: add "B" rules to ruff config 2023-03-24 09:41:23 -07:00
bastimeyer 49988a0923 chore: aware datetimes
- Add "DTZ" ruff rule
- Add utility functions to `streamlink.utils.times` which use
  "aware" datetimes with explicit timezone information,
  and use `isodate`'s local timezone implementation
- Replace all "naive" datetimes without timezone information
- Replace all custom ISO8601 parsers with `isodate`'s implementation
- Add tests for new utility functions
2023-03-01 09:59:47 -08:00
bastimeyer 9cfe62d03c chore: add "PL{C,E,W}" rules to ruff config 2023-02-20 14:10:06 -08:00
bastimeyer 4622c9728e chore: add "ISC" rules to ruff config 2023-02-09 11:48:40 -08:00
bastimeyer 17daf563a3 chore: add "C4" rules to ruff config 2023-02-09 11:48:40 -08:00
bastimeyer 57fa6f80e8 chore: add "COM" rules to ruff config 2023-02-09 10:26:50 -08:00
bastimeyer 0589750c8f options: deprecate global plugin arguments
- Raise `StreamlinkDeprecationWarning` when `is_global=True`
- Remove all global plugin arguments and replace them with simple
  option lookups on the Streamlink session instance
- Remove global argument detection in custom Sphinx plugins extension
- Remove supported plugins list from custom Sphinx argparse extension
  and remove respective setup from `setup_plugin_args` in the CLI module
- Add deprecation note to the docs
- Update tests
  - Check whether builtin plugins define global plugin arguments
  - Capture deprecation warnings
2023-02-03 10:21:31 -08:00
bastimeyer 777d1be3de logger: add StreamlinkWarning
- Add `StreamlinkWarning` and `StreamlinkDeprecationWarning`
  and replace `FutureWarning`s
- Don't include the warning's origin in the warning logger if it's a
  subclass of `StreamlinkWarning`
- Update tests
2023-01-12 15:54:18 -08:00
bastimeyer d5b2981fbe logger: turn deprecation log msgs into warnings
- Replace all deprecation warning log messages with `FutureWarning`s
- Fix deprecated session options in tests
- Update and fix tests
2023-01-12 15:54:18 -08:00
bastimeyer a76cefe7ca logger: capture warnings
Add support for capturing warnings and logging them via Streamlink's
root logger on the warning log level. Use custom `WarningLogRecord`s
with a custom warning message format, and replace the record's
logger name with "warnings" and use the name of the warning type
as log level name when showing warnings.

This enables having proper `DeprecationWarning`s and `FutureWarning`s
in the code instead of just using the logger and its warning log level,
and those warnings can be filtered via the regular filtering mechanisms.

For example:
```
[warnings][deprecationwarning] Calling this method is deprecated
[warnings][futurewarning] Using this config file path is deprecated
```
2023-01-12 15:54:18 -08:00
bastimeyer b1c6d8bffa cli: replace read_stream() with StreamRunner
Replace `read_stream()` with a new `StreamRunner` class, refactor stream
reads and output writes, as well as the progress thread data feeding,
and move player polling into a separate thread which closes the stream
once the player process gets terminated/killed.

This fixes the player polling issue, or rather the detection of its
broken pipe while reading the stream, as stream read calls can stall
for various reasons, e.g. due to segmented streams or the filtering
of stream data which pauses the stream output and disables any timeouts.

- Implement `StreamRunner` and `PlayerPollThread` classes
  in dedicated `streamlink_cli.streamrunner` module
- Remove old `streamlink_cli.main.read_stream()` implementation
- Keep the same log messages
- Make `StreamRunner.run()` raise `OSError` on read/write error and
  catch `OSError`s in main module where `console.exit()` gets called
- Remove `Progress.iter()`, as it's not needed anymore
- Add extensive tests with full code coverage
2023-01-01 10:07:37 +01:00
bastimeyer 3d02f6c9cd cli: fix log format for log level "all" 2022-11-13 20:30:39 +01:00
bastimeyer ca4fff1453 chore: fix new mypy warnings and errors 2022-11-07 15:16:51 -08:00
bastimeyer 87a84424a3 chore: lists
- collection-builtin-to-comprehension
- list-comprehension
- identity-comprehension
- for-append-to-extend
2022-09-02 20:23:20 -07:00
bastimeyer 1c731644b4 cli: remove custom `ignored` contextmanager
and replace it with `contextlib.suppress`
2022-09-02 11:36:20 -07:00
bastimeyer 075cf32a57 cli: refactor setup of session options
- Merge `setup_options()` and `setup_http_session()` and always
  initialize all session options:
  It doesn't make sense having this split up into two functions, just
  so that some CLI functions can have a partially configured session
  with just the HTTP options set
- Move merged session setup logic into the `argparser` module
  and replace the dozens of if-statements and `session.set_option()`
  calls with a list of "CLI argument -> session option" mappings
- Add tests and update `streamlink_cli.main` tests respectively
2022-09-02 00:24:13 -07:00
bastimeyer 7e5aa3dda5 cli: refactor version check
- Move to separate module (slim down main module)
- Catch and log errors
- Don't call `sys.exit()` and instead alter flow of `main.main()`
  if `--version-check` is set
- Handle `KeyboardInterrupt`
- Add tests
2022-09-01 19:22:53 -07:00
bastimeyer f020259832 logger: log deprecation messages as warnings 2022-08-29 22:56:58 -07:00
bastimeyer 6ec535f88f plugin: remove Plugin.bind()
This changes the way how the Streamlink session and other objects like
the plugin cache and logger are stored on each plugin.

Previously, those objects were set as class attributes on every `Plugin`
class via `Plugin.bind()` when loading plugins via the session's
`load_plugins()` method that gets called on initialization.

This meant that whenever a new Streamlink session was initialized,
references to it (including a dict of every loaded plugin) were set
on each `Plugin` class as a class attribute, and Python's garbage
collector could not get rid of this memory when deleting the session
instance that was created last.

Removing `Plugin.bind()`, passing the session via the `Plugin.__init__`
constructor, and setting the cache, logger, etc. on `Plugin` instances
instead (only one gets initialized by `streamlink_cli`), removes those
static references that prevent the garbage collector to work.

Since the plugin "module" name now doesn't get set via `Plugin.bind()`
anymore, it derives its name via `self.__class__.__module__` on its own,
which means a change of the return type of `Streamlink.resolve_url()`
is necessary in order to pass the plugin name to `streamlink_cli`,
so that it can load config files and initialize plugin arguments, etc.

Breaking changes:

- Remove `Plugin.bind()`
- Pass the `session` instance via the Plugin constructor and set the
  `module`, `cache` and `logger` on the plugin instance instead.
  Derive `module` from the actual module name.
- Change the return type of `Session.resolve_url()` and include the
  resolved plugin name in the returned tuple

Other changes:

- Remove `pluginclass.bind()` call from `Session.load_plugins()` and
  use the loader's module name directly on the `Session.plugins` dict
- Remove initialization check from `Plugin` cookie methods
- Update streamlink_cli.main module according to breaking changes
- Update tests respectively
  - Add explicit plugin initialization test
  - Update tests with plugin constructors and custom plugin names
  - Move testplugin override module, so that it shares the same module
    name as the main testplugin module. Rel `Session.load_plugins()`
  - Refactor most session tests and replace unneeded `resolve_url()`
    wrappers in favor of calling `session.streams()`
2022-08-28 12:24:56 -07:00
bastimeyer 880ac1e2b3 chore: use f-strings for concatenation 2022-08-27 11:33:38 +02:00
bastimeyer b5df8aadfe cli.utils.progress: add output path to formats
- Add `path` argument to the `Progress` class
- Update formats and add path variable with a min-width of 15
- Truncate paths according to the available space
- Add and update tests
2022-08-24 10:25:40 -07:00
bastimeyer 2699ca9024 cli.utils.progress: refactor
- Remove `streamlink_cli.utils.terminal` module again and move logic
  to the `Progress` and `ProgressFormatter` classes
- Replace `output` and `formatter` arguments of the `Progress` class
  with a simple `stream` argument
- Rewrite `ProgressFormatter.format()`
  - Pre-parse format strings via `string.Formatter.parse()`
    and store the parsing results
  - Add support for substitutions with dynamic/variable lengths
- Update tests
2022-08-24 10:25:40 -07:00
bastimeyer 8c1430ea05 cli: implement --player-external-http-continuous 2022-08-21 10:13:00 +02:00
bastimeyer e529337acc plugin: set Plugin.arguments class attr to None
Don't let plugins with no arguments share a common Arguments instance
2022-08-18 20:27:30 -07:00
bastimeyer 349c4ca8e2 refactor: is_darwin/is_win32 compat exports
- Move `is_darwin` from `streamlink_cli.compat` to `streamlink.compat`
- Remove `is_win32` from `streamlink_cli.compat`
- Replace last remaining `unittest.skipIf` test decorators with the
  `posix_only`/`windows_only` pytest decorators from the `tests` module
- Remove `is_win32` check when importing from `ctypes` in `named_pipe`
2022-07-27 09:53:07 -07:00
bastimeyer 424aac28f0 cli: rewrite progress output
- Split `streamlink_cli.utils.progress` into `progress` + `terminal`
  - `terminal`:
    Responsible for calculating character output widths and for writing
    data to the output text stream
  - `progress`:
    Responsible for calculating the download speed and for formatting
    the progress output
- Use a separate thread for the progress output, with the output being
  written in constant time intervals, independent of the data that gets
  fed by the stream iterator of the main thread
- Output format changes:
  - Remove "prefix" from output format (full path already gets logged)
  - Avoid showing download speeds until a time threshold is reached
  - Use binary values for file size and download speed values
  - Add leading zeros to time units (only ever grow output text size)
- Don't import from alias module in `streamlink_cli.main`
- Split, move and rewrite tests
2022-07-17 11:54:10 -07:00
bastimeyer 5985f08570 cli: refactor nested argparse argument groups
- Use a custom mapping of parent->child argument group relations
  instead of calling `group.add_argument_group()` for creating
  a tree structure of argument groups (deprecated in Python 3.11)
- Override `parser.add_argument_group()` and add the `parent` keyword
  for being able to have nested argument group defintions
- Update custom `--help` format output
- Update the argparser Sphinx extension
- Fix tests
2022-07-12 09:42:09 -07:00
bastimeyer 1685c6caf2 cli: fix pluginerror in handle_url if json is True 2022-06-13 08:44:01 +02:00
bastimeyer c319aa445e cli: list all dependencies in debug output
- Require importlib-metadata as fallback on Python < 3.8
- Add importlib_metadata to streamlink_cli.compat
- List all dependencies in `log_current_versions`
- Update tests
2022-06-06 08:44:02 +02:00
bastimeyer 8fc178276e chore: fix import typing issues 2022-05-24 14:07:47 -07:00
bastimeyer 3f508d1bed cli: fix typing issues 2022-05-22 19:30:15 +02:00
bastimeyer ecb21f9781 stream: refactor to_url and string representation
- Change format of Stream string representations and error messages
  of `to_url` and `to_manifest_url`
- Don't override `__repr__` in `Stream` subclasses
- Raise TypeError if url or manifest URL is None
- Fix CLI
  - Abort passthrough if stream can't be translated to URL
  - Remove unneeded streamlink_cli.utils.stream module
- Rewrite stream URL tests
- Move FilmOnHLS stream URL tests to plugin test module (and rewrite)
2022-05-13 21:07:32 +02:00
Alexis Murzeau 081595cfed tests: mock user euid to be able to run tests as root
When running test as root, there is an additional log done to warn the
user that it is running streamlink as root. This causes some logging
tests to fail.

For these logging test, ensure we run with a mocked user euid when not
testing for the root euid.
2022-05-09 23:13:50 +02:00
bastimeyer 6325c74e68 chore: remove unnecessary collection.OrderedDict
- Replace collection.OrderedDict with builtins.dict where possible:
  Python 3.7+ ensures the correct order in builtins.dict objects and is
  no longer an implementation detail of cpython.
- Fix OrderedDict type annotation in streamlink.utils.cache.LRUCache
- Add unit test for streamlink.utils.cache.LRUCache
2022-04-17 11:06:29 -07:00
bastimeyer bfd0933675 cli: add support for --record=-
Write stream to stdout while also opening it in the player.
2022-04-17 13:16:33 +02:00
bastimeyer 5b78085fd8 cli: add `plugin` metadata variable 2022-04-07 18:46:36 -07:00
bastimeyer 40b995aa7b cli: log resolved file output path 2022-02-07 22:02:13 -08:00
Mozi 665f76ab99
cli: tell users the stream could be saved or piped
- change missing player exit message, suggest --output / --stdout
- refactor create_output
- add missing test case

Co-authored-by: bastimeyer <mail@bastimeyer.de>
2022-01-23 21:17:58 +01:00
bastimeyer 59496acce6 cli: create file output before opening the stream 2021-12-14 09:26:05 -08:00
bastimeyer 52820c4107 cli: prioritize --help and fix its output 2021-11-26 15:11:45 -08:00
bastimeyer 01df2f00c8 plugin: add 'id' metadata property 2021-11-24 18:41:21 +01:00
bastimeyer 6ef5605e76 cli: override default signal handlers
- Don't raise a KeyboardInterrupt until streamlink_cli has been fully
  initialized. This prevents a stack call from being printed to stderr
  when streamlink_cli gets interrupted early.
- Restore default signal handlers once streamlink_cli has finished its
  initialization, so that KeyboardInterrupt exceptions can be caught to
  perform clean up tasks.
- Restore default signal handlers in test immediately after importing
  streamlink_cli once as early as possible, to be able to cancel the
  test runners regularly via a KeyboardInterrupt.
- Refactor CLI logging tests, which test the log output of the
  initialization and need to stop execution at some point.
2021-11-20 11:02:19 -08:00
bastimeyer a7063e1d70 stream: remove RTMP and RTMPDump dependency
- drop RTMP stream implementation
- drop RTMP plugin
- drop RTMPDump dependency
- remove stream.streamprocess and utils.{rtmp,swf}
- remove "rtmp" from default stream types list
- remove "rtmp" from player-passthrough options list
- remove all `--rtmp*` CLI args and `rtmp-*` session options
- remove all `--subprocess-*` CLI args and `subprocess` session options,
  as these were used only by StreamProcess which is no longer needed
- update tests
- update docs and CLI argument help texts
2021-11-13 13:24:25 +01:00
bastimeyer 46817a1cff streams: remove HDS/AkamaiHD and flashmedia pkg
- drop HDS and AkamaiHD stream implementations
- drop HDS and AkamaiHD plugins
- drop streamlink.packages.flashmedia
- remove stream.flvconcat, stream.playlist and plugins.common_swf
- remove "hds" from default stream types list
- remove all `--hds-*` CLI arguments and `hds-*` session options
- remove unneeded flashmedia license text files
- update tests
- update docs and CLI argument help texts
2021-11-12 20:09:53 +01:00
bastimeyer 0ed1d6731c cli.main: add plugin type annotations 2021-11-12 16:23:24 +01:00
bastimeyer 26f115f7eb session.resolve_url: return plugin class + URL
Breaking change:
Instead of resolving a plugin instance in `Streamlink.resolve_url(url)`
from the provided input URL (which can be redirected), resolve the
plugin class and the resulting URL, and cache the tuple. Also affects
`Streamlink.resolve_url_no_redirect(url)`.

The main reason for this change is streamlink_cli and how the plugin
options get set. The plugin options need to be set on the class before
it gets initialized, so that the instance can access the options in its
constructor method. This also fixes any kind of state stored on the
plugin instance.

- resolve and cache a tuple of the plugin class and resulting URL
- initialize plugin in streamlink_cli after applying its options
- remove plugin variable from global scope in streamlink_cli
- fix, rewrite and add tests
2021-11-12 16:23:24 +01:00
bastimeyer 47b715a369 revert: stream.hls: remove hls-segment-stream-data option
This reverts commit ccdd84bf76

- resolve merge conflicts
- keep stream=False parameter in HLSStreamWriter.fetch_map
- keep updated help text of --twitch-low-latency
2021-11-09 18:17:55 +01:00
bastimeyer e152c539eb cli: refactor FileOutput and Formatter
- add streamlink_cli.utils.path.replace_path
- refactor streamlink_cli.utils.formatter.Formatter
  - rename filename() to path() and return a pathlib.Path object
  - format path parts separately
  - replace dot and double-dot path parts if not explicit user input
  - replace unsupported chars in path parts (like before)
- refactor streamlink_cli.output.FileOutput
  - turn filename into a Path object
  - create parent directories before creating the file descriptor
- update streamlink_cli.main accordingly
- update and add new tests
2021-10-10 20:26:49 +02:00
bastimeyer a0437c2e7b stream: clean up imports 2021-09-20 15:31:39 +00:00