1
mirror of https://github.com/streamlink/streamlink synced 2024-10-30 23:16:39 +01:00
Commit Graph

3238 Commits

Author SHA1 Message Date
bastimeyer
5e6f03c3cd chore: add "B" rules to ruff config 2023-03-24 09:41:23 -07:00
bastimeyer
786f0f0be1 plugins.twitcasting: rewrite plugin
- Refactor API query
- Add HLS streams, but prefer websocket streams due to lower latency
- Enforce TLS for HLS and websocket streams
2023-03-21 14:46:50 +01:00
bastimeyer
98355078f1 stream.dash: fix segment duration
Fix segment duration in representations without a segment list
or segment template and add test for segments of this kind.
2023-03-18 11:47:53 -07:00
Mehavoid
88772da939 plugins.vkplay: fix category null 2023-03-18 10:12:25 +01:00
bastimeyer
17493ba20b stream.dash: refactor SegmentList+SegmentTemplate
- Make `SegmentBase`, `SegmentList` and `SegmentTemplate` inherit from
  the common `_SegmentBaseType` and `_MultipleSegmentBaseType` classes
- Logically move class definitions
- Define common attributes in `_{,Multiple}SegmentBaseType`
  that get inherited from nodes of the same type in ancestor nodes,
  and find common child nodes
- Find inherited attributes independently
- Fix, clean up, reorder and selectively add attributes
  - Add `_SegmentBaseType.availabilityTimeOffset` (currently unused)
- Fix `MPD.minBufferTime` type (required attribute which can't be None)
2023-03-16 09:32:37 -07:00
bastimeyer
c04048d52e stream.dash: refactor AdaptationSet+Representation
- Make `AdaptationSet` and `Representation` inherit from the common
  `_RepresentationBaseType` class
- Move and logically group class definitions
- Define common attributes in `_RepresentationBaseType` that get
  inherited from ancestor nodes of the same base type, and find
  common child nodes
- Fix, clean up, reorder and selectively add attributes
  - Remove `Representation.numChannels` (invalid and unused)
  - Turn `Representation.lang` into a property which reads its value
    from the parent `AdaptationSet.lang`
    (it's not an attribute on `Representation` according to the spec,
    but it gets read by `DASHStream.parse_manifest`, so keep the alias)
  - Rename `contentProtection` attribute to `contentProtections`
    and `subRepresentation` to `subRepresentations`, to stay consistent
    with other attributes of child node lists
- Fix tests with old assertions: non-inhertied attributes that led to
  different expectations (e.g. the resulting stream name)
2023-03-16 09:32:37 -07:00
bastimeyer
fae1be42f3 stream.dash: inherit attrs from specific ancestors
Refactor `walk_back()`:
- compare ancestor nodes by type and not by tag name
- rename mapper parameter and check its result before yielding the node

Refactor `walk_back_get_attr()`:
- allow finding attributes from specific ancestor nodes only
- stop iterating after the first match

Refactor `attr(inherited=...)`:
- allow inheriting from ancestor nodes, optionally from a specific type,
  and not just the parent node
2023-03-16 09:32:37 -07:00
bastimeyer
b2159a98a1 stream.dash: find ancestor SegmentTimeline nodes
- Fix `SegmentTimeline` in `SegmentTemplate` and find ancestor timelines
- Add `SegmentTimeline` to `SegmentList` (currently unused)
2023-03-14 09:18:48 -07:00
bastimeyer
c5f2789202 stream.dash: make SegmentList.SegmentURL optional
The number of `SegmentURL` nodes in `SegmentList` can be zero, e.g. when
the `SegmentList` gets overridden in a descendant node
2023-03-14 09:18:48 -07:00
bastimeyer
456dd53b7d stream.dash: inherit SegmentList attributes
Inherit attributes from ancestor `SegmentList` nodes, if set,
similar to `SegmentTemplate`.

Also fix `presentationTimeOffset` not being inherited in
`SegmentTemplate`.

TODO: use common `MultipleSegmentBaseType` class for these attributes
2023-03-14 09:18:48 -07:00
bastimeyer
23b8af5b15 stream.dash: fix SegmentList child nodes
- Turn `SegmentList` into an only-child in `Period`, `AdaptationSet`
  and `Representation`
- Set `parent` type in `SegmentList`
- Turn `SegmentList.segments` from a property into a regular method,
  similar to `SegmentTemplate.segments()`
2023-03-14 09:18:48 -07:00
bastimeyer
a866a374cc stream.dash: refactor segment timeline generator
- Move segment timeline logic from `SegmentTemplate.format_media()`
  into the new method `SegmentTemplate.segment_timeline()`, similar to
  `SegmentTemplate.segment_numbers()`
- Simplify segment timeline generator
  - Use a threshold `datetime` for the presentation delay
  - Don't collect every segment on subsequent manifests
- Include segment number for timeline segments (for the debug log)
- Rewrite continued dynamic timeline test (same results)
2023-03-13 09:32:31 -07:00
bastimeyer
f635705cbe cli.output: use subprocess.DEVNULL in PlayerOutput
Instead of passing `os.devnull` file-handles to `subprocess.Popen`,
use the `subprocess.DEVNULL` identifier for stdout/stderr streams.

This avoids `ResourceWarning`s in tests when the file-handles don't get
closed despite never calling `PlayerOutput.open()`, as they were already
opened in the `PlayerOutput`'s constructor, e.g. during test collection
in parametrized tests without indirect fixtures.

Also call `PlayerOutput.close()` in tests where a *mocked* player output
was explicitly opened.
2023-03-12 12:16:21 -07:00
bastimeyer
33d9790b07 utils.args: update num() argparse utility function
- Replace `min` and `max` with `ge`, `gt`, `le` and `lt`
- Update and fix params/values of numeric CLI argparser arguments
- Rewrite `num()` tests
2023-03-12 01:32:29 +01:00
bastimeyer
142d17f819 stream.segmented: properly fix wait() in writer
Follow-up of f234c690

Properly fix the `SegmentedStreamWriter`'s `_wait` Event when closing
the thread.

Depending on which thread attempted to close the writer thread,
the `close()` method's `self.reader.close()` call can block, leading
to the `_wait` event to not get set until all threads have terminated
and joined. So make sure to set the event first, so that all threads
of the writer's thread-pool executor can be stopped first,
for example segment downloads waiting for the segment's availability.
2023-03-11 04:10:38 +01:00
bastimeyer
bd7d15c799 stream.dash: refactor Segment, improve debug log
- Add `number` attribute to `Segment` and move `available_at` further up
- Pass all attributes on each `Segment` initialization
- Return "initialization", the segment number or the URL's path name
  as the segment's `name`
- Log the segment `availability` time in addition to the current time
- Use better debug log messages while fetching segments
2023-03-10 16:20:02 -08:00
bastimeyer
b03093558a stream.dash: refactor datetime import
- Import `datetime` and `timedelta` from `datetime` in order to
  improve readability and avoid wrapping of long lines
- Use `streamlink.utils.times` utilities and constants
2023-03-10 16:20:02 -08:00
bastimeyer
f234c69022 stream.segmented: fix wait() in writer thread-pool
The `SegmentedStreamWriter`'s `_wait` event needs to be set before
awaiting the shutdown of the thread-pool executor, otherwise the threads
of the executor won't ever receive this signal. For example, queued
segment downloads can't be cancelled because the event never gets set
before awaiting the shutdown of the executor, so `wait()` calls in the
executor's threads time out regularly and return `True`.
2023-03-10 16:19:17 -08:00
bastimeyer
d388acd6d9 stream.dash: fix segment number iterator 2023-03-10 01:36:44 +01:00
bastimeyer
49682ddda4 plugins.steam: fix validation schema 2023-03-09 14:37:00 -08:00
Oleg Koloskov
7f2d83b2bc
plugins.vimeo: fix playerConfig regex (#5227) 2023-03-09 12:30:51 +01:00
bastimeyer
e0e13b646b stream.dash: fix period/mediaPresentation duration
Calculate the `total_seconds()` and don't just read the `seconds` value
2023-03-08 15:41:15 -08:00
bastimeyer
3178ac3a03 stream.dash: refactor Segment availability
Add `available_in` property to `Segment` and add tests
2023-03-07 18:42:13 -08:00
bastimeyer
4ed7be94a0 stream.dash: add period param to parse_manifest()
- Add `period` parameter to `DASHStream.parse_manifest()`:
  This can be used by the DASH plugin, where parameters can be passed
  from the input URL
- Remove `period` parameter from the `DASHStream` constructor
  and reference the period from the chosen `Representation`
- Move `get_representation()` from `DASHStreamWorker` to `MPD`
- Update tests
2023-03-07 13:09:52 -08:00
bastimeyer
4bd1bf87cb stream.dash: refactor DASHStream.parse_manifest()
- Move MPD manifest fetching+parsing logic to separate methods
- Rewrite tests based on pytest
2023-03-07 13:09:52 -08:00
bastimeyer
ff654873bb plugins.steam: fix missing CDN auth URL params
CDN auth data is only required at certain times of the day,
and if missing, DASH segment requests result in a 403 response.

Also remove retry-loop with `time.sleep()` calls when trying to access
a stream while it's in the waiting state. Just return `None` and let
the user try again on their own.
2023-03-06 11:18:59 -08:00
bastimeyer
236116bbc9 stream.dash: typed MPDNode constructors
- Add typing annotations to all `MPDNode` children
- Fix various typing issues
  - Override optional types of required attributes
  - Add checks for optional stuff in segment processing
  - Fix processing of empty node texts
  - Fix typing in `DASHStream` caused by enabling typing in `MPDNode`s
2023-03-06 09:32:46 -08:00
bastimeyer
6c824eb304 stream.dash: change signatures of MPDNode+children
- Fix order of `*args` and keywords
- Remove `*args` from `MPDNode`
- Capture `node`, `root` and `parent` in `*args` and `**kwargs`
- TODO/PEP570: set positional-only and keyword-only parameters (py38)
2023-03-06 09:32:46 -08:00
bastimeyer
f565f8d89f stream.dash: overload typing of MPDNode.attr()
Add overloaded typing information to `MPDNode.attr()` to be able to
have more precise typing of return values. Return values of required
attributes are still typed as optional and need to be overridden.

Also change the order of parameters.
2023-03-06 09:32:46 -08:00
bastimeyer
dfff6a9900 stream.dash: optional Segment duration 2023-03-06 09:32:46 -08:00
bastimeyer
29bdea792c stream.dash: refactor Segment name 2023-03-06 09:32:46 -08:00
bastimeyer
ec8b6af442 stream.dash: refactor fetch attempts in writer
Don't fetch recursively, just pass the `retries` parameter to `get()`
2023-03-06 09:32:46 -08:00
bastimeyer
a55d803c2c stream.dash: fix dynamic timeline-less streams
- Fix segment number iterator and segment availability-time iterator
  for dynamic timeline-less DASH streams by calculating the right
  segment number offset and basing the segment availability-time on
  the segment number, and not just on the current time alone
- Set a default value of 0 seconds for presentationTimeOffset
- Ensure that segment durations are known
- Never calculate negative segment number offsets
- Reduce unnecessary delay by starting with the upcoming segment in the
  generated timeline to be as close to the live-edge as possible
  (with minBufferTime and presentation offset+delay in mind)
- Add debug log output
- Update MPD manifest test fixture with better time data

Next:

- Consider removing the default value of suggestedPresentationDelay,
  as it seems to be a relict of Streamlink's initial DASH implementation
- Also take a look at properly synchronizing the generated timeline
  between multiple substreams, so that the correct segment numbers
  always get calculated in all threads
- Avoid interweaving debug log and make it flush per-thread
2023-03-05 12:09:03 -08:00
bastimeyer
350cb58a46 stream.dash: fix suggestedPresentationDelay
- Convert `isodate.Duration` instances to `datetime.timedelta`
  in `SegmentTemplate.format_media()` in order to be able to compare
  with `datetime.datetime` in the timeline generator
- Keep `isodate.Duration` or `datetime.timedelta` instances
  in `SegmentTemplate.segment_numbers()` before subtracting from
  `since_start`, as `isodate.Duration()` can't be converted to seconds
  without a reference `datetime.datetime`
- Replace all `isodate.Duration()` attribute default values with
  `datetime.timedelta()` for simplicity/performance reasons
- Set the 3 seconds default value of `MPD.suggestedPresentationDelay`
  on the attribute's default definition instead of using constants
  in the `SegmentTemplate` methods
2023-03-04 13:07:42 -08:00
bastimeyer
85b5e857a5 stream.dash: log mime-type of segment downloads 2023-03-04 12:00:11 -08:00
bastimeyer
651739f236 stream.dash: fix segment availability times
The segment availability "anchor time" depends on the sum of the
manifest's `availabilityStartTime` and the period's `start` attribute,
for both static and dynamic manifests.

Fix segment availability times:
- Set the correct `available_at` value for static manifests with segment
  templates and segment timelines instead of using the current time
- Set the `available_at` value for `SegmentList` segments instead of
  defaulting to `EPOCH_START`
- Set the `available_at` value for all initialization segments

Fix segment numbers:
- The number offset now also takes the period start into consideration

Also:
- Allow passing keywords to child node constructors
- Keep the `Period` reference on `SegmentList`, `SegmentTemplate`
  and `Representation`
- Check segment availability times in certain tests
- Rename DASH manifest fixture files of updated tests
2023-03-02 11:37:24 -08:00
Pablo Hoffman
1b9ce00454
plugins.telemadrid: new plugin (#5212) 2023-03-02 19:16:39 +01:00
bastimeyer
49988a0923 chore: aware datetimes
- Add "DTZ" ruff rule
- Add utility functions to `streamlink.utils.times` which use
  "aware" datetimes with explicit timezone information,
  and use `isodate`'s local timezone implementation
- Replace all "naive" datetimes without timezone information
- Replace all custom ISO8601 parsers with `isodate`'s implementation
- Add tests for new utility functions
2023-03-01 09:59:47 -08:00
streamlinkbot
86002adbce plugin.api: update useragents 2023-03-01 01:22:13 +01:00
bastimeyer
0ad0a4183a stream.dash: add --dash-manifest-reload-attempts 2023-02-28 12:55:05 -08:00
bastimeyer
9d853b3896 stream.dash: refactor DASHStream{Reader,Worker}
Use the newly added `Representation.ident` for finding the same
`Representation` after a manifest reload of a dynamic DASH stream.

Also use the worker's `period` attribute instead of the hardcoded
zero index, even though only the first period is supported.
2023-02-27 23:56:36 -08:00
bastimeyer
1373b1609b stream.dash: fix old timeline ID workaround
When building segment timelines, don't use the `mimeType` attribute
of the `SegmentTemplate`'s parent node as a fallback when the parent's
`id` attribute is missing.

This doesn't work if the parent node is a `Period`, as it doesn't have
the `mimeType` attribute, and it also generates timeline keys which
are not unique if two streams of the same `mimeType` would get
muxed (currently not supported).

Instead, build an `ident` tuple on the `Representation` instance
consisting of the ids of the parent `Period` and `AdaptationSet`, and
the `Representation` itself, with its required `id` attribute. This
ensures a unique key for the `timelines` dict.

Then simply pass the `ident` tuple to the `SegmentTemplate.segments()`
generator, similar to the `BaseURL` fixes of e684913.
2023-02-27 23:56:36 -08:00
bastimeyer
3fa7e7646d tools: ignore PLW0603 ruff rule 2023-02-27 20:28:10 +01:00
bastimeyer
8c01136537 plugins.tv5monde: rewrite and fix plugin 2023-02-27 09:34:02 -08:00
bastimeyer
e2e41987ae stream.dash: ContentProtection in Representation 2023-02-23 18:51:24 -08:00
bastimeyer
e6849137c2 stream.dash: fix SegmentTemplate's BaseURL context 2023-02-23 16:21:49 -08:00
bastimeyer
2a24f314f2 session: fix http-trust-env option 2023-02-23 22:26:38 +01:00
bastimeyer
77e67a7002 stream.dash: add basic typing to parser 2023-02-23 11:52:05 -08:00
bastimeyer
07bd742ddf stream.dash: fix missing initialization byterange 2023-02-22 15:51:27 -08:00
bastimeyer
65bb38e42e stream.dash: refactor and reformat code
- Fix incorrect typing
- Fix error messages, as well as docstrings and comments
- Fix indentation
- Replace str.format with f-strings
- Use consistent order of `MPDNode.attr()` call keywords,
  but keep the order of the method's arguments
- Use keywords when initializing the `Segment` dataclass
- Flatten nested if-blocks in `SegmentTemplate.format_media()`
- Remove unnecessary if-block in `Representation.segments()`
2023-02-23 00:33:54 +01:00