docs: add plugin.api.validate API documentation

This commit is contained in:
bastimeyer 2023-11-05 17:29:36 +01:00 committed by Sebastian Meyer
parent ae71c9d933
commit d3ae06f91a
6 changed files with 822 additions and 49 deletions

View File

@ -9,6 +9,7 @@ This is an incomplete reference of the relevant Streamlink APIs.
api/options
api/session
api/plugin
api/validate
api/stream
api/webbrowser
api/exceptions

247
docs/api/validate.rst Normal file
View File

@ -0,0 +1,247 @@
Validation schemas
------------------
.. ..
Sphinx's autodoc doesn't properly document imported module members and it just outputs "alias of" for re-exported classes.
This means we'll have to run `automodule` twice if we want to document the original classes:
1. the main public interface (which contains aliases, like `all` for `AllSchema` for example)
2. the original schema classes with their full signatures and docstrings
.
Ignore unneeded classes like `SchemaContainer` which are not useful for the API docs.
.
Ignore `validate` as well, as `functools.singledispatch` functions are not fully supported by autodoc.
Instead, manually document `validate` and its overloading functions for base schema types here at the top,
just below the manually imported `Schema` (the main validation schema interface).
The documentations for any custom schemas like `AllSchema` for example is done on the schemas themselves.
.
Ideally, we'd just run autodoc on the main module and configure the order of items. :(
.. autoclass:: streamlink.plugin.api.validate.Schema
:members:
:undoc-members:
.. py:function:: validate(schema, value)
:module: streamlink.plugin.api.validate
The core of the :mod:`streamlink.plugin.api.validate` module.
It validates the given input ``value`` and returns a value according to the specific validation rules of the ``schema``.
If the validation fails, a :exc:`ValidationError <_exception.ValidationError>` is raised with a detailed error message.
The ``schema`` can be any kind of object. Depending on the ``schema``, different validation rules apply.
Simple schema objects like ``"abc"`` or ``123`` for example test the equality of ``value`` and ``schema``
and return ``value`` again, while type schema objects like ``str`` test whether ``value`` is an instance of ``schema``.
``schema`` objects which are callable receive ``value`` as a single argument and must return a truthy value, otherwise the
validation fails. These are just a few examples.
The ``validate`` module implements lots of special schemas, like :class:`validate.all <all>` or :class:`validate.any <any>`
for example, which are schema containers that receive a sequence of sub-schemas as arguments and where each sub-schema
then gets validated one after another.
:class:`validate.all <all>` requires each sub-schema to successfully validate. It passes the return value of each
sub-schema to the next one and then returns the return value of the last sub-schema.
:class:`validate.any <any>` on the other hand requires at least one sub-schema to be valid and returns the return value of
the first valid sub-schema. Any validation failures prior will be ignored, but at least one must succeed.
Other special ``schema`` cases for example are instances of sequences like ``list`` or ``tuple``, or mappings like ``dict``.
Here, each sequence item or key-value mapping pair is validated against the input ``value``
and a new sequence/mapping object according to the ``schema`` validation is returned.
:func:`validate()` should usually not be called directly when validating schemas. Instead, the wrapper method
:meth:`Schema.validate() <Schema.validate>` of the main :class:`Schema` class should be called. Other Streamlink APIs
like the methods of the :class:`HTTPSession <streamlink.session.Streamlink.http>` or the various
:mod:`streamlink.utils.parse` functions for example expect this interface when the ``schema`` keyword is set,
which allows for immediate validation of the data using a :class:`Schema` object.
:func:`validate()` is implemented using the stdlib's :func:`functools.singledispatch` decorator, where more specific
schemas overload the default implementation with more validation logic.
----
By default, :func:`validate()` compares ``value`` and ``schema`` for equality. This means that simple schema objects
like booleans, strings, numbers, None, etc. are validated here, as well as anything unknown.
Example:
.. code-block:: python
schema = validate.Schema(123)
assert schema.validate(123) == 123
assert schema.validate(123.0) == 123.0
schema.validate(456) # raises ValidationError
schema.validate(None) # raises ValidationError
:param Any schema: Any kind of object not handled by a more specific validation function
:param Any value: The input value
:raise ValidationError: If ``value`` and ``schema`` are not equal
:return: Unmodified ``value``
.. py:function:: _validate_type(schema, value)
:module: streamlink.plugin.api.validate
:class:`type` validation.
Checks if ``value`` is an instance of ``schema``.
Example:
.. code-block:: python
schema = validate.Schema(int)
assert schema.validate(123) == 123
assert schema.validate(True) is True # `bool` is a subclass of `int`
schema.validate("123") # raises ValidationError
*This function is included for documentation purposes only! (singledispatch overload)*
:param type schema: A :class:`type` object
:param Any value: The input value
:raise ValidationError: If ``value`` is not an instance of ``schema``
:return: Unmodified ``value``
.. py:function:: _validate_callable(schema, value)
:module: streamlink.plugin.api.validate
``Callable`` validation.
Validates a ``schema`` function where ``value`` gets passed as a single argument.
Must return a truthy value.
Example:
.. code-block:: python
schema = validate.Schema(
lambda val: val < 2,
)
assert schema.validate(1) == 1
schema.validate(2) # raises ValidationError
*This function is included for documentation purposes only! (singledispatch overload)*
:param Callable schema: A function with one argument
:param Any value: The input value
:raise ValidationError: If ``schema`` returns a non-truthy value
:return: Unmodified ``value``
.. py:function:: _validate_sequence(schema, value)
:module: streamlink.plugin.api.validate
:class:`list <builtins.list>`, :class:`tuple`, :class:`set` and :class:`frozenset` validation.
Each item of ``value`` gets validated against **any** of the items of ``schema``.
Please note the difference between :class:`list <builtins.list>`
and the :class:`ListSchema <_schemas.ListSchema>` validation.
Example:
.. code-block:: python
schema = validate.Schema([1, 2, 3])
assert schema.validate([]) == []
assert schema.validate([1, 2]) == [1, 2]
assert schema.validate([3, 2, 1]) == [3, 2, 1]
schema.validate({1, 2, 3}) # raises ValidationError
schema.validate([1, 2, 3, 4]) # raises ValidationError
*This function is included for documentation purposes only! (singledispatch overload)*
:param Union[list, tuple, set, frozenset] schema: A sequence of validation schemas
:param Any value: The input value
:raise ValidationError: If ``value`` is not an instance of the ``schema``'s own type
:return: A new sequence of the same type as ``schema`` with each item of ``value`` being validated
.. py:function:: _validate_dict(schema, value)
:module: streamlink.plugin.api.validate
:class:`dict` validation.
Each key-value pair of ``schema`` gets validated against the respective key-value pair of ``value``.
Additional keys in ``value`` are ignored and not included in the validation result.
If a ``schema`` key is an instance of :class:`OptionalSchema <_schemas.OptionalSchema>`, then ``value`` may omit it.
If one of the ``schema``'s keys is a :class:`type`,
:class:`AllSchema <_schemas.AllSchema>`, :class:`AnySchema <_schemas.AnySchema>`,
:class:`TransformSchema <_schemas.TransformSchema>`, or :class:`UnionSchema <_schemas.UnionSchema>`,
then all key-value pairs of ``value`` are validated against the ``schema``'s key-value pair.
Example:
.. code-block:: python
schema = validate.Schema({
"key": str,
validate.optional("opt"): 123,
})
assert schema.validate({"key": "val", "other": 123}) == {"key": "val"}
assert schema.validate({"key": "val", "opt": 123}) == {"key": "val", "opt": 123}
schema.validate(None) # raises ValidationError
schema.validate({}) # raises ValidationError
schema.validate({"key": 123}) # raises ValidationError
schema.validate({"key": "val", "opt": 456}) # raises ValidationError
.. code-block:: python
schema = validate.Schema({
validate.any("a", "b"): int,
})
assert schema.validate({}) == {}
assert schema.validate({"a": 1}) == {"a": 1}
assert schema.validate({"b": 2}) == {"b": 2}
assert schema.validate({"a": 1, "b": 2}) == {"a": 1, "b": 2}
schema.validate({"a": 1, "b": 2, "other": 0}) # raises ValidationError
schema.validate({"a": None}) # raises ValidationError
*This function is included for documentation purposes only! (singledispatch overload)*
:param dict schema: A :class:`dict`
:param Any value: The input value
:raise ValidationError: If ``value`` is not a :class:`dict`
:raise ValidationError: If any of the ``schema``'s non-optional keys are not part of the input ``value``
:return: A new :class:`dict`
.. py:function:: _validate_pattern(schema, value)
:module: streamlink.plugin.api.validate
:class:`re.Pattern` validation.
Calls the :meth:`re.Pattern.search()` method on the ``schema`` pattern.
Please note the difference between :class:`re.Pattern` and the :class:`RegexSchema <_schemas.RegexSchema>` validation.
Example:
.. code-block:: python
schema = validate.Schema(
re.compile(r"^Hello, (?P<name>\w+)!$"),
)
assert schema.validate("Does not match") is None
assert schema.validate("Hello, world!")["name"] == "world"
schema.validate(123) # raises ValidationError
schema.validate(b"Hello, world!") # raises ValidationError
*This function is included for documentation purposes only! (singledispatch overload)*
:param re.Pattern schema: A compiled :class:`re.Pattern` object (:func:`re.compile()` return value)
:param Any value: The input value
:raise ValidationError: If ``value`` is not an instance of :class:`str` or :class:`bytes`
:raise ValidationError: If the type of ``value`` doesn't match ``schema``'s :class:`str`/:class:`bytes` type
:return: ``None`` if ``value`` doesn't match ``schema``, or the resulting :class:`re.Match` object
.. automodule:: streamlink.plugin.api.validate
:imported-members:
:exclude-members: Schema, SchemaContainer, validate
.. automodule:: streamlink.plugin.api.validate._schemas
:exclude-members: SchemaContainer
:no-show-inheritance:
.. autoexception:: streamlink.plugin.api.validate._exception.ValidationError

View File

@ -3,6 +3,10 @@ from typing import Optional, Sequence, Union
class ValidationError(ValueError):
"""
Currently not exposed in the public API.
"""
MAX_LENGTH = 60
errors: Union[str, Exception, Sequence[Union[str, Exception]]]

View File

@ -17,39 +17,146 @@ class _CollectionSchemaContainer(SchemaContainer):
class AllSchema(_CollectionSchemaContainer):
"""
Collection of schemas where every schema must be valid.
The last validation result gets returned.
A collection of schemas where each schema must be valid.
Validates one schema after another with the input value of the return value of the previous one.
Example:
.. code-block:: python
# `validate.Schema` is a subclass of `AllSchema` (`validate.all`)
schema = validate.Schema(
int,
validate.transform(lambda val: val + 1),
lambda val: val < 3,
)
assert schema.validate(1) == 2
schema.validate("a") # raises ValidationError
schema.validate(2) # raises ValidationError
:param Any \\*schemas: Schemas where each one must be valid
:return: The return value of the last schema
"""
class AnySchema(_CollectionSchemaContainer):
"""
Collection of schemas where at least one schema must be valid.
The first successful validation result gets returned.
A collection of schemas where at least one schema must be valid.
Validates one schema after another with the same input value until the first one succeeds.
Example:
.. code-block:: python
schema = validate.Schema(
validate.any(int, float, str),
)
assert schema.validate(123) == 123
assert schema.validate(123.0) == 123.0
assert schema.validate("123") == "123"
schema.validate(None) # raises ValidationError
:param Any \\*schemas: Schemas where at least one must be valid
:raise ValidationError: Error collection of all schema validations if none succeeded
:return: The return value of the first valid schema
"""
class NoneOrAllSchema(_CollectionSchemaContainer):
"""
Collection of schemas where every schema must be valid. If the initial input is None, all validations will be skipped.
The last validation result gets returned.
Similar to :class:`AllSchema`, but skips the validation if the input value is ``None``.
This is useful for optional validation results, e.g. when validating a potential match of a regular expression.
Example:
.. code-block:: python
schema = validate.Schema(
validate.none_or_all(
int,
lambda val: val < 2,
),
)
assert schema.validate(None) is None
assert schema.validate(1) == 1
schema.validate("123") # raises ValidationError
schema.validate(2) # raises ValidationError
:param Any \\*schemas: Schemas where each one must be valid, unless the input is ``None``
:raise ValidationError: Error wrapper of the failed schema validation
:return: ``None`` if the input is ``None``, or the return value of the last schema
"""
class ListSchema(_CollectionSchemaContainer):
"""
Collection of schemas where every indexed schema must be valid, as well as the input type and length.
A new list of the validated input gets returned.
A list of schemas where every item must be valid, as well as the input type and length.
Please note the difference between :class:`ListSchema`
and the :func:`list <streamlink.plugin.api.validate.validate_sequence()>` validation.
Example:
.. code-block:: python
schema = validate.Schema(
validate.list(1, 2, int),
)
assert schema.validate([1, 2, 3]) == [1, 2, 3]
schema.validate(None) # raises ValidationError
schema.validate([1, 2]) # raises ValidationError
schema.validate([3, 2, 1]) # raises ValidationError
:param Any \\*schema: Schemas where each one must be valid
:raise ValidationError: If the input is not a :class:`list`
:raise ValidationError: If the input's length is not equal to the number of schemas
:return: A new :class:`list <builtins.list>` with the validated input
"""
class GetItemSchema:
"""
Get an item from the input.
Get an ``item`` from the input.
Unless strict is set to True, item can be a tuple of items for recursive lookups.
If the item is not found in the last object of a recursive lookup, return the default.
Supported inputs are XML elements, regex matches and anything that implements __getitem__.
The input can be anything that implements :func:`__getitem__()`,
as well as :class:`lxml.etree.Element` objects where element attributes are looked up.
Returns the ``default`` value if ``item`` was not found.
Unless ``strict`` is set to ``True``, the ``item`` can be a :class:`tuple` of items for a recursive lookup.
In this case, the ``default`` value is only returned if the leaf-input-object doesn't contain the current ``item``.
Example:
.. code-block:: python
schema = validate.Schema(
validate.get("name", default="unknown"),
)
assert schema.validate({"name": "user"}) == "user"
assert schema.validate(re.match(r"Hello, (?P<name>\\w+)!", "Hello, user!")) == "user"
assert schema.validate(lxml.etree.XML(\"\"\"<elem name="abc"/>\"\"\")) == "abc"
assert schema.validate({}) == "unknown"
schema.validate(None) # raises ValidationError
.. code-block:: python
schema = validate.Schema(
validate.get(("a", "b", "c")),
)
assert schema.validate({"a": {"b": {"c": "d"}}}) == "d"
assert schema.validate({"a": {"b": {}}}) is None
schema.validate({"a": {}}) # raises ValidationError
:param item: The lookup key, or a :class:`tuple` of recursive lookup keys
:param default: Optional custom default value
:param strict: If ``True``, don't perform recursive lookups with the :class:`tuple` item
:raise ValidationError: If the input doesn't implement :func:`__getitem__()`
:raise ValidationError: If the input doesn't have the current ``item`` in a recursive non-leaf-input-object lookup
:return: The :func:`__getitem__()` return value, or an :class:`lxml.etree.Element` attribute
"""
def __init__(
@ -65,7 +172,38 @@ class GetItemSchema:
class RegexSchema:
"""
A regex pattern that must match using the provided method.
A :class:`re.Pattern` that **must** match.
Allows selecting a different regex pattern method (default is :meth:`re.Pattern.search()`).
Please note the difference between :class:`RegexSchema`
and the :func:`re.Pattern <streamlink.plugin.api.validate.validate_pattern()>` validation.
Example:
.. code-block:: python
schema = validate.Schema(
validate.regex(re.compile(r"Hello, (?P<name>\\w+)!")),
)
assert schema.validate("Hello, world!")["name"] == "world"
schema.validate("Does not match") # raises ValidationError
schema.validate(123) # raises ValidationError
schema.validate(b"Hello, world!") # raises ValidationError
.. code-block:: python
schema = validate.Schema(
validate.regex(re.compile(r"Hello, (?P<name>\\w+)!"), method="findall"),
)
assert schema.validate("Hello, userA! Hello, userB!") == ["userA", "userB"]
assert schema.validate("Does not match") == [] # findall does not return None
:param pattern: A compiled pattern (:func:`re.compile` return value)
:param method: The pattern's method which will be called when validating
:raise ValidationError: If the input is not an instance of ``str`` or ``bytes``
:raise ValidationError: If the type of the input doesn't match the pattern's ``str``/``bytes`` type
:raise ValidationError: If the return value of the chosen regex pattern method is ``None``
"""
def __init__(
@ -79,7 +217,24 @@ class RegexSchema:
class TransformSchema:
"""
Transform the input using the specified function and args/keywords.
A transform function which receives the input value as the argument, with optional custom arguments and keywords.
Example:
.. code-block:: python
schema = validate.Schema(
validate.transform(lambda val: val + 1),
validate.transform(operator.lt, 3),
)
assert schema.validate(1) is True
assert schema.validate(2) is False
:param func: A transform function
:param \\*args: Additional arguments
:param \\*\\*kwargs: Additional keywords
:raise ValidationError: If the transform function is not callable
:return: The return value of the transform function
"""
def __init__(
@ -95,7 +250,9 @@ class TransformSchema:
class OptionalSchema:
"""
An optional key set in a dict or dict in a :class:`UnionSchema`.
An optional key set in a :class:`dict`.
See the :func:`dict <streamlink.plugin.api.validate.validate_dict>` validation and the :class:`UnionSchema`.
"""
def __init__(self, key: Any):
@ -104,13 +261,60 @@ class OptionalSchema:
class AttrSchema(SchemaContainer):
"""
Validate attributes of an input object.
Validate attributes of an input object according to a :class:`dict`'s key-value pairs.
Example:
.. code-block:: python
schema = validate.Schema(
validate.attr({
"a": str,
"b": int,
}),
)
assert schema.validate(obj) is not obj
schema.validate(obj_without_a) # raises ValidationError
schema.validate(obj_b_is_str) # raises ValidationError
:param dict[str, Any] schema: A :class:`dict` with attribute validations
:raise ValidationError: If the input doesn't have one of the schema's attributes
:return: A copy of the input object with validated attributes
"""
class XmlElementSchema:
"""
Validate an XML element.
Example:
.. code-block:: python
schema = validate.Schema(
validate.xml_element(
tag="foo",
attrib={"bar": str},
text=validate.transform(str.upper),
),
)
elem = lxml.etree.XML(\"\"\"<foo bar="baz">qux</foo>\"\"\")
new_elem = schema.validate(elem)
assert new_elem is not elem
assert new_elem.tag == "foo"
assert new_elem.attrib == {"bar": "baz"}
assert new_elem.text == "QUX"
assert new_elem.tail is None
schema.validate(123) # raises ValidationError
schema.validate(lxml.etree.XML(\"\"\"<unknown/>\"\"\")) # raises ValidationError
:param tag: Optional element tag validation
:param text: Optional element text validation
:param attrib: Optional element attributes validation
:param tail: Optional element tail validation
:raise ValidationError: If ``value`` is not an :class:`lxml.etree.Element`
:return: A new :class:`lxml.etree.Element` object, including a deep-copy of the input's child nodes,
with optionally validated ``tag``, ``attrib`` mapping, ``text`` or ``tail``.
"""
# signature is weird because of backwards compatiblity
@ -130,12 +334,26 @@ class XmlElementSchema:
class UnionGetSchema:
"""
Validate multiple :class:`GetItemSchema` schemas on the same input.
Convenience wrapper for ``validate.union((validate.get(...), validate.get(...), ...))``.
Example:
.. code-block:: python
schema = validate.Schema(
validate.union_get("a", "b", ("c", "d")),
)
assert schema.validate({"a": 1, "b": 2, "c": {"d": 3}}) == (1, 2, 3)
:param \\*getters: Inputs for each :class:`GetItemSchema`
:return: A :class:`tuple` (default ``seq`` type) with items of the respective :class:`GetItemSchema` validations
"""
def __init__(
self,
*getters,
seq: Type[Union[List, FrozenSet, Set, Tuple]] = tuple,
seq: Type[Union[Tuple, List, Set, FrozenSet]] = tuple,
):
self.getters: Sequence[GetItemSchema] = tuple(GetItemSchema(getter) for getter in getters)
self.seq = seq
@ -145,5 +363,32 @@ class UnionSchema(SchemaContainer):
"""
Validate multiple schemas on the same input.
Can be a tuple, list, set, frozenset or dict of schemas.
Example:
.. code-block:: python
schema = validate.Schema(
validate.union((
validate.transform(str.format, one="abc", two="def"),
validate.transform(str.format, one="123", two="456"),
)),
)
assert schema.validate("{one} {two}") == ("abc def", "123 456")
.. code-block:: python
schema = validate.Schema(
validate.union({
"one": lambda val: val < 3,
validate.optional("two"): lambda val: val > 1,
}),
)
assert schema.validate(1) == {"one": 1}
assert schema.validate(2) == {"one": 2, "two": 2}
schema.validate(3) # raises ValidationError
:param Union[tuple, list, set, frozenset, dict] schema: A :class:`tuple`, :class:`list`, :class:`set`, :class:`frozenset`
or :class:`dict` of schemas
:raises ValidationError: If a sequence item or the value of a non-optional key-value pair doesn't validate
:return: A new object of the same type, with each item or key-value pair being validated against the same input value
"""

View File

@ -2,6 +2,7 @@ from collections import abc
from copy import copy, deepcopy
from functools import singledispatch
from re import Pattern
from typing import Any, Type, Union
from lxml.etree import Element, iselement
@ -25,10 +26,13 @@ from streamlink.plugin.api.validate._schemas import (
class Schema(AllSchema):
"""
Wrapper class for :class:`AllSchema` with a validate method which raises :class:`PluginError` by default on error.
The base class for creating validation schemas.
A wrapper for :class:`AllSchema <_schemas.AllSchema>` with a wrapper method for :func:`validate`
which by default raises :class:`PluginError <streamlink.exceptions.PluginError>` on error.
"""
def validate(self, value, name="result", exception=PluginError):
def validate(self, value: Any, name: str = "result", exception: Type[Exception] = PluginError) -> Any:
try:
return validate(self, value)
except ValidationError as err:
@ -51,8 +55,8 @@ def validate(schema, value):
return value
@validate.register(type)
def _validate_type(schema, value):
@validate.register
def _validate_type(schema: type, value):
if not isinstance(value, schema):
raise ValidationError(
"Type of {value} should be {expected}, but is {actual}",
@ -65,11 +69,12 @@ def _validate_type(schema, value):
return value
# singledispatch doesn't support typing.Union/types.UnionType on py<311, so keep each register() call for now
@validate.register(list)
@validate.register(tuple)
@validate.register(set)
@validate.register(frozenset)
def _validate_sequence(schema, value):
def _validate_sequence(schema: Union[list, tuple, set, frozenset], value):
cls = type(schema)
validate(cls, value)
any_schemas = AnySchema(*schema)
@ -79,8 +84,8 @@ def _validate_sequence(schema, value):
)
@validate.register(dict)
def _validate_dict(schema, value):
@validate.register
def _validate_dict(schema: dict, value):
cls = type(schema)
validate(cls, value)
new = cls()
@ -372,8 +377,8 @@ def validate_union(schema, value):
)
@validate_union.register(dict)
def _validate_union_dict(schema, value):
@validate_union.register
def _validate_union_dict(schema: dict, value):
new = type(schema)()
for key, subschema in schema.items():
is_optional = isinstance(key, OptionalSchema)
@ -395,11 +400,12 @@ def _validate_union_dict(schema, value):
return new
# singledispatch doesn't support typing.Union/types.UnionType on py<311, so keep each register() call for now
@validate_union.register(list)
@validate_union.register(tuple)
@validate_union.register(set)
@validate_union.register(frozenset)
def _validate_union_sequence(schemas, value):
def _validate_union_sequence(schemas: Union[list, tuple, set, frozenset], value):
return type(schemas)(
validate(schema, value) for schema in schemas
)

View File

@ -18,7 +18,19 @@ from streamlink.utils.parse import (
def validator_length(number: int) -> Callable[[str], bool]:
"""
Check input for minimum length using len().
Utility function for checking whether the input has a minimum length, by using :func:`len()`.
Example:
.. code-block:: python
schema = validate.Schema(
validate.length(3),
)
assert schema.validate("abc") == "abc"
assert schema.validate([1, 2, 3]) == [1, 2, 3]
schema.validate("a") # raises ValidationError
schema.validate([1]) # raises ValidationError
"""
def min_len(value):
@ -37,7 +49,21 @@ def validator_length(number: int) -> Callable[[str], bool]:
def validator_startswith(string: str) -> Callable[[str], bool]:
"""
Check if the input string starts with another string.
Utility function for checking whether the input string starts with another string.
Example:
.. code-block:: python
schema = validate.Schema(
validate.startswith("1"),
)
assert schema.validate("123") == "123"
schema.validate("321") # raises ValidationError
schema.validate(None) # raises ValidationError
:raise ValidationError: If input is not an instance of :class:`str`
:raise ValidationError: If input doesn't start with ``string``
"""
def starts_with(value):
@ -57,7 +83,21 @@ def validator_startswith(string: str) -> Callable[[str], bool]:
def validator_endswith(string: str) -> Callable[[str], bool]:
"""
Check if the input string ends with another string.
Utility function for checking whether the input string ends with another string.
Example:
.. code-block:: python
schema = validate.Schema(
validate.endswith("3"),
)
assert schema.validate("123") == "123"
schema.validate("321") # raises ValidationError
schema.validate(None) # raises ValidationError
:raise ValidationError: If input is not an instance of :class:`str`
:raise ValidationError: If input doesn't end with ``string``
"""
def ends_with(value):
@ -77,7 +117,21 @@ def validator_endswith(string: str) -> Callable[[str], bool]:
def validator_contains(string: str) -> Callable[[str], bool]:
"""
Check if the input string contains another string.
Utility function for checking whether the input string contains another string.
Example:
.. code-block:: python
schema = validate.Schema(
validate.contains("456"),
)
assert schema.validate("123456789") == "123456789"
schema.validate("987654321") # raises ValidationError
schema.validate(None) # raises ValidationError
:raise ValidationError: If input is not an instance of :class:`str`
:raise ValidationError: If input doesn't contain ``string``
"""
def contains_str(value):
@ -97,7 +151,36 @@ def validator_contains(string: str) -> Callable[[str], bool]:
def validator_url(**attributes) -> Callable[[str], bool]:
"""
Parse a URL and validate its attributes using sub-schemas.
Utility function for validating a URL using schemas.
Allows validating all URL attributes returned by :func:`urllib.parse.urlparse()`:
- ``scheme`` - updated to ``AnySchema("http", "https")`` if set to ``"http"``
- ``netloc``
- ``path``
- ``params``
- ``query``
- ``fragment``
- ``username``
- ``password``
- ``hostname``
- ``port``
Example:
.. code-block:: python
schema = validate.Schema(
validate.url(path=validate.endswith(".m3u8")),
)
assert schema.validate("https://host/pl.m3u8?query") == "https://host/pl.m3u8?query"
schema.validate(None) # raises ValidationError
schema.validate("not a URL") # raises ValidationError
schema.validate("https://host/no-pl?pl.m3u8") # raises ValidationError
:raise ValidationError: If input is not a string
:raise ValidationError: If input is not a URL (doesn't have a ``netloc`` parsing result)
:raise ValidationError: If an unknown URL attribute is passed as an option
"""
# Convert "http" to AnySchema("http", "https") for convenience
@ -141,9 +224,19 @@ def validator_url(**attributes) -> Callable[[str], bool]:
def validator_getattr(attr: Any, default: Any = None) -> TransformSchema:
"""
Get a named attribute from the input object.
Utility function for getting an attribute from the input object.
If a default is set, it is returned when the attribute doesn't exist.
Example:
.. code-block:: python
schema = validate.Schema(
validate.getattr("year", "unknown"),
)
assert schema.validate(datetime.date.fromisoformat("2000-01-01")) == 2000
assert schema.validate("not a date/datetime object") == "unknown"
"""
def getter(value):
@ -154,7 +247,18 @@ def validator_getattr(attr: Any, default: Any = None) -> TransformSchema:
def validator_hasattr(attr: Any) -> Callable[[Any], bool]:
"""
Verify that the input object has an attribute with the given name.
Utility function for checking whether an attribute exists on the input object.
Example:
.. code-block:: python
schema = validate.Schema(
validate.hasattr("year"),
)
date = datetime.date.fromisoformat("2000-01-01")
assert schema.validate(date) is date
schema.validate("not a date/datetime object") # raises ValidationError
"""
def getter(value):
@ -168,9 +272,26 @@ def validator_hasattr(attr: Any) -> Callable[[Any], bool]:
def validator_filter(func: Callable[..., bool]) -> TransformSchema:
"""
Filter out unwanted items from the input using the specified function.
Utility function for filtering out unwanted items from the input using the specified function
via the built-in :func:`filter() <builtins.filter>`.
Supports both dicts and sequences. key/value pairs are expanded when applied to a dict.
Supports iterables, as well as instances of :class:`dict` where key-value pairs are expanded.
Example:
.. code-block:: python
schema = validate.Schema(
validate.filter(lambda val: val < 3),
)
assert schema.validate([1, 2, 3, 4]) == [1, 2]
.. code-block:: python
schema = validate.Schema(
validate.filter(lambda key, val: key > 1 and val < 3),
)
assert schema.validate({0: 0, 1: 1, 2: 2, 3: 3, 4: 4}) == {2: 2}
"""
def expand_kv(kv):
@ -188,9 +309,26 @@ def validator_filter(func: Callable[..., bool]) -> TransformSchema:
def validator_map(func: Callable[..., Any]) -> TransformSchema:
"""
Transform items from the input using the specified function.
Utility function for mapping/transforming items from the input using the specified function,
via the built-in :func:`map() <builtins.map>`.
Supports both dicts and sequences. key/value pairs are expanded when applied to a dict.
Supports iterables, as well as instances of :class:`dict` where key-value pairs are expanded.
Example:
.. code-block:: python
schema = validate.Schema(
validate.map(lambda val: val + 1),
)
assert schema.validate([1, 2, 3, 4]) == [2, 3, 4, 5]
.. code-block:: python
schema = validate.Schema(
validate.map(lambda key, val: (key + 1, val * 2)),
)
assert schema.validate({0: 0, 1: 1, 2: 2, 3: 3, 4: 4}) == {1: 0, 2: 2, 3: 4, 4: 6, 5: 8}
"""
def expand_kv(kv):
@ -214,8 +352,24 @@ def validator_xml_find(
namespaces: Optional[Dict[str, str]] = None,
) -> TransformSchema:
"""
Find an XML element (:meth:`Element.find`).
Utility function for finding an XML element using :meth:`Element.find()`.
This method uses the ElementPath query language, which is a subset of XPath.
Example:
.. code-block:: python
schema = validate.Schema(
validate.xml_find(".//b/c"),
)
assert schema.validate(lxml.etree.XML("<a><b><c>d</c></b></a>")).text == "d"
schema.validate(lxml.etree.XML("<a><b></b></a>")) # raises ValidationError
schema.validate("<a><b><c>d</c></b></a>") # raises ValidationError
:raise ValidationError: If the input is not an :class:`lxml.etree.Element`
:raise ValidationError: On ElementPath evaluation error
:raise ValidationError: If the query didn't return an XML element
"""
def xpath_find(value):
@ -247,8 +401,24 @@ def validator_xml_findall(
namespaces: Optional[Dict[str, str]] = None,
) -> TransformSchema:
"""
Find a list of XML elements (:meth:`Element.findall`).
Utility function for finding XML elements using :meth:`Element.findall()`.
This method uses the ElementPath query language, which is a subset of XPath.
Example:
.. code-block:: python
schema = validate.Schema(
validate.xml_findall(".//b"),
validate.map(lambda elem: elem.text),
)
assert schema.validate(lxml.etree.XML("<a><b>1</b><b>2</b></a>")) == ["1", "2"]
assert schema.validate(lxml.etree.XML("<a><c></c></a>")) == []
schema.validate("<a><b>1</b><b>2</b></a>") # raises ValidationError
:raise ValidationError: If the input is not an :class:`lxml.etree.Element`
:raise ValidationError: On ElementPath evaluation error
"""
def xpath_findall(value):
@ -263,8 +433,24 @@ def validator_xml_findtext(
namespaces: Optional[Dict[str, str]] = None,
) -> AllSchema:
"""
Find an XML element (:meth:`Element.find`) and return its text.
Utility function for finding an XML element using :meth:`Element.find()` and returning its ``text`` attribute.
This method uses the ElementPath query language, which is a subset of XPath.
Example:
.. code-block:: python
schema = validate.Schema(
validate.xml_findtext(".//b/c"),
)
assert schema.validate(lxml.etree.XML("<a><b><c>d</c></b></a>")) == "d"
schema.validate(lxml.etree.XML("<a><b></b></a>")) # raises ValidationError
schema.validate("<a><b><c>d</c></b></a>") # raises ValidationError
:raise ValidationError: If the input is not an :class:`lxml.etree.Element`
:raise ValidationError: On ElementPath evaluation error
:raise ValidationError: If the query didn't return an XML element
"""
return AllSchema(
@ -281,7 +467,25 @@ def validator_xml_xpath(
**variables,
) -> TransformSchema:
"""
Query XML elements via XPath (:meth:`Element.xpath`) and return None if the result is falsy.
Utility function for querying XML elements using XPath (:meth:`Element.xpath()`).
XPath queries always return a result set, but if the result is an empty set, this function instead returns ``None``.
Allows setting XPath variables (``$var``) as additional keywords.
Example:
.. code-block:: python
schema = validate.Schema(
validate.xml_xpath(".//b[@c=$c][1]/@d", c="2"),
)
assert schema.validate(lxml.etree.XML("<a><b c='1' d='A'/><b c='2' d='B'/></a>")) == ["B"]
assert schema.validate(lxml.etree.XML("<a></a>")) is None
schema.validate("<a><b c='1' d='A'/><b c='2' d='B'/></a>") # raises ValidationError
:raise ValidationError: If the input is not an :class:`lxml.etree.Element`
:raise ValidationError: On XPath evaluation error
"""
def transform_xpath(value):
@ -313,8 +517,26 @@ def validator_xml_xpath_string(
**variables,
) -> TransformSchema:
"""
Query XML elements via XPath (:meth:`Element.xpath`),
transform the result into a string and return None if the result is falsy.
Utility function for querying XML elements using XPath (:meth:`Element.xpath()`) and turning the result into a string.
XPath queries always return a result set, so be aware when querying multiple elements.
If the result is an empty set, this function instead returns ``None``.
Allows setting XPath variables (``$var``) as additional keywords.
Example:
.. code-block:: python
schema = validate.Schema(
validate.xml_xpath_string(".//b[2]/text()"),
)
assert schema.validate(lxml.etree.XML("<a><b>A</b><b>B</b><b>C</b></a>")) == "B"
assert schema.validate(lxml.etree.XML("<a></a>")) is None
schema.validate("<a><b>A</b><b>B</b><b>C</b></a>") # raises ValidationError
:raise ValidationError: If the input is not an :class:`lxml.etree.Element`
:raise ValidationError: On XPath evaluation error
"""
return validator_xml_xpath(
@ -331,7 +553,19 @@ def validator_xml_xpath_string(
def validator_parse_json(*args, **kwargs) -> TransformSchema:
"""
Parse JSON data via the :func:`streamlink.utils.parse.parse_json` utility function.
Utility function for parsing JSON data using :func:`streamlink.utils.parse.parse_json()`.
Example:
.. code-block:: python
schema = validate.Schema(
validate.parse_json(),
)
assert schema.validate(\"\"\"{"a":[1,2,3],"b":null}\"\"\") == {"a": [1, 2, 3], "b": None}
schema.validate(123) # raises ValidationError
:raise ValidationError: On parsing error
"""
return TransformSchema(_parse_json, *args, **kwargs, exception=ValidationError, schema=None)
@ -339,7 +573,19 @@ def validator_parse_json(*args, **kwargs) -> TransformSchema:
def validator_parse_html(*args, **kwargs) -> TransformSchema:
"""
Parse HTML data via the :func:`streamlink.utils.parse.parse_html` utility function.
Utility function for parsing HTML data using :func:`streamlink.utils.parse.parse_html()`.
Example:
.. code-block:: python
schema = validate.Schema(
validate.parse_html(),
)
assert schema.validate(\"\"\"<html lang="en">\"\"\").attrib["lang"] == "en"
schema.validate(123) # raises ValidationError
:raise ValidationError: On parsing error
"""
return TransformSchema(_parse_html, *args, **kwargs, exception=ValidationError, schema=None)
@ -347,7 +593,19 @@ def validator_parse_html(*args, **kwargs) -> TransformSchema:
def validator_parse_xml(*args, **kwargs) -> TransformSchema:
"""
Parse XML data via the :func:`streamlink.utils.parse.parse_xml` utility function.
Utility function for parsing XML data using :func:`streamlink.utils.parse.parse_xml()`.
Example:
.. code-block:: python
schema = validate.Schema(
validate.parse_xml(),
)
assert schema.validate(\"\"\"<a b="c"/>\"\"\").attrib["b"] == "c"
schema.validate(123) # raises ValidationError
:raise ValidationError: On parsing error
"""
return TransformSchema(_parse_xml, *args, **kwargs, exception=ValidationError, schema=None)
@ -355,7 +613,19 @@ def validator_parse_xml(*args, **kwargs) -> TransformSchema:
def validator_parse_qsd(*args, **kwargs) -> TransformSchema:
"""
Parse a query string via the :func:`streamlink.utils.parse.parse_qsd` utility function.
Utility function for parsing a query string using :func:`streamlink.utils.parse.parse_qsd()`.
Example:
.. code-block:: python
schema = validate.Schema(
validate.parse_qsd(),
)
assert schema.validate("a=b&a=c&foo=bar") == {"a": "c", "foo": "bar"}
schema.validate(123) # raises ValidationError
:raise ValidationError: On parsing error
"""
return TransformSchema(_parse_qsd, *args, **kwargs, exception=ValidationError, schema=None)