Vulkan Video Decoding has finally become a reality, as it's now
showing up in shipping drivers, and the ffmpeg support has been
merged.
With that in mind, this change introduces HW interop support for
ffmpeg Vulkan frames. The implementation is functionally complete - it
can display frames produced by hardware decoding, and it can work with
ffmpeg vulkan filters. There are still various caveats due to gaps and
bugs in drivers, so YMMV, as always.
Primary testing has been done on Intel, AMD, and nvidia hardware on
Linux with basic Windows testing on nvidia.
Notable caveats:
* Due to driver bugs, video decoding on nvidia does not work right now,
unless you use the Vulkan Beta driver. It can be worked around, but
requires ffmpeg changes that are not considered acceptable to merge.
* Even if those work-arounds are applied, Vulkan filters will not work
on video that was decoded by Vulkan, due to additional bugs in the
nvidia drivers. The filters do work correctly on content decoded some
other way, and then uploaded to Vulkan (eg: Decode with nvdec, upload
with --vf=format=vulkan)
* Vulkan filters can only be used with drivers that support
VK_EXT_descriptor_buffer which doesn't include Intel ANV as yet.
There is an MR outstanding for this.
* When dealing with 1080p content, there may be some visual distortion
in the bottom lines of frames due to chroma scaling incorporating the
extra hidden lines at the bottom of the frame (1080p content is
actually stored as 1088 lines), depending on the hardware/driver
combination and the scaling algorithm. This cannot be easily
addressed as the mechanical fix for it violates the Vulkan spec, and
probably requires a spec change to resolve properly.
All of these caveats will be fixed in either drivers or ffmpeg, and so
will not require mpv changes (unless something unexpected happens)
If you want to run on nvidia with the non-beta drivers, you can this
ffmpeg tree with the work-around patches:
* https://github.com/philipl/FFmpeg/tree/vulkan-nvidia-workarounds
This reworks all of mpv's unit tests so they are compiled as separate
executables (optional) and run via meson test. Because most of the tests
are dependant on mpv's internals, existing compiled objects are
leveraged to create static libs and used when necessary. As an aside, a
function was moved into video/out/gpu/utils for sanity's sake (otherwise
most of vo would have been needed). As a plus, meson multithreads
running tests automatically and also the output no longer pollutes the
source directory. There are tests that can break due to ffmpeg changes,
so they require a specific minimum libavutil version to be built.
Changes:
* fixed hangups in the loop function and in some other cases
* refactoring according to @michaelforney's recommendations in #8314
* a few minor and/or cosmetic changes
* ability to build ao_sndio using meson
The AO provides a way for mpv to directly submit audio to the PipeWire
audio server.
Doing this directly instead of going through the various compatibility
layers provided by PipeWire has the following advantages:
* It reduces complexity of going through the compatibility layers
* It allows a richer integration between mpv and PipeWire
(for example for metadata)
* Some users report issues with the compatibility layers that to not
occur with the native AO
For now the AO is ordered after all the other relevant AOs, so it will
most probably not be picked up by default.
This is for the following reasons:
* Currently it is not possible to detect if the PipeWire daemon that mpv
connects to is actually driving the system audio.
(https://gitlab.freedesktop.org/pipewire/pipewire/-/issues/1835)
* It gives the AO time to stabilize before it is used by everyone.
Based-on-patch-by: Oschowa <oschowa@web.de>
Based-on-patch-by: Andreas Kempf <aakempf@gmail.com>
Helped-by: Ivan <etircopyhdot@gmail.com>
Philip Langdale