Image areas with subtitles are upsampled to 4:4:4 in order to render
the subtitles (makes blending easier). Try not to copy the Y plane on
upsampling. The libswscale API requires this, but this commit works it
around by scaling the chroma planes separately as AV_PIX_FMT_GRAY8. The
Y plane is not touched at all. This is done for 420p8 only, which is the
most commonly needed case. For other formats, the old way is used.
Seems to make ASS rendering faster about 15% in the setup I tested.
Allocate it even if it's needed. The actually done work is almost the
same, except that the code is a bit simpler. May need more memory at
once for RGB subs that use more than one part, which is rare.
This was an awkward hack that attempted to avoid the use of 16 bit
textures, while still allowing rendering 10-16 bit YUV formats. The
idea was that even if the hardware doesn't support 16 bit textures,
an A8L8 textures could be used to convert 10 bit (etc.) to 8 bit in
the shader, instead of doing this on the CPU.
This was an experiment, disabled by default, and was (probably) rarely
used. I've never heard of this being used successfully. Remove it.
This is more correct. Not all frame specific fields are in AVFrame,
such as colorspace and color_range, and these are still queried
through the decoder context.
This field contained the "average" bit depth per pixel. It serves no
purpose anymore. Remove it.
Only vo_opengl_old still uses this in order to allocate a buffer that is
shared between all planes.
This used to mean that there is more than one plane. This is not very
useful: IMGFMT_Y8 was not considered planar, but it's just a Y plane,
and could be treated the same as actual planar formats. On the other
hand, IMGFMT_NV12 is partially packed, and usually requires special
handling, but was considered planar.
Change its meaning. Now the flag is set if the format has a separate
plane for each component. IMGFMT_Y8 is now planar, IMGFMT_NV12 is not.
As an odd special case, IMGFMT_MONO (1 bit per pixel) is like a planar
RGB format with a single plane.
In theory, vf_sub could take any format supported by swscale. But to be
sure that it's reasonably fast, only 420P was allowed. However, other
similar 8 bit planar formats will be just as fast and there's no reason
to exclude them. Even for completely different formats there doesn't
seem to be any significant advantage to force vf_sub to convert to a
simpler/more common format.
This did random things with some image formats, including 10 bit
formats. Fixes the mp_image_clear() function too.
This still has some caveats:
- doesn't clear alpha to opaque (too hard with packed RGB, and is rarely
needed)
- sets luma to 0 for mpeg-range YUV, instead of the lowest possible
value (e.g. 16 for 8 bit)
Needed because mplayer* basically tracks DAR, which makes it harder for
filters which want to keep the SAR.
The contents of this function are duplicated in a few filters, and will
be used instead of custom code as the filters are updated later.
This makes it behave like vo_lavc.
Unfortunately, the code for setting up the OSD dimensions (mp_osd_res)
is copied from vo_lavc, but it doesn't look like something that should
be factored out.
redraw_frame() copied the image into the currently visible buffer. This
resulted in flicker when doing heavy OSD redrawing (like changing the
subtitle size to something absurdly large).
Use the same logic as draw_image instead.
In order to support OSD redrawing for vo_xv and vo_x11, draw_bmp.c
included an awkward "backup" mechanism to copy and restore image
regions that have been changed by OSD/subtitles.
Replace this by a much simpler mechanism: keep a reference to the
original image, and use that to restore the Xv/X framebuffers.
In the worst case, this may increase cache pressure and memory usage,
even if no OSD or subtitles are rendered. In practice, it seems to be
always faster.
It's not easy to tell whether the OSD/subs are empty, or if something is
drawn. In general you have to use osd_draw() with a custom callback. If
nothing is visible, the callback is never invoked. (The actual reason
why this is so "hard" is the implementation of osd_libass.c, which
doesn't allow separating rendering and drawing of OSD elements, because
all OSD elements share the same ASS_Renderer.)
To simplify avoiding copies, make osd_draw_on_image() instead of the
caller use mp_image_make_writeable(). Introduce osd_draw_on_image_p(),
which works like osd_draw_on_image(), but gets the new image allocation
from an image pool. This is supposed to be an optimization, because it
reduces the frequency of large allocations/deallocations for image data.
The result of this is that the frequency of copies needed in conjunction
with vf_sub, screenshots, and vo_lavc (encoding) should be reduced.
vf_sub now always does true pass-through if no subs are shown.
Drop the pts check from vf_sub. This didn't make much sense.
Remove the strange things the old mp_image_setfmt() code did to the
image format parameters. This includes setting chroma shift to 31 for
gray (Y8) formats and more.
Y8 + vo_opengl_old didn't actually work for unknown reasons (regression
in this branch). Fix this. The difference is that Y8 is now interpreted
as gray RGB (LUMINANCE texture) instead of involving YUV (and levels)
conversion.
Get rid of distinguishing RGB and BGR. There wasn't really any good
reason for this.
Remove mp_get_chroma_shift() and IMGFMT_IS_YUVP16*(). mp_imgfmt_desc
gives the same information and more.
Even though #ifdef ACCURATE is removed, the result should be about the
same. The fallback is only used by packed YUV formats (YUYV, NV12), and
doing 16 bit for them instead of 8 bit is not useful.
A side effect is that Y8 (gray) is not converted drawing subs, and for
alpha formats, the alpha plane is not removed. This means the number of
planes after upsampling can be 1-4 (1: gray, 2: gray+alpha, 3: planar,
4: planar+alpha). The code has to be adjusted accordingly to work on the
color planes only. Also remove the workaround for the chroma shift 31
hack.
mplayer's video chain traditionally used FourCCs for pixel formats. For
example, it used IMGFMT_YV12 for 4:2:0 YUV, which was defined to the
string 'YV12' interpreted as unsigned int. Additionally, it used to
encode information into the numeric values of some formats. The RGB
formats had their bit depth and endian encoded into the least
significant byte. Extended planar formats (420P10 etc.) had chroma
shift, endian, and component bit depth encoded. (This has been removed
in recent commits.)
Replace the FourCC mess with a simple enum. Remove all the redundant
formats like YV12/I420/IYUV. Replace some image format names by
something more intuitive, most importantly IMGFMT_YV12 -> IMGFMT_420P.
Add img_fourcc.h, which contains the old IDs for code that actually uses
FourCCs. Change the way demuxers, that output raw video, identify the
video format: they set either MP_FOURCC_RAWVIDEO or MP_FOURCC_IMGFMT to
request the rawvideo decoder, and sh_video->imgfmt specifies the pixel
format. Like the previous hack, this is supposed to avoid the need for
a complete codecs.cfg entry per format, or other lookup tables. (Note
that the RGB raw video FourCCs mostly rely on ffmpeg's mappings for NUT
raw video, but this is still considered better than adding a raw video
decoder - even if trivial, it would be full of annoying lookup tables.)
The TV code has not been tested.
Some corrective changes regarding endian and other image format flags
creep in.
These functions weren't specific to video filters and were misplaced
in vf.c. Move them to mp_image.c.
Fix the big endian test in vf_mpi_clear/mp_image_clear, which has been
messed up in 74df1d.
According to DOCS/OUTDATED-tech/colorspaces.txt, the following formats
are supposed to be palettized:
IMGFMT_BGR8
IMGFMT_RGB8,
IMGFMT_BGR4_CHAR
IMGFMT_RGB4_CHAR
IMGFMT_BGR4
IMGFMT_RGB4
Of these, only BGR8 and RGB8 are actually treated as palettized in some
way. ffmpeg has only one palettized format (AV_PIX_FMT_PAL8), and
IMGFMT_BGR8 was inconsistently mapped to packed non-palettized RGB
formats too (AV_PIX_FMT_BGR8). Moreover, vf_scale.c contained messy
hacks to generate a palette when AV_PIX_FMT_BGR8 is output. (libswscale
does not support AV_PIX_FMT_PAL8 output in the first place.)
Get rid of all of this, and introduce IMGFMT_PAL8, which directly maps
to AV_PIX_FMT_PAL8. Remove the palette creation code from vf_scale.c.
IMGFMT_BGR8 maps to AV_PIX_FMT_RGB8 (don't ask me why it's swapped),
without any palette use. Enabling it in vo_x11 or using it as vf_scale
input seems to give correct results.
mp_image_alloc_planes() allocated images with minimal stride, even if
the resulting stride was unaligned. It was the responsibility of
vf_get_image() to set an image's width to something larger than
required to get an aligned stride, and then crop it. Always allocate
with aligned strides instead.
Get rid of IMGFMT_IF09 special handling. This format is not used
anymore. (IF09 has 4x4 chroma sub-sampling, and that is what it was
mainly used for - this is still supported.) Get rid of swapped chroma
plane allocation. This is not used anywhere, and VOs like vo_xv,
vo_direct3d and vo_sdl do their own swapping.
Always round chroma width/height up instead of down. Consider 4:2:0 and
an uneven image size. For luma, the size was left uneven, and the chroma
size was rounded down. This doesn't make sense, because chroma would be
missing for the bottom/right border.
Remove mp_image_new_empty() and mp_image_alloc_planes(), they were not
used anymore, except in draw_bmp.c. (It's still allowed to setup
mp_images manually, you just can't allocate image data with them
anymore - this is also done in draw_bmp.c.)
Replace the internal pixel format stuff with code that queries the
libavutil list of pixel format descriptors.
Trying to map IMGFMT_IS_RGB() etc. turned out extremely hacky.
Instead of using a callback to "capture" the image next time the filter
function is called, do it the other way around: on every filter
invocation, create a reference to the image, and return it if a
screenshot is requested. This also fixes the 1-frame delay when taking
screenshots with the filter.
This also allows simplifying screenshot.c.
VFCAP_TIMER disables any additional waiting done by mpv in the
playloop. Remove VFCAP_TIMER, but re-use the idea for vo_image and
vo_lavc.
This means --untimed doesn't have to be passed when using --vo=image.
This allows avoiding a copy.
The logic about using the AVFrame.reference field if buffer hints are
not available is similar to mplayer-svn's direct rendering code. This is
needed because h264 decoding doesn't set buffer hints.
<wm4> michaelni: couldn't it set buffer hints from AVFrame.reference then?
<michaelni> i see no reason ATM why that wouldnt be possible
OK...
Remove mp_image.width/height. The w/h members are the ones to use.
width/height were used internally by vf_get_image(), and sometimes for
other purposes.
Remove some image flags, most of which are now useless or completely
unused. This includes VFCAP_ACCEPT_STRIDE: the vf_expand insertion in
vf.c does nothing.
Remove some other unused mp_image fields.
Some rather messy changes in vo_opengl[_old] to get rid of legacy
mp_image flags and fields. This is left from when vo_gl supported DR.
Change the entire filter API to use reference counted images instead
of vf_get_image().
Remove filter "direct rendering". This was useful for vf_expand and (in
rare cases) vf_sub: DR allowed these filters to pass a cropped image to
the filters before them. Then, on filtering, the image was "uncropped",
so that black bars could be added around the image without copying. This
means that in some cases, vf_expand will be slower (-vf gradfun,expand
for example).
Note that another form of DR used for in-place filters has been replaced
by simpler logic. Instead of trying to do DR, filters can check if the
image is writeable (with mp_image_is_writeable()), and do true in-place
if that's the case. This affects filters like vf_gradfun and vf_sub.
Everything has to support strides now. If something doesn't, making a
copy of the image data is required.
The code was entirely correct, as the VO doesn't report VFCAP_ACCEPT_STRIDE
in query_format. Add stride capability in preparation for changing the
video chain: soon all VOs will have to support arbitrary strides.
The code assumed mp_image_alloc() would allocate an image large enough
for corevideo's stride, which doesn't have to be the case. If
corevideo's stride was larger than the stride of mp_image, the memcpy()
would write beyond the mp_image allocation.
This probably didn't actually happen, but fix the code to be more
correct anyway.
Setting the size of a mp_image must be done with mp_image_set_size()
now. Do this to guarantee that the redundant fields (like chroma_width)
are updated consistently. Replacing the redundant fields by function
calls would probably be better, but there are too many uses of them,
and is a bit less convenient.
Most code actually called mp_image_setfmt(), which did this as well.
This commit just makes things a bit more explicit.
Warning: the video filter chain still sets up mp_images manually,
and vf_get_image() is not updated.
Refcounting will conceptually allocate and free images all the time
when using the filter chain. Add a pool that makes these reallocations
cheap.
This only affects the image data, not mp_image structs and similar small
allocations. Small allocations are always fast with reasonable memory
managers, while large image data will trigger mmap/munmap calls each
time.
Note that if the codec doesn't support DR1, the image has to be copied.
There is no other way to guarantee that the image will be valid after
decoding the next image.
The only important codec that doesn't support DR1 yet is rawvideo. It's
likely that ffmpeg/Libav will fix this at some time. For now, this
decoder uses an evil hack and puts pointers to the packet data into the
returned frame. This means the image will actually get invalid as soon
as the corresponding video packet is free'd, so copying the image is the
only reasonable thing anyway.
