Fixes: 673/clusterfuzz-testcase-5948736536576000
Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/targets/ffmpeg
Reviewed-by: Paul B Mahol <onemda@gmail.com>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
This fixes ubsan warnings in non debug builds by using unsigned operations
in debug builds the correct signed operations are retained so that overflows
(which should not occur in valid files and may indicate problems in the DSP code
or decoder) can be detected.
Alternatively they can be changed to unsigned unconditionally, then its
not possible though to detect overflows easily if someone wants to test
the DSP code for overflows.
The 2nd alternative would be to leave the code as it is and accept that
there are undefined operations in the DSP code and that ubsan output is
full of them in some cases.
Similar changes would be needed in some other DSP routines
Suggested-by: Matt Wolenetz <wolenetz@google.com>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
benchmarked on sandybridge x86_64:
1358232 decicycles in flac_lpc_32_c
1244575 decicycles in flac_lpc_32_sse4, James Almer's patch
650045 decicycles in flac_lpc_32_sse4, this patch
I haven't tested the edgecases such as odd block lengths
odd block length tested-by: James Almer <jamrial@gmail.com>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
* qatar/master:
FATE: add a 24-bit FLAC encoding test
FATE: rename FLAC tests from flac-* to flac-16-*
flacenc: use RICE2 entropy coding mode for 24-bit
flacenc: add 24-bit encoding
flacdsp: move lpc encoding from FLAC encoder to FLACDSPContext
Merged-by: Michael Niedermayer <michaelni@gmx.at>
* qatar/master:
mpegvideo: drop unnecessary arguments to hpel_motion()
mpegvideo: drop 'inline' from some functions
nellymoserdec: drop support for s16 output.
bmpdec: only initialize palette for pal8.
build: Properly remove object files while cleaning
flacdsp: arm optimised lpc filter
compat/vsnprintf: return number of bytes required on truncation.
Merged-by: Michael Niedermayer <michaelni@gmx.at>
* qatar/master:
Print full compiler identification, not only version number
flacdec: reverse lpc coeff order, simplify filter
x86: dsputil: drop some unused CPU flag debug code
Conflicts:
cmdutils.c
configure
Merged-by: Michael Niedermayer <michaelni@gmx.at>
* qatar/master:
configure: add functions for testing code fragments
af_amix: avoid spurious EAGAIN.
af_amix: return AVERROR(EAGAIN) when request_frame didn't produce output.
af_amix: only consider negative return codes as errors.
avconv: use only meaningful timestamps in start time check.
avconv: fix the check for -ss as an output option.
mss3: add forgotten 'static' qualifier for private table
lavc: options: add planar names for request_sample_fmt
flacdec: add planar output support
flvdec: Treat all nellymoser versions as the same codec
Conflicts:
ffmpeg.c
Merged-by: Michael Niedermayer <michaelni@gmx.at>
* qatar/master: (29 commits)
lavfi: reclassify showfiltfmts as a TESTPROG
graph2dot: fix printf format specifier
swscale: yuv2planeX 8bit >=sse2 functions need aligned stack on x86-32.
vp8: loopfilter >=sse2 functions need aligned stack on x86-32.
amr: remove shift out of the AMR_BIT() macro.
dsputilenc: group yasm and inline asm function pointer assignment.
mov: use forward declaration of a function instead of a table.
Clarify Doxygen comment for FF_API_* #defines.
configure: simplify get_version()
Create version.h headers for libraries that lack them
gitignore: Use full path instead of relative path to specify patterns
mpegvideo: remove VLAs
Add XTEA encryption support in libavutil
Add Blowfish encryption support in libavutil
eval: Add the isinf() function and tests for it
flacdec: move lpc filter to flacdsp
flacdec: split off channel decorrelation as flacdsp
avplay: Add an option for not limiting the input buffer size
FATE: add a test for WMA cover art.
FATE: add a test for apetag cover art
...
Conflicts:
.gitignore
configure
ffplay.c
libavcodec/Makefile
libavcodec/error_resilience.c
libavcodec/mpegvideo.c
libavcodec/ratecontrol.c
libavdevice/avdevice.h
libavfilter/Makefile
libavfilter/filtfmts.c
libavfilter/version.h
libavformat/mov.c
libavformat/version.h
libavutil/Makefile
libavutil/avutil.h
libavutil/version.h
libswscale/swscale.h
libswscale/x86/swscale_mmx.c
tests/fate/libavutil.mak
tests/lavfi-regression.sh
tools/graph2dot.c
Merged-by: Michael Niedermayer <michaelni@gmx.at>
