videolan/ffmpeg
1
Fork 0
mirror of https://git.videolan.org/git/ffmpeg.git synced 2024-07-26 06:01:30 +02:00
Code Releases Activity

avcodec/hqxdsp: Fix multiple runtime error: signed integer overflow: 248220 * 21407 cannot be represented in type 'int' in idct_col()

Browse Source 
Fixes: 1405/clusterfuzz-testcase-minimized-5011491835084800

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/targets/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
This commit is contained in:
Michael Niedermayer 2017-05-08 12:04:09 +02:00
parent 279420b5a6
commit 5d5118f81b
1 changed files with 6 additions and 6 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

12
libavcodec/hqxdsp.c
View File

@ -39,18 +39,18 @@ static inline void idct_col(int16_t *blk, const uint8_t *quant)
s6 = (int) blk[6 * 8] * quant[6 * 8];
s7 = (int) blk[7 * 8] * quant[7 * 8];
t0 = (s3 * 19266 + s5 * 12873) >> 15;
t1 = (s5 * 19266 - s3 * 12873) >> 15;
t2 = ((s7 * 4520 + s1 * 22725) >> 15) - t0;
t3 = ((s1 * 4520 - s7 * 22725) >> 15) - t1;
t0 = (int)(s3 * 19266U + s5 * 12873U) >> 15;
t1 = (int)(s5 * 19266U - s3 * 12873U) >> 15;
t2 = ((int)(s7 * 4520U + s1 * 22725U) >> 15) - t0;
t3 = ((int)(s1 * 4520U - s7 * 22725U) >> 15) - t1;
t4 = t0 * 2 + t2;
t5 = t1 * 2 + t3;
t6 = t2 - t3;
t7 = t3 * 2 + t6;
t8 = (t6 * 11585) >> 14;
t9 = (t7 * 11585) >> 14;
tA = (s2 * 8867 - s6 * 21407) >> 14;
tB = (s6 * 8867 + s2 * 21407) >> 14;
tA = (int)(s2 * 8867U - s6 * 21407U) >> 14;
tB = (int)(s6 * 8867U + s2 * 21407U) >> 14;
tC = (s0 >> 1) - (s4 >> 1);
tD = (s4 >> 1) * 2 + tC;
tE = tC - (tA >> 1);