The structure of a ProRes frame in mov/mp4 is that of a typical atom:
First a 32 bit BE size field, then a tag detailling the content. Said
size field includes the eight bytes of the atom header.
This header is actually redundant, as the size of the atom is already
known from the containing atom. It is therefore stripped away when muxed
into Matroska and so the Matroska demuxer has to recreate upon demuxing.
But it did not account for the fact that the size field includes the
size of the header and this can lead to problems when a decoder uses the
in-band size field.
Fixes ticket #8210.
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
Signed-off-by: James Almer <jamrial@gmail.com>
matroska_reset_status (a function that is used during seeking (among
other things)) used an int for the return value of avio_seek which
returns an int64_t. Checking the return value then indicated an error
even though the seek was successfull for targets in the range of
2GB-4GB, 6GB-8GB, ... This error implied that the status hasn't been
reset and in particular, the old level was still considered to be in
force, so that ebml_parse returned errors because the newly parsed
elements were of course not contained in the previously active and still
wrongly considered active master element any more.
Addresses ticket #8084.
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
Signed-off-by: James Almer <jamrial@gmail.com>
The typedef used to define EbmlSyntax already includes a const qualifier
so that it is unnecessary to include another const qualifier in future
definitions and declarations. Given that MSVC warns about this, this
commit removes these redundant const qualifiers.
Suggested-by: Hendrik Leppkes <h.leppkes@gmail.com>
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
Unknown-length elements end when an element not allowed in them, but
allowed at a higher level is encountered. In order to check for this,
c1abd95a added a pointer to every syntax level's parent to each
EbmlSyntax. Given that the parent must of course also reference the
child in order to be able to enter said child level, one needs to use
forward declarations.
These forward declarations constitute tentative definitions and tentative
definitions with internal linkage (like our syntaxes) must not be an
incomplete type. Yet they were an incomplete type and while GCC and
Clang did not even warn about this (on default warning levels), it
broke compilation with MSVC. Therefore this commit adds the sizes.
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
If a file uses unknown-length level 1 elements besides clusters and such
elements are after the first cluster, then these elements will usually
be parsed twice: Once during parsing of the file header and once when
reading the file reaches the position where these elements are located.
The second time the element is parsed leads to a "Duplicate element"
error message. Known-length elements are not affected by this as they
are skipped except during parsing the header.
This commit fixes this by explicitly adding a check for whether the
position of the element to be parsed is the same as the position of the
already known level 1 element.
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
This commit converts the MatroskaLevel1Element struct to use file-based
offsets, as opposed to the current practice of using offsets relative to
the beginning of the segment in it. This also includes a change from
uint64_t to int64_t.
This is in preparation to another patch that improves the check for
duplicate level 1 elements.
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
Up until now, one last kind of unknown-length element hasn't been
properly handled: Unknown-length elements that are supposed to be
skipped, i.e. the level 1 elements that might reside after the
clusters.
This commit changes this. To do this, ebml_parse got a mode that
essentially tries to skip everything except when parsing is needed
(namely for unknown-length elements for which parsing is necessary
as they can't be skipped). This mode is selected by using a NULL
as destination where the parsed data should be written to.
It is used to parse the level 1 elements in matroska_parse_cluster.
The syntax list used for parsing must of course include links to
the syntax of all the master elements that might need to be parsed.
In other words: Instead of matroska_clusters (which contained every
level 1 element except clusters as EBML_NONE elements designated to
be skipped) matroska_segment is needed and used; matroska_clusters has
been removed.
Furthermore, matroska_segment has been reordered so that clusters are at
the front as this is now the most common case for this list.
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
matroska_probe did not support the case of an unknown-length EBML header
at all; given that libavformat's Matroska muxer used to produce such
files in the streaming case, support for them has been added.
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
The current Matroska specifications mandate that only two elements may
use an unknown-length length: Segments and clusters. But this was not
always so: For the greater part of Matroska's existence, all master
elements were allowed to make use of the unknown-length feature.
And there were muxers creating such files: For several years
libavformat's Matroska muxer used unknown-length for all master
elements when the output wasn't seekable. This only stopped in March
2010 with 2529bb30. And even afterwards it was possible (albeit
unlikely) for libavformat to create unknown-length master elements
that are in violation of today's specifications, namely if the master
element was so big that the seek backwards to update the size could
no longer be performed inside the AVIOContext's write buffer. This
has only been fixed in October 2016 (with the patches that introduced
support for writing CRC-32 elements).
Libavformat's Matroska demuxer meanwhile has never really supported
unknown-length elements besides segments and clusters. Support for the
latter was hardcoded. This commit changes this: Now all master elements
for which a syntax to parse them is available are supported. This
includes the files produced by old versions of libavformat's muxer.
More precisely, master elements that have unknown length and are about
to be parsed (not skipped) are supported; only a warning is emitted for
them. For normal files, this means that level 1 elements after the
clusters that are encountered after the clusters have been parsed (i.e.
not because they are referenced by the seekhead at the beginning of the
file) are still unsupported (they would be skipped at this point if
their length were known).
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
1. Up until now, the error message for EBML numbers whose length exceeds
the limits imposed upon them because of the element's type did not
distinguish between known-length and unknown-length elements. As a
consequence, the numerical value of the define constant
EBML_UNKNOWN_LENGTH was emitted as part of the error message which is
of course not appropriate. This commit changes this by adding error
messages designed for unknown-length elements.
2. We impose some (arbitrary) sanity checks on the lengths of certain
element types; these checks were conducted before the checks depending
on whether the element exceeds its containing master element. Now the
order has been reversed, because a failure at the (formerly) latter
check implies that the file is truly erroneous and not only fails our
arbitrary length limit. Moreover, this increases the informativeness of
the error messages.
3. Furthermore, the error message in general has been changed by replacing
the type of the element (something internal to this demuxer and
therefore suitable as debug output at best, not as an error message
intended for ordinary users) with the element ID. The element's position
has been added, too.
4. Finally, the length limit for EBML_NONE elements has been changed so
that all unknown-length elements of EBML_NONE-type trigger an error.
This is done because unknown-length elements can't be skipped and need
to be parsed, but there is no syntax to parse available for EBML_NONE
elements. This is done in preparation for a further patch which allows
more unknown-length elements than just clusters and segments.
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
The Matroska (and WebM) file format achieves forward-compability by
insisting that demuxers ignore and skip elements they don't know about.
Unfortunately, this complicates the detection of errors as errors
resulting from loosing sync can't be reliably distinguished from
unknown elements that are part of a future version of the standard.
Up until now, the strategy to deal with this situation was to skip all
unknown elements that are not obviously erroneous; if an error happened,
it was tried to seek to the last known good position to resync from (and
resync to level 1 elements). This is working fine if the input is
seekable, but if it is not, then the skipped data can usually not be
rechecked lateron. This is particularly acute if unknown-length clusters
are in use, as the check for whether a child element exceeds the
containing master element is ineffective in this situation.
To remedy this, a new heuristic has been introduced: If an unknown
element is encountered in non-seekable mode, an error is presumed to
have happened based upon a combination of the length of the row of the
already encountered unknown elements and of how far away skipping this
element would take us.
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
Cosmetics include reordering EbmlType so that EBML_SINT is adjacent to
the other numbers (and matches the order in the switch in ebml_parse)
and also reordering the switch for assignment of default values so that
it matches the order in EbmlType.
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
Up until now, avio_tell was used multiple times in ebml_parse and its
subroutines, although the result of these calls can usually be simply
derived from the result of earlier calls to avio_tell. This has been
changed. Unnecessary calls to avio_tell in ebml_parse are avoided now.
Furthermore, there has been a slight change in the output of some error
messages relating to elements exceeding their containing master element:
The reported position of the element now points to the first byte of the
element ID and no longer to the first byte of the element's payload.
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
This commit closes the last hole in the system of checks for a
known-length file ending too early: Now an error message is emitted
in case the file ends directly after an EBML element.
Furthermore, this commit adds a check and a corresponding warning
whether there is data beyond the Matroska segment (only reasonable for
known-length segments). If everything looks alright, then parsing is
stopped as soon as EOF is reached (in contrast, the earlier code would
always call matroska_resync at the end).
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
By including SimpleBlocks and BlockGroups twice in the same EbmlSyntax
array (with different semantics), one can reduce the duplication of the
other values.
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
The new code does not rely on whether the cluster's position is set or
not to infer whether a cluster needs to be closed or not (instead, this
is done in ebml_parse), so there is no need to reset the cluster's
position at all any more. It will be automatically set to the correct
value when a cluster is entered.
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
Before this commit, the parsing of clusters mixed EBML levels by
allowing elements from different levels in a EbmlSyntax (namely
matroska_cluster_parsing). This has been changed. And the level
is now explicitly used to determine how to parse.
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
This commit changes how levels are handled: If the level used for
ebml_parse ends directly after an element that has been consumed, then
ebml_parse ends the level itself (and any known-length levels that end
there as well) and informs the caller via the return value; if the
current level is of unknown-length, then the level is ended as soon as
an element that is not valid on the current level, but on a higher
level is encountered (or if EOF has been encountered).
This is designed for situations where one wants to parse master elements
incrementally, i.e. not in one go via ebml_parse_nest.
The (incremental) parsing of clusters still mixes levels by using a
syntax list that contains elements from different levels and the level
is still ended manually via a call to ebml_level_end if the last cluster
was an unknown-length cluster (known-length clusters are already ended
when their last element is read), but only if the next element is a
cluster, too. A different level 1 element following an unknown-length
cluster will currently simply be presumed to be part of the earlier
cluster. Fixing this will be done in a future patch. The modifications
to matroska_parse_cluster contained in this patch are only intended not
to cause regressions.
Nevertheless, the fact that known-length levels are automatically ended
in ebml_parse when their last element has been read already fixes a bogus
error message introduced in 9326117b that was emitted when a known-length
cluster is followed by another level 1 element other than a cluster in
which case the cluster's level was not ended (which only happened when
a new cluster has been encountered) so that the length check (introduced
in 9326117b) failed for the level 1 element as it is of course not
contained in the previous cluster. Most Matroska files were affected by
this.
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
By linking to the syntax of the parent (i.e. the containing master
element) one can check whether an element is actually part of a higher
level in the EBML hierarchy. Knowing this is important for
unknown-length levels, because they end when an element that doesn't
belong to this, but to a higher hierarchy level is encountered.
Sometimes there are different syntaxes dealing with the same elements.
In this case it is important to use a parent that contains all the
elements at the parent level; whether this is the syntax actually used
to enter the child's level is irrelevant. This affects the list of level
1 elements (which has been used as parent for matroska_cluster, too) and
it affects recursive elements (currently only the SimpleTag), where the
non-recursive parent has to be choosen.
This is in preparation for a patch that redoes level handling.
Finally, the segment id has been added to ebml_syntax. This will enable
handling of unknown-length EBML headers.
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
Currently, resyncing during reading packets works as follows:
The current position is recorded, then a call to matroska_parse_cluster
is made and if said call fails, the demuxer tries to resync from the
earlier position. If the call doesn't fail, but also doesn't deliver a
packet, then this is looped.
There are two problems with this approach:
1. The Matroska file format aims to be forward-compatible; to achieve
this, a demuxer should simply ignore and skip elements it doesn't
know about. But it is not possible to reliably distinguish unknown
elements from junk. If matroska_parse_cluster encounters an unknown
element, it can therefore not simply error out; instead it returns zero
and the loop is iterated which includes an update of the position that
is intended to be used in case of errors, i.e. the element that is
skipped is not searched for level 1 element ids to resync to at all if
later calls to matroska_parse_cluster return an error.
Notice that in case that sync has been lost there can be a chain of
several unknown/possibly junk elements before an error is detected.
2. Even if a call to matroska_parse_cluster delivers a packet, this does
not mean that everything is fine. E.g. it might be that some of the
block's data is missing and that the data that was presumed to be from
the block just read actually contains the beginning of the next element.
This will only be apparent at the next call of matroska_read_packet,
which uses the (false) end of the earlier block as resync position so
that in the (not unlikely) case that the call to matroska_parse_cluster
fails, the data believed to be part of the earlier block is not searched
for a level 1 element to resync to.
To counter this, a "last known good" position is introduced. When an
element id that is known to be allowed at this position in the hierarchy
(according to the syntax currently in use for parsing) is read and some
further checks (regarding the length of the element and its containing
master element) are passed, then the beginning of the current element is
treated as a "good" position and recorded as such in the
MatroskaDemuxContext. Because of 2., only the start of the element is
treated as a "good" position, not the whole element. If an error occurs
later during parsing of clusters, the resync process starts at the last
known good position.
Given that when the header is damaged the subsequent resync never skips over
data and is therefore unaffected by both issues, the "last known good"
concept is not used there.
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
Since the changes to the parsing of SimpleBlocks, both ebml_parse_id and
ebml_parse_elem are only called from one place, so that it is possible
to inline these two function calls. This is done, but not completely:
ebml_parse_id still exists in a modified form. This is done in
preparation for a further patch regarding the handling of
unknown-length elements.
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
The earlier code set the level to zero upon seeking and after a
discontinuity although in both cases parsing (re)starts at a level 1
element.
Also set the segment's length to unkown if an error occured in order not
to drop any valid data that happens to be beyond the designated end of
the segment.
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
This function will be useful later to reset the status (e.g. current
level and the already parsed id).
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
When an error happens, the Matroska demuxer tries to resync to level 1
elements from an earlier position onwards. If the seek to said earlier
position fails, the demuxer currently treats this as an unrecoverable
error. And that behaviour is suboptimal as said failure is nothing
unrecoverable or unexpected (when the input isn't seekable).
It is preferable to simply resync from the earliest position available
(i.e. the start of the AVIOContext's buffer) onwards if the seek failed.
Here are some scenarios that might be treated as unrecoverable errors
by the current code if the input isn't seekable. They all have in
common that the current position is so far away from the desired
position that the seek can't be fulfilled from the AVIOContext's buffer:
1. Blocks (both SimpleBlocks as well as a Block in a BlockGroup) for
which reading them as binary EBML elements succeeds, but whose parsing
triggers an error (e.g. an invalid TrackNumber). In this case the
earlier position from which resyncing begins is at the start of the block
(or even earlier).
2. BlockGroups, whose parsing fails in one of the latter elements. Just
as in 1., the start of the BlockGroup (the target of the seek) might be
so far away from the current position that it is no longer in the
buffer.
3. At the beginning of parsing a cluster, the cluster is parsed until a
SimpleBlock or a BlockGroup is encountered. So if the input is damaged
between the beginning of the cluster and the first occurrence of a
SimpleBlock/BlockGroup and if said damage makes the demuxer read/skip so
much data that the beginning of the cluster is no longer in the buffer,
demuxing will currently fail completely.
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
This commit fixes an overflow introduced in a569a7b3 that affected EBML
elements that the Matroska demuxer doesn't want to parse like CRC-32
elements. The return value of avio_skip (the new position on success or
an AVERROR on failure) has been assigned to an integer which meant that
new positions in the range of 2GB to 4GB-1 etc. were considered errors.
Fixes ticket #8001.
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
Signed-off-by: James Almer <jamrial@gmail.com>
Up until now, when an element was skipped, it was relied upon
ffio_limit to make sure that there is enough data available to skip.
ffio_limit itself relies upon the availability of the file's size. As
this needn't be available, the check has been refined: First one byte
less than intended is skipped, then another byte is read, followed by a
check of the error flags.
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
This commit fixes a number of bugs:
1. There was no check that no read error/EOF occured during
ebml_read_uint, ebml_read_sint and ebml_read_float.
2. ebml_read_ascii and ebml_read_binary did sometimes not forward
error codes; instead they simply returned AVERROR(EIO).
3. In particular, AVERROR_EOF hasn't been used and no dedicated error
message for it existed. This has been changed.
In order to reduce code duplication, the new error code NEEDS_CHECKING
has been introduced which makes ebml_parse check the AVIOContext's
status for errors.
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
ebml_read_num had a number of flaws:
1. The check for read errors/EOF was totally wrong. E.g. an EBML number
beginning with the invalid 0x00 would be considered a read error,
although it is just invalid data.
2. The check for read errors/EOF was done just once, after reading the
first byte of the EBML number. But errors/EOF can happen inbetween, of
course, and this wasn't checked.
3. There was no way to distinguish when EOF should be an error (because
the data has to be there) for which an error message should be emitted
and when it is not necessarily an error (namely during parsing of EBML
IDs). Such a possibility has been added and used.
All this was fixed; furthermore, the error messages for invalid EBML
numbers were improved and useless initializations were removed.
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
Up until now, webm_dash_manifest_cues used the return values of
ebml_read_num and ebml_read_length without checking for errors,
i.e. return values < 0. This has been changed.
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
It is only necessary to zero the initial allocated memory used to store
the size of laced frames if the block used Xiph lacing. Otherwise no
unintialized data was ever used, so use av_malloc instead of av_mallocz.
Also use the correct type for the allocations.
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
Up until now, the SimpleBlock was treated specially: It basically had
its own EBML category and it was also included in the BlockGroup EBML
syntax (although a SimpleBlock must not exist in a BlockGroup according
to the Matroska specifications). The latter fact also meant that
a MatroskaBlock's buffer was always unreferenced twice.
This has been changed: The type of a SimpleBlock is now an EBML_BIN.
The only way in which SimpleBlocks are still different is that they
share their associated structure with another unit (namely BlockGroup).
This is also used to unref the block: It is always unreferenced via the
BlockGroup syntax.
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
Before this commit, the Matroska muxer would read a block when required
to do so, parse the block, create and return the necessary AVPackets and
yet keep the blocks (in a dynamically allocated list), although they
aren't used at all any more. This has been changed. There is no list any
more and the block is immediately discarded after parsing.
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
Every new element of an EbmlList is zeroed initially in
ebml_parse_elem, so that in particular a SimpleBlock's duration is
initialized to zero. Therefore it is unnecessary to initialize this
field again (for SimpleBlocks) in matroska_parse_cluster_incremental.
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
By default, the data_offset member of the AVFormatInternal of the
AVFormatContext associated with the MatroskaDemuxContext has not been
initialized explicitly by any Matroska-specific function, so that it was
initialized by default to the offset at the end of matroska_read_header,
i.e. usually to the offset of the length field of the first encountered
cluster. This meant that in case that the Matroska-specific seek-code
fails because there are no index entries for the target track a seek to
data_offset would be performed and ordinary parsing would start from
there which is nonsense: The length field would be treated as EBML ID and
(if the length field is not longer than four bytes (EBML numbers that
long are rejected as invalid EBML IDs)) whatever comes next would be
treated as its EBML size although it simply isn't.
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
The earlier code relied on the length of clusters always being coded on
eight bytes as was the behaviour of libavformat's Matroska muxer until
recently. But given that our own Matroska muxer now (and mkvmerge from
time immemorial) creates files that don't conform to this assumption,
it is high time to get rid of this assumption.
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
When the new incremental parser was introduced, the old parser was
kept, because the new parser was unable to handle the way SSA packets
are put into Matroska. But since 2014 (since c7d8dbad) this is no
longer needed, so that the old parser can be completely removed.
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
This commit replaces copying attached pictures by using references to
the already existing buffers.
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
Matroska EBML IDs can be only four bytes long maximally, so it is
natural to use uint32_t for them. By doing this and rearranging the
elements of the MatroskaLevel1Element structure, one can reduce the size
of said structure.
Notice that this field is not read via the generic reading process for
EBML_UINT, so one is not forced to use an uint64_t for it.
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
This error message is outdated since d31fb1a9.
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
The earlier code had three flaws:
1. The case of an unknown-sized element inside a finite-sized element
(which is against the specifications) was not caught.
2. The error message wasn't helpful: It compared the length of the child
with the offset of the end of the parent and claimed that the first
exceeds the latter, although that is not necessarily true.
3. Unknown-sized elements that are not parsed can't be skipped. Given
that according to the Matroska specifications only the segment and the
clusters can be of unknown-size, this is handled by not allowing any
other units to have infinite size whereas the earlier code would seek
back by 1 byte upon encountering an infinite-size element that ought
to be skipped.
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@googlemail.com>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
Unifying the way the EBML unknown length is signaled, rather than using two
incompatible values. UINT64_MAX cannot be read as a valid EBML length with the
current code.
Co-authored-by: Steve Lhomme <robux4@ycbcr.xyz>
Co-authored-by: Dale Curtis <dalecurtis@chromium.org>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
This was found through the Hacker One program on VLC but is not a security issue in libavformat
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
Instead add the character to the snprintf above as suggested by Mark.
Silences a warning:
libavformat/matroskadec.c: In function 'webm_dash_manifest_cues':
libavformat/matroskadec.c:3947:13: warning: 'strncat' specified bound 1 equals source length [-Wstringop-overflow=]
strncat(buf, ",", 1);
^~~~~~~~~~~~~~~~~~~~
This will get ISOBMFF and Matroska up to date with the revised AV1 Codec
Configuration Box spec.
For now keep propagating raw OBUs as extradata until all libavcodec modules
are adapted to handle AV1CodecConfigurationRecord formatted extradata.
Tested-by: Thomas Daede <bztdlinux@gmail.com>
Signed-off-by: James Almer <jamrial@gmail.com>
Newly allocated data buffers (wavpack, prores, compressed buffers)
are padded to meet the requirements of AVPacket.
About 10x speed up in matroska_parse_frame().
Reviewed-by: Michael Niedermayer <michael@niedermayer.cc>
Signed-off-by: James Almer <jamrial@gmail.com>
Simplifies code in matroska_parse_frame(). This is in preparation for
the following patch.
Reviewed-by: Michael Niedermayer <michael@niedermayer.cc>
Signed-off-by: James Almer <jamrial@gmail.com>
Data in EbmlBin objects is never changed after being read from the
input file (save for two specific cases with encoded CodePrivate), so
using AVBufferRef we can prevent unnecessary copy of data by instead
creating new references to said constant data.
Signed-off-by: James Almer <jamrial@gmail.com>
Defined in a recent revision of https://www.webmproject.org/docs/container/
This prevents storing the contents of CodecPrivate into extradata for
a codec that doesn't need nor expect any. It will among other things
prevent matroska specific binary data from being dumped onto other
formats during remuxing.
Signed-off-by: James Almer <jamrial@gmail.com>
There's at least one known file with a TrueHD stream that hasn't
been correctly muxed, and requires full frame parsing and repack.
Signed-off-by: James Almer <jamrial@gmail.com>
* commit '55fe72a841ba306370e68e86c88f34b4456aa4dd':
matroskadec: don't warn about unknown spherical medata when none is present
Merged-by: James Almer <jamrial@gmail.com>
track->video.projection.type is set to 0 (a Matroska specific "No spherical
metadata present" value, with no related AVSphericalMapping) by default on
files without the element.
This removes bogus warnings on every single matroska file without Spherical
metadata.
Signed-off-by: James Almer <jamrial@gmail.com>
* commit '251849f06ce36ce8dc076e0fca2922119fa7e39e':
mkv: Add support for Spherical Video elements
See 445204cd57
Merged-by: James Almer <jamrial@gmail.com>
Fixes Coverity CID: 1405453
Reviewed-by: wm4 <nfxjfg@googlemail.com>
Reviewed-by: Hendrik Leppkes <h.leppkes@gmail.com>
Signed-off-by: Steven Liu <lq@chinaffmpeg.org>
The WebM DASH spec states:
The Initialization Segment shall not contain Clusters or Cues.
The Segment Index corresponds to the Cues.
Previously, it included the cues if they were at the front.
Signed-off-by: Derek Buitenhuis <derek.buitenhuis@gmail.com>
Output was apparently not tested for correctness. Passing overlapping
memory to snprintf causes undefined behavior, and usually resulted in
only the very last timestamp being written to metadata, and not a list
at all.
Signed-off-by: Derek Buitenhuis <derek.buitenhuis@gmail.com>
Add an option to webm_dash_manifest demuxer to specify a value for
"bandwidth" field in the DASH manifest. The value is then used by
the muxer. Fixes an existing FIXME in the code.
Signed-off-by: Vignesh Venkatasubramanian <vigneshv@google.com>
Signed-off-by: James Zern <jzern@google.com>
These values are defined to be 32bit in the specification,
so it makes more sense to store them as fixed width.
Based on a patch by Micahel Niedermayer <michael@niedermayer.cc>.
Signed-off-by: Vittorio Giovara <vittorio.giovara@gmail.com>
These values are defined to be 32bit in the specification,
so it makes more sense to store them as fixed width.
Based on a patch by Micahel Niedermayer <michael@niedermayer.cc>.
Signed-off-by: Vittorio Giovara <vittorio.giovara@gmail.com>
Blocks are marked as key frames whenever the "reference" field is
zero. This breaks for non-keyframe Blocks with a reference timestamp
of zero.
The likelihood of reference timestamp being zero is increased by a
longstanding bug in muxing that encodes reference timestamp as the
absolute time of the referenced frame (rather than relative to the
current Block timestamp, as described in MKV spec).
Now using INT64_MIN to denote "no reference".
Reported to chromium at http://crbug.com/497889 (contains sample)
If a read fails, the current code will free the data but leave the size
non-zero. Make sure the size is zeroed in such a case.
CC: libav-stable@libav.org
Bug-Id: 1001
Found-By: Kamil Frankowicz
Signed-off-by: Sean McGovern <gseanmcg@gmail.com>
The demuxer doesn't fill the defaults if the master isn't present.
This results in codecpar->color_space being set with a value of
zero (RGB) on such files.
Signed-off-by: James Almer <jamrial@gmail.com>
This reverts parts of c16582579b. The hard
coded 30 seconds are a lot, and finishing the seek can takes several
seconds when the source is on a network share. Remove this code
entirely, because it does more bad than good.
(Commit message provided by committer, based on the original messages
by the patch author.)
Signed-off-by: Rainer Hochecker <fernetmenta@online.de>
Signed-off-by: wm4 <nfxjfg@googlemail.com>
Using the decode interrupt feature of ffmpeg may cause crashes by
accessing previously freed pointers in matroska_read_close.
To prevent this reset nb_elem to zero after freeing the elements,
because ffmpeg normally tests for nb_elem.
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
The code assumes that s->streams[0] is valid.
Reviewed-by: Michael Niedermayer <michael@niedermayer.cc>
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
If the value is negative then it means padding at the start of the packet
instead of at the end.
Based on a patch by Hendrik Leppkes.
Reviewed-by: James Zern <jzern-at-google.com@ffmpeg.org>
Signed-off-by: James Almer <jamrial@gmail.com>
This way it's more clear what the default values refer to, as
Undetermined/Unspecified is 0 for some, 2 for others.
Signed-off-by: James Almer <jamrial@gmail.com>
The problem was introduced in commit 1273bc6.
Reviewed-by: James Almer <jamrial@gmail.com>
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
The code works just fine regardless of unit, so only make sure DisplayUnit
is not "unknown".
Found-by: Nicolas George <george@nsup.org>
Reviewed-by: Nicolas George <george@nsup.org>
Signed-off-by: James Almer <jamrial@gmail.com>
A missing DisplayUnit element or one with the default value of 0 means
DisplayWidth and DisplayHeight should be interpreted as pixels.
The current code setting st->sample_aspect_ratio is wrong when DisplayUnit
is anything else.
Reviewed-by: Carl Eugen Hoyos <ceffmpeg@gmail.com>
Signed-off-by: James Almer <jamrial@gmail.com>
It is supposed to be a flag. The only currently defined value is
AVIO_SEEKABLE_NORMAL, but other ones may be added in the future.
However all the current lavf code treats this field as a bool (mainly
for historical reasons).
Change all those cases to properly check for AVIO_SEEKABLE_NORMAL.
Use proper ISO 8601 timestamps which also signal that they are in UTC.
This changes the format of creation_time and modification_date metadata values
from 2016-06-01 22:30:00 to 2016-01-01T22:30:00.000000Z
Fixes ticket #5673.
Signed-off-by: Marton Balint <cus@passwd.hu>
When seeking a file where codec delay is greater than 0, the timecode
can become negative after offsetting by the codec delay. Failing to cast
to a signed int64 will cause the check against skip_to_timecode to evaluate
true for these negative values. This breaks the "skip_to" seek mechanism.
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
Mkvtoolnix stores the sample rate of the original stream as reported by the
"OpusHead" stream header instead of 48kHz, the actual sample rate of the Opus
stream.
Ignoring the stored sample rate and forcing 48kHz preserves the correct initial
padding when remuxing such files.
Reviewed-by: Michael Niedermayer <michael@niedermayer.cc>
Signed-off-by: James Almer <jamrial@gmail.com>
Check if the size is written the first 4 bytes and read the next 4
as fourcc candidate, fallback checking the initial for 4 bytes.
"The CodecPrivate contains all additional data that is stored in the
'stsd' (sample description) atom in the QuickTime file after the
mandatory video descriptor structure (starting with the size and FourCC
fields)"
CC: libav-stable@libav.org
Adding early support for a subset of the proposed colour elements
according to the latest version of spec:
https://mailarchive.ietf.org/arch/search/?email_list=cellar&gbt=1&index=hIKLhMdgTMTEwUTeA4ct38h0tmE
I've left out elements for pix_fmt related things as there still
seems to be some discussion around these, and the max_cll/max_fall
are currently not propagated as there is not yet side data for them.
The new elements are exposed under strict experimental mode.
Signed-off-by: Neil Birkbeck <neil.birkbeck@gmail.com>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
Currently, AVStream contains an embedded AVCodecContext instance, which
is used by demuxers to export stream parameters to the caller and by
muxers to receive stream parameters from the caller. It is also used
internally as the codec context that is passed to parsers.
In addition, it is also widely used by the callers as the decoding (when
demuxer) or encoding (when muxing) context, though this has been
officially discouraged since Libav 11.
There are multiple important problems with this approach:
- the fields in AVCodecContext are in general one of
* stream parameters
* codec options
* codec state
However, it's not clear which ones are which. It is consequently
unclear which fields are a demuxer allowed to set or a muxer allowed to
read. This leads to erratic behaviour depending on whether decoding or
encoding is being performed or not (and whether it uses the AVStream
embedded codec context).
- various synchronization issues arising from the fact that the same
context is used by several different APIs (muxers/demuxers,
parsers, bitstream filters and encoders/decoders) simultaneously, with
there being no clear rules for who can modify what and the different
processes being typically delayed with respect to each other.
- avformat_find_stream_info() making it necessary to support opening
and closing a single codec context multiple times, thus
complicating the semantics of freeing various allocated objects in the
codec context.
Those problems are resolved by replacing the AVStream embedded codec
context with a newly added AVCodecParameters instance, which stores only
the stream parameters exported by the demuxers or read by the muxers.
Use av_realloc() rather than av_malloc() when normalizing noncompliant
private data in get_qt_codec().
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
This patch adds a new static function get_qt_codec() that takes care of
the initial retrieval of the fourcc and codec ID for A_QUICKTIME and
V_QUICKTIME. It also normalizes noncompliant private data found in some
older files that incorrectly starts with the fourcc by expanding/shifting
the data by 4 bytes, and storing the data size at the start. This is
necessary in order for the rest of the code in the A_QUICKTIME and
V_QUICKTIME blocks (and most likely other code as well) to correctly
parse the private data.
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
In many older QuickTime files, the audio format, or "fourcc", is
0x00000000. The QuickTime File Format Specification states the following
regarding this situation:
"This format descriptor should not be used, but may be found in some
files. Samples are assumed to be stored in either 'raw ' or 'twos'
format, depending on the sample size field in the sound description."
MPlayer handles this logic by itself, but FFmpeg/FFplay currently does
not.
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
Palettized QuickTime video in Matroska has hitherto not been recognized
whatsoever, and the "palette" used has been completely random.
The patch for matroskadec.c fixes this issue by adding a palette side
data packet in matroska_deliver_packet(), much in the same way as it's
done in mov.c.
Video samples for testing are available at
https://drive.google.com/open?id=0B3_pEBoLs0faWElmM2FnLTZYNlk.
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
matroskaenc applies divisors to the display width/height when generating
stereo content. This patch adds the corresponding multipliers to matroskadec
so that the original sample aspect ratio can be recovered.
Signed-off-by: wm4 <nfxjfg@googlemail.com>
Signed-off-by: Vittorio Giovara <vittorio.giovara@gmail.com>
* commit 'a0fa6d06b848f26b16ba12f0a9a4a85b93ab8022':
matroska: Warn when metadata references a non-existent element
Merged-by: Hendrik Leppkes <h.leppkes@gmail.com>
matroskaenc.c applies divisors to the display width/height when generating
stereo content. This patch adds the corresponding multipliers to matroskadec.c
so that the original sample aspect ratio can be recovered.
Signed-off-by: wm4 <nfxjfg@googlemail.com>
Unrecognized streams are not allocated
Fixes: flicker-1.color1.vp91447030769.08.webm
Found-by: Chris Cunningham <chcunningham@google.com>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
* commit '948f3c19a8bd069768ca411212aaf8c1ed96b10d':
lavc: Make AVPacket.duration int64, and deprecate convergence_duration
Merged-by: Hendrik Leppkes <h.leppkes@gmail.com>
Note that convergence_duration had another meaning, one which was in
practice never used. The only real use for it was a 64 bit replacement
for the duration field. It's better just to make duration 64 bits, and
to get rid of it.
Signed-off-by: Vittorio Giovara <vittorio.giovara@gmail.com>
In the TTA extradata re-construction the values are written with
avio_wl16 and if they don't fit into uint16_t, this triggers an
av_assert2 in avio_w8.
Reviewed-by: Michael Niedermayer <michaelni@gmx.at>
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
And default to 8000 if it is invalid.
An invalid sample rate can trigger av_assert2 in av_rescale_rnd.
Reviewed-by: Michael Niedermayer <michaelni@gmx.at>
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
Some files have SeekHead elements with broken IDs. They mismatch with
the ID of the destination element. These files are written by
"IDMmkvlib0.1" (as identified by the MuxingApp and WritingApp elements),
and the SeekHead IDs are actually endian-swapped.
This confuses the SeekHead logic of the demuxer. It will read some
elements twice, because the SeekHead ID is used to identify and remember
already read elements. With the file at hand, the stream list was
duplicated by reading the Tracks element twice.
Fix this by rejecting invalid EBML IDs in SeekHead entries. (This fix is
relatively specific to the broken file at hand, and doesn't protect
against some other cases of broken SeekHead, such as valid but
mismatching target element IDs.)
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
This fixes a segmentation fault.
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
index_scale is set to matroska->time_scale of type uint64_t.
When index_scale is int, the assignment can overflow and e.g. result
in index_scale = 0. This causes a floating point exception due to the
division by index_scale.
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
Generally, libavformat exports cover art pictures as video streams with
1 packet and AV_DISPOSITION_ATTACHED_PIC set. Only matroskadec exported
it as attachment with codec_id set to AV_CODEC_ID_MJPEG.
Obviously, this should be consistent, so change the Matroska demuxer to
export a AV_DISPOSITION_ATTACHED_PIC pseudo video stream.
Matroska muxing is probably incorrect too. I know that it can create
broken files with an audio track and just 1 video frame when e.g.
remuxing mp3 with APIC to mkv. But for now this commit does not change
anything about muxing, and also continues to write attachments with
AV_CODEC_ID_MJPEG should the muxer application have special knowledge
that the Matroska is broken in this way.
Signed-off-by: Anton Khirnov <anton@khirnov.net>
This patch adds support for parsing live files (produced by
-f webm_chunk) which contains only the headers but no packets. This
is only used when using -f webm_dash_manifest. There will be a
follow up patch which adds live support to WebM DASH Manifest
muxer.
Signed-off-by: Vignesh Venkatasubramanian <vigneshv@google.com>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
Generally, libavformat exports cover art pictures as video streams with
1 packet and AV_DISPOSITION_ATTACHED_PIC set. Only matroskadec exported
it as attachment with codec_id set to AV_CODEC_ID_MJPEG.
Obviously, this should be consistent, so change the Matroska demuxer to
export a AV_DISPOSITION_ATTACHED_PIC pseudo video stream.
Matroska muxing is probably incorrect too. I know that it can create
broken files with an audio track and just 1 video frame when e.g.
remuxing mp3 with APIC to mkv. But for now this commit does not change
anything about muxing, and also continues to write attachments with
AV_CODEC_ID_MJPEG should the muxer application have special knowledge
that the Matroska is broken in this way.
Fixes trac #4423.
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
Just because the user requested the seek index to be ignored, we can't
just skip essential headers. At least tags are often located at the end
of the file, and the old code simply ignored the seekhead for all
elements, not just the cue index. Also, it looks like it used the index
even if IGNIDX was set if the cue index was located in the beginning of
the file.
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
In particular, this reads chained seekheads. This makes seeking faster
in files which have the index indirectly linked through 2 seekheads.
As a side-effect, this warns when reading level-1 (toplevel) elements
multiple times (other than seekheads, clusters, and void/crc). Such
elements are not valid and likely break everything.
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
Matroska is an extensible format - unknown elements must be expected. It
shouldn't complain about such elements to the user either; it'll just
generate noise. The "error_recognition & AV_EF_EXPLODE" is completely,
wrong why would it explode on valid files?
It's still useful for debugging, so the message is left in place with a
higher log level.
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
In matroska_read_seek(), |tracks| is assigned at the begining of the
function. However, functions like matroska_parse_cues() could reallocate
the tracks and invalidate |tracks|.
This assigns |tracks| only before using it, so that it will not get
invalidated elsewhere.
Bug-Id: chromium/427266
When matroska_execute_seekhead() is called, it goes through the list of
seekhead entries and attempts to read elements not read yet. When doing
this, the parser can find further SeekHead elements, and will extend the
matroska->seekhead list. This can lead to a (practically) infinite loop
with certain broken files. (Maybe it can happen even with valid files.
The demuxer doesn't seem to check correctly whether an element has
already been read.)
Fix this by ignoring elements that were added to the seekhead field
during executing seekhead entries.
This does not fix the possible situation when multiple SeekHead elements
after the file header (i.e. occur after the "before_pos" file position)
point to the same elements. These elements will probably be parsed
multiple times, likely leading to bugs.
Fixes ticket #4162.
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
Ensures that the header include order is such that winsock2.h is always
included before windows.h or that windows.h does not include winsock.h.
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
In matroska_read_seek(), |tracks| is assigned at the begining of the function.
However, functions like matroska_parse_cues() could reallocate the tracks so
that |tracks| can get invalidated.
This CL assigns |tracks| only before we use it so that it won't be invalidated.
BUG=427266
TEST=Test case in associated bug passes now.
Change-Id: I9c7065fe8f4311ca846076281df2282d190ed344
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
Fix basename computation code to allow just file names without any
directories in the path.
Signed-off-by: Vignesh Venkatasubramanian <vigneshv@google.com>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
Fix incorrect bandwidth computation in some cases. When the cue end
descriptor is null (i.e.) start_time_ns == -1, existing bandwidth
computed (if any) should be returned rather than returning 0.
Signed-off-by: Vignesh Venkatasubramanian <vigneshv@google.com>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
Fix an incorrect hard code in cues_end computation. Updating the fate
test reference files related to the fix as well. The earlier computation
was clearly wrong as the cues_end field was greater than the file size
itself in some cases.
Signed-off-by: Vignesh Venkatasubramanian <vigneshv@google.com>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
James Almer