Until now, stream_peek() read only the bare minimum it had to read from
the stream. But this could cause problems, such as being very
inefficient when peeking a lot, or conflicting with ability to seek
back. (The latter issue can be caused by peeking a few bytes, and then
doing a stream_read() with a size that is 1 byte longer: this would read
the peeked data, then call stream_fill_buffer(), which throws away the
previously peeked bytes - so you can't seek back anymore. This is
mitigated by a hack in demux_open(): it peeks a full buffer, to avoid
that peeking/reading during demuxer probing [or before that, in a stream
filter] can cause the buffer to be dropped.)
Apparently, it is popular to store large files in uncompressed rar
archives. Extracting files is not practical, and some media players
suport playing directly from uncompressed rar (at least VLC and some
DirectShow components).
Storing or accessing files this way is completely idiotic, but it is
a common practice, and the ones subjected to this practice can't do
much to change this (at least that's what I assume/hope). Also, it's
a feature request, so we say yes.
This code is mostly taken from VLC (commit f6e7240 from their git tree).
We also copy the way this is done: opening a rar file by itself yields
a playlist, which contains URLs to the actual entries in the rar file.
Compressed entries are simply skipped.
Add a stream filter concept, in which streams can be opened on top of
an underlying "source" stream. Change the open code to make this
easier, and also to account for some mechanisms that will be needed
for this.
The following commit will add stream_rar, which contains such a stream
filter.
This is really not needed. While we really can't take a loaded buffer
over to the cache, there's no reason why the cache couldn't read this
buffer normally.
On the other hand, this code could cause trouble when probing from a
stream before the cache has been enabled.
The way the url_options field was handled was not entirely sane: it's
actually a flexible array member, so it points to garbage for streams
which do not initialize this member (it just points to the data right
after the struct, which is garbage in theory and practice). This was
not actually a problem, since the field is only used if priv_size is
set (due to how this stuff is used). But it doesn't allow setting
priv_size only, which might be useful in some cases.
Also, make the protocols array not a fixed size array. Most stream
implementations have only 1 protocol prefix, but stream_lavf.c has
over 10 (whitelists ffmpeg protocols). The high size of the fixed
size protocol array wastes space, and it is _still_ annoying to
add new prefixes to stream_lavf (have to bump the maximum length),
so make it arbitrary length.
The two changes (plus some more cosmetic changes) arte conflated into
one, because it was annoying going over all the stream implementations.
Modeled after the old playlist_parser.c, but actually new code, and it
works a bit differently.
Demuxers (and sometimes streams) are the component that should be used
to open files and to determine the file format. This was already done
for subtitles, but playlists still use a separate code path.
Instead of always skipping in STREAM_BUFFER_SIZE blocks, allow an
arbitrary size. This allows - in theory - faster forward seeking in
pipes.
(Maybe not a very significant change, but it reduces the number of
things that depend on STREAM_BUFFER_SIZE for no good reason. Though
we still use that value as minimum read size.)
stream_file.c contains some code meant for forward seeking with pipes.
This simply reads data until the seek position is reached. Move this
code to stream.c. This stops stream_file from doing strange things
(messing with stream internals), and removes the code duplication too.
We also make stream_seek_long() use the new skip code. This is shorter
and much easier to follow than the old code, which basically did strange
things.
So for example "file:///file%20name.mkv" will open "file name.mkv".
I'm not sure whether we want/need this. The old code didn't do it.
Also, it's not really clear whether this is handled correctly. It
seems the corresponding freedesktop.org "standard" allows a (useless)
hostname part, which we should skip in theory. The number of slashes
is not really clear either. We can open relative filenames (by removing
one of the slashes from the example above), which is perhaps an
unneeded feature. How does this even work with Windows paths?
This issues can probably be corrected later.
The URL unescape code is based on code from m_option.c removed with
a recent commit.
Move the URL parsing code from m_option.c to stream.c, and simplify it
dramatically. This code originates from times when http code used this,
but now it's just relict from other stream implementations reusing this
code. Remove the unused bits and simplify the rest.
stream_vcd is insane, and the priv struct is different on every
platform, so drop the URL parsing. This means you can't specify a track
anymore, only the device. (Does anyone use stream_vcd? Not like this
couldn't be fixed, but it doesn't seem worth the effort, especially
because it'd require potentially touching platform specific code.)
These were printed only with -v. Most streams had them set to useless
or redundant values, so it's just badly maintained bloat.
Since we remove the "author" field too, and since this may have
copyright implications, we add the contents of the author fields to
the file headers, except if the name is already part of the file header.
Stream implementations could set this to a unix file descriptor. The
generic stream code could use it as fallback for a few things. This
was confusing and insane. In most cases, the stream implementations
defined all callbacks, so setting the fd member didn't have any
advantages, other than avoiding defining a private struct to store it.
It appears that even if the stream implementation used close() on the
fd (or something equivalent), stream.c would close() it a second time
(and on windows, even would call closesocket()), which should be proof
for the insanity of this code.
For stream_file.c, additionally make sure we don't close stdin or
stdout if "-" is used as filename.
For stream_vcd.c, remove the control() code. This code most likely
didn't make the slightest sense, because it used a different type
for stream->priv. It also leaked memory. Maybe it worked, but it's
incorrect and insignificant anyway, so kill it. This code was added
with commit 9521c19 (svn commit 31019).
Untested for all protocols other than stream_file.c.
This removes the dependency on DEMUXER_TYPE_* and the file_format
parameter from the stream open functions.
Remove some of the playlist handling code. It looks like this was
needed only for loading linked mov files with demux_mov (which was
removed long ago).
Delete a minor bit of dead network-related code from stream.c as well.
Add this option, which lets users set the cache size without forcing it
even when playing from the local filesystem.
Also document the default value explicitly.
The Matroska linked segments case is slightly simplified: they can
never come from network (mostly because it'd be insane, and we can't
even list files from network sources), so the cache will never be
enabled automatically.
Commit 7b16d4b changed some stream implementations to check the buffer
size passed to them. This made stream_cdda stop working, because the
default buffer size is smaller than the CDIO frame size. So pass the
sector size instead of the (arbitrary) default buffer size.
Seeking to position 0 meant to try reconnecting with some streams,
actually just the internal http implementation. This has been removed,
so we don't need the special handling anymore.
This means we don't have to be stuck in a retry loop if the stream
doesn't even support reconnect.
This was an old leftover from an earlier cleanup (which happened in
2003), and which used "special" stuff for streams that could be only
forward-seeked.
Also, don't add mode flags to s->flags; they're supposed to be in
s->mode instead.
This was under CONFIG_NETWORKING, so in theory it should have been save
to remove. But actually this disables forward skipping when reading
from a pipe. (Still a questionable feature, because it doesn't behave
well with libavformat - but it was not supposed to be changed.)
This commit removes the "old" networking code in favor of libavformat's
code.
The code was still used for mp_http, udp, ftp, cddb. http has been
mapped to libavformat's http support since approximately 6 months ago.
udp and ftp have support in ffmpeg (though ftp was added only last
month). cddb support is removed with this commit - it's probably not
important and rarely used if at all, so we don't care about it.
When reading something successfully, the eof flag should never be 1, so
clear it in these situations. The eof flag will be set properly on the
next read call.
stream_read_unbuffered() can sometimes return negative values on error.
Change that to return 0 - the negative values are nowhere used anyway.
If distinguishing errors and EOF is really needed, a flag could be added
instead.
This also fixes the stream_read_partial() call in cache.c, which assumes
the return values is always >= 0.
Seems like a completely unnecessary complication. Instead, always add a
1 byte padding (could be extended if a caller needs it), and clear it.
Also add some documentation. There was some, but it was outdated and
incomplete.
This is useless on the cache side. The sector is needed only to deal
with stream implementations which are not byte addressable, and the
cache is always byte addressable.
Also set a default read_chunk value. (This value is never used unless
you chain multiple caches, but it's cleaner.)
This was probably done this way to ensure that after a successful seek,
the reported stream position is the same as the requested seek position.
But it doesn't make too much sense, since both stream->pos and the
stream implementation's internal position will go out of sync.
The stream EOF flag should only be set when trying to read past the end
of the file (relatively similar to unix files). Always clear the EOF
flag on seeking. Trying to set it "properly" (depending whether data is
available at seek destination or not) might be an ok idea, but would
require attention to too many special cases. I suspect before this
commit (and in MPlayer etc. too), the EOF flag wasn't handled
consistently when the stream position was at the end of the file.
Fix one special case in ebml.c and stream_skip(): this function couldn't
distinguish between at-EOF and past-EOF either.
EOF should be set when reading more data fails. The stream
implementations have nothing to say here and should behave correctly
when trying to read when EOF was actually read.
Even when seeking, a correct EOF flag should be guaranteed. stream_seek()
(or actually stream_seek_long()) calls stream_fill_buffer() at least
once, which also updates the EOF flag.
This function was called in various places. Most time, it was used
before a seek. In other cases, the purpose was apparently resetting
the EOF flag. As far as I can see, this makes no sense anymore. At
least the stream_reset() calls paired with stream_seek() are completely
pointless. A seek will either seek inside the buffer (and reset the
EOF flag), or do an actual seek and reset all state.
This happens with something like "mpv https://www.youtube.com/watch".
The URL is obviously not valid, but the stream layer tries to reconnect.
This commit at least allows to use the terminal to abort gracefully.
(Other than killing the process.)
Basically rewrite all the code supporting the cache (i.e. anything other
than the ringbuffer logic). The underlying design is untouched.
Note that the old cache2.c (on which this code is based) already had a
threading implementation. This was mostly unused on Linux, and had some
problems, such as using shared volatile variables for communication and
uninterruptible timeouts, instead of using locks for synchronization.
This commit does use proper locking, while still retaining the way the
old cache worked. It's basically a big refactor.
Simplify the code too. Since we don't need to copy stream ctrl args
anymore (we're always guaranteed a shared address space now), lots of
annoying code just goes away. Likewise, we don't need to care about
sector sizes. The cache uses the high-level stream API to read from
other streams, and sector sizes are handled transparently.
demux_lavf probes up to 2 MB of data in the worst case. When the ffmpeg
demuxer is actually opened, the stream is seeked back to 0, and the
previously read data is thrown away.
This wasn't a problem for playback of local files, but it's less than
ideal for playing from slow media (like web streams), and breaks
completely if the media is not seekable (pipes, some web streams).
This new function is intended to allow fixing this. demux_lavf will use
it to put the read probe data back into the buffer.
The simplest way of implementing this function is by making it
transparently extend the normal stream buffer. This makes sure no
existing code is broken by new weird special cases. For simplicity
and to avoid possible performance loss due to extra dereferencing
when accessing the buffer, we just extend the static buffer from
8 KB to 2 MB. Normally, most of these 2 MB will stay uncommitted, so
there's no associated waste of memory. If demux_lavf really reads all
2 MB, the memory will be committed and stay unused, though.
Before this commit, the cache was franken-hacked on top of the stream
API. You had to use special functions (like cache_stream_fill_buffer()
instead of stream_fill_buffer()), which would access the stream in a
cached manner.
The whole idea about the previous design was that the cache runs in a
thread or in a forked process, while the cache awa functions made sure
the stream instance looked consistent to the user. If you used the
normal functions instead of the special ones while the cache was
running, you were out of luck.
Make it a bit more reasonable by turning the cache into a stream on its
own. This makes it behave exactly like a normal stream. The stream
callbacks call into the original (uncached) stream to do work. No
special cache functions or redirections are needed. The only different
thing about cache streams is that they are created by special functions,
instead of being part of the auto_open_streams[] array.
To make things simpler, remove the threading implementation, which was
messed into the code. The threading code could perhaps be kept, but I
don't really want to have to worry about this special case. A proper
threaded implementation will be added later.
Remove the cache enabling code from stream_radio.c. Since enabling the
cache involves replacing the old stream with a new one, the code as-is
can't be kept. It would be easily possible to enable the cache by
requesting a cache size (which is also much simpler). But nobody uses
stream_radio.c and I can't even test this thing, and the cache is
probably not really important for it either.
