This changes log statements from log to fs package, which is required for --use-json-log
to properly make log output in JSON format. The recently added custom linting rule,
handled by ruleguard via gocritic via golangci-lint, warns about these and suggests
the alternative. Fixing was therefore basically running "golangci-lint run --fix",
although some manual fixup of mainly imports are necessary following that.
This adds a new optional parameter to the backend, to specify a path
to a unix domain socket to connect to, instead the specified URL.
The URL itself is still used for the rest of the HTTP client, allowing
host and subpath to stay intact.
This allows using rclone with the webdav backend to connect to a WebDAV
server provided at a Unix Domain socket:
rclone serve webdav --addr unix:///tmp/my.socket remote:path
rclone --webdav-unix-socket /tmp/my.socket --webdav-url http://localhost lsf :webdav:
In corporate environments, client certificates have short life times
for added security, and they get renewed automatically. This means
that client certificate can expire in the middle of long running
command such as `mount`.
This commit attempts to reload the client certificates 30s before they
expire.
This will be active for all backends which use HTTP.
The following command will block for 60s(default) when the network is slow or unavailable:
```
rclone --contimeout 10s --low-level-retries 0 lsd dropbox:
```
This change will make it timeout after the expected 10s.
Signed-off-by: rkonfj <rkonfj@gmail.com>
This patch adds rclone_http_status_code counter vector labeled by
* host,
* method,
* code.
It allows to see HTTP errors, backoffs etc.
The Metrics struct is designed for extensibility.
Adding new metrics is a matter of adding them to Metrics struct and including them in the response handling.
This feature has been discussed in the forum [1].
[1] https://forum.rclone.org/t/prometheus-metrics/14484
Reapply missing bwlimiting which was inserted in
0a932dc1f2 Add --bwlimit for upload and download #1873
But accidentally removed when merging
edfe183ba2 fshttp: add DSCP support with --dscp for QoS with differentiated services
This change uses the bwlimit code to apply limits to the receive and
transmit data functions in the HTTP Transport.
This means that all HTTP transactions will have limiting applied -
this includes listings for example.
For HTTP based transorts this makes the limiting in Accounting
redundant and possibly counter productive
This is done by making fs.Config private and attaching it to the
context instead.
The Config should be obtained with fs.GetConfig and fs.AddConfig
should be used to get a new mutable config that can be changed.
Before this change the expect/continue timeout was set to
--conntimeout which was 60s by default which is too long to wait.
This was noticed when using s3 with a proxy which apparently didn't
support expect / continue properly.
Set --expect-continue-timeout 0 to disable expect/continue.
Cookies are handled by cookiejar in memory with fshttp module through
the entire session.
One useful scenario is, with HTTP storage system where index server
adds authentication cookie while redirecting to CDN for actual files.
Also, it can be helpful to reuse fshttp in other storage systems
requiring cookie.
The purpose of this is to make it easier to maintain and eventually to
allow the rclone backends to be re-used in other projects without
having to use the rclone configuration system.
The new code layout is documented in CONTRIBUTING.
