mirror of
https://github.com/rclone/rclone
synced 2024-12-23 14:23:44 +01:00
49320 lines
1.3 MiB
Generated
49320 lines
1.3 MiB
Generated
.\"t
|
|
.\" Automatically generated by Pandoc 2.5
|
|
.\"
|
|
.TH "rclone" "1" "Mar 31, 2021" "User Manual" ""
|
|
.hy
|
|
.SH Rclone syncs your files to cloud storage
|
|
.PP
|
|
.IP \[bu] 2
|
|
About rclone
|
|
.IP \[bu] 2
|
|
What can rclone do for you?
|
|
.IP \[bu] 2
|
|
What features does rclone have?
|
|
.IP \[bu] 2
|
|
What providers does rclone support?
|
|
.IP \[bu] 2
|
|
Download (https://rclone.org/downloads/)
|
|
.IP \[bu] 2
|
|
Install (https://rclone.org/install/)
|
|
.IP \[bu] 2
|
|
Donate. (https://rclone.org/donate/)
|
|
.SS About rclone
|
|
.PP
|
|
Rclone is a command line program to manage files on cloud storage.
|
|
It is a feature rich alternative to cloud vendors\[aq] web storage
|
|
interfaces.
|
|
Over 40 cloud storage products support rclone including S3 object
|
|
stores, business & consumer file storage services, as well as standard
|
|
transfer protocols.
|
|
.PP
|
|
Rclone has powerful cloud equivalents to the unix commands rsync, cp,
|
|
mv, mount, ls, ncdu, tree, rm, and cat.
|
|
Rclone\[aq]s familiar syntax includes shell pipeline support, and
|
|
\f[C]\-\-dry\-run\f[R] protection.
|
|
It is used at the command line, in scripts or via its API (/rc).
|
|
.PP
|
|
Users call rclone \f[I]\[dq]The Swiss army knife of cloud
|
|
storage\[dq]\f[R], and \f[I]\[dq]Technology indistinguishable from
|
|
magic\[dq]\f[R].
|
|
.PP
|
|
Rclone really looks after your data.
|
|
It preserves timestamps and verifies checksums at all times.
|
|
Transfers over limited bandwidth; intermittent connections, or subject
|
|
to quota can be restarted, from the last good file transferred.
|
|
You can check (https://rclone.org/commands/rclone_check/) the integrity
|
|
of your files.
|
|
Where possible, rclone employs server\-side transfers to minimise local
|
|
bandwidth use and transfers from one provider to another without using
|
|
local disk.
|
|
.PP
|
|
Virtual backends wrap local and cloud file systems to apply
|
|
encryption (https://rclone.org/crypt/),
|
|
caching (https://rclone.org/cache/),
|
|
compression (https://rclone.org/compress/)
|
|
chunking (https://rclone.org/chunker/) and
|
|
joining (https://rclone.org/union/).
|
|
.PP
|
|
Rclone mounts (https://rclone.org/commands/rclone_mount/) any local,
|
|
cloud or virtual filesystem as a disk on Windows, macOS, linux and
|
|
FreeBSD, and also serves these over
|
|
SFTP (https://rclone.org/commands/rclone_serve_sftp/),
|
|
HTTP (https://rclone.org/commands/rclone_serve_http/),
|
|
WebDAV (https://rclone.org/commands/rclone_serve_webdav/),
|
|
FTP (https://rclone.org/commands/rclone_serve_ftp/) and
|
|
DLNA (https://rclone.org/commands/rclone_serve_dlna/).
|
|
.PP
|
|
Rclone is mature, open source software originally inspired by rsync and
|
|
written in Go (https://golang.org).
|
|
The friendly support community are familiar with varied use cases.
|
|
Official Ubuntu, Debian, Fedora, Brew and Chocolatey repos.
|
|
include rclone.
|
|
For the latest version downloading from
|
|
rclone.org (https://rclone.org/downloads/) is recommended.
|
|
.PP
|
|
Rclone is widely used on Linux, Windows and Mac.
|
|
Third party developers create innovative backup, restore, GUI and
|
|
business process solutions using the rclone command line or API.
|
|
.PP
|
|
Rclone does the heavy lifting of communicating with cloud storage.
|
|
.SS What can rclone do for you?
|
|
.PP
|
|
Rclone helps you:
|
|
.IP \[bu] 2
|
|
Backup (and encrypt) files to cloud storage
|
|
.IP \[bu] 2
|
|
Restore (and decrypt) files from cloud storage
|
|
.IP \[bu] 2
|
|
Mirror cloud data to other cloud services or locally
|
|
.IP \[bu] 2
|
|
Migrate data to cloud, or between cloud storage vendors
|
|
.IP \[bu] 2
|
|
Mount multiple, encrypted, cached or diverse cloud storage as a disk
|
|
.IP \[bu] 2
|
|
Analyse and account for data held on cloud storage using
|
|
lsf (https://rclone.org/commands/rclone_lsf/),
|
|
ljson (https://rclone.org/commands/rclone_lsjson/),
|
|
size (https://rclone.org/commands/rclone_size/),
|
|
ncdu (https://rclone.org/commands/rclone_ncdu/)
|
|
.IP \[bu] 2
|
|
Union (https://rclone.org/union/) file systems together to present
|
|
multiple local and/or cloud file systems as one
|
|
.SS Features
|
|
.IP \[bu] 2
|
|
Transfers
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
MD5, SHA1 hashes are checked at all times for file integrity
|
|
.IP \[bu] 2
|
|
Timestamps are preserved on files
|
|
.IP \[bu] 2
|
|
Operations can be restarted at any time
|
|
.IP \[bu] 2
|
|
Can be to and from network, e.g.
|
|
two different cloud providers
|
|
.IP \[bu] 2
|
|
Can use multi\-threaded downloads to local disk
|
|
.RE
|
|
.IP \[bu] 2
|
|
Copy (https://rclone.org/commands/rclone_copy/) new or changed files to
|
|
cloud storage
|
|
.IP \[bu] 2
|
|
Sync (https://rclone.org/commands/rclone_sync/) (one way) to make a
|
|
directory identical
|
|
.IP \[bu] 2
|
|
Move (https://rclone.org/commands/rclone_move/) files to cloud storage
|
|
deleting the local after verification
|
|
.IP \[bu] 2
|
|
Check (https://rclone.org/commands/rclone_check/) hashes and for
|
|
missing/extra files
|
|
.IP \[bu] 2
|
|
Mount (https://rclone.org/commands/rclone_mount/) your cloud storage as
|
|
a network disk
|
|
.IP \[bu] 2
|
|
Serve (https://rclone.org/commands/rclone_serve/) local or remote files
|
|
over
|
|
HTTP (https://rclone.org/commands/rclone_serve_http/)/WebDav (https://rclone.org/commands/rclone_serve_webdav/)/FTP (https://rclone.org/commands/rclone_serve_ftp/)/SFTP (https://rclone.org/commands/rclone_serve_sftp/)/dlna (https://rclone.org/commands/rclone_serve_dlna/)
|
|
.IP \[bu] 2
|
|
Experimental Web based GUI (https://rclone.org/gui/)
|
|
.SS Supported providers
|
|
.PP
|
|
(There are many others, built on standard protocols such as WebDAV or
|
|
S3, that work out of the box.)
|
|
.IP \[bu] 2
|
|
1Fichier
|
|
.IP \[bu] 2
|
|
Alibaba Cloud (Aliyun) Object Storage System (OSS)
|
|
.IP \[bu] 2
|
|
Amazon Drive
|
|
.IP \[bu] 2
|
|
Amazon S3
|
|
.IP \[bu] 2
|
|
Backblaze B2
|
|
.IP \[bu] 2
|
|
Box
|
|
.IP \[bu] 2
|
|
Ceph
|
|
.IP \[bu] 2
|
|
Citrix ShareFile
|
|
.IP \[bu] 2
|
|
C14
|
|
.IP \[bu] 2
|
|
DigitalOcean Spaces
|
|
.IP \[bu] 2
|
|
Dreamhost
|
|
.IP \[bu] 2
|
|
Dropbox
|
|
.IP \[bu] 2
|
|
Enterprise File Fabric
|
|
.IP \[bu] 2
|
|
FTP
|
|
.IP \[bu] 2
|
|
Google Cloud Storage
|
|
.IP \[bu] 2
|
|
Google Drive
|
|
.IP \[bu] 2
|
|
Google Photos
|
|
.IP \[bu] 2
|
|
HDFS
|
|
.IP \[bu] 2
|
|
HTTP
|
|
.IP \[bu] 2
|
|
Hubic
|
|
.IP \[bu] 2
|
|
Jottacloud
|
|
.IP \[bu] 2
|
|
IBM COS S3
|
|
.IP \[bu] 2
|
|
Koofr
|
|
.IP \[bu] 2
|
|
Mail.ru Cloud
|
|
.IP \[bu] 2
|
|
Memset Memstore
|
|
.IP \[bu] 2
|
|
Mega
|
|
.IP \[bu] 2
|
|
Memory
|
|
.IP \[bu] 2
|
|
Microsoft Azure Blob Storage
|
|
.IP \[bu] 2
|
|
Microsoft OneDrive
|
|
.IP \[bu] 2
|
|
Minio
|
|
.IP \[bu] 2
|
|
Nextcloud
|
|
.IP \[bu] 2
|
|
OVH
|
|
.IP \[bu] 2
|
|
OpenDrive
|
|
.IP \[bu] 2
|
|
OpenStack Swift
|
|
.IP \[bu] 2
|
|
Oracle Cloud Storage
|
|
.IP \[bu] 2
|
|
ownCloud
|
|
.IP \[bu] 2
|
|
pCloud
|
|
.IP \[bu] 2
|
|
premiumize.me
|
|
.IP \[bu] 2
|
|
put.io
|
|
.IP \[bu] 2
|
|
QingStor
|
|
.IP \[bu] 2
|
|
Rackspace Cloud Files
|
|
.IP \[bu] 2
|
|
rsync.net
|
|
.IP \[bu] 2
|
|
Scaleway
|
|
.IP \[bu] 2
|
|
Seafile
|
|
.IP \[bu] 2
|
|
SFTP
|
|
.IP \[bu] 2
|
|
StackPath
|
|
.IP \[bu] 2
|
|
SugarSync
|
|
.IP \[bu] 2
|
|
Tardigrade
|
|
.IP \[bu] 2
|
|
Tencent Cloud Object Storage (COS)
|
|
.IP \[bu] 2
|
|
Wasabi
|
|
.IP \[bu] 2
|
|
WebDAV
|
|
.IP \[bu] 2
|
|
Yandex Disk
|
|
.IP \[bu] 2
|
|
Zoho WorkDrive
|
|
.IP \[bu] 2
|
|
The local filesystem
|
|
.PP
|
|
Links
|
|
.IP \[bu] 2
|
|
Home page (https://rclone.org/)
|
|
.IP \[bu] 2
|
|
GitHub project page for source and bug
|
|
tracker (https://github.com/rclone/rclone)
|
|
.IP \[bu] 2
|
|
Rclone Forum (https://forum.rclone.org)
|
|
.IP \[bu] 2
|
|
Downloads (https://rclone.org/downloads/)
|
|
.SH Install
|
|
.PP
|
|
Rclone is a Go program and comes as a single binary file.
|
|
.SS Quickstart
|
|
.IP \[bu] 2
|
|
Download (https://rclone.org/downloads/) the relevant binary.
|
|
.IP \[bu] 2
|
|
Extract the \f[C]rclone\f[R] or \f[C]rclone.exe\f[R] binary from the
|
|
archive
|
|
.IP \[bu] 2
|
|
Run \f[C]rclone config\f[R] to setup.
|
|
See rclone config docs (https://rclone.org/docs/) for more details.
|
|
.PP
|
|
See below for some expanded Linux / macOS instructions.
|
|
.PP
|
|
See the Usage section (https://rclone.org/docs/#usage) of the docs for
|
|
how to use rclone, or run \f[C]rclone \-h\f[R].
|
|
.PP
|
|
Already installed rclone can be easily updated to the latest version
|
|
using the rclone
|
|
selfupdate (https://rclone.org/commands/rclone_selfupdate/) command.
|
|
.SS Script installation
|
|
.PP
|
|
To install rclone on Linux/macOS/BSD systems, run:
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
curl https://rclone.org/install.sh | sudo bash
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
For beta installation, run:
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
curl https://rclone.org/install.sh | sudo bash \-s beta
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
Note that this script checks the version of rclone installed first and
|
|
won\[aq]t re\-download if not needed.
|
|
.SS Linux installation from precompiled binary
|
|
.PP
|
|
Fetch and unpack
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
curl \-O https://downloads.rclone.org/rclone\-current\-linux\-amd64.zip
|
|
unzip rclone\-current\-linux\-amd64.zip
|
|
cd rclone\-*\-linux\-amd64
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
Copy binary file
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
sudo cp rclone /usr/bin/
|
|
sudo chown root:root /usr/bin/rclone
|
|
sudo chmod 755 /usr/bin/rclone
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
Install manpage
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
sudo mkdir \-p /usr/local/share/man/man1
|
|
sudo cp rclone.1 /usr/local/share/man/man1/
|
|
sudo mandb
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
Run \f[C]rclone config\f[R] to setup.
|
|
See rclone config docs (https://rclone.org/docs/) for more details.
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone config
|
|
\f[R]
|
|
.fi
|
|
.SS macOS installation with brew
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
brew install rclone
|
|
\f[R]
|
|
.fi
|
|
.SS macOS installation from precompiled binary, using curl
|
|
.PP
|
|
To avoid problems with macOS gatekeeper enforcing the binary to be
|
|
signed and notarized it is enough to download with \f[C]curl\f[R].
|
|
.PP
|
|
Download the latest version of rclone.
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
cd && curl \-O https://downloads.rclone.org/rclone\-current\-osx\-amd64.zip
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
Unzip the download and cd to the extracted folder.
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
unzip \-a rclone\-current\-osx\-amd64.zip && cd rclone\-*\-osx\-amd64
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
Move rclone to your $PATH.
|
|
You will be prompted for your password.
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
sudo mkdir \-p /usr/local/bin
|
|
sudo mv rclone /usr/local/bin/
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
(the \f[C]mkdir\f[R] command is safe to run, even if the directory
|
|
already exists).
|
|
.PP
|
|
Remove the leftover files.
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
cd .. && rm \-rf rclone\-*\-osx\-amd64 rclone\-current\-osx\-amd64.zip
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
Run \f[C]rclone config\f[R] to setup.
|
|
See rclone config docs (https://rclone.org/docs/) for more details.
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone config
|
|
\f[R]
|
|
.fi
|
|
.SS macOS installation from precompiled binary, using a web browser
|
|
.PP
|
|
When downloading a binary with a web browser, the browser will set the
|
|
macOS gatekeeper quarantine attribute.
|
|
Starting from Catalina, when attempting to run \f[C]rclone\f[R], a
|
|
pop\-up will appear saying:
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
\[lq]rclone\[rq] cannot be opened because the developer cannot be verified.
|
|
macOS cannot verify that this app is free from malware.
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
The simplest fix is to run
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
xattr \-d com.apple.quarantine rclone
|
|
\f[R]
|
|
.fi
|
|
.SS Install with docker
|
|
.PP
|
|
The rclone maintains a docker image for
|
|
rclone (https://hub.docker.com/r/rclone/rclone).
|
|
These images are autobuilt by docker hub from the rclone source based on
|
|
a minimal Alpine linux image.
|
|
.PP
|
|
The \f[C]:latest\f[R] tag will always point to the latest stable
|
|
release.
|
|
You can use the \f[C]:beta\f[R] tag to get the latest build from master.
|
|
You can also use version tags, e.g.
|
|
\f[C]:1.49.1\f[R], \f[C]:1.49\f[R] or \f[C]:1\f[R].
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
$ docker pull rclone/rclone:latest
|
|
latest: Pulling from rclone/rclone
|
|
Digest: sha256:0e0ced72671989bb837fea8e88578b3fc48371aa45d209663683e24cfdaa0e11
|
|
\&...
|
|
$ docker run \-\-rm rclone/rclone:latest version
|
|
rclone v1.49.1
|
|
\- os/arch: linux/amd64
|
|
\- go version: go1.12.9
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
There are a few command line options to consider when starting an rclone
|
|
Docker container from the rclone image.
|
|
.IP \[bu] 2
|
|
You need to mount the host rclone config dir at \f[C]/config/rclone\f[R]
|
|
into the Docker container.
|
|
Due to the fact that rclone updates tokens inside its config file, and
|
|
that the update process involves a file rename, you need to mount the
|
|
whole host rclone config dir, not just the single host rclone config
|
|
file.
|
|
.IP \[bu] 2
|
|
You need to mount a host data dir at \f[C]/data\f[R] into the Docker
|
|
container.
|
|
.IP \[bu] 2
|
|
By default, the rclone binary inside a Docker container runs with UID=0
|
|
(root).
|
|
As a result, all files created in a run will have UID=0.
|
|
If your config and data files reside on the host with a non\-root
|
|
UID:GID, you need to pass these on the container start command line.
|
|
.IP \[bu] 2
|
|
If you want to access the RC interface (either via the API or the Web
|
|
UI), it is required to set the \f[C]\-\-rc\-addr\f[R] to \f[C]:5572\f[R]
|
|
in order to connect to it from outside the container.
|
|
An explanation about why this is necessary is present
|
|
here (https://web.archive.org/web/20200808071950/https://pythonspeed.com/articles/docker-connection-refused/).
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
NOTE: Users running this container with the docker network set to
|
|
\f[C]host\f[R] should probably set it to listen to localhost only, with
|
|
\f[C]127.0.0.1:5572\f[R] as the value for \f[C]\-\-rc\-addr\f[R]
|
|
.RE
|
|
.IP \[bu] 2
|
|
It is possible to use \f[C]rclone mount\f[R] inside a userspace Docker
|
|
container, and expose the resulting fuse mount to the host.
|
|
The exact \f[C]docker run\f[R] options to do that might vary slightly
|
|
between hosts.
|
|
See, e.g.
|
|
the discussion in this
|
|
thread (https://github.com/moby/moby/issues/9448).
|
|
.RS 2
|
|
.PP
|
|
You also need to mount the host \f[C]/etc/passwd\f[R] and
|
|
\f[C]/etc/group\f[R] for fuse to work inside the container.
|
|
.RE
|
|
.PP
|
|
Here are some commands tested on an Ubuntu 18.04.3 host:
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
# config on host at \[ti]/.config/rclone/rclone.conf
|
|
# data on host at \[ti]/data
|
|
|
|
# make sure the config is ok by listing the remotes
|
|
docker run \-\-rm \[rs]
|
|
\-\-volume \[ti]/.config/rclone:/config/rclone \[rs]
|
|
\-\-volume \[ti]/data:/data:shared \[rs]
|
|
\-\-user $(id \-u):$(id \-g) \[rs]
|
|
rclone/rclone \[rs]
|
|
listremotes
|
|
|
|
# perform mount inside Docker container, expose result to host
|
|
mkdir \-p \[ti]/data/mount
|
|
docker run \-\-rm \[rs]
|
|
\-\-volume \[ti]/.config/rclone:/config/rclone \[rs]
|
|
\-\-volume \[ti]/data:/data:shared \[rs]
|
|
\-\-user $(id \-u):$(id \-g) \[rs]
|
|
\-\-volume /etc/passwd:/etc/passwd:ro \-\-volume /etc/group:/etc/group:ro \[rs]
|
|
\-\-device /dev/fuse \-\-cap\-add SYS_ADMIN \-\-security\-opt apparmor:unconfined \[rs]
|
|
rclone/rclone \[rs]
|
|
mount dropbox:Photos /data/mount &
|
|
ls \[ti]/data/mount
|
|
kill %1
|
|
\f[R]
|
|
.fi
|
|
.SS Install from source
|
|
.PP
|
|
Make sure you have at least Go (https://golang.org/) go1.13 installed.
|
|
Download go (https://golang.org/dl/) if necessary.
|
|
The latest release is recommended.
|
|
Then
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
git clone https://github.com/rclone/rclone.git
|
|
cd rclone
|
|
go build
|
|
\&./rclone version
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
This will leave you a checked out version of rclone you can modify and
|
|
send pull requests with.
|
|
If you use \f[C]make\f[R] instead of \f[C]go build\f[R] then the rclone
|
|
build will have the correct version information in it.
|
|
.PP
|
|
You can also build the latest stable rclone with:
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
go get github.com/rclone/rclone
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
or the latest version (equivalent to the beta) with
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
go get github.com/rclone/rclone\[at]master
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
These will build the binary in \f[C]$(go env GOPATH)/bin\f[R]
|
|
(\f[C]\[ti]/go/bin/rclone\f[R] by default) after downloading the source
|
|
to the go module cache.
|
|
Note \- do \f[B]not\f[R] use the \f[C]\-u\f[R] flag here.
|
|
This causes go to try to update the dependencies that rclone uses and
|
|
sometimes these don\[aq]t work with the current version of rclone.
|
|
.SS Installation with Ansible
|
|
.PP
|
|
This can be done with Stefan Weichinger\[aq]s ansible
|
|
role (https://github.com/stefangweichinger/ansible-rclone).
|
|
.PP
|
|
Instructions
|
|
.IP "1." 3
|
|
\f[C]git clone https://github.com/stefangweichinger/ansible\-rclone.git\f[R]
|
|
into your local roles\-directory
|
|
.IP "2." 3
|
|
add the role to the hosts you want rclone installed to:
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
\- hosts: rclone\-hosts
|
|
roles:
|
|
\- rclone
|
|
\f[R]
|
|
.fi
|
|
.SS Configure
|
|
.PP
|
|
First, you\[aq]ll need to configure rclone.
|
|
As the object storage systems have quite complicated authentication
|
|
these are kept in a config file.
|
|
(See the \f[C]\-\-config\f[R] entry for how to find the config file and
|
|
choose its location.)
|
|
.PP
|
|
The easiest way to make the config is to run rclone with the config
|
|
option:
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone config
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
See the following for detailed instructions for
|
|
.IP \[bu] 2
|
|
1Fichier (https://rclone.org/fichier/)
|
|
.IP \[bu] 2
|
|
Alias (https://rclone.org/alias/)
|
|
.IP \[bu] 2
|
|
Amazon Drive (https://rclone.org/amazonclouddrive/)
|
|
.IP \[bu] 2
|
|
Amazon S3 (https://rclone.org/s3/)
|
|
.IP \[bu] 2
|
|
Backblaze B2 (https://rclone.org/b2/)
|
|
.IP \[bu] 2
|
|
Box (https://rclone.org/box/)
|
|
.IP \[bu] 2
|
|
Cache (https://rclone.org/cache/)
|
|
.IP \[bu] 2
|
|
Chunker (https://rclone.org/chunker/) \- transparently splits large
|
|
files for other remotes
|
|
.IP \[bu] 2
|
|
Citrix ShareFile (https://rclone.org/sharefile/)
|
|
.IP \[bu] 2
|
|
Compress (https://rclone.org/compress/)
|
|
.IP \[bu] 2
|
|
Crypt (https://rclone.org/crypt/) \- to encrypt other remotes
|
|
.IP \[bu] 2
|
|
DigitalOcean Spaces (https://rclone.org/s3/#digitalocean-spaces)
|
|
.IP \[bu] 2
|
|
Dropbox (https://rclone.org/dropbox/)
|
|
.IP \[bu] 2
|
|
Enterprise File Fabric (https://rclone.org/filefabric/)
|
|
.IP \[bu] 2
|
|
FTP (https://rclone.org/ftp/)
|
|
.IP \[bu] 2
|
|
Google Cloud Storage (https://rclone.org/googlecloudstorage/)
|
|
.IP \[bu] 2
|
|
Google Drive (https://rclone.org/drive/)
|
|
.IP \[bu] 2
|
|
Google Photos (https://rclone.org/googlephotos/)
|
|
.IP \[bu] 2
|
|
HDFS (https://rclone.org/hdfs/)
|
|
.IP \[bu] 2
|
|
HTTP (https://rclone.org/http/)
|
|
.IP \[bu] 2
|
|
Hubic (https://rclone.org/hubic/)
|
|
.IP \[bu] 2
|
|
Jottacloud / GetSky.no (https://rclone.org/jottacloud/)
|
|
.IP \[bu] 2
|
|
Koofr (https://rclone.org/koofr/)
|
|
.IP \[bu] 2
|
|
Mail.ru Cloud (https://rclone.org/mailru/)
|
|
.IP \[bu] 2
|
|
Mega (https://rclone.org/mega/)
|
|
.IP \[bu] 2
|
|
Memory (https://rclone.org/memory/)
|
|
.IP \[bu] 2
|
|
Microsoft Azure Blob Storage (https://rclone.org/azureblob/)
|
|
.IP \[bu] 2
|
|
Microsoft OneDrive (https://rclone.org/onedrive/)
|
|
.IP \[bu] 2
|
|
OpenStack Swift / Rackspace Cloudfiles / Memset
|
|
Memstore (https://rclone.org/swift/)
|
|
.IP \[bu] 2
|
|
OpenDrive (https://rclone.org/opendrive/)
|
|
.IP \[bu] 2
|
|
Pcloud (https://rclone.org/pcloud/)
|
|
.IP \[bu] 2
|
|
premiumize.me (https://rclone.org/premiumizeme/)
|
|
.IP \[bu] 2
|
|
put.io (https://rclone.org/putio/)
|
|
.IP \[bu] 2
|
|
QingStor (https://rclone.org/qingstor/)
|
|
.IP \[bu] 2
|
|
Seafile (https://rclone.org/seafile/)
|
|
.IP \[bu] 2
|
|
SFTP (https://rclone.org/sftp/)
|
|
.IP \[bu] 2
|
|
SugarSync (https://rclone.org/sugarsync/)
|
|
.IP \[bu] 2
|
|
Tardigrade (https://rclone.org/tardigrade/)
|
|
.IP \[bu] 2
|
|
Union (https://rclone.org/union/)
|
|
.IP \[bu] 2
|
|
WebDAV (https://rclone.org/webdav/)
|
|
.IP \[bu] 2
|
|
Yandex Disk (https://rclone.org/yandex/)
|
|
.IP \[bu] 2
|
|
Zoho WorkDrive (https://rclone.org/zoho/)
|
|
.IP \[bu] 2
|
|
The local filesystem (https://rclone.org/local/)
|
|
.SS Usage
|
|
.PP
|
|
Rclone syncs a directory tree from one storage system to another.
|
|
.PP
|
|
Its syntax is like this
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
Syntax: [options] subcommand <parameters> <parameters...>
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
Source and destination paths are specified by the name you gave the
|
|
storage system in the config file then the sub path, e.g.
|
|
\[dq]drive:myfolder\[dq] to look at \[dq]myfolder\[dq] in Google drive.
|
|
.PP
|
|
You can define as many storage paths as you like in the config file.
|
|
.PP
|
|
Please use the \f[C]\-i\f[R] / \f[C]\-\-interactive\f[R] flag while
|
|
learning rclone to avoid accidental data loss.
|
|
.SS Subcommands
|
|
.PP
|
|
rclone uses a system of subcommands.
|
|
For example
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone ls remote:path # lists a remote
|
|
rclone copy /local/path remote:path # copies /local/path to the remote
|
|
rclone sync \-i /local/path remote:path # syncs /local/path to the remote
|
|
\f[R]
|
|
.fi
|
|
.SH rclone config
|
|
.PP
|
|
Enter an interactive configuration session.
|
|
.SS Synopsis
|
|
.PP
|
|
Enter an interactive configuration session where you can setup new
|
|
remotes and manage existing ones.
|
|
You may also set or remove a password to protect your configuration.
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone config [flags]
|
|
\f[R]
|
|
.fi
|
|
.SS Options
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
\-h, \-\-help help for config
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
See the global flags page (https://rclone.org/flags/) for global options
|
|
not listed here.
|
|
.SS SEE ALSO
|
|
.IP \[bu] 2
|
|
rclone (https://rclone.org/commands/rclone/) \- Show help for rclone
|
|
commands, flags and backends.
|
|
.IP \[bu] 2
|
|
rclone config create (https://rclone.org/commands/rclone_config_create/)
|
|
\- Create a new remote with name, type and options.
|
|
.IP \[bu] 2
|
|
rclone config delete (https://rclone.org/commands/rclone_config_delete/)
|
|
\- Delete an existing remote \f[C]name\f[R].
|
|
.IP \[bu] 2
|
|
rclone config
|
|
disconnect (https://rclone.org/commands/rclone_config_disconnect/) \-
|
|
Disconnects user from remote
|
|
.IP \[bu] 2
|
|
rclone config dump (https://rclone.org/commands/rclone_config_dump/) \-
|
|
Dump the config file as JSON.
|
|
.IP \[bu] 2
|
|
rclone config edit (https://rclone.org/commands/rclone_config_edit/) \-
|
|
Enter an interactive configuration session.
|
|
.IP \[bu] 2
|
|
rclone config file (https://rclone.org/commands/rclone_config_file/) \-
|
|
Show path of configuration file in use.
|
|
.IP \[bu] 2
|
|
rclone config
|
|
password (https://rclone.org/commands/rclone_config_password/) \- Update
|
|
password in an existing remote.
|
|
.IP \[bu] 2
|
|
rclone config
|
|
providers (https://rclone.org/commands/rclone_config_providers/) \- List
|
|
in JSON format all the providers and options.
|
|
.IP \[bu] 2
|
|
rclone config
|
|
reconnect (https://rclone.org/commands/rclone_config_reconnect/) \-
|
|
Re\-authenticates user with remote.
|
|
.IP \[bu] 2
|
|
rclone config show (https://rclone.org/commands/rclone_config_show/) \-
|
|
Print (decrypted) config file, or the config for a single remote.
|
|
.IP \[bu] 2
|
|
rclone config update (https://rclone.org/commands/rclone_config_update/)
|
|
\- Update options in an existing remote.
|
|
.IP \[bu] 2
|
|
rclone config
|
|
userinfo (https://rclone.org/commands/rclone_config_userinfo/) \- Prints
|
|
info about logged in user of remote.
|
|
.SH rclone copy
|
|
.PP
|
|
Copy files from source to dest, skipping already copied.
|
|
.SS Synopsis
|
|
.PP
|
|
Copy the source to the destination.
|
|
Doesn\[aq]t transfer unchanged files, testing by size and modification
|
|
time or MD5SUM.
|
|
Doesn\[aq]t delete files from the destination.
|
|
.PP
|
|
Note that it is always the contents of the directory that is synced, not
|
|
the directory so when source:path is a directory, it\[aq]s the contents
|
|
of source:path that are copied, not the directory name and contents.
|
|
.PP
|
|
If dest:path doesn\[aq]t exist, it is created and the source:path
|
|
contents go there.
|
|
.PP
|
|
For example
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone copy source:sourcepath dest:destpath
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
Let\[aq]s say there are two files in sourcepath
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
sourcepath/one.txt
|
|
sourcepath/two.txt
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
This copies them to
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
destpath/one.txt
|
|
destpath/two.txt
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
Not to
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
destpath/sourcepath/one.txt
|
|
destpath/sourcepath/two.txt
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
If you are familiar with \f[C]rsync\f[R], rclone always works as if you
|
|
had written a trailing \f[C]/\f[R] \- meaning \[dq]copy the contents of
|
|
this directory\[dq].
|
|
This applies to all commands and whether you are talking about the
|
|
source or destination.
|
|
.PP
|
|
See the \-\-no\-traverse (https://rclone.org/docs/#no-traverse) option
|
|
for controlling whether rclone lists the destination directory or not.
|
|
Supplying this option when copying a small number of files into a large
|
|
destination can speed transfers up greatly.
|
|
.PP
|
|
For example, if you have many files in /path/to/src but only a few of
|
|
them change every day, you can copy all the files which have changed
|
|
recently very efficiently like this:
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone copy \-\-max\-age 24h \-\-no\-traverse /path/to/src remote:
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
\f[B]Note\f[R]: Use the \f[C]\-P\f[R]/\f[C]\-\-progress\f[R] flag to
|
|
view real\-time transfer statistics.
|
|
.PP
|
|
\f[B]Note\f[R]: Use the \f[C]\-\-dry\-run\f[R] or the
|
|
\f[C]\-\-interactive\f[R]/\f[C]\-i\f[R] flag to test without copying
|
|
anything.
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone copy source:path dest:path [flags]
|
|
\f[R]
|
|
.fi
|
|
.SS Options
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
\-\-create\-empty\-src\-dirs Create empty source dirs on destination after copy
|
|
\-h, \-\-help help for copy
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
See the global flags page (https://rclone.org/flags/) for global options
|
|
not listed here.
|
|
.SS SEE ALSO
|
|
.IP \[bu] 2
|
|
rclone (https://rclone.org/commands/rclone/) \- Show help for rclone
|
|
commands, flags and backends.
|
|
.SH rclone sync
|
|
.PP
|
|
Make source and dest identical, modifying destination only.
|
|
.SS Synopsis
|
|
.PP
|
|
Sync the source to the destination, changing the destination only.
|
|
Doesn\[aq]t transfer unchanged files, testing by size and modification
|
|
time or MD5SUM.
|
|
Destination is updated to match source, including deleting files if
|
|
necessary (except duplicate objects, see below).
|
|
.PP
|
|
\f[B]Important\f[R]: Since this can cause data loss, test first with the
|
|
\f[C]\-\-dry\-run\f[R] or the \f[C]\-\-interactive\f[R]/\f[C]\-i\f[R]
|
|
flag.
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone sync \-i SOURCE remote:DESTINATION
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
Note that files in the destination won\[aq]t be deleted if there were
|
|
any errors at any point.
|
|
Duplicate objects (files with the same name, on those providers that
|
|
support it) are also not yet handled.
|
|
.PP
|
|
It is always the contents of the directory that is synced, not the
|
|
directory so when source:path is a directory, it\[aq]s the contents of
|
|
source:path that are copied, not the directory name and contents.
|
|
See extended explanation in the \f[C]copy\f[R] command above if unsure.
|
|
.PP
|
|
If dest:path doesn\[aq]t exist, it is created and the source:path
|
|
contents go there.
|
|
.PP
|
|
\f[B]Note\f[R]: Use the \f[C]\-P\f[R]/\f[C]\-\-progress\f[R] flag to
|
|
view real\-time transfer statistics
|
|
.PP
|
|
\f[B]Note\f[R]: Use the \f[C]rclone dedupe\f[R] command to deal with
|
|
\[dq]Duplicate object/directory found in source/destination \-
|
|
ignoring\[dq] errors.
|
|
See this forum
|
|
post (https://forum.rclone.org/t/sync-not-clearing-duplicates/14372) for
|
|
more info.
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone sync source:path dest:path [flags]
|
|
\f[R]
|
|
.fi
|
|
.SS Options
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
\-\-create\-empty\-src\-dirs Create empty source dirs on destination after sync
|
|
\-h, \-\-help help for sync
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
See the global flags page (https://rclone.org/flags/) for global options
|
|
not listed here.
|
|
.SS SEE ALSO
|
|
.IP \[bu] 2
|
|
rclone (https://rclone.org/commands/rclone/) \- Show help for rclone
|
|
commands, flags and backends.
|
|
.SH rclone move
|
|
.PP
|
|
Move files from source to dest.
|
|
.SS Synopsis
|
|
.PP
|
|
Moves the contents of the source directory to the destination directory.
|
|
Rclone will error if the source and destination overlap and the remote
|
|
does not support a server\-side directory move operation.
|
|
.PP
|
|
If no filters are in use and if possible this will server\-side move
|
|
\f[C]source:path\f[R] into \f[C]dest:path\f[R].
|
|
After this \f[C]source:path\f[R] will no longer exist.
|
|
.PP
|
|
Otherwise for each file in \f[C]source:path\f[R] selected by the filters
|
|
(if any) this will move it into \f[C]dest:path\f[R].
|
|
If possible a server\-side move will be used, otherwise it will copy it
|
|
(server\-side if possible) into \f[C]dest:path\f[R] then delete the
|
|
original (if no errors on copy) in \f[C]source:path\f[R].
|
|
.PP
|
|
If you want to delete empty source directories after move, use the
|
|
\-\-delete\-empty\-src\-dirs flag.
|
|
.PP
|
|
See the \-\-no\-traverse (https://rclone.org/docs/#no-traverse) option
|
|
for controlling whether rclone lists the destination directory or not.
|
|
Supplying this option when moving a small number of files into a large
|
|
destination can speed transfers up greatly.
|
|
.PP
|
|
\f[B]Important\f[R]: Since this can cause data loss, test first with the
|
|
\f[C]\-\-dry\-run\f[R] or the \f[C]\-\-interactive\f[R]/\f[C]\-i\f[R]
|
|
flag.
|
|
.PP
|
|
\f[B]Note\f[R]: Use the \f[C]\-P\f[R]/\f[C]\-\-progress\f[R] flag to
|
|
view real\-time transfer statistics.
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone move source:path dest:path [flags]
|
|
\f[R]
|
|
.fi
|
|
.SS Options
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
\-\-create\-empty\-src\-dirs Create empty source dirs on destination after move
|
|
\-\-delete\-empty\-src\-dirs Delete empty source dirs after move
|
|
\-h, \-\-help help for move
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
See the global flags page (https://rclone.org/flags/) for global options
|
|
not listed here.
|
|
.SS SEE ALSO
|
|
.IP \[bu] 2
|
|
rclone (https://rclone.org/commands/rclone/) \- Show help for rclone
|
|
commands, flags and backends.
|
|
.SH rclone delete
|
|
.PP
|
|
Remove the files in path.
|
|
.SS Synopsis
|
|
.PP
|
|
Remove the files in path.
|
|
Unlike \f[C]purge\f[R] it obeys include/exclude filters so can be used
|
|
to selectively delete files.
|
|
.PP
|
|
\f[C]rclone delete\f[R] only deletes files but leaves the directory
|
|
structure alone.
|
|
If you want to delete a directory and all of its contents use the
|
|
\f[C]purge\f[R] command.
|
|
.PP
|
|
If you supply the \f[C]\-\-rmdirs\f[R] flag, it will remove all empty
|
|
directories along with it.
|
|
You can also use the separate command \f[C]rmdir\f[R] or
|
|
\f[C]rmdirs\f[R] to delete empty directories only.
|
|
.PP
|
|
For example, to delete all files bigger than 100MBytes, you may first
|
|
want to check what would be deleted (use either):
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone \-\-min\-size 100M lsl remote:path
|
|
rclone \-\-dry\-run \-\-min\-size 100M delete remote:path
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
Then proceed with the actual delete:
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone \-\-min\-size 100M delete remote:path
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
That reads \[dq]delete everything with a minimum size of 100 MB\[dq],
|
|
hence delete all files bigger than 100MBytes.
|
|
.PP
|
|
\f[B]Important\f[R]: Since this can cause data loss, test first with the
|
|
\f[C]\-\-dry\-run\f[R] or the \f[C]\-\-interactive\f[R]/\f[C]\-i\f[R]
|
|
flag.
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone delete remote:path [flags]
|
|
\f[R]
|
|
.fi
|
|
.SS Options
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
\-h, \-\-help help for delete
|
|
\-\-rmdirs rmdirs removes empty directories but leaves root intact
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
See the global flags page (https://rclone.org/flags/) for global options
|
|
not listed here.
|
|
.SS SEE ALSO
|
|
.IP \[bu] 2
|
|
rclone (https://rclone.org/commands/rclone/) \- Show help for rclone
|
|
commands, flags and backends.
|
|
.SH rclone purge
|
|
.PP
|
|
Remove the path and all of its contents.
|
|
.SS Synopsis
|
|
.PP
|
|
Remove the path and all of its contents.
|
|
Note that this does not obey include/exclude filters \- everything will
|
|
be removed.
|
|
Use the \f[C]delete\f[R] command if you want to selectively delete
|
|
files.
|
|
To delete empty directories only, use command \f[C]rmdir\f[R] or
|
|
\f[C]rmdirs\f[R].
|
|
.PP
|
|
\f[B]Important\f[R]: Since this can cause data loss, test first with the
|
|
\f[C]\-\-dry\-run\f[R] or the \f[C]\-\-interactive\f[R]/\f[C]\-i\f[R]
|
|
flag.
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone purge remote:path [flags]
|
|
\f[R]
|
|
.fi
|
|
.SS Options
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
\-h, \-\-help help for purge
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
See the global flags page (https://rclone.org/flags/) for global options
|
|
not listed here.
|
|
.SS SEE ALSO
|
|
.IP \[bu] 2
|
|
rclone (https://rclone.org/commands/rclone/) \- Show help for rclone
|
|
commands, flags and backends.
|
|
.SH rclone mkdir
|
|
.PP
|
|
Make the path if it doesn\[aq]t already exist.
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone mkdir remote:path [flags]
|
|
\f[R]
|
|
.fi
|
|
.SS Options
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
\-h, \-\-help help for mkdir
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
See the global flags page (https://rclone.org/flags/) for global options
|
|
not listed here.
|
|
.SS SEE ALSO
|
|
.IP \[bu] 2
|
|
rclone (https://rclone.org/commands/rclone/) \- Show help for rclone
|
|
commands, flags and backends.
|
|
.SH rclone rmdir
|
|
.PP
|
|
Remove the empty directory at path.
|
|
.SS Synopsis
|
|
.PP
|
|
This removes empty directory given by path.
|
|
Will not remove the path if it has any objects in it, not even empty
|
|
subdirectories.
|
|
Use command \f[C]rmdirs\f[R] (or \f[C]delete\f[R] with option
|
|
\f[C]\-\-rmdirs\f[R]) to do that.
|
|
.PP
|
|
To delete a path and any objects in it, use \f[C]purge\f[R] command.
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone rmdir remote:path [flags]
|
|
\f[R]
|
|
.fi
|
|
.SS Options
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
\-h, \-\-help help for rmdir
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
See the global flags page (https://rclone.org/flags/) for global options
|
|
not listed here.
|
|
.SS SEE ALSO
|
|
.IP \[bu] 2
|
|
rclone (https://rclone.org/commands/rclone/) \- Show help for rclone
|
|
commands, flags and backends.
|
|
.SH rclone check
|
|
.PP
|
|
Checks the files in the source and destination match.
|
|
.SS Synopsis
|
|
.PP
|
|
Checks the files in the source and destination match.
|
|
It compares sizes and hashes (MD5 or SHA1) and logs a report of files
|
|
which don\[aq]t match.
|
|
It doesn\[aq]t alter the source or destination.
|
|
.PP
|
|
If you supply the \f[C]\-\-size\-only\f[R] flag, it will only compare
|
|
the sizes not the hashes as well.
|
|
Use this for a quick check.
|
|
.PP
|
|
If you supply the \f[C]\-\-download\f[R] flag, it will download the data
|
|
from both remotes and check them against each other on the fly.
|
|
This can be useful for remotes that don\[aq]t support hashes or if you
|
|
really want to check all the data.
|
|
.PP
|
|
If you supply the \f[C]\-\-one\-way\f[R] flag, it will only check that
|
|
files in the source match the files in the destination, not the other
|
|
way around.
|
|
This means that extra files in the destination that are not in the
|
|
source will not be detected.
|
|
.PP
|
|
The \f[C]\-\-differ\f[R], \f[C]\-\-missing\-on\-dst\f[R],
|
|
\f[C]\-\-missing\-on\-src\f[R], \f[C]\-\-match\f[R] and
|
|
\f[C]\-\-error\f[R] flags write paths, one per line, to the file name
|
|
(or stdout if it is \f[C]\-\f[R]) supplied.
|
|
What they write is described in the help below.
|
|
For example \f[C]\-\-differ\f[R] will write all paths which are present
|
|
on both the source and destination but different.
|
|
.PP
|
|
The \f[C]\-\-combined\f[R] flag will write a file (or stdout) which
|
|
contains all file paths with a symbol and then a space and then the path
|
|
to tell you what happened to it.
|
|
These are reminiscent of diff files.
|
|
.IP \[bu] 2
|
|
\f[C]= path\f[R] means path was found in source and destination and was
|
|
identical
|
|
.IP \[bu] 2
|
|
\f[C]\- path\f[R] means path was missing on the source, so only in the
|
|
destination
|
|
.IP \[bu] 2
|
|
\f[C]+ path\f[R] means path was missing on the destination, so only in
|
|
the source
|
|
.IP \[bu] 2
|
|
\f[C]* path\f[R] means path was present in source and destination but
|
|
different.
|
|
.IP \[bu] 2
|
|
\f[C]! path\f[R] means there was an error reading or hashing the source
|
|
or dest.
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone check source:path dest:path [flags]
|
|
\f[R]
|
|
.fi
|
|
.SS Options
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
\-\-combined string Make a combined report of changes to this file
|
|
\-\-differ string Report all non\-matching files to this file
|
|
\-\-download Check by downloading rather than with hash.
|
|
\-\-error string Report all files with errors (hashing or reading) to this file
|
|
\-h, \-\-help help for check
|
|
\-\-match string Report all matching files to this file
|
|
\-\-missing\-on\-dst string Report all files missing from the destination to this file
|
|
\-\-missing\-on\-src string Report all files missing from the source to this file
|
|
\-\-one\-way Check one way only, source files must exist on remote
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
See the global flags page (https://rclone.org/flags/) for global options
|
|
not listed here.
|
|
.SS SEE ALSO
|
|
.IP \[bu] 2
|
|
rclone (https://rclone.org/commands/rclone/) \- Show help for rclone
|
|
commands, flags and backends.
|
|
.SH rclone ls
|
|
.PP
|
|
List the objects in the path with size and path.
|
|
.SS Synopsis
|
|
.PP
|
|
Lists the objects in the source path to standard output in a human
|
|
readable format with size and path.
|
|
Recurses by default.
|
|
.PP
|
|
Eg
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
$ rclone ls swift:bucket
|
|
60295 bevajer5jef
|
|
90613 canole
|
|
94467 diwogej7
|
|
37600 fubuwic
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
Any of the filtering options can be applied to this command.
|
|
.PP
|
|
There are several related list commands
|
|
.IP \[bu] 2
|
|
\f[C]ls\f[R] to list size and path of objects only
|
|
.IP \[bu] 2
|
|
\f[C]lsl\f[R] to list modification time, size and path of objects only
|
|
.IP \[bu] 2
|
|
\f[C]lsd\f[R] to list directories only
|
|
.IP \[bu] 2
|
|
\f[C]lsf\f[R] to list objects and directories in easy to parse format
|
|
.IP \[bu] 2
|
|
\f[C]lsjson\f[R] to list objects and directories in JSON format
|
|
.PP
|
|
\f[C]ls\f[R],\f[C]lsl\f[R],\f[C]lsd\f[R] are designed to be human
|
|
readable.
|
|
\f[C]lsf\f[R] is designed to be human and machine readable.
|
|
\f[C]lsjson\f[R] is designed to be machine readable.
|
|
.PP
|
|
Note that \f[C]ls\f[R] and \f[C]lsl\f[R] recurse by default \- use
|
|
\f[C]\-\-max\-depth 1\f[R] to stop the recursion.
|
|
.PP
|
|
The other list commands \f[C]lsd\f[R],\f[C]lsf\f[R],\f[C]lsjson\f[R] do
|
|
not recurse by default \- use \f[C]\-R\f[R] to make them recurse.
|
|
.PP
|
|
Listing a non existent directory will produce an error except for
|
|
remotes which can\[aq]t have empty directories (e.g.
|
|
s3, swift, or gcs \- the bucket based remotes).
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone ls remote:path [flags]
|
|
\f[R]
|
|
.fi
|
|
.SS Options
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
\-h, \-\-help help for ls
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
See the global flags page (https://rclone.org/flags/) for global options
|
|
not listed here.
|
|
.SS SEE ALSO
|
|
.IP \[bu] 2
|
|
rclone (https://rclone.org/commands/rclone/) \- Show help for rclone
|
|
commands, flags and backends.
|
|
.SH rclone lsd
|
|
.PP
|
|
List all directories/containers/buckets in the path.
|
|
.SS Synopsis
|
|
.PP
|
|
Lists the directories in the source path to standard output.
|
|
Does not recurse by default.
|
|
Use the \-R flag to recurse.
|
|
.PP
|
|
This command lists the total size of the directory (if known, \-1 if
|
|
not), the modification time (if known, the current time if not), the
|
|
number of objects in the directory (if known, \-1 if not) and the name
|
|
of the directory, Eg
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
$ rclone lsd swift:
|
|
494000 2018\-04\-26 08:43:20 10000 10000files
|
|
65 2018\-04\-26 08:43:20 1 1File
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
Or
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
$ rclone lsd drive:test
|
|
\-1 2016\-10\-17 17:41:53 \-1 1000files
|
|
\-1 2017\-01\-03 14:40:54 \-1 2500files
|
|
\-1 2017\-07\-08 14:39:28 \-1 4000files
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
If you just want the directory names use \[dq]rclone lsf
|
|
\-\-dirs\-only\[dq].
|
|
.PP
|
|
Any of the filtering options can be applied to this command.
|
|
.PP
|
|
There are several related list commands
|
|
.IP \[bu] 2
|
|
\f[C]ls\f[R] to list size and path of objects only
|
|
.IP \[bu] 2
|
|
\f[C]lsl\f[R] to list modification time, size and path of objects only
|
|
.IP \[bu] 2
|
|
\f[C]lsd\f[R] to list directories only
|
|
.IP \[bu] 2
|
|
\f[C]lsf\f[R] to list objects and directories in easy to parse format
|
|
.IP \[bu] 2
|
|
\f[C]lsjson\f[R] to list objects and directories in JSON format
|
|
.PP
|
|
\f[C]ls\f[R],\f[C]lsl\f[R],\f[C]lsd\f[R] are designed to be human
|
|
readable.
|
|
\f[C]lsf\f[R] is designed to be human and machine readable.
|
|
\f[C]lsjson\f[R] is designed to be machine readable.
|
|
.PP
|
|
Note that \f[C]ls\f[R] and \f[C]lsl\f[R] recurse by default \- use
|
|
\f[C]\-\-max\-depth 1\f[R] to stop the recursion.
|
|
.PP
|
|
The other list commands \f[C]lsd\f[R],\f[C]lsf\f[R],\f[C]lsjson\f[R] do
|
|
not recurse by default \- use \f[C]\-R\f[R] to make them recurse.
|
|
.PP
|
|
Listing a non existent directory will produce an error except for
|
|
remotes which can\[aq]t have empty directories (e.g.
|
|
s3, swift, or gcs \- the bucket based remotes).
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone lsd remote:path [flags]
|
|
\f[R]
|
|
.fi
|
|
.SS Options
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
\-h, \-\-help help for lsd
|
|
\-R, \-\-recursive Recurse into the listing.
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
See the global flags page (https://rclone.org/flags/) for global options
|
|
not listed here.
|
|
.SS SEE ALSO
|
|
.IP \[bu] 2
|
|
rclone (https://rclone.org/commands/rclone/) \- Show help for rclone
|
|
commands, flags and backends.
|
|
.SH rclone lsl
|
|
.PP
|
|
List the objects in path with modification time, size and path.
|
|
.SS Synopsis
|
|
.PP
|
|
Lists the objects in the source path to standard output in a human
|
|
readable format with modification time, size and path.
|
|
Recurses by default.
|
|
.PP
|
|
Eg
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
$ rclone lsl swift:bucket
|
|
60295 2016\-06\-25 18:55:41.062626927 bevajer5jef
|
|
90613 2016\-06\-25 18:55:43.302607074 canole
|
|
94467 2016\-06\-25 18:55:43.046609333 diwogej7
|
|
37600 2016\-06\-25 18:55:40.814629136 fubuwic
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
Any of the filtering options can be applied to this command.
|
|
.PP
|
|
There are several related list commands
|
|
.IP \[bu] 2
|
|
\f[C]ls\f[R] to list size and path of objects only
|
|
.IP \[bu] 2
|
|
\f[C]lsl\f[R] to list modification time, size and path of objects only
|
|
.IP \[bu] 2
|
|
\f[C]lsd\f[R] to list directories only
|
|
.IP \[bu] 2
|
|
\f[C]lsf\f[R] to list objects and directories in easy to parse format
|
|
.IP \[bu] 2
|
|
\f[C]lsjson\f[R] to list objects and directories in JSON format
|
|
.PP
|
|
\f[C]ls\f[R],\f[C]lsl\f[R],\f[C]lsd\f[R] are designed to be human
|
|
readable.
|
|
\f[C]lsf\f[R] is designed to be human and machine readable.
|
|
\f[C]lsjson\f[R] is designed to be machine readable.
|
|
.PP
|
|
Note that \f[C]ls\f[R] and \f[C]lsl\f[R] recurse by default \- use
|
|
\f[C]\-\-max\-depth 1\f[R] to stop the recursion.
|
|
.PP
|
|
The other list commands \f[C]lsd\f[R],\f[C]lsf\f[R],\f[C]lsjson\f[R] do
|
|
not recurse by default \- use \f[C]\-R\f[R] to make them recurse.
|
|
.PP
|
|
Listing a non existent directory will produce an error except for
|
|
remotes which can\[aq]t have empty directories (e.g.
|
|
s3, swift, or gcs \- the bucket based remotes).
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone lsl remote:path [flags]
|
|
\f[R]
|
|
.fi
|
|
.SS Options
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
\-h, \-\-help help for lsl
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
See the global flags page (https://rclone.org/flags/) for global options
|
|
not listed here.
|
|
.SS SEE ALSO
|
|
.IP \[bu] 2
|
|
rclone (https://rclone.org/commands/rclone/) \- Show help for rclone
|
|
commands, flags and backends.
|
|
.SH rclone md5sum
|
|
.PP
|
|
Produces an md5sum file for all the objects in the path.
|
|
.SS Synopsis
|
|
.PP
|
|
Produces an md5sum file for all the objects in the path.
|
|
This is in the same format as the standard md5sum tool produces.
|
|
.PP
|
|
By default, the hash is requested from the remote.
|
|
If MD5 is not supported by the remote, no hash will be returned.
|
|
With the download flag, the file will be downloaded from the remote and
|
|
hashed locally enabling MD5 for any remote.
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone md5sum remote:path [flags]
|
|
\f[R]
|
|
.fi
|
|
.SS Options
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
\-\-base64 Output base64 encoded hashsum
|
|
\-\-download Download the file and hash it locally; if this flag is not specified, the hash is requested from the remote
|
|
\-h, \-\-help help for md5sum
|
|
\-\-output\-file string Output hashsums to a file rather than the terminal
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
See the global flags page (https://rclone.org/flags/) for global options
|
|
not listed here.
|
|
.SS SEE ALSO
|
|
.IP \[bu] 2
|
|
rclone (https://rclone.org/commands/rclone/) \- Show help for rclone
|
|
commands, flags and backends.
|
|
.SH rclone sha1sum
|
|
.PP
|
|
Produces an sha1sum file for all the objects in the path.
|
|
.SS Synopsis
|
|
.PP
|
|
Produces an sha1sum file for all the objects in the path.
|
|
This is in the same format as the standard sha1sum tool produces.
|
|
.PP
|
|
By default, the hash is requested from the remote.
|
|
If SHA\-1 is not supported by the remote, no hash will be returned.
|
|
With the download flag, the file will be downloaded from the remote and
|
|
hashed locally enabling SHA\-1 for any remote.
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone sha1sum remote:path [flags]
|
|
\f[R]
|
|
.fi
|
|
.SS Options
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
\-\-base64 Output base64 encoded hashsum
|
|
\-\-download Download the file and hash it locally; if this flag is not specified, the hash is requested from the remote
|
|
\-h, \-\-help help for sha1sum
|
|
\-\-output\-file string Output hashsums to a file rather than the terminal
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
See the global flags page (https://rclone.org/flags/) for global options
|
|
not listed here.
|
|
.SS SEE ALSO
|
|
.IP \[bu] 2
|
|
rclone (https://rclone.org/commands/rclone/) \- Show help for rclone
|
|
commands, flags and backends.
|
|
.SH rclone size
|
|
.PP
|
|
Prints the total size and number of objects in remote:path.
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone size remote:path [flags]
|
|
\f[R]
|
|
.fi
|
|
.SS Options
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
\-h, \-\-help help for size
|
|
\-\-json format output as JSON
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
See the global flags page (https://rclone.org/flags/) for global options
|
|
not listed here.
|
|
.SS SEE ALSO
|
|
.IP \[bu] 2
|
|
rclone (https://rclone.org/commands/rclone/) \- Show help for rclone
|
|
commands, flags and backends.
|
|
.SH rclone version
|
|
.PP
|
|
Show the version number.
|
|
.SS Synopsis
|
|
.PP
|
|
Show the rclone version number, the go version, the build target OS and
|
|
architecture, build tags and the type of executable (static or dynamic).
|
|
.PP
|
|
For example:
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
$ rclone version
|
|
rclone v1.54
|
|
\- os/type: linux
|
|
\- os/arch: amd64
|
|
\- go/version: go1.16
|
|
\- go/linking: static
|
|
\- go/tags: none
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
Note: before rclone version 1.55 the os/type and os/arch lines were
|
|
merged, and the \[dq]go/version\[dq] line was tagged as \[dq]go
|
|
version\[dq].
|
|
.PP
|
|
If you supply the \-\-check flag, then it will do an online check to
|
|
compare your version with the latest release and the latest beta.
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
$ rclone version \-\-check
|
|
yours: 1.42.0.6
|
|
latest: 1.42 (released 2018\-06\-16)
|
|
beta: 1.42.0.5 (released 2018\-06\-17)
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
Or
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
$ rclone version \-\-check
|
|
yours: 1.41
|
|
latest: 1.42 (released 2018\-06\-16)
|
|
upgrade: https://downloads.rclone.org/v1.42
|
|
beta: 1.42.0.5 (released 2018\-06\-17)
|
|
upgrade: https://beta.rclone.org/v1.42\-005\-g56e1e820
|
|
\f[R]
|
|
.fi
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone version [flags]
|
|
\f[R]
|
|
.fi
|
|
.SS Options
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
\-\-check Check for new version.
|
|
\-h, \-\-help help for version
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
See the global flags page (https://rclone.org/flags/) for global options
|
|
not listed here.
|
|
.SS SEE ALSO
|
|
.IP \[bu] 2
|
|
rclone (https://rclone.org/commands/rclone/) \- Show help for rclone
|
|
commands, flags and backends.
|
|
.SH rclone cleanup
|
|
.PP
|
|
Clean up the remote if possible.
|
|
.SS Synopsis
|
|
.PP
|
|
Clean up the remote if possible.
|
|
Empty the trash or delete old file versions.
|
|
Not supported by all remotes.
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone cleanup remote:path [flags]
|
|
\f[R]
|
|
.fi
|
|
.SS Options
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
\-h, \-\-help help for cleanup
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
See the global flags page (https://rclone.org/flags/) for global options
|
|
not listed here.
|
|
.SS SEE ALSO
|
|
.IP \[bu] 2
|
|
rclone (https://rclone.org/commands/rclone/) \- Show help for rclone
|
|
commands, flags and backends.
|
|
.SH rclone dedupe
|
|
.PP
|
|
Interactively find duplicate filenames and delete/rename them.
|
|
.SS Synopsis
|
|
.PP
|
|
By default \f[C]dedupe\f[R] interactively finds files with duplicate
|
|
names and offers to delete all but one or rename them to be different.
|
|
This is known as deduping by name.
|
|
.PP
|
|
Deduping by name is only useful with a small group of backends (e.g.
|
|
Google Drive, Opendrive) that can have duplicate file names.
|
|
It can be run on wrapping backends (e.g.
|
|
crypt) if they wrap a backend which supports duplicate file names.
|
|
.PP
|
|
However if \-\-by\-hash is passed in then dedupe will find files with
|
|
duplicate hashes instead which will work on any backend which supports
|
|
at least one hash.
|
|
This can be used to find files with duplicate content.
|
|
This is known as deduping by hash.
|
|
.PP
|
|
If deduping by name, first rclone will merge directories with the same
|
|
name.
|
|
It will do this iteratively until all the identically named directories
|
|
have been merged.
|
|
.PP
|
|
Next, if deduping by name, for every group of duplicate file names /
|
|
hashes, it will delete all but one identical files it finds without
|
|
confirmation.
|
|
This means that for most duplicated files the \f[C]dedupe\f[R] command
|
|
will not be interactive.
|
|
.PP
|
|
\f[C]dedupe\f[R] considers files to be identical if they have the same
|
|
file path and the same hash.
|
|
If the backend does not support hashes (e.g.
|
|
crypt wrapping Google Drive) then they will never be found to be
|
|
identical.
|
|
If you use the \f[C]\-\-size\-only\f[R] flag then files will be
|
|
considered identical if they have the same size (any hash will be
|
|
ignored).
|
|
This can be useful on crypt backends which do not support hashes.
|
|
.PP
|
|
Next rclone will resolve the remaining duplicates.
|
|
Exactly which action is taken depends on the dedupe mode.
|
|
By default rclone will interactively query the user for each one.
|
|
.PP
|
|
\f[B]Important\f[R]: Since this can cause data loss, test first with the
|
|
\f[C]\-\-dry\-run\f[R] or the \f[C]\-\-interactive\f[R]/\f[C]\-i\f[R]
|
|
flag.
|
|
.PP
|
|
Here is an example run.
|
|
.PP
|
|
Before \- with duplicates
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
$ rclone lsl drive:dupes
|
|
6048320 2016\-03\-05 16:23:16.798000000 one.txt
|
|
6048320 2016\-03\-05 16:23:11.775000000 one.txt
|
|
564374 2016\-03\-05 16:23:06.731000000 one.txt
|
|
6048320 2016\-03\-05 16:18:26.092000000 one.txt
|
|
6048320 2016\-03\-05 16:22:46.185000000 two.txt
|
|
1744073 2016\-03\-05 16:22:38.104000000 two.txt
|
|
564374 2016\-03\-05 16:22:52.118000000 two.txt
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
Now the \f[C]dedupe\f[R] session
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
$ rclone dedupe drive:dupes
|
|
2016/03/05 16:24:37 Google drive root \[aq]dupes\[aq]: Looking for duplicates using interactive mode.
|
|
one.txt: Found 4 files with duplicate names
|
|
one.txt: Deleting 2/3 identical duplicates (MD5 \[dq]1eedaa9fe86fd4b8632e2ac549403b36\[dq])
|
|
one.txt: 2 duplicates remain
|
|
1: 6048320 bytes, 2016\-03\-05 16:23:16.798000000, MD5 1eedaa9fe86fd4b8632e2ac549403b36
|
|
2: 564374 bytes, 2016\-03\-05 16:23:06.731000000, MD5 7594e7dc9fc28f727c42ee3e0749de81
|
|
s) Skip and do nothing
|
|
k) Keep just one (choose which in next step)
|
|
r) Rename all to be different (by changing file.jpg to file\-1.jpg)
|
|
s/k/r> k
|
|
Enter the number of the file to keep> 1
|
|
one.txt: Deleted 1 extra copies
|
|
two.txt: Found 3 files with duplicate names
|
|
two.txt: 3 duplicates remain
|
|
1: 564374 bytes, 2016\-03\-05 16:22:52.118000000, MD5 7594e7dc9fc28f727c42ee3e0749de81
|
|
2: 6048320 bytes, 2016\-03\-05 16:22:46.185000000, MD5 1eedaa9fe86fd4b8632e2ac549403b36
|
|
3: 1744073 bytes, 2016\-03\-05 16:22:38.104000000, MD5 851957f7fb6f0bc4ce76be966d336802
|
|
s) Skip and do nothing
|
|
k) Keep just one (choose which in next step)
|
|
r) Rename all to be different (by changing file.jpg to file\-1.jpg)
|
|
s/k/r> r
|
|
two\-1.txt: renamed from: two.txt
|
|
two\-2.txt: renamed from: two.txt
|
|
two\-3.txt: renamed from: two.txt
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
The result being
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
$ rclone lsl drive:dupes
|
|
6048320 2016\-03\-05 16:23:16.798000000 one.txt
|
|
564374 2016\-03\-05 16:22:52.118000000 two\-1.txt
|
|
6048320 2016\-03\-05 16:22:46.185000000 two\-2.txt
|
|
1744073 2016\-03\-05 16:22:38.104000000 two\-3.txt
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
Dedupe can be run non interactively using the \f[C]\-\-dedupe\-mode\f[R]
|
|
flag or by using an extra parameter with the same value
|
|
.IP \[bu] 2
|
|
\f[C]\-\-dedupe\-mode interactive\f[R] \- interactive as above.
|
|
.IP \[bu] 2
|
|
\f[C]\-\-dedupe\-mode skip\f[R] \- removes identical files then skips
|
|
anything left.
|
|
.IP \[bu] 2
|
|
\f[C]\-\-dedupe\-mode first\f[R] \- removes identical files then keeps
|
|
the first one.
|
|
.IP \[bu] 2
|
|
\f[C]\-\-dedupe\-mode newest\f[R] \- removes identical files then keeps
|
|
the newest one.
|
|
.IP \[bu] 2
|
|
\f[C]\-\-dedupe\-mode oldest\f[R] \- removes identical files then keeps
|
|
the oldest one.
|
|
.IP \[bu] 2
|
|
\f[C]\-\-dedupe\-mode largest\f[R] \- removes identical files then keeps
|
|
the largest one.
|
|
.IP \[bu] 2
|
|
\f[C]\-\-dedupe\-mode smallest\f[R] \- removes identical files then
|
|
keeps the smallest one.
|
|
.IP \[bu] 2
|
|
\f[C]\-\-dedupe\-mode rename\f[R] \- removes identical files then
|
|
renames the rest to be different.
|
|
.IP \[bu] 2
|
|
\f[C]\-\-dedupe\-mode list\f[R] \- lists duplicate dirs and files only
|
|
and changes nothing.
|
|
.PP
|
|
For example to rename all the identically named photos in your Google
|
|
Photos directory, do
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone dedupe \-\-dedupe\-mode rename \[dq]drive:Google Photos\[dq]
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
Or
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone dedupe rename \[dq]drive:Google Photos\[dq]
|
|
\f[R]
|
|
.fi
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone dedupe [mode] remote:path [flags]
|
|
\f[R]
|
|
.fi
|
|
.SS Options
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
\-\-by\-hash Find indentical hashes rather than names
|
|
\-\-dedupe\-mode string Dedupe mode interactive|skip|first|newest|oldest|largest|smallest|rename. (default \[dq]interactive\[dq])
|
|
\-h, \-\-help help for dedupe
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
See the global flags page (https://rclone.org/flags/) for global options
|
|
not listed here.
|
|
.SS SEE ALSO
|
|
.IP \[bu] 2
|
|
rclone (https://rclone.org/commands/rclone/) \- Show help for rclone
|
|
commands, flags and backends.
|
|
.SH rclone about
|
|
.PP
|
|
Get quota information from the remote.
|
|
.SS Synopsis
|
|
.PP
|
|
\f[C]rclone about\f[R]prints quota information about a remote to
|
|
standard output.
|
|
The output is typically used, free, quota and trash contents.
|
|
.PP
|
|
E.g.
|
|
Typical output from\f[C]rclone about remote:\f[R]is:
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
Total: 17G
|
|
Used: 7.444G
|
|
Free: 1.315G
|
|
Trashed: 100.000M
|
|
Other: 8.241G
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
Where the fields are:
|
|
.IP \[bu] 2
|
|
Total: total size available.
|
|
.IP \[bu] 2
|
|
Used: total size used
|
|
.IP \[bu] 2
|
|
Free: total space available to this user.
|
|
.IP \[bu] 2
|
|
Trashed: total space used by trash
|
|
.IP \[bu] 2
|
|
Other: total amount in other storage (e.g.
|
|
Gmail, Google Photos)
|
|
.IP \[bu] 2
|
|
Objects: total number of objects in the storage
|
|
.PP
|
|
Not all backends print all fields.
|
|
Information is not included if it is not provided by a backend.
|
|
Where the value is unlimited it is omitted.
|
|
.PP
|
|
Applying a \f[C]\-\-full\f[R] flag to the command prints the bytes in
|
|
full, e.g.
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
Total: 18253611008
|
|
Used: 7993453766
|
|
Free: 1411001220
|
|
Trashed: 104857602
|
|
Other: 8849156022
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
A \f[C]\-\-json\f[R]flag generates conveniently computer readable
|
|
output, e.g.
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
{
|
|
\[dq]total\[dq]: 18253611008,
|
|
\[dq]used\[dq]: 7993453766,
|
|
\[dq]trashed\[dq]: 104857602,
|
|
\[dq]other\[dq]: 8849156022,
|
|
\[dq]free\[dq]: 1411001220
|
|
}
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
Not all backends support the \f[C]rclone about\f[R] command.
|
|
.PP
|
|
See List of backends that do not support
|
|
about (https://rclone.org/overview/#optional-features)
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone about remote: [flags]
|
|
\f[R]
|
|
.fi
|
|
.SS Options
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
\-\-full Full numbers instead of SI units
|
|
\-h, \-\-help help for about
|
|
\-\-json Format output as JSON
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
See the global flags page (https://rclone.org/flags/) for global options
|
|
not listed here.
|
|
.SS SEE ALSO
|
|
.IP \[bu] 2
|
|
rclone (https://rclone.org/commands/rclone/) \- Show help for rclone
|
|
commands, flags and backends.
|
|
.SH rclone authorize
|
|
.PP
|
|
Remote authorization.
|
|
.SS Synopsis
|
|
.PP
|
|
Remote authorization.
|
|
Used to authorize a remote or headless rclone from a machine with a
|
|
browser \- use as instructed by rclone config.
|
|
.PP
|
|
Use the \-\-auth\-no\-open\-browser to prevent rclone to open auth link
|
|
in default browser automatically.
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone authorize [flags]
|
|
\f[R]
|
|
.fi
|
|
.SS Options
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
\-\-auth\-no\-open\-browser Do not automatically open auth link in default browser
|
|
\-h, \-\-help help for authorize
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
See the global flags page (https://rclone.org/flags/) for global options
|
|
not listed here.
|
|
.SS SEE ALSO
|
|
.IP \[bu] 2
|
|
rclone (https://rclone.org/commands/rclone/) \- Show help for rclone
|
|
commands, flags and backends.
|
|
.SH rclone backend
|
|
.PP
|
|
Run a backend specific command.
|
|
.SS Synopsis
|
|
.PP
|
|
This runs a backend specific command.
|
|
The commands themselves (except for \[dq]help\[dq] and
|
|
\[dq]features\[dq]) are defined by the backends and you should see the
|
|
backend docs for definitions.
|
|
.PP
|
|
You can discover what commands a backend implements by using
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone backend help remote:
|
|
rclone backend help <backendname>
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
You can also discover information about the backend using (see
|
|
operations/fsinfo (https://rclone.org/rc/#operations/fsinfo) in the
|
|
remote control docs for more info).
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone backend features remote:
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
Pass options to the backend command with \-o.
|
|
This should be key=value or key, e.g.:
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone backend stats remote:path stats \-o format=json \-o long
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
Pass arguments to the backend by placing them on the end of the line
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone backend cleanup remote:path file1 file2 file3
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
Note to run these commands on a running backend then see
|
|
backend/command (https://rclone.org/rc/#backend/command) in the rc docs.
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone backend <command> remote:path [opts] <args> [flags]
|
|
\f[R]
|
|
.fi
|
|
.SS Options
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
\-h, \-\-help help for backend
|
|
\-\-json Always output in JSON format.
|
|
\-o, \-\-option stringArray Option in the form name=value or name.
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
See the global flags page (https://rclone.org/flags/) for global options
|
|
not listed here.
|
|
.SS SEE ALSO
|
|
.IP \[bu] 2
|
|
rclone (https://rclone.org/commands/rclone/) \- Show help for rclone
|
|
commands, flags and backends.
|
|
.SH rclone cat
|
|
.PP
|
|
Concatenates any files and sends them to stdout.
|
|
.SS Synopsis
|
|
.PP
|
|
rclone cat sends any files to standard output.
|
|
.PP
|
|
You can use it like this to output a single file
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone cat remote:path/to/file
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
Or like this to output any file in dir or its subdirectories.
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone cat remote:path/to/dir
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
Or like this to output any .txt files in dir or its subdirectories.
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone \-\-include \[dq]*.txt\[dq] cat remote:path/to/dir
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
Use the \f[C]\-\-head\f[R] flag to print characters only at the start,
|
|
\f[C]\-\-tail\f[R] for the end and \f[C]\-\-offset\f[R] and
|
|
\f[C]\-\-count\f[R] to print a section in the middle.
|
|
Note that if offset is negative it will count from the end, so
|
|
\f[C]\-\-offset \-1 \-\-count 1\f[R] is equivalent to
|
|
\f[C]\-\-tail 1\f[R].
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone cat remote:path [flags]
|
|
\f[R]
|
|
.fi
|
|
.SS Options
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
\-\-count int Only print N characters. (default \-1)
|
|
\-\-discard Discard the output instead of printing.
|
|
\-\-head int Only print the first N characters.
|
|
\-h, \-\-help help for cat
|
|
\-\-offset int Start printing at offset N (or from end if \-ve).
|
|
\-\-tail int Only print the last N characters.
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
See the global flags page (https://rclone.org/flags/) for global options
|
|
not listed here.
|
|
.SS SEE ALSO
|
|
.IP \[bu] 2
|
|
rclone (https://rclone.org/commands/rclone/) \- Show help for rclone
|
|
commands, flags and backends.
|
|
.SH rclone config create
|
|
.PP
|
|
Create a new remote with name, type and options.
|
|
.SS Synopsis
|
|
.PP
|
|
Create a new remote of \f[C]name\f[R] with \f[C]type\f[R] and options.
|
|
The options should be passed in pairs of \f[C]key\f[R] \f[C]value\f[R].
|
|
.PP
|
|
For example to make a swift remote of name myremote using auto config
|
|
you would do:
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone config create myremote swift env_auth true
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
Note that if the config process would normally ask a question the
|
|
default is taken.
|
|
Each time that happens rclone will print a message saying how to affect
|
|
the value taken.
|
|
.PP
|
|
If any of the parameters passed is a password field, then rclone will
|
|
automatically obscure them if they aren\[aq]t already obscured before
|
|
putting them in the config file.
|
|
.PP
|
|
\f[B]NB\f[R] If the password parameter is 22 characters or longer and
|
|
consists only of base64 characters then rclone can get confused about
|
|
whether the password is already obscured or not and put unobscured
|
|
passwords into the config file.
|
|
If you want to be 100% certain that the passwords get obscured then use
|
|
the \[dq]\-\-obscure\[dq] flag, or if you are 100% certain you are
|
|
already passing obscured passwords then use \[dq]\-\-no\-obscure\[dq].
|
|
You can also set obscured passwords using the \[dq]rclone config
|
|
password\[dq] command.
|
|
.PP
|
|
So for example if you wanted to configure a Google Drive remote but
|
|
using remote authorization you would do this:
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone config create mydrive drive config_is_local false
|
|
\f[R]
|
|
.fi
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone config create \[ga]name\[ga] \[ga]type\[ga] [\[ga]key\[ga] \[ga]value\[ga]]* [flags]
|
|
\f[R]
|
|
.fi
|
|
.SS Options
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
\-h, \-\-help help for create
|
|
\-\-no\-obscure Force any passwords not to be obscured.
|
|
\-\-obscure Force any passwords to be obscured.
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
See the global flags page (https://rclone.org/flags/) for global options
|
|
not listed here.
|
|
.SS SEE ALSO
|
|
.IP \[bu] 2
|
|
rclone config (https://rclone.org/commands/rclone_config/) \- Enter an
|
|
interactive configuration session.
|
|
.SH rclone config delete
|
|
.PP
|
|
Delete an existing remote \f[C]name\f[R].
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone config delete \[ga]name\[ga] [flags]
|
|
\f[R]
|
|
.fi
|
|
.SS Options
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
\-h, \-\-help help for delete
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
See the global flags page (https://rclone.org/flags/) for global options
|
|
not listed here.
|
|
.SS SEE ALSO
|
|
.IP \[bu] 2
|
|
rclone config (https://rclone.org/commands/rclone_config/) \- Enter an
|
|
interactive configuration session.
|
|
.SH rclone config disconnect
|
|
.PP
|
|
Disconnects user from remote
|
|
.SS Synopsis
|
|
.PP
|
|
This disconnects the remote: passed in to the cloud storage system.
|
|
.PP
|
|
This normally means revoking the oauth token.
|
|
.PP
|
|
To reconnect use \[dq]rclone config reconnect\[dq].
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone config disconnect remote: [flags]
|
|
\f[R]
|
|
.fi
|
|
.SS Options
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
\-h, \-\-help help for disconnect
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
See the global flags page (https://rclone.org/flags/) for global options
|
|
not listed here.
|
|
.SS SEE ALSO
|
|
.IP \[bu] 2
|
|
rclone config (https://rclone.org/commands/rclone_config/) \- Enter an
|
|
interactive configuration session.
|
|
.SH rclone config dump
|
|
.PP
|
|
Dump the config file as JSON.
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone config dump [flags]
|
|
\f[R]
|
|
.fi
|
|
.SS Options
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
\-h, \-\-help help for dump
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
See the global flags page (https://rclone.org/flags/) for global options
|
|
not listed here.
|
|
.SS SEE ALSO
|
|
.IP \[bu] 2
|
|
rclone config (https://rclone.org/commands/rclone_config/) \- Enter an
|
|
interactive configuration session.
|
|
.SH rclone config edit
|
|
.PP
|
|
Enter an interactive configuration session.
|
|
.SS Synopsis
|
|
.PP
|
|
Enter an interactive configuration session where you can setup new
|
|
remotes and manage existing ones.
|
|
You may also set or remove a password to protect your configuration.
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone config edit [flags]
|
|
\f[R]
|
|
.fi
|
|
.SS Options
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
\-h, \-\-help help for edit
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
See the global flags page (https://rclone.org/flags/) for global options
|
|
not listed here.
|
|
.SS SEE ALSO
|
|
.IP \[bu] 2
|
|
rclone config (https://rclone.org/commands/rclone_config/) \- Enter an
|
|
interactive configuration session.
|
|
.SH rclone config file
|
|
.PP
|
|
Show path of configuration file in use.
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone config file [flags]
|
|
\f[R]
|
|
.fi
|
|
.SS Options
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
\-h, \-\-help help for file
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
See the global flags page (https://rclone.org/flags/) for global options
|
|
not listed here.
|
|
.SS SEE ALSO
|
|
.IP \[bu] 2
|
|
rclone config (https://rclone.org/commands/rclone_config/) \- Enter an
|
|
interactive configuration session.
|
|
.SH rclone config password
|
|
.PP
|
|
Update password in an existing remote.
|
|
.SS Synopsis
|
|
.PP
|
|
Update an existing remote\[aq]s password.
|
|
The password should be passed in pairs of \f[C]key\f[R] \f[C]value\f[R].
|
|
.PP
|
|
For example to set password of a remote of name myremote you would do:
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone config password myremote fieldname mypassword
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
This command is obsolete now that \[dq]config update\[dq] and
|
|
\[dq]config create\[dq] both support obscuring passwords directly.
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone config password \[ga]name\[ga] [\[ga]key\[ga] \[ga]value\[ga]]+ [flags]
|
|
\f[R]
|
|
.fi
|
|
.SS Options
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
\-h, \-\-help help for password
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
See the global flags page (https://rclone.org/flags/) for global options
|
|
not listed here.
|
|
.SS SEE ALSO
|
|
.IP \[bu] 2
|
|
rclone config (https://rclone.org/commands/rclone_config/) \- Enter an
|
|
interactive configuration session.
|
|
.SH rclone config providers
|
|
.PP
|
|
List in JSON format all the providers and options.
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone config providers [flags]
|
|
\f[R]
|
|
.fi
|
|
.SS Options
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
\-h, \-\-help help for providers
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
See the global flags page (https://rclone.org/flags/) for global options
|
|
not listed here.
|
|
.SS SEE ALSO
|
|
.IP \[bu] 2
|
|
rclone config (https://rclone.org/commands/rclone_config/) \- Enter an
|
|
interactive configuration session.
|
|
.SH rclone config reconnect
|
|
.PP
|
|
Re\-authenticates user with remote.
|
|
.SS Synopsis
|
|
.PP
|
|
This reconnects remote: passed in to the cloud storage system.
|
|
.PP
|
|
To disconnect the remote use \[dq]rclone config disconnect\[dq].
|
|
.PP
|
|
This normally means going through the interactive oauth flow again.
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone config reconnect remote: [flags]
|
|
\f[R]
|
|
.fi
|
|
.SS Options
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
\-h, \-\-help help for reconnect
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
See the global flags page (https://rclone.org/flags/) for global options
|
|
not listed here.
|
|
.SS SEE ALSO
|
|
.IP \[bu] 2
|
|
rclone config (https://rclone.org/commands/rclone_config/) \- Enter an
|
|
interactive configuration session.
|
|
.SH rclone config show
|
|
.PP
|
|
Print (decrypted) config file, or the config for a single remote.
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone config show [<remote>] [flags]
|
|
\f[R]
|
|
.fi
|
|
.SS Options
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
\-h, \-\-help help for show
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
See the global flags page (https://rclone.org/flags/) for global options
|
|
not listed here.
|
|
.SS SEE ALSO
|
|
.IP \[bu] 2
|
|
rclone config (https://rclone.org/commands/rclone_config/) \- Enter an
|
|
interactive configuration session.
|
|
.SH rclone config update
|
|
.PP
|
|
Update options in an existing remote.
|
|
.SS Synopsis
|
|
.PP
|
|
Update an existing remote\[aq]s options.
|
|
The options should be passed in in pairs of \f[C]key\f[R]
|
|
\f[C]value\f[R].
|
|
.PP
|
|
For example to update the env_auth field of a remote of name myremote
|
|
you would do:
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone config update myremote swift env_auth true
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
If any of the parameters passed is a password field, then rclone will
|
|
automatically obscure them if they aren\[aq]t already obscured before
|
|
putting them in the config file.
|
|
.PP
|
|
\f[B]NB\f[R] If the password parameter is 22 characters or longer and
|
|
consists only of base64 characters then rclone can get confused about
|
|
whether the password is already obscured or not and put unobscured
|
|
passwords into the config file.
|
|
If you want to be 100% certain that the passwords get obscured then use
|
|
the \[dq]\-\-obscure\[dq] flag, or if you are 100% certain you are
|
|
already passing obscured passwords then use \[dq]\-\-no\-obscure\[dq].
|
|
You can also set obscured passwords using the \[dq]rclone config
|
|
password\[dq] command.
|
|
.PP
|
|
If the remote uses OAuth the token will be updated, if you don\[aq]t
|
|
require this add an extra parameter thus:
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone config update myremote swift env_auth true config_refresh_token false
|
|
\f[R]
|
|
.fi
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone config update \[ga]name\[ga] [\[ga]key\[ga] \[ga]value\[ga]]+ [flags]
|
|
\f[R]
|
|
.fi
|
|
.SS Options
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
\-h, \-\-help help for update
|
|
\-\-no\-obscure Force any passwords not to be obscured.
|
|
\-\-obscure Force any passwords to be obscured.
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
See the global flags page (https://rclone.org/flags/) for global options
|
|
not listed here.
|
|
.SS SEE ALSO
|
|
.IP \[bu] 2
|
|
rclone config (https://rclone.org/commands/rclone_config/) \- Enter an
|
|
interactive configuration session.
|
|
.SH rclone config userinfo
|
|
.PP
|
|
Prints info about logged in user of remote.
|
|
.SS Synopsis
|
|
.PP
|
|
This prints the details of the person logged in to the cloud storage
|
|
system.
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone config userinfo remote: [flags]
|
|
\f[R]
|
|
.fi
|
|
.SS Options
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
\-h, \-\-help help for userinfo
|
|
\-\-json Format output as JSON
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
See the global flags page (https://rclone.org/flags/) for global options
|
|
not listed here.
|
|
.SS SEE ALSO
|
|
.IP \[bu] 2
|
|
rclone config (https://rclone.org/commands/rclone_config/) \- Enter an
|
|
interactive configuration session.
|
|
.SH rclone copyto
|
|
.PP
|
|
Copy files from source to dest, skipping already copied.
|
|
.SS Synopsis
|
|
.PP
|
|
If source:path is a file or directory then it copies it to a file or
|
|
directory named dest:path.
|
|
.PP
|
|
This can be used to upload single files to other than their current
|
|
name.
|
|
If the source is a directory then it acts exactly like the copy command.
|
|
.PP
|
|
So
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone copyto src dst
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
where src and dst are rclone paths, either remote:path or /path/to/local
|
|
or C:.
|
|
.PP
|
|
This will:
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
if src is file
|
|
copy it to dst, overwriting an existing file if it exists
|
|
if src is directory
|
|
copy it to dst, overwriting existing files if they exist
|
|
see copy command for full details
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
This doesn\[aq]t transfer unchanged files, testing by size and
|
|
modification time or MD5SUM.
|
|
It doesn\[aq]t delete files from the destination.
|
|
.PP
|
|
\f[B]Note\f[R]: Use the \f[C]\-P\f[R]/\f[C]\-\-progress\f[R] flag to
|
|
view real\-time transfer statistics
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone copyto source:path dest:path [flags]
|
|
\f[R]
|
|
.fi
|
|
.SS Options
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
\-h, \-\-help help for copyto
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
See the global flags page (https://rclone.org/flags/) for global options
|
|
not listed here.
|
|
.SS SEE ALSO
|
|
.IP \[bu] 2
|
|
rclone (https://rclone.org/commands/rclone/) \- Show help for rclone
|
|
commands, flags and backends.
|
|
.SH rclone copyurl
|
|
.PP
|
|
Copy url content to dest.
|
|
.SS Synopsis
|
|
.PP
|
|
Download a URL\[aq]s content and copy it to the destination without
|
|
saving it in temporary storage.
|
|
.PP
|
|
Setting \f[C]\-\-auto\-filename\f[R]will cause the file name to be
|
|
retrieved from the from URL (after any redirections) and used in the
|
|
destination path.
|
|
With \f[C]\-\-print\-filename\f[R] in addition, the resuling file name
|
|
will be printed.
|
|
.PP
|
|
Setting \f[C]\-\-no\-clobber\f[R] will prevent overwriting file on the
|
|
destination if there is one with the same name.
|
|
.PP
|
|
Setting \f[C]\-\-stdout\f[R] or making the output file name \f[C]\-\f[R]
|
|
will cause the output to be written to standard output.
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone copyurl https://example.com dest:path [flags]
|
|
\f[R]
|
|
.fi
|
|
.SS Options
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
\-a, \-\-auto\-filename Get the file name from the URL and use it for destination file path
|
|
\-h, \-\-help help for copyurl
|
|
\-\-no\-clobber Prevent overwriting file with same name
|
|
\-p, \-\-print\-filename Print the resulting name from \-\-auto\-filename
|
|
\-\-stdout Write the output to stdout rather than a file
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
See the global flags page (https://rclone.org/flags/) for global options
|
|
not listed here.
|
|
.SS SEE ALSO
|
|
.IP \[bu] 2
|
|
rclone (https://rclone.org/commands/rclone/) \- Show help for rclone
|
|
commands, flags and backends.
|
|
.SH rclone cryptcheck
|
|
.PP
|
|
Cryptcheck checks the integrity of a crypted remote.
|
|
.SS Synopsis
|
|
.PP
|
|
rclone cryptcheck checks a remote against a crypted remote.
|
|
This is the equivalent of running rclone check, but able to check the
|
|
checksums of the crypted remote.
|
|
.PP
|
|
For it to work the underlying remote of the cryptedremote must support
|
|
some kind of checksum.
|
|
.PP
|
|
It works by reading the nonce from each file on the cryptedremote: and
|
|
using that to encrypt each file on the remote:.
|
|
It then checks the checksum of the underlying file on the cryptedremote:
|
|
against the checksum of the file it has just encrypted.
|
|
.PP
|
|
Use it like this
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone cryptcheck /path/to/files encryptedremote:path
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
You can use it like this also, but that will involve downloading all the
|
|
files in remote:path.
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone cryptcheck remote:path encryptedremote:path
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
After it has run it will log the status of the encryptedremote:.
|
|
.PP
|
|
If you supply the \f[C]\-\-one\-way\f[R] flag, it will only check that
|
|
files in the source match the files in the destination, not the other
|
|
way around.
|
|
This means that extra files in the destination that are not in the
|
|
source will not be detected.
|
|
.PP
|
|
The \f[C]\-\-differ\f[R], \f[C]\-\-missing\-on\-dst\f[R],
|
|
\f[C]\-\-missing\-on\-src\f[R], \f[C]\-\-match\f[R] and
|
|
\f[C]\-\-error\f[R] flags write paths, one per line, to the file name
|
|
(or stdout if it is \f[C]\-\f[R]) supplied.
|
|
What they write is described in the help below.
|
|
For example \f[C]\-\-differ\f[R] will write all paths which are present
|
|
on both the source and destination but different.
|
|
.PP
|
|
The \f[C]\-\-combined\f[R] flag will write a file (or stdout) which
|
|
contains all file paths with a symbol and then a space and then the path
|
|
to tell you what happened to it.
|
|
These are reminiscent of diff files.
|
|
.IP \[bu] 2
|
|
\f[C]= path\f[R] means path was found in source and destination and was
|
|
identical
|
|
.IP \[bu] 2
|
|
\f[C]\- path\f[R] means path was missing on the source, so only in the
|
|
destination
|
|
.IP \[bu] 2
|
|
\f[C]+ path\f[R] means path was missing on the destination, so only in
|
|
the source
|
|
.IP \[bu] 2
|
|
\f[C]* path\f[R] means path was present in source and destination but
|
|
different.
|
|
.IP \[bu] 2
|
|
\f[C]! path\f[R] means there was an error reading or hashing the source
|
|
or dest.
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone cryptcheck remote:path cryptedremote:path [flags]
|
|
\f[R]
|
|
.fi
|
|
.SS Options
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
\-\-combined string Make a combined report of changes to this file
|
|
\-\-differ string Report all non\-matching files to this file
|
|
\-\-error string Report all files with errors (hashing or reading) to this file
|
|
\-h, \-\-help help for cryptcheck
|
|
\-\-match string Report all matching files to this file
|
|
\-\-missing\-on\-dst string Report all files missing from the destination to this file
|
|
\-\-missing\-on\-src string Report all files missing from the source to this file
|
|
\-\-one\-way Check one way only, source files must exist on remote
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
See the global flags page (https://rclone.org/flags/) for global options
|
|
not listed here.
|
|
.SS SEE ALSO
|
|
.IP \[bu] 2
|
|
rclone (https://rclone.org/commands/rclone/) \- Show help for rclone
|
|
commands, flags and backends.
|
|
.SH rclone cryptdecode
|
|
.PP
|
|
Cryptdecode returns unencrypted file names.
|
|
.SS Synopsis
|
|
.PP
|
|
rclone cryptdecode returns unencrypted file names when provided with a
|
|
list of encrypted file names.
|
|
List limit is 10 items.
|
|
.PP
|
|
If you supply the \-\-reverse flag, it will return encrypted file names.
|
|
.PP
|
|
use it like this
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone cryptdecode encryptedremote: encryptedfilename1 encryptedfilename2
|
|
|
|
rclone cryptdecode \-\-reverse encryptedremote: filename1 filename2
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
Another way to accomplish this is by using the
|
|
\f[C]rclone backend encode\f[R] (or \f[C]decode\f[R])command.
|
|
See the documentation on the \f[C]crypt\f[R] overlay for more info.
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone cryptdecode encryptedremote: encryptedfilename [flags]
|
|
\f[R]
|
|
.fi
|
|
.SS Options
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
\-h, \-\-help help for cryptdecode
|
|
\-\-reverse Reverse cryptdecode, encrypts filenames
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
See the global flags page (https://rclone.org/flags/) for global options
|
|
not listed here.
|
|
.SS SEE ALSO
|
|
.IP \[bu] 2
|
|
rclone (https://rclone.org/commands/rclone/) \- Show help for rclone
|
|
commands, flags and backends.
|
|
.SH rclone deletefile
|
|
.PP
|
|
Remove a single file from remote.
|
|
.SS Synopsis
|
|
.PP
|
|
Remove a single file from remote.
|
|
Unlike \f[C]delete\f[R] it cannot be used to remove a directory and it
|
|
doesn\[aq]t obey include/exclude filters \- if the specified file
|
|
exists, it will always be removed.
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone deletefile remote:path [flags]
|
|
\f[R]
|
|
.fi
|
|
.SS Options
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
\-h, \-\-help help for deletefile
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
See the global flags page (https://rclone.org/flags/) for global options
|
|
not listed here.
|
|
.SS SEE ALSO
|
|
.IP \[bu] 2
|
|
rclone (https://rclone.org/commands/rclone/) \- Show help for rclone
|
|
commands, flags and backends.
|
|
.SH rclone genautocomplete
|
|
.PP
|
|
Output completion script for a given shell.
|
|
.SS Synopsis
|
|
.PP
|
|
Generates a shell completion script for rclone.
|
|
Run with \-\-help to list the supported shells.
|
|
.SS Options
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
\-h, \-\-help help for genautocomplete
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
See the global flags page (https://rclone.org/flags/) for global options
|
|
not listed here.
|
|
.SS SEE ALSO
|
|
.IP \[bu] 2
|
|
rclone (https://rclone.org/commands/rclone/) \- Show help for rclone
|
|
commands, flags and backends.
|
|
.IP \[bu] 2
|
|
rclone genautocomplete
|
|
bash (https://rclone.org/commands/rclone_genautocomplete_bash/) \-
|
|
Output bash completion script for rclone.
|
|
.IP \[bu] 2
|
|
rclone genautocomplete
|
|
fish (https://rclone.org/commands/rclone_genautocomplete_fish/) \-
|
|
Output fish completion script for rclone.
|
|
.IP \[bu] 2
|
|
rclone genautocomplete
|
|
zsh (https://rclone.org/commands/rclone_genautocomplete_zsh/) \- Output
|
|
zsh completion script for rclone.
|
|
.SH rclone genautocomplete bash
|
|
.PP
|
|
Output bash completion script for rclone.
|
|
.SS Synopsis
|
|
.PP
|
|
Generates a bash shell autocompletion script for rclone.
|
|
.PP
|
|
This writes to /etc/bash_completion.d/rclone by default so will probably
|
|
need to be run with sudo or as root, e.g.
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
sudo rclone genautocomplete bash
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
Logout and login again to use the autocompletion scripts, or source them
|
|
directly
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
\&. /etc/bash_completion
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
If you supply a command line argument the script will be written there.
|
|
.PP
|
|
If output_file is \[dq]\-\[dq], then the output will be written to
|
|
stdout.
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone genautocomplete bash [output_file] [flags]
|
|
\f[R]
|
|
.fi
|
|
.SS Options
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
\-h, \-\-help help for bash
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
See the global flags page (https://rclone.org/flags/) for global options
|
|
not listed here.
|
|
.SS SEE ALSO
|
|
.IP \[bu] 2
|
|
rclone
|
|
genautocomplete (https://rclone.org/commands/rclone_genautocomplete/) \-
|
|
Output completion script for a given shell.
|
|
.SH rclone genautocomplete fish
|
|
.PP
|
|
Output fish completion script for rclone.
|
|
.SS Synopsis
|
|
.PP
|
|
Generates a fish autocompletion script for rclone.
|
|
.PP
|
|
This writes to /etc/fish/completions/rclone.fish by default so will
|
|
probably need to be run with sudo or as root, e.g.
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
sudo rclone genautocomplete fish
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
Logout and login again to use the autocompletion scripts, or source them
|
|
directly
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
\&. /etc/fish/completions/rclone.fish
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
If you supply a command line argument the script will be written there.
|
|
.PP
|
|
If output_file is \[dq]\-\[dq], then the output will be written to
|
|
stdout.
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone genautocomplete fish [output_file] [flags]
|
|
\f[R]
|
|
.fi
|
|
.SS Options
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
\-h, \-\-help help for fish
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
See the global flags page (https://rclone.org/flags/) for global options
|
|
not listed here.
|
|
.SS SEE ALSO
|
|
.IP \[bu] 2
|
|
rclone
|
|
genautocomplete (https://rclone.org/commands/rclone_genautocomplete/) \-
|
|
Output completion script for a given shell.
|
|
.SH rclone genautocomplete zsh
|
|
.PP
|
|
Output zsh completion script for rclone.
|
|
.SS Synopsis
|
|
.PP
|
|
Generates a zsh autocompletion script for rclone.
|
|
.PP
|
|
This writes to /usr/share/zsh/vendor\-completions/_rclone by default so
|
|
will probably need to be run with sudo or as root, e.g.
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
sudo rclone genautocomplete zsh
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
Logout and login again to use the autocompletion scripts, or source them
|
|
directly
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
autoload \-U compinit && compinit
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
If you supply a command line argument the script will be written there.
|
|
.PP
|
|
If output_file is \[dq]\-\[dq], then the output will be written to
|
|
stdout.
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone genautocomplete zsh [output_file] [flags]
|
|
\f[R]
|
|
.fi
|
|
.SS Options
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
\-h, \-\-help help for zsh
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
See the global flags page (https://rclone.org/flags/) for global options
|
|
not listed here.
|
|
.SS SEE ALSO
|
|
.IP \[bu] 2
|
|
rclone
|
|
genautocomplete (https://rclone.org/commands/rclone_genautocomplete/) \-
|
|
Output completion script for a given shell.
|
|
.SH rclone gendocs
|
|
.PP
|
|
Output markdown docs for rclone to the directory supplied.
|
|
.SS Synopsis
|
|
.PP
|
|
This produces markdown docs for the rclone commands to the directory
|
|
supplied.
|
|
These are in a format suitable for hugo to render into the rclone.org
|
|
website.
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone gendocs output_directory [flags]
|
|
\f[R]
|
|
.fi
|
|
.SS Options
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
\-h, \-\-help help for gendocs
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
See the global flags page (https://rclone.org/flags/) for global options
|
|
not listed here.
|
|
.SS SEE ALSO
|
|
.IP \[bu] 2
|
|
rclone (https://rclone.org/commands/rclone/) \- Show help for rclone
|
|
commands, flags and backends.
|
|
.SH rclone hashsum
|
|
.PP
|
|
Produces a hashsum file for all the objects in the path.
|
|
.SS Synopsis
|
|
.PP
|
|
Produces a hash file for all the objects in the path using the hash
|
|
named.
|
|
The output is in the same format as the standard md5sum/sha1sum tool.
|
|
.PP
|
|
By default, the hash is requested from the remote.
|
|
If the hash is not supported by the remote, no hash will be returned.
|
|
With the download flag, the file will be downloaded from the remote and
|
|
hashed locally enabling any hash for any remote.
|
|
.PP
|
|
Run without a hash to see the list of all supported hashes, e.g.
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
$ rclone hashsum
|
|
Supported hashes are:
|
|
* MD5
|
|
* SHA\-1
|
|
* DropboxHash
|
|
* QuickXorHash
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
Then
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
$ rclone hashsum MD5 remote:path
|
|
\f[R]
|
|
.fi
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone hashsum <hash> remote:path [flags]
|
|
\f[R]
|
|
.fi
|
|
.SS Options
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
\-\-base64 Output base64 encoded hashsum
|
|
\-\-download Download the file and hash it locally; if this flag is not specified, the hash is requested from the remote
|
|
\-h, \-\-help help for hashsum
|
|
\-\-output\-file string Output hashsums to a file rather than the terminal
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
See the global flags page (https://rclone.org/flags/) for global options
|
|
not listed here.
|
|
.SS SEE ALSO
|
|
.IP \[bu] 2
|
|
rclone (https://rclone.org/commands/rclone/) \- Show help for rclone
|
|
commands, flags and backends.
|
|
.SH rclone link
|
|
.PP
|
|
Generate public link to file/folder.
|
|
.SS Synopsis
|
|
.PP
|
|
rclone link will create, retrieve or remove a public link to the given
|
|
file or folder.
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone link remote:path/to/file
|
|
rclone link remote:path/to/folder/
|
|
rclone link \-\-unlink remote:path/to/folder/
|
|
rclone link \-\-expire 1d remote:path/to/file
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
If you supply the \-\-expire flag, it will set the expiration time
|
|
otherwise it will use the default (100 years).
|
|
\f[B]Note\f[R] not all backends support the \-\-expire flag \- if the
|
|
backend doesn\[aq]t support it then the link returned won\[aq]t expire.
|
|
.PP
|
|
Use the \-\-unlink flag to remove existing public links to the file or
|
|
folder.
|
|
\f[B]Note\f[R] not all backends support \[dq]\-\-unlink\[dq] flag \-
|
|
those that don\[aq]t will just ignore it.
|
|
.PP
|
|
If successful, the last line of the output will contain the link.
|
|
Exact capabilities depend on the remote, but the link will always by
|
|
default be created with the least constraints \[en] e.g.
|
|
no expiry, no password protection, accessible without account.
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone link remote:path [flags]
|
|
\f[R]
|
|
.fi
|
|
.SS Options
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
\-\-expire Duration The amount of time that the link will be valid (default 100y)
|
|
\-h, \-\-help help for link
|
|
\-\-unlink Remove existing public link to file/folder
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
See the global flags page (https://rclone.org/flags/) for global options
|
|
not listed here.
|
|
.SS SEE ALSO
|
|
.IP \[bu] 2
|
|
rclone (https://rclone.org/commands/rclone/) \- Show help for rclone
|
|
commands, flags and backends.
|
|
.SH rclone listremotes
|
|
.PP
|
|
List all the remotes in the config file.
|
|
.SS Synopsis
|
|
.PP
|
|
rclone listremotes lists all the available remotes from the config file.
|
|
.PP
|
|
When uses with the \-l flag it lists the types too.
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone listremotes [flags]
|
|
\f[R]
|
|
.fi
|
|
.SS Options
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
\-h, \-\-help help for listremotes
|
|
\-\-long Show the type as well as names.
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
See the global flags page (https://rclone.org/flags/) for global options
|
|
not listed here.
|
|
.SS SEE ALSO
|
|
.IP \[bu] 2
|
|
rclone (https://rclone.org/commands/rclone/) \- Show help for rclone
|
|
commands, flags and backends.
|
|
.SH rclone lsf
|
|
.PP
|
|
List directories and objects in remote:path formatted for parsing.
|
|
.SS Synopsis
|
|
.PP
|
|
List the contents of the source path (directories and objects) to
|
|
standard output in a form which is easy to parse by scripts.
|
|
By default this will just be the names of the objects and directories,
|
|
one per line.
|
|
The directories will have a / suffix.
|
|
.PP
|
|
Eg
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
$ rclone lsf swift:bucket
|
|
bevajer5jef
|
|
canole
|
|
diwogej7
|
|
ferejej3gux/
|
|
fubuwic
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
Use the \-\-format option to control what gets listed.
|
|
By default this is just the path, but you can use these parameters to
|
|
control the output:
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
p \- path
|
|
s \- size
|
|
t \- modification time
|
|
h \- hash
|
|
i \- ID of object
|
|
o \- Original ID of underlying object
|
|
m \- MimeType of object if known
|
|
e \- encrypted name
|
|
T \- tier of storage if known, e.g. \[dq]Hot\[dq] or \[dq]Cool\[dq]
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
So if you wanted the path, size and modification time, you would use
|
|
\-\-format \[dq]pst\[dq], or maybe \-\-format \[dq]tsp\[dq] to put the
|
|
path last.
|
|
.PP
|
|
Eg
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
$ rclone lsf \-\-format \[dq]tsp\[dq] swift:bucket
|
|
2016\-06\-25 18:55:41;60295;bevajer5jef
|
|
2016\-06\-25 18:55:43;90613;canole
|
|
2016\-06\-25 18:55:43;94467;diwogej7
|
|
2018\-04\-26 08:50:45;0;ferejej3gux/
|
|
2016\-06\-25 18:55:40;37600;fubuwic
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
If you specify \[dq]h\[dq] in the format you will get the MD5 hash by
|
|
default, use the \[dq]\-\-hash\[dq] flag to change which hash you want.
|
|
Note that this can be returned as an empty string if it isn\[aq]t
|
|
available on the object (and for directories), \[dq]ERROR\[dq] if there
|
|
was an error reading it from the object and \[dq]UNSUPPORTED\[dq] if
|
|
that object does not support that hash type.
|
|
.PP
|
|
For example to emulate the md5sum command you can use
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone lsf \-R \-\-hash MD5 \-\-format hp \-\-separator \[dq] \[dq] \-\-files\-only .
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
Eg
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
$ rclone lsf \-R \-\-hash MD5 \-\-format hp \-\-separator \[dq] \[dq] \-\-files\-only swift:bucket
|
|
7908e352297f0f530b84a756f188baa3 bevajer5jef
|
|
cd65ac234e6fea5925974a51cdd865cc canole
|
|
03b5341b4f234b9d984d03ad076bae91 diwogej7
|
|
8fd37c3810dd660778137ac3a66cc06d fubuwic
|
|
99713e14a4c4ff553acaf1930fad985b gixacuh7ku
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
(Though \[dq]rclone md5sum .\[dq] is an easier way of typing this.)
|
|
.PP
|
|
By default the separator is \[dq];\[dq] this can be changed with the
|
|
\-\-separator flag.
|
|
Note that separators aren\[aq]t escaped in the path so putting it last
|
|
is a good strategy.
|
|
.PP
|
|
Eg
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
$ rclone lsf \-\-separator \[dq],\[dq] \-\-format \[dq]tshp\[dq] swift:bucket
|
|
2016\-06\-25 18:55:41,60295,7908e352297f0f530b84a756f188baa3,bevajer5jef
|
|
2016\-06\-25 18:55:43,90613,cd65ac234e6fea5925974a51cdd865cc,canole
|
|
2016\-06\-25 18:55:43,94467,03b5341b4f234b9d984d03ad076bae91,diwogej7
|
|
2018\-04\-26 08:52:53,0,,ferejej3gux/
|
|
2016\-06\-25 18:55:40,37600,8fd37c3810dd660778137ac3a66cc06d,fubuwic
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
You can output in CSV standard format.
|
|
This will escape things in \[dq] if they contain ,
|
|
.PP
|
|
Eg
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
$ rclone lsf \-\-csv \-\-files\-only \-\-format ps remote:path
|
|
test.log,22355
|
|
test.sh,449
|
|
\[dq]this file contains a comma, in the file name.txt\[dq],6
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
Note that the \-\-absolute parameter is useful for making lists of files
|
|
to pass to an rclone copy with the \-\-files\-from\-raw flag.
|
|
.PP
|
|
For example to find all the files modified within one day and copy those
|
|
only (without traversing the whole directory structure):
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone lsf \-\-absolute \-\-files\-only \-\-max\-age 1d /path/to/local > new_files
|
|
rclone copy \-\-files\-from\-raw new_files /path/to/local remote:path
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
Any of the filtering options can be applied to this command.
|
|
.PP
|
|
There are several related list commands
|
|
.IP \[bu] 2
|
|
\f[C]ls\f[R] to list size and path of objects only
|
|
.IP \[bu] 2
|
|
\f[C]lsl\f[R] to list modification time, size and path of objects only
|
|
.IP \[bu] 2
|
|
\f[C]lsd\f[R] to list directories only
|
|
.IP \[bu] 2
|
|
\f[C]lsf\f[R] to list objects and directories in easy to parse format
|
|
.IP \[bu] 2
|
|
\f[C]lsjson\f[R] to list objects and directories in JSON format
|
|
.PP
|
|
\f[C]ls\f[R],\f[C]lsl\f[R],\f[C]lsd\f[R] are designed to be human
|
|
readable.
|
|
\f[C]lsf\f[R] is designed to be human and machine readable.
|
|
\f[C]lsjson\f[R] is designed to be machine readable.
|
|
.PP
|
|
Note that \f[C]ls\f[R] and \f[C]lsl\f[R] recurse by default \- use
|
|
\f[C]\-\-max\-depth 1\f[R] to stop the recursion.
|
|
.PP
|
|
The other list commands \f[C]lsd\f[R],\f[C]lsf\f[R],\f[C]lsjson\f[R] do
|
|
not recurse by default \- use \f[C]\-R\f[R] to make them recurse.
|
|
.PP
|
|
Listing a non existent directory will produce an error except for
|
|
remotes which can\[aq]t have empty directories (e.g.
|
|
s3, swift, or gcs \- the bucket based remotes).
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone lsf remote:path [flags]
|
|
\f[R]
|
|
.fi
|
|
.SS Options
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
\-\-absolute Put a leading / in front of path names.
|
|
\-\-csv Output in CSV format.
|
|
\-d, \-\-dir\-slash Append a slash to directory names. (default true)
|
|
\-\-dirs\-only Only list directories.
|
|
\-\-files\-only Only list files.
|
|
\-F, \-\-format string Output format \- see help for details (default \[dq]p\[dq])
|
|
\-\-hash h Use this hash when h is used in the format MD5|SHA\-1|DropboxHash (default \[dq]MD5\[dq])
|
|
\-h, \-\-help help for lsf
|
|
\-R, \-\-recursive Recurse into the listing.
|
|
\-s, \-\-separator string Separator for the items in the format. (default \[dq];\[dq])
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
See the global flags page (https://rclone.org/flags/) for global options
|
|
not listed here.
|
|
.SS SEE ALSO
|
|
.IP \[bu] 2
|
|
rclone (https://rclone.org/commands/rclone/) \- Show help for rclone
|
|
commands, flags and backends.
|
|
.SH rclone lsjson
|
|
.PP
|
|
List directories and objects in the path in JSON format.
|
|
.SS Synopsis
|
|
.PP
|
|
List directories and objects in the path in JSON format.
|
|
.PP
|
|
The output is an array of Items, where each Item looks like this
|
|
.PP
|
|
{ \[dq]Hashes\[dq] : { \[dq]SHA\-1\[dq] :
|
|
\[dq]f572d396fae9206628714fb2ce00f72e94f2258f\[dq], \[dq]MD5\[dq] :
|
|
\[dq]b1946ac92492d2347c6235b4d2611184\[dq], \[dq]DropboxHash\[dq] :
|
|
\[dq]ecb65bb98f9d905b70458986c39fcbad7715e5f2fcc3b1f07767d7c83e2438cc\[dq]
|
|
}, \[dq]ID\[dq]: \[dq]y2djkhiujf83u33\[dq], \[dq]OrigID\[dq]:
|
|
\[dq]UYOJVTUW00Q1RzTDA\[dq], \[dq]IsBucket\[dq] : false, \[dq]IsDir\[dq]
|
|
: false, \[dq]MimeType\[dq] : \[dq]application/octet\-stream\[dq],
|
|
\[dq]ModTime\[dq] : \[dq]2017\-05\-31T16:15:57.034468261+01:00\[dq],
|
|
\[dq]Name\[dq] : \[dq]file.txt\[dq], \[dq]Encrypted\[dq] :
|
|
\[dq]v0qpsdq8anpci8n929v3uu9338\[dq], \[dq]EncryptedPath\[dq] :
|
|
\[dq]kja9098349023498/v0qpsdq8anpci8n929v3uu9338\[dq], \[dq]Path\[dq] :
|
|
\[dq]full/path/goes/here/file.txt\[dq], \[dq]Size\[dq] : 6,
|
|
\[dq]Tier\[dq] : \[dq]hot\[dq], }
|
|
.PP
|
|
If \-\-hash is not specified the Hashes property won\[aq]t be emitted.
|
|
The types of hash can be specified with the \-\-hash\-type parameter
|
|
(which may be repeated).
|
|
If \-\-hash\-type is set then it implies \-\-hash.
|
|
.PP
|
|
If \-\-no\-modtime is specified then ModTime will be blank.
|
|
This can speed things up on remotes where reading the ModTime takes an
|
|
extra request (e.g.
|
|
s3, swift).
|
|
.PP
|
|
If \-\-no\-mimetype is specified then MimeType will be blank.
|
|
This can speed things up on remotes where reading the MimeType takes an
|
|
extra request (e.g.
|
|
s3, swift).
|
|
.PP
|
|
If \-\-encrypted is not specified the Encrypted won\[aq]t be emitted.
|
|
.PP
|
|
If \-\-dirs\-only is not specified files in addition to directories are
|
|
returned
|
|
.PP
|
|
If \-\-files\-only is not specified directories in addition to the files
|
|
will be returned.
|
|
.PP
|
|
The Path field will only show folders below the remote path being
|
|
listed.
|
|
If \[dq]remote:path\[dq] contains the file \[dq]subfolder/file.txt\[dq],
|
|
the Path for \[dq]file.txt\[dq] will be \[dq]subfolder/file.txt\[dq],
|
|
not \[dq]remote:path/subfolder/file.txt\[dq].
|
|
When used without \-\-recursive the Path will always be the same as
|
|
Name.
|
|
.PP
|
|
If the directory is a bucket in a bucket based backend, then
|
|
\[dq]IsBucket\[dq] will be set to true.
|
|
This key won\[aq]t be present unless it is \[dq]true\[dq].
|
|
.PP
|
|
The time is in RFC3339 format with up to nanosecond precision.
|
|
The number of decimal digits in the seconds will depend on the precision
|
|
that the remote can hold the times, so if times are accurate to the
|
|
nearest millisecond (e.g.
|
|
Google Drive) then 3 digits will always be shown
|
|
(\[dq]2017\-05\-31T16:15:57.034+01:00\[dq]) whereas if the times are
|
|
accurate to the nearest second (Dropbox, Box, WebDav, etc.) no digits
|
|
will be shown (\[dq]2017\-05\-31T16:15:57+01:00\[dq]).
|
|
.PP
|
|
The whole output can be processed as a JSON blob, or alternatively it
|
|
can be processed line by line as each item is written one to a line.
|
|
.PP
|
|
Any of the filtering options can be applied to this command.
|
|
.PP
|
|
There are several related list commands
|
|
.IP \[bu] 2
|
|
\f[C]ls\f[R] to list size and path of objects only
|
|
.IP \[bu] 2
|
|
\f[C]lsl\f[R] to list modification time, size and path of objects only
|
|
.IP \[bu] 2
|
|
\f[C]lsd\f[R] to list directories only
|
|
.IP \[bu] 2
|
|
\f[C]lsf\f[R] to list objects and directories in easy to parse format
|
|
.IP \[bu] 2
|
|
\f[C]lsjson\f[R] to list objects and directories in JSON format
|
|
.PP
|
|
\f[C]ls\f[R],\f[C]lsl\f[R],\f[C]lsd\f[R] are designed to be human
|
|
readable.
|
|
\f[C]lsf\f[R] is designed to be human and machine readable.
|
|
\f[C]lsjson\f[R] is designed to be machine readable.
|
|
.PP
|
|
Note that \f[C]ls\f[R] and \f[C]lsl\f[R] recurse by default \- use
|
|
\f[C]\-\-max\-depth 1\f[R] to stop the recursion.
|
|
.PP
|
|
The other list commands \f[C]lsd\f[R],\f[C]lsf\f[R],\f[C]lsjson\f[R] do
|
|
not recurse by default \- use \f[C]\-R\f[R] to make them recurse.
|
|
.PP
|
|
Listing a non existent directory will produce an error except for
|
|
remotes which can\[aq]t have empty directories (e.g.
|
|
s3, swift, or gcs \- the bucket based remotes).
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone lsjson remote:path [flags]
|
|
\f[R]
|
|
.fi
|
|
.SS Options
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
\-\-dirs\-only Show only directories in the listing.
|
|
\-M, \-\-encrypted Show the encrypted names.
|
|
\-\-files\-only Show only files in the listing.
|
|
\-\-hash Include hashes in the output (may take longer).
|
|
\-\-hash\-type stringArray Show only this hash type (may be repeated).
|
|
\-h, \-\-help help for lsjson
|
|
\-\-no\-mimetype Don\[aq]t read the mime type (can speed things up).
|
|
\-\-no\-modtime Don\[aq]t read the modification time (can speed things up).
|
|
\-\-original Show the ID of the underlying Object.
|
|
\-R, \-\-recursive Recurse into the listing.
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
See the global flags page (https://rclone.org/flags/) for global options
|
|
not listed here.
|
|
.SS SEE ALSO
|
|
.IP \[bu] 2
|
|
rclone (https://rclone.org/commands/rclone/) \- Show help for rclone
|
|
commands, flags and backends.
|
|
.SH rclone mount
|
|
.PP
|
|
Mount the remote as file system on a mountpoint.
|
|
.SS Synopsis
|
|
.PP
|
|
rclone mount allows Linux, FreeBSD, macOS and Windows to mount any of
|
|
Rclone\[aq]s cloud storage systems as a file system with FUSE.
|
|
.PP
|
|
First set up your remote using \f[C]rclone config\f[R].
|
|
Check it works with \f[C]rclone ls\f[R] etc.
|
|
.PP
|
|
On Linux and OSX, you can either run mount in foreground mode or
|
|
background (daemon) mode.
|
|
Mount runs in foreground mode by default, use the \f[C]\-\-daemon\f[R]
|
|
flag to specify background mode.
|
|
You can only run mount in foreground mode on Windows.
|
|
.PP
|
|
On Linux/macOS/FreeBSD start the mount like this, where
|
|
\f[C]/path/to/local/mount\f[R] is an \f[B]empty\f[R] \f[B]existing\f[R]
|
|
directory:
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone mount remote:path/to/files /path/to/local/mount
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
On Windows you can start a mount in different ways.
|
|
See below for details.
|
|
The following examples will mount to an automatically assigned drive, to
|
|
specific drive letter \f[C]X:\f[R], to path
|
|
\f[C]C:\[rs]path\[rs]parent\[rs]mount\f[R] (where parent directory or
|
|
drive must exist, and mount must \f[B]not\f[R] exist, and is not
|
|
supported when mounting as a network drive), and the last example will
|
|
mount as network share \f[C]\[rs]\[rs]cloud\[rs]remote\f[R] and map it
|
|
to an automatically assigned drive:
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone mount remote:path/to/files *
|
|
rclone mount remote:path/to/files X:
|
|
rclone mount remote:path/to/files C:\[rs]path\[rs]parent\[rs]mount
|
|
rclone mount remote:path/to/files \[rs]\[rs]cloud\[rs]remote
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
When the program ends while in foreground mode, either via Ctrl+C or
|
|
receiving a SIGINT or SIGTERM signal, the mount should be automatically
|
|
stopped.
|
|
.PP
|
|
When running in background mode the user will have to stop the mount
|
|
manually:
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
# Linux
|
|
fusermount \-u /path/to/local/mount
|
|
# OS X
|
|
umount /path/to/local/mount
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
The umount operation can fail, for example when the mountpoint is busy.
|
|
When that happens, it is the user\[aq]s responsibility to stop the mount
|
|
manually.
|
|
.PP
|
|
The size of the mounted file system will be set according to information
|
|
retrieved from the remote, the same as returned by the rclone
|
|
about (https://rclone.org/commands/rclone_about/) command.
|
|
Remotes with unlimited storage may report the used size only, then an
|
|
additional 1PB of free space is assumed.
|
|
If the remote does not
|
|
support (https://rclone.org/overview/#optional-features) the about
|
|
feature at all, then 1PB is set as both the total and the free size.
|
|
.PP
|
|
\f[B]Note\f[R]: As of \f[C]rclone\f[R] 1.52.2, \f[C]rclone mount\f[R]
|
|
now requires Go version 1.13 or newer on some platforms depending on the
|
|
underlying FUSE library in use.
|
|
.SS Installing on Windows
|
|
.PP
|
|
To run rclone mount on Windows, you will need to download and install
|
|
WinFsp (http://www.secfs.net/winfsp/).
|
|
.PP
|
|
WinFsp (https://github.com/billziss-gh/winfsp) is an open source Windows
|
|
File System Proxy which makes it easy to write user space file systems
|
|
for Windows.
|
|
It provides a FUSE emulation layer which rclone uses combination with
|
|
cgofuse (https://github.com/billziss-gh/cgofuse).
|
|
Both of these packages are by Bill Zissimopoulos who was very helpful
|
|
during the implementation of rclone mount for Windows.
|
|
.SS Mounting modes on windows
|
|
.PP
|
|
Unlike other operating systems, Microsoft Windows provides a different
|
|
filesystem type for network and fixed drives.
|
|
It optimises access on the assumption fixed disk drives are fast and
|
|
reliable, while network drives have relatively high latency and less
|
|
reliability.
|
|
Some settings can also be differentiated between the two types, for
|
|
example that Windows Explorer should just display icons and not create
|
|
preview thumbnails for image and video files on network drives.
|
|
.PP
|
|
In most cases, rclone will mount the remote as a normal, fixed disk
|
|
drive by default.
|
|
However, you can also choose to mount it as a remote network drive,
|
|
often described as a network share.
|
|
If you mount an rclone remote using the default, fixed drive mode and
|
|
experience unexpected program errors, freezes or other issues, consider
|
|
mounting as a network drive instead.
|
|
.PP
|
|
When mounting as a fixed disk drive you can either mount to an unused
|
|
drive letter, or to a path representing a \f[B]non\-existent\f[R]
|
|
subdirectory of an \f[B]existing\f[R] parent directory or drive.
|
|
Using the special value \f[C]*\f[R] will tell rclone to automatically
|
|
assign the next available drive letter, starting with Z: and moving
|
|
backward.
|
|
Examples:
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone mount remote:path/to/files *
|
|
rclone mount remote:path/to/files X:
|
|
rclone mount remote:path/to/files C:\[rs]path\[rs]parent\[rs]mount
|
|
rclone mount remote:path/to/files X:
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
Option \f[C]\-\-volname\f[R] can be used to set a custom volume name for
|
|
the mounted file system.
|
|
The default is to use the remote name and path.
|
|
.PP
|
|
To mount as network drive, you can add option
|
|
\f[C]\-\-network\-mode\f[R] to your mount command.
|
|
Mounting to a directory path is not supported in this mode, it is a
|
|
limitation Windows imposes on junctions, so the remote must always be
|
|
mounted to a drive letter.
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone mount remote:path/to/files X: \-\-network\-mode
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
A volume name specified with \f[C]\-\-volname\f[R] will be used to
|
|
create the network share path.
|
|
A complete UNC path, such as \f[C]\[rs]\[rs]cloud\[rs]remote\f[R],
|
|
optionally with path
|
|
\f[C]\[rs]\[rs]cloud\[rs]remote\[rs]madeup\[rs]path\f[R], will be used
|
|
as is.
|
|
Any other string will be used as the share part, after a default prefix
|
|
\f[C]\[rs]\[rs]server\[rs]\f[R].
|
|
If no volume name is specified then \f[C]\[rs]\[rs]server\[rs]share\f[R]
|
|
will be used.
|
|
You must make sure the volume name is unique when you are mounting more
|
|
than one drive, or else the mount command will fail.
|
|
The share name will treated as the volume label for the mapped drive,
|
|
shown in Windows Explorer etc, while the complete
|
|
\f[C]\[rs]\[rs]server\[rs]share\f[R] will be reported as the remote UNC
|
|
path by \f[C]net use\f[R] etc, just like a normal network drive mapping.
|
|
.PP
|
|
If you specify a full network share UNC path with \f[C]\-\-volname\f[R],
|
|
this will implicitely set the \f[C]\-\-network\-mode\f[R] option, so the
|
|
following two examples have same result:
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone mount remote:path/to/files X: \-\-network\-mode
|
|
rclone mount remote:path/to/files X: \-\-volname \[rs]\[rs]server\[rs]share
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
You may also specify the network share UNC path as the mountpoint
|
|
itself.
|
|
Then rclone will automatically assign a drive letter, same as with
|
|
\f[C]*\f[R] and use that as mountpoint, and instead use the UNC path
|
|
specified as the volume name, as if it were specified with the
|
|
\f[C]\-\-volname\f[R] option.
|
|
This will also implicitely set the \f[C]\-\-network\-mode\f[R] option.
|
|
This means the following two examples have same result:
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone mount remote:path/to/files \[rs]\[rs]cloud\[rs]remote
|
|
rclone mount remote:path/to/files * \-\-volname \[rs]\[rs]cloud\[rs]remote
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
There is yet another way to enable network mode, and to set the share
|
|
path, and that is to pass the \[dq]native\[dq] libfuse/WinFsp option
|
|
directly:
|
|
\f[C]\-\-fuse\-flag \-\-VolumePrefix=\[rs]server\[rs]share\f[R].
|
|
Note that the path must be with just a single backslash prefix in this
|
|
case.
|
|
.PP
|
|
\f[I]Note:\f[R] In previous versions of rclone this was the only
|
|
supported method.
|
|
.PP
|
|
Read more about drive
|
|
mapping (https://en.wikipedia.org/wiki/Drive_mapping)
|
|
.PP
|
|
See also Limitations section below.
|
|
.SS Windows filesystem permissions
|
|
.PP
|
|
The FUSE emulation layer on Windows must convert between the
|
|
POSIX\-based permission model used in FUSE, and the permission model
|
|
used in Windows, based on access\-control lists (ACL).
|
|
.PP
|
|
The mounted filesystem will normally get three entries in its
|
|
access\-control list (ACL), representing permissions for the POSIX
|
|
permission scopes: Owner, group and others.
|
|
By default, the owner and group will be taken from the current user, and
|
|
the built\-in group \[dq]Everyone\[dq] will be used to represent others.
|
|
The user/group can be customized with FUSE options \[dq]UserName\[dq]
|
|
and \[dq]GroupName\[dq], e.g.
|
|
\f[C]\-o UserName=user123 \-o GroupName=\[dq]Authenticated Users\[dq]\f[R].
|
|
.PP
|
|
The permissions on each entry will be set according to options
|
|
\f[C]\-\-dir\-perms\f[R] and \f[C]\-\-file\-perms\f[R], which takes a
|
|
value in traditional numeric
|
|
notation (https://en.wikipedia.org/wiki/File-system_permissions#Numeric_notation),
|
|
where the default corresponds to
|
|
\f[C]\-\-file\-perms 0666 \-\-dir\-perms 0777\f[R].
|
|
.PP
|
|
Note that the mapping of permissions is not always trivial, and the
|
|
result you see in Windows Explorer may not be exactly like you expected.
|
|
For example, when setting a value that includes write access, this will
|
|
be mapped to individual permissions \[dq]write attributes\[dq],
|
|
\[dq]write data\[dq] and \[dq]append data\[dq], but not \[dq]write
|
|
extended attributes\[dq].
|
|
Windows will then show this as basic permission \[dq]Special\[dq]
|
|
instead of \[dq]Write\[dq], because \[dq]Write\[dq] includes the
|
|
\[dq]write extended attributes\[dq] permission.
|
|
.PP
|
|
If you set POSIX permissions for only allowing access to the owner,
|
|
using \f[C]\-\-file\-perms 0600 \-\-dir\-perms 0700\f[R], the user group
|
|
and the built\-in \[dq]Everyone\[dq] group will still be given some
|
|
special permissions, such as \[dq]read attributes\[dq] and \[dq]read
|
|
permissions\[dq], in Windows.
|
|
This is done for compatibility reasons, e.g.
|
|
to allow users without additional permissions to be able to read basic
|
|
metadata about files like in UNIX.
|
|
One case that may arise is that other programs (incorrectly) interprets
|
|
this as the file being accessible by everyone.
|
|
For example an SSH client may warn about \[dq]unprotected private key
|
|
file\[dq].
|
|
.PP
|
|
WinFsp 2021 (version 1.9, still in beta) introduces a new FUSE option
|
|
\[dq]FileSecurity\[dq], that allows the complete specification of file
|
|
security descriptors using
|
|
SDDL (https://docs.microsoft.com/en-us/windows/win32/secauthz/security-descriptor-string-format).
|
|
With this you can work around issues such as the mentioned
|
|
\[dq]unprotected private key file\[dq] by specifying
|
|
\f[C]\-o FileSecurity=\[dq]D:P(A;;FA;;;OW)\[dq]\f[R], for file all
|
|
access (FA) to the owner (OW).
|
|
.SS Windows caveats
|
|
.PP
|
|
Note that drives created as Administrator are not visible by other
|
|
accounts (including the account that was elevated as Administrator).
|
|
So if you start a Windows drive from an Administrative Command Prompt
|
|
and then try to access the same drive from Explorer (which does not run
|
|
as Administrator), you will not be able to see the new drive.
|
|
.PP
|
|
The easiest way around this is to start the drive from a normal command
|
|
prompt.
|
|
It is also possible to start a drive from the SYSTEM account (using the
|
|
WinFsp.Launcher
|
|
infrastructure (https://github.com/billziss-gh/winfsp/wiki/WinFsp-Service-Architecture))
|
|
which creates drives accessible for everyone on the system or
|
|
alternatively using the nssm service manager (https://nssm.cc/usage).
|
|
.SS Limitations
|
|
.PP
|
|
Without the use of \f[C]\-\-vfs\-cache\-mode\f[R] this can only write
|
|
files sequentially, it can only seek when reading.
|
|
This means that many applications won\[aq]t work with their files on an
|
|
rclone mount without \f[C]\-\-vfs\-cache\-mode writes\f[R] or
|
|
\f[C]\-\-vfs\-cache\-mode full\f[R].
|
|
See the VFS File Caching section for more info.
|
|
.PP
|
|
The bucket based remotes (e.g.
|
|
Swift, S3, Google Compute Storage, B2, Hubic) do not support the concept
|
|
of empty directories, so empty directories will have a tendency to
|
|
disappear once they fall out of the directory cache.
|
|
.PP
|
|
Only supported on Linux, FreeBSD, OS X and Windows at the moment.
|
|
.SS rclone mount vs rclone sync/copy
|
|
.PP
|
|
File systems expect things to be 100% reliable, whereas cloud storage
|
|
systems are a long way from 100% reliable.
|
|
The rclone sync/copy commands cope with this with lots of retries.
|
|
However rclone mount can\[aq]t use retries in the same way without
|
|
making local copies of the uploads.
|
|
Look at the VFS File Caching for solutions to make mount more reliable.
|
|
.SS Attribute caching
|
|
.PP
|
|
You can use the flag \f[C]\-\-attr\-timeout\f[R] to set the time the
|
|
kernel caches the attributes (size, modification time, etc.) for
|
|
directory entries.
|
|
.PP
|
|
The default is \f[C]1s\f[R] which caches files just long enough to avoid
|
|
too many callbacks to rclone from the kernel.
|
|
.PP
|
|
In theory 0s should be the correct value for filesystems which can
|
|
change outside the control of the kernel.
|
|
However this causes quite a few problems such as rclone using too much
|
|
memory (https://github.com/rclone/rclone/issues/2157), rclone not
|
|
serving files to
|
|
samba (https://forum.rclone.org/t/rclone-1-39-vs-1-40-mount-issue/5112)
|
|
and excessive time listing
|
|
directories (https://github.com/rclone/rclone/issues/2095#issuecomment-371141147).
|
|
.PP
|
|
The kernel can cache the info about a file for the time given by
|
|
\f[C]\-\-attr\-timeout\f[R].
|
|
You may see corruption if the remote file changes length during this
|
|
window.
|
|
It will show up as either a truncated file or a file with garbage on the
|
|
end.
|
|
With \f[C]\-\-attr\-timeout 1s\f[R] this is very unlikely but not
|
|
impossible.
|
|
The higher you set \f[C]\-\-attr\-timeout\f[R] the more likely it is.
|
|
The default setting of \[dq]1s\[dq] is the lowest setting which
|
|
mitigates the problems above.
|
|
.PP
|
|
If you set it higher (\f[C]10s\f[R] or \f[C]1m\f[R] say) then the kernel
|
|
will call back to rclone less often making it more efficient, however
|
|
there is more chance of the corruption issue above.
|
|
.PP
|
|
If files don\[aq]t change on the remote outside of the control of rclone
|
|
then there is no chance of corruption.
|
|
.PP
|
|
This is the same as setting the attr_timeout option in mount.fuse.
|
|
.SS Filters
|
|
.PP
|
|
Note that all the rclone filters can be used to select a subset of the
|
|
files to be visible in the mount.
|
|
.SS systemd
|
|
.PP
|
|
When running rclone mount as a systemd service, it is possible to use
|
|
Type=notify.
|
|
In this case the service will enter the started state after the
|
|
mountpoint has been successfully set up.
|
|
Units having the rclone mount service specified as a requirement will
|
|
see all files and folders immediately in this mode.
|
|
.SS chunked reading
|
|
.PP
|
|
\f[C]\-\-vfs\-read\-chunk\-size\f[R] will enable reading the source
|
|
objects in parts.
|
|
This can reduce the used download quota for some remotes by requesting
|
|
only chunks from the remote that are actually read at the cost of an
|
|
increased number of requests.
|
|
.PP
|
|
When \f[C]\-\-vfs\-read\-chunk\-size\-limit\f[R] is also specified and
|
|
greater than \f[C]\-\-vfs\-read\-chunk\-size\f[R], the chunk size for
|
|
each open file will get doubled for each chunk read, until the specified
|
|
value is reached.
|
|
A value of \f[C]\-1\f[R] will disable the limit and the chunk size will
|
|
grow indefinitely.
|
|
.PP
|
|
With \f[C]\-\-vfs\-read\-chunk\-size 100M\f[R] and
|
|
\f[C]\-\-vfs\-read\-chunk\-size\-limit 0\f[R] the following parts will
|
|
be downloaded: 0\-100M, 100M\-200M, 200M\-300M, 300M\-400M and so on.
|
|
When \f[C]\-\-vfs\-read\-chunk\-size\-limit 500M\f[R] is specified, the
|
|
result would be 0\-100M, 100M\-300M, 300M\-700M, 700M\-1200M,
|
|
1200M\-1700M and so on.
|
|
.SS VFS \- Virtual File System
|
|
.PP
|
|
This command uses the VFS layer.
|
|
This adapts the cloud storage objects that rclone uses into something
|
|
which looks much more like a disk filing system.
|
|
.PP
|
|
Cloud storage objects have lots of properties which aren\[aq]t like disk
|
|
files \- you can\[aq]t extend them or write to the middle of them, so
|
|
the VFS layer has to deal with that.
|
|
Because there is no one right way of doing this there are various
|
|
options explained below.
|
|
.PP
|
|
The VFS layer also implements a directory cache \- this caches info
|
|
about files and directories (but not the data) in memory.
|
|
.SS VFS Directory Cache
|
|
.PP
|
|
Using the \f[C]\-\-dir\-cache\-time\f[R] flag, you can control how long
|
|
a directory should be considered up to date and not refreshed from the
|
|
backend.
|
|
Changes made through the mount will appear immediately or invalidate the
|
|
cache.
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
\-\-dir\-cache\-time duration Time to cache directory entries for. (default 5m0s)
|
|
\-\-poll\-interval duration Time to wait between polling for changes.
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
However, changes made directly on the cloud storage by the web interface
|
|
or a different copy of rclone will only be picked up once the directory
|
|
cache expires if the backend configured does not support polling for
|
|
changes.
|
|
If the backend supports polling, changes will be picked up within the
|
|
polling interval.
|
|
.PP
|
|
You can send a \f[C]SIGHUP\f[R] signal to rclone for it to flush all
|
|
directory caches, regardless of how old they are.
|
|
Assuming only one rclone instance is running, you can reset the cache
|
|
like this:
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
kill \-SIGHUP $(pidof rclone)
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
If you configure rclone with a remote control (/rc) then you can use
|
|
rclone rc to flush the whole directory cache:
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone rc vfs/forget
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
Or individual files or directories:
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone rc vfs/forget file=path/to/file dir=path/to/dir
|
|
\f[R]
|
|
.fi
|
|
.SS VFS File Buffering
|
|
.PP
|
|
The \f[C]\-\-buffer\-size\f[R] flag determines the amount of memory,
|
|
that will be used to buffer data in advance.
|
|
.PP
|
|
Each open file will try to keep the specified amount of data in memory
|
|
at all times.
|
|
The buffered data is bound to one open file and won\[aq]t be shared.
|
|
.PP
|
|
This flag is a upper limit for the used memory per open file.
|
|
The buffer will only use memory for data that is downloaded but not not
|
|
yet read.
|
|
If the buffer is empty, only a small amount of memory will be used.
|
|
.PP
|
|
The maximum memory used by rclone for buffering can be up to
|
|
\f[C]\-\-buffer\-size * open files\f[R].
|
|
.SS VFS File Caching
|
|
.PP
|
|
These flags control the VFS file caching options.
|
|
File caching is necessary to make the VFS layer appear compatible with a
|
|
normal file system.
|
|
It can be disabled at the cost of some compatibility.
|
|
.PP
|
|
For example you\[aq]ll need to enable VFS caching if you want to read
|
|
and write simultaneously to a file.
|
|
See below for more details.
|
|
.PP
|
|
Note that the VFS cache is separate from the cache backend and you may
|
|
find that you need one or the other or both.
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
\-\-cache\-dir string Directory rclone will use for caching.
|
|
\-\-vfs\-cache\-mode CacheMode Cache mode off|minimal|writes|full (default off)
|
|
\-\-vfs\-cache\-max\-age duration Max age of objects in the cache. (default 1h0m0s)
|
|
\-\-vfs\-cache\-max\-size SizeSuffix Max total size of objects in the cache. (default off)
|
|
\-\-vfs\-cache\-poll\-interval duration Interval to poll the cache for stale objects. (default 1m0s)
|
|
\-\-vfs\-write\-back duration Time to writeback files after last use when using cache. (default 5s)
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
If run with \f[C]\-vv\f[R] rclone will print the location of the file
|
|
cache.
|
|
The files are stored in the user cache file area which is OS dependent
|
|
but can be controlled with \f[C]\-\-cache\-dir\f[R] or setting the
|
|
appropriate environment variable.
|
|
.PP
|
|
The cache has 4 different modes selected by
|
|
\f[C]\-\-vfs\-cache\-mode\f[R].
|
|
The higher the cache mode the more compatible rclone becomes at the cost
|
|
of using disk space.
|
|
.PP
|
|
Note that files are written back to the remote only when they are closed
|
|
and if they haven\[aq]t been accessed for \-\-vfs\-write\-back second.
|
|
If rclone is quit or dies with files that haven\[aq]t been uploaded,
|
|
these will be uploaded next time rclone is run with the same flags.
|
|
.PP
|
|
If using \f[C]\-\-vfs\-cache\-max\-size\f[R] note that the cache may
|
|
exceed this size for two reasons.
|
|
Firstly because it is only checked every
|
|
\f[C]\-\-vfs\-cache\-poll\-interval\f[R].
|
|
Secondly because open files cannot be evicted from the cache.
|
|
.PP
|
|
You \f[B]should not\f[R] run two copies of rclone using the same VFS
|
|
cache with the same or overlapping remotes if using
|
|
\f[C]\-\-vfs\-cache\-mode > off\f[R].
|
|
This can potentially cause data corruption if you do.
|
|
You can work around this by giving each rclone its own cache hierarchy
|
|
with \f[C]\-\-cache\-dir\f[R].
|
|
You don\[aq]t need to worry about this if the remotes in use don\[aq]t
|
|
overlap.
|
|
.SS \-\-vfs\-cache\-mode off
|
|
.PP
|
|
In this mode (the default) the cache will read directly from the remote
|
|
and write directly to the remote without caching anything on disk.
|
|
.PP
|
|
This will mean some operations are not possible
|
|
.IP \[bu] 2
|
|
Files can\[aq]t be opened for both read AND write
|
|
.IP \[bu] 2
|
|
Files opened for write can\[aq]t be seeked
|
|
.IP \[bu] 2
|
|
Existing files opened for write must have O_TRUNC set
|
|
.IP \[bu] 2
|
|
Files open for read with O_TRUNC will be opened write only
|
|
.IP \[bu] 2
|
|
Files open for write only will behave as if O_TRUNC was supplied
|
|
.IP \[bu] 2
|
|
Open modes O_APPEND, O_TRUNC are ignored
|
|
.IP \[bu] 2
|
|
If an upload fails it can\[aq]t be retried
|
|
.SS \-\-vfs\-cache\-mode minimal
|
|
.PP
|
|
This is very similar to \[dq]off\[dq] except that files opened for read
|
|
AND write will be buffered to disk.
|
|
This means that files opened for write will be a lot more compatible,
|
|
but uses the minimal disk space.
|
|
.PP
|
|
These operations are not possible
|
|
.IP \[bu] 2
|
|
Files opened for write only can\[aq]t be seeked
|
|
.IP \[bu] 2
|
|
Existing files opened for write must have O_TRUNC set
|
|
.IP \[bu] 2
|
|
Files opened for write only will ignore O_APPEND, O_TRUNC
|
|
.IP \[bu] 2
|
|
If an upload fails it can\[aq]t be retried
|
|
.SS \-\-vfs\-cache\-mode writes
|
|
.PP
|
|
In this mode files opened for read only are still read directly from the
|
|
remote, write only and read/write files are buffered to disk first.
|
|
.PP
|
|
This mode should support all normal file system operations.
|
|
.PP
|
|
If an upload fails it will be retried at exponentially increasing
|
|
intervals up to 1 minute.
|
|
.SS \-\-vfs\-cache\-mode full
|
|
.PP
|
|
In this mode all reads and writes are buffered to and from disk.
|
|
When data is read from the remote this is buffered to disk as well.
|
|
.PP
|
|
In this mode the files in the cache will be sparse files and rclone will
|
|
keep track of which bits of the files it has downloaded.
|
|
.PP
|
|
So if an application only reads the starts of each file, then rclone
|
|
will only buffer the start of the file.
|
|
These files will appear to be their full size in the cache, but they
|
|
will be sparse files with only the data that has been downloaded present
|
|
in them.
|
|
.PP
|
|
This mode should support all normal file system operations and is
|
|
otherwise identical to \-\-vfs\-cache\-mode writes.
|
|
.PP
|
|
When reading a file rclone will read \-\-buffer\-size plus
|
|
\-\-vfs\-read\-ahead bytes ahead.
|
|
The \-\-buffer\-size is buffered in memory whereas the
|
|
\-\-vfs\-read\-ahead is buffered on disk.
|
|
.PP
|
|
When using this mode it is recommended that \-\-buffer\-size is not set
|
|
too big and \-\-vfs\-read\-ahead is set large if required.
|
|
.PP
|
|
\f[B]IMPORTANT\f[R] not all file systems support sparse files.
|
|
In particular FAT/exFAT do not.
|
|
Rclone will perform very badly if the cache directory is on a filesystem
|
|
which doesn\[aq]t support sparse files and it will log an ERROR message
|
|
if one is detected.
|
|
.SS VFS Performance
|
|
.PP
|
|
These flags may be used to enable/disable features of the VFS for
|
|
performance or other reasons.
|
|
.PP
|
|
In particular S3 and Swift benefit hugely from the \-\-no\-modtime flag
|
|
(or use \-\-use\-server\-modtime for a slightly different effect) as
|
|
each read of the modification time takes a transaction.
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
\-\-no\-checksum Don\[aq]t compare checksums on up/download.
|
|
\-\-no\-modtime Don\[aq]t read/write the modification time (can speed things up).
|
|
\-\-no\-seek Don\[aq]t allow seeking in files.
|
|
\-\-read\-only Mount read\-only.
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
When rclone reads files from a remote it reads them in chunks.
|
|
This means that rather than requesting the whole file rclone reads the
|
|
chunk specified.
|
|
This is advantageous because some cloud providers account for reads
|
|
being all the data requested, not all the data delivered.
|
|
.PP
|
|
Rclone will keep doubling the chunk size requested starting at
|
|
\-\-vfs\-read\-chunk\-size with a maximum of
|
|
\-\-vfs\-read\-chunk\-size\-limit unless it is set to \[dq]off\[dq] in
|
|
which case there will be no limit.
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
\-\-vfs\-read\-chunk\-size SizeSuffix Read the source objects in chunks. (default 128M)
|
|
\-\-vfs\-read\-chunk\-size\-limit SizeSuffix Max chunk doubling size (default \[dq]off\[dq])
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
Sometimes rclone is delivered reads or writes out of order.
|
|
Rather than seeking rclone will wait a short time for the in sequence
|
|
read or write to come in.
|
|
These flags only come into effect when not using an on disk cache file.
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
\-\-vfs\-read\-wait duration Time to wait for in\-sequence read before seeking. (default 20ms)
|
|
\-\-vfs\-write\-wait duration Time to wait for in\-sequence write before giving error. (default 1s)
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
When using VFS write caching (\-\-vfs\-cache\-mode with value writes or
|
|
full), the global flag \-\-transfers can be set to adjust the number of
|
|
parallel uploads of modified files from cache (the related global flag
|
|
\-\-checkers have no effect on mount).
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
\-\-transfers int Number of file transfers to run in parallel. (default 4)
|
|
\f[R]
|
|
.fi
|
|
.SS VFS Case Sensitivity
|
|
.PP
|
|
Linux file systems are case\-sensitive: two files can differ only by
|
|
case, and the exact case must be used when opening a file.
|
|
.PP
|
|
File systems in modern Windows are case\-insensitive but
|
|
case\-preserving: although existing files can be opened using any case,
|
|
the exact case used to create the file is preserved and available for
|
|
programs to query.
|
|
It is not allowed for two files in the same directory to differ only by
|
|
case.
|
|
.PP
|
|
Usually file systems on macOS are case\-insensitive.
|
|
It is possible to make macOS file systems case\-sensitive but that is
|
|
not the default
|
|
.PP
|
|
The \f[C]\-\-vfs\-case\-insensitive\f[R] mount flag controls how rclone
|
|
handles these two cases.
|
|
If its value is \[dq]false\[dq], rclone passes file names to the mounted
|
|
file system as\-is.
|
|
If the flag is \[dq]true\[dq] (or appears without a value on command
|
|
line), rclone may perform a \[dq]fixup\[dq] as explained below.
|
|
.PP
|
|
The user may specify a file name to open/delete/rename/etc with a case
|
|
different than what is stored on mounted file system.
|
|
If an argument refers to an existing file with exactly the same name,
|
|
then the case of the existing file on the disk will be used.
|
|
However, if a file name with exactly the same name is not found but a
|
|
name differing only by case exists, rclone will transparently fixup the
|
|
name.
|
|
This fixup happens only when an existing file is requested.
|
|
Case sensitivity of file names created anew by rclone is controlled by
|
|
an underlying mounted file system.
|
|
.PP
|
|
Note that case sensitivity of the operating system running rclone (the
|
|
target) may differ from case sensitivity of a file system mounted by
|
|
rclone (the source).
|
|
The flag controls whether \[dq]fixup\[dq] is performed to satisfy the
|
|
target.
|
|
.PP
|
|
If the flag is not provided on the command line, then its default value
|
|
depends on the operating system where rclone runs: \[dq]true\[dq] on
|
|
Windows and macOS, \[dq]false\[dq] otherwise.
|
|
If the flag is provided without a value, then it is \[dq]true\[dq].
|
|
.SS Alternate report of used bytes
|
|
.PP
|
|
Some backends, most notably S3, do not report the amount of bytes used.
|
|
If you need this information to be available when running \f[C]df\f[R]
|
|
on the filesystem, then pass the flag \f[C]\-\-vfs\-used\-is\-size\f[R]
|
|
to rclone.
|
|
With this flag set, instead of relying on the backend to report this
|
|
information, rclone will scan the whole remote similar to
|
|
\f[C]rclone size\f[R] and compute the total used space itself.
|
|
.PP
|
|
\f[I]WARNING.\f[R] Contrary to \f[C]rclone size\f[R], this flag ignores
|
|
filters so that the result is accurate.
|
|
However, this is very inefficient and may cost lots of API calls
|
|
resulting in extra charges.
|
|
Use it as a last resort and only with caching.
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone mount remote:path /path/to/mountpoint [flags]
|
|
\f[R]
|
|
.fi
|
|
.SS Options
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
\-\-allow\-non\-empty Allow mounting over a non\-empty directory. Not supported on Windows.
|
|
\-\-allow\-other Allow access to other users. Not supported on Windows.
|
|
\-\-allow\-root Allow access to root user. Not supported on Windows.
|
|
\-\-async\-read Use asynchronous reads. Not supported on Windows. (default true)
|
|
\-\-attr\-timeout duration Time for which file/directory attributes are cached. (default 1s)
|
|
\-\-daemon Run mount as a daemon (background mode). Not supported on Windows.
|
|
\-\-daemon\-timeout duration Time limit for rclone to respond to kernel. Not supported on Windows.
|
|
\-\-debug\-fuse Debug the FUSE internals \- needs \-v.
|
|
\-\-default\-permissions Makes kernel enforce access control based on the file mode. Not supported on Windows.
|
|
\-\-dir\-cache\-time duration Time to cache directory entries for. (default 5m0s)
|
|
\-\-dir\-perms FileMode Directory permissions (default 0777)
|
|
\-\-file\-perms FileMode File permissions (default 0666)
|
|
\-\-fuse\-flag stringArray Flags or arguments to be passed direct to libfuse/WinFsp. Repeat if required.
|
|
\-\-gid uint32 Override the gid field set by the filesystem. Not supported on Windows. (default 1000)
|
|
\-h, \-\-help help for mount
|
|
\-\-max\-read\-ahead SizeSuffix The number of bytes that can be prefetched for sequential reads. Not supported on Windows. (default 128k)
|
|
\-\-network\-mode Mount as remote network drive, instead of fixed disk drive. Supported on Windows only
|
|
\-\-no\-checksum Don\[aq]t compare checksums on up/download.
|
|
\-\-no\-modtime Don\[aq]t read/write the modification time (can speed things up).
|
|
\-\-no\-seek Don\[aq]t allow seeking in files.
|
|
\-\-noappledouble Ignore Apple Double (._) and .DS_Store files. Supported on OSX only. (default true)
|
|
\-\-noapplexattr Ignore all \[dq]com.apple.*\[dq] extended attributes. Supported on OSX only.
|
|
\-o, \-\-option stringArray Option for libfuse/WinFsp. Repeat if required.
|
|
\-\-poll\-interval duration Time to wait between polling for changes. Must be smaller than dir\-cache\-time. Only on supported remotes. Set to 0 to disable. (default 1m0s)
|
|
\-\-read\-only Mount read\-only.
|
|
\-\-uid uint32 Override the uid field set by the filesystem. Not supported on Windows. (default 1000)
|
|
\-\-umask int Override the permission bits set by the filesystem. Not supported on Windows.
|
|
\-\-vfs\-cache\-max\-age duration Max age of objects in the cache. (default 1h0m0s)
|
|
\-\-vfs\-cache\-max\-size SizeSuffix Max total size of objects in the cache. (default off)
|
|
\-\-vfs\-cache\-mode CacheMode Cache mode off|minimal|writes|full (default off)
|
|
\-\-vfs\-cache\-poll\-interval duration Interval to poll the cache for stale objects. (default 1m0s)
|
|
\-\-vfs\-case\-insensitive If a file name not found, find a case insensitive match.
|
|
\-\-vfs\-read\-ahead SizeSuffix Extra read ahead over \-\-buffer\-size when using cache\-mode full.
|
|
\-\-vfs\-read\-chunk\-size SizeSuffix Read the source objects in chunks. (default 128M)
|
|
\-\-vfs\-read\-chunk\-size\-limit SizeSuffix If greater than \-\-vfs\-read\-chunk\-size, double the chunk size after each chunk read, until the limit is reached. \[aq]off\[aq] is unlimited. (default off)
|
|
\-\-vfs\-read\-wait duration Time to wait for in\-sequence read before seeking. (default 20ms)
|
|
\-\-vfs\-used\-is\-size rclone size Use the rclone size algorithm for Used size.
|
|
\-\-vfs\-write\-back duration Time to writeback files after last use when using cache. (default 5s)
|
|
\-\-vfs\-write\-wait duration Time to wait for in\-sequence write before giving error. (default 1s)
|
|
\-\-volname string Set the volume name. Supported on Windows and OSX only.
|
|
\-\-write\-back\-cache Makes kernel buffer writes before sending them to rclone. Without this, writethrough caching is used. Not supported on Windows.
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
See the global flags page (https://rclone.org/flags/) for global options
|
|
not listed here.
|
|
.SS SEE ALSO
|
|
.IP \[bu] 2
|
|
rclone (https://rclone.org/commands/rclone/) \- Show help for rclone
|
|
commands, flags and backends.
|
|
.SH rclone moveto
|
|
.PP
|
|
Move file or directory from source to dest.
|
|
.SS Synopsis
|
|
.PP
|
|
If source:path is a file or directory then it moves it to a file or
|
|
directory named dest:path.
|
|
.PP
|
|
This can be used to rename files or upload single files to other than
|
|
their existing name.
|
|
If the source is a directory then it acts exactly like the move command.
|
|
.PP
|
|
So
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone moveto src dst
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
where src and dst are rclone paths, either remote:path or /path/to/local
|
|
or C:.
|
|
.PP
|
|
This will:
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
if src is file
|
|
move it to dst, overwriting an existing file if it exists
|
|
if src is directory
|
|
move it to dst, overwriting existing files if they exist
|
|
see move command for full details
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
This doesn\[aq]t transfer unchanged files, testing by size and
|
|
modification time or MD5SUM.
|
|
src will be deleted on successful transfer.
|
|
.PP
|
|
\f[B]Important\f[R]: Since this can cause data loss, test first with the
|
|
\f[C]\-\-dry\-run\f[R] or the \f[C]\-\-interactive\f[R]/\f[C]\-i\f[R]
|
|
flag.
|
|
.PP
|
|
\f[B]Note\f[R]: Use the \f[C]\-P\f[R]/\f[C]\-\-progress\f[R] flag to
|
|
view real\-time transfer statistics.
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone moveto source:path dest:path [flags]
|
|
\f[R]
|
|
.fi
|
|
.SS Options
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
\-h, \-\-help help for moveto
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
See the global flags page (https://rclone.org/flags/) for global options
|
|
not listed here.
|
|
.SS SEE ALSO
|
|
.IP \[bu] 2
|
|
rclone (https://rclone.org/commands/rclone/) \- Show help for rclone
|
|
commands, flags and backends.
|
|
.SH rclone ncdu
|
|
.PP
|
|
Explore a remote with a text based user interface.
|
|
.SS Synopsis
|
|
.PP
|
|
This displays a text based user interface allowing the navigation of a
|
|
remote.
|
|
It is most useful for answering the question \- \[dq]What is using all
|
|
my disk space?\[dq].
|
|
.PP
|
|
To make the user interface it first scans the entire remote given and
|
|
builds an in memory representation.
|
|
rclone ncdu can be used during this scanning phase and you will see it
|
|
building up the directory structure as it goes along.
|
|
.PP
|
|
Here are the keys \- press \[aq]?\[aq] to toggle the help on and off
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
\[ua],\[da] or k,j to Move
|
|
\[->],l to enter
|
|
\[<-],h to return
|
|
c toggle counts
|
|
g toggle graph
|
|
a toggle average size in directory
|
|
n,s,C,A sort by name,size,count,average size
|
|
d delete file/directory
|
|
y copy current path to clipboard
|
|
Y display current path
|
|
\[ha]L refresh screen
|
|
? to toggle help on and off
|
|
q/ESC/c\-C to quit
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
This an homage to the ncdu tool (https://dev.yorhel.nl/ncdu) but for
|
|
rclone remotes.
|
|
It is missing lots of features at the moment but is useful as it stands.
|
|
.PP
|
|
Note that it might take some time to delete big files/folders.
|
|
The UI won\[aq]t respond in the meantime since the deletion is done
|
|
synchronously.
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone ncdu remote:path [flags]
|
|
\f[R]
|
|
.fi
|
|
.SS Options
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
\-h, \-\-help help for ncdu
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
See the global flags page (https://rclone.org/flags/) for global options
|
|
not listed here.
|
|
.SS SEE ALSO
|
|
.IP \[bu] 2
|
|
rclone (https://rclone.org/commands/rclone/) \- Show help for rclone
|
|
commands, flags and backends.
|
|
.SH rclone obscure
|
|
.PP
|
|
Obscure password for use in the rclone config file.
|
|
.SS Synopsis
|
|
.PP
|
|
In the rclone config file, human readable passwords are obscured.
|
|
Obscuring them is done by encrypting them and writing them out in
|
|
base64.
|
|
This is \f[B]not\f[R] a secure way of encrypting these passwords as
|
|
rclone can decrypt them \- it is to prevent \[dq]eyedropping\[dq] \-
|
|
namely someone seeing a password in the rclone config file by accident.
|
|
.PP
|
|
Many equally important things (like access tokens) are not obscured in
|
|
the config file.
|
|
However it is very hard to shoulder surf a 64 character hex token.
|
|
.PP
|
|
This command can also accept a password through STDIN instead of an
|
|
argument by passing a hyphen as an argument.
|
|
This will use the first line of STDIN as the password not including the
|
|
trailing newline.
|
|
.PP
|
|
echo \[dq]secretpassword\[dq] | rclone obscure \-
|
|
.PP
|
|
If there is no data on STDIN to read, rclone obscure will default to
|
|
obfuscating the hyphen itself.
|
|
.PP
|
|
If you want to encrypt the config file then please use config file
|
|
encryption \- see rclone
|
|
config (https://rclone.org/commands/rclone_config/) for more info.
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone obscure password [flags]
|
|
\f[R]
|
|
.fi
|
|
.SS Options
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
\-h, \-\-help help for obscure
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
See the global flags page (https://rclone.org/flags/) for global options
|
|
not listed here.
|
|
.SS SEE ALSO
|
|
.IP \[bu] 2
|
|
rclone (https://rclone.org/commands/rclone/) \- Show help for rclone
|
|
commands, flags and backends.
|
|
.SH rclone rc
|
|
.PP
|
|
Run a command against a running rclone.
|
|
.SS Synopsis
|
|
.PP
|
|
This runs a command against a running rclone.
|
|
Use the \-\-url flag to specify an non default URL to connect on.
|
|
This can be either a \[dq]:port\[dq] which is taken to mean
|
|
\[dq]http://localhost:port\[dq] or a \[dq]host:port\[dq] which is taken
|
|
to mean \[dq]http://host:port\[dq]
|
|
.PP
|
|
A username and password can be passed in with \-\-user and \-\-pass.
|
|
.PP
|
|
Note that \-\-rc\-addr, \-\-rc\-user, \-\-rc\-pass will be read also for
|
|
\-\-url, \-\-user, \-\-pass.
|
|
.PP
|
|
Arguments should be passed in as parameter=value.
|
|
.PP
|
|
The result will be returned as a JSON object by default.
|
|
.PP
|
|
The \-\-json parameter can be used to pass in a JSON blob as an input
|
|
instead of key=value arguments.
|
|
This is the only way of passing in more complicated values.
|
|
.PP
|
|
The \-o/\-\-opt option can be used to set a key \[dq]opt\[dq] with key,
|
|
value options in the form \[dq]\-o key=value\[dq] or \[dq]\-o key\[dq].
|
|
It can be repeated as many times as required.
|
|
This is useful for rc commands which take the \[dq]opt\[dq] parameter
|
|
which by convention is a dictionary of strings.
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
\-o key=value \-o key2
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
Will place this in the \[dq]opt\[dq] value
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
{\[dq]key\[dq]:\[dq]value\[dq], \[dq]key2\[dq],\[dq]\[dq])
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
The \-a/\-\-arg option can be used to set strings in the \[dq]arg\[dq]
|
|
value.
|
|
It can be repeated as many times as required.
|
|
This is useful for rc commands which take the \[dq]arg\[dq] parameter
|
|
which by convention is a list of strings.
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
\-a value \-a value2
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
Will place this in the \[dq]arg\[dq] value
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
[\[dq]value\[dq], \[dq]value2\[dq]]
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
Use \-\-loopback to connect to the rclone instance running \[dq]rclone
|
|
rc\[dq].
|
|
This is very useful for testing commands without having to run an rclone
|
|
rc server, e.g.:
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone rc \-\-loopback operations/about fs=/
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
Use \[dq]rclone rc\[dq] to see a list of all possible commands.
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone rc commands parameter [flags]
|
|
\f[R]
|
|
.fi
|
|
.SS Options
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
\-a, \-\-arg stringArray Argument placed in the \[dq]arg\[dq] array.
|
|
\-h, \-\-help help for rc
|
|
\-\-json string Input JSON \- use instead of key=value args.
|
|
\-\-loopback If set connect to this rclone instance not via HTTP.
|
|
\-\-no\-output If set, don\[aq]t output the JSON result.
|
|
\-o, \-\-opt stringArray Option in the form name=value or name placed in the \[dq]opt\[dq] array.
|
|
\-\-pass string Password to use to connect to rclone remote control.
|
|
\-\-url string URL to connect to rclone remote control. (default \[dq]http://localhost:5572/\[dq])
|
|
\-\-user string Username to use to rclone remote control.
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
See the global flags page (https://rclone.org/flags/) for global options
|
|
not listed here.
|
|
.SS SEE ALSO
|
|
.IP \[bu] 2
|
|
rclone (https://rclone.org/commands/rclone/) \- Show help for rclone
|
|
commands, flags and backends.
|
|
.SH rclone rcat
|
|
.PP
|
|
Copies standard input to file on remote.
|
|
.SS Synopsis
|
|
.PP
|
|
rclone rcat reads from standard input (stdin) and copies it to a single
|
|
remote file.
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
echo \[dq]hello world\[dq] | rclone rcat remote:path/to/file
|
|
ffmpeg \- | rclone rcat remote:path/to/file
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
If the remote file already exists, it will be overwritten.
|
|
.PP
|
|
rcat will try to upload small files in a single request, which is
|
|
usually more efficient than the streaming/chunked upload endpoints,
|
|
which use multiple requests.
|
|
Exact behaviour depends on the remote.
|
|
What is considered a small file may be set through
|
|
\f[C]\-\-streaming\-upload\-cutoff\f[R].
|
|
Uploading only starts after the cutoff is reached or if the file ends
|
|
before that.
|
|
The data must fit into RAM.
|
|
The cutoff needs to be small enough to adhere the limits of your remote,
|
|
please see there.
|
|
Generally speaking, setting this cutoff too high will decrease your
|
|
performance.
|
|
.PP
|
|
Note that the upload can also not be retried because the data is not
|
|
kept around until the upload succeeds.
|
|
If you need to transfer a lot of data, you\[aq]re better off caching
|
|
locally and then \f[C]rclone move\f[R] it to the destination.
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone rcat remote:path [flags]
|
|
\f[R]
|
|
.fi
|
|
.SS Options
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
\-h, \-\-help help for rcat
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
See the global flags page (https://rclone.org/flags/) for global options
|
|
not listed here.
|
|
.SS SEE ALSO
|
|
.IP \[bu] 2
|
|
rclone (https://rclone.org/commands/rclone/) \- Show help for rclone
|
|
commands, flags and backends.
|
|
.SH rclone rcd
|
|
.PP
|
|
Run rclone listening to remote control commands only.
|
|
.SS Synopsis
|
|
.PP
|
|
This runs rclone so that it only listens to remote control commands.
|
|
.PP
|
|
This is useful if you are controlling rclone via the rc API.
|
|
.PP
|
|
If you pass in a path to a directory, rclone will serve that directory
|
|
for GET requests on the URL passed in.
|
|
It will also open the URL in the browser when rclone is run.
|
|
.PP
|
|
See the rc documentation (https://rclone.org/rc/) for more info on the
|
|
rc flags.
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone rcd <path to files to serve>* [flags]
|
|
\f[R]
|
|
.fi
|
|
.SS Options
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
\-h, \-\-help help for rcd
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
See the global flags page (https://rclone.org/flags/) for global options
|
|
not listed here.
|
|
.SS SEE ALSO
|
|
.IP \[bu] 2
|
|
rclone (https://rclone.org/commands/rclone/) \- Show help for rclone
|
|
commands, flags and backends.
|
|
.SH rclone rmdirs
|
|
.PP
|
|
Remove empty directories under the path.
|
|
.SS Synopsis
|
|
.PP
|
|
This recursively removes any empty directories (including directories
|
|
that only contain empty directories), that it finds under the path.
|
|
The root path itself will also be removed if it is empty, unless you
|
|
supply the \f[C]\-\-leave\-root\f[R] flag.
|
|
.PP
|
|
Use command \f[C]rmdir\f[R] to delete just the empty directory given by
|
|
path, not recurse.
|
|
.PP
|
|
This is useful for tidying up remotes that rclone has left a lot of
|
|
empty directories in.
|
|
For example the \f[C]delete\f[R] command will delete files but leave the
|
|
directory structure (unless used with option \f[C]\-\-rmdirs\f[R]).
|
|
.PP
|
|
To delete a path and any objects in it, use \f[C]purge\f[R] command.
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone rmdirs remote:path [flags]
|
|
\f[R]
|
|
.fi
|
|
.SS Options
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
\-h, \-\-help help for rmdirs
|
|
\-\-leave\-root Do not remove root directory if empty
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
See the global flags page (https://rclone.org/flags/) for global options
|
|
not listed here.
|
|
.SS SEE ALSO
|
|
.IP \[bu] 2
|
|
rclone (https://rclone.org/commands/rclone/) \- Show help for rclone
|
|
commands, flags and backends.
|
|
.SH rclone selfupdate
|
|
.PP
|
|
Update the rclone binary.
|
|
.SS Synopsis
|
|
.PP
|
|
This command downloads the latest release of rclone and replaces the
|
|
currently running binary.
|
|
The download is verified with a hashsum and cryptographically signed
|
|
signature.
|
|
.PP
|
|
If used without flags (or with implied \f[C]\-\-stable\f[R] flag), this
|
|
command will install the latest stable release.
|
|
However, some issues may be fixed (or features added) only in the latest
|
|
beta release.
|
|
In such cases you should run the command with the \f[C]\-\-beta\f[R]
|
|
flag, i.e.
|
|
\f[C]rclone selfupdate \-\-beta\f[R].
|
|
You can check in advance what version would be installed by adding the
|
|
\f[C]\-\-check\f[R] flag, then repeat the command without it when you
|
|
are satisfied.
|
|
.PP
|
|
Sometimes the rclone team may recommend you a concrete beta or stable
|
|
rclone release to troubleshoot your issue or add a bleeding edge
|
|
feature.
|
|
The \f[C]\-\-version VER\f[R] flag, if given, will update to the
|
|
concrete version instead of the latest one.
|
|
If you omit micro version from \f[C]VER\f[R] (for example
|
|
\f[C]1.53\f[R]), the latest matching micro version will be used.
|
|
.PP
|
|
Upon successful update rclone will print a message that contains a
|
|
previous version number.
|
|
You will need it if you later decide to revert your update for some
|
|
reason.
|
|
Then you\[aq]ll have to note the previous version and run the following
|
|
command: \f[C]rclone selfupdate [\-\-beta] OLDVER\f[R].
|
|
If the old version contains only dots and digits (for example
|
|
\f[C]v1.54.0\f[R]) then it\[aq]s a stable release so you won\[aq]t need
|
|
the \f[C]\-\-beta\f[R] flag.
|
|
Beta releases have an additional information similar to
|
|
\f[C]v1.54.0\-beta.5111.06f1c0c61\f[R].
|
|
(if you are a developer and use a locally built rclone, the version
|
|
number will end with \f[C]\-DEV\f[R], you will have to rebuild it as it
|
|
obvisously can\[aq]t be distributed).
|
|
.PP
|
|
If you previously installed rclone via a package manager, the package
|
|
may include local documentation or configure services.
|
|
You may wish to update with the flag \f[C]\-\-package deb\f[R] or
|
|
\f[C]\-\-package rpm\f[R] (whichever is correct for your OS) to update
|
|
these too.
|
|
This command with the default \f[C]\-\-package zip\f[R] will update only
|
|
the rclone executable so the local manual may become inaccurate after
|
|
it.
|
|
.PP
|
|
The \f[C]rclone mount\f[R] command
|
|
(https://rclone.org/commands/rclone_mount/) may or may not support
|
|
extended FUSE options depending on the build and OS.
|
|
\f[C]selfupdate\f[R] will refuse to update if the capability would be
|
|
discarded.
|
|
.PP
|
|
Note: Windows forbids deletion of a currently running executable so this
|
|
command will rename the old executable to \[aq]rclone.old.exe\[aq] upon
|
|
success.
|
|
.PP
|
|
Please note that this command was not available before rclone version
|
|
1.55.
|
|
If it fails for you with the message
|
|
\f[C]unknown command \[dq]selfupdate\[dq]\f[R] then you will need to
|
|
update manually following the install instructions located at
|
|
https://rclone.org/install/
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone selfupdate [flags]
|
|
\f[R]
|
|
.fi
|
|
.SS Options
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
\-\-beta Install beta release.
|
|
\-\-check Check for latest release, do not download.
|
|
\-h, \-\-help help for selfupdate
|
|
\-\-output string Save the downloaded binary at a given path (default: replace running binary)
|
|
\-\-package string Package format: zip|deb|rpm (default: zip)
|
|
\-\-stable Install stable release (this is the default)
|
|
\-\-version string Install the given rclone version (default: latest)
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
See the global flags page (https://rclone.org/flags/) for global options
|
|
not listed here.
|
|
.SS SEE ALSO
|
|
.IP \[bu] 2
|
|
rclone (https://rclone.org/commands/rclone/) \- Show help for rclone
|
|
commands, flags and backends.
|
|
.SH rclone serve
|
|
.PP
|
|
Serve a remote over a protocol.
|
|
.SS Synopsis
|
|
.PP
|
|
rclone serve is used to serve a remote over a given protocol.
|
|
This command requires the use of a subcommand to specify the protocol,
|
|
e.g.
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone serve http remote:
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
Each subcommand has its own options which you can see in their help.
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone serve <protocol> [opts] <remote> [flags]
|
|
\f[R]
|
|
.fi
|
|
.SS Options
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
\-h, \-\-help help for serve
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
See the global flags page (https://rclone.org/flags/) for global options
|
|
not listed here.
|
|
.SS SEE ALSO
|
|
.IP \[bu] 2
|
|
rclone (https://rclone.org/commands/rclone/) \- Show help for rclone
|
|
commands, flags and backends.
|
|
.IP \[bu] 2
|
|
rclone serve dlna (https://rclone.org/commands/rclone_serve_dlna/) \-
|
|
Serve remote:path over DLNA
|
|
.IP \[bu] 2
|
|
rclone serve ftp (https://rclone.org/commands/rclone_serve_ftp/) \-
|
|
Serve remote:path over FTP.
|
|
.IP \[bu] 2
|
|
rclone serve http (https://rclone.org/commands/rclone_serve_http/) \-
|
|
Serve the remote over HTTP.
|
|
.IP \[bu] 2
|
|
rclone serve restic (https://rclone.org/commands/rclone_serve_restic/)
|
|
\- Serve the remote for restic\[aq]s REST API.
|
|
.IP \[bu] 2
|
|
rclone serve sftp (https://rclone.org/commands/rclone_serve_sftp/) \-
|
|
Serve the remote over SFTP.
|
|
.IP \[bu] 2
|
|
rclone serve webdav (https://rclone.org/commands/rclone_serve_webdav/)
|
|
\- Serve remote:path over webdav.
|
|
.SH rclone serve dlna
|
|
.PP
|
|
Serve remote:path over DLNA
|
|
.SS Synopsis
|
|
.PP
|
|
rclone serve dlna is a DLNA media server for media stored in an rclone
|
|
remote.
|
|
Many devices, such as the Xbox and PlayStation, can automatically
|
|
discover this server in the LAN and play audio/video from it.
|
|
VLC is also supported.
|
|
Service discovery uses UDP multicast packets (SSDP) and will thus only
|
|
work on LANs.
|
|
.PP
|
|
Rclone will list all files present in the remote, without filtering
|
|
based on media formats or file extensions.
|
|
Additionally, there is no media transcoding support.
|
|
This means that some players might show files that they are not able to
|
|
play back correctly.
|
|
.SS Server options
|
|
.PP
|
|
Use \f[C]\-\-addr\f[R] to specify which IP address and port the server
|
|
should listen on, e.g.
|
|
\f[C]\-\-addr 1.2.3.4:8000\f[R] or \f[C]\-\-addr :8080\f[R] to listen to
|
|
all IPs.
|
|
.PP
|
|
Use \f[C]\-\-name\f[R] to choose the friendly server name, which is by
|
|
default \[dq]rclone (hostname)\[dq].
|
|
.PP
|
|
Use \f[C]\-\-log\-trace\f[R] in conjunction with \f[C]\-vv\f[R] to
|
|
enable additional debug logging of all UPNP traffic.
|
|
.SS VFS \- Virtual File System
|
|
.PP
|
|
This command uses the VFS layer.
|
|
This adapts the cloud storage objects that rclone uses into something
|
|
which looks much more like a disk filing system.
|
|
.PP
|
|
Cloud storage objects have lots of properties which aren\[aq]t like disk
|
|
files \- you can\[aq]t extend them or write to the middle of them, so
|
|
the VFS layer has to deal with that.
|
|
Because there is no one right way of doing this there are various
|
|
options explained below.
|
|
.PP
|
|
The VFS layer also implements a directory cache \- this caches info
|
|
about files and directories (but not the data) in memory.
|
|
.SS VFS Directory Cache
|
|
.PP
|
|
Using the \f[C]\-\-dir\-cache\-time\f[R] flag, you can control how long
|
|
a directory should be considered up to date and not refreshed from the
|
|
backend.
|
|
Changes made through the mount will appear immediately or invalidate the
|
|
cache.
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
\-\-dir\-cache\-time duration Time to cache directory entries for. (default 5m0s)
|
|
\-\-poll\-interval duration Time to wait between polling for changes.
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
However, changes made directly on the cloud storage by the web interface
|
|
or a different copy of rclone will only be picked up once the directory
|
|
cache expires if the backend configured does not support polling for
|
|
changes.
|
|
If the backend supports polling, changes will be picked up within the
|
|
polling interval.
|
|
.PP
|
|
You can send a \f[C]SIGHUP\f[R] signal to rclone for it to flush all
|
|
directory caches, regardless of how old they are.
|
|
Assuming only one rclone instance is running, you can reset the cache
|
|
like this:
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
kill \-SIGHUP $(pidof rclone)
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
If you configure rclone with a remote control (/rc) then you can use
|
|
rclone rc to flush the whole directory cache:
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone rc vfs/forget
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
Or individual files or directories:
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone rc vfs/forget file=path/to/file dir=path/to/dir
|
|
\f[R]
|
|
.fi
|
|
.SS VFS File Buffering
|
|
.PP
|
|
The \f[C]\-\-buffer\-size\f[R] flag determines the amount of memory,
|
|
that will be used to buffer data in advance.
|
|
.PP
|
|
Each open file will try to keep the specified amount of data in memory
|
|
at all times.
|
|
The buffered data is bound to one open file and won\[aq]t be shared.
|
|
.PP
|
|
This flag is a upper limit for the used memory per open file.
|
|
The buffer will only use memory for data that is downloaded but not not
|
|
yet read.
|
|
If the buffer is empty, only a small amount of memory will be used.
|
|
.PP
|
|
The maximum memory used by rclone for buffering can be up to
|
|
\f[C]\-\-buffer\-size * open files\f[R].
|
|
.SS VFS File Caching
|
|
.PP
|
|
These flags control the VFS file caching options.
|
|
File caching is necessary to make the VFS layer appear compatible with a
|
|
normal file system.
|
|
It can be disabled at the cost of some compatibility.
|
|
.PP
|
|
For example you\[aq]ll need to enable VFS caching if you want to read
|
|
and write simultaneously to a file.
|
|
See below for more details.
|
|
.PP
|
|
Note that the VFS cache is separate from the cache backend and you may
|
|
find that you need one or the other or both.
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
\-\-cache\-dir string Directory rclone will use for caching.
|
|
\-\-vfs\-cache\-mode CacheMode Cache mode off|minimal|writes|full (default off)
|
|
\-\-vfs\-cache\-max\-age duration Max age of objects in the cache. (default 1h0m0s)
|
|
\-\-vfs\-cache\-max\-size SizeSuffix Max total size of objects in the cache. (default off)
|
|
\-\-vfs\-cache\-poll\-interval duration Interval to poll the cache for stale objects. (default 1m0s)
|
|
\-\-vfs\-write\-back duration Time to writeback files after last use when using cache. (default 5s)
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
If run with \f[C]\-vv\f[R] rclone will print the location of the file
|
|
cache.
|
|
The files are stored in the user cache file area which is OS dependent
|
|
but can be controlled with \f[C]\-\-cache\-dir\f[R] or setting the
|
|
appropriate environment variable.
|
|
.PP
|
|
The cache has 4 different modes selected by
|
|
\f[C]\-\-vfs\-cache\-mode\f[R].
|
|
The higher the cache mode the more compatible rclone becomes at the cost
|
|
of using disk space.
|
|
.PP
|
|
Note that files are written back to the remote only when they are closed
|
|
and if they haven\[aq]t been accessed for \-\-vfs\-write\-back second.
|
|
If rclone is quit or dies with files that haven\[aq]t been uploaded,
|
|
these will be uploaded next time rclone is run with the same flags.
|
|
.PP
|
|
If using \f[C]\-\-vfs\-cache\-max\-size\f[R] note that the cache may
|
|
exceed this size for two reasons.
|
|
Firstly because it is only checked every
|
|
\f[C]\-\-vfs\-cache\-poll\-interval\f[R].
|
|
Secondly because open files cannot be evicted from the cache.
|
|
.PP
|
|
You \f[B]should not\f[R] run two copies of rclone using the same VFS
|
|
cache with the same or overlapping remotes if using
|
|
\f[C]\-\-vfs\-cache\-mode > off\f[R].
|
|
This can potentially cause data corruption if you do.
|
|
You can work around this by giving each rclone its own cache hierarchy
|
|
with \f[C]\-\-cache\-dir\f[R].
|
|
You don\[aq]t need to worry about this if the remotes in use don\[aq]t
|
|
overlap.
|
|
.SS \-\-vfs\-cache\-mode off
|
|
.PP
|
|
In this mode (the default) the cache will read directly from the remote
|
|
and write directly to the remote without caching anything on disk.
|
|
.PP
|
|
This will mean some operations are not possible
|
|
.IP \[bu] 2
|
|
Files can\[aq]t be opened for both read AND write
|
|
.IP \[bu] 2
|
|
Files opened for write can\[aq]t be seeked
|
|
.IP \[bu] 2
|
|
Existing files opened for write must have O_TRUNC set
|
|
.IP \[bu] 2
|
|
Files open for read with O_TRUNC will be opened write only
|
|
.IP \[bu] 2
|
|
Files open for write only will behave as if O_TRUNC was supplied
|
|
.IP \[bu] 2
|
|
Open modes O_APPEND, O_TRUNC are ignored
|
|
.IP \[bu] 2
|
|
If an upload fails it can\[aq]t be retried
|
|
.SS \-\-vfs\-cache\-mode minimal
|
|
.PP
|
|
This is very similar to \[dq]off\[dq] except that files opened for read
|
|
AND write will be buffered to disk.
|
|
This means that files opened for write will be a lot more compatible,
|
|
but uses the minimal disk space.
|
|
.PP
|
|
These operations are not possible
|
|
.IP \[bu] 2
|
|
Files opened for write only can\[aq]t be seeked
|
|
.IP \[bu] 2
|
|
Existing files opened for write must have O_TRUNC set
|
|
.IP \[bu] 2
|
|
Files opened for write only will ignore O_APPEND, O_TRUNC
|
|
.IP \[bu] 2
|
|
If an upload fails it can\[aq]t be retried
|
|
.SS \-\-vfs\-cache\-mode writes
|
|
.PP
|
|
In this mode files opened for read only are still read directly from the
|
|
remote, write only and read/write files are buffered to disk first.
|
|
.PP
|
|
This mode should support all normal file system operations.
|
|
.PP
|
|
If an upload fails it will be retried at exponentially increasing
|
|
intervals up to 1 minute.
|
|
.SS \-\-vfs\-cache\-mode full
|
|
.PP
|
|
In this mode all reads and writes are buffered to and from disk.
|
|
When data is read from the remote this is buffered to disk as well.
|
|
.PP
|
|
In this mode the files in the cache will be sparse files and rclone will
|
|
keep track of which bits of the files it has downloaded.
|
|
.PP
|
|
So if an application only reads the starts of each file, then rclone
|
|
will only buffer the start of the file.
|
|
These files will appear to be their full size in the cache, but they
|
|
will be sparse files with only the data that has been downloaded present
|
|
in them.
|
|
.PP
|
|
This mode should support all normal file system operations and is
|
|
otherwise identical to \-\-vfs\-cache\-mode writes.
|
|
.PP
|
|
When reading a file rclone will read \-\-buffer\-size plus
|
|
\-\-vfs\-read\-ahead bytes ahead.
|
|
The \-\-buffer\-size is buffered in memory whereas the
|
|
\-\-vfs\-read\-ahead is buffered on disk.
|
|
.PP
|
|
When using this mode it is recommended that \-\-buffer\-size is not set
|
|
too big and \-\-vfs\-read\-ahead is set large if required.
|
|
.PP
|
|
\f[B]IMPORTANT\f[R] not all file systems support sparse files.
|
|
In particular FAT/exFAT do not.
|
|
Rclone will perform very badly if the cache directory is on a filesystem
|
|
which doesn\[aq]t support sparse files and it will log an ERROR message
|
|
if one is detected.
|
|
.SS VFS Performance
|
|
.PP
|
|
These flags may be used to enable/disable features of the VFS for
|
|
performance or other reasons.
|
|
.PP
|
|
In particular S3 and Swift benefit hugely from the \-\-no\-modtime flag
|
|
(or use \-\-use\-server\-modtime for a slightly different effect) as
|
|
each read of the modification time takes a transaction.
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
\-\-no\-checksum Don\[aq]t compare checksums on up/download.
|
|
\-\-no\-modtime Don\[aq]t read/write the modification time (can speed things up).
|
|
\-\-no\-seek Don\[aq]t allow seeking in files.
|
|
\-\-read\-only Mount read\-only.
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
When rclone reads files from a remote it reads them in chunks.
|
|
This means that rather than requesting the whole file rclone reads the
|
|
chunk specified.
|
|
This is advantageous because some cloud providers account for reads
|
|
being all the data requested, not all the data delivered.
|
|
.PP
|
|
Rclone will keep doubling the chunk size requested starting at
|
|
\-\-vfs\-read\-chunk\-size with a maximum of
|
|
\-\-vfs\-read\-chunk\-size\-limit unless it is set to \[dq]off\[dq] in
|
|
which case there will be no limit.
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
\-\-vfs\-read\-chunk\-size SizeSuffix Read the source objects in chunks. (default 128M)
|
|
\-\-vfs\-read\-chunk\-size\-limit SizeSuffix Max chunk doubling size (default \[dq]off\[dq])
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
Sometimes rclone is delivered reads or writes out of order.
|
|
Rather than seeking rclone will wait a short time for the in sequence
|
|
read or write to come in.
|
|
These flags only come into effect when not using an on disk cache file.
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
\-\-vfs\-read\-wait duration Time to wait for in\-sequence read before seeking. (default 20ms)
|
|
\-\-vfs\-write\-wait duration Time to wait for in\-sequence write before giving error. (default 1s)
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
When using VFS write caching (\-\-vfs\-cache\-mode with value writes or
|
|
full), the global flag \-\-transfers can be set to adjust the number of
|
|
parallel uploads of modified files from cache (the related global flag
|
|
\-\-checkers have no effect on mount).
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
\-\-transfers int Number of file transfers to run in parallel. (default 4)
|
|
\f[R]
|
|
.fi
|
|
.SS VFS Case Sensitivity
|
|
.PP
|
|
Linux file systems are case\-sensitive: two files can differ only by
|
|
case, and the exact case must be used when opening a file.
|
|
.PP
|
|
File systems in modern Windows are case\-insensitive but
|
|
case\-preserving: although existing files can be opened using any case,
|
|
the exact case used to create the file is preserved and available for
|
|
programs to query.
|
|
It is not allowed for two files in the same directory to differ only by
|
|
case.
|
|
.PP
|
|
Usually file systems on macOS are case\-insensitive.
|
|
It is possible to make macOS file systems case\-sensitive but that is
|
|
not the default
|
|
.PP
|
|
The \f[C]\-\-vfs\-case\-insensitive\f[R] mount flag controls how rclone
|
|
handles these two cases.
|
|
If its value is \[dq]false\[dq], rclone passes file names to the mounted
|
|
file system as\-is.
|
|
If the flag is \[dq]true\[dq] (or appears without a value on command
|
|
line), rclone may perform a \[dq]fixup\[dq] as explained below.
|
|
.PP
|
|
The user may specify a file name to open/delete/rename/etc with a case
|
|
different than what is stored on mounted file system.
|
|
If an argument refers to an existing file with exactly the same name,
|
|
then the case of the existing file on the disk will be used.
|
|
However, if a file name with exactly the same name is not found but a
|
|
name differing only by case exists, rclone will transparently fixup the
|
|
name.
|
|
This fixup happens only when an existing file is requested.
|
|
Case sensitivity of file names created anew by rclone is controlled by
|
|
an underlying mounted file system.
|
|
.PP
|
|
Note that case sensitivity of the operating system running rclone (the
|
|
target) may differ from case sensitivity of a file system mounted by
|
|
rclone (the source).
|
|
The flag controls whether \[dq]fixup\[dq] is performed to satisfy the
|
|
target.
|
|
.PP
|
|
If the flag is not provided on the command line, then its default value
|
|
depends on the operating system where rclone runs: \[dq]true\[dq] on
|
|
Windows and macOS, \[dq]false\[dq] otherwise.
|
|
If the flag is provided without a value, then it is \[dq]true\[dq].
|
|
.SS Alternate report of used bytes
|
|
.PP
|
|
Some backends, most notably S3, do not report the amount of bytes used.
|
|
If you need this information to be available when running \f[C]df\f[R]
|
|
on the filesystem, then pass the flag \f[C]\-\-vfs\-used\-is\-size\f[R]
|
|
to rclone.
|
|
With this flag set, instead of relying on the backend to report this
|
|
information, rclone will scan the whole remote similar to
|
|
\f[C]rclone size\f[R] and compute the total used space itself.
|
|
.PP
|
|
\f[I]WARNING.\f[R] Contrary to \f[C]rclone size\f[R], this flag ignores
|
|
filters so that the result is accurate.
|
|
However, this is very inefficient and may cost lots of API calls
|
|
resulting in extra charges.
|
|
Use it as a last resort and only with caching.
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone serve dlna remote:path [flags]
|
|
\f[R]
|
|
.fi
|
|
.SS Options
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
\-\-addr string ip:port or :port to bind the DLNA http server to. (default \[dq]:7879\[dq])
|
|
\-\-dir\-cache\-time duration Time to cache directory entries for. (default 5m0s)
|
|
\-\-dir\-perms FileMode Directory permissions (default 0777)
|
|
\-\-file\-perms FileMode File permissions (default 0666)
|
|
\-\-gid uint32 Override the gid field set by the filesystem. Not supported on Windows. (default 1000)
|
|
\-h, \-\-help help for dlna
|
|
\-\-log\-trace enable trace logging of SOAP traffic
|
|
\-\-name string name of DLNA server
|
|
\-\-no\-checksum Don\[aq]t compare checksums on up/download.
|
|
\-\-no\-modtime Don\[aq]t read/write the modification time (can speed things up).
|
|
\-\-no\-seek Don\[aq]t allow seeking in files.
|
|
\-\-poll\-interval duration Time to wait between polling for changes. Must be smaller than dir\-cache\-time. Only on supported remotes. Set to 0 to disable. (default 1m0s)
|
|
\-\-read\-only Mount read\-only.
|
|
\-\-uid uint32 Override the uid field set by the filesystem. Not supported on Windows. (default 1000)
|
|
\-\-umask int Override the permission bits set by the filesystem. Not supported on Windows. (default 2)
|
|
\-\-vfs\-cache\-max\-age duration Max age of objects in the cache. (default 1h0m0s)
|
|
\-\-vfs\-cache\-max\-size SizeSuffix Max total size of objects in the cache. (default off)
|
|
\-\-vfs\-cache\-mode CacheMode Cache mode off|minimal|writes|full (default off)
|
|
\-\-vfs\-cache\-poll\-interval duration Interval to poll the cache for stale objects. (default 1m0s)
|
|
\-\-vfs\-case\-insensitive If a file name not found, find a case insensitive match.
|
|
\-\-vfs\-read\-ahead SizeSuffix Extra read ahead over \-\-buffer\-size when using cache\-mode full.
|
|
\-\-vfs\-read\-chunk\-size SizeSuffix Read the source objects in chunks. (default 128M)
|
|
\-\-vfs\-read\-chunk\-size\-limit SizeSuffix If greater than \-\-vfs\-read\-chunk\-size, double the chunk size after each chunk read, until the limit is reached. \[aq]off\[aq] is unlimited. (default off)
|
|
\-\-vfs\-read\-wait duration Time to wait for in\-sequence read before seeking. (default 20ms)
|
|
\-\-vfs\-used\-is\-size rclone size Use the rclone size algorithm for Used size.
|
|
\-\-vfs\-write\-back duration Time to writeback files after last use when using cache. (default 5s)
|
|
\-\-vfs\-write\-wait duration Time to wait for in\-sequence write before giving error. (default 1s)
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
See the global flags page (https://rclone.org/flags/) for global options
|
|
not listed here.
|
|
.SS SEE ALSO
|
|
.IP \[bu] 2
|
|
rclone serve (https://rclone.org/commands/rclone_serve/) \- Serve a
|
|
remote over a protocol.
|
|
.SH rclone serve ftp
|
|
.PP
|
|
Serve remote:path over FTP.
|
|
.SS Synopsis
|
|
.PP
|
|
rclone serve ftp implements a basic ftp server to serve the remote over
|
|
FTP protocol.
|
|
This can be viewed with a ftp client or you can make a remote of type
|
|
ftp to read and write it.
|
|
.SS Server options
|
|
.PP
|
|
Use \-\-addr to specify which IP address and port the server should
|
|
listen on, e.g.
|
|
\-\-addr 1.2.3.4:8000 or \-\-addr :8080 to listen to all IPs.
|
|
By default it only listens on localhost.
|
|
You can use port :0 to let the OS choose an available port.
|
|
.PP
|
|
If you set \-\-addr to listen on a public or LAN accessible IP address
|
|
then using Authentication is advised \- see the next section for info.
|
|
.SS Authentication
|
|
.PP
|
|
By default this will serve files without needing a login.
|
|
.PP
|
|
You can set a single username and password with the \-\-user and
|
|
\-\-pass flags.
|
|
.SS VFS \- Virtual File System
|
|
.PP
|
|
This command uses the VFS layer.
|
|
This adapts the cloud storage objects that rclone uses into something
|
|
which looks much more like a disk filing system.
|
|
.PP
|
|
Cloud storage objects have lots of properties which aren\[aq]t like disk
|
|
files \- you can\[aq]t extend them or write to the middle of them, so
|
|
the VFS layer has to deal with that.
|
|
Because there is no one right way of doing this there are various
|
|
options explained below.
|
|
.PP
|
|
The VFS layer also implements a directory cache \- this caches info
|
|
about files and directories (but not the data) in memory.
|
|
.SS VFS Directory Cache
|
|
.PP
|
|
Using the \f[C]\-\-dir\-cache\-time\f[R] flag, you can control how long
|
|
a directory should be considered up to date and not refreshed from the
|
|
backend.
|
|
Changes made through the mount will appear immediately or invalidate the
|
|
cache.
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
\-\-dir\-cache\-time duration Time to cache directory entries for. (default 5m0s)
|
|
\-\-poll\-interval duration Time to wait between polling for changes.
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
However, changes made directly on the cloud storage by the web interface
|
|
or a different copy of rclone will only be picked up once the directory
|
|
cache expires if the backend configured does not support polling for
|
|
changes.
|
|
If the backend supports polling, changes will be picked up within the
|
|
polling interval.
|
|
.PP
|
|
You can send a \f[C]SIGHUP\f[R] signal to rclone for it to flush all
|
|
directory caches, regardless of how old they are.
|
|
Assuming only one rclone instance is running, you can reset the cache
|
|
like this:
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
kill \-SIGHUP $(pidof rclone)
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
If you configure rclone with a remote control (/rc) then you can use
|
|
rclone rc to flush the whole directory cache:
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone rc vfs/forget
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
Or individual files or directories:
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone rc vfs/forget file=path/to/file dir=path/to/dir
|
|
\f[R]
|
|
.fi
|
|
.SS VFS File Buffering
|
|
.PP
|
|
The \f[C]\-\-buffer\-size\f[R] flag determines the amount of memory,
|
|
that will be used to buffer data in advance.
|
|
.PP
|
|
Each open file will try to keep the specified amount of data in memory
|
|
at all times.
|
|
The buffered data is bound to one open file and won\[aq]t be shared.
|
|
.PP
|
|
This flag is a upper limit for the used memory per open file.
|
|
The buffer will only use memory for data that is downloaded but not not
|
|
yet read.
|
|
If the buffer is empty, only a small amount of memory will be used.
|
|
.PP
|
|
The maximum memory used by rclone for buffering can be up to
|
|
\f[C]\-\-buffer\-size * open files\f[R].
|
|
.SS VFS File Caching
|
|
.PP
|
|
These flags control the VFS file caching options.
|
|
File caching is necessary to make the VFS layer appear compatible with a
|
|
normal file system.
|
|
It can be disabled at the cost of some compatibility.
|
|
.PP
|
|
For example you\[aq]ll need to enable VFS caching if you want to read
|
|
and write simultaneously to a file.
|
|
See below for more details.
|
|
.PP
|
|
Note that the VFS cache is separate from the cache backend and you may
|
|
find that you need one or the other or both.
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
\-\-cache\-dir string Directory rclone will use for caching.
|
|
\-\-vfs\-cache\-mode CacheMode Cache mode off|minimal|writes|full (default off)
|
|
\-\-vfs\-cache\-max\-age duration Max age of objects in the cache. (default 1h0m0s)
|
|
\-\-vfs\-cache\-max\-size SizeSuffix Max total size of objects in the cache. (default off)
|
|
\-\-vfs\-cache\-poll\-interval duration Interval to poll the cache for stale objects. (default 1m0s)
|
|
\-\-vfs\-write\-back duration Time to writeback files after last use when using cache. (default 5s)
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
If run with \f[C]\-vv\f[R] rclone will print the location of the file
|
|
cache.
|
|
The files are stored in the user cache file area which is OS dependent
|
|
but can be controlled with \f[C]\-\-cache\-dir\f[R] or setting the
|
|
appropriate environment variable.
|
|
.PP
|
|
The cache has 4 different modes selected by
|
|
\f[C]\-\-vfs\-cache\-mode\f[R].
|
|
The higher the cache mode the more compatible rclone becomes at the cost
|
|
of using disk space.
|
|
.PP
|
|
Note that files are written back to the remote only when they are closed
|
|
and if they haven\[aq]t been accessed for \-\-vfs\-write\-back second.
|
|
If rclone is quit or dies with files that haven\[aq]t been uploaded,
|
|
these will be uploaded next time rclone is run with the same flags.
|
|
.PP
|
|
If using \f[C]\-\-vfs\-cache\-max\-size\f[R] note that the cache may
|
|
exceed this size for two reasons.
|
|
Firstly because it is only checked every
|
|
\f[C]\-\-vfs\-cache\-poll\-interval\f[R].
|
|
Secondly because open files cannot be evicted from the cache.
|
|
.PP
|
|
You \f[B]should not\f[R] run two copies of rclone using the same VFS
|
|
cache with the same or overlapping remotes if using
|
|
\f[C]\-\-vfs\-cache\-mode > off\f[R].
|
|
This can potentially cause data corruption if you do.
|
|
You can work around this by giving each rclone its own cache hierarchy
|
|
with \f[C]\-\-cache\-dir\f[R].
|
|
You don\[aq]t need to worry about this if the remotes in use don\[aq]t
|
|
overlap.
|
|
.SS \-\-vfs\-cache\-mode off
|
|
.PP
|
|
In this mode (the default) the cache will read directly from the remote
|
|
and write directly to the remote without caching anything on disk.
|
|
.PP
|
|
This will mean some operations are not possible
|
|
.IP \[bu] 2
|
|
Files can\[aq]t be opened for both read AND write
|
|
.IP \[bu] 2
|
|
Files opened for write can\[aq]t be seeked
|
|
.IP \[bu] 2
|
|
Existing files opened for write must have O_TRUNC set
|
|
.IP \[bu] 2
|
|
Files open for read with O_TRUNC will be opened write only
|
|
.IP \[bu] 2
|
|
Files open for write only will behave as if O_TRUNC was supplied
|
|
.IP \[bu] 2
|
|
Open modes O_APPEND, O_TRUNC are ignored
|
|
.IP \[bu] 2
|
|
If an upload fails it can\[aq]t be retried
|
|
.SS \-\-vfs\-cache\-mode minimal
|
|
.PP
|
|
This is very similar to \[dq]off\[dq] except that files opened for read
|
|
AND write will be buffered to disk.
|
|
This means that files opened for write will be a lot more compatible,
|
|
but uses the minimal disk space.
|
|
.PP
|
|
These operations are not possible
|
|
.IP \[bu] 2
|
|
Files opened for write only can\[aq]t be seeked
|
|
.IP \[bu] 2
|
|
Existing files opened for write must have O_TRUNC set
|
|
.IP \[bu] 2
|
|
Files opened for write only will ignore O_APPEND, O_TRUNC
|
|
.IP \[bu] 2
|
|
If an upload fails it can\[aq]t be retried
|
|
.SS \-\-vfs\-cache\-mode writes
|
|
.PP
|
|
In this mode files opened for read only are still read directly from the
|
|
remote, write only and read/write files are buffered to disk first.
|
|
.PP
|
|
This mode should support all normal file system operations.
|
|
.PP
|
|
If an upload fails it will be retried at exponentially increasing
|
|
intervals up to 1 minute.
|
|
.SS \-\-vfs\-cache\-mode full
|
|
.PP
|
|
In this mode all reads and writes are buffered to and from disk.
|
|
When data is read from the remote this is buffered to disk as well.
|
|
.PP
|
|
In this mode the files in the cache will be sparse files and rclone will
|
|
keep track of which bits of the files it has downloaded.
|
|
.PP
|
|
So if an application only reads the starts of each file, then rclone
|
|
will only buffer the start of the file.
|
|
These files will appear to be their full size in the cache, but they
|
|
will be sparse files with only the data that has been downloaded present
|
|
in them.
|
|
.PP
|
|
This mode should support all normal file system operations and is
|
|
otherwise identical to \-\-vfs\-cache\-mode writes.
|
|
.PP
|
|
When reading a file rclone will read \-\-buffer\-size plus
|
|
\-\-vfs\-read\-ahead bytes ahead.
|
|
The \-\-buffer\-size is buffered in memory whereas the
|
|
\-\-vfs\-read\-ahead is buffered on disk.
|
|
.PP
|
|
When using this mode it is recommended that \-\-buffer\-size is not set
|
|
too big and \-\-vfs\-read\-ahead is set large if required.
|
|
.PP
|
|
\f[B]IMPORTANT\f[R] not all file systems support sparse files.
|
|
In particular FAT/exFAT do not.
|
|
Rclone will perform very badly if the cache directory is on a filesystem
|
|
which doesn\[aq]t support sparse files and it will log an ERROR message
|
|
if one is detected.
|
|
.SS VFS Performance
|
|
.PP
|
|
These flags may be used to enable/disable features of the VFS for
|
|
performance or other reasons.
|
|
.PP
|
|
In particular S3 and Swift benefit hugely from the \-\-no\-modtime flag
|
|
(or use \-\-use\-server\-modtime for a slightly different effect) as
|
|
each read of the modification time takes a transaction.
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
\-\-no\-checksum Don\[aq]t compare checksums on up/download.
|
|
\-\-no\-modtime Don\[aq]t read/write the modification time (can speed things up).
|
|
\-\-no\-seek Don\[aq]t allow seeking in files.
|
|
\-\-read\-only Mount read\-only.
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
When rclone reads files from a remote it reads them in chunks.
|
|
This means that rather than requesting the whole file rclone reads the
|
|
chunk specified.
|
|
This is advantageous because some cloud providers account for reads
|
|
being all the data requested, not all the data delivered.
|
|
.PP
|
|
Rclone will keep doubling the chunk size requested starting at
|
|
\-\-vfs\-read\-chunk\-size with a maximum of
|
|
\-\-vfs\-read\-chunk\-size\-limit unless it is set to \[dq]off\[dq] in
|
|
which case there will be no limit.
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
\-\-vfs\-read\-chunk\-size SizeSuffix Read the source objects in chunks. (default 128M)
|
|
\-\-vfs\-read\-chunk\-size\-limit SizeSuffix Max chunk doubling size (default \[dq]off\[dq])
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
Sometimes rclone is delivered reads or writes out of order.
|
|
Rather than seeking rclone will wait a short time for the in sequence
|
|
read or write to come in.
|
|
These flags only come into effect when not using an on disk cache file.
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
\-\-vfs\-read\-wait duration Time to wait for in\-sequence read before seeking. (default 20ms)
|
|
\-\-vfs\-write\-wait duration Time to wait for in\-sequence write before giving error. (default 1s)
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
When using VFS write caching (\-\-vfs\-cache\-mode with value writes or
|
|
full), the global flag \-\-transfers can be set to adjust the number of
|
|
parallel uploads of modified files from cache (the related global flag
|
|
\-\-checkers have no effect on mount).
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
\-\-transfers int Number of file transfers to run in parallel. (default 4)
|
|
\f[R]
|
|
.fi
|
|
.SS VFS Case Sensitivity
|
|
.PP
|
|
Linux file systems are case\-sensitive: two files can differ only by
|
|
case, and the exact case must be used when opening a file.
|
|
.PP
|
|
File systems in modern Windows are case\-insensitive but
|
|
case\-preserving: although existing files can be opened using any case,
|
|
the exact case used to create the file is preserved and available for
|
|
programs to query.
|
|
It is not allowed for two files in the same directory to differ only by
|
|
case.
|
|
.PP
|
|
Usually file systems on macOS are case\-insensitive.
|
|
It is possible to make macOS file systems case\-sensitive but that is
|
|
not the default
|
|
.PP
|
|
The \f[C]\-\-vfs\-case\-insensitive\f[R] mount flag controls how rclone
|
|
handles these two cases.
|
|
If its value is \[dq]false\[dq], rclone passes file names to the mounted
|
|
file system as\-is.
|
|
If the flag is \[dq]true\[dq] (or appears without a value on command
|
|
line), rclone may perform a \[dq]fixup\[dq] as explained below.
|
|
.PP
|
|
The user may specify a file name to open/delete/rename/etc with a case
|
|
different than what is stored on mounted file system.
|
|
If an argument refers to an existing file with exactly the same name,
|
|
then the case of the existing file on the disk will be used.
|
|
However, if a file name with exactly the same name is not found but a
|
|
name differing only by case exists, rclone will transparently fixup the
|
|
name.
|
|
This fixup happens only when an existing file is requested.
|
|
Case sensitivity of file names created anew by rclone is controlled by
|
|
an underlying mounted file system.
|
|
.PP
|
|
Note that case sensitivity of the operating system running rclone (the
|
|
target) may differ from case sensitivity of a file system mounted by
|
|
rclone (the source).
|
|
The flag controls whether \[dq]fixup\[dq] is performed to satisfy the
|
|
target.
|
|
.PP
|
|
If the flag is not provided on the command line, then its default value
|
|
depends on the operating system where rclone runs: \[dq]true\[dq] on
|
|
Windows and macOS, \[dq]false\[dq] otherwise.
|
|
If the flag is provided without a value, then it is \[dq]true\[dq].
|
|
.SS Alternate report of used bytes
|
|
.PP
|
|
Some backends, most notably S3, do not report the amount of bytes used.
|
|
If you need this information to be available when running \f[C]df\f[R]
|
|
on the filesystem, then pass the flag \f[C]\-\-vfs\-used\-is\-size\f[R]
|
|
to rclone.
|
|
With this flag set, instead of relying on the backend to report this
|
|
information, rclone will scan the whole remote similar to
|
|
\f[C]rclone size\f[R] and compute the total used space itself.
|
|
.PP
|
|
\f[I]WARNING.\f[R] Contrary to \f[C]rclone size\f[R], this flag ignores
|
|
filters so that the result is accurate.
|
|
However, this is very inefficient and may cost lots of API calls
|
|
resulting in extra charges.
|
|
Use it as a last resort and only with caching.
|
|
.SS Auth Proxy
|
|
.PP
|
|
If you supply the parameter \f[C]\-\-auth\-proxy /path/to/program\f[R]
|
|
then rclone will use that program to generate backends on the fly which
|
|
then are used to authenticate incoming requests.
|
|
This uses a simple JSON based protocol with input on STDIN and output on
|
|
STDOUT.
|
|
.PP
|
|
\f[B]PLEASE NOTE:\f[R] \f[C]\-\-auth\-proxy\f[R] and
|
|
\f[C]\-\-authorized\-keys\f[R] cannot be used together, if
|
|
\f[C]\-\-auth\-proxy\f[R] is set the authorized keys option will be
|
|
ignored.
|
|
.PP
|
|
There is an example program
|
|
bin/test_proxy.py (https://github.com/rclone/rclone/blob/master/test_proxy.py)
|
|
in the rclone source code.
|
|
.PP
|
|
The program\[aq]s job is to take a \f[C]user\f[R] and \f[C]pass\f[R] on
|
|
the input and turn those into the config for a backend on STDOUT in JSON
|
|
format.
|
|
This config will have any default parameters for the backend added, but
|
|
it won\[aq]t use configuration from environment variables or command
|
|
line options \- it is the job of the proxy program to make a complete
|
|
config.
|
|
.PP
|
|
This config generated must have this extra parameter \- \f[C]_root\f[R]
|
|
\- root to use for the backend
|
|
.PP
|
|
And it may have this parameter \- \f[C]_obscure\f[R] \- comma separated
|
|
strings for parameters to obscure
|
|
.PP
|
|
If password authentication was used by the client, input to the proxy
|
|
process (on STDIN) would look similar to this:
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
{
|
|
\[dq]user\[dq]: \[dq]me\[dq],
|
|
\[dq]pass\[dq]: \[dq]mypassword\[dq]
|
|
}
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
If public\-key authentication was used by the client, input to the proxy
|
|
process (on STDIN) would look similar to this:
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
{
|
|
\[dq]user\[dq]: \[dq]me\[dq],
|
|
\[dq]public_key\[dq]: \[dq]AAAAB3NzaC1yc2EAAAADAQABAAABAQDuwESFdAe14hVS6omeyX7edc...JQdf\[dq]
|
|
}
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
And as an example return this on STDOUT
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
{
|
|
\[dq]type\[dq]: \[dq]sftp\[dq],
|
|
\[dq]_root\[dq]: \[dq]\[dq],
|
|
\[dq]_obscure\[dq]: \[dq]pass\[dq],
|
|
\[dq]user\[dq]: \[dq]me\[dq],
|
|
\[dq]pass\[dq]: \[dq]mypassword\[dq],
|
|
\[dq]host\[dq]: \[dq]sftp.example.com\[dq]
|
|
}
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
This would mean that an SFTP backend would be created on the fly for the
|
|
\f[C]user\f[R] and \f[C]pass\f[R]/\f[C]public_key\f[R] returned in the
|
|
output to the host given.
|
|
Note that since \f[C]_obscure\f[R] is set to \f[C]pass\f[R], rclone will
|
|
obscure the \f[C]pass\f[R] parameter before creating the backend (which
|
|
is required for sftp backends).
|
|
.PP
|
|
The program can manipulate the supplied \f[C]user\f[R] in any way, for
|
|
example to make proxy to many different sftp backends, you could make
|
|
the \f[C]user\f[R] be \f[C]user\[at]example.com\f[R] and then set the
|
|
\f[C]host\f[R] to \f[C]example.com\f[R] in the output and the user to
|
|
\f[C]user\f[R].
|
|
For security you\[aq]d probably want to restrict the \f[C]host\f[R] to a
|
|
limited list.
|
|
.PP
|
|
Note that an internal cache is keyed on \f[C]user\f[R] so only use that
|
|
for configuration, don\[aq]t use \f[C]pass\f[R] or \f[C]public_key\f[R].
|
|
This also means that if a user\[aq]s password or public\-key is changed
|
|
the cache will need to expire (which takes 5 mins) before it takes
|
|
effect.
|
|
.PP
|
|
This can be used to build general purpose proxies to any kind of backend
|
|
that rclone supports.
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone serve ftp remote:path [flags]
|
|
\f[R]
|
|
.fi
|
|
.SS Options
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
\-\-addr string IPaddress:Port or :Port to bind server to. (default \[dq]localhost:2121\[dq])
|
|
\-\-auth\-proxy string A program to use to create the backend from the auth.
|
|
\-\-cert string TLS PEM key (concatenation of certificate and CA certificate)
|
|
\-\-dir\-cache\-time duration Time to cache directory entries for. (default 5m0s)
|
|
\-\-dir\-perms FileMode Directory permissions (default 0777)
|
|
\-\-file\-perms FileMode File permissions (default 0666)
|
|
\-\-gid uint32 Override the gid field set by the filesystem. Not supported on Windows. (default 1000)
|
|
\-h, \-\-help help for ftp
|
|
\-\-key string TLS PEM Private key
|
|
\-\-no\-checksum Don\[aq]t compare checksums on up/download.
|
|
\-\-no\-modtime Don\[aq]t read/write the modification time (can speed things up).
|
|
\-\-no\-seek Don\[aq]t allow seeking in files.
|
|
\-\-pass string Password for authentication. (empty value allow every password)
|
|
\-\-passive\-port string Passive port range to use. (default \[dq]30000\-32000\[dq])
|
|
\-\-poll\-interval duration Time to wait between polling for changes. Must be smaller than dir\-cache\-time. Only on supported remotes. Set to 0 to disable. (default 1m0s)
|
|
\-\-public\-ip string Public IP address to advertise for passive connections.
|
|
\-\-read\-only Mount read\-only.
|
|
\-\-uid uint32 Override the uid field set by the filesystem. Not supported on Windows. (default 1000)
|
|
\-\-umask int Override the permission bits set by the filesystem. Not supported on Windows. (default 2)
|
|
\-\-user string User name for authentication. (default \[dq]anonymous\[dq])
|
|
\-\-vfs\-cache\-max\-age duration Max age of objects in the cache. (default 1h0m0s)
|
|
\-\-vfs\-cache\-max\-size SizeSuffix Max total size of objects in the cache. (default off)
|
|
\-\-vfs\-cache\-mode CacheMode Cache mode off|minimal|writes|full (default off)
|
|
\-\-vfs\-cache\-poll\-interval duration Interval to poll the cache for stale objects. (default 1m0s)
|
|
\-\-vfs\-case\-insensitive If a file name not found, find a case insensitive match.
|
|
\-\-vfs\-read\-ahead SizeSuffix Extra read ahead over \-\-buffer\-size when using cache\-mode full.
|
|
\-\-vfs\-read\-chunk\-size SizeSuffix Read the source objects in chunks. (default 128M)
|
|
\-\-vfs\-read\-chunk\-size\-limit SizeSuffix If greater than \-\-vfs\-read\-chunk\-size, double the chunk size after each chunk read, until the limit is reached. \[aq]off\[aq] is unlimited. (default off)
|
|
\-\-vfs\-read\-wait duration Time to wait for in\-sequence read before seeking. (default 20ms)
|
|
\-\-vfs\-used\-is\-size rclone size Use the rclone size algorithm for Used size.
|
|
\-\-vfs\-write\-back duration Time to writeback files after last use when using cache. (default 5s)
|
|
\-\-vfs\-write\-wait duration Time to wait for in\-sequence write before giving error. (default 1s)
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
See the global flags page (https://rclone.org/flags/) for global options
|
|
not listed here.
|
|
.SS SEE ALSO
|
|
.IP \[bu] 2
|
|
rclone serve (https://rclone.org/commands/rclone_serve/) \- Serve a
|
|
remote over a protocol.
|
|
.SH rclone serve http
|
|
.PP
|
|
Serve the remote over HTTP.
|
|
.SS Synopsis
|
|
.PP
|
|
rclone serve http implements a basic web server to serve the remote over
|
|
HTTP.
|
|
This can be viewed in a web browser or you can make a remote of type
|
|
http read from it.
|
|
.PP
|
|
You can use the filter flags (e.g.
|
|
\-\-include, \-\-exclude) to control what is served.
|
|
.PP
|
|
The server will log errors.
|
|
Use \-v to see access logs.
|
|
.PP
|
|
\-\-bwlimit will be respected for file transfers.
|
|
Use \-\-stats to control the stats printing.
|
|
.SS Server options
|
|
.PP
|
|
Use \-\-addr to specify which IP address and port the server should
|
|
listen on, e.g.
|
|
\-\-addr 1.2.3.4:8000 or \-\-addr :8080 to listen to all IPs.
|
|
By default it only listens on localhost.
|
|
You can use port :0 to let the OS choose an available port.
|
|
.PP
|
|
If you set \-\-addr to listen on a public or LAN accessible IP address
|
|
then using Authentication is advised \- see the next section for info.
|
|
.PP
|
|
\-\-server\-read\-timeout and \-\-server\-write\-timeout can be used to
|
|
control the timeouts on the server.
|
|
Note that this is the total time for a transfer.
|
|
.PP
|
|
\-\-max\-header\-bytes controls the maximum number of bytes the server
|
|
will accept in the HTTP header.
|
|
.PP
|
|
\-\-baseurl controls the URL prefix that rclone serves from.
|
|
By default rclone will serve from the root.
|
|
If you used \-\-baseurl \[dq]/rclone\[dq] then rclone would serve from a
|
|
URL starting with \[dq]/rclone/\[dq].
|
|
This is useful if you wish to proxy rclone serve.
|
|
Rclone automatically inserts leading and trailing \[dq]/\[dq] on
|
|
\-\-baseurl, so \-\-baseurl \[dq]rclone\[dq], \-\-baseurl
|
|
\[dq]/rclone\[dq] and \-\-baseurl \[dq]/rclone/\[dq] are all treated
|
|
identically.
|
|
.PP
|
|
\-\-template allows a user to specify a custom markup template for http
|
|
and webdav serve functions.
|
|
The server exports the following markup to be used within the template
|
|
to server pages:
|
|
.PP
|
|
.TS
|
|
tab(@);
|
|
lw(35.0n) lw(35.0n).
|
|
T{
|
|
Parameter
|
|
T}@T{
|
|
Description
|
|
T}
|
|
_
|
|
T{
|
|
\&.Name
|
|
T}@T{
|
|
The full path of a file/directory.
|
|
T}
|
|
T{
|
|
\&.Title
|
|
T}@T{
|
|
Directory listing of .Name
|
|
T}
|
|
T{
|
|
\&.Sort
|
|
T}@T{
|
|
The current sort used.
|
|
This is changeable via ?sort= parameter
|
|
T}
|
|
T{
|
|
T}@T{
|
|
Sort Options: namedirfirst,name,size,time (default namedirfirst)
|
|
T}
|
|
T{
|
|
\&.Order
|
|
T}@T{
|
|
The current ordering used.
|
|
This is changeable via ?order= parameter
|
|
T}
|
|
T{
|
|
T}@T{
|
|
Order Options: asc,desc (default asc)
|
|
T}
|
|
T{
|
|
\&.Query
|
|
T}@T{
|
|
Currently unused.
|
|
T}
|
|
T{
|
|
\&.Breadcrumb
|
|
T}@T{
|
|
Allows for creating a relative navigation
|
|
T}
|
|
T{
|
|
\-\- .Link
|
|
T}@T{
|
|
The relative to the root link of the Text.
|
|
T}
|
|
T{
|
|
\-\- .Text
|
|
T}@T{
|
|
The Name of the directory.
|
|
T}
|
|
T{
|
|
\&.Entries
|
|
T}@T{
|
|
Information about a specific file/directory.
|
|
T}
|
|
T{
|
|
\-\- .URL
|
|
T}@T{
|
|
The \[aq]url\[aq] of an entry.
|
|
T}
|
|
T{
|
|
\-\- .Leaf
|
|
T}@T{
|
|
Currently same as \[aq]URL\[aq] but intended to be \[aq]just\[aq] the
|
|
name.
|
|
T}
|
|
T{
|
|
\-\- .IsDir
|
|
T}@T{
|
|
Boolean for if an entry is a directory or not.
|
|
T}
|
|
T{
|
|
\-\- .Size
|
|
T}@T{
|
|
Size in Bytes of the entry.
|
|
T}
|
|
T{
|
|
\-\- .ModTime
|
|
T}@T{
|
|
The UTC timestamp of an entry.
|
|
T}
|
|
.TE
|
|
.SS Authentication
|
|
.PP
|
|
By default this will serve files without needing a login.
|
|
.PP
|
|
You can either use an htpasswd file which can take lots of users, or set
|
|
a single username and password with the \-\-user and \-\-pass flags.
|
|
.PP
|
|
Use \-\-htpasswd /path/to/htpasswd to provide an htpasswd file.
|
|
This is in standard apache format and supports MD5, SHA1 and BCrypt for
|
|
basic authentication.
|
|
Bcrypt is recommended.
|
|
.PP
|
|
To create an htpasswd file:
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
touch htpasswd
|
|
htpasswd \-B htpasswd user
|
|
htpasswd \-B htpasswd anotherUser
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
The password file can be updated while rclone is running.
|
|
.PP
|
|
Use \-\-realm to set the authentication realm.
|
|
.SS SSL/TLS
|
|
.PP
|
|
By default this will serve over http.
|
|
If you want you can serve over https.
|
|
You will need to supply the \-\-cert and \-\-key flags.
|
|
If you wish to do client side certificate validation then you will need
|
|
to supply \-\-client\-ca also.
|
|
.PP
|
|
\-\-cert should be either a PEM encoded certificate or a concatenation
|
|
of that with the CA certificate.
|
|
\-\-key should be the PEM encoded private key and \-\-client\-ca should
|
|
be the PEM encoded client certificate authority certificate.
|
|
.SS VFS \- Virtual File System
|
|
.PP
|
|
This command uses the VFS layer.
|
|
This adapts the cloud storage objects that rclone uses into something
|
|
which looks much more like a disk filing system.
|
|
.PP
|
|
Cloud storage objects have lots of properties which aren\[aq]t like disk
|
|
files \- you can\[aq]t extend them or write to the middle of them, so
|
|
the VFS layer has to deal with that.
|
|
Because there is no one right way of doing this there are various
|
|
options explained below.
|
|
.PP
|
|
The VFS layer also implements a directory cache \- this caches info
|
|
about files and directories (but not the data) in memory.
|
|
.SS VFS Directory Cache
|
|
.PP
|
|
Using the \f[C]\-\-dir\-cache\-time\f[R] flag, you can control how long
|
|
a directory should be considered up to date and not refreshed from the
|
|
backend.
|
|
Changes made through the mount will appear immediately or invalidate the
|
|
cache.
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
\-\-dir\-cache\-time duration Time to cache directory entries for. (default 5m0s)
|
|
\-\-poll\-interval duration Time to wait between polling for changes.
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
However, changes made directly on the cloud storage by the web interface
|
|
or a different copy of rclone will only be picked up once the directory
|
|
cache expires if the backend configured does not support polling for
|
|
changes.
|
|
If the backend supports polling, changes will be picked up within the
|
|
polling interval.
|
|
.PP
|
|
You can send a \f[C]SIGHUP\f[R] signal to rclone for it to flush all
|
|
directory caches, regardless of how old they are.
|
|
Assuming only one rclone instance is running, you can reset the cache
|
|
like this:
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
kill \-SIGHUP $(pidof rclone)
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
If you configure rclone with a remote control (/rc) then you can use
|
|
rclone rc to flush the whole directory cache:
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone rc vfs/forget
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
Or individual files or directories:
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone rc vfs/forget file=path/to/file dir=path/to/dir
|
|
\f[R]
|
|
.fi
|
|
.SS VFS File Buffering
|
|
.PP
|
|
The \f[C]\-\-buffer\-size\f[R] flag determines the amount of memory,
|
|
that will be used to buffer data in advance.
|
|
.PP
|
|
Each open file will try to keep the specified amount of data in memory
|
|
at all times.
|
|
The buffered data is bound to one open file and won\[aq]t be shared.
|
|
.PP
|
|
This flag is a upper limit for the used memory per open file.
|
|
The buffer will only use memory for data that is downloaded but not not
|
|
yet read.
|
|
If the buffer is empty, only a small amount of memory will be used.
|
|
.PP
|
|
The maximum memory used by rclone for buffering can be up to
|
|
\f[C]\-\-buffer\-size * open files\f[R].
|
|
.SS VFS File Caching
|
|
.PP
|
|
These flags control the VFS file caching options.
|
|
File caching is necessary to make the VFS layer appear compatible with a
|
|
normal file system.
|
|
It can be disabled at the cost of some compatibility.
|
|
.PP
|
|
For example you\[aq]ll need to enable VFS caching if you want to read
|
|
and write simultaneously to a file.
|
|
See below for more details.
|
|
.PP
|
|
Note that the VFS cache is separate from the cache backend and you may
|
|
find that you need one or the other or both.
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
\-\-cache\-dir string Directory rclone will use for caching.
|
|
\-\-vfs\-cache\-mode CacheMode Cache mode off|minimal|writes|full (default off)
|
|
\-\-vfs\-cache\-max\-age duration Max age of objects in the cache. (default 1h0m0s)
|
|
\-\-vfs\-cache\-max\-size SizeSuffix Max total size of objects in the cache. (default off)
|
|
\-\-vfs\-cache\-poll\-interval duration Interval to poll the cache for stale objects. (default 1m0s)
|
|
\-\-vfs\-write\-back duration Time to writeback files after last use when using cache. (default 5s)
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
If run with \f[C]\-vv\f[R] rclone will print the location of the file
|
|
cache.
|
|
The files are stored in the user cache file area which is OS dependent
|
|
but can be controlled with \f[C]\-\-cache\-dir\f[R] or setting the
|
|
appropriate environment variable.
|
|
.PP
|
|
The cache has 4 different modes selected by
|
|
\f[C]\-\-vfs\-cache\-mode\f[R].
|
|
The higher the cache mode the more compatible rclone becomes at the cost
|
|
of using disk space.
|
|
.PP
|
|
Note that files are written back to the remote only when they are closed
|
|
and if they haven\[aq]t been accessed for \-\-vfs\-write\-back second.
|
|
If rclone is quit or dies with files that haven\[aq]t been uploaded,
|
|
these will be uploaded next time rclone is run with the same flags.
|
|
.PP
|
|
If using \f[C]\-\-vfs\-cache\-max\-size\f[R] note that the cache may
|
|
exceed this size for two reasons.
|
|
Firstly because it is only checked every
|
|
\f[C]\-\-vfs\-cache\-poll\-interval\f[R].
|
|
Secondly because open files cannot be evicted from the cache.
|
|
.PP
|
|
You \f[B]should not\f[R] run two copies of rclone using the same VFS
|
|
cache with the same or overlapping remotes if using
|
|
\f[C]\-\-vfs\-cache\-mode > off\f[R].
|
|
This can potentially cause data corruption if you do.
|
|
You can work around this by giving each rclone its own cache hierarchy
|
|
with \f[C]\-\-cache\-dir\f[R].
|
|
You don\[aq]t need to worry about this if the remotes in use don\[aq]t
|
|
overlap.
|
|
.SS \-\-vfs\-cache\-mode off
|
|
.PP
|
|
In this mode (the default) the cache will read directly from the remote
|
|
and write directly to the remote without caching anything on disk.
|
|
.PP
|
|
This will mean some operations are not possible
|
|
.IP \[bu] 2
|
|
Files can\[aq]t be opened for both read AND write
|
|
.IP \[bu] 2
|
|
Files opened for write can\[aq]t be seeked
|
|
.IP \[bu] 2
|
|
Existing files opened for write must have O_TRUNC set
|
|
.IP \[bu] 2
|
|
Files open for read with O_TRUNC will be opened write only
|
|
.IP \[bu] 2
|
|
Files open for write only will behave as if O_TRUNC was supplied
|
|
.IP \[bu] 2
|
|
Open modes O_APPEND, O_TRUNC are ignored
|
|
.IP \[bu] 2
|
|
If an upload fails it can\[aq]t be retried
|
|
.SS \-\-vfs\-cache\-mode minimal
|
|
.PP
|
|
This is very similar to \[dq]off\[dq] except that files opened for read
|
|
AND write will be buffered to disk.
|
|
This means that files opened for write will be a lot more compatible,
|
|
but uses the minimal disk space.
|
|
.PP
|
|
These operations are not possible
|
|
.IP \[bu] 2
|
|
Files opened for write only can\[aq]t be seeked
|
|
.IP \[bu] 2
|
|
Existing files opened for write must have O_TRUNC set
|
|
.IP \[bu] 2
|
|
Files opened for write only will ignore O_APPEND, O_TRUNC
|
|
.IP \[bu] 2
|
|
If an upload fails it can\[aq]t be retried
|
|
.SS \-\-vfs\-cache\-mode writes
|
|
.PP
|
|
In this mode files opened for read only are still read directly from the
|
|
remote, write only and read/write files are buffered to disk first.
|
|
.PP
|
|
This mode should support all normal file system operations.
|
|
.PP
|
|
If an upload fails it will be retried at exponentially increasing
|
|
intervals up to 1 minute.
|
|
.SS \-\-vfs\-cache\-mode full
|
|
.PP
|
|
In this mode all reads and writes are buffered to and from disk.
|
|
When data is read from the remote this is buffered to disk as well.
|
|
.PP
|
|
In this mode the files in the cache will be sparse files and rclone will
|
|
keep track of which bits of the files it has downloaded.
|
|
.PP
|
|
So if an application only reads the starts of each file, then rclone
|
|
will only buffer the start of the file.
|
|
These files will appear to be their full size in the cache, but they
|
|
will be sparse files with only the data that has been downloaded present
|
|
in them.
|
|
.PP
|
|
This mode should support all normal file system operations and is
|
|
otherwise identical to \-\-vfs\-cache\-mode writes.
|
|
.PP
|
|
When reading a file rclone will read \-\-buffer\-size plus
|
|
\-\-vfs\-read\-ahead bytes ahead.
|
|
The \-\-buffer\-size is buffered in memory whereas the
|
|
\-\-vfs\-read\-ahead is buffered on disk.
|
|
.PP
|
|
When using this mode it is recommended that \-\-buffer\-size is not set
|
|
too big and \-\-vfs\-read\-ahead is set large if required.
|
|
.PP
|
|
\f[B]IMPORTANT\f[R] not all file systems support sparse files.
|
|
In particular FAT/exFAT do not.
|
|
Rclone will perform very badly if the cache directory is on a filesystem
|
|
which doesn\[aq]t support sparse files and it will log an ERROR message
|
|
if one is detected.
|
|
.SS VFS Performance
|
|
.PP
|
|
These flags may be used to enable/disable features of the VFS for
|
|
performance or other reasons.
|
|
.PP
|
|
In particular S3 and Swift benefit hugely from the \-\-no\-modtime flag
|
|
(or use \-\-use\-server\-modtime for a slightly different effect) as
|
|
each read of the modification time takes a transaction.
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
\-\-no\-checksum Don\[aq]t compare checksums on up/download.
|
|
\-\-no\-modtime Don\[aq]t read/write the modification time (can speed things up).
|
|
\-\-no\-seek Don\[aq]t allow seeking in files.
|
|
\-\-read\-only Mount read\-only.
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
When rclone reads files from a remote it reads them in chunks.
|
|
This means that rather than requesting the whole file rclone reads the
|
|
chunk specified.
|
|
This is advantageous because some cloud providers account for reads
|
|
being all the data requested, not all the data delivered.
|
|
.PP
|
|
Rclone will keep doubling the chunk size requested starting at
|
|
\-\-vfs\-read\-chunk\-size with a maximum of
|
|
\-\-vfs\-read\-chunk\-size\-limit unless it is set to \[dq]off\[dq] in
|
|
which case there will be no limit.
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
\-\-vfs\-read\-chunk\-size SizeSuffix Read the source objects in chunks. (default 128M)
|
|
\-\-vfs\-read\-chunk\-size\-limit SizeSuffix Max chunk doubling size (default \[dq]off\[dq])
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
Sometimes rclone is delivered reads or writes out of order.
|
|
Rather than seeking rclone will wait a short time for the in sequence
|
|
read or write to come in.
|
|
These flags only come into effect when not using an on disk cache file.
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
\-\-vfs\-read\-wait duration Time to wait for in\-sequence read before seeking. (default 20ms)
|
|
\-\-vfs\-write\-wait duration Time to wait for in\-sequence write before giving error. (default 1s)
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
When using VFS write caching (\-\-vfs\-cache\-mode with value writes or
|
|
full), the global flag \-\-transfers can be set to adjust the number of
|
|
parallel uploads of modified files from cache (the related global flag
|
|
\-\-checkers have no effect on mount).
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
\-\-transfers int Number of file transfers to run in parallel. (default 4)
|
|
\f[R]
|
|
.fi
|
|
.SS VFS Case Sensitivity
|
|
.PP
|
|
Linux file systems are case\-sensitive: two files can differ only by
|
|
case, and the exact case must be used when opening a file.
|
|
.PP
|
|
File systems in modern Windows are case\-insensitive but
|
|
case\-preserving: although existing files can be opened using any case,
|
|
the exact case used to create the file is preserved and available for
|
|
programs to query.
|
|
It is not allowed for two files in the same directory to differ only by
|
|
case.
|
|
.PP
|
|
Usually file systems on macOS are case\-insensitive.
|
|
It is possible to make macOS file systems case\-sensitive but that is
|
|
not the default
|
|
.PP
|
|
The \f[C]\-\-vfs\-case\-insensitive\f[R] mount flag controls how rclone
|
|
handles these two cases.
|
|
If its value is \[dq]false\[dq], rclone passes file names to the mounted
|
|
file system as\-is.
|
|
If the flag is \[dq]true\[dq] (or appears without a value on command
|
|
line), rclone may perform a \[dq]fixup\[dq] as explained below.
|
|
.PP
|
|
The user may specify a file name to open/delete/rename/etc with a case
|
|
different than what is stored on mounted file system.
|
|
If an argument refers to an existing file with exactly the same name,
|
|
then the case of the existing file on the disk will be used.
|
|
However, if a file name with exactly the same name is not found but a
|
|
name differing only by case exists, rclone will transparently fixup the
|
|
name.
|
|
This fixup happens only when an existing file is requested.
|
|
Case sensitivity of file names created anew by rclone is controlled by
|
|
an underlying mounted file system.
|
|
.PP
|
|
Note that case sensitivity of the operating system running rclone (the
|
|
target) may differ from case sensitivity of a file system mounted by
|
|
rclone (the source).
|
|
The flag controls whether \[dq]fixup\[dq] is performed to satisfy the
|
|
target.
|
|
.PP
|
|
If the flag is not provided on the command line, then its default value
|
|
depends on the operating system where rclone runs: \[dq]true\[dq] on
|
|
Windows and macOS, \[dq]false\[dq] otherwise.
|
|
If the flag is provided without a value, then it is \[dq]true\[dq].
|
|
.SS Alternate report of used bytes
|
|
.PP
|
|
Some backends, most notably S3, do not report the amount of bytes used.
|
|
If you need this information to be available when running \f[C]df\f[R]
|
|
on the filesystem, then pass the flag \f[C]\-\-vfs\-used\-is\-size\f[R]
|
|
to rclone.
|
|
With this flag set, instead of relying on the backend to report this
|
|
information, rclone will scan the whole remote similar to
|
|
\f[C]rclone size\f[R] and compute the total used space itself.
|
|
.PP
|
|
\f[I]WARNING.\f[R] Contrary to \f[C]rclone size\f[R], this flag ignores
|
|
filters so that the result is accurate.
|
|
However, this is very inefficient and may cost lots of API calls
|
|
resulting in extra charges.
|
|
Use it as a last resort and only with caching.
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone serve http remote:path [flags]
|
|
\f[R]
|
|
.fi
|
|
.SS Options
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
\-\-addr string IPaddress:Port or :Port to bind server to. (default \[dq]localhost:8080\[dq])
|
|
\-\-baseurl string Prefix for URLs \- leave blank for root.
|
|
\-\-cert string SSL PEM key (concatenation of certificate and CA certificate)
|
|
\-\-client\-ca string Client certificate authority to verify clients with
|
|
\-\-dir\-cache\-time duration Time to cache directory entries for. (default 5m0s)
|
|
\-\-dir\-perms FileMode Directory permissions (default 0777)
|
|
\-\-file\-perms FileMode File permissions (default 0666)
|
|
\-\-gid uint32 Override the gid field set by the filesystem. Not supported on Windows. (default 1000)
|
|
\-h, \-\-help help for http
|
|
\-\-htpasswd string htpasswd file \- if not provided no authentication is done
|
|
\-\-key string SSL PEM Private key
|
|
\-\-max\-header\-bytes int Maximum size of request header (default 4096)
|
|
\-\-no\-checksum Don\[aq]t compare checksums on up/download.
|
|
\-\-no\-modtime Don\[aq]t read/write the modification time (can speed things up).
|
|
\-\-no\-seek Don\[aq]t allow seeking in files.
|
|
\-\-pass string Password for authentication.
|
|
\-\-poll\-interval duration Time to wait between polling for changes. Must be smaller than dir\-cache\-time. Only on supported remotes. Set to 0 to disable. (default 1m0s)
|
|
\-\-read\-only Mount read\-only.
|
|
\-\-realm string realm for authentication (default \[dq]rclone\[dq])
|
|
\-\-server\-read\-timeout duration Timeout for server reading data (default 1h0m0s)
|
|
\-\-server\-write\-timeout duration Timeout for server writing data (default 1h0m0s)
|
|
\-\-template string User Specified Template.
|
|
\-\-uid uint32 Override the uid field set by the filesystem. Not supported on Windows. (default 1000)
|
|
\-\-umask int Override the permission bits set by the filesystem. Not supported on Windows. (default 2)
|
|
\-\-user string User name for authentication.
|
|
\-\-vfs\-cache\-max\-age duration Max age of objects in the cache. (default 1h0m0s)
|
|
\-\-vfs\-cache\-max\-size SizeSuffix Max total size of objects in the cache. (default off)
|
|
\-\-vfs\-cache\-mode CacheMode Cache mode off|minimal|writes|full (default off)
|
|
\-\-vfs\-cache\-poll\-interval duration Interval to poll the cache for stale objects. (default 1m0s)
|
|
\-\-vfs\-case\-insensitive If a file name not found, find a case insensitive match.
|
|
\-\-vfs\-read\-ahead SizeSuffix Extra read ahead over \-\-buffer\-size when using cache\-mode full.
|
|
\-\-vfs\-read\-chunk\-size SizeSuffix Read the source objects in chunks. (default 128M)
|
|
\-\-vfs\-read\-chunk\-size\-limit SizeSuffix If greater than \-\-vfs\-read\-chunk\-size, double the chunk size after each chunk read, until the limit is reached. \[aq]off\[aq] is unlimited. (default off)
|
|
\-\-vfs\-read\-wait duration Time to wait for in\-sequence read before seeking. (default 20ms)
|
|
\-\-vfs\-used\-is\-size rclone size Use the rclone size algorithm for Used size.
|
|
\-\-vfs\-write\-back duration Time to writeback files after last use when using cache. (default 5s)
|
|
\-\-vfs\-write\-wait duration Time to wait for in\-sequence write before giving error. (default 1s)
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
See the global flags page (https://rclone.org/flags/) for global options
|
|
not listed here.
|
|
.SS SEE ALSO
|
|
.IP \[bu] 2
|
|
rclone serve (https://rclone.org/commands/rclone_serve/) \- Serve a
|
|
remote over a protocol.
|
|
.SH rclone serve restic
|
|
.PP
|
|
Serve the remote for restic\[aq]s REST API.
|
|
.SS Synopsis
|
|
.PP
|
|
rclone serve restic implements restic\[aq]s REST backend API over HTTP.
|
|
This allows restic to use rclone as a data storage mechanism for cloud
|
|
providers that restic does not support directly.
|
|
.PP
|
|
Restic (https://restic.net/) is a command line program for doing
|
|
backups.
|
|
.PP
|
|
The server will log errors.
|
|
Use \-v to see access logs.
|
|
.PP
|
|
\-\-bwlimit will be respected for file transfers.
|
|
Use \-\-stats to control the stats printing.
|
|
.SS Setting up rclone for use by restic
|
|
.PP
|
|
First set up a remote for your chosen cloud
|
|
provider (https://rclone.org/docs/#configure).
|
|
.PP
|
|
Once you have set up the remote, check it is working with, for example
|
|
\[dq]rclone lsd remote:\[dq].
|
|
You may have called the remote something other than \[dq]remote:\[dq] \-
|
|
just substitute whatever you called it in the following instructions.
|
|
.PP
|
|
Now start the rclone restic server
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone serve restic \-v remote:backup
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
Where you can replace \[dq]backup\[dq] in the above by whatever path in
|
|
the remote you wish to use.
|
|
.PP
|
|
By default this will serve on \[dq]localhost:8080\[dq] you can change
|
|
this with use of the \[dq]\-\-addr\[dq] flag.
|
|
.PP
|
|
You might wish to start this server on boot.
|
|
.PP
|
|
Adding \-\-cache\-objects=false will cause rclone to stop caching
|
|
objects returned from the List call.
|
|
Caching is normally desirable as it speeds up downloading objects, saves
|
|
transactions and uses very little memory.
|
|
.SS Setting up restic to use rclone
|
|
.PP
|
|
Now you can follow the restic
|
|
instructions (http://restic.readthedocs.io/en/latest/030_preparing_a_new_repo.html#rest-server)
|
|
on setting up restic.
|
|
.PP
|
|
Note that you will need restic 0.8.2 or later to interoperate with
|
|
rclone.
|
|
.PP
|
|
For the example above you will want to use
|
|
\[dq]http://localhost:8080/\[dq] as the URL for the REST server.
|
|
.PP
|
|
For example:
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
$ export RESTIC_REPOSITORY=rest:http://localhost:8080/
|
|
$ export RESTIC_PASSWORD=yourpassword
|
|
$ restic init
|
|
created restic backend 8b1a4b56ae at rest:http://localhost:8080/
|
|
|
|
Please note that knowledge of your password is required to access
|
|
the repository. Losing your password means that your data is
|
|
irrecoverably lost.
|
|
$ restic backup /path/to/files/to/backup
|
|
scan [/path/to/files/to/backup]
|
|
scanned 189 directories, 312 files in 0:00
|
|
[0:00] 100.00% 38.128 MiB / 38.128 MiB 501 / 501 items 0 errors ETA 0:00
|
|
duration: 0:00
|
|
snapshot 45c8fdd8 saved
|
|
\f[R]
|
|
.fi
|
|
.SS Multiple repositories
|
|
.PP
|
|
Note that you can use the endpoint to host multiple repositories.
|
|
Do this by adding a directory name or path after the URL.
|
|
Note that these \f[B]must\f[R] end with /.
|
|
Eg
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
$ export RESTIC_REPOSITORY=rest:http://localhost:8080/user1repo/
|
|
# backup user1 stuff
|
|
$ export RESTIC_REPOSITORY=rest:http://localhost:8080/user2repo/
|
|
# backup user2 stuff
|
|
\f[R]
|
|
.fi
|
|
.SS Private repositories
|
|
.PP
|
|
The \[dq]\-\-private\-repos\[dq] flag can be used to limit users to
|
|
repositories starting with a path of \f[C]/<username>/\f[R].
|
|
.SS Server options
|
|
.PP
|
|
Use \-\-addr to specify which IP address and port the server should
|
|
listen on, e.g.
|
|
\-\-addr 1.2.3.4:8000 or \-\-addr :8080 to listen to all IPs.
|
|
By default it only listens on localhost.
|
|
You can use port :0 to let the OS choose an available port.
|
|
.PP
|
|
If you set \-\-addr to listen on a public or LAN accessible IP address
|
|
then using Authentication is advised \- see the next section for info.
|
|
.PP
|
|
\-\-server\-read\-timeout and \-\-server\-write\-timeout can be used to
|
|
control the timeouts on the server.
|
|
Note that this is the total time for a transfer.
|
|
.PP
|
|
\-\-max\-header\-bytes controls the maximum number of bytes the server
|
|
will accept in the HTTP header.
|
|
.PP
|
|
\-\-baseurl controls the URL prefix that rclone serves from.
|
|
By default rclone will serve from the root.
|
|
If you used \-\-baseurl \[dq]/rclone\[dq] then rclone would serve from a
|
|
URL starting with \[dq]/rclone/\[dq].
|
|
This is useful if you wish to proxy rclone serve.
|
|
Rclone automatically inserts leading and trailing \[dq]/\[dq] on
|
|
\-\-baseurl, so \-\-baseurl \[dq]rclone\[dq], \-\-baseurl
|
|
\[dq]/rclone\[dq] and \-\-baseurl \[dq]/rclone/\[dq] are all treated
|
|
identically.
|
|
.PP
|
|
\-\-template allows a user to specify a custom markup template for http
|
|
and webdav serve functions.
|
|
The server exports the following markup to be used within the template
|
|
to server pages:
|
|
.PP
|
|
.TS
|
|
tab(@);
|
|
lw(35.0n) lw(35.0n).
|
|
T{
|
|
Parameter
|
|
T}@T{
|
|
Description
|
|
T}
|
|
_
|
|
T{
|
|
\&.Name
|
|
T}@T{
|
|
The full path of a file/directory.
|
|
T}
|
|
T{
|
|
\&.Title
|
|
T}@T{
|
|
Directory listing of .Name
|
|
T}
|
|
T{
|
|
\&.Sort
|
|
T}@T{
|
|
The current sort used.
|
|
This is changeable via ?sort= parameter
|
|
T}
|
|
T{
|
|
T}@T{
|
|
Sort Options: namedirfirst,name,size,time (default namedirfirst)
|
|
T}
|
|
T{
|
|
\&.Order
|
|
T}@T{
|
|
The current ordering used.
|
|
This is changeable via ?order= parameter
|
|
T}
|
|
T{
|
|
T}@T{
|
|
Order Options: asc,desc (default asc)
|
|
T}
|
|
T{
|
|
\&.Query
|
|
T}@T{
|
|
Currently unused.
|
|
T}
|
|
T{
|
|
\&.Breadcrumb
|
|
T}@T{
|
|
Allows for creating a relative navigation
|
|
T}
|
|
T{
|
|
\-\- .Link
|
|
T}@T{
|
|
The relative to the root link of the Text.
|
|
T}
|
|
T{
|
|
\-\- .Text
|
|
T}@T{
|
|
The Name of the directory.
|
|
T}
|
|
T{
|
|
\&.Entries
|
|
T}@T{
|
|
Information about a specific file/directory.
|
|
T}
|
|
T{
|
|
\-\- .URL
|
|
T}@T{
|
|
The \[aq]url\[aq] of an entry.
|
|
T}
|
|
T{
|
|
\-\- .Leaf
|
|
T}@T{
|
|
Currently same as \[aq]URL\[aq] but intended to be \[aq]just\[aq] the
|
|
name.
|
|
T}
|
|
T{
|
|
\-\- .IsDir
|
|
T}@T{
|
|
Boolean for if an entry is a directory or not.
|
|
T}
|
|
T{
|
|
\-\- .Size
|
|
T}@T{
|
|
Size in Bytes of the entry.
|
|
T}
|
|
T{
|
|
\-\- .ModTime
|
|
T}@T{
|
|
The UTC timestamp of an entry.
|
|
T}
|
|
.TE
|
|
.SS Authentication
|
|
.PP
|
|
By default this will serve files without needing a login.
|
|
.PP
|
|
You can either use an htpasswd file which can take lots of users, or set
|
|
a single username and password with the \-\-user and \-\-pass flags.
|
|
.PP
|
|
Use \-\-htpasswd /path/to/htpasswd to provide an htpasswd file.
|
|
This is in standard apache format and supports MD5, SHA1 and BCrypt for
|
|
basic authentication.
|
|
Bcrypt is recommended.
|
|
.PP
|
|
To create an htpasswd file:
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
touch htpasswd
|
|
htpasswd \-B htpasswd user
|
|
htpasswd \-B htpasswd anotherUser
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
The password file can be updated while rclone is running.
|
|
.PP
|
|
Use \-\-realm to set the authentication realm.
|
|
.SS SSL/TLS
|
|
.PP
|
|
By default this will serve over http.
|
|
If you want you can serve over https.
|
|
You will need to supply the \-\-cert and \-\-key flags.
|
|
If you wish to do client side certificate validation then you will need
|
|
to supply \-\-client\-ca also.
|
|
.PP
|
|
\-\-cert should be either a PEM encoded certificate or a concatenation
|
|
of that with the CA certificate.
|
|
\-\-key should be the PEM encoded private key and \-\-client\-ca should
|
|
be the PEM encoded client certificate authority certificate.
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone serve restic remote:path [flags]
|
|
\f[R]
|
|
.fi
|
|
.SS Options
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
\-\-addr string IPaddress:Port or :Port to bind server to. (default \[dq]localhost:8080\[dq])
|
|
\-\-append\-only disallow deletion of repository data
|
|
\-\-baseurl string Prefix for URLs \- leave blank for root.
|
|
\-\-cache\-objects cache listed objects (default true)
|
|
\-\-cert string SSL PEM key (concatenation of certificate and CA certificate)
|
|
\-\-client\-ca string Client certificate authority to verify clients with
|
|
\-h, \-\-help help for restic
|
|
\-\-htpasswd string htpasswd file \- if not provided no authentication is done
|
|
\-\-key string SSL PEM Private key
|
|
\-\-max\-header\-bytes int Maximum size of request header (default 4096)
|
|
\-\-pass string Password for authentication.
|
|
\-\-private\-repos users can only access their private repo
|
|
\-\-realm string realm for authentication (default \[dq]rclone\[dq])
|
|
\-\-server\-read\-timeout duration Timeout for server reading data (default 1h0m0s)
|
|
\-\-server\-write\-timeout duration Timeout for server writing data (default 1h0m0s)
|
|
\-\-stdio run an HTTP2 server on stdin/stdout
|
|
\-\-template string User Specified Template.
|
|
\-\-user string User name for authentication.
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
See the global flags page (https://rclone.org/flags/) for global options
|
|
not listed here.
|
|
.SS SEE ALSO
|
|
.IP \[bu] 2
|
|
rclone serve (https://rclone.org/commands/rclone_serve/) \- Serve a
|
|
remote over a protocol.
|
|
.SH rclone serve sftp
|
|
.PP
|
|
Serve the remote over SFTP.
|
|
.SS Synopsis
|
|
.PP
|
|
rclone serve sftp implements an SFTP server to serve the remote over
|
|
SFTP.
|
|
This can be used with an SFTP client or you can make a remote of type
|
|
sftp to use with it.
|
|
.PP
|
|
You can use the filter flags (e.g.
|
|
\-\-include, \-\-exclude) to control what is served.
|
|
.PP
|
|
The server will log errors.
|
|
Use \-v to see access logs.
|
|
.PP
|
|
\-\-bwlimit will be respected for file transfers.
|
|
Use \-\-stats to control the stats printing.
|
|
.PP
|
|
You must provide some means of authentication, either with
|
|
\-\-user/\-\-pass, an authorized keys file (specify location with
|
|
\-\-authorized\-keys \- the default is the same as ssh), an
|
|
\-\-auth\-proxy, or set the \-\-no\-auth flag for no authentication when
|
|
logging in.
|
|
.PP
|
|
Note that this also implements a small number of shell commands so that
|
|
it can provide md5sum/sha1sum/df information for the rclone sftp
|
|
backend.
|
|
This means that is can support SHA1SUMs, MD5SUMs and the about command
|
|
when paired with the rclone sftp backend.
|
|
.PP
|
|
If you don\[aq]t supply a \-\-key then rclone will generate one and
|
|
cache it for later use.
|
|
.PP
|
|
By default the server binds to localhost:2022 \- if you want it to be
|
|
reachable externally then supply \[dq]\-\-addr :2022\[dq] for example.
|
|
.PP
|
|
Note that the default of \[dq]\-\-vfs\-cache\-mode off\[dq] is fine for
|
|
the rclone sftp backend, but it may not be with other SFTP clients.
|
|
.SS VFS \- Virtual File System
|
|
.PP
|
|
This command uses the VFS layer.
|
|
This adapts the cloud storage objects that rclone uses into something
|
|
which looks much more like a disk filing system.
|
|
.PP
|
|
Cloud storage objects have lots of properties which aren\[aq]t like disk
|
|
files \- you can\[aq]t extend them or write to the middle of them, so
|
|
the VFS layer has to deal with that.
|
|
Because there is no one right way of doing this there are various
|
|
options explained below.
|
|
.PP
|
|
The VFS layer also implements a directory cache \- this caches info
|
|
about files and directories (but not the data) in memory.
|
|
.SS VFS Directory Cache
|
|
.PP
|
|
Using the \f[C]\-\-dir\-cache\-time\f[R] flag, you can control how long
|
|
a directory should be considered up to date and not refreshed from the
|
|
backend.
|
|
Changes made through the mount will appear immediately or invalidate the
|
|
cache.
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
\-\-dir\-cache\-time duration Time to cache directory entries for. (default 5m0s)
|
|
\-\-poll\-interval duration Time to wait between polling for changes.
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
However, changes made directly on the cloud storage by the web interface
|
|
or a different copy of rclone will only be picked up once the directory
|
|
cache expires if the backend configured does not support polling for
|
|
changes.
|
|
If the backend supports polling, changes will be picked up within the
|
|
polling interval.
|
|
.PP
|
|
You can send a \f[C]SIGHUP\f[R] signal to rclone for it to flush all
|
|
directory caches, regardless of how old they are.
|
|
Assuming only one rclone instance is running, you can reset the cache
|
|
like this:
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
kill \-SIGHUP $(pidof rclone)
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
If you configure rclone with a remote control (/rc) then you can use
|
|
rclone rc to flush the whole directory cache:
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone rc vfs/forget
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
Or individual files or directories:
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone rc vfs/forget file=path/to/file dir=path/to/dir
|
|
\f[R]
|
|
.fi
|
|
.SS VFS File Buffering
|
|
.PP
|
|
The \f[C]\-\-buffer\-size\f[R] flag determines the amount of memory,
|
|
that will be used to buffer data in advance.
|
|
.PP
|
|
Each open file will try to keep the specified amount of data in memory
|
|
at all times.
|
|
The buffered data is bound to one open file and won\[aq]t be shared.
|
|
.PP
|
|
This flag is a upper limit for the used memory per open file.
|
|
The buffer will only use memory for data that is downloaded but not not
|
|
yet read.
|
|
If the buffer is empty, only a small amount of memory will be used.
|
|
.PP
|
|
The maximum memory used by rclone for buffering can be up to
|
|
\f[C]\-\-buffer\-size * open files\f[R].
|
|
.SS VFS File Caching
|
|
.PP
|
|
These flags control the VFS file caching options.
|
|
File caching is necessary to make the VFS layer appear compatible with a
|
|
normal file system.
|
|
It can be disabled at the cost of some compatibility.
|
|
.PP
|
|
For example you\[aq]ll need to enable VFS caching if you want to read
|
|
and write simultaneously to a file.
|
|
See below for more details.
|
|
.PP
|
|
Note that the VFS cache is separate from the cache backend and you may
|
|
find that you need one or the other or both.
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
\-\-cache\-dir string Directory rclone will use for caching.
|
|
\-\-vfs\-cache\-mode CacheMode Cache mode off|minimal|writes|full (default off)
|
|
\-\-vfs\-cache\-max\-age duration Max age of objects in the cache. (default 1h0m0s)
|
|
\-\-vfs\-cache\-max\-size SizeSuffix Max total size of objects in the cache. (default off)
|
|
\-\-vfs\-cache\-poll\-interval duration Interval to poll the cache for stale objects. (default 1m0s)
|
|
\-\-vfs\-write\-back duration Time to writeback files after last use when using cache. (default 5s)
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
If run with \f[C]\-vv\f[R] rclone will print the location of the file
|
|
cache.
|
|
The files are stored in the user cache file area which is OS dependent
|
|
but can be controlled with \f[C]\-\-cache\-dir\f[R] or setting the
|
|
appropriate environment variable.
|
|
.PP
|
|
The cache has 4 different modes selected by
|
|
\f[C]\-\-vfs\-cache\-mode\f[R].
|
|
The higher the cache mode the more compatible rclone becomes at the cost
|
|
of using disk space.
|
|
.PP
|
|
Note that files are written back to the remote only when they are closed
|
|
and if they haven\[aq]t been accessed for \-\-vfs\-write\-back second.
|
|
If rclone is quit or dies with files that haven\[aq]t been uploaded,
|
|
these will be uploaded next time rclone is run with the same flags.
|
|
.PP
|
|
If using \f[C]\-\-vfs\-cache\-max\-size\f[R] note that the cache may
|
|
exceed this size for two reasons.
|
|
Firstly because it is only checked every
|
|
\f[C]\-\-vfs\-cache\-poll\-interval\f[R].
|
|
Secondly because open files cannot be evicted from the cache.
|
|
.PP
|
|
You \f[B]should not\f[R] run two copies of rclone using the same VFS
|
|
cache with the same or overlapping remotes if using
|
|
\f[C]\-\-vfs\-cache\-mode > off\f[R].
|
|
This can potentially cause data corruption if you do.
|
|
You can work around this by giving each rclone its own cache hierarchy
|
|
with \f[C]\-\-cache\-dir\f[R].
|
|
You don\[aq]t need to worry about this if the remotes in use don\[aq]t
|
|
overlap.
|
|
.SS \-\-vfs\-cache\-mode off
|
|
.PP
|
|
In this mode (the default) the cache will read directly from the remote
|
|
and write directly to the remote without caching anything on disk.
|
|
.PP
|
|
This will mean some operations are not possible
|
|
.IP \[bu] 2
|
|
Files can\[aq]t be opened for both read AND write
|
|
.IP \[bu] 2
|
|
Files opened for write can\[aq]t be seeked
|
|
.IP \[bu] 2
|
|
Existing files opened for write must have O_TRUNC set
|
|
.IP \[bu] 2
|
|
Files open for read with O_TRUNC will be opened write only
|
|
.IP \[bu] 2
|
|
Files open for write only will behave as if O_TRUNC was supplied
|
|
.IP \[bu] 2
|
|
Open modes O_APPEND, O_TRUNC are ignored
|
|
.IP \[bu] 2
|
|
If an upload fails it can\[aq]t be retried
|
|
.SS \-\-vfs\-cache\-mode minimal
|
|
.PP
|
|
This is very similar to \[dq]off\[dq] except that files opened for read
|
|
AND write will be buffered to disk.
|
|
This means that files opened for write will be a lot more compatible,
|
|
but uses the minimal disk space.
|
|
.PP
|
|
These operations are not possible
|
|
.IP \[bu] 2
|
|
Files opened for write only can\[aq]t be seeked
|
|
.IP \[bu] 2
|
|
Existing files opened for write must have O_TRUNC set
|
|
.IP \[bu] 2
|
|
Files opened for write only will ignore O_APPEND, O_TRUNC
|
|
.IP \[bu] 2
|
|
If an upload fails it can\[aq]t be retried
|
|
.SS \-\-vfs\-cache\-mode writes
|
|
.PP
|
|
In this mode files opened for read only are still read directly from the
|
|
remote, write only and read/write files are buffered to disk first.
|
|
.PP
|
|
This mode should support all normal file system operations.
|
|
.PP
|
|
If an upload fails it will be retried at exponentially increasing
|
|
intervals up to 1 minute.
|
|
.SS \-\-vfs\-cache\-mode full
|
|
.PP
|
|
In this mode all reads and writes are buffered to and from disk.
|
|
When data is read from the remote this is buffered to disk as well.
|
|
.PP
|
|
In this mode the files in the cache will be sparse files and rclone will
|
|
keep track of which bits of the files it has downloaded.
|
|
.PP
|
|
So if an application only reads the starts of each file, then rclone
|
|
will only buffer the start of the file.
|
|
These files will appear to be their full size in the cache, but they
|
|
will be sparse files with only the data that has been downloaded present
|
|
in them.
|
|
.PP
|
|
This mode should support all normal file system operations and is
|
|
otherwise identical to \-\-vfs\-cache\-mode writes.
|
|
.PP
|
|
When reading a file rclone will read \-\-buffer\-size plus
|
|
\-\-vfs\-read\-ahead bytes ahead.
|
|
The \-\-buffer\-size is buffered in memory whereas the
|
|
\-\-vfs\-read\-ahead is buffered on disk.
|
|
.PP
|
|
When using this mode it is recommended that \-\-buffer\-size is not set
|
|
too big and \-\-vfs\-read\-ahead is set large if required.
|
|
.PP
|
|
\f[B]IMPORTANT\f[R] not all file systems support sparse files.
|
|
In particular FAT/exFAT do not.
|
|
Rclone will perform very badly if the cache directory is on a filesystem
|
|
which doesn\[aq]t support sparse files and it will log an ERROR message
|
|
if one is detected.
|
|
.SS VFS Performance
|
|
.PP
|
|
These flags may be used to enable/disable features of the VFS for
|
|
performance or other reasons.
|
|
.PP
|
|
In particular S3 and Swift benefit hugely from the \-\-no\-modtime flag
|
|
(or use \-\-use\-server\-modtime for a slightly different effect) as
|
|
each read of the modification time takes a transaction.
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
\-\-no\-checksum Don\[aq]t compare checksums on up/download.
|
|
\-\-no\-modtime Don\[aq]t read/write the modification time (can speed things up).
|
|
\-\-no\-seek Don\[aq]t allow seeking in files.
|
|
\-\-read\-only Mount read\-only.
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
When rclone reads files from a remote it reads them in chunks.
|
|
This means that rather than requesting the whole file rclone reads the
|
|
chunk specified.
|
|
This is advantageous because some cloud providers account for reads
|
|
being all the data requested, not all the data delivered.
|
|
.PP
|
|
Rclone will keep doubling the chunk size requested starting at
|
|
\-\-vfs\-read\-chunk\-size with a maximum of
|
|
\-\-vfs\-read\-chunk\-size\-limit unless it is set to \[dq]off\[dq] in
|
|
which case there will be no limit.
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
\-\-vfs\-read\-chunk\-size SizeSuffix Read the source objects in chunks. (default 128M)
|
|
\-\-vfs\-read\-chunk\-size\-limit SizeSuffix Max chunk doubling size (default \[dq]off\[dq])
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
Sometimes rclone is delivered reads or writes out of order.
|
|
Rather than seeking rclone will wait a short time for the in sequence
|
|
read or write to come in.
|
|
These flags only come into effect when not using an on disk cache file.
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
\-\-vfs\-read\-wait duration Time to wait for in\-sequence read before seeking. (default 20ms)
|
|
\-\-vfs\-write\-wait duration Time to wait for in\-sequence write before giving error. (default 1s)
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
When using VFS write caching (\-\-vfs\-cache\-mode with value writes or
|
|
full), the global flag \-\-transfers can be set to adjust the number of
|
|
parallel uploads of modified files from cache (the related global flag
|
|
\-\-checkers have no effect on mount).
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
\-\-transfers int Number of file transfers to run in parallel. (default 4)
|
|
\f[R]
|
|
.fi
|
|
.SS VFS Case Sensitivity
|
|
.PP
|
|
Linux file systems are case\-sensitive: two files can differ only by
|
|
case, and the exact case must be used when opening a file.
|
|
.PP
|
|
File systems in modern Windows are case\-insensitive but
|
|
case\-preserving: although existing files can be opened using any case,
|
|
the exact case used to create the file is preserved and available for
|
|
programs to query.
|
|
It is not allowed for two files in the same directory to differ only by
|
|
case.
|
|
.PP
|
|
Usually file systems on macOS are case\-insensitive.
|
|
It is possible to make macOS file systems case\-sensitive but that is
|
|
not the default
|
|
.PP
|
|
The \f[C]\-\-vfs\-case\-insensitive\f[R] mount flag controls how rclone
|
|
handles these two cases.
|
|
If its value is \[dq]false\[dq], rclone passes file names to the mounted
|
|
file system as\-is.
|
|
If the flag is \[dq]true\[dq] (or appears without a value on command
|
|
line), rclone may perform a \[dq]fixup\[dq] as explained below.
|
|
.PP
|
|
The user may specify a file name to open/delete/rename/etc with a case
|
|
different than what is stored on mounted file system.
|
|
If an argument refers to an existing file with exactly the same name,
|
|
then the case of the existing file on the disk will be used.
|
|
However, if a file name with exactly the same name is not found but a
|
|
name differing only by case exists, rclone will transparently fixup the
|
|
name.
|
|
This fixup happens only when an existing file is requested.
|
|
Case sensitivity of file names created anew by rclone is controlled by
|
|
an underlying mounted file system.
|
|
.PP
|
|
Note that case sensitivity of the operating system running rclone (the
|
|
target) may differ from case sensitivity of a file system mounted by
|
|
rclone (the source).
|
|
The flag controls whether \[dq]fixup\[dq] is performed to satisfy the
|
|
target.
|
|
.PP
|
|
If the flag is not provided on the command line, then its default value
|
|
depends on the operating system where rclone runs: \[dq]true\[dq] on
|
|
Windows and macOS, \[dq]false\[dq] otherwise.
|
|
If the flag is provided without a value, then it is \[dq]true\[dq].
|
|
.SS Alternate report of used bytes
|
|
.PP
|
|
Some backends, most notably S3, do not report the amount of bytes used.
|
|
If you need this information to be available when running \f[C]df\f[R]
|
|
on the filesystem, then pass the flag \f[C]\-\-vfs\-used\-is\-size\f[R]
|
|
to rclone.
|
|
With this flag set, instead of relying on the backend to report this
|
|
information, rclone will scan the whole remote similar to
|
|
\f[C]rclone size\f[R] and compute the total used space itself.
|
|
.PP
|
|
\f[I]WARNING.\f[R] Contrary to \f[C]rclone size\f[R], this flag ignores
|
|
filters so that the result is accurate.
|
|
However, this is very inefficient and may cost lots of API calls
|
|
resulting in extra charges.
|
|
Use it as a last resort and only with caching.
|
|
.SS Auth Proxy
|
|
.PP
|
|
If you supply the parameter \f[C]\-\-auth\-proxy /path/to/program\f[R]
|
|
then rclone will use that program to generate backends on the fly which
|
|
then are used to authenticate incoming requests.
|
|
This uses a simple JSON based protocol with input on STDIN and output on
|
|
STDOUT.
|
|
.PP
|
|
\f[B]PLEASE NOTE:\f[R] \f[C]\-\-auth\-proxy\f[R] and
|
|
\f[C]\-\-authorized\-keys\f[R] cannot be used together, if
|
|
\f[C]\-\-auth\-proxy\f[R] is set the authorized keys option will be
|
|
ignored.
|
|
.PP
|
|
There is an example program
|
|
bin/test_proxy.py (https://github.com/rclone/rclone/blob/master/test_proxy.py)
|
|
in the rclone source code.
|
|
.PP
|
|
The program\[aq]s job is to take a \f[C]user\f[R] and \f[C]pass\f[R] on
|
|
the input and turn those into the config for a backend on STDOUT in JSON
|
|
format.
|
|
This config will have any default parameters for the backend added, but
|
|
it won\[aq]t use configuration from environment variables or command
|
|
line options \- it is the job of the proxy program to make a complete
|
|
config.
|
|
.PP
|
|
This config generated must have this extra parameter \- \f[C]_root\f[R]
|
|
\- root to use for the backend
|
|
.PP
|
|
And it may have this parameter \- \f[C]_obscure\f[R] \- comma separated
|
|
strings for parameters to obscure
|
|
.PP
|
|
If password authentication was used by the client, input to the proxy
|
|
process (on STDIN) would look similar to this:
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
{
|
|
\[dq]user\[dq]: \[dq]me\[dq],
|
|
\[dq]pass\[dq]: \[dq]mypassword\[dq]
|
|
}
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
If public\-key authentication was used by the client, input to the proxy
|
|
process (on STDIN) would look similar to this:
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
{
|
|
\[dq]user\[dq]: \[dq]me\[dq],
|
|
\[dq]public_key\[dq]: \[dq]AAAAB3NzaC1yc2EAAAADAQABAAABAQDuwESFdAe14hVS6omeyX7edc...JQdf\[dq]
|
|
}
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
And as an example return this on STDOUT
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
{
|
|
\[dq]type\[dq]: \[dq]sftp\[dq],
|
|
\[dq]_root\[dq]: \[dq]\[dq],
|
|
\[dq]_obscure\[dq]: \[dq]pass\[dq],
|
|
\[dq]user\[dq]: \[dq]me\[dq],
|
|
\[dq]pass\[dq]: \[dq]mypassword\[dq],
|
|
\[dq]host\[dq]: \[dq]sftp.example.com\[dq]
|
|
}
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
This would mean that an SFTP backend would be created on the fly for the
|
|
\f[C]user\f[R] and \f[C]pass\f[R]/\f[C]public_key\f[R] returned in the
|
|
output to the host given.
|
|
Note that since \f[C]_obscure\f[R] is set to \f[C]pass\f[R], rclone will
|
|
obscure the \f[C]pass\f[R] parameter before creating the backend (which
|
|
is required for sftp backends).
|
|
.PP
|
|
The program can manipulate the supplied \f[C]user\f[R] in any way, for
|
|
example to make proxy to many different sftp backends, you could make
|
|
the \f[C]user\f[R] be \f[C]user\[at]example.com\f[R] and then set the
|
|
\f[C]host\f[R] to \f[C]example.com\f[R] in the output and the user to
|
|
\f[C]user\f[R].
|
|
For security you\[aq]d probably want to restrict the \f[C]host\f[R] to a
|
|
limited list.
|
|
.PP
|
|
Note that an internal cache is keyed on \f[C]user\f[R] so only use that
|
|
for configuration, don\[aq]t use \f[C]pass\f[R] or \f[C]public_key\f[R].
|
|
This also means that if a user\[aq]s password or public\-key is changed
|
|
the cache will need to expire (which takes 5 mins) before it takes
|
|
effect.
|
|
.PP
|
|
This can be used to build general purpose proxies to any kind of backend
|
|
that rclone supports.
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone serve sftp remote:path [flags]
|
|
\f[R]
|
|
.fi
|
|
.SS Options
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
\-\-addr string IPaddress:Port or :Port to bind server to. (default \[dq]localhost:2022\[dq])
|
|
\-\-auth\-proxy string A program to use to create the backend from the auth.
|
|
\-\-authorized\-keys string Authorized keys file (default \[dq]\[ti]/.ssh/authorized_keys\[dq])
|
|
\-\-dir\-cache\-time duration Time to cache directory entries for. (default 5m0s)
|
|
\-\-dir\-perms FileMode Directory permissions (default 0777)
|
|
\-\-file\-perms FileMode File permissions (default 0666)
|
|
\-\-gid uint32 Override the gid field set by the filesystem. Not supported on Windows. (default 1000)
|
|
\-h, \-\-help help for sftp
|
|
\-\-key stringArray SSH private host key file (Can be multi\-valued, leave blank to auto generate)
|
|
\-\-no\-auth Allow connections with no authentication if set.
|
|
\-\-no\-checksum Don\[aq]t compare checksums on up/download.
|
|
\-\-no\-modtime Don\[aq]t read/write the modification time (can speed things up).
|
|
\-\-no\-seek Don\[aq]t allow seeking in files.
|
|
\-\-pass string Password for authentication.
|
|
\-\-poll\-interval duration Time to wait between polling for changes. Must be smaller than dir\-cache\-time. Only on supported remotes. Set to 0 to disable. (default 1m0s)
|
|
\-\-read\-only Mount read\-only.
|
|
\-\-uid uint32 Override the uid field set by the filesystem. Not supported on Windows. (default 1000)
|
|
\-\-umask int Override the permission bits set by the filesystem. Not supported on Windows. (default 2)
|
|
\-\-user string User name for authentication.
|
|
\-\-vfs\-cache\-max\-age duration Max age of objects in the cache. (default 1h0m0s)
|
|
\-\-vfs\-cache\-max\-size SizeSuffix Max total size of objects in the cache. (default off)
|
|
\-\-vfs\-cache\-mode CacheMode Cache mode off|minimal|writes|full (default off)
|
|
\-\-vfs\-cache\-poll\-interval duration Interval to poll the cache for stale objects. (default 1m0s)
|
|
\-\-vfs\-case\-insensitive If a file name not found, find a case insensitive match.
|
|
\-\-vfs\-read\-ahead SizeSuffix Extra read ahead over \-\-buffer\-size when using cache\-mode full.
|
|
\-\-vfs\-read\-chunk\-size SizeSuffix Read the source objects in chunks. (default 128M)
|
|
\-\-vfs\-read\-chunk\-size\-limit SizeSuffix If greater than \-\-vfs\-read\-chunk\-size, double the chunk size after each chunk read, until the limit is reached. \[aq]off\[aq] is unlimited. (default off)
|
|
\-\-vfs\-read\-wait duration Time to wait for in\-sequence read before seeking. (default 20ms)
|
|
\-\-vfs\-used\-is\-size rclone size Use the rclone size algorithm for Used size.
|
|
\-\-vfs\-write\-back duration Time to writeback files after last use when using cache. (default 5s)
|
|
\-\-vfs\-write\-wait duration Time to wait for in\-sequence write before giving error. (default 1s)
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
See the global flags page (https://rclone.org/flags/) for global options
|
|
not listed here.
|
|
.SS SEE ALSO
|
|
.IP \[bu] 2
|
|
rclone serve (https://rclone.org/commands/rclone_serve/) \- Serve a
|
|
remote over a protocol.
|
|
.SH rclone serve webdav
|
|
.PP
|
|
Serve remote:path over webdav.
|
|
.SS Synopsis
|
|
.PP
|
|
rclone serve webdav implements a basic webdav server to serve the remote
|
|
over HTTP via the webdav protocol.
|
|
This can be viewed with a webdav client, through a web browser, or you
|
|
can make a remote of type webdav to read and write it.
|
|
.SS Webdav options
|
|
.SS \-\-etag\-hash
|
|
.PP
|
|
This controls the ETag header.
|
|
Without this flag the ETag will be based on the ModTime and Size of the
|
|
object.
|
|
.PP
|
|
If this flag is set to \[dq]auto\[dq] then rclone will choose the first
|
|
supported hash on the backend or you can use a named hash such as
|
|
\[dq]MD5\[dq] or \[dq]SHA\-1\[dq].
|
|
.PP
|
|
Use \[dq]rclone hashsum\[dq] to see the full list.
|
|
.SS Server options
|
|
.PP
|
|
Use \-\-addr to specify which IP address and port the server should
|
|
listen on, e.g.
|
|
\-\-addr 1.2.3.4:8000 or \-\-addr :8080 to listen to all IPs.
|
|
By default it only listens on localhost.
|
|
You can use port :0 to let the OS choose an available port.
|
|
.PP
|
|
If you set \-\-addr to listen on a public or LAN accessible IP address
|
|
then using Authentication is advised \- see the next section for info.
|
|
.PP
|
|
\-\-server\-read\-timeout and \-\-server\-write\-timeout can be used to
|
|
control the timeouts on the server.
|
|
Note that this is the total time for a transfer.
|
|
.PP
|
|
\-\-max\-header\-bytes controls the maximum number of bytes the server
|
|
will accept in the HTTP header.
|
|
.PP
|
|
\-\-baseurl controls the URL prefix that rclone serves from.
|
|
By default rclone will serve from the root.
|
|
If you used \-\-baseurl \[dq]/rclone\[dq] then rclone would serve from a
|
|
URL starting with \[dq]/rclone/\[dq].
|
|
This is useful if you wish to proxy rclone serve.
|
|
Rclone automatically inserts leading and trailing \[dq]/\[dq] on
|
|
\-\-baseurl, so \-\-baseurl \[dq]rclone\[dq], \-\-baseurl
|
|
\[dq]/rclone\[dq] and \-\-baseurl \[dq]/rclone/\[dq] are all treated
|
|
identically.
|
|
.PP
|
|
\-\-template allows a user to specify a custom markup template for http
|
|
and webdav serve functions.
|
|
The server exports the following markup to be used within the template
|
|
to server pages:
|
|
.PP
|
|
.TS
|
|
tab(@);
|
|
lw(35.0n) lw(35.0n).
|
|
T{
|
|
Parameter
|
|
T}@T{
|
|
Description
|
|
T}
|
|
_
|
|
T{
|
|
\&.Name
|
|
T}@T{
|
|
The full path of a file/directory.
|
|
T}
|
|
T{
|
|
\&.Title
|
|
T}@T{
|
|
Directory listing of .Name
|
|
T}
|
|
T{
|
|
\&.Sort
|
|
T}@T{
|
|
The current sort used.
|
|
This is changeable via ?sort= parameter
|
|
T}
|
|
T{
|
|
T}@T{
|
|
Sort Options: namedirfirst,name,size,time (default namedirfirst)
|
|
T}
|
|
T{
|
|
\&.Order
|
|
T}@T{
|
|
The current ordering used.
|
|
This is changeable via ?order= parameter
|
|
T}
|
|
T{
|
|
T}@T{
|
|
Order Options: asc,desc (default asc)
|
|
T}
|
|
T{
|
|
\&.Query
|
|
T}@T{
|
|
Currently unused.
|
|
T}
|
|
T{
|
|
\&.Breadcrumb
|
|
T}@T{
|
|
Allows for creating a relative navigation
|
|
T}
|
|
T{
|
|
\-\- .Link
|
|
T}@T{
|
|
The relative to the root link of the Text.
|
|
T}
|
|
T{
|
|
\-\- .Text
|
|
T}@T{
|
|
The Name of the directory.
|
|
T}
|
|
T{
|
|
\&.Entries
|
|
T}@T{
|
|
Information about a specific file/directory.
|
|
T}
|
|
T{
|
|
\-\- .URL
|
|
T}@T{
|
|
The \[aq]url\[aq] of an entry.
|
|
T}
|
|
T{
|
|
\-\- .Leaf
|
|
T}@T{
|
|
Currently same as \[aq]URL\[aq] but intended to be \[aq]just\[aq] the
|
|
name.
|
|
T}
|
|
T{
|
|
\-\- .IsDir
|
|
T}@T{
|
|
Boolean for if an entry is a directory or not.
|
|
T}
|
|
T{
|
|
\-\- .Size
|
|
T}@T{
|
|
Size in Bytes of the entry.
|
|
T}
|
|
T{
|
|
\-\- .ModTime
|
|
T}@T{
|
|
The UTC timestamp of an entry.
|
|
T}
|
|
.TE
|
|
.SS Authentication
|
|
.PP
|
|
By default this will serve files without needing a login.
|
|
.PP
|
|
You can either use an htpasswd file which can take lots of users, or set
|
|
a single username and password with the \-\-user and \-\-pass flags.
|
|
.PP
|
|
Use \-\-htpasswd /path/to/htpasswd to provide an htpasswd file.
|
|
This is in standard apache format and supports MD5, SHA1 and BCrypt for
|
|
basic authentication.
|
|
Bcrypt is recommended.
|
|
.PP
|
|
To create an htpasswd file:
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
touch htpasswd
|
|
htpasswd \-B htpasswd user
|
|
htpasswd \-B htpasswd anotherUser
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
The password file can be updated while rclone is running.
|
|
.PP
|
|
Use \-\-realm to set the authentication realm.
|
|
.SS SSL/TLS
|
|
.PP
|
|
By default this will serve over http.
|
|
If you want you can serve over https.
|
|
You will need to supply the \-\-cert and \-\-key flags.
|
|
If you wish to do client side certificate validation then you will need
|
|
to supply \-\-client\-ca also.
|
|
.PP
|
|
\-\-cert should be either a PEM encoded certificate or a concatenation
|
|
of that with the CA certificate.
|
|
\-\-key should be the PEM encoded private key and \-\-client\-ca should
|
|
be the PEM encoded client certificate authority certificate.
|
|
.SS VFS \- Virtual File System
|
|
.PP
|
|
This command uses the VFS layer.
|
|
This adapts the cloud storage objects that rclone uses into something
|
|
which looks much more like a disk filing system.
|
|
.PP
|
|
Cloud storage objects have lots of properties which aren\[aq]t like disk
|
|
files \- you can\[aq]t extend them or write to the middle of them, so
|
|
the VFS layer has to deal with that.
|
|
Because there is no one right way of doing this there are various
|
|
options explained below.
|
|
.PP
|
|
The VFS layer also implements a directory cache \- this caches info
|
|
about files and directories (but not the data) in memory.
|
|
.SS VFS Directory Cache
|
|
.PP
|
|
Using the \f[C]\-\-dir\-cache\-time\f[R] flag, you can control how long
|
|
a directory should be considered up to date and not refreshed from the
|
|
backend.
|
|
Changes made through the mount will appear immediately or invalidate the
|
|
cache.
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
\-\-dir\-cache\-time duration Time to cache directory entries for. (default 5m0s)
|
|
\-\-poll\-interval duration Time to wait between polling for changes.
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
However, changes made directly on the cloud storage by the web interface
|
|
or a different copy of rclone will only be picked up once the directory
|
|
cache expires if the backend configured does not support polling for
|
|
changes.
|
|
If the backend supports polling, changes will be picked up within the
|
|
polling interval.
|
|
.PP
|
|
You can send a \f[C]SIGHUP\f[R] signal to rclone for it to flush all
|
|
directory caches, regardless of how old they are.
|
|
Assuming only one rclone instance is running, you can reset the cache
|
|
like this:
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
kill \-SIGHUP $(pidof rclone)
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
If you configure rclone with a remote control (/rc) then you can use
|
|
rclone rc to flush the whole directory cache:
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone rc vfs/forget
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
Or individual files or directories:
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone rc vfs/forget file=path/to/file dir=path/to/dir
|
|
\f[R]
|
|
.fi
|
|
.SS VFS File Buffering
|
|
.PP
|
|
The \f[C]\-\-buffer\-size\f[R] flag determines the amount of memory,
|
|
that will be used to buffer data in advance.
|
|
.PP
|
|
Each open file will try to keep the specified amount of data in memory
|
|
at all times.
|
|
The buffered data is bound to one open file and won\[aq]t be shared.
|
|
.PP
|
|
This flag is a upper limit for the used memory per open file.
|
|
The buffer will only use memory for data that is downloaded but not not
|
|
yet read.
|
|
If the buffer is empty, only a small amount of memory will be used.
|
|
.PP
|
|
The maximum memory used by rclone for buffering can be up to
|
|
\f[C]\-\-buffer\-size * open files\f[R].
|
|
.SS VFS File Caching
|
|
.PP
|
|
These flags control the VFS file caching options.
|
|
File caching is necessary to make the VFS layer appear compatible with a
|
|
normal file system.
|
|
It can be disabled at the cost of some compatibility.
|
|
.PP
|
|
For example you\[aq]ll need to enable VFS caching if you want to read
|
|
and write simultaneously to a file.
|
|
See below for more details.
|
|
.PP
|
|
Note that the VFS cache is separate from the cache backend and you may
|
|
find that you need one or the other or both.
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
\-\-cache\-dir string Directory rclone will use for caching.
|
|
\-\-vfs\-cache\-mode CacheMode Cache mode off|minimal|writes|full (default off)
|
|
\-\-vfs\-cache\-max\-age duration Max age of objects in the cache. (default 1h0m0s)
|
|
\-\-vfs\-cache\-max\-size SizeSuffix Max total size of objects in the cache. (default off)
|
|
\-\-vfs\-cache\-poll\-interval duration Interval to poll the cache for stale objects. (default 1m0s)
|
|
\-\-vfs\-write\-back duration Time to writeback files after last use when using cache. (default 5s)
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
If run with \f[C]\-vv\f[R] rclone will print the location of the file
|
|
cache.
|
|
The files are stored in the user cache file area which is OS dependent
|
|
but can be controlled with \f[C]\-\-cache\-dir\f[R] or setting the
|
|
appropriate environment variable.
|
|
.PP
|
|
The cache has 4 different modes selected by
|
|
\f[C]\-\-vfs\-cache\-mode\f[R].
|
|
The higher the cache mode the more compatible rclone becomes at the cost
|
|
of using disk space.
|
|
.PP
|
|
Note that files are written back to the remote only when they are closed
|
|
and if they haven\[aq]t been accessed for \-\-vfs\-write\-back second.
|
|
If rclone is quit or dies with files that haven\[aq]t been uploaded,
|
|
these will be uploaded next time rclone is run with the same flags.
|
|
.PP
|
|
If using \f[C]\-\-vfs\-cache\-max\-size\f[R] note that the cache may
|
|
exceed this size for two reasons.
|
|
Firstly because it is only checked every
|
|
\f[C]\-\-vfs\-cache\-poll\-interval\f[R].
|
|
Secondly because open files cannot be evicted from the cache.
|
|
.PP
|
|
You \f[B]should not\f[R] run two copies of rclone using the same VFS
|
|
cache with the same or overlapping remotes if using
|
|
\f[C]\-\-vfs\-cache\-mode > off\f[R].
|
|
This can potentially cause data corruption if you do.
|
|
You can work around this by giving each rclone its own cache hierarchy
|
|
with \f[C]\-\-cache\-dir\f[R].
|
|
You don\[aq]t need to worry about this if the remotes in use don\[aq]t
|
|
overlap.
|
|
.SS \-\-vfs\-cache\-mode off
|
|
.PP
|
|
In this mode (the default) the cache will read directly from the remote
|
|
and write directly to the remote without caching anything on disk.
|
|
.PP
|
|
This will mean some operations are not possible
|
|
.IP \[bu] 2
|
|
Files can\[aq]t be opened for both read AND write
|
|
.IP \[bu] 2
|
|
Files opened for write can\[aq]t be seeked
|
|
.IP \[bu] 2
|
|
Existing files opened for write must have O_TRUNC set
|
|
.IP \[bu] 2
|
|
Files open for read with O_TRUNC will be opened write only
|
|
.IP \[bu] 2
|
|
Files open for write only will behave as if O_TRUNC was supplied
|
|
.IP \[bu] 2
|
|
Open modes O_APPEND, O_TRUNC are ignored
|
|
.IP \[bu] 2
|
|
If an upload fails it can\[aq]t be retried
|
|
.SS \-\-vfs\-cache\-mode minimal
|
|
.PP
|
|
This is very similar to \[dq]off\[dq] except that files opened for read
|
|
AND write will be buffered to disk.
|
|
This means that files opened for write will be a lot more compatible,
|
|
but uses the minimal disk space.
|
|
.PP
|
|
These operations are not possible
|
|
.IP \[bu] 2
|
|
Files opened for write only can\[aq]t be seeked
|
|
.IP \[bu] 2
|
|
Existing files opened for write must have O_TRUNC set
|
|
.IP \[bu] 2
|
|
Files opened for write only will ignore O_APPEND, O_TRUNC
|
|
.IP \[bu] 2
|
|
If an upload fails it can\[aq]t be retried
|
|
.SS \-\-vfs\-cache\-mode writes
|
|
.PP
|
|
In this mode files opened for read only are still read directly from the
|
|
remote, write only and read/write files are buffered to disk first.
|
|
.PP
|
|
This mode should support all normal file system operations.
|
|
.PP
|
|
If an upload fails it will be retried at exponentially increasing
|
|
intervals up to 1 minute.
|
|
.SS \-\-vfs\-cache\-mode full
|
|
.PP
|
|
In this mode all reads and writes are buffered to and from disk.
|
|
When data is read from the remote this is buffered to disk as well.
|
|
.PP
|
|
In this mode the files in the cache will be sparse files and rclone will
|
|
keep track of which bits of the files it has downloaded.
|
|
.PP
|
|
So if an application only reads the starts of each file, then rclone
|
|
will only buffer the start of the file.
|
|
These files will appear to be their full size in the cache, but they
|
|
will be sparse files with only the data that has been downloaded present
|
|
in them.
|
|
.PP
|
|
This mode should support all normal file system operations and is
|
|
otherwise identical to \-\-vfs\-cache\-mode writes.
|
|
.PP
|
|
When reading a file rclone will read \-\-buffer\-size plus
|
|
\-\-vfs\-read\-ahead bytes ahead.
|
|
The \-\-buffer\-size is buffered in memory whereas the
|
|
\-\-vfs\-read\-ahead is buffered on disk.
|
|
.PP
|
|
When using this mode it is recommended that \-\-buffer\-size is not set
|
|
too big and \-\-vfs\-read\-ahead is set large if required.
|
|
.PP
|
|
\f[B]IMPORTANT\f[R] not all file systems support sparse files.
|
|
In particular FAT/exFAT do not.
|
|
Rclone will perform very badly if the cache directory is on a filesystem
|
|
which doesn\[aq]t support sparse files and it will log an ERROR message
|
|
if one is detected.
|
|
.SS VFS Performance
|
|
.PP
|
|
These flags may be used to enable/disable features of the VFS for
|
|
performance or other reasons.
|
|
.PP
|
|
In particular S3 and Swift benefit hugely from the \-\-no\-modtime flag
|
|
(or use \-\-use\-server\-modtime for a slightly different effect) as
|
|
each read of the modification time takes a transaction.
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
\-\-no\-checksum Don\[aq]t compare checksums on up/download.
|
|
\-\-no\-modtime Don\[aq]t read/write the modification time (can speed things up).
|
|
\-\-no\-seek Don\[aq]t allow seeking in files.
|
|
\-\-read\-only Mount read\-only.
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
When rclone reads files from a remote it reads them in chunks.
|
|
This means that rather than requesting the whole file rclone reads the
|
|
chunk specified.
|
|
This is advantageous because some cloud providers account for reads
|
|
being all the data requested, not all the data delivered.
|
|
.PP
|
|
Rclone will keep doubling the chunk size requested starting at
|
|
\-\-vfs\-read\-chunk\-size with a maximum of
|
|
\-\-vfs\-read\-chunk\-size\-limit unless it is set to \[dq]off\[dq] in
|
|
which case there will be no limit.
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
\-\-vfs\-read\-chunk\-size SizeSuffix Read the source objects in chunks. (default 128M)
|
|
\-\-vfs\-read\-chunk\-size\-limit SizeSuffix Max chunk doubling size (default \[dq]off\[dq])
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
Sometimes rclone is delivered reads or writes out of order.
|
|
Rather than seeking rclone will wait a short time for the in sequence
|
|
read or write to come in.
|
|
These flags only come into effect when not using an on disk cache file.
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
\-\-vfs\-read\-wait duration Time to wait for in\-sequence read before seeking. (default 20ms)
|
|
\-\-vfs\-write\-wait duration Time to wait for in\-sequence write before giving error. (default 1s)
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
When using VFS write caching (\-\-vfs\-cache\-mode with value writes or
|
|
full), the global flag \-\-transfers can be set to adjust the number of
|
|
parallel uploads of modified files from cache (the related global flag
|
|
\-\-checkers have no effect on mount).
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
\-\-transfers int Number of file transfers to run in parallel. (default 4)
|
|
\f[R]
|
|
.fi
|
|
.SS VFS Case Sensitivity
|
|
.PP
|
|
Linux file systems are case\-sensitive: two files can differ only by
|
|
case, and the exact case must be used when opening a file.
|
|
.PP
|
|
File systems in modern Windows are case\-insensitive but
|
|
case\-preserving: although existing files can be opened using any case,
|
|
the exact case used to create the file is preserved and available for
|
|
programs to query.
|
|
It is not allowed for two files in the same directory to differ only by
|
|
case.
|
|
.PP
|
|
Usually file systems on macOS are case\-insensitive.
|
|
It is possible to make macOS file systems case\-sensitive but that is
|
|
not the default
|
|
.PP
|
|
The \f[C]\-\-vfs\-case\-insensitive\f[R] mount flag controls how rclone
|
|
handles these two cases.
|
|
If its value is \[dq]false\[dq], rclone passes file names to the mounted
|
|
file system as\-is.
|
|
If the flag is \[dq]true\[dq] (or appears without a value on command
|
|
line), rclone may perform a \[dq]fixup\[dq] as explained below.
|
|
.PP
|
|
The user may specify a file name to open/delete/rename/etc with a case
|
|
different than what is stored on mounted file system.
|
|
If an argument refers to an existing file with exactly the same name,
|
|
then the case of the existing file on the disk will be used.
|
|
However, if a file name with exactly the same name is not found but a
|
|
name differing only by case exists, rclone will transparently fixup the
|
|
name.
|
|
This fixup happens only when an existing file is requested.
|
|
Case sensitivity of file names created anew by rclone is controlled by
|
|
an underlying mounted file system.
|
|
.PP
|
|
Note that case sensitivity of the operating system running rclone (the
|
|
target) may differ from case sensitivity of a file system mounted by
|
|
rclone (the source).
|
|
The flag controls whether \[dq]fixup\[dq] is performed to satisfy the
|
|
target.
|
|
.PP
|
|
If the flag is not provided on the command line, then its default value
|
|
depends on the operating system where rclone runs: \[dq]true\[dq] on
|
|
Windows and macOS, \[dq]false\[dq] otherwise.
|
|
If the flag is provided without a value, then it is \[dq]true\[dq].
|
|
.SS Alternate report of used bytes
|
|
.PP
|
|
Some backends, most notably S3, do not report the amount of bytes used.
|
|
If you need this information to be available when running \f[C]df\f[R]
|
|
on the filesystem, then pass the flag \f[C]\-\-vfs\-used\-is\-size\f[R]
|
|
to rclone.
|
|
With this flag set, instead of relying on the backend to report this
|
|
information, rclone will scan the whole remote similar to
|
|
\f[C]rclone size\f[R] and compute the total used space itself.
|
|
.PP
|
|
\f[I]WARNING.\f[R] Contrary to \f[C]rclone size\f[R], this flag ignores
|
|
filters so that the result is accurate.
|
|
However, this is very inefficient and may cost lots of API calls
|
|
resulting in extra charges.
|
|
Use it as a last resort and only with caching.
|
|
.SS Auth Proxy
|
|
.PP
|
|
If you supply the parameter \f[C]\-\-auth\-proxy /path/to/program\f[R]
|
|
then rclone will use that program to generate backends on the fly which
|
|
then are used to authenticate incoming requests.
|
|
This uses a simple JSON based protocol with input on STDIN and output on
|
|
STDOUT.
|
|
.PP
|
|
\f[B]PLEASE NOTE:\f[R] \f[C]\-\-auth\-proxy\f[R] and
|
|
\f[C]\-\-authorized\-keys\f[R] cannot be used together, if
|
|
\f[C]\-\-auth\-proxy\f[R] is set the authorized keys option will be
|
|
ignored.
|
|
.PP
|
|
There is an example program
|
|
bin/test_proxy.py (https://github.com/rclone/rclone/blob/master/test_proxy.py)
|
|
in the rclone source code.
|
|
.PP
|
|
The program\[aq]s job is to take a \f[C]user\f[R] and \f[C]pass\f[R] on
|
|
the input and turn those into the config for a backend on STDOUT in JSON
|
|
format.
|
|
This config will have any default parameters for the backend added, but
|
|
it won\[aq]t use configuration from environment variables or command
|
|
line options \- it is the job of the proxy program to make a complete
|
|
config.
|
|
.PP
|
|
This config generated must have this extra parameter \- \f[C]_root\f[R]
|
|
\- root to use for the backend
|
|
.PP
|
|
And it may have this parameter \- \f[C]_obscure\f[R] \- comma separated
|
|
strings for parameters to obscure
|
|
.PP
|
|
If password authentication was used by the client, input to the proxy
|
|
process (on STDIN) would look similar to this:
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
{
|
|
\[dq]user\[dq]: \[dq]me\[dq],
|
|
\[dq]pass\[dq]: \[dq]mypassword\[dq]
|
|
}
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
If public\-key authentication was used by the client, input to the proxy
|
|
process (on STDIN) would look similar to this:
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
{
|
|
\[dq]user\[dq]: \[dq]me\[dq],
|
|
\[dq]public_key\[dq]: \[dq]AAAAB3NzaC1yc2EAAAADAQABAAABAQDuwESFdAe14hVS6omeyX7edc...JQdf\[dq]
|
|
}
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
And as an example return this on STDOUT
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
{
|
|
\[dq]type\[dq]: \[dq]sftp\[dq],
|
|
\[dq]_root\[dq]: \[dq]\[dq],
|
|
\[dq]_obscure\[dq]: \[dq]pass\[dq],
|
|
\[dq]user\[dq]: \[dq]me\[dq],
|
|
\[dq]pass\[dq]: \[dq]mypassword\[dq],
|
|
\[dq]host\[dq]: \[dq]sftp.example.com\[dq]
|
|
}
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
This would mean that an SFTP backend would be created on the fly for the
|
|
\f[C]user\f[R] and \f[C]pass\f[R]/\f[C]public_key\f[R] returned in the
|
|
output to the host given.
|
|
Note that since \f[C]_obscure\f[R] is set to \f[C]pass\f[R], rclone will
|
|
obscure the \f[C]pass\f[R] parameter before creating the backend (which
|
|
is required for sftp backends).
|
|
.PP
|
|
The program can manipulate the supplied \f[C]user\f[R] in any way, for
|
|
example to make proxy to many different sftp backends, you could make
|
|
the \f[C]user\f[R] be \f[C]user\[at]example.com\f[R] and then set the
|
|
\f[C]host\f[R] to \f[C]example.com\f[R] in the output and the user to
|
|
\f[C]user\f[R].
|
|
For security you\[aq]d probably want to restrict the \f[C]host\f[R] to a
|
|
limited list.
|
|
.PP
|
|
Note that an internal cache is keyed on \f[C]user\f[R] so only use that
|
|
for configuration, don\[aq]t use \f[C]pass\f[R] or \f[C]public_key\f[R].
|
|
This also means that if a user\[aq]s password or public\-key is changed
|
|
the cache will need to expire (which takes 5 mins) before it takes
|
|
effect.
|
|
.PP
|
|
This can be used to build general purpose proxies to any kind of backend
|
|
that rclone supports.
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone serve webdav remote:path [flags]
|
|
\f[R]
|
|
.fi
|
|
.SS Options
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
\-\-addr string IPaddress:Port or :Port to bind server to. (default \[dq]localhost:8080\[dq])
|
|
\-\-auth\-proxy string A program to use to create the backend from the auth.
|
|
\-\-baseurl string Prefix for URLs \- leave blank for root.
|
|
\-\-cert string SSL PEM key (concatenation of certificate and CA certificate)
|
|
\-\-client\-ca string Client certificate authority to verify clients with
|
|
\-\-dir\-cache\-time duration Time to cache directory entries for. (default 5m0s)
|
|
\-\-dir\-perms FileMode Directory permissions (default 0777)
|
|
\-\-disable\-dir\-list Disable HTML directory list on GET request for a directory
|
|
\-\-etag\-hash string Which hash to use for the ETag, or auto or blank for off
|
|
\-\-file\-perms FileMode File permissions (default 0666)
|
|
\-\-gid uint32 Override the gid field set by the filesystem. Not supported on Windows. (default 1000)
|
|
\-h, \-\-help help for webdav
|
|
\-\-htpasswd string htpasswd file \- if not provided no authentication is done
|
|
\-\-key string SSL PEM Private key
|
|
\-\-max\-header\-bytes int Maximum size of request header (default 4096)
|
|
\-\-no\-checksum Don\[aq]t compare checksums on up/download.
|
|
\-\-no\-modtime Don\[aq]t read/write the modification time (can speed things up).
|
|
\-\-no\-seek Don\[aq]t allow seeking in files.
|
|
\-\-pass string Password for authentication.
|
|
\-\-poll\-interval duration Time to wait between polling for changes. Must be smaller than dir\-cache\-time. Only on supported remotes. Set to 0 to disable. (default 1m0s)
|
|
\-\-read\-only Mount read\-only.
|
|
\-\-realm string realm for authentication (default \[dq]rclone\[dq])
|
|
\-\-server\-read\-timeout duration Timeout for server reading data (default 1h0m0s)
|
|
\-\-server\-write\-timeout duration Timeout for server writing data (default 1h0m0s)
|
|
\-\-template string User Specified Template.
|
|
\-\-uid uint32 Override the uid field set by the filesystem. Not supported on Windows. (default 1000)
|
|
\-\-umask int Override the permission bits set by the filesystem. Not supported on Windows. (default 2)
|
|
\-\-user string User name for authentication.
|
|
\-\-vfs\-cache\-max\-age duration Max age of objects in the cache. (default 1h0m0s)
|
|
\-\-vfs\-cache\-max\-size SizeSuffix Max total size of objects in the cache. (default off)
|
|
\-\-vfs\-cache\-mode CacheMode Cache mode off|minimal|writes|full (default off)
|
|
\-\-vfs\-cache\-poll\-interval duration Interval to poll the cache for stale objects. (default 1m0s)
|
|
\-\-vfs\-case\-insensitive If a file name not found, find a case insensitive match.
|
|
\-\-vfs\-read\-ahead SizeSuffix Extra read ahead over \-\-buffer\-size when using cache\-mode full.
|
|
\-\-vfs\-read\-chunk\-size SizeSuffix Read the source objects in chunks. (default 128M)
|
|
\-\-vfs\-read\-chunk\-size\-limit SizeSuffix If greater than \-\-vfs\-read\-chunk\-size, double the chunk size after each chunk read, until the limit is reached. \[aq]off\[aq] is unlimited. (default off)
|
|
\-\-vfs\-read\-wait duration Time to wait for in\-sequence read before seeking. (default 20ms)
|
|
\-\-vfs\-used\-is\-size rclone size Use the rclone size algorithm for Used size.
|
|
\-\-vfs\-write\-back duration Time to writeback files after last use when using cache. (default 5s)
|
|
\-\-vfs\-write\-wait duration Time to wait for in\-sequence write before giving error. (default 1s)
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
See the global flags page (https://rclone.org/flags/) for global options
|
|
not listed here.
|
|
.SS SEE ALSO
|
|
.IP \[bu] 2
|
|
rclone serve (https://rclone.org/commands/rclone_serve/) \- Serve a
|
|
remote over a protocol.
|
|
.SH rclone settier
|
|
.PP
|
|
Changes storage class/tier of objects in remote.
|
|
.SS Synopsis
|
|
.PP
|
|
rclone settier changes storage tier or class at remote if supported.
|
|
Few cloud storage services provides different storage classes on
|
|
objects, for example AWS S3 and Glacier, Azure Blob storage \- Hot, Cool
|
|
and Archive, Google Cloud Storage, Regional Storage, Nearline, Coldline
|
|
etc.
|
|
.PP
|
|
Note that, certain tier changes make objects not available to access
|
|
immediately.
|
|
For example tiering to archive in azure blob storage makes objects in
|
|
frozen state, user can restore by setting tier to Hot/Cool, similarly S3
|
|
to Glacier makes object inaccessible.true
|
|
.PP
|
|
You can use it to tier single object
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone settier Cool remote:path/file
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
Or use rclone filters to set tier on only specific files
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone \-\-include \[dq]*.txt\[dq] settier Hot remote:path/dir
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
Or just provide remote directory and all files in directory will be
|
|
tiered
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone settier tier remote:path/dir
|
|
\f[R]
|
|
.fi
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone settier tier remote:path [flags]
|
|
\f[R]
|
|
.fi
|
|
.SS Options
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
\-h, \-\-help help for settier
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
See the global flags page (https://rclone.org/flags/) for global options
|
|
not listed here.
|
|
.SS SEE ALSO
|
|
.IP \[bu] 2
|
|
rclone (https://rclone.org/commands/rclone/) \- Show help for rclone
|
|
commands, flags and backends.
|
|
.SH rclone test
|
|
.PP
|
|
Run a test command
|
|
.SS Synopsis
|
|
.PP
|
|
Rclone test is used to run test commands.
|
|
.PP
|
|
Select which test comand you want with the subcommand, eg
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone test memory remote:
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
Each subcommand has its own options which you can see in their help.
|
|
.PP
|
|
\f[B]NB\f[R] Be careful running these commands, they may do strange
|
|
things so reading their documentation first is recommended.
|
|
.SS Options
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
\-h, \-\-help help for test
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
See the global flags page (https://rclone.org/flags/) for global options
|
|
not listed here.
|
|
.SS SEE ALSO
|
|
.IP \[bu] 2
|
|
rclone (https://rclone.org/commands/rclone/) \- Show help for rclone
|
|
commands, flags and backends.
|
|
.IP \[bu] 2
|
|
rclone test
|
|
histogram (https://rclone.org/commands/rclone_test_histogram/) \- Makes
|
|
a histogram of file name characters.
|
|
.IP \[bu] 2
|
|
rclone test info (https://rclone.org/commands/rclone_test_info/) \-
|
|
Discovers file name or other limitations for paths.
|
|
.IP \[bu] 2
|
|
rclone test
|
|
makefiles (https://rclone.org/commands/rclone_test_makefiles/) \- Make a
|
|
random file hierarchy in
|
|
.RS 2
|
|
.RE
|
|
.IP \[bu] 2
|
|
rclone test memory (https://rclone.org/commands/rclone_test_memory/) \-
|
|
Load all the objects at remote:path into memory and report memory stats.
|
|
.SH rclone test histogram
|
|
.PP
|
|
Makes a histogram of file name characters.
|
|
.SS Synopsis
|
|
.PP
|
|
This command outputs JSON which shows the histogram of characters used
|
|
in filenames in the remote:path specified.
|
|
.PP
|
|
The data doesn\[aq]t contain any identifying information but is useful
|
|
for the rclone developers when developing filename compression.
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone test histogram [remote:path] [flags]
|
|
\f[R]
|
|
.fi
|
|
.SS Options
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
\-h, \-\-help help for histogram
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
See the global flags page (https://rclone.org/flags/) for global options
|
|
not listed here.
|
|
.SS SEE ALSO
|
|
.IP \[bu] 2
|
|
rclone test (https://rclone.org/commands/rclone_test/) \- Run a test
|
|
command
|
|
.SH rclone test info
|
|
.PP
|
|
Discovers file name or other limitations for paths.
|
|
.SS Synopsis
|
|
.PP
|
|
rclone info discovers what filenames and upload methods are possible to
|
|
write to the paths passed in and how long they can be.
|
|
It can take some time.
|
|
It will write test files into the remote:path passed in.
|
|
It outputs a bit of go code for each one.
|
|
.PP
|
|
\f[B]NB\f[R] this can create undeletable files and other hazards \- use
|
|
with care
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone test info [remote:path]+ [flags]
|
|
\f[R]
|
|
.fi
|
|
.SS Options
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
\-\-all Run all tests.
|
|
\-\-check\-control Check control characters.
|
|
\-\-check\-length Check max filename length.
|
|
\-\-check\-normalization Check UTF\-8 Normalization.
|
|
\-\-check\-streaming Check uploads with indeterminate file size.
|
|
\-h, \-\-help help for info
|
|
\-\-upload\-wait duration Wait after writing a file.
|
|
\-\-write\-json string Write results to file.
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
See the global flags page (https://rclone.org/flags/) for global options
|
|
not listed here.
|
|
.SS SEE ALSO
|
|
.IP \[bu] 2
|
|
rclone test (https://rclone.org/commands/rclone_test/) \- Run a test
|
|
command
|
|
.SH rclone test makefiles
|
|
Make a random file hierarchy in
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone test makefiles <dir> [flags]
|
|
\f[R]
|
|
.fi
|
|
.SS Options
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
\-\-files int Number of files to create (default 1000)
|
|
\-\-files\-per\-directory int Average number of files per directory (default 10)
|
|
\-h, \-\-help help for makefiles
|
|
\-\-max\-file\-size SizeSuffix Maximum size of files to create (default 100)
|
|
\-\-max\-name\-length int Maximum size of file names (default 12)
|
|
\-\-min\-file\-size SizeSuffix Minimum size of file to create
|
|
\-\-min\-name\-length int Minimum size of file names (default 4)
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
See the global flags page (https://rclone.org/flags/) for global options
|
|
not listed here.
|
|
.SS SEE ALSO
|
|
.IP \[bu] 2
|
|
rclone test (https://rclone.org/commands/rclone_test/) \- Run a test
|
|
command
|
|
.SH rclone test memory
|
|
.PP
|
|
Load all the objects at remote:path into memory and report memory stats.
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone test memory remote:path [flags]
|
|
\f[R]
|
|
.fi
|
|
.SS Options
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
\-h, \-\-help help for memory
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
See the global flags page (https://rclone.org/flags/) for global options
|
|
not listed here.
|
|
.SS SEE ALSO
|
|
.IP \[bu] 2
|
|
rclone test (https://rclone.org/commands/rclone_test/) \- Run a test
|
|
command
|
|
.SH rclone touch
|
|
.PP
|
|
Create new file or change file modification time.
|
|
.SS Synopsis
|
|
.PP
|
|
Set the modification time on object(s) as specified by remote:path to
|
|
have the current time.
|
|
.PP
|
|
If remote:path does not exist then a zero sized object will be created
|
|
unless the \-\-no\-create flag is provided.
|
|
.PP
|
|
If \-\-timestamp is used then it will set the modification time to that
|
|
time instead of the current time.
|
|
Times may be specified as one of:
|
|
.IP \[bu] 2
|
|
\[aq]YYMMDD\[aq] \- e.g.
|
|
17.10.30
|
|
.IP \[bu] 2
|
|
\[aq]YYYY\-MM\-DDTHH:MM:SS\[aq] \- e.g.
|
|
2006\-01\-02T15:04:05
|
|
.IP \[bu] 2
|
|
\[aq]YYYY\-MM\-DDTHH:MM:SS.SSS\[aq] \- e.g.
|
|
2006\-01\-02T15:04:05.123456789
|
|
.PP
|
|
Note that \-\-timestamp is in UTC if you want local time then add the
|
|
\-\-localtime flag.
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone touch remote:path [flags]
|
|
\f[R]
|
|
.fi
|
|
.SS Options
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
\-h, \-\-help help for touch
|
|
\-\-localtime Use localtime for timestamp, not UTC.
|
|
\-C, \-\-no\-create Do not create the file if it does not exist.
|
|
\-t, \-\-timestamp string Use specified time instead of the current time of day.
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
See the global flags page (https://rclone.org/flags/) for global options
|
|
not listed here.
|
|
.SS SEE ALSO
|
|
.IP \[bu] 2
|
|
rclone (https://rclone.org/commands/rclone/) \- Show help for rclone
|
|
commands, flags and backends.
|
|
.SH rclone tree
|
|
.PP
|
|
List the contents of the remote in a tree like fashion.
|
|
.SS Synopsis
|
|
.PP
|
|
rclone tree lists the contents of a remote in a similar way to the unix
|
|
tree command.
|
|
.PP
|
|
For example
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
$ rclone tree remote:path
|
|
/
|
|
\[u251C]\[u2500]\[u2500] file1
|
|
\[u251C]\[u2500]\[u2500] file2
|
|
\[u251C]\[u2500]\[u2500] file3
|
|
\[u2514]\[u2500]\[u2500] subdir
|
|
\[u251C]\[u2500]\[u2500] file4
|
|
\[u2514]\[u2500]\[u2500] file5
|
|
|
|
1 directories, 5 files
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
You can use any of the filtering options with the tree command (e.g.
|
|
\-\-include and \-\-exclude).
|
|
You can also use \-\-fast\-list.
|
|
.PP
|
|
The tree command has many options for controlling the listing which are
|
|
compatible with the tree command.
|
|
Note that not all of them have short options as they conflict with
|
|
rclone\[aq]s short options.
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone tree remote:path [flags]
|
|
\f[R]
|
|
.fi
|
|
.SS Options
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
\-a, \-\-all All files are listed (list . files too).
|
|
\-C, \-\-color Turn colorization on always.
|
|
\-d, \-\-dirs\-only List directories only.
|
|
\-\-dirsfirst List directories before files (\-U disables).
|
|
\-\-full\-path Print the full path prefix for each file.
|
|
\-h, \-\-help help for tree
|
|
\-\-human Print the size in a more human readable way.
|
|
\-\-level int Descend only level directories deep.
|
|
\-D, \-\-modtime Print the date of last modification.
|
|
\-\-noindent Don\[aq]t print indentation lines.
|
|
\-\-noreport Turn off file/directory count at end of tree listing.
|
|
\-o, \-\-output string Output to file instead of stdout.
|
|
\-p, \-\-protections Print the protections for each file.
|
|
\-Q, \-\-quote Quote filenames with double quotes.
|
|
\-s, \-\-size Print the size in bytes of each file.
|
|
\-\-sort string Select sort: name,version,size,mtime,ctime.
|
|
\-\-sort\-ctime Sort files by last status change time.
|
|
\-t, \-\-sort\-modtime Sort files by last modification time.
|
|
\-r, \-\-sort\-reverse Reverse the order of the sort.
|
|
\-U, \-\-unsorted Leave files unsorted.
|
|
\-\-version Sort files alphanumerically by version.
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
See the global flags page (https://rclone.org/flags/) for global options
|
|
not listed here.
|
|
.SS SEE ALSO
|
|
.IP \[bu] 2
|
|
rclone (https://rclone.org/commands/rclone/) \- Show help for rclone
|
|
commands, flags and backends.
|
|
.SS Copying single files
|
|
.PP
|
|
rclone normally syncs or copies directories.
|
|
However, if the source remote points to a file, rclone will just copy
|
|
that file.
|
|
The destination remote must point to a directory \- rclone will give the
|
|
error
|
|
\f[C]Failed to create file system for \[dq]remote:file\[dq]: is a file not a directory\f[R]
|
|
if it isn\[aq]t.
|
|
.PP
|
|
For example, suppose you have a remote with a file in called
|
|
\f[C]test.jpg\f[R], then you could copy just that file like this
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone copy remote:test.jpg /tmp/download
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
The file \f[C]test.jpg\f[R] will be placed inside
|
|
\f[C]/tmp/download\f[R].
|
|
.PP
|
|
This is equivalent to specifying
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone copy \-\-files\-from /tmp/files remote: /tmp/download
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
Where \f[C]/tmp/files\f[R] contains the single line
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
test.jpg
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
It is recommended to use \f[C]copy\f[R] when copying individual files,
|
|
not \f[C]sync\f[R].
|
|
They have pretty much the same effect but \f[C]copy\f[R] will use a lot
|
|
less memory.
|
|
.SS Syntax of remote paths
|
|
.PP
|
|
The syntax of the paths passed to the rclone command are as follows.
|
|
.SS /path/to/dir
|
|
.PP
|
|
This refers to the local file system.
|
|
.PP
|
|
On Windows \f[C]\[rs]\f[R] may be used instead of \f[C]/\f[R] in local
|
|
paths \f[B]only\f[R], non local paths must use \f[C]/\f[R].
|
|
See local filesystem (https://rclone.org/local/#windows-paths)
|
|
documentation for more about Windows\-specific paths.
|
|
.PP
|
|
These paths needn\[aq]t start with a leading \f[C]/\f[R] \- if they
|
|
don\[aq]t then they will be relative to the current directory.
|
|
.SS remote:path/to/dir
|
|
.PP
|
|
This refers to a directory \f[C]path/to/dir\f[R] on \f[C]remote:\f[R] as
|
|
defined in the config file (configured with \f[C]rclone config\f[R]).
|
|
.SS remote:/path/to/dir
|
|
.PP
|
|
On most backends this is refers to the same directory as
|
|
\f[C]remote:path/to/dir\f[R] and that format should be preferred.
|
|
On a very small number of remotes (FTP, SFTP, Dropbox for business) this
|
|
will refer to a different directory.
|
|
On these, paths without a leading \f[C]/\f[R] will refer to your
|
|
\[dq]home\[dq] directory and paths with a leading \f[C]/\f[R] will refer
|
|
to the root.
|
|
.SS :backend:path/to/dir
|
|
.PP
|
|
This is an advanced form for creating remotes on the fly.
|
|
\f[C]backend\f[R] should be the name or prefix of a backend (the
|
|
\f[C]type\f[R] in the config file) and all the configuration for the
|
|
backend should be provided on the command line (or in environment
|
|
variables).
|
|
.PP
|
|
Here are some examples:
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone lsd \-\-http\-url https://pub.rclone.org :http:
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
To list all the directories in the root of
|
|
\f[C]https://pub.rclone.org/\f[R].
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone lsf \-\-http\-url https://example.com :http:path/to/dir
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
To list files and directories in
|
|
\f[C]https://example.com/path/to/dir/\f[R]
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone copy \-\-http\-url https://example.com :http:path/to/dir /tmp/dir
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
To copy files and directories in
|
|
\f[C]https://example.com/path/to/dir\f[R] to \f[C]/tmp/dir\f[R].
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone copy \-\-sftp\-host example.com :sftp:path/to/dir /tmp/dir
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
To copy files and directories from \f[C]example.com\f[R] in the relative
|
|
directory \f[C]path/to/dir\f[R] to \f[C]/tmp/dir\f[R] using sftp.
|
|
.SS Connection strings
|
|
.PP
|
|
The above examples can also be written using a connection string syntax,
|
|
so instead of providing the arguments as command line parameters
|
|
\f[C]\-\-http\-url https://pub.rclone.org\f[R] they are provided as part
|
|
of the remote specification as a kind of connection string.
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone lsd \[dq]:http,url=\[aq]https://pub.rclone.org\[aq]:\[dq]
|
|
rclone lsf \[dq]:http,url=\[aq]https://example.com\[aq]:path/to/dir\[dq]
|
|
rclone copy \[dq]:http,url=\[aq]https://example.com\[aq]:path/to/dir\[dq] /tmp/dir
|
|
rclone copy :sftp,host=example.com:path/to/dir /tmp/dir
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
These can apply to modify existing remotes as well as create new remotes
|
|
with the on the fly syntax.
|
|
This example is equivalent to adding the
|
|
\f[C]\-\-drive\-shared\-with\-me\f[R] parameter to the remote
|
|
\f[C]gdrive:\f[R].
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone lsf \[dq]gdrive,shared_with_me:path/to/dir\[dq]
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
The major advantage to using the connection string style syntax is that
|
|
it only applies the the remote, not to all the remotes of that type of
|
|
the command line.
|
|
A common confusion is this attempt to copy a file shared on google drive
|
|
to the normal drive which \f[B]does not work\f[R] because the
|
|
\f[C]\-\-drive\-shared\-with\-me\f[R] flag applies to both the source
|
|
and the destination.
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone copy \-\-drive\-shared\-with\-me gdrive:shared\-file.txt gdrive:
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
However using the connection string syntax, this does work.
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone copy \[dq]gdrive,shared_with_me:shared\-file.txt\[dq] gdrive:
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
The connection strings have the following syntax
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
remote,parameter=value,parameter2=value2:path/to/dir
|
|
:backend,parameter=value,parameter2=value2:path/to/dir
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
If the \f[C]parameter\f[R] has a \f[C]:\f[R] or \f[C],\f[R] then it must
|
|
be placed in quotes \f[C]\[dq]\f[R] or \f[C]\[aq]\f[R], so
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
remote,parameter=\[dq]colon:value\[dq],parameter2=\[dq]comma,value\[dq]:path/to/dir
|
|
:backend,parameter=\[aq]colon:value\[aq],parameter2=\[aq]comma,value\[aq]:path/to/dir
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
If a quoted value needs to include that quote, then it should be
|
|
doubled, so
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
remote,parameter=\[dq]with\[dq]\[dq]quote\[dq],parameter2=\[aq]with\[aq]\[aq]quote\[aq]:path/to/dir
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
This will make \f[C]parameter\f[R] be \f[C]with\[dq]quote\f[R] and
|
|
\f[C]parameter2\f[R] be \f[C]with\[aq]quote\f[R].
|
|
.PP
|
|
If you leave off the \f[C]=parameter\f[R] then rclone will substitute
|
|
\f[C]=true\f[R] which works very well with flags.
|
|
For example to use s3 configured in the environment you could use:
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone lsd :s3,env_auth:
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
Which is equivalent to
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone lsd :s3,env_auth=true:
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
Note that on the command line you might need to surround these
|
|
connection strings with \f[C]\[dq]\f[R] or \f[C]\[aq]\f[R] to stop the
|
|
shell interpreting any special characters within them.
|
|
.PP
|
|
If you are a shell master then you\[aq]ll know which strings are OK and
|
|
which aren\[aq]t, but if you aren\[aq]t sure then enclose them in
|
|
\f[C]\[dq]\f[R] and use \f[C]\[aq]\f[R] as the inside quote.
|
|
This syntax works on all OSes.
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone copy \[dq]:http,url=\[aq]https://example.com\[aq]:path/to/dir\[dq] /tmp/dir
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
On Linux/macOS some characters are still interpreted inside
|
|
\f[C]\[dq]\f[R] strings in the shell (notably \f[C]\[rs]\f[R] and
|
|
\f[C]$\f[R] and \f[C]\[dq]\f[R]) so if your strings contain those you
|
|
can swap the roles of \f[C]\[dq]\f[R] and \f[C]\[aq]\f[R] thus.
|
|
(This syntax does not work on Windows.)
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone copy \[aq]:http,url=\[dq]https://example.com\[dq]:path/to/dir\[aq] /tmp/dir
|
|
\f[R]
|
|
.fi
|
|
.SS Connection strings, config and logging
|
|
.PP
|
|
If you supply extra configuration to a backend by command line flag,
|
|
environment variable or connection string then rclone will add a suffix
|
|
based on the hash of the config to the name of the remote, eg
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone \-vv lsf \-\-s3\-chunk\-size 20M s3:
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
Has the log message
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
DEBUG : s3: detected overridden config \- adding \[dq]{Srj1p}\[dq] suffix to name
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
This is so rclone can tell the modified remote apart from the unmodified
|
|
remote when caching the backends.
|
|
.PP
|
|
This should only be noticeable in the logs.
|
|
.PP
|
|
This means that on the fly backends such as
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone \-vv lsf :s3,env_auth:
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
Will get their own names
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
DEBUG : :s3: detected overridden config \- adding \[dq]{YTu53}\[dq] suffix to name
|
|
\f[R]
|
|
.fi
|
|
.SS Valid remote names
|
|
.IP \[bu] 2
|
|
Remote names may only contain 0\-9, A\-Z ,a\-z ,_ , \- and space.
|
|
.IP \[bu] 2
|
|
Remote names may not start with \-.
|
|
.SS Quoting and the shell
|
|
.PP
|
|
When you are typing commands to your computer you are using something
|
|
called the command line shell.
|
|
This interprets various characters in an OS specific way.
|
|
.PP
|
|
Here are some gotchas which may help users unfamiliar with the shell
|
|
rules
|
|
.SS Linux / OSX
|
|
.PP
|
|
If your names have spaces or shell metacharacters (e.g.
|
|
\f[C]*\f[R], \f[C]?\f[R], \f[C]$\f[R], \f[C]\[aq]\f[R], \f[C]\[dq]\f[R],
|
|
etc.) then you must quote them.
|
|
Use single quotes \f[C]\[aq]\f[R] by default.
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone copy \[aq]Important files?\[aq] remote:backup
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
If you want to send a \f[C]\[aq]\f[R] you will need to use
|
|
\f[C]\[dq]\f[R], e.g.
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone copy \[dq]O\[aq]Reilly Reviews\[dq] remote:backup
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
The rules for quoting metacharacters are complicated and if you want the
|
|
full details you\[aq]ll have to consult the manual page for your shell.
|
|
.SS Windows
|
|
.PP
|
|
If your names have spaces in you need to put them in \f[C]\[dq]\f[R],
|
|
e.g.
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone copy \[dq]E:\[rs]folder name\[rs]folder name\[rs]folder name\[dq] remote:backup
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
If you are using the root directory on its own then don\[aq]t quote it
|
|
(see #464 (https://github.com/rclone/rclone/issues/464) for why), e.g.
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone copy E:\[rs] remote:backup
|
|
\f[R]
|
|
.fi
|
|
.SS Copying files or directories with \f[C]:\f[R] in the names
|
|
.PP
|
|
rclone uses \f[C]:\f[R] to mark a remote name.
|
|
This is, however, a valid filename component in non\-Windows OSes.
|
|
The remote name parser will only search for a \f[C]:\f[R] up to the
|
|
first \f[C]/\f[R] so if you need to act on a file or directory like this
|
|
then use the full path starting with a \f[C]/\f[R], or use \f[C]./\f[R]
|
|
as a current directory prefix.
|
|
.PP
|
|
So to sync a directory called \f[C]sync:me\f[R] to a remote called
|
|
\f[C]remote:\f[R] use
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone sync \-i ./sync:me remote:path
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
or
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone sync \-i /full/path/to/sync:me remote:path
|
|
\f[R]
|
|
.fi
|
|
.SS Server Side Copy
|
|
.PP
|
|
Most remotes (but not all \- see the
|
|
overview (https://rclone.org/overview/#optional-features)) support
|
|
server\-side copy.
|
|
.PP
|
|
This means if you want to copy one folder to another then rclone
|
|
won\[aq]t download all the files and re\-upload them; it will instruct
|
|
the server to copy them in place.
|
|
.PP
|
|
Eg
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone copy s3:oldbucket s3:newbucket
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
Will copy the contents of \f[C]oldbucket\f[R] to \f[C]newbucket\f[R]
|
|
without downloading and re\-uploading.
|
|
.PP
|
|
Remotes which don\[aq]t support server\-side copy \f[B]will\f[R]
|
|
download and re\-upload in this case.
|
|
.PP
|
|
Server side copies are used with \f[C]sync\f[R] and \f[C]copy\f[R] and
|
|
will be identified in the log when using the \f[C]\-v\f[R] flag.
|
|
The \f[C]move\f[R] command may also use them if remote doesn\[aq]t
|
|
support server\-side move directly.
|
|
This is done by issuing a server\-side copy then a delete which is much
|
|
quicker than a download and re\-upload.
|
|
.PP
|
|
Server side copies will only be attempted if the remote names are the
|
|
same.
|
|
.PP
|
|
This can be used when scripting to make aged backups efficiently, e.g.
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone sync \-i remote:current\-backup remote:previous\-backup
|
|
rclone sync \-i /path/to/files remote:current\-backup
|
|
\f[R]
|
|
.fi
|
|
.SS Options
|
|
.PP
|
|
Rclone has a number of options to control its behaviour.
|
|
.PP
|
|
Options that take parameters can have the values passed in two ways,
|
|
\f[C]\-\-option=value\f[R] or \f[C]\-\-option value\f[R].
|
|
However boolean (true/false) options behave slightly differently to the
|
|
other options in that \f[C]\-\-boolean\f[R] sets the option to
|
|
\f[C]true\f[R] and the absence of the flag sets it to \f[C]false\f[R].
|
|
It is also possible to specify \f[C]\-\-boolean=false\f[R] or
|
|
\f[C]\-\-boolean=true\f[R].
|
|
Note that \f[C]\-\-boolean false\f[R] is not valid \- this is parsed as
|
|
\f[C]\-\-boolean\f[R] and the \f[C]false\f[R] is parsed as an extra
|
|
command line argument for rclone.
|
|
.PP
|
|
Options which use TIME use the go time parser.
|
|
A duration string is a possibly signed sequence of decimal numbers, each
|
|
with optional fraction and a unit suffix, such as \[dq]300ms\[dq],
|
|
\[dq]\-1.5h\[dq] or \[dq]2h45m\[dq].
|
|
Valid time units are \[dq]ns\[dq], \[dq]us\[dq] (or \[dq]\[mc]s\[dq]),
|
|
\[dq]ms\[dq], \[dq]s\[dq], \[dq]m\[dq], \[dq]h\[dq].
|
|
.PP
|
|
Options which use SIZE use kByte by default.
|
|
However, a suffix of \f[C]b\f[R] for bytes, \f[C]k\f[R] for kBytes,
|
|
\f[C]M\f[R] for MBytes, \f[C]G\f[R] for GBytes, \f[C]T\f[R] for TBytes
|
|
and \f[C]P\f[R] for PBytes may be used.
|
|
These are the binary units, e.g.
|
|
1, 2**10, 2**20, 2**30 respectively.
|
|
.SS \-\-backup\-dir=DIR
|
|
.PP
|
|
When using \f[C]sync\f[R], \f[C]copy\f[R] or \f[C]move\f[R] any files
|
|
which would have been overwritten or deleted are moved in their original
|
|
hierarchy into this directory.
|
|
.PP
|
|
If \f[C]\-\-suffix\f[R] is set, then the moved files will have the
|
|
suffix added to them.
|
|
If there is a file with the same path (after the suffix has been added)
|
|
in DIR, then it will be overwritten.
|
|
.PP
|
|
The remote in use must support server\-side move or copy and you must
|
|
use the same remote as the destination of the sync.
|
|
The backup directory must not overlap the destination directory.
|
|
.PP
|
|
For example
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone sync \-i /path/to/local remote:current \-\-backup\-dir remote:old
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
will sync \f[C]/path/to/local\f[R] to \f[C]remote:current\f[R], but for
|
|
any files which would have been updated or deleted will be stored in
|
|
\f[C]remote:old\f[R].
|
|
.PP
|
|
If running rclone from a script you might want to use today\[aq]s date
|
|
as the directory name passed to \f[C]\-\-backup\-dir\f[R] to store the
|
|
old files, or you might want to pass \f[C]\-\-suffix\f[R] with
|
|
today\[aq]s date.
|
|
.PP
|
|
See \f[C]\-\-compare\-dest\f[R] and \f[C]\-\-copy\-dest\f[R].
|
|
.SS \-\-bind string
|
|
.PP
|
|
Local address to bind to for outgoing connections.
|
|
This can be an IPv4 address (1.2.3.4), an IPv6 address (1234::789A) or
|
|
host name.
|
|
If the host name doesn\[aq]t resolve or resolves to more than one IP
|
|
address it will give an error.
|
|
.SS \-\-bwlimit=BANDWIDTH_SPEC
|
|
.PP
|
|
This option controls the bandwidth limit.
|
|
For example
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
\-\-bwlimit 10M
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
would mean limit the upload and download bandwidth to 10 MByte/s.
|
|
\f[B]NB\f[R] this is \f[B]bytes\f[R] per second not \f[B]bits\f[R] per
|
|
second.
|
|
To use a single limit, specify the desired bandwidth in kBytes/s, or use
|
|
a suffix b|k|M|G.
|
|
The default is \f[C]0\f[R] which means to not limit bandwidth.
|
|
.PP
|
|
The upload and download bandwidth can be specified seperately, as
|
|
\f[C]\-\-bwlimit UP:DOWN\f[R], so
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
\-\-bwlimit 10M:100k
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
would mean limit the upload bandwidth to 10 MByte/s and the download
|
|
bandwidth to 100 kByte/s.
|
|
Either limit can be \[dq]off\[dq] meaning no limit, so to just limit the
|
|
upload bandwidth you would use
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
\-\-bwlimit 10M:off
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
this would limit the upload bandwidth to 10MByte/s but the download
|
|
bandwidth would be unlimited.
|
|
.PP
|
|
When specified as above the bandwidth limits last for the duration of
|
|
run of the rclone binary.
|
|
.PP
|
|
It is also possible to specify a \[dq]timetable\[dq] of limits, which
|
|
will cause certain limits to be applied at certain times.
|
|
To specify a timetable, format your entries as
|
|
\f[C]WEEKDAY\-HH:MM,BANDWIDTH WEEKDAY\-HH:MM,BANDWIDTH...\f[R] where:
|
|
\f[C]WEEKDAY\f[R] is optional element.
|
|
.IP \[bu] 2
|
|
\f[C]BANDWIDTH\f[R] can be a single number, e.g.\f[C]100k\f[R] or a pair
|
|
of numbers for upload:download, e.g.\f[C]10M:1M\f[R].
|
|
.IP \[bu] 2
|
|
\f[C]WEEKDAY\f[R] can be written as the whole word or only using the
|
|
first 3 characters.
|
|
It is optional.
|
|
.IP \[bu] 2
|
|
\f[C]HH:MM\f[R] is an hour from 00:00 to 23:59.
|
|
.PP
|
|
An example of a typical timetable to avoid link saturation during
|
|
daytime working hours could be:
|
|
.PP
|
|
\f[C]\-\-bwlimit \[dq]08:00,512k 12:00,10M 13:00,512k 18:00,30M 23:00,off\[dq]\f[R]
|
|
.PP
|
|
In this example, the transfer bandwidth will be set to 512kBytes/sec at
|
|
8am every day.
|
|
At noon, it will rise to 10MByte/s, and drop back to 512kBytes/sec at
|
|
1pm.
|
|
At 6pm, the bandwidth limit will be set to 30MByte/s, and at 11pm it
|
|
will be completely disabled (full speed).
|
|
Anything between 11pm and 8am will remain unlimited.
|
|
.PP
|
|
An example of timetable with \f[C]WEEKDAY\f[R] could be:
|
|
.PP
|
|
\f[C]\-\-bwlimit \[dq]Mon\-00:00,512 Fri\-23:59,10M Sat\-10:00,1M Sun\-20:00,off\[dq]\f[R]
|
|
.PP
|
|
It means that, the transfer bandwidth will be set to 512kBytes/sec on
|
|
Monday.
|
|
It will rise to 10MByte/s before the end of Friday.
|
|
At 10:00 on Saturday it will be set to 1MByte/s.
|
|
From 20:00 on Sunday it will be unlimited.
|
|
.PP
|
|
Timeslots without \f[C]WEEKDAY\f[R] are extended to the whole week.
|
|
So this example:
|
|
.PP
|
|
\f[C]\-\-bwlimit \[dq]Mon\-00:00,512 12:00,1M Sun\-20:00,off\[dq]\f[R]
|
|
.PP
|
|
Is equivalent to this:
|
|
.PP
|
|
\f[C]\-\-bwlimit \[dq]Mon\-00:00,512Mon\-12:00,1M Tue\-12:00,1M Wed\-12:00,1M Thu\-12:00,1M Fri\-12:00,1M Sat\-12:00,1M Sun\-12:00,1M Sun\-20:00,off\[dq]\f[R]
|
|
.PP
|
|
Bandwidth limit apply to the data transfer for all backends.
|
|
For most backends the directory listing bandwidth is also included
|
|
(exceptions being the non HTTP backends, \f[C]ftp\f[R], \f[C]sftp\f[R]
|
|
and \f[C]tardigrade\f[R]).
|
|
.PP
|
|
Note that the units are \f[B]Bytes/s\f[R], not \f[B]Bits/s\f[R].
|
|
Typically connections are measured in Bits/s \- to convert divide by 8.
|
|
For example, let\[aq]s say you have a 10 Mbit/s connection and you wish
|
|
rclone to use half of it \- 5 Mbit/s.
|
|
This is 5/8 = 0.625MByte/s so you would use a
|
|
\f[C]\-\-bwlimit 0.625M\f[R] parameter for rclone.
|
|
.PP
|
|
On Unix systems (Linux, macOS, \&...) the bandwidth limiter can be
|
|
toggled by sending a \f[C]SIGUSR2\f[R] signal to rclone.
|
|
This allows to remove the limitations of a long running rclone transfer
|
|
and to restore it back to the value specified with \f[C]\-\-bwlimit\f[R]
|
|
quickly when needed.
|
|
Assuming there is only one rclone instance running, you can toggle the
|
|
limiter like this:
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
kill \-SIGUSR2 $(pidof rclone)
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
If you configure rclone with a remote control (/rc) then you can use
|
|
change the bwlimit dynamically:
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone rc core/bwlimit rate=1M
|
|
\f[R]
|
|
.fi
|
|
.SS \-\-bwlimit\-file=BANDWIDTH_SPEC
|
|
.PP
|
|
This option controls per file bandwidth limit.
|
|
For the options see the \f[C]\-\-bwlimit\f[R] flag.
|
|
.PP
|
|
For example use this to allow no transfers to be faster than 1MByte/s
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
\-\-bwlimit\-file 1M
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
This can be used in conjunction with \f[C]\-\-bwlimit\f[R].
|
|
.PP
|
|
Note that if a schedule is provided the file will use the schedule in
|
|
effect at the start of the transfer.
|
|
.SS \-\-buffer\-size=SIZE
|
|
.PP
|
|
Use this sized buffer to speed up file transfers.
|
|
Each \f[C]\-\-transfer\f[R] will use this much memory for buffering.
|
|
.PP
|
|
When using \f[C]mount\f[R] or \f[C]cmount\f[R] each open file descriptor
|
|
will use this much memory for buffering.
|
|
See the mount (https://rclone.org/commands/rclone_mount/#file-buffering)
|
|
documentation for more details.
|
|
.PP
|
|
Set to \f[C]0\f[R] to disable the buffering for the minimum memory
|
|
usage.
|
|
.PP
|
|
Note that the memory allocation of the buffers is influenced by the
|
|
\-\-use\-mmap flag.
|
|
.SS \-\-check\-first
|
|
.PP
|
|
If this flag is set then in a \f[C]sync\f[R], \f[C]copy\f[R] or
|
|
\f[C]move\f[R], rclone will do all the checks to see whether files need
|
|
to be transferred before doing any of the transfers.
|
|
Normally rclone would start running transfers as soon as possible.
|
|
.PP
|
|
This flag can be useful on IO limited systems where transfers interfere
|
|
with checking.
|
|
.PP
|
|
It can also be useful to ensure perfect ordering when using
|
|
\f[C]\-\-order\-by\f[R].
|
|
.PP
|
|
Using this flag can use more memory as it effectively sets
|
|
\f[C]\-\-max\-backlog\f[R] to infinite.
|
|
This means that all the info on the objects to transfer is held in
|
|
memory before the transfers start.
|
|
.SS \-\-checkers=N
|
|
.PP
|
|
The number of checkers to run in parallel.
|
|
Checkers do the equality checking of files during a sync.
|
|
For some storage systems (e.g.
|
|
S3, Swift, Dropbox) this can take a significant amount of time so they
|
|
are run in parallel.
|
|
.PP
|
|
The default is to run 8 checkers in parallel.
|
|
.SS \-c, \-\-checksum
|
|
.PP
|
|
Normally rclone will look at modification time and size of files to see
|
|
if they are equal.
|
|
If you set this flag then rclone will check the file hash and size to
|
|
determine if files are equal.
|
|
.PP
|
|
This is useful when the remote doesn\[aq]t support setting modified time
|
|
and a more accurate sync is desired than just checking the file size.
|
|
.PP
|
|
This is very useful when transferring between remotes which store the
|
|
same hash type on the object, e.g.
|
|
Drive and Swift.
|
|
For details of which remotes support which hash type see the table in
|
|
the overview section (https://rclone.org/overview/).
|
|
.PP
|
|
Eg \f[C]rclone \-\-checksum sync s3:/bucket swift:/bucket\f[R] would run
|
|
much quicker than without the \f[C]\-\-checksum\f[R] flag.
|
|
.PP
|
|
When using this flag, rclone won\[aq]t update mtimes of remote files if
|
|
they are incorrect as it would normally.
|
|
.SS \-\-compare\-dest=DIR
|
|
.PP
|
|
When using \f[C]sync\f[R], \f[C]copy\f[R] or \f[C]move\f[R] DIR is
|
|
checked in addition to the destination for files.
|
|
If a file identical to the source is found that file is NOT copied from
|
|
source.
|
|
This is useful to copy just files that have changed since the last
|
|
backup.
|
|
.PP
|
|
You must use the same remote as the destination of the sync.
|
|
The compare directory must not overlap the destination directory.
|
|
.PP
|
|
See \f[C]\-\-copy\-dest\f[R] and \f[C]\-\-backup\-dir\f[R].
|
|
.SS \-\-config=CONFIG_FILE
|
|
.PP
|
|
Specify the location of the rclone configuration file.
|
|
.PP
|
|
Normally the config file is in your home directory as a file called
|
|
\f[C].config/rclone/rclone.conf\f[R] (or \f[C].rclone.conf\f[R] if
|
|
created with an older version).
|
|
If \f[C]$XDG_CONFIG_HOME\f[R] is set it will be at
|
|
\f[C]$XDG_CONFIG_HOME/rclone/rclone.conf\f[R].
|
|
.PP
|
|
If there is a file \f[C]rclone.conf\f[R] in the same directory as the
|
|
rclone executable it will be preferred.
|
|
This file must be created manually for Rclone to use it, it will never
|
|
be created automatically.
|
|
.PP
|
|
If you run \f[C]rclone config file\f[R] you will see where the default
|
|
location is for you.
|
|
.PP
|
|
Use this flag to override the config location, e.g.
|
|
\f[C]rclone \-\-config=\[dq].myconfig\[dq] .config\f[R].
|
|
.PP
|
|
If the location is set to empty string \f[C]\[dq]\[dq]\f[R] or the
|
|
special value \f[C]/notfound\f[R], or the os null device represented by
|
|
value \f[C]NUL\f[R] on Windows and \f[C]/dev/null\f[R] on Unix systems,
|
|
then rclone will keep the config file in memory only.
|
|
.PP
|
|
The file format is basic
|
|
INI (https://en.wikipedia.org/wiki/INI_file#Format): Sections of text,
|
|
led by a \f[C][section]\f[R] header and followed by \f[C]key=value\f[R]
|
|
entries on separate lines.
|
|
In rclone each remote is represented by its own section, where the
|
|
section name defines the name of the remote.
|
|
Options are specified as the \f[C]key=value\f[R] entries, where the key
|
|
is the option name without the \f[C]\-\-backend\-\f[R] prefix, in
|
|
lowercase and with \f[C]_\f[R] instead of \f[C]\-\f[R].
|
|
E.g.
|
|
option \f[C]\-\-mega\-hard\-delete\f[R] corresponds to key
|
|
\f[C]hard_delete\f[R].
|
|
Only backend options can be specified.
|
|
A special, and required, key \f[C]type\f[R] identifies the storage
|
|
system (https://rclone.org/overview/), where the value is the internal
|
|
lowercase name as returned by command \f[C]rclone help backends\f[R].
|
|
Comments are indicated by \f[C];\f[R] or \f[C]#\f[R] at the beginning of
|
|
a line.
|
|
.PP
|
|
Example:
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
[megaremote]
|
|
type = mega
|
|
user = you\[at]example.com
|
|
pass = PDPcQVVjVtzFY\-GTdDFozqBhTdsPg3qH
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
Note that passwords are in
|
|
obscured (https://rclone.org/commands/rclone_obscure/) form.
|
|
Also, many storage systems uses token\-based authentication instead of
|
|
passwords, and this requires additional steps.
|
|
It is easier, and safer, to use the interactive command
|
|
\f[C]rclone config\f[R] instead of manually editing the configuration
|
|
file.
|
|
.PP
|
|
The configuration file will typically contain login information, and
|
|
should therefore have restricted permissions so that only the current
|
|
user can read it.
|
|
Rclone tries to ensure this when it writes the file.
|
|
You may also choose to encrypt the file.
|
|
.PP
|
|
When token\-based authentication are used, the configuration file must
|
|
be writable, because rclone needs to update the tokens inside it.
|
|
.SS \-\-contimeout=TIME
|
|
.PP
|
|
Set the connection timeout.
|
|
This should be in go time format which looks like \f[C]5s\f[R] for 5
|
|
seconds, \f[C]10m\f[R] for 10 minutes, or \f[C]3h30m\f[R].
|
|
.PP
|
|
The connection timeout is the amount of time rclone will wait for a
|
|
connection to go through to a remote object storage system.
|
|
It is \f[C]1m\f[R] by default.
|
|
.SS \-\-copy\-dest=DIR
|
|
.PP
|
|
When using \f[C]sync\f[R], \f[C]copy\f[R] or \f[C]move\f[R] DIR is
|
|
checked in addition to the destination for files.
|
|
If a file identical to the source is found that file is server\-side
|
|
copied from DIR to the destination.
|
|
This is useful for incremental backup.
|
|
.PP
|
|
The remote in use must support server\-side copy and you must use the
|
|
same remote as the destination of the sync.
|
|
The compare directory must not overlap the destination directory.
|
|
.PP
|
|
See \f[C]\-\-compare\-dest\f[R] and \f[C]\-\-backup\-dir\f[R].
|
|
.SS \-\-dedupe\-mode MODE
|
|
.PP
|
|
Mode to run dedupe command in.
|
|
One of \f[C]interactive\f[R], \f[C]skip\f[R], \f[C]first\f[R],
|
|
\f[C]newest\f[R], \f[C]oldest\f[R], \f[C]rename\f[R].
|
|
The default is \f[C]interactive\f[R].
|
|
See the dedupe command for more information as to what these options
|
|
mean.
|
|
.SS \-\-disable FEATURE,FEATURE,...
|
|
.PP
|
|
This disables a comma separated list of optional features.
|
|
For example to disable server\-side move and server\-side copy use:
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
\-\-disable move,copy
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
The features can be put in any case.
|
|
.PP
|
|
To see a list of which features can be disabled use:
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
\-\-disable help
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
See the overview features (https://rclone.org/overview/#features) and
|
|
optional features (https://rclone.org/overview/#optional-features) to
|
|
get an idea of which feature does what.
|
|
.PP
|
|
This flag can be useful for debugging and in exceptional circumstances
|
|
(e.g.
|
|
Google Drive limiting the total volume of Server Side Copies to
|
|
100GB/day).
|
|
.SS \-\-dscp VALUE
|
|
.PP
|
|
Specify a DSCP value or name to use in connections.
|
|
This could help QoS system to identify traffic class.
|
|
BE, EF, DF, LE, CSx and AFxx are allowed.
|
|
.PP
|
|
See the description of differentiated
|
|
services (https://en.wikipedia.org/wiki/Differentiated_services) to get
|
|
an idea of this field.
|
|
Setting this to 1 (LE) to identify the flow to SCAVENGER class can avoid
|
|
occupying too much bandwidth in a network with DiffServ support (RFC
|
|
8622 (https://tools.ietf.org/html/rfc8622)).
|
|
.PP
|
|
For example, if you configured QoS on router to handle LE properly.
|
|
Running:
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone copy \-\-dscp LE from:/from to:/to
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
would make the priority lower than usual internet flows.
|
|
.SS \-n, \-\-dry\-run
|
|
.PP
|
|
Do a trial run with no permanent changes.
|
|
Use this to see what rclone would do without actually doing it.
|
|
Useful when setting up the \f[C]sync\f[R] command which deletes files in
|
|
the destination.
|
|
.SS \-\-expect\-continue\-timeout=TIME
|
|
.PP
|
|
This specifies the amount of time to wait for a server\[aq]s first
|
|
response headers after fully writing the request headers if the request
|
|
has an \[dq]Expect: 100\-continue\[dq] header.
|
|
Not all backends support using this.
|
|
.PP
|
|
Zero means no timeout and causes the body to be sent immediately,
|
|
without waiting for the server to approve.
|
|
This time does not include the time to send the request header.
|
|
.PP
|
|
The default is \f[C]1s\f[R].
|
|
Set to \f[C]0\f[R] to disable.
|
|
.SS \-\-error\-on\-no\-transfer
|
|
.PP
|
|
By default, rclone will exit with return code 0 if there were no errors.
|
|
.PP
|
|
This option allows rclone to return exit code 9 if no files were
|
|
transferred between the source and destination.
|
|
This allows using rclone in scripts, and triggering follow\-on actions
|
|
if data was copied, or skipping if not.
|
|
.PP
|
|
NB: Enabling this option turns a usually non\-fatal error into a
|
|
potentially fatal one \- please check and adjust your scripts
|
|
accordingly!
|
|
.SS \-\-fs\-cache\-expire\-duration=TIME
|
|
.PP
|
|
When using rclone via the API rclone caches created remotes for 5
|
|
minutes by default in the \[dq]fs cache\[dq].
|
|
This means that if you do repeated actions on the same remote then
|
|
rclone won\[aq]t have to build it again from scratch, which makes it
|
|
more efficient.
|
|
.PP
|
|
This flag sets the time that the remotes are cached for.
|
|
If you set it to \f[C]0\f[R] (or negative) then rclone won\[aq]t cache
|
|
the remotes at all.
|
|
.PP
|
|
Note that if you use some flags, eg \f[C]\-\-backup\-dir\f[R] and if
|
|
this is set to \f[C]0\f[R] rclone may build two remotes (one for the
|
|
source or destination and one for the \f[C]\-\-backup\-dir\f[R] where it
|
|
may have only built one before.
|
|
.SS \-\-fs\-cache\-expire\-interval=TIME
|
|
.PP
|
|
This controls how often rclone checks for cached remotes to expire.
|
|
See the \f[C]\-\-fs\-cache\-expire\-duration\f[R] documentation above
|
|
for more info.
|
|
The default is 60s, set to 0 to disable expiry.
|
|
.SS \-\-header
|
|
.PP
|
|
Add an HTTP header for all transactions.
|
|
The flag can be repeated to add multiple headers.
|
|
.PP
|
|
If you want to add headers only for uploads use
|
|
\f[C]\-\-header\-upload\f[R] and if you want to add headers only for
|
|
downloads use \f[C]\-\-header\-download\f[R].
|
|
.PP
|
|
This flag is supported for all HTTP based backends even those not
|
|
supported by \f[C]\-\-header\-upload\f[R] and
|
|
\f[C]\-\-header\-download\f[R] so may be used as a workaround for those
|
|
with care.
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone ls remote:test \-\-header \[dq]X\-Rclone: Foo\[dq] \-\-header \[dq]X\-LetMeIn: Yes\[dq]
|
|
\f[R]
|
|
.fi
|
|
.SS \-\-header\-download
|
|
.PP
|
|
Add an HTTP header for all download transactions.
|
|
The flag can be repeated to add multiple headers.
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone sync \-i s3:test/src \[ti]/dst \-\-header\-download \[dq]X\-Amz\-Meta\-Test: Foo\[dq] \-\-header\-download \[dq]X\-Amz\-Meta\-Test2: Bar\[dq]
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
See the GitHub issue here (https://github.com/rclone/rclone/issues/59)
|
|
for currently supported backends.
|
|
.SS \-\-header\-upload
|
|
.PP
|
|
Add an HTTP header for all upload transactions.
|
|
The flag can be repeated to add multiple headers.
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone sync \-i \[ti]/src s3:test/dst \-\-header\-upload \[dq]Content\-Disposition: attachment; filename=\[aq]cool.html\[aq]\[dq] \-\-header\-upload \[dq]X\-Amz\-Meta\-Test: FooBar\[dq]
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
See the GitHub issue here (https://github.com/rclone/rclone/issues/59)
|
|
for currently supported backends.
|
|
.SS \-\-ignore\-case\-sync
|
|
.PP
|
|
Using this option will cause rclone to ignore the case of the files when
|
|
synchronizing so files will not be copied/synced when the existing
|
|
filenames are the same, even if the casing is different.
|
|
.SS \-\-ignore\-checksum
|
|
.PP
|
|
Normally rclone will check that the checksums of transferred files
|
|
match, and give an error \[dq]corrupted on transfer\[dq] if they
|
|
don\[aq]t.
|
|
.PP
|
|
You can use this option to skip that check.
|
|
You should only use it if you have had the \[dq]corrupted on
|
|
transfer\[dq] error message and you are sure you might want to transfer
|
|
potentially corrupted data.
|
|
.SS \-\-ignore\-existing
|
|
.PP
|
|
Using this option will make rclone unconditionally skip all files that
|
|
exist on the destination, no matter the content of these files.
|
|
.PP
|
|
While this isn\[aq]t a generally recommended option, it can be useful in
|
|
cases where your files change due to encryption.
|
|
However, it cannot correct partial transfers in case a transfer was
|
|
interrupted.
|
|
.SS \-\-ignore\-size
|
|
.PP
|
|
Normally rclone will look at modification time and size of files to see
|
|
if they are equal.
|
|
If you set this flag then rclone will check only the modification time.
|
|
If \f[C]\-\-checksum\f[R] is set then it only checks the checksum.
|
|
.PP
|
|
It will also cause rclone to skip verifying the sizes are the same after
|
|
transfer.
|
|
.PP
|
|
This can be useful for transferring files to and from OneDrive which
|
|
occasionally misreports the size of image files (see
|
|
#399 (https://github.com/rclone/rclone/issues/399) for more info).
|
|
.SS \-I, \-\-ignore\-times
|
|
.PP
|
|
Using this option will cause rclone to unconditionally upload all files
|
|
regardless of the state of files on the destination.
|
|
.PP
|
|
Normally rclone would skip any files that have the same modification
|
|
time and are the same size (or have the same checksum if using
|
|
\f[C]\-\-checksum\f[R]).
|
|
.SS \-\-immutable
|
|
.PP
|
|
Treat source and destination files as immutable and disallow
|
|
modification.
|
|
.PP
|
|
With this option set, files will be created and deleted as requested,
|
|
but existing files will never be updated.
|
|
If an existing file does not match between the source and destination,
|
|
rclone will give the error
|
|
\f[C]Source and destination exist but do not match: immutable file modified\f[R].
|
|
.PP
|
|
Note that only commands which transfer files (e.g.
|
|
\f[C]sync\f[R], \f[C]copy\f[R], \f[C]move\f[R]) are affected by this
|
|
behavior, and only modification is disallowed.
|
|
Files may still be deleted explicitly (e.g.
|
|
\f[C]delete\f[R], \f[C]purge\f[R]) or implicitly (e.g.
|
|
\f[C]sync\f[R], \f[C]move\f[R]).
|
|
Use \f[C]copy \-\-immutable\f[R] if it is desired to avoid deletion as
|
|
well as modification.
|
|
.PP
|
|
This can be useful as an additional layer of protection for immutable or
|
|
append\-only data sets (notably backup archives), where modification
|
|
implies corruption and should not be propagated.
|
|
.SS \-i / \-\-interactive
|
|
.PP
|
|
This flag can be used to tell rclone that you wish a manual confirmation
|
|
before destructive operations.
|
|
.PP
|
|
It is \f[B]recommended\f[R] that you use this flag while learning rclone
|
|
especially with \f[C]rclone sync\f[R].
|
|
.PP
|
|
For example
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
$ rclone delete \-i /tmp/dir
|
|
rclone: delete \[dq]important\-file.txt\[dq]?
|
|
y) Yes, this is OK (default)
|
|
n) No, skip this
|
|
s) Skip all delete operations with no more questions
|
|
!) Do all delete operations with no more questions
|
|
q) Exit rclone now.
|
|
y/n/s/!/q> n
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
The options mean
|
|
.IP \[bu] 2
|
|
\f[C]y\f[R]: \f[B]Yes\f[R], this operation should go ahead.
|
|
You can also press Return for this to happen.
|
|
You\[aq]ll be asked every time unless you choose \f[C]s\f[R] or
|
|
\f[C]!\f[R].
|
|
.IP \[bu] 2
|
|
\f[C]n\f[R]: \f[B]No\f[R], do not do this operation.
|
|
You\[aq]ll be asked every time unless you choose \f[C]s\f[R] or
|
|
\f[C]!\f[R].
|
|
.IP \[bu] 2
|
|
\f[C]s\f[R]: \f[B]Skip\f[R] all the following operations of this type
|
|
with no more questions.
|
|
This takes effect until rclone exits.
|
|
If there are any different kind of operations you\[aq]ll be prompted for
|
|
them.
|
|
.IP \[bu] 2
|
|
\f[C]!\f[R]: \f[B]Do all\f[R] the following operations with no more
|
|
questions.
|
|
Useful if you\[aq]ve decided that you don\[aq]t mind rclone doing that
|
|
kind of operation.
|
|
This takes effect until rclone exits .
|
|
If there are any different kind of operations you\[aq]ll be prompted for
|
|
them.
|
|
.IP \[bu] 2
|
|
\f[C]q\f[R]: \f[B]Quit\f[R] rclone now, just in case!
|
|
.SS \-\-leave\-root
|
|
.PP
|
|
During rmdirs it will not remove root directory, even if it\[aq]s empty.
|
|
.SS \-\-log\-file=FILE
|
|
.PP
|
|
Log all of rclone\[aq]s output to FILE.
|
|
This is not active by default.
|
|
This can be useful for tracking down problems with syncs in combination
|
|
with the \f[C]\-v\f[R] flag.
|
|
See the Logging section for more info.
|
|
.PP
|
|
If FILE exists then rclone will append to it.
|
|
.PP
|
|
Note that if you are using the \f[C]logrotate\f[R] program to manage
|
|
rclone\[aq]s logs, then you should use the \f[C]copytruncate\f[R] option
|
|
as rclone doesn\[aq]t have a signal to rotate logs.
|
|
.SS \-\-log\-format LIST
|
|
.PP
|
|
Comma separated list of log format options.
|
|
\f[C]date\f[R], \f[C]time\f[R], \f[C]microseconds\f[R],
|
|
\f[C]longfile\f[R], \f[C]shortfile\f[R], \f[C]UTC\f[R].
|
|
The default is \[dq]\f[C]date\f[R],\f[C]time\f[R]\[dq].
|
|
.SS \-\-log\-level LEVEL
|
|
.PP
|
|
This sets the log level for rclone.
|
|
The default log level is \f[C]NOTICE\f[R].
|
|
.PP
|
|
\f[C]DEBUG\f[R] is equivalent to \f[C]\-vv\f[R].
|
|
It outputs lots of debug info \- useful for bug reports and really
|
|
finding out what rclone is doing.
|
|
.PP
|
|
\f[C]INFO\f[R] is equivalent to \f[C]\-v\f[R].
|
|
It outputs information about each transfer and prints stats once a
|
|
minute by default.
|
|
.PP
|
|
\f[C]NOTICE\f[R] is the default log level if no logging flags are
|
|
supplied.
|
|
It outputs very little when things are working normally.
|
|
It outputs warnings and significant events.
|
|
.PP
|
|
\f[C]ERROR\f[R] is equivalent to \f[C]\-q\f[R].
|
|
It only outputs error messages.
|
|
.SS \-\-use\-json\-log
|
|
.PP
|
|
This switches the log format to JSON for rclone.
|
|
The fields of json log are level, msg, source, time.
|
|
.SS \-\-low\-level\-retries NUMBER
|
|
.PP
|
|
This controls the number of low level retries rclone does.
|
|
.PP
|
|
A low level retry is used to retry a failing operation \- typically one
|
|
HTTP request.
|
|
This might be uploading a chunk of a big file for example.
|
|
You will see low level retries in the log with the \f[C]\-v\f[R] flag.
|
|
.PP
|
|
This shouldn\[aq]t need to be changed from the default in normal
|
|
operations.
|
|
However, if you get a lot of low level retries you may wish to reduce
|
|
the value so rclone moves on to a high level retry (see the
|
|
\f[C]\-\-retries\f[R] flag) quicker.
|
|
.PP
|
|
Disable low level retries with \f[C]\-\-low\-level\-retries 1\f[R].
|
|
.SS \-\-max\-backlog=N
|
|
.PP
|
|
This is the maximum allowable backlog of files in a sync/copy/move
|
|
queued for being checked or transferred.
|
|
.PP
|
|
This can be set arbitrarily large.
|
|
It will only use memory when the queue is in use.
|
|
Note that it will use in the order of N kB of memory when the backlog is
|
|
in use.
|
|
.PP
|
|
Setting this large allows rclone to calculate how many files are pending
|
|
more accurately, give a more accurate estimated finish time and make
|
|
\f[C]\-\-order\-by\f[R] work more accurately.
|
|
.PP
|
|
Setting this small will make rclone more synchronous to the listings of
|
|
the remote which may be desirable.
|
|
.PP
|
|
Setting this to a negative number will make the backlog as large as
|
|
possible.
|
|
.SS \-\-max\-delete=N
|
|
.PP
|
|
This tells rclone not to delete more than N files.
|
|
If that limit is exceeded then a fatal error will be generated and
|
|
rclone will stop the operation in progress.
|
|
.SS \-\-max\-depth=N
|
|
.PP
|
|
This modifies the recursion depth for all the commands except purge.
|
|
.PP
|
|
So if you do \f[C]rclone \-\-max\-depth 1 ls remote:path\f[R] you will
|
|
see only the files in the top level directory.
|
|
Using \f[C]\-\-max\-depth 2\f[R] means you will see all the files in
|
|
first two directory levels and so on.
|
|
.PP
|
|
For historical reasons the \f[C]lsd\f[R] command defaults to using a
|
|
\f[C]\-\-max\-depth\f[R] of 1 \- you can override this with the command
|
|
line flag.
|
|
.PP
|
|
You can use this command to disable recursion (with
|
|
\f[C]\-\-max\-depth 1\f[R]).
|
|
.PP
|
|
Note that if you use this with \f[C]sync\f[R] and
|
|
\f[C]\-\-delete\-excluded\f[R] the files not recursed through are
|
|
considered excluded and will be deleted on the destination.
|
|
Test first with \f[C]\-\-dry\-run\f[R] if you are not sure what will
|
|
happen.
|
|
.SS \-\-max\-duration=TIME
|
|
.PP
|
|
Rclone will stop scheduling new transfers when it has run for the
|
|
duration specified.
|
|
.PP
|
|
Defaults to off.
|
|
.PP
|
|
When the limit is reached any existing transfers will complete.
|
|
.PP
|
|
Rclone won\[aq]t exit with an error if the transfer limit is reached.
|
|
.SS \-\-max\-transfer=SIZE
|
|
.PP
|
|
Rclone will stop transferring when it has reached the size specified.
|
|
Defaults to off.
|
|
.PP
|
|
When the limit is reached all transfers will stop immediately.
|
|
.PP
|
|
Rclone will exit with exit code 8 if the transfer limit is reached.
|
|
.SS \-\-cutoff\-mode=hard|soft|cautious
|
|
.PP
|
|
This modifies the behavior of \f[C]\-\-max\-transfer\f[R] Defaults to
|
|
\f[C]\-\-cutoff\-mode=hard\f[R].
|
|
.PP
|
|
Specifying \f[C]\-\-cutoff\-mode=hard\f[R] will stop transferring
|
|
immediately when Rclone reaches the limit.
|
|
.PP
|
|
Specifying \f[C]\-\-cutoff\-mode=soft\f[R] will stop starting new
|
|
transfers when Rclone reaches the limit.
|
|
.PP
|
|
Specifying \f[C]\-\-cutoff\-mode=cautious\f[R] will try to prevent
|
|
Rclone from reaching the limit.
|
|
.SS \-\-modify\-window=TIME
|
|
.PP
|
|
When checking whether a file has been modified, this is the maximum
|
|
allowed time difference that a file can have and still be considered
|
|
equivalent.
|
|
.PP
|
|
The default is \f[C]1ns\f[R] unless this is overridden by a remote.
|
|
For example OS X only stores modification times to the nearest second so
|
|
if you are reading and writing to an OS X filing system this will be
|
|
\f[C]1s\f[R] by default.
|
|
.PP
|
|
This command line flag allows you to override that computed default.
|
|
.SS \-\-multi\-thread\-cutoff=SIZE
|
|
.PP
|
|
When downloading files to the local backend above this size, rclone will
|
|
use multiple threads to download the file (default 250M).
|
|
.PP
|
|
Rclone preallocates the file (using
|
|
\f[C]fallocate(FALLOC_FL_KEEP_SIZE)\f[R] on unix or
|
|
\f[C]NTSetInformationFile\f[R] on Windows both of which takes no time)
|
|
then each thread writes directly into the file at the correct place.
|
|
This means that rclone won\[aq]t create fragmented or sparse files and
|
|
there won\[aq]t be any assembly time at the end of the transfer.
|
|
.PP
|
|
The number of threads used to download is controlled by
|
|
\f[C]\-\-multi\-thread\-streams\f[R].
|
|
.PP
|
|
Use \f[C]\-vv\f[R] if you wish to see info about the threads.
|
|
.PP
|
|
This will work with the \f[C]sync\f[R]/\f[C]copy\f[R]/\f[C]move\f[R]
|
|
commands and friends \f[C]copyto\f[R]/\f[C]moveto\f[R].
|
|
Multi thread downloads will be used with \f[C]rclone mount\f[R] and
|
|
\f[C]rclone serve\f[R] if \f[C]\-\-vfs\-cache\-mode\f[R] is set to
|
|
\f[C]writes\f[R] or above.
|
|
.PP
|
|
\f[B]NB\f[R] that this \f[B]only\f[R] works for a local destination but
|
|
will work with any source.
|
|
.PP
|
|
\f[B]NB\f[R] that multi thread copies are disabled for local to local
|
|
copies as they are faster without unless
|
|
\f[C]\-\-multi\-thread\-streams\f[R] is set explicitly.
|
|
.PP
|
|
\f[B]NB\f[R] on Windows using multi\-thread downloads will cause the
|
|
resulting files to be
|
|
sparse (https://en.wikipedia.org/wiki/Sparse_file).
|
|
Use \f[C]\-\-local\-no\-sparse\f[R] to disable sparse files (which may
|
|
cause long delays at the start of downloads) or disable multi\-thread
|
|
downloads with \f[C]\-\-multi\-thread\-streams 0\f[R]
|
|
.SS \-\-multi\-thread\-streams=N
|
|
.PP
|
|
When using multi thread downloads (see above
|
|
\f[C]\-\-multi\-thread\-cutoff\f[R]) this sets the maximum number of
|
|
streams to use.
|
|
Set to \f[C]0\f[R] to disable multi thread downloads (Default 4).
|
|
.PP
|
|
Exactly how many streams rclone uses for the download depends on the
|
|
size of the file.
|
|
To calculate the number of download streams Rclone divides the size of
|
|
the file by the \f[C]\-\-multi\-thread\-cutoff\f[R] and rounds up, up to
|
|
the maximum set with \f[C]\-\-multi\-thread\-streams\f[R].
|
|
.PP
|
|
So if \f[C]\-\-multi\-thread\-cutoff 250MB\f[R] and
|
|
\f[C]\-\-multi\-thread\-streams 4\f[R] are in effect (the defaults):
|
|
.IP \[bu] 2
|
|
0MB..250MB files will be downloaded with 1 stream
|
|
.IP \[bu] 2
|
|
250MB..500MB files will be downloaded with 2 streams
|
|
.IP \[bu] 2
|
|
500MB..750MB files will be downloaded with 3 streams
|
|
.IP \[bu] 2
|
|
750MB+ files will be downloaded with 4 streams
|
|
.SS \-\-no\-check\-dest
|
|
.PP
|
|
The \f[C]\-\-no\-check\-dest\f[R] can be used with \f[C]move\f[R] or
|
|
\f[C]copy\f[R] and it causes rclone not to check the destination at all
|
|
when copying files.
|
|
.PP
|
|
This means that:
|
|
.IP \[bu] 2
|
|
the destination is not listed minimising the API calls
|
|
.IP \[bu] 2
|
|
files are always transferred
|
|
.IP \[bu] 2
|
|
this can cause duplicates on remotes which allow it (e.g.
|
|
Google Drive)
|
|
.IP \[bu] 2
|
|
\f[C]\-\-retries 1\f[R] is recommended otherwise you\[aq]ll transfer
|
|
everything again on a retry
|
|
.PP
|
|
This flag is useful to minimise the transactions if you know that none
|
|
of the files are on the destination.
|
|
.PP
|
|
This is a specialized flag which should be ignored by most users!
|
|
.SS \-\-no\-gzip\-encoding
|
|
.PP
|
|
Don\[aq]t set \f[C]Accept\-Encoding: gzip\f[R].
|
|
This means that rclone won\[aq]t ask the server for compressed files
|
|
automatically.
|
|
Useful if you\[aq]ve set the server to return files with
|
|
\f[C]Content\-Encoding: gzip\f[R] but you uploaded compressed files.
|
|
.PP
|
|
There is no need to set this in normal operation, and doing so will
|
|
decrease the network transfer efficiency of rclone.
|
|
.SS \-\-no\-traverse
|
|
.PP
|
|
The \f[C]\-\-no\-traverse\f[R] flag controls whether the destination
|
|
file system is traversed when using the \f[C]copy\f[R] or \f[C]move\f[R]
|
|
commands.
|
|
\f[C]\-\-no\-traverse\f[R] is not compatible with \f[C]sync\f[R] and
|
|
will be ignored if you supply it with \f[C]sync\f[R].
|
|
.PP
|
|
If you are only copying a small number of files (or are filtering most
|
|
of the files) and/or have a large number of files on the destination
|
|
then \f[C]\-\-no\-traverse\f[R] will stop rclone listing the destination
|
|
and save time.
|
|
.PP
|
|
However, if you are copying a large number of files, especially if you
|
|
are doing a copy where lots of the files under consideration haven\[aq]t
|
|
changed and won\[aq]t need copying then you shouldn\[aq]t use
|
|
\f[C]\-\-no\-traverse\f[R].
|
|
.PP
|
|
See rclone copy (https://rclone.org/commands/rclone_copy/) for an
|
|
example of how to use it.
|
|
.SS \-\-no\-unicode\-normalization
|
|
.PP
|
|
Don\[aq]t normalize unicode characters in filenames during the sync
|
|
routine.
|
|
.PP
|
|
Sometimes, an operating system will store filenames containing unicode
|
|
parts in their decomposed form (particularly macOS).
|
|
Some cloud storage systems will then recompose the unicode, resulting in
|
|
duplicate files if the data is ever copied back to a local filesystem.
|
|
.PP
|
|
Using this flag will disable that functionality, treating each unicode
|
|
character as unique.
|
|
For example, by default e\[u0301] and \['e] will be normalized into the
|
|
same character.
|
|
With \f[C]\-\-no\-unicode\-normalization\f[R] they will be treated as
|
|
unique characters.
|
|
.SS \-\-no\-update\-modtime
|
|
.PP
|
|
When using this flag, rclone won\[aq]t update modification times of
|
|
remote files if they are incorrect as it would normally.
|
|
.PP
|
|
This can be used if the remote is being synced with another tool also
|
|
(e.g.
|
|
the Google Drive client).
|
|
.SS \-\-order\-by string
|
|
.PP
|
|
The \f[C]\-\-order\-by\f[R] flag controls the order in which files in
|
|
the backlog are processed in \f[C]rclone sync\f[R],
|
|
\f[C]rclone copy\f[R] and \f[C]rclone move\f[R].
|
|
.PP
|
|
The order by string is constructed like this.
|
|
The first part describes what aspect is being measured:
|
|
.IP \[bu] 2
|
|
\f[C]size\f[R] \- order by the size of the files
|
|
.IP \[bu] 2
|
|
\f[C]name\f[R] \- order by the full path of the files
|
|
.IP \[bu] 2
|
|
\f[C]modtime\f[R] \- order by the modification date of the files
|
|
.PP
|
|
This can have a modifier appended with a comma:
|
|
.IP \[bu] 2
|
|
\f[C]ascending\f[R] or \f[C]asc\f[R] \- order so that the smallest (or
|
|
oldest) is processed first
|
|
.IP \[bu] 2
|
|
\f[C]descending\f[R] or \f[C]desc\f[R] \- order so that the largest (or
|
|
newest) is processed first
|
|
.IP \[bu] 2
|
|
\f[C]mixed\f[R] \- order so that the smallest is processed first for
|
|
some threads and the largest for others
|
|
.PP
|
|
If the modifier is \f[C]mixed\f[R] then it can have an optional
|
|
percentage (which defaults to \f[C]50\f[R]), e.g.
|
|
\f[C]size,mixed,25\f[R] which means that 25% of the threads should be
|
|
taking the smallest items and 75% the largest.
|
|
The threads which take the smallest first will always take the smallest
|
|
first and likewise the largest first threads.
|
|
The \f[C]mixed\f[R] mode can be useful to minimise the transfer time
|
|
when you are transferring a mixture of large and small files \- the
|
|
large files are guaranteed upload threads and bandwidth and the small
|
|
files will be processed continuously.
|
|
.PP
|
|
If no modifier is supplied then the order is \f[C]ascending\f[R].
|
|
.PP
|
|
For example
|
|
.IP \[bu] 2
|
|
\f[C]\-\-order\-by size,desc\f[R] \- send the largest files first
|
|
.IP \[bu] 2
|
|
\f[C]\-\-order\-by modtime,ascending\f[R] \- send the oldest files first
|
|
.IP \[bu] 2
|
|
\f[C]\-\-order\-by name\f[R] \- send the files with alphabetically by
|
|
path first
|
|
.PP
|
|
If the \f[C]\-\-order\-by\f[R] flag is not supplied or it is supplied
|
|
with an empty string then the default ordering will be used which is as
|
|
scanned.
|
|
With \f[C]\-\-checkers 1\f[R] this is mostly alphabetical, however with
|
|
the default \f[C]\-\-checkers 8\f[R] it is somewhat random.
|
|
.SS Limitations
|
|
.PP
|
|
The \f[C]\-\-order\-by\f[R] flag does not do a separate pass over the
|
|
data.
|
|
This means that it may transfer some files out of the order specified if
|
|
.IP \[bu] 2
|
|
there are no files in the backlog or the source has not been fully
|
|
scanned yet
|
|
.IP \[bu] 2
|
|
there are more than \-\-max\-backlog files in the backlog
|
|
.PP
|
|
Rclone will do its best to transfer the best file it has so in practice
|
|
this should not cause a problem.
|
|
Think of \f[C]\-\-order\-by\f[R] as being more of a best efforts flag
|
|
rather than a perfect ordering.
|
|
.PP
|
|
If you want perfect ordering then you will need to specify
|
|
\-\-check\-first which will find all the files which need transferring
|
|
first before transferring any.
|
|
.SS \-\-password\-command SpaceSepList
|
|
.PP
|
|
This flag supplies a program which should supply the config password
|
|
when run.
|
|
This is an alternative to rclone prompting for the password or setting
|
|
the \f[C]RCLONE_CONFIG_PASS\f[R] variable.
|
|
.PP
|
|
The argument to this should be a command with a space separated list of
|
|
arguments.
|
|
If one of the arguments has a space in then enclose it in
|
|
\f[C]\[dq]\f[R], if you want a literal \f[C]\[dq]\f[R] in an argument
|
|
then enclose the argument in \f[C]\[dq]\f[R] and double the
|
|
\f[C]\[dq]\f[R].
|
|
See CSV encoding (https://godoc.org/encoding/csv) for more info.
|
|
.PP
|
|
Eg
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
\-\-password\-command echo hello
|
|
\-\-password\-command echo \[dq]hello with space\[dq]
|
|
\-\-password\-command echo \[dq]hello with \[dq]\[dq]quotes\[dq]\[dq] and space\[dq]
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
See the Configuration Encryption for more info.
|
|
.PP
|
|
See a Windows PowerShell example on the
|
|
Wiki (https://github.com/rclone/rclone/wiki/Windows-Powershell-use-rclone-password-command-for-Config-file-password).
|
|
.SS \-P, \-\-progress
|
|
.PP
|
|
This flag makes rclone update the stats in a static block in the
|
|
terminal providing a realtime overview of the transfer.
|
|
.PP
|
|
Any log messages will scroll above the static block.
|
|
Log messages will push the static block down to the bottom of the
|
|
terminal where it will stay.
|
|
.PP
|
|
Normally this is updated every 500mS but this period can be overridden
|
|
with the \f[C]\-\-stats\f[R] flag.
|
|
.PP
|
|
This can be used with the \f[C]\-\-stats\-one\-line\f[R] flag for a
|
|
simpler display.
|
|
.PP
|
|
Note: On Windows until this
|
|
bug (https://github.com/Azure/go-ansiterm/issues/26) is fixed all
|
|
non\-ASCII characters will be replaced with \f[C].\f[R] when
|
|
\f[C]\-\-progress\f[R] is in use.
|
|
.SS \-\-progress\-terminal\-title
|
|
.PP
|
|
This flag, when used with \f[C]\-P/\-\-progress\f[R], will print the
|
|
string \f[C]ETA: %s\f[R] to the terminal title.
|
|
.SS \-q, \-\-quiet
|
|
.PP
|
|
This flag will limit rclone\[aq]s output to error messages only.
|
|
.SS \-\-refresh\-times
|
|
.PP
|
|
The \f[C]\-\-refresh\-times\f[R] flag can be used to update modification
|
|
times of existing files when they are out of sync on backends which
|
|
don\[aq]t support hashes.
|
|
.PP
|
|
This is useful if you uploaded files with the incorrect timestamps and
|
|
you now wish to correct them.
|
|
.PP
|
|
This flag is \f[B]only\f[R] useful for destinations which don\[aq]t
|
|
support hashes (e.g.
|
|
\f[C]crypt\f[R]).
|
|
.PP
|
|
This can be used any of the sync commands \f[C]sync\f[R], \f[C]copy\f[R]
|
|
or \f[C]move\f[R].
|
|
.PP
|
|
To use this flag you will need to be doing a modification time sync (so
|
|
not using \f[C]\-\-size\-only\f[R] or \f[C]\-\-checksum\f[R]).
|
|
The flag will have no effect when using \f[C]\-\-size\-only\f[R] or
|
|
\f[C]\-\-checksum\f[R].
|
|
.PP
|
|
If this flag is used when rclone comes to upload a file it will check to
|
|
see if there is an existing file on the destination.
|
|
If this file matches the source with size (and checksum if available)
|
|
but has a differing timestamp then instead of re\-uploading it, rclone
|
|
will update the timestamp on the destination file.
|
|
If the checksum does not match rclone will upload the new file.
|
|
If the checksum is absent (e.g.
|
|
on a \f[C]crypt\f[R] backend) then rclone will update the timestamp.
|
|
.PP
|
|
Note that some remotes can\[aq]t set the modification time without
|
|
re\-uploading the file so this flag is less useful on them.
|
|
.PP
|
|
Normally if you are doing a modification time sync rclone will update
|
|
modification times without \f[C]\-\-refresh\-times\f[R] provided that
|
|
the remote supports checksums \f[B]and\f[R] the checksums match on the
|
|
file.
|
|
However if the checksums are absent then rclone will upload the file
|
|
rather than setting the timestamp as this is the safe behaviour.
|
|
.SS \-\-retries int
|
|
.PP
|
|
Retry the entire sync if it fails this many times it fails (default 3).
|
|
.PP
|
|
Some remotes can be unreliable and a few retries help pick up the files
|
|
which didn\[aq]t get transferred because of errors.
|
|
.PP
|
|
Disable retries with \f[C]\-\-retries 1\f[R].
|
|
.SS \-\-retries\-sleep=TIME
|
|
.PP
|
|
This sets the interval between each retry specified by
|
|
\f[C]\-\-retries\f[R]
|
|
.PP
|
|
The default is \f[C]0\f[R].
|
|
Use \f[C]0\f[R] to disable.
|
|
.SS \-\-size\-only
|
|
.PP
|
|
Normally rclone will look at modification time and size of files to see
|
|
if they are equal.
|
|
If you set this flag then rclone will check only the size.
|
|
.PP
|
|
This can be useful transferring files from Dropbox which have been
|
|
modified by the desktop sync client which doesn\[aq]t set checksums of
|
|
modification times in the same way as rclone.
|
|
.SS \-\-stats=TIME
|
|
.PP
|
|
Commands which transfer data (\f[C]sync\f[R], \f[C]copy\f[R],
|
|
\f[C]copyto\f[R], \f[C]move\f[R], \f[C]moveto\f[R]) will print data
|
|
transfer stats at regular intervals to show their progress.
|
|
.PP
|
|
This sets the interval.
|
|
.PP
|
|
The default is \f[C]1m\f[R].
|
|
Use \f[C]0\f[R] to disable.
|
|
.PP
|
|
If you set the stats interval then all commands can show stats.
|
|
This can be useful when running other commands, \f[C]check\f[R] or
|
|
\f[C]mount\f[R] for example.
|
|
.PP
|
|
Stats are logged at \f[C]INFO\f[R] level by default which means they
|
|
won\[aq]t show at default log level \f[C]NOTICE\f[R].
|
|
Use \f[C]\-\-stats\-log\-level NOTICE\f[R] or \f[C]\-v\f[R] to make them
|
|
show.
|
|
See the Logging section for more info on log levels.
|
|
.PP
|
|
Note that on macOS you can send a SIGINFO (which is normally ctrl\-T in
|
|
the terminal) to make the stats print immediately.
|
|
.SS \-\-stats\-file\-name\-length integer
|
|
.PP
|
|
By default, the \f[C]\-\-stats\f[R] output will truncate file names and
|
|
paths longer than 40 characters.
|
|
This is equivalent to providing
|
|
\f[C]\-\-stats\-file\-name\-length 40\f[R].
|
|
Use \f[C]\-\-stats\-file\-name\-length 0\f[R] to disable any truncation
|
|
of file names printed by stats.
|
|
.SS \-\-stats\-log\-level string
|
|
.PP
|
|
Log level to show \f[C]\-\-stats\f[R] output at.
|
|
This can be \f[C]DEBUG\f[R], \f[C]INFO\f[R], \f[C]NOTICE\f[R], or
|
|
\f[C]ERROR\f[R].
|
|
The default is \f[C]INFO\f[R].
|
|
This means at the default level of logging which is \f[C]NOTICE\f[R] the
|
|
stats won\[aq]t show \- if you want them to then use
|
|
\f[C]\-\-stats\-log\-level NOTICE\f[R].
|
|
See the Logging section for more info on log levels.
|
|
.SS \-\-stats\-one\-line
|
|
.PP
|
|
When this is specified, rclone condenses the stats into a single line
|
|
showing the most important stats only.
|
|
.SS \-\-stats\-one\-line\-date
|
|
.PP
|
|
When this is specified, rclone enables the single\-line stats and
|
|
prepends the display with a date string.
|
|
The default is \f[C]2006/01/02 15:04:05 \-\f[R]
|
|
.SS \-\-stats\-one\-line\-date\-format
|
|
.PP
|
|
When this is specified, rclone enables the single\-line stats and
|
|
prepends the display with a user\-supplied date string.
|
|
The date string MUST be enclosed in quotes.
|
|
Follow golang specs (https://golang.org/pkg/time/#Time.Format) for date
|
|
formatting syntax.
|
|
.SS \-\-stats\-unit=bits|bytes
|
|
.PP
|
|
By default, data transfer rates will be printed in bytes/second.
|
|
.PP
|
|
This option allows the data rate to be printed in bits/second.
|
|
.PP
|
|
Data transfer volume will still be reported in bytes.
|
|
.PP
|
|
The rate is reported as a binary unit, not SI unit.
|
|
So 1 Mbit/s equals 1,048,576 bits/s and not 1,000,000 bits/s.
|
|
.PP
|
|
The default is \f[C]bytes\f[R].
|
|
.SS \-\-suffix=SUFFIX
|
|
.PP
|
|
When using \f[C]sync\f[R], \f[C]copy\f[R] or \f[C]move\f[R] any files
|
|
which would have been overwritten or deleted will have the suffix added
|
|
to them.
|
|
If there is a file with the same path (after the suffix has been added),
|
|
then it will be overwritten.
|
|
.PP
|
|
The remote in use must support server\-side move or copy and you must
|
|
use the same remote as the destination of the sync.
|
|
.PP
|
|
This is for use with files to add the suffix in the current directory or
|
|
with \f[C]\-\-backup\-dir\f[R].
|
|
See \f[C]\-\-backup\-dir\f[R] for more info.
|
|
.PP
|
|
For example
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone copy \-i /path/to/local/file remote:current \-\-suffix .bak
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
will copy \f[C]/path/to/local\f[R] to \f[C]remote:current\f[R], but for
|
|
any files which would have been updated or deleted have .bak added.
|
|
.PP
|
|
If using \f[C]rclone sync\f[R] with \f[C]\-\-suffix\f[R] and without
|
|
\f[C]\-\-backup\-dir\f[R] then it is recommended to put a filter rule in
|
|
excluding the suffix otherwise the \f[C]sync\f[R] will delete the backup
|
|
files.
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone sync \-i /path/to/local/file remote:current \-\-suffix .bak \-\-exclude \[dq]*.bak\[dq]
|
|
\f[R]
|
|
.fi
|
|
.SS \-\-suffix\-keep\-extension
|
|
.PP
|
|
When using \f[C]\-\-suffix\f[R], setting this causes rclone put the
|
|
SUFFIX before the extension of the files that it backs up rather than
|
|
after.
|
|
.PP
|
|
So let\[aq]s say we had \f[C]\-\-suffix \-2019\-01\-01\f[R], without the
|
|
flag \f[C]file.txt\f[R] would be backed up to
|
|
\f[C]file.txt\-2019\-01\-01\f[R] and with the flag it would be backed up
|
|
to \f[C]file\-2019\-01\-01.txt\f[R].
|
|
This can be helpful to make sure the suffixed files can still be opened.
|
|
.SS \-\-syslog
|
|
.PP
|
|
On capable OSes (not Windows or Plan9) send all log output to syslog.
|
|
.PP
|
|
This can be useful for running rclone in a script or
|
|
\f[C]rclone mount\f[R].
|
|
.SS \-\-syslog\-facility string
|
|
.PP
|
|
If using \f[C]\-\-syslog\f[R] this sets the syslog facility (e.g.
|
|
\f[C]KERN\f[R], \f[C]USER\f[R]).
|
|
See \f[C]man syslog\f[R] for a list of possible facilities.
|
|
The default facility is \f[C]DAEMON\f[R].
|
|
.SS \-\-tpslimit float
|
|
.PP
|
|
Limit transactions per second to this number.
|
|
Default is 0 which is used to mean unlimited transactions per second.
|
|
.PP
|
|
A transaction is roughly defined as an API call; its exact meaning will
|
|
depend on the backend.
|
|
For HTTP based backends it is an HTTP PUT/GET/POST/etc and its response.
|
|
For FTP/SFTP it is a round trip transaction over TCP.
|
|
.PP
|
|
For example to limit rclone to 10 transactions per second use
|
|
\f[C]\-\-tpslimit 10\f[R], or to 1 transaction every 2 seconds use
|
|
\f[C]\-\-tpslimit 0.5\f[R].
|
|
.PP
|
|
Use this when the number of transactions per second from rclone is
|
|
causing a problem with the cloud storage provider (e.g.
|
|
getting you banned or rate limited).
|
|
.PP
|
|
This can be very useful for \f[C]rclone mount\f[R] to control the
|
|
behaviour of applications using it.
|
|
.PP
|
|
This limit applies to all HTTP based backends and to the FTP and SFTP
|
|
backends.
|
|
It does not apply to the local backend or the Tardigrade backend.
|
|
.PP
|
|
See also \f[C]\-\-tpslimit\-burst\f[R].
|
|
.SS \-\-tpslimit\-burst int
|
|
.PP
|
|
Max burst of transactions for \f[C]\-\-tpslimit\f[R] (default
|
|
\f[C]1\f[R]).
|
|
.PP
|
|
Normally \f[C]\-\-tpslimit\f[R] will do exactly the number of
|
|
transaction per second specified.
|
|
However if you supply \f[C]\-\-tps\-burst\f[R] then rclone can save up
|
|
some transactions from when it was idle giving a burst of up to the
|
|
parameter supplied.
|
|
.PP
|
|
For example if you provide \f[C]\-\-tpslimit\-burst 10\f[R] then if
|
|
rclone has been idle for more than 10*\f[C]\-\-tpslimit\f[R] then it can
|
|
do 10 transactions very quickly before they are limited again.
|
|
.PP
|
|
This may be used to increase performance of \f[C]\-\-tpslimit\f[R]
|
|
without changing the long term average number of transactions per
|
|
second.
|
|
.SS \-\-track\-renames
|
|
.PP
|
|
By default, rclone doesn\[aq]t keep track of renamed files, so if you
|
|
rename a file locally then sync it to a remote, rclone will delete the
|
|
old file on the remote and upload a new copy.
|
|
.PP
|
|
If you use this flag, and the remote supports server\-side copy or
|
|
server\-side move, and the source and destination have a compatible
|
|
hash, then this will track renames during \f[C]sync\f[R] operations and
|
|
perform renaming server\-side.
|
|
.PP
|
|
Files will be matched by size and hash \- if both match then a rename
|
|
will be considered.
|
|
.PP
|
|
If the destination does not support server\-side copy or move, rclone
|
|
will fall back to the default behaviour and log an error level message
|
|
to the console.
|
|
.PP
|
|
Encrypted destinations are not currently supported by
|
|
\f[C]\-\-track\-renames\f[R] if \f[C]\-\-track\-renames\-strategy\f[R]
|
|
includes \f[C]hash\f[R].
|
|
.PP
|
|
Note that \f[C]\-\-track\-renames\f[R] is incompatible with
|
|
\f[C]\-\-no\-traverse\f[R] and that it uses extra memory to keep track
|
|
of all the rename candidates.
|
|
.PP
|
|
Note also that \f[C]\-\-track\-renames\f[R] is incompatible with
|
|
\f[C]\-\-delete\-before\f[R] and will select \f[C]\-\-delete\-after\f[R]
|
|
instead of \f[C]\-\-delete\-during\f[R].
|
|
.SS \-\-track\-renames\-strategy (hash,modtime,leaf,size)
|
|
.PP
|
|
This option changes the matching criteria for
|
|
\f[C]\-\-track\-renames\f[R].
|
|
.PP
|
|
The matching is controlled by a comma separated selection of these
|
|
tokens:
|
|
.IP \[bu] 2
|
|
\f[C]modtime\f[R] \- the modification time of the file \- not supported
|
|
on all backends
|
|
.IP \[bu] 2
|
|
\f[C]hash\f[R] \- the hash of the file contents \- not supported on all
|
|
backends
|
|
.IP \[bu] 2
|
|
\f[C]leaf\f[R] \- the name of the file not including its directory name
|
|
.IP \[bu] 2
|
|
\f[C]size\f[R] \- the size of the file (this is always enabled)
|
|
.PP
|
|
So using \f[C]\-\-track\-renames\-strategy modtime,leaf\f[R] would match
|
|
files based on modification time, the leaf of the file name and the size
|
|
only.
|
|
.PP
|
|
Using \f[C]\-\-track\-renames\-strategy modtime\f[R] or \f[C]leaf\f[R]
|
|
can enable \f[C]\-\-track\-renames\f[R] support for encrypted
|
|
destinations.
|
|
.PP
|
|
If nothing is specified, the default option is matching by
|
|
\f[C]hash\f[R]es.
|
|
.PP
|
|
Note that the \f[C]hash\f[R] strategy is not supported with encrypted
|
|
destinations.
|
|
.SS \-\-delete\-(before,during,after)
|
|
.PP
|
|
This option allows you to specify when files on your destination are
|
|
deleted when you sync folders.
|
|
.PP
|
|
Specifying the value \f[C]\-\-delete\-before\f[R] will delete all files
|
|
present on the destination, but not on the source \f[I]before\f[R]
|
|
starting the transfer of any new or updated files.
|
|
This uses two passes through the file systems, one for the deletions and
|
|
one for the copies.
|
|
.PP
|
|
Specifying \f[C]\-\-delete\-during\f[R] will delete files while checking
|
|
and uploading files.
|
|
This is the fastest option and uses the least memory.
|
|
.PP
|
|
Specifying \f[C]\-\-delete\-after\f[R] (the default value) will delay
|
|
deletion of files until all new/updated files have been successfully
|
|
transferred.
|
|
The files to be deleted are collected in the copy pass then deleted
|
|
after the copy pass has completed successfully.
|
|
The files to be deleted are held in memory so this mode may use more
|
|
memory.
|
|
This is the safest mode as it will only delete files if there have been
|
|
no errors subsequent to that.
|
|
If there have been errors before the deletions start then you will get
|
|
the message \f[C]not deleting files as there were IO errors\f[R].
|
|
.SS \-\-fast\-list
|
|
.PP
|
|
When doing anything which involves a directory listing (e.g.
|
|
\f[C]sync\f[R], \f[C]copy\f[R], \f[C]ls\f[R] \- in fact nearly every
|
|
command), rclone normally lists a directory and processes it before
|
|
using more directory lists to process any subdirectories.
|
|
This can be parallelised and works very quickly using the least amount
|
|
of memory.
|
|
.PP
|
|
However, some remotes have a way of listing all files beneath a
|
|
directory in one (or a small number) of transactions.
|
|
These tend to be the bucket based remotes (e.g.
|
|
S3, B2, GCS, Swift, Hubic).
|
|
.PP
|
|
If you use the \f[C]\-\-fast\-list\f[R] flag then rclone will use this
|
|
method for listing directories.
|
|
This will have the following consequences for the listing:
|
|
.IP \[bu] 2
|
|
It \f[B]will\f[R] use fewer transactions (important if you pay for them)
|
|
.IP \[bu] 2
|
|
It \f[B]will\f[R] use more memory.
|
|
Rclone has to load the whole listing into memory.
|
|
.IP \[bu] 2
|
|
It \f[I]may\f[R] be faster because it uses fewer transactions
|
|
.IP \[bu] 2
|
|
It \f[I]may\f[R] be slower because it can\[aq]t be parallelized
|
|
.PP
|
|
rclone should always give identical results with and without
|
|
\f[C]\-\-fast\-list\f[R].
|
|
.PP
|
|
If you pay for transactions and can fit your entire sync listing into
|
|
memory then \f[C]\-\-fast\-list\f[R] is recommended.
|
|
If you have a very big sync to do then don\[aq]t use
|
|
\f[C]\-\-fast\-list\f[R] otherwise you will run out of memory.
|
|
.PP
|
|
If you use \f[C]\-\-fast\-list\f[R] on a remote which doesn\[aq]t
|
|
support it, then rclone will just ignore it.
|
|
.SS \-\-timeout=TIME
|
|
.PP
|
|
This sets the IO idle timeout.
|
|
If a transfer has started but then becomes idle for this long it is
|
|
considered broken and disconnected.
|
|
.PP
|
|
The default is \f[C]5m\f[R].
|
|
Set to \f[C]0\f[R] to disable.
|
|
.SS \-\-transfers=N
|
|
.PP
|
|
The number of file transfers to run in parallel.
|
|
It can sometimes be useful to set this to a smaller number if the remote
|
|
is giving a lot of timeouts or bigger if you have lots of bandwidth and
|
|
a fast remote.
|
|
.PP
|
|
The default is to run 4 file transfers in parallel.
|
|
.SS \-u, \-\-update
|
|
.PP
|
|
This forces rclone to skip any files which exist on the destination and
|
|
have a modified time that is newer than the source file.
|
|
.PP
|
|
This can be useful when transferring to a remote which doesn\[aq]t
|
|
support mod times directly (or when using
|
|
\f[C]\-\-use\-server\-modtime\f[R] to avoid extra API calls) as it is
|
|
more accurate than a \f[C]\-\-size\-only\f[R] check and faster than
|
|
using \f[C]\-\-checksum\f[R].
|
|
.PP
|
|
If an existing destination file has a modification time equal (within
|
|
the computed modify window precision) to the source file\[aq]s, it will
|
|
be updated if the sizes are different.
|
|
If \f[C]\-\-checksum\f[R] is set then rclone will update the destination
|
|
if the checksums differ too.
|
|
.PP
|
|
If an existing destination file is older than the source file then it
|
|
will be updated if the size or checksum differs from the source file.
|
|
.PP
|
|
On remotes which don\[aq]t support mod time directly (or when using
|
|
\f[C]\-\-use\-server\-modtime\f[R]) the time checked will be the
|
|
uploaded time.
|
|
This means that if uploading to one of these remotes, rclone will skip
|
|
any files which exist on the destination and have an uploaded time that
|
|
is newer than the modification time of the source file.
|
|
.SS \-\-use\-mmap
|
|
.PP
|
|
If this flag is set then rclone will use anonymous memory allocated by
|
|
mmap on Unix based platforms and VirtualAlloc on Windows for its
|
|
transfer buffers (size controlled by \f[C]\-\-buffer\-size\f[R]).
|
|
Memory allocated like this does not go on the Go heap and can be
|
|
returned to the OS immediately when it is finished with.
|
|
.PP
|
|
If this flag is not set then rclone will allocate and free the buffers
|
|
using the Go memory allocator which may use more memory as memory pages
|
|
are returned less aggressively to the OS.
|
|
.PP
|
|
It is possible this does not work well on all platforms so it is
|
|
disabled by default; in the future it may be enabled by default.
|
|
.SS \-\-use\-server\-modtime
|
|
.PP
|
|
Some object\-store backends (e.g, Swift, S3) do not preserve file
|
|
modification times (modtime).
|
|
On these backends, rclone stores the original modtime as additional
|
|
metadata on the object.
|
|
By default it will make an API call to retrieve the metadata when the
|
|
modtime is needed by an operation.
|
|
.PP
|
|
Use this flag to disable the extra API call and rely instead on the
|
|
server\[aq]s modified time.
|
|
In cases such as a local to remote sync using \f[C]\-\-update\f[R],
|
|
knowing the local file is newer than the time it was last uploaded to
|
|
the remote is sufficient.
|
|
In those cases, this flag can speed up the process and reduce the number
|
|
of API calls necessary.
|
|
.PP
|
|
Using this flag on a sync operation without also using
|
|
\f[C]\-\-update\f[R] would cause all files modified at any time other
|
|
than the last upload time to be uploaded again, which is probably not
|
|
what you want.
|
|
.SS \-v, \-vv, \-\-verbose
|
|
.PP
|
|
With \f[C]\-v\f[R] rclone will tell you about each file that is
|
|
transferred and a small number of significant events.
|
|
.PP
|
|
With \f[C]\-vv\f[R] rclone will become very verbose telling you about
|
|
every file it considers and transfers.
|
|
Please send bug reports with a log with this setting.
|
|
.SS \-V, \-\-version
|
|
.PP
|
|
Prints the version number
|
|
.SS SSL/TLS options
|
|
.PP
|
|
The outgoing SSL/TLS connections rclone makes can be controlled with
|
|
these options.
|
|
For example this can be very useful with the HTTP or WebDAV backends.
|
|
Rclone HTTP servers have their own set of configuration for SSL/TLS
|
|
which you can find in their documentation.
|
|
.SS \-\-ca\-cert string
|
|
.PP
|
|
This loads the PEM encoded certificate authority certificate and uses it
|
|
to verify the certificates of the servers rclone connects to.
|
|
.PP
|
|
If you have generated certificates signed with a local CA then you will
|
|
need this flag to connect to servers using those certificates.
|
|
.SS \-\-client\-cert string
|
|
.PP
|
|
This loads the PEM encoded client side certificate.
|
|
.PP
|
|
This is used for mutual TLS
|
|
authentication (https://en.wikipedia.org/wiki/Mutual_authentication).
|
|
.PP
|
|
The \f[C]\-\-client\-key\f[R] flag is required too when using this.
|
|
.SS \-\-client\-key string
|
|
.PP
|
|
This loads the PEM encoded client side private key used for mutual TLS
|
|
authentication.
|
|
Used in conjunction with \f[C]\-\-client\-cert\f[R].
|
|
.SS \-\-no\-check\-certificate=true/false
|
|
.PP
|
|
\f[C]\-\-no\-check\-certificate\f[R] controls whether a client verifies
|
|
the server\[aq]s certificate chain and host name.
|
|
If \f[C]\-\-no\-check\-certificate\f[R] is true, TLS accepts any
|
|
certificate presented by the server and any host name in that
|
|
certificate.
|
|
In this mode, TLS is susceptible to man\-in\-the\-middle attacks.
|
|
.PP
|
|
This option defaults to \f[C]false\f[R].
|
|
.PP
|
|
\f[B]This should be used only for testing.\f[R]
|
|
.SS Configuration Encryption
|
|
.PP
|
|
Your configuration file contains information for logging in to your
|
|
cloud services.
|
|
This means that you should keep your \f[C]rclone.conf\f[R] file in a
|
|
secure location.
|
|
.PP
|
|
If you are in an environment where that isn\[aq]t possible, you can add
|
|
a password to your configuration.
|
|
This means that you will have to supply the password every time you
|
|
start rclone.
|
|
.PP
|
|
To add a password to your rclone configuration, execute
|
|
\f[C]rclone config\f[R].
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
>rclone config
|
|
Current remotes:
|
|
|
|
e) Edit existing remote
|
|
n) New remote
|
|
d) Delete remote
|
|
s) Set configuration password
|
|
q) Quit config
|
|
e/n/d/s/q>
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
Go into \f[C]s\f[R], Set configuration password:
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
e/n/d/s/q> s
|
|
Your configuration is not encrypted.
|
|
If you add a password, you will protect your login information to cloud services.
|
|
a) Add Password
|
|
q) Quit to main menu
|
|
a/q> a
|
|
Enter NEW configuration password:
|
|
password:
|
|
Confirm NEW password:
|
|
password:
|
|
Password set
|
|
Your configuration is encrypted.
|
|
c) Change Password
|
|
u) Unencrypt configuration
|
|
q) Quit to main menu
|
|
c/u/q>
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
Your configuration is now encrypted, and every time you start rclone you
|
|
will have to supply the password.
|
|
See below for details.
|
|
In the same menu, you can change the password or completely remove
|
|
encryption from your configuration.
|
|
.PP
|
|
There is no way to recover the configuration if you lose your password.
|
|
.PP
|
|
rclone uses nacl
|
|
secretbox (https://godoc.org/golang.org/x/crypto/nacl/secretbox) which
|
|
in turn uses XSalsa20 and Poly1305 to encrypt and authenticate your
|
|
configuration with secret\-key cryptography.
|
|
The password is SHA\-256 hashed, which produces the key for secretbox.
|
|
The hashed password is not stored.
|
|
.PP
|
|
While this provides very good security, we do not recommend storing your
|
|
encrypted rclone configuration in public if it contains sensitive
|
|
information, maybe except if you use a very strong password.
|
|
.PP
|
|
If it is safe in your environment, you can set the
|
|
\f[C]RCLONE_CONFIG_PASS\f[R] environment variable to contain your
|
|
password, in which case it will be used for decrypting the
|
|
configuration.
|
|
.PP
|
|
You can set this for a session from a script.
|
|
For unix like systems save this to a file called
|
|
\f[C]set\-rclone\-password\f[R]:
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
#!/bin/echo Source this file don\[aq]t run it
|
|
|
|
read \-s RCLONE_CONFIG_PASS
|
|
export RCLONE_CONFIG_PASS
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
Then source the file when you want to use it.
|
|
From the shell you would do \f[C]source set\-rclone\-password\f[R].
|
|
It will then ask you for the password and set it in the environment
|
|
variable.
|
|
.PP
|
|
An alternate means of supplying the password is to provide a script
|
|
which will retrieve the password and print on standard output.
|
|
This script should have a fully specified path name and not rely on any
|
|
environment variables.
|
|
The script is supplied either via
|
|
\f[C]\-\-password\-command=\[dq]...\[dq]\f[R] command line argument or
|
|
via the \f[C]RCLONE_PASSWORD_COMMAND\f[R] environment variable.
|
|
.PP
|
|
One useful example of this is using the \f[C]passwordstore\f[R]
|
|
application to retrieve the password:
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
export RCLONE_PASSWORD_COMMAND=\[dq]pass rclone/config\[dq]
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
If the \f[C]passwordstore\f[R] password manager holds the password for
|
|
the rclone configuration, using the script method means the password is
|
|
primarily protected by the \f[C]passwordstore\f[R] system, and is never
|
|
embedded in the clear in scripts, nor available for examination using
|
|
the standard commands available.
|
|
It is quite possible with long running rclone sessions for copies of
|
|
passwords to be innocently captured in log files or terminal scroll
|
|
buffers, etc.
|
|
Using the script method of supplying the password enhances the security
|
|
of the config password considerably.
|
|
.PP
|
|
If you are running rclone inside a script, unless you are using the
|
|
\f[C]\-\-password\-command\f[R] method, you might want to disable
|
|
password prompts.
|
|
To do that, pass the parameter \f[C]\-\-ask\-password=false\f[R] to
|
|
rclone.
|
|
This will make rclone fail instead of asking for a password if
|
|
\f[C]RCLONE_CONFIG_PASS\f[R] doesn\[aq]t contain a valid password, and
|
|
\f[C]\-\-password\-command\f[R] has not been supplied.
|
|
.PP
|
|
Some rclone commands, such as \f[C]genautocomplete\f[R], do not require
|
|
configuration.
|
|
Nevertheless, rclone will read any configuration file found according to
|
|
the rules described above (https://rclone.org/docs/#config-config-file).
|
|
If an encrypted configuration file is found, this means you will be
|
|
prompted for password (unless using \f[C]\-\-password\-command\f[R]).
|
|
To avoid this, you can bypass the loading of the configuration file by
|
|
overriding the location with an empty string \f[C]\[dq]\[dq]\f[R] or the
|
|
special value \f[C]/notfound\f[R], or the os null device represented by
|
|
value \f[C]NUL\f[R] on Windows and \f[C]/dev/null\f[R] on Unix systems
|
|
(before rclone version 1.55 only this null device alternative was
|
|
supported).
|
|
E.g.
|
|
\f[C]rclone \-\-config=\[dq]\[dq] genautocomplete bash\f[R].
|
|
.SS Developer options
|
|
.PP
|
|
These options are useful when developing or debugging rclone.
|
|
There are also some more remote specific options which aren\[aq]t
|
|
documented here which are used for testing.
|
|
These start with remote name e.g.
|
|
\f[C]\-\-drive\-test\-option\f[R] \- see the docs for the remote in
|
|
question.
|
|
.SS \-\-cpuprofile=FILE
|
|
.PP
|
|
Write CPU profile to file.
|
|
This can be analysed with \f[C]go tool pprof\f[R].
|
|
.SS \-\-dump flag,flag,flag
|
|
.PP
|
|
The \f[C]\-\-dump\f[R] flag takes a comma separated list of flags to
|
|
dump info about.
|
|
.PP
|
|
Note that some headers including \f[C]Accept\-Encoding\f[R] as shown may
|
|
not be correct in the request and the response may not show
|
|
\f[C]Content\-Encoding\f[R] if the go standard libraries auto gzip
|
|
encoding was in effect.
|
|
In this case the body of the request will be gunzipped before showing
|
|
it.
|
|
.PP
|
|
The available flags are:
|
|
.SS \-\-dump headers
|
|
.PP
|
|
Dump HTTP headers with \f[C]Authorization:\f[R] lines removed.
|
|
May still contain sensitive info.
|
|
Can be very verbose.
|
|
Useful for debugging only.
|
|
.PP
|
|
Use \f[C]\-\-dump auth\f[R] if you do want the \f[C]Authorization:\f[R]
|
|
headers.
|
|
.SS \-\-dump bodies
|
|
.PP
|
|
Dump HTTP headers and bodies \- may contain sensitive info.
|
|
Can be very verbose.
|
|
Useful for debugging only.
|
|
.PP
|
|
Note that the bodies are buffered in memory so don\[aq]t use this for
|
|
enormous files.
|
|
.SS \-\-dump requests
|
|
.PP
|
|
Like \f[C]\-\-dump bodies\f[R] but dumps the request bodies and the
|
|
response headers.
|
|
Useful for debugging download problems.
|
|
.SS \-\-dump responses
|
|
.PP
|
|
Like \f[C]\-\-dump bodies\f[R] but dumps the response bodies and the
|
|
request headers.
|
|
Useful for debugging upload problems.
|
|
.SS \-\-dump auth
|
|
.PP
|
|
Dump HTTP headers \- will contain sensitive info such as
|
|
\f[C]Authorization:\f[R] headers \- use \f[C]\-\-dump headers\f[R] to
|
|
dump without \f[C]Authorization:\f[R] headers.
|
|
Can be very verbose.
|
|
Useful for debugging only.
|
|
.SS \-\-dump filters
|
|
.PP
|
|
Dump the filters to the output.
|
|
Useful to see exactly what include and exclude options are filtering on.
|
|
.SS \-\-dump goroutines
|
|
.PP
|
|
This dumps a list of the running go\-routines at the end of the command
|
|
to standard output.
|
|
.SS \-\-dump openfiles
|
|
.PP
|
|
This dumps a list of the open files at the end of the command.
|
|
It uses the \f[C]lsof\f[R] command to do that so you\[aq]ll need that
|
|
installed to use it.
|
|
.SS \-\-memprofile=FILE
|
|
.PP
|
|
Write memory profile to file.
|
|
This can be analysed with \f[C]go tool pprof\f[R].
|
|
.SS Filtering
|
|
.PP
|
|
For the filtering options
|
|
.IP \[bu] 2
|
|
\f[C]\-\-delete\-excluded\f[R]
|
|
.IP \[bu] 2
|
|
\f[C]\-\-filter\f[R]
|
|
.IP \[bu] 2
|
|
\f[C]\-\-filter\-from\f[R]
|
|
.IP \[bu] 2
|
|
\f[C]\-\-exclude\f[R]
|
|
.IP \[bu] 2
|
|
\f[C]\-\-exclude\-from\f[R]
|
|
.IP \[bu] 2
|
|
\f[C]\-\-include\f[R]
|
|
.IP \[bu] 2
|
|
\f[C]\-\-include\-from\f[R]
|
|
.IP \[bu] 2
|
|
\f[C]\-\-files\-from\f[R]
|
|
.IP \[bu] 2
|
|
\f[C]\-\-files\-from\-raw\f[R]
|
|
.IP \[bu] 2
|
|
\f[C]\-\-min\-size\f[R]
|
|
.IP \[bu] 2
|
|
\f[C]\-\-max\-size\f[R]
|
|
.IP \[bu] 2
|
|
\f[C]\-\-min\-age\f[R]
|
|
.IP \[bu] 2
|
|
\f[C]\-\-max\-age\f[R]
|
|
.IP \[bu] 2
|
|
\f[C]\-\-dump filters\f[R]
|
|
.PP
|
|
See the filtering section (https://rclone.org/filtering/).
|
|
.SS Remote control
|
|
.PP
|
|
For the remote control options and for instructions on how to remote
|
|
control rclone
|
|
.IP \[bu] 2
|
|
\f[C]\-\-rc\f[R]
|
|
.IP \[bu] 2
|
|
and anything starting with \f[C]\-\-rc\-\f[R]
|
|
.PP
|
|
See the remote control section (https://rclone.org/rc/).
|
|
.SS Logging
|
|
.PP
|
|
rclone has 4 levels of logging, \f[C]ERROR\f[R], \f[C]NOTICE\f[R],
|
|
\f[C]INFO\f[R] and \f[C]DEBUG\f[R].
|
|
.PP
|
|
By default, rclone logs to standard error.
|
|
This means you can redirect standard error and still see the normal
|
|
output of rclone commands (e.g.
|
|
\f[C]rclone ls\f[R]).
|
|
.PP
|
|
By default, rclone will produce \f[C]Error\f[R] and \f[C]Notice\f[R]
|
|
level messages.
|
|
.PP
|
|
If you use the \f[C]\-q\f[R] flag, rclone will only produce
|
|
\f[C]Error\f[R] messages.
|
|
.PP
|
|
If you use the \f[C]\-v\f[R] flag, rclone will produce \f[C]Error\f[R],
|
|
\f[C]Notice\f[R] and \f[C]Info\f[R] messages.
|
|
.PP
|
|
If you use the \f[C]\-vv\f[R] flag, rclone will produce \f[C]Error\f[R],
|
|
\f[C]Notice\f[R], \f[C]Info\f[R] and \f[C]Debug\f[R] messages.
|
|
.PP
|
|
You can also control the log levels with the \f[C]\-\-log\-level\f[R]
|
|
flag.
|
|
.PP
|
|
If you use the \f[C]\-\-log\-file=FILE\f[R] option, rclone will redirect
|
|
\f[C]Error\f[R], \f[C]Info\f[R] and \f[C]Debug\f[R] messages along with
|
|
standard error to FILE.
|
|
.PP
|
|
If you use the \f[C]\-\-syslog\f[R] flag then rclone will log to syslog
|
|
and the \f[C]\-\-syslog\-facility\f[R] control which facility it uses.
|
|
.PP
|
|
Rclone prefixes all log messages with their level in capitals, e.g.
|
|
INFO which makes it easy to grep the log file for different kinds of
|
|
information.
|
|
.SS Exit Code
|
|
.PP
|
|
If any errors occur during the command execution, rclone will exit with
|
|
a non\-zero exit code.
|
|
This allows scripts to detect when rclone operations have failed.
|
|
.PP
|
|
During the startup phase, rclone will exit immediately if an error is
|
|
detected in the configuration.
|
|
There will always be a log message immediately before exiting.
|
|
.PP
|
|
When rclone is running it will accumulate errors as it goes along, and
|
|
only exit with a non\-zero exit code if (after retries) there were still
|
|
failed transfers.
|
|
For every error counted there will be a high priority log message
|
|
(visible with \f[C]\-q\f[R]) showing the message and which file caused
|
|
the problem.
|
|
A high priority message is also shown when starting a retry so the user
|
|
can see that any previous error messages may not be valid after the
|
|
retry.
|
|
If rclone has done a retry it will log a high priority message if the
|
|
retry was successful.
|
|
.SS List of exit codes
|
|
.IP \[bu] 2
|
|
\f[C]0\f[R] \- success
|
|
.IP \[bu] 2
|
|
\f[C]1\f[R] \- Syntax or usage error
|
|
.IP \[bu] 2
|
|
\f[C]2\f[R] \- Error not otherwise categorised
|
|
.IP \[bu] 2
|
|
\f[C]3\f[R] \- Directory not found
|
|
.IP \[bu] 2
|
|
\f[C]4\f[R] \- File not found
|
|
.IP \[bu] 2
|
|
\f[C]5\f[R] \- Temporary error (one that more retries might fix) (Retry
|
|
errors)
|
|
.IP \[bu] 2
|
|
\f[C]6\f[R] \- Less serious errors (like 461 errors from dropbox)
|
|
(NoRetry errors)
|
|
.IP \[bu] 2
|
|
\f[C]7\f[R] \- Fatal error (one that more retries won\[aq]t fix, like
|
|
account suspended) (Fatal errors)
|
|
.IP \[bu] 2
|
|
\f[C]8\f[R] \- Transfer exceeded \- limit set by \-\-max\-transfer
|
|
reached
|
|
.IP \[bu] 2
|
|
\f[C]9\f[R] \- Operation successful, but no files transferred
|
|
.SS Environment Variables
|
|
.PP
|
|
Rclone can be configured entirely using environment variables.
|
|
These can be used to set defaults for options or config file entries.
|
|
.SS Options
|
|
.PP
|
|
Every option in rclone can have its default set by environment variable.
|
|
.PP
|
|
To find the name of the environment variable, first, take the long
|
|
option name, strip the leading \f[C]\-\-\f[R], change \f[C]\-\f[R] to
|
|
\f[C]_\f[R], make upper case and prepend \f[C]RCLONE_\f[R].
|
|
.PP
|
|
For example, to always set \f[C]\-\-stats 5s\f[R], set the environment
|
|
variable \f[C]RCLONE_STATS=5s\f[R].
|
|
If you set stats on the command line this will override the environment
|
|
variable setting.
|
|
.PP
|
|
Or to always use the trash in drive \f[C]\-\-drive\-use\-trash\f[R], set
|
|
\f[C]RCLONE_DRIVE_USE_TRASH=true\f[R].
|
|
.PP
|
|
The same parser is used for the options and the environment variables so
|
|
they take exactly the same form.
|
|
.SS Config file
|
|
.PP
|
|
You can set defaults for values in the config file on an individual
|
|
remote basis.
|
|
The names of the config items are documented in the page for each
|
|
backend.
|
|
.PP
|
|
To find the name of the environment variable, you need to set, take
|
|
\f[C]RCLONE_CONFIG_\f[R] + name of remote + \f[C]_\f[R] + name of config
|
|
file option and make it all uppercase.
|
|
.PP
|
|
For example, to configure an S3 remote named \f[C]mys3:\f[R] without a
|
|
config file (using unix ways of setting environment variables):
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
$ export RCLONE_CONFIG_MYS3_TYPE=s3
|
|
$ export RCLONE_CONFIG_MYS3_ACCESS_KEY_ID=XXX
|
|
$ export RCLONE_CONFIG_MYS3_SECRET_ACCESS_KEY=XXX
|
|
$ rclone lsd MYS3:
|
|
\-1 2016\-09\-21 12:54:21 \-1 my\-bucket
|
|
$ rclone listremotes | grep mys3
|
|
mys3:
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
Note that if you want to create a remote using environment variables you
|
|
must create the \f[C]..._TYPE\f[R] variable as above.
|
|
.PP
|
|
Note also that now rclone has connectionstrings, it is probably easier
|
|
to use those instead which makes the above example
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone lsd :s3,access_key_id=XXX,secret_access_key=XXX:
|
|
\f[R]
|
|
.fi
|
|
.SS Precedence
|
|
.PP
|
|
The various different methods of backend configuration are read in this
|
|
order and the first one with a value is used.
|
|
.IP \[bu] 2
|
|
Flag values as supplied on the command line, e.g.
|
|
\f[C]\-\-drive\-use\-trash\f[R].
|
|
.IP \[bu] 2
|
|
Remote specific environment vars, e.g.
|
|
\f[C]RCLONE_CONFIG_MYREMOTE_USE_TRASH\f[R] (see above).
|
|
.IP \[bu] 2
|
|
Backend specific environment vars, e.g.
|
|
\f[C]RCLONE_DRIVE_USE_TRASH\f[R].
|
|
.IP \[bu] 2
|
|
Config file, e.g.
|
|
\f[C]use_trash = false\f[R].
|
|
.IP \[bu] 2
|
|
Default values, e.g.
|
|
\f[C]true\f[R] \- these can\[aq]t be changed.
|
|
.PP
|
|
So if both \f[C]\-\-drive\-use\-trash\f[R] is supplied on the config
|
|
line and an environment variable \f[C]RCLONE_DRIVE_USE_TRASH\f[R] is
|
|
set, the command line flag will take preference.
|
|
.PP
|
|
For non backend configuration the order is as follows:
|
|
.IP \[bu] 2
|
|
Flag values as supplied on the command line, e.g.
|
|
\f[C]\-\-stats 5s\f[R].
|
|
.IP \[bu] 2
|
|
Environment vars, e.g.
|
|
\f[C]RCLONE_STATS=5s\f[R].
|
|
.IP \[bu] 2
|
|
Default values, e.g.
|
|
\f[C]1m\f[R] \- these can\[aq]t be changed.
|
|
.SS Other environment variables
|
|
.IP \[bu] 2
|
|
\f[C]RCLONE_CONFIG_PASS\f[R] set to contain your config file password
|
|
(see Configuration Encryption section)
|
|
.IP \[bu] 2
|
|
\f[C]HTTP_PROXY\f[R], \f[C]HTTPS_PROXY\f[R] and \f[C]NO_PROXY\f[R] (or
|
|
the lowercase versions thereof).
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
\f[C]HTTPS_PROXY\f[R] takes precedence over \f[C]HTTP_PROXY\f[R] for
|
|
https requests.
|
|
.IP \[bu] 2
|
|
The environment values may be either a complete URL or a
|
|
\[dq]host[:port]\[dq] for, in which case the \[dq]http\[dq] scheme is
|
|
assumed.
|
|
.RE
|
|
.IP \[bu] 2
|
|
\f[C]RCLONE_CONFIG_DIR\f[R] \- rclone \f[B]sets\f[R] this variable for
|
|
use in config files and sub processes to point to the directory holding
|
|
the config file.
|
|
.SH Configuring rclone on a remote / headless machine
|
|
.PP
|
|
Some of the configurations (those involving oauth2) require an Internet
|
|
connected web browser.
|
|
.PP
|
|
If you are trying to set rclone up on a remote or headless box with no
|
|
browser available on it (e.g.
|
|
a NAS or a server in a datacenter) then you will need to use an
|
|
alternative means of configuration.
|
|
There are two ways of doing it, described below.
|
|
.SS Configuring using rclone authorize
|
|
.PP
|
|
On the headless box run \f[C]rclone\f[R] config but answer \f[C]N\f[R]
|
|
to the \f[C]Use auto config?\f[R] question.
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
\&...
|
|
Remote config
|
|
Use auto config?
|
|
* Say Y if not sure
|
|
* Say N if you are working on a remote or headless machine
|
|
y) Yes (default)
|
|
n) No
|
|
y/n> n
|
|
For this to work, you will need rclone available on a machine that has
|
|
a web browser available.
|
|
|
|
For more help and alternate methods see: https://rclone.org/remote_setup/
|
|
|
|
Execute the following on the machine with the web browser (same rclone
|
|
version recommended):
|
|
|
|
rclone authorize \[dq]amazon cloud drive\[dq]
|
|
|
|
Then paste the result below:
|
|
result>
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
Then on your main desktop machine
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone authorize \[dq]amazon cloud drive\[dq]
|
|
If your browser doesn\[aq]t open automatically go to the following link: http://127.0.0.1:53682/auth
|
|
Log in and authorize rclone for access
|
|
Waiting for code...
|
|
Got code
|
|
Paste the following into your remote machine \-\-\->
|
|
SECRET_TOKEN
|
|
<\-\-\-End paste
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
Then back to the headless box, paste in the code
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
result> SECRET_TOKEN
|
|
\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-
|
|
[acd12]
|
|
client_id =
|
|
client_secret =
|
|
token = SECRET_TOKEN
|
|
\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-
|
|
y) Yes this is OK
|
|
e) Edit this remote
|
|
d) Delete this remote
|
|
y/e/d>
|
|
\f[R]
|
|
.fi
|
|
.SS Configuring by copying the config file
|
|
.PP
|
|
Rclone stores all of its config in a single configuration file.
|
|
This can easily be copied to configure a remote rclone.
|
|
.PP
|
|
So first configure rclone on your desktop machine with
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone config
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
to set up the config file.
|
|
.PP
|
|
Find the config file by running \f[C]rclone config file\f[R], for
|
|
example
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
$ rclone config file
|
|
Configuration file is stored at:
|
|
/home/user/.rclone.conf
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
Now transfer it to the remote box (scp, cut paste, ftp, sftp, etc.) and
|
|
place it in the correct place (use \f[C]rclone config file\f[R] on the
|
|
remote box to find out where).
|
|
.SH Filtering, includes and excludes
|
|
.PP
|
|
Filter flags determine which files rclone \f[C]sync\f[R],
|
|
\f[C]move\f[R], \f[C]ls\f[R], \f[C]lsl\f[R], \f[C]md5sum\f[R],
|
|
\f[C]sha1sum\f[R], \f[C]size\f[R], \f[C]delete\f[R], \f[C]check\f[R] and
|
|
similar commands apply to.
|
|
.PP
|
|
They are specified in terms of path/file name patterns; path/file lists;
|
|
file age and size, or presence of a file in a directory.
|
|
Bucket based remotes without the concept of directory apply filters to
|
|
object key, age and size in an analogous way.
|
|
.PP
|
|
Rclone \f[C]purge\f[R] does not obey filters.
|
|
.PP
|
|
To test filters without risk of damage to data, apply them to
|
|
\f[C]rclone ls\f[R], or with the \f[C]\-\-dry\-run\f[R] and
|
|
\f[C]\-vv\f[R] flags.
|
|
.PP
|
|
Rclone filter patterns can only be used in filter command line options,
|
|
not in the specification of a remote.
|
|
.PP
|
|
E.g.
|
|
\f[C]rclone copy \[dq]remote:dir*.jpg\[dq] /path/to/dir\f[R] does not
|
|
have a filter effect.
|
|
\f[C]rclone copy remote:dir /path/to/dir \-\-include \[dq]*.jpg\[dq]\f[R]
|
|
does.
|
|
.PP
|
|
\f[B]Important\f[R] Avoid mixing any two of \f[C]\-\-include...\f[R],
|
|
\f[C]\-\-exclude...\f[R] or \f[C]\-\-filter...\f[R] flags in an rclone
|
|
command.
|
|
The results may not be what you expect.
|
|
Instead use a \f[C]\-\-filter...\f[R] flag.
|
|
.SS Patterns for matching path/file names
|
|
.SS Pattern syntax
|
|
.PP
|
|
Rclone matching rules follow a glob style:
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
\[ga]*\[ga] matches any sequence of non\-separator (\[ga]/\[ga]) characters
|
|
\[ga]**\[ga] matches any sequence of characters including \[ga]/\[ga] separators
|
|
\[ga]?\[ga] matches any single non\-separator (\[ga]/\[ga]) character
|
|
\[ga][\[ga] [ \[ga]!\[ga] ] { character\-range } \[ga]]\[ga]
|
|
character class (must be non\-empty)
|
|
\[ga]{\[ga] pattern\-list \[ga]}\[ga]
|
|
pattern alternatives
|
|
c matches character c (c != \[ga]*\[ga], \[ga]**\[ga], \[ga]?\[ga], \[ga]\[rs]\[ga], \[ga][\[ga], \[ga]{\[ga], \[ga]}\[ga])
|
|
\[ga]\[rs]\[ga] c matches character c
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
character\-range:
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
c matches character c (c != \[ga]\[rs]\[rs]\[ga], \[ga]\-\[ga], \[ga]]\[ga])
|
|
\[ga]\[rs]\[ga] c matches character c
|
|
lo \[ga]\-\[ga] hi matches character c for lo <= c <= hi
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
pattern\-list:
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
pattern { \[ga],\[ga] pattern }
|
|
comma\-separated (without spaces) patterns
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
character classes (see Go regular expression
|
|
reference (https://golang.org/pkg/regexp/syntax/)) include:
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
Named character classes (e.g. [\[rs]d], [\[ha]\[rs]d], [\[rs]D], [\[ha]\[rs]D])
|
|
Perl character classes (e.g. \[rs]s, \[rs]S, \[rs]w, \[rs]W)
|
|
ASCII character classes (e.g. [[:alnum:]], [[:alpha:]], [[:punct:]], [[:xdigit:]])
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
If the filter pattern starts with a \f[C]/\f[R] then it only matches at
|
|
the top level of the directory tree, \f[B]relative to the root of the
|
|
remote\f[R] (not necessarily the root of the drive).
|
|
If it does not start with \f[C]/\f[R] then it is matched starting at the
|
|
\f[B]end of the path/file name\f[R] but it only matches a complete path
|
|
element \- it must match from a \f[C]/\f[R] separator or the beginning
|
|
of the path/file.
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
file.jpg \- matches \[dq]file.jpg\[dq]
|
|
\- matches \[dq]directory/file.jpg\[dq]
|
|
\- doesn\[aq]t match \[dq]afile.jpg\[dq]
|
|
\- doesn\[aq]t match \[dq]directory/afile.jpg\[dq]
|
|
/file.jpg \- matches \[dq]file.jpg\[dq] in the root directory of the remote
|
|
\- doesn\[aq]t match \[dq]afile.jpg\[dq]
|
|
\- doesn\[aq]t match \[dq]directory/file.jpg\[dq]
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
The top level of the remote may not be the top level of the drive.
|
|
.PP
|
|
E.g.
|
|
for a Microsoft Windows local directory structure
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
F:
|
|
\[u251C]\[u2500]\[u2500] bkp
|
|
\[u251C]\[u2500]\[u2500] data
|
|
\[br] \[u251C]\[u2500]\[u2500] excl
|
|
\[br] \[br] \[u251C]\[u2500]\[u2500] 123.jpg
|
|
\[br] \[br] \[u2514]\[u2500]\[u2500] 456.jpg
|
|
\[br] \[u251C]\[u2500]\[u2500] incl
|
|
\[br] \[br] \[u2514]\[u2500]\[u2500] document.pdf
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
To copy the contents of folder \f[C]data\f[R] into folder \f[C]bkp\f[R]
|
|
excluding the contents of subfolder \f[C]excl\f[R]the following command
|
|
treats \f[C]F:\[rs]data\f[R] and \f[C]F:\[rs]bkp\f[R] as top level for
|
|
filtering.
|
|
.PP
|
|
\f[C]rclone copy F:\[rs]data\[rs] F:\[rs]bkp\[rs] \-\-exclude=/excl/**\f[R]
|
|
.PP
|
|
\f[B]Important\f[R] Use \f[C]/\f[R] in path/file name patterns and not
|
|
\f[C]\[rs]\f[R] even if running on Microsoft Windows.
|
|
.PP
|
|
Simple patterns are case sensitive unless the \f[C]\-\-ignore\-case\f[R]
|
|
flag is used.
|
|
.PP
|
|
Without \f[C]\-\-ignore\-case\f[R] (default)
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
potato \- matches \[dq]potato\[dq]
|
|
\- doesn\[aq]t match \[dq]POTATO\[dq]
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
With \f[C]\-\-ignore\-case\f[R]
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
potato \- matches \[dq]potato\[dq]
|
|
\- matches \[dq]POTATO\[dq]
|
|
\f[R]
|
|
.fi
|
|
.SS How filter rules are applied to files
|
|
.PP
|
|
Rclone path/file name filters are made up of one or more of the
|
|
following flags:
|
|
.IP \[bu] 2
|
|
\f[C]\-\-include\f[R]
|
|
.IP \[bu] 2
|
|
\f[C]\-\-include\-from\f[R]
|
|
.IP \[bu] 2
|
|
\f[C]\-\-exclude\f[R]
|
|
.IP \[bu] 2
|
|
\f[C]\-\-exclude\-from\f[R]
|
|
.IP \[bu] 2
|
|
\f[C]\-\-filter\f[R]
|
|
.IP \[bu] 2
|
|
\f[C]\-\-filter\-from\f[R]
|
|
.PP
|
|
There can be more than one instance of individual flags.
|
|
.PP
|
|
Rclone internally uses a combined list of all the include and exclude
|
|
rules.
|
|
The order in which rules are processed can influence the result of the
|
|
filter.
|
|
.PP
|
|
All flags of the same type are processed together in the order above,
|
|
regardless of what order the different types of flags are included on
|
|
the command line.
|
|
.PP
|
|
Multiple instances of the same flag are processed from left to right
|
|
according to their position in the command line.
|
|
.PP
|
|
To mix up the order of processing includes and excludes use
|
|
\f[C]\-\-filter...\f[R] flags.
|
|
.PP
|
|
Within \f[C]\-\-include\-from\f[R], \f[C]\-\-exclude\-from\f[R] and
|
|
\f[C]\-\-filter\-from\f[R] flags rules are processed from top to bottom
|
|
of the referenced file.
|
|
.PP
|
|
If there is an \f[C]\-\-include\f[R] or \f[C]\-\-include\-from\f[R] flag
|
|
specified, rclone implies a \f[C]\- **\f[R] rule which it adds to the
|
|
bottom of the internal rule list.
|
|
Specifying a \f[C]+\f[R] rule with a \f[C]\-\-filter...\f[R] flag does
|
|
not imply that rule.
|
|
.PP
|
|
Each path/file name passed through rclone is matched against the
|
|
combined filter list.
|
|
At first match to a rule the path/file name is included or excluded and
|
|
no further filter rules are processed for that path/file.
|
|
.PP
|
|
If rclone does not find a match, after testing against all rules
|
|
(including the implied rule if appropriate), the path/file name is
|
|
included.
|
|
.PP
|
|
Any path/file included at that stage is processed by the rclone command.
|
|
.PP
|
|
\f[C]\-\-files\-from\f[R] and \f[C]\-\-files\-from\-raw\f[R] flags
|
|
over\-ride and cannot be combined with other filter options.
|
|
.PP
|
|
To see the internal combined rule list, in regular expression form, for
|
|
a command add the \f[C]\-\-dump filters\f[R] flag.
|
|
Running an rclone command with \f[C]\-\-dump filters\f[R] and
|
|
\f[C]\-vv\f[R] flags lists the internal filter elements and shows how
|
|
they are applied to each source path/file.
|
|
There is not currently a means provided to pass regular expression
|
|
filter options into rclone directly though character class filter rules
|
|
contain character classes.
|
|
Go regular expression reference (https://golang.org/pkg/regexp/syntax/)
|
|
.SS How filter rules are applied to directories
|
|
.PP
|
|
Rclone commands are applied to path/file names not directories.
|
|
The entire contents of a directory can be matched to a filter by the
|
|
pattern \f[C]directory/*\f[R] or recursively by \f[C]directory/**\f[R].
|
|
.PP
|
|
Directory filter rules are defined with a closing \f[C]/\f[R] separator.
|
|
.PP
|
|
E.g.
|
|
\f[C]/directory/subdirectory/\f[R] is an rclone directory filter rule.
|
|
.PP
|
|
Rclone commands can use directory filter rules to determine whether they
|
|
recurse into subdirectories.
|
|
This potentially optimises access to a remote by avoiding listing
|
|
unnecessary directories.
|
|
Whether optimisation is desirable depends on the specific filter rules
|
|
and source remote content.
|
|
.PP
|
|
Directory recursion optimisation occurs if either:
|
|
.IP \[bu] 2
|
|
A source remote does not support the rclone \f[C]ListR\f[R] primitive.
|
|
local, sftp, Microsoft OneDrive and WebDav do not support
|
|
\f[C]ListR\f[R].
|
|
Google Drive and most bucket type storage do.
|
|
Full list (https://rclone.org/overview/#optional-features)
|
|
.IP \[bu] 2
|
|
On other remotes (those that support \f[C]ListR\f[R]), if the rclone
|
|
command is not naturally recursive, and provided it is not run with the
|
|
\f[C]\-\-fast\-list\f[R] flag.
|
|
\f[C]ls\f[R], \f[C]lsf \-R\f[R] and \f[C]size\f[R] are naturally
|
|
recursive but \f[C]sync\f[R], \f[C]copy\f[R] and \f[C]move\f[R] are not.
|
|
.IP \[bu] 2
|
|
Whenever the \f[C]\-\-disable ListR\f[R] flag is applied to an rclone
|
|
command.
|
|
.PP
|
|
Rclone commands imply directory filter rules from path/file filter
|
|
rules.
|
|
To view the directory filter rules rclone has implied for a command
|
|
specify the \f[C]\-\-dump filters\f[R] flag.
|
|
.PP
|
|
E.g.
|
|
for an include rule
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
/a/*.jpg
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
Rclone implies the directory include rule
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
/a/
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
Directory filter rules specified in an rclone command can limit the
|
|
scope of an rclone command but path/file filters still have to be
|
|
specified.
|
|
.PP
|
|
E.g.
|
|
\f[C]rclone ls remote: \-\-include /directory/\f[R] will not match any
|
|
files.
|
|
Because it is an \f[C]\-\-include\f[R] option the
|
|
\f[C]\-\-exclude **\f[R] rule is implied, and the \f[C]/directory/\f[R]
|
|
pattern serves only to optimise access to the remote by ignoring
|
|
everything outside of that directory.
|
|
.PP
|
|
E.g.
|
|
\f[C]rclone ls remote: \-\-filter\-from filter\-list.txt\f[R] with a
|
|
file \f[C]filter\-list.txt\f[R]:
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
\- /dir1/
|
|
\- /dir2/
|
|
+ *.pdf
|
|
\- **
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
All files in directories \f[C]dir1\f[R] or \f[C]dir2\f[R] or their
|
|
subdirectories are completely excluded from the listing.
|
|
Only files of suffix \f[C]pdf\f[R] in the root of \f[C]remote:\f[R] or
|
|
its subdirectories are listed.
|
|
The \f[C]\- **\f[R] rule prevents listing of any path/files not
|
|
previously matched by the rules above.
|
|
.PP
|
|
Option \f[C]exclude\-if\-present\f[R] creates a directory exclude rule
|
|
based on the presence of a file in a directory and takes precedence over
|
|
other rclone directory filter rules.
|
|
.PP
|
|
When using pattern list syntax, if a pattern item contains either
|
|
\f[C]/\f[R] or \f[C]**\f[R], then rclone will not able to imply a
|
|
directory filter rule from this pattern list.
|
|
.PP
|
|
E.g.
|
|
for an include rule
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
{dir1/**,dir2/**}
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
Rclone will match files below directories \f[C]dir1\f[R] or
|
|
\f[C]dir2\f[R] only, but will not be able to use this filter to exclude
|
|
a directory \f[C]dir3\f[R] from being traversed.
|
|
.PP
|
|
Directory recursion optimisation may affect performance, but normally
|
|
not the result.
|
|
One exception to this is sync operations with option
|
|
\f[C]\-\-create\-empty\-src\-dirs\f[R], where any traversed empty
|
|
directories will be created.
|
|
With the pattern list example \f[C]{dir1/**,dir2/**}\f[R] above, this
|
|
would create an empty directory \f[C]dir3\f[R] on destination (when it
|
|
exists on source).
|
|
Changing the filter to \f[C]{dir1,dir2}/**\f[R], or splitting it into
|
|
two include rules \f[C]\-\-include dir1/** \-\-include dir2/**\f[R],
|
|
will match the same files while also filtering directories, with the
|
|
result that an empty directory \f[C]dir3\f[R] will no longer be created.
|
|
.SS \f[C]\-\-exclude\f[R] \- Exclude files matching pattern
|
|
.PP
|
|
Excludes path/file names from an rclone command based on a single
|
|
exclude rule.
|
|
.PP
|
|
This flag can be repeated.
|
|
See above for the order filter flags are processed in.
|
|
.PP
|
|
\f[C]\-\-exclude\f[R] should not be used with \f[C]\-\-include\f[R],
|
|
\f[C]\-\-include\-from\f[R], \f[C]\-\-filter\f[R] or
|
|
\f[C]\-\-filter\-from\f[R] flags.
|
|
.PP
|
|
\f[C]\-\-exclude\f[R] has no effect when combined with
|
|
\f[C]\-\-files\-from\f[R] or \f[C]\-\-files\-from\-raw\f[R] flags.
|
|
.PP
|
|
E.g.
|
|
\f[C]rclone ls remote: \-\-exclude *.bak\f[R] excludes all .bak files
|
|
from listing.
|
|
.PP
|
|
E.g.
|
|
\f[C]rclone size remote: \[dq]\-\-exclude /dir/**\[dq]\f[R] returns the
|
|
total size of all files on \f[C]remote:\f[R] excluding those in root
|
|
directory \f[C]dir\f[R] and sub directories.
|
|
.PP
|
|
E.g.
|
|
on Microsoft Windows
|
|
\f[C]rclone ls remote: \-\-exclude \[dq]*\[rs][{JP,KR,HK}\[rs]]*\[dq]\f[R]
|
|
lists the files in \f[C]remote:\f[R] with \f[C][JP]\f[R] or
|
|
\f[C][KR]\f[R] or \f[C][HK]\f[R] in their name.
|
|
Quotes prevent the shell from interpreting the \f[C]\[rs]\f[R]
|
|
characters.\f[C]\[rs]\f[R] characters escape the \f[C][\f[R] and
|
|
\f[C]]\f[R] so an rclone filter treats them literally rather than as a
|
|
character\-range.
|
|
The \f[C]{\f[R] and \f[C]}\f[R] define an rclone pattern list.
|
|
For other operating systems single quotes are required ie
|
|
\f[C]rclone ls remote: \-\-exclude \[aq]*\[rs][{JP,KR,HK}\[rs]]*\[aq]\f[R]
|
|
.SS \f[C]\-\-exclude\-from\f[R] \- Read exclude patterns from file
|
|
.PP
|
|
Excludes path/file names from an rclone command based on rules in a
|
|
named file.
|
|
The file contains a list of remarks and pattern rules.
|
|
.PP
|
|
For an example \f[C]exclude\-file.txt\f[R]:
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
# a sample exclude rule file
|
|
*.bak
|
|
file2.jpg
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
\f[C]rclone ls remote: \-\-exclude\-from exclude\-file.txt\f[R] lists
|
|
the files on \f[C]remote:\f[R] except those named \f[C]file2.jpg\f[R] or
|
|
with a suffix \f[C].bak\f[R].
|
|
That is equivalent to
|
|
\f[C]rclone ls remote: \-\-exclude file2.jpg \-\-exclude \[dq]*.bak\[dq]\f[R].
|
|
.PP
|
|
This flag can be repeated.
|
|
See above for the order filter flags are processed in.
|
|
.PP
|
|
The \f[C]\-\-exclude\-from\f[R] flag is useful where multiple exclude
|
|
filter rules are applied to an rclone command.
|
|
.PP
|
|
\f[C]\-\-exclude\-from\f[R] should not be used with
|
|
\f[C]\-\-include\f[R], \f[C]\-\-include\-from\f[R], \f[C]\-\-filter\f[R]
|
|
or \f[C]\-\-filter\-from\f[R] flags.
|
|
.PP
|
|
\f[C]\-\-exclude\-from\f[R] has no effect when combined with
|
|
\f[C]\-\-files\-from\f[R] or \f[C]\-\-files\-from\-raw\f[R] flags.
|
|
.PP
|
|
\f[C]\-\-exclude\-from\f[R] followed by \f[C]\-\f[R] reads filter rules
|
|
from standard input.
|
|
.SS \f[C]\-\-include\f[R] \- Include files matching pattern
|
|
.PP
|
|
Adds a single include rule based on path/file names to an rclone
|
|
command.
|
|
.PP
|
|
This flag can be repeated.
|
|
See above for the order filter flags are processed in.
|
|
.PP
|
|
\f[C]\-\-include\f[R] has no effect when combined with
|
|
\f[C]\-\-files\-from\f[R] or \f[C]\-\-files\-from\-raw\f[R] flags.
|
|
.PP
|
|
\f[C]\-\-include\f[R] implies \f[C]\-\-exclude **\f[R] at the end of an
|
|
rclone internal filter list.
|
|
Therefore if you mix \f[C]\-\-include\f[R] and
|
|
\f[C]\-\-include\-from\f[R] flags with \f[C]\-\-exclude\f[R],
|
|
\f[C]\-\-exclude\-from\f[R], \f[C]\-\-filter\f[R] or
|
|
\f[C]\-\-filter\-from\f[R], you must use include rules for all the files
|
|
you want in the include statement.
|
|
For more flexibility use the \f[C]\-\-filter\-from\f[R] flag.
|
|
.PP
|
|
E.g.
|
|
\f[C]rclone ls remote: \-\-include \[dq]*.{png,jpg}\[dq]\f[R] lists the
|
|
files on \f[C]remote:\f[R] with suffix \f[C].png\f[R] and
|
|
\f[C].jpg\f[R].
|
|
All other files are excluded.
|
|
.PP
|
|
E.g.
|
|
multiple rclone copy commands can be combined with \f[C]\-\-include\f[R]
|
|
and a pattern\-list.
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone copy /vol1/A remote:A
|
|
rclone copy /vol1/B remote:B
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
is equivalent to:
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone copy /vol1 remote: \-\-include \[dq]{A,B}/**\[dq]
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
E.g.
|
|
\f[C]rclone ls remote:/wheat \-\-include \[dq]??[\[ha][:punct:]]*\[dq]\f[R]
|
|
lists the files \f[C]remote:\f[R] directory \f[C]wheat\f[R] (and
|
|
subdirectories) whose third character is not punctuation.
|
|
This example uses an ASCII character
|
|
class (https://golang.org/pkg/regexp/syntax/).
|
|
.SS \f[C]\-\-include\-from\f[R] \- Read include patterns from file
|
|
.PP
|
|
Adds path/file names to an rclone command based on rules in a named
|
|
file.
|
|
The file contains a list of remarks and pattern rules.
|
|
.PP
|
|
For an example \f[C]include\-file.txt\f[R]:
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
# a sample include rule file
|
|
*.jpg
|
|
file2.avi
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
\f[C]rclone ls remote: \-\-include\-from include\-file.txt\f[R] lists
|
|
the files on \f[C]remote:\f[R] with name \f[C]file2.avi\f[R] or suffix
|
|
\f[C].jpg\f[R].
|
|
That is equivalent to
|
|
\f[C]rclone ls remote: \-\-include file2.avi \-\-include \[dq]*.jpg\[dq]\f[R].
|
|
.PP
|
|
This flag can be repeated.
|
|
See above for the order filter flags are processed in.
|
|
.PP
|
|
The \f[C]\-\-include\-from\f[R] flag is useful where multiple include
|
|
filter rules are applied to an rclone command.
|
|
.PP
|
|
\f[C]\-\-include\-from\f[R] implies \f[C]\-\-exclude **\f[R] at the end
|
|
of an rclone internal filter list.
|
|
Therefore if you mix \f[C]\-\-include\f[R] and
|
|
\f[C]\-\-include\-from\f[R] flags with \f[C]\-\-exclude\f[R],
|
|
\f[C]\-\-exclude\-from\f[R], \f[C]\-\-filter\f[R] or
|
|
\f[C]\-\-filter\-from\f[R], you must use include rules for all the files
|
|
you want in the include statement.
|
|
For more flexibility use the \f[C]\-\-filter\-from\f[R] flag.
|
|
.PP
|
|
\f[C]\-\-exclude\-from\f[R] has no effect when combined with
|
|
\f[C]\-\-files\-from\f[R] or \f[C]\-\-files\-from\-raw\f[R] flags.
|
|
.PP
|
|
\f[C]\-\-exclude\-from\f[R] followed by \f[C]\-\f[R] reads filter rules
|
|
from standard input.
|
|
.SS \f[C]\-\-filter\f[R] \- Add a file\-filtering rule
|
|
.PP
|
|
Specifies path/file names to an rclone command, based on a single
|
|
include or exclude rule, in \f[C]+\f[R] or \f[C]\-\f[R] format.
|
|
.PP
|
|
This flag can be repeated.
|
|
See above for the order filter flags are processed in.
|
|
.PP
|
|
\f[C]\-\-filter +\f[R] differs from \f[C]\-\-include\f[R].
|
|
In the case of \f[C]\-\-include\f[R] rclone implies an
|
|
\f[C]\-\-exclude *\f[R] rule which it adds to the bottom of the internal
|
|
rule list.
|
|
\f[C]\-\-filter...+\f[R] does not imply that rule.
|
|
.PP
|
|
\f[C]\-\-filter\f[R] has no effect when combined with
|
|
\f[C]\-\-files\-from\f[R] or \f[C]\-\-files\-from\-raw\f[R] flags.
|
|
.PP
|
|
\f[C]\-\-filter\f[R] should not be used with \f[C]\-\-include\f[R],
|
|
\f[C]\-\-include\-from\f[R], \f[C]\-\-exclude\f[R] or
|
|
\f[C]\-\-exclude\-from\f[R] flags.
|
|
.PP
|
|
E.g.
|
|
\f[C]rclone ls remote: \-\-filter \[dq]\- *.bak\[dq]\f[R] excludes all
|
|
\f[C].bak\f[R] files from a list of \f[C]remote:\f[R].
|
|
.SS \f[C]\-\-filter\-from\f[R] \- Read filtering patterns from a file
|
|
.PP
|
|
Adds path/file names to an rclone command based on rules in a named
|
|
file.
|
|
The file contains a list of remarks and pattern rules.
|
|
Include rules start with \f[C]+\f[R] and exclude rules with
|
|
\f[C]\-\f[R].
|
|
\f[C]!\f[R] clears existing rules.
|
|
Rules are processed in the order they are defined.
|
|
.PP
|
|
This flag can be repeated.
|
|
See above for the order filter flags are processed in.
|
|
.PP
|
|
Arrange the order of filter rules with the most restrictive first and
|
|
work down.
|
|
.PP
|
|
E.g.
|
|
for \f[C]filter\-file.txt\f[R]:
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
# a sample filter rule file
|
|
\- secret*.jpg
|
|
+ *.jpg
|
|
+ *.png
|
|
+ file2.avi
|
|
\- /dir/Trash/**
|
|
+ /dir/**
|
|
# exclude everything else
|
|
\- *
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
\f[C]rclone ls remote: \-\-filter\-from filter\-file.txt\f[R] lists the
|
|
path/files on \f[C]remote:\f[R] including all \f[C]jpg\f[R] and
|
|
\f[C]png\f[R] files, excluding any matching \f[C]secret*.jpg\f[R] and
|
|
including \f[C]file2.avi\f[R].
|
|
It also includes everything in the directory \f[C]dir\f[R] at the root
|
|
of \f[C]remote\f[R], except \f[C]remote:dir/Trash\f[R] which it
|
|
excludes.
|
|
Everything else is excluded.
|
|
.PP
|
|
E.g.
|
|
for an alternative \f[C]filter\-file.txt\f[R]:
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
\- secret*.jpg
|
|
+ *.jpg
|
|
+ *.png
|
|
+ file2.avi
|
|
\- *
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
Files \f[C]file1.jpg\f[R], \f[C]file3.png\f[R] and \f[C]file2.avi\f[R]
|
|
are listed whilst \f[C]secret17.jpg\f[R] and files without the suffix
|
|
\&.jpg\f[C]or\f[R].png\[ga] are excluded.
|
|
.PP
|
|
E.g.
|
|
for an alternative \f[C]filter\-file.txt\f[R]:
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
+ *.jpg
|
|
+ *.gif
|
|
!
|
|
+ 42.doc
|
|
\- *
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
Only file 42.doc is listed.
|
|
Prior rules are cleared by the \f[C]!\f[R].
|
|
.SS \f[C]\-\-files\-from\f[R] \- Read list of source\-file names
|
|
.PP
|
|
Adds path/files to an rclone command from a list in a named file.
|
|
Rclone processes the path/file names in the order of the list, and no
|
|
others.
|
|
.PP
|
|
Other filter flags (\f[C]\-\-include\f[R], \f[C]\-\-include\-from\f[R],
|
|
\f[C]\-\-exclude\f[R], \f[C]\-\-exclude\-from\f[R], \f[C]\-\-filter\f[R]
|
|
and \f[C]\-\-filter\-from\f[R]) are ignored when
|
|
\f[C]\-\-files\-from\f[R] is used.
|
|
.PP
|
|
\f[C]\-\-files\-from\f[R] expects a list of files as its input.
|
|
Leading or trailing whitespace is stripped from the input lines.
|
|
Lines starting with \f[C]#\f[R] or \f[C];\f[R] are ignored.
|
|
.PP
|
|
Rclone commands with a \f[C]\-\-files\-from\f[R] flag traverse the
|
|
remote, treating the names in \f[C]\-\-files\-from\f[R] as a set of
|
|
filters.
|
|
.PP
|
|
If the \f[C]\-\-no\-traverse\f[R] and \f[C]\-\-files\-from\f[R] flags
|
|
are used together an rclone command does not traverse the remote.
|
|
Instead it addresses each path/file named in the file individually.
|
|
For each path/file name, that requires typically 1 API call.
|
|
This can be efficient for a short \f[C]\-\-files\-from\f[R] list and a
|
|
remote containing many files.
|
|
.PP
|
|
Rclone commands do not error if any names in the
|
|
\f[C]\-\-files\-from\f[R] file are missing from the source remote.
|
|
.PP
|
|
The \f[C]\-\-files\-from\f[R] flag can be repeated in a single rclone
|
|
command to read path/file names from more than one file.
|
|
The files are read from left to right along the command line.
|
|
.PP
|
|
Paths within the \f[C]\-\-files\-from\f[R] file are interpreted as
|
|
starting with the root specified in the rclone command.
|
|
Leading \f[C]/\f[R] separators are ignored.
|
|
See \-\-files\-from\-raw if you need the input to be processed in a raw
|
|
manner.
|
|
.PP
|
|
E.g.
|
|
for a file \f[C]files\-from.txt\f[R]:
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
# comment
|
|
file1.jpg
|
|
subdir/file2.jpg
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
\f[C]rclone copy \-\-files\-from files\-from.txt /home/me/pics remote:pics\f[R]
|
|
copies the following, if they exist, and only those files.
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
/home/me/pics/file1.jpg \[->] remote:pics/file1.jpg
|
|
/home/me/pics/subdir/file2.jpg \[->] remote:pics/subdir/file2.jpg
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
E.g.
|
|
to copy the following files referenced by their absolute paths:
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
/home/user1/42
|
|
/home/user1/dir/ford
|
|
/home/user2/prefect
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
First find a common subdirectory \- in this case \f[C]/home\f[R] and put
|
|
the remaining files in \f[C]files\-from.txt\f[R] with or without leading
|
|
\f[C]/\f[R], e.g.
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
user1/42
|
|
user1/dir/ford
|
|
user2/prefect
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
Then copy these to a remote:
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone copy \-\-files\-from files\-from.txt /home remote:backup
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
The three files are transferred as follows:
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
/home/user1/42 \[->] remote:backup/user1/important
|
|
/home/user1/dir/ford \[->] remote:backup/user1/dir/file
|
|
/home/user2/prefect \[->] remote:backup/user2/stuff
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
Alternatively if \f[C]/\f[R] is chosen as root \f[C]files\-from.txt\f[R]
|
|
will be:
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
/home/user1/42
|
|
/home/user1/dir/ford
|
|
/home/user2/prefect
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
The copy command will be:
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone copy \-\-files\-from files\-from.txt / remote:backup
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
Then there will be an extra \f[C]home\f[R] directory on the remote:
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
/home/user1/42 \[->] remote:backup/home/user1/42
|
|
/home/user1/dir/ford \[->] remote:backup/home/user1/dir/ford
|
|
/home/user2/prefect \[->] remote:backup/home/user2/prefect
|
|
\f[R]
|
|
.fi
|
|
.SS \f[C]\-\-files\-from\-raw\f[R] \- Read list of source\-file names without any processing
|
|
.PP
|
|
This flag is the same as \f[C]\-\-files\-from\f[R] except that input is
|
|
read in a raw manner.
|
|
Lines with leading / trailing whitespace, and lines starting with
|
|
\f[C];\f[R] or \f[C]#\f[R] are read without any processing.
|
|
rclone lsf (https://rclone.org/commands/rclone_lsf/) has a compatible
|
|
format that can be used to export file lists from remotes for input to
|
|
\f[C]\-\-files\-from\-raw\f[R].
|
|
.SS \f[C]\-\-ignore\-case\f[R] \- make searches case insensitive
|
|
.PP
|
|
By default rclone filter patterns are case sensitive.
|
|
The \f[C]\-\-ignore\-case\f[R] flag makes all of the filters patterns on
|
|
the command line case insensitive.
|
|
.PP
|
|
E.g.
|
|
\f[C]\-\-include \[dq]zaphod.txt\[dq]\f[R] does not match a file
|
|
\f[C]Zaphod.txt\f[R].
|
|
With \f[C]\-\-ignore\-case\f[R] a match is made.
|
|
.SS Quoting shell metacharacters
|
|
.PP
|
|
Rclone commands with filter patterns containing shell metacharacters may
|
|
not as work as expected in your shell and may require quoting.
|
|
.PP
|
|
E.g.
|
|
linux, OSX (\f[C]*\f[R] metacharacter)
|
|
.IP \[bu] 2
|
|
\f[C]\-\-include \[rs]*.jpg\f[R]
|
|
.IP \[bu] 2
|
|
\f[C]\-\-include \[aq]*.jpg\[aq]\f[R]
|
|
.IP \[bu] 2
|
|
\f[C]\-\-include=\[aq]*.jpg\[aq]\f[R]
|
|
.PP
|
|
Microsoft Windows expansion is done by the command, not shell, so
|
|
\f[C]\-\-include *.jpg\f[R] does not require quoting.
|
|
.PP
|
|
If the rclone error
|
|
\f[C]Command .... needs .... arguments maximum: you provided .... non flag arguments:\f[R]
|
|
is encountered, the cause is commonly spaces within the name of a remote
|
|
or flag value.
|
|
The fix then is to quote values containing spaces.
|
|
.SS Other filters
|
|
.SS \f[C]\-\-min\-size\f[R] \- Don\[aq]t transfer any file smaller than this
|
|
.PP
|
|
Controls the minimum size file within the scope of an rclone command.
|
|
Default units are \f[C]kBytes\f[R] but abbreviations \f[C]k\f[R],
|
|
\f[C]M\f[R], or \f[C]G\f[R] are valid.
|
|
.PP
|
|
E.g.
|
|
\f[C]rclone ls remote: \-\-min\-size 50k\f[R] lists files on
|
|
\f[C]remote:\f[R] of 50kByte size or larger.
|
|
.SS \f[C]\-\-max\-size\f[R] \- Don\[aq]t transfer any file larger than this
|
|
.PP
|
|
Controls the maximum size file within the scope of an rclone command.
|
|
Default units are \f[C]kBytes\f[R] but abbreviations \f[C]k\f[R],
|
|
\f[C]M\f[R], or \f[C]G\f[R] are valid.
|
|
.PP
|
|
E.g.
|
|
\f[C]rclone ls remote: \-\-max\-size 1G\f[R] lists files on
|
|
\f[C]remote:\f[R] of 1GByte size or smaller.
|
|
.SS \f[C]\-\-max\-age\f[R] \- Don\[aq]t transfer any file older than this
|
|
.PP
|
|
Controls the maximum age of files within the scope of an rclone command.
|
|
Default units are seconds or the following abbreviations are valid:
|
|
.IP \[bu] 2
|
|
\f[C]ms\f[R] \- Milliseconds
|
|
.IP \[bu] 2
|
|
\f[C]s\f[R] \- Seconds
|
|
.IP \[bu] 2
|
|
\f[C]m\f[R] \- Minutes
|
|
.IP \[bu] 2
|
|
\f[C]h\f[R] \- Hours
|
|
.IP \[bu] 2
|
|
\f[C]d\f[R] \- Days
|
|
.IP \[bu] 2
|
|
\f[C]w\f[R] \- Weeks
|
|
.IP \[bu] 2
|
|
\f[C]M\f[R] \- Months
|
|
.IP \[bu] 2
|
|
\f[C]y\f[R] \- Years
|
|
.PP
|
|
\f[C]\-\-max\-age\f[R] can also be specified as an absolute time in the
|
|
following formats:
|
|
.IP \[bu] 2
|
|
RFC3339 \- e.g.
|
|
\f[C]2006\-01\-02T15:04:05Z\f[R] or
|
|
\f[C]2006\-01\-02T15:04:05+07:00\f[R]
|
|
.IP \[bu] 2
|
|
ISO8601 Date and time, local timezone \- \f[C]2006\-01\-02T15:04:05\f[R]
|
|
.IP \[bu] 2
|
|
ISO8601 Date and time, local timezone \- \f[C]2006\-01\-02 15:04:05\f[R]
|
|
.IP \[bu] 2
|
|
ISO8601 Date \- \f[C]2006\-01\-02\f[R] (YYYY\-MM\-DD)
|
|
.PP
|
|
\f[C]\-\-max\-age\f[R] applies only to files and not to directories.
|
|
.PP
|
|
E.g.
|
|
\f[C]rclone ls remote: \-\-max\-age 2d\f[R] lists files on
|
|
\f[C]remote:\f[R] of 2 days old or less.
|
|
.SS \f[C]\-\-min\-age\f[R] \- Don\[aq]t transfer any file younger than this
|
|
.PP
|
|
Controls the minimum age of files within the scope of an rclone command.
|
|
(see \f[C]\-\-max\-age\f[R] for valid formats)
|
|
.PP
|
|
\f[C]\-\-min\-age\f[R] applies only to files and not to directories.
|
|
.PP
|
|
E.g.
|
|
\f[C]rclone ls remote: \-\-min\-age 2d\f[R] lists files on
|
|
\f[C]remote:\f[R] of 2 days old or more.
|
|
.SS Other flags
|
|
.SS \f[C]\-\-delete\-excluded\f[R] \- Delete files on dest excluded from sync
|
|
.PP
|
|
\f[B]Important\f[R] this flag is dangerous to your data \- use with
|
|
\f[C]\-\-dry\-run\f[R] and \f[C]\-v\f[R] first.
|
|
.PP
|
|
In conjunction with \f[C]rclone sync\f[R],
|
|
\f[C]\-\-delete\-excluded\f[R] deletes any files on the destination
|
|
which are excluded from the command.
|
|
.PP
|
|
E.g.
|
|
the scope of \f[C]rclone sync \-i A: B:\f[R] can be restricted:
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone \-\-min\-size 50k \-\-delete\-excluded sync A: B:
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
All files on \f[C]B:\f[R] which are less than 50 kBytes are deleted
|
|
because they are excluded from the rclone sync command.
|
|
.SS \f[C]\-\-dump filters\f[R] \- dump the filters to the output
|
|
.PP
|
|
Dumps the defined filters to standard output in regular expression
|
|
format.
|
|
.PP
|
|
Useful for debugging.
|
|
.SS Exclude directory based on a file
|
|
.PP
|
|
The \f[C]\-\-exclude\-if\-present\f[R] flag controls whether a directory
|
|
is within the scope of an rclone command based on the presence of a
|
|
named file within it.
|
|
.PP
|
|
This flag has a priority over other filter flags.
|
|
.PP
|
|
E.g.
|
|
for the following directory structure:
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
dir1/file1
|
|
dir1/dir2/file2
|
|
dir1/dir2/dir3/file3
|
|
dir1/dir2/dir3/.ignore
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
The command \f[C]rclone ls \-\-exclude\-if\-present .ignore dir1\f[R]
|
|
does not list \f[C]dir3\f[R], \f[C]file3\f[R] or \f[C].ignore\f[R].
|
|
.PP
|
|
\f[C]\-\-exclude\-if\-present\f[R] can only be used once in an rclone
|
|
command.
|
|
.SS Common pitfalls
|
|
.PP
|
|
The most frequent filter support issues on the rclone
|
|
forum (https://forum.rclone.org/) are:
|
|
.IP \[bu] 2
|
|
Not using paths relative to the root of the remote
|
|
.IP \[bu] 2
|
|
Not using \f[C]/\f[R] to match from the root of a remote
|
|
.IP \[bu] 2
|
|
Not using \f[C]**\f[R] to match the contents of a directory
|
|
.SH GUI (Experimental)
|
|
.PP
|
|
Rclone can serve a web based GUI (graphical user interface).
|
|
This is somewhat experimental at the moment so things may be subject to
|
|
change.
|
|
.PP
|
|
Run this command in a terminal and rclone will download and then display
|
|
the GUI in a web browser.
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone rcd \-\-rc\-web\-gui
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
This will produce logs like this and rclone needs to continue to run to
|
|
serve the GUI:
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
2019/08/25 11:40:14 NOTICE: A new release for gui is present at https://github.com/rclone/rclone\-webui\-react/releases/download/v0.0.6/currentbuild.zip
|
|
2019/08/25 11:40:14 NOTICE: Downloading webgui binary. Please wait. [Size: 3813937, Path : /home/USER/.cache/rclone/webgui/v0.0.6.zip]
|
|
2019/08/25 11:40:16 NOTICE: Unzipping
|
|
2019/08/25 11:40:16 NOTICE: Serving remote control on http://127.0.0.1:5572/
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
This assumes you are running rclone locally on your machine.
|
|
It is possible to separate the rclone and the GUI \- see below for
|
|
details.
|
|
.PP
|
|
If you wish to check for updates then you can add
|
|
\f[C]\-\-rc\-web\-gui\-update\f[R] to the command line.
|
|
.PP
|
|
If you find your GUI broken, you may force it to update by add
|
|
\f[C]\-\-rc\-web\-gui\-force\-update\f[R].
|
|
.PP
|
|
By default, rclone will open your browser.
|
|
Add \f[C]\-\-rc\-web\-gui\-no\-open\-browser\f[R] to disable this
|
|
feature.
|
|
.SS Using the GUI
|
|
.PP
|
|
Once the GUI opens, you will be looking at the dashboard which has an
|
|
overall overview.
|
|
.PP
|
|
On the left hand side you will see a series of view buttons you can
|
|
click on:
|
|
.IP \[bu] 2
|
|
Dashboard \- main overview
|
|
.IP \[bu] 2
|
|
Configs \- examine and create new configurations
|
|
.IP \[bu] 2
|
|
Explorer \- view, download and upload files to the cloud storage systems
|
|
.IP \[bu] 2
|
|
Backend \- view or alter the backend config
|
|
.IP \[bu] 2
|
|
Log out
|
|
.PP
|
|
(More docs and walkthrough video to come!)
|
|
.SS How it works
|
|
.PP
|
|
When you run the \f[C]rclone rcd \-\-rc\-web\-gui\f[R] this is what
|
|
happens
|
|
.IP \[bu] 2
|
|
Rclone starts but only runs the remote control API (\[dq]rc\[dq]).
|
|
.IP \[bu] 2
|
|
The API is bound to localhost with an auto generated username and
|
|
password.
|
|
.IP \[bu] 2
|
|
If the API bundle is missing then rclone will download it.
|
|
.IP \[bu] 2
|
|
rclone will start serving the files from the API bundle over the same
|
|
port as the API
|
|
.IP \[bu] 2
|
|
rclone will open the browser with a \f[C]login_token\f[R] so it can log
|
|
straight in.
|
|
.SS Advanced use
|
|
.PP
|
|
The \f[C]rclone rcd\f[R] may use any of the flags documented on the rc
|
|
page (https://rclone.org/rc/#supported-parameters).
|
|
.PP
|
|
The flag \f[C]\-\-rc\-web\-gui\f[R] is shorthand for
|
|
.IP \[bu] 2
|
|
Download the web GUI if necessary
|
|
.IP \[bu] 2
|
|
Check we are using some authentication
|
|
.IP \[bu] 2
|
|
\f[C]\-\-rc\-user gui\f[R]
|
|
.IP \[bu] 2
|
|
\f[C]\-\-rc\-pass <random password>\f[R]
|
|
.IP \[bu] 2
|
|
\f[C]\-\-rc\-serve\f[R]
|
|
.PP
|
|
These flags can be overridden as desired.
|
|
.PP
|
|
See also the rclone rcd
|
|
documentation (https://rclone.org/commands/rclone_rcd/).
|
|
.SS Example: Running a public GUI
|
|
.PP
|
|
For example the GUI could be served on a public port over SSL using an
|
|
htpasswd file using the following flags:
|
|
.IP \[bu] 2
|
|
\f[C]\-\-rc\-web\-gui\f[R]
|
|
.IP \[bu] 2
|
|
\f[C]\-\-rc\-addr :443\f[R]
|
|
.IP \[bu] 2
|
|
\f[C]\-\-rc\-htpasswd /path/to/htpasswd\f[R]
|
|
.IP \[bu] 2
|
|
\f[C]\-\-rc\-cert /path/to/ssl.crt\f[R]
|
|
.IP \[bu] 2
|
|
\f[C]\-\-rc\-key /path/to/ssl.key\f[R]
|
|
.SS Example: Running a GUI behind a proxy
|
|
.PP
|
|
If you want to run the GUI behind a proxy at \f[C]/rclone\f[R] you could
|
|
use these flags:
|
|
.IP \[bu] 2
|
|
\f[C]\-\-rc\-web\-gui\f[R]
|
|
.IP \[bu] 2
|
|
\f[C]\-\-rc\-baseurl rclone\f[R]
|
|
.IP \[bu] 2
|
|
\f[C]\-\-rc\-htpasswd /path/to/htpasswd\f[R]
|
|
.PP
|
|
Or instead of htpasswd if you just want a single user and password:
|
|
.IP \[bu] 2
|
|
\f[C]\-\-rc\-user me\f[R]
|
|
.IP \[bu] 2
|
|
\f[C]\-\-rc\-pass mypassword\f[R]
|
|
.SS Project
|
|
.PP
|
|
The GUI is being developed in the: rclone/rclone\-webui\-react
|
|
repository (https://github.com/rclone/rclone-webui-react).
|
|
.PP
|
|
Bug reports and contributions are very welcome :\-)
|
|
.PP
|
|
If you have questions then please ask them on the rclone
|
|
forum (https://forum.rclone.org/).
|
|
.SH Remote controlling rclone with its API
|
|
.PP
|
|
If rclone is run with the \f[C]\-\-rc\f[R] flag then it starts an HTTP
|
|
server which can be used to remote control rclone using its API.
|
|
.PP
|
|
You can either use the rclone rc command to access the API or use HTTP
|
|
directly.
|
|
.PP
|
|
If you just want to run a remote control then see the rcd
|
|
command (https://rclone.org/commands/rclone_rcd/).
|
|
.SS Supported parameters
|
|
.SS \-\-rc
|
|
.PP
|
|
Flag to start the http server listen on remote requests
|
|
.SS \-\-rc\-addr=IP
|
|
.PP
|
|
IPaddress:Port or :Port to bind server to.
|
|
(default \[dq]localhost:5572\[dq])
|
|
.SS \-\-rc\-cert=KEY
|
|
.PP
|
|
SSL PEM key (concatenation of certificate and CA certificate)
|
|
.SS \-\-rc\-client\-ca=PATH
|
|
.PP
|
|
Client certificate authority to verify clients with
|
|
.SS \-\-rc\-htpasswd=PATH
|
|
.PP
|
|
htpasswd file \- if not provided no authentication is done
|
|
.SS \-\-rc\-key=PATH
|
|
.PP
|
|
SSL PEM Private key
|
|
.SS \-\-rc\-max\-header\-bytes=VALUE
|
|
.PP
|
|
Maximum size of request header (default 4096)
|
|
.SS \-\-rc\-user=VALUE
|
|
.PP
|
|
User name for authentication.
|
|
.SS \-\-rc\-pass=VALUE
|
|
.PP
|
|
Password for authentication.
|
|
.SS \-\-rc\-realm=VALUE
|
|
.PP
|
|
Realm for authentication (default \[dq]rclone\[dq])
|
|
.SS \-\-rc\-server\-read\-timeout=DURATION
|
|
.PP
|
|
Timeout for server reading data (default 1h0m0s)
|
|
.SS \-\-rc\-server\-write\-timeout=DURATION
|
|
.PP
|
|
Timeout for server writing data (default 1h0m0s)
|
|
.SS \-\-rc\-serve
|
|
.PP
|
|
Enable the serving of remote objects via the HTTP interface.
|
|
This means objects will be accessible at http://127.0.0.1:5572/ by
|
|
default, so you can browse to http://127.0.0.1:5572/ or
|
|
http://127.0.0.1:5572/* to see a listing of the remotes.
|
|
Objects may be requested from remotes using this syntax
|
|
http://127.0.0.1:5572/[remote:path]/path/to/object
|
|
.PP
|
|
Default Off.
|
|
.SS \-\-rc\-files /path/to/directory
|
|
.PP
|
|
Path to local files to serve on the HTTP server.
|
|
.PP
|
|
If this is set then rclone will serve the files in that directory.
|
|
It will also open the root in the web browser if specified.
|
|
This is for implementing browser based GUIs for rclone functions.
|
|
.PP
|
|
If \f[C]\-\-rc\-user\f[R] or \f[C]\-\-rc\-pass\f[R] is set then the URL
|
|
that is opened will have the authorization in the URL in the
|
|
\f[C]http://user:pass\[at]localhost/\f[R] style.
|
|
.PP
|
|
Default Off.
|
|
.SS \-\-rc\-enable\-metrics
|
|
.PP
|
|
Enable OpenMetrics/Prometheus compatible endpoint at \f[C]/metrics\f[R].
|
|
.PP
|
|
Default Off.
|
|
.SS \-\-rc\-web\-gui
|
|
.PP
|
|
Set this flag to serve the default web gui on the same port as rclone.
|
|
.PP
|
|
Default Off.
|
|
.SS \-\-rc\-allow\-origin
|
|
.PP
|
|
Set the allowed Access\-Control\-Allow\-Origin for rc requests.
|
|
.PP
|
|
Can be used with \-\-rc\-web\-gui if the rclone is running on different
|
|
IP than the web\-gui.
|
|
.PP
|
|
Default is IP address on which rc is running.
|
|
.SS \-\-rc\-web\-fetch\-url
|
|
.PP
|
|
Set the URL to fetch the rclone\-web\-gui files from.
|
|
.PP
|
|
Default
|
|
https://api.github.com/repos/rclone/rclone\-webui\-react/releases/latest.
|
|
.SS \-\-rc\-web\-gui\-update
|
|
.PP
|
|
Set this flag to check and update rclone\-webui\-react from the
|
|
rc\-web\-fetch\-url.
|
|
.PP
|
|
Default Off.
|
|
.SS \-\-rc\-web\-gui\-force\-update
|
|
.PP
|
|
Set this flag to force update rclone\-webui\-react from the
|
|
rc\-web\-fetch\-url.
|
|
.PP
|
|
Default Off.
|
|
.SS \-\-rc\-web\-gui\-no\-open\-browser
|
|
.PP
|
|
Set this flag to disable opening browser automatically when using
|
|
web\-gui.
|
|
.PP
|
|
Default Off.
|
|
.SS \-\-rc\-job\-expire\-duration=DURATION
|
|
.PP
|
|
Expire finished async jobs older than DURATION (default 60s).
|
|
.SS \-\-rc\-job\-expire\-interval=DURATION
|
|
.PP
|
|
Interval duration to check for expired async jobs (default 10s).
|
|
.SS \-\-rc\-no\-auth
|
|
.PP
|
|
By default rclone will require authorisation to have been set up on the
|
|
rc interface in order to use any methods which access any rclone
|
|
remotes.
|
|
Eg \f[C]operations/list\f[R] is denied as it involved creating a remote
|
|
as is \f[C]sync/copy\f[R].
|
|
.PP
|
|
If this is set then no authorisation will be required on the server to
|
|
use these methods.
|
|
The alternative is to use \f[C]\-\-rc\-user\f[R] and
|
|
\f[C]\-\-rc\-pass\f[R] and use these credentials in the request.
|
|
.PP
|
|
Default Off.
|
|
.SS Accessing the remote control via the rclone rc command
|
|
.PP
|
|
Rclone itself implements the remote control protocol in its
|
|
\f[C]rclone rc\f[R] command.
|
|
.PP
|
|
You can use it like this
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
$ rclone rc rc/noop param1=one param2=two
|
|
{
|
|
\[dq]param1\[dq]: \[dq]one\[dq],
|
|
\[dq]param2\[dq]: \[dq]two\[dq]
|
|
}
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
Run \f[C]rclone rc\f[R] on its own to see the help for the installed
|
|
remote control commands.
|
|
.SS JSON input
|
|
.PP
|
|
\f[C]rclone rc\f[R] also supports a \f[C]\-\-json\f[R] flag which can be
|
|
used to send more complicated input parameters.
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
$ rclone rc \-\-json \[aq]{ \[dq]p1\[dq]: [1,\[dq]2\[dq],null,4], \[dq]p2\[dq]: { \[dq]a\[dq]:1, \[dq]b\[dq]:2 } }\[aq] rc/noop
|
|
{
|
|
\[dq]p1\[dq]: [
|
|
1,
|
|
\[dq]2\[dq],
|
|
null,
|
|
4
|
|
],
|
|
\[dq]p2\[dq]: {
|
|
\[dq]a\[dq]: 1,
|
|
\[dq]b\[dq]: 2
|
|
}
|
|
}
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
If the parameter being passed is an object then it can be passed as a
|
|
JSON string rather than using the \f[C]\-\-json\f[R] flag which
|
|
simplifies the command line.
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone rc operations/list fs=/tmp remote=test opt=\[aq]{\[dq]showHash\[dq]: true}\[aq]
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
Rather than
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone rc operations/list \-\-json \[aq]{\[dq]fs\[dq]: \[dq]/tmp\[dq], \[dq]remote\[dq]: \[dq]test\[dq], \[dq]opt\[dq]: {\[dq]showHash\[dq]: true}}\[aq]
|
|
\f[R]
|
|
.fi
|
|
.SS Special parameters
|
|
.PP
|
|
The rc interface supports some special parameters which apply to
|
|
\f[B]all\f[R] commands.
|
|
These start with \f[C]_\f[R] to show they are different.
|
|
.SS Running asynchronous jobs with _async = true
|
|
.PP
|
|
Each rc call is classified as a job and it is assigned its own id.
|
|
By default jobs are executed immediately as they are created or
|
|
synchronously.
|
|
.PP
|
|
If \f[C]_async\f[R] has a true value when supplied to an rc call then it
|
|
will return immediately with a job id and the task will be run in the
|
|
background.
|
|
The \f[C]job/status\f[R] call can be used to get information of the
|
|
background job.
|
|
The job can be queried for up to 1 minute after it has finished.
|
|
.PP
|
|
It is recommended that potentially long running jobs, e.g.
|
|
\f[C]sync/sync\f[R], \f[C]sync/copy\f[R], \f[C]sync/move\f[R],
|
|
\f[C]operations/purge\f[R] are run with the \f[C]_async\f[R] flag to
|
|
avoid any potential problems with the HTTP request and response timing
|
|
out.
|
|
.PP
|
|
Starting a job with the \f[C]_async\f[R] flag:
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
$ rclone rc \-\-json \[aq]{ \[dq]p1\[dq]: [1,\[dq]2\[dq],null,4], \[dq]p2\[dq]: { \[dq]a\[dq]:1, \[dq]b\[dq]:2 }, \[dq]_async\[dq]: true }\[aq] rc/noop
|
|
{
|
|
\[dq]jobid\[dq]: 2
|
|
}
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
Query the status to see if the job has finished.
|
|
For more information on the meaning of these return parameters see the
|
|
\f[C]job/status\f[R] call.
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
$ rclone rc \-\-json \[aq]{ \[dq]jobid\[dq]:2 }\[aq] job/status
|
|
{
|
|
\[dq]duration\[dq]: 0.000124163,
|
|
\[dq]endTime\[dq]: \[dq]2018\-10\-27T11:38:07.911245881+01:00\[dq],
|
|
\[dq]error\[dq]: \[dq]\[dq],
|
|
\[dq]finished\[dq]: true,
|
|
\[dq]id\[dq]: 2,
|
|
\[dq]output\[dq]: {
|
|
\[dq]_async\[dq]: true,
|
|
\[dq]p1\[dq]: [
|
|
1,
|
|
\[dq]2\[dq],
|
|
null,
|
|
4
|
|
],
|
|
\[dq]p2\[dq]: {
|
|
\[dq]a\[dq]: 1,
|
|
\[dq]b\[dq]: 2
|
|
}
|
|
},
|
|
\[dq]startTime\[dq]: \[dq]2018\-10\-27T11:38:07.911121728+01:00\[dq],
|
|
\[dq]success\[dq]: true
|
|
}
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
\f[C]job/list\f[R] can be used to show the running or recently completed
|
|
jobs
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
$ rclone rc job/list
|
|
{
|
|
\[dq]jobids\[dq]: [
|
|
2
|
|
]
|
|
}
|
|
\f[R]
|
|
.fi
|
|
.SS Setting config flags with _config
|
|
.PP
|
|
If you wish to set config (the equivalent of the global flags) for the
|
|
duration of an rc call only then pass in the \f[C]_config\f[R]
|
|
parameter.
|
|
.PP
|
|
This should be in the same format as the \f[C]config\f[R] key returned
|
|
by options/get.
|
|
.PP
|
|
For example, if you wished to run a sync with the \f[C]\-\-checksum\f[R]
|
|
parameter, you would pass this parameter in your JSON blob.
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
\[dq]_config\[dq]:{\[dq]CheckSum\[dq]: true}
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
If using \f[C]rclone rc\f[R] this could be passed as
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone rc operations/sync ... _config=\[aq]{\[dq]CheckSum\[dq]: true}\[aq]
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
Any config parameters you don\[aq]t set will inherit the global defaults
|
|
which were set with command line flags or environment variables.
|
|
.PP
|
|
Note that it is possible to set some values as strings or integers \-
|
|
see data types (/#data-types) for more info.
|
|
Here is an example setting the equivalent of \f[C]\-\-buffer\-size\f[R]
|
|
in string or integer format.
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
\[dq]_config\[dq]:{\[dq]BufferSize\[dq]: \[dq]42M\[dq]}
|
|
\[dq]_config\[dq]:{\[dq]BufferSize\[dq]: 44040192}
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
If you wish to check the \f[C]_config\f[R] assignment has worked
|
|
properly then calling \f[C]options/local\f[R] will show what the value
|
|
got set to.
|
|
.SS Setting filter flags with _filter
|
|
.PP
|
|
If you wish to set filters for the duration of an rc call only then pass
|
|
in the \f[C]_filter\f[R] parameter.
|
|
.PP
|
|
This should be in the same format as the \f[C]filter\f[R] key returned
|
|
by options/get.
|
|
.PP
|
|
For example, if you wished to run a sync with these flags
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
\-\-max\-size 1M \-\-max\-age 42s \-\-include \[dq]a\[dq] \-\-include \[dq]b\[dq]
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
you would pass this parameter in your JSON blob.
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
\[dq]_filter\[dq]:{\[dq]MaxSize\[dq]:\[dq]1M\[dq], \[dq]IncludeRule\[dq]:[\[dq]a\[dq],\[dq]b\[dq]], \[dq]MaxAge\[dq]:\[dq]42s\[dq]}
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
If using \f[C]rclone rc\f[R] this could be passed as
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone rc ... _filter=\[aq]{\[dq]MaxSize\[dq]:\[dq]1M\[dq], \[dq]IncludeRule\[dq]:[\[dq]a\[dq],\[dq]b\[dq]], \[dq]MaxAge\[dq]:\[dq]42s\[dq]}\[aq]
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
Any filter parameters you don\[aq]t set will inherit the global defaults
|
|
which were set with command line flags or environment variables.
|
|
.PP
|
|
Note that it is possible to set some values as strings or integers \-
|
|
see data types (/#data-types) for more info.
|
|
Here is an example setting the equivalent of \f[C]\-\-buffer\-size\f[R]
|
|
in string or integer format.
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
\[dq]_filter\[dq]:{\[dq]MinSize\[dq]: \[dq]42M\[dq]}
|
|
\[dq]_filter\[dq]:{\[dq]MinSize\[dq]: 44040192}
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
If you wish to check the \f[C]_filter\f[R] assignment has worked
|
|
properly then calling \f[C]options/local\f[R] will show what the value
|
|
got set to.
|
|
.SS Assigning operations to groups with _group = value
|
|
.PP
|
|
Each rc call has its own stats group for tracking its metrics.
|
|
By default grouping is done by the composite group name from prefix
|
|
\f[C]job/\f[R] and id of the job like so \f[C]job/1\f[R].
|
|
.PP
|
|
If \f[C]_group\f[R] has a value then stats for that request will be
|
|
grouped under that value.
|
|
This allows caller to group stats under their own name.
|
|
.PP
|
|
Stats for specific group can be accessed by passing \f[C]group\f[R] to
|
|
\f[C]core/stats\f[R]:
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
$ rclone rc \-\-json \[aq]{ \[dq]group\[dq]: \[dq]job/1\[dq] }\[aq] core/stats
|
|
{
|
|
\[dq]speed\[dq]: 12345
|
|
...
|
|
}
|
|
\f[R]
|
|
.fi
|
|
.SS Data types
|
|
.PP
|
|
When the API returns types, these will mostly be straight forward
|
|
integer, string or boolean types.
|
|
.PP
|
|
However some of the types returned by the options/get call and taken by
|
|
the options/set calls as well as the \f[C]vfsOpt\f[R],
|
|
\f[C]mountOpt\f[R] and the \f[C]_config\f[R] parameters.
|
|
.IP \[bu] 2
|
|
\f[C]Duration\f[R] \- these are returned as an integer duration in
|
|
nanoseconds.
|
|
They may be set as an integer, or they may be set with time string, eg
|
|
\[dq]5s\[dq].
|
|
See the options section (https://rclone.org/docs/#options) for more
|
|
info.
|
|
.IP \[bu] 2
|
|
\f[C]Size\f[R] \- these are returned as an integer number of bytes.
|
|
They may be set as an integer or they may be set with a size suffix
|
|
string, eg \[dq]10M\[dq].
|
|
See the options section (https://rclone.org/docs/#options) for more
|
|
info.
|
|
.IP \[bu] 2
|
|
Enumerated type (such as \f[C]CutoffMode\f[R], \f[C]DumpFlags\f[R],
|
|
\f[C]LogLevel\f[R], \f[C]VfsCacheMode\f[R] \- these will be returned as
|
|
an integer and may be set as an integer but more conveniently they can
|
|
be set as a string, eg \[dq]HARD\[dq] for \f[C]CutoffMode\f[R] or
|
|
\f[C]DEBUG\f[R] for \f[C]LogLevel\f[R].
|
|
.IP \[bu] 2
|
|
\f[C]BandwidthSpec\f[R] \- this will be set and returned as a string, eg
|
|
\[dq]1M\[dq].
|
|
.SS Specifying remotes to work on
|
|
.PP
|
|
Remotes are specified with the \f[C]fs=\f[R], \f[C]srcFs=\f[R],
|
|
\f[C]dstFs=\f[R] parameters depending on the command being used.
|
|
.PP
|
|
The parameters can be a string as per the rest of rclone, eg
|
|
\f[C]s3:bucket/path\f[R] or \f[C]:sftp:/my/dir\f[R].
|
|
They can also be specified as JSON blobs.
|
|
.PP
|
|
If specifyng a JSON blob it should be a object mapping strings to
|
|
strings.
|
|
These values will be used to configure the remote.
|
|
There are 3 special values which may be set:
|
|
.IP \[bu] 2
|
|
\f[C]type\f[R] \- set to \f[C]type\f[R] to specify a remote called
|
|
\f[C]:type:\f[R]
|
|
.IP \[bu] 2
|
|
\f[C]_name\f[R] \- set to \f[C]name\f[R] to specify a remote called
|
|
\f[C]name:\f[R]
|
|
.IP \[bu] 2
|
|
\f[C]_root\f[R] \- sets the root of the remote \- may be empty
|
|
.PP
|
|
One of \f[C]_name\f[R] or \f[C]type\f[R] should normally be set.
|
|
If the \f[C]local\f[R] backend is desired then \f[C]type\f[R] should be
|
|
set to \f[C]local\f[R].
|
|
If \f[C]_root\f[R] isn\[aq]t specified then it defaults to the root of
|
|
the remote.
|
|
.PP
|
|
For example this JSON is equivalent to \f[C]remote:/tmp\f[R]
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
{
|
|
\[dq]_name\[dq]: \[dq]remote\[dq],
|
|
\[dq]_path\[dq]: \[dq]/tmp\[dq]
|
|
}
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
And this is equivalent to
|
|
\f[C]:sftp,host=\[aq]example.com\[aq]:/tmp\f[R]
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
{
|
|
\[dq]type\[dq]: \[dq]sftp\[dq],
|
|
\[dq]host\[dq]: \[dq]example.com\[dq],
|
|
\[dq]_path\[dq]: \[dq]/tmp\[dq]
|
|
}
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
And this is equivalent to \f[C]/tmp/dir\f[R]
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
{
|
|
type = \[dq]local\[dq],
|
|
_ path = \[dq]/tmp/dir\[dq]
|
|
}
|
|
\f[R]
|
|
.fi
|
|
.SS Supported commands
|
|
.SS backend/command: Runs a backend command.
|
|
.PP
|
|
This takes the following parameters
|
|
.IP \[bu] 2
|
|
command \- a string with the command name
|
|
.IP \[bu] 2
|
|
fs \- a remote name string e.g.
|
|
\[dq]drive:\[dq]
|
|
.IP \[bu] 2
|
|
arg \- a list of arguments for the backend command
|
|
.IP \[bu] 2
|
|
opt \- a map of string to string of options
|
|
.PP
|
|
Returns
|
|
.IP \[bu] 2
|
|
result \- result from the backend command
|
|
.PP
|
|
For example
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone rc backend/command command=noop fs=. \-o echo=yes \-o blue \-a path1 \-a path2
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
Returns
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
{
|
|
\[dq]result\[dq]: {
|
|
\[dq]arg\[dq]: [
|
|
\[dq]path1\[dq],
|
|
\[dq]path2\[dq]
|
|
],
|
|
\[dq]name\[dq]: \[dq]noop\[dq],
|
|
\[dq]opt\[dq]: {
|
|
\[dq]blue\[dq]: \[dq]\[dq],
|
|
\[dq]echo\[dq]: \[dq]yes\[dq]
|
|
}
|
|
}
|
|
}
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
Note that this is the direct equivalent of using this \[dq]backend\[dq]
|
|
command:
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone backend noop . \-o echo=yes \-o blue path1 path2
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
Note that arguments must be preceded by the \[dq]\-a\[dq] flag
|
|
.PP
|
|
See the backend (https://rclone.org/commands/rclone_backend/) command
|
|
for more information.
|
|
.PP
|
|
\f[B]Authentication is required for this call.\f[R]
|
|
.SS cache/expire: Purge a remote from cache
|
|
.PP
|
|
Purge a remote from the cache backend.
|
|
Supports either a directory or a file.
|
|
Params: \- remote = path to remote (required) \- withData = true/false
|
|
to delete cached data (chunks) as well (optional)
|
|
.PP
|
|
Eg
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone rc cache/expire remote=path/to/sub/folder/
|
|
rclone rc cache/expire remote=/ withData=true
|
|
\f[R]
|
|
.fi
|
|
.SS cache/fetch: Fetch file chunks
|
|
.PP
|
|
Ensure the specified file chunks are cached on disk.
|
|
.PP
|
|
The chunks= parameter specifies the file chunks to check.
|
|
It takes a comma separated list of array slice indices.
|
|
The slice indices are similar to Python slices: start[:end]
|
|
.PP
|
|
start is the 0 based chunk number from the beginning of the file to
|
|
fetch inclusive.
|
|
end is 0 based chunk number from the beginning of the file to fetch
|
|
exclusive.
|
|
Both values can be negative, in which case they count from the back of
|
|
the file.
|
|
The value \[dq]\-5:\[dq] represents the last 5 chunks of a file.
|
|
.PP
|
|
Some valid examples are: \[dq]:5,\-5:\[dq] \-> the first and last five
|
|
chunks \[dq]0,\-2\[dq] \-> the first and the second last chunk
|
|
\[dq]0:10\[dq] \-> the first ten chunks
|
|
.PP
|
|
Any parameter with a key that starts with \[dq]file\[dq] can be used to
|
|
specify files to fetch, e.g.
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone rc cache/fetch chunks=0 file=hello file2=home/goodbye
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
File names will automatically be encrypted when the a crypt remote is
|
|
used on top of the cache.
|
|
.SS cache/stats: Get cache stats
|
|
.PP
|
|
Show statistics for the cache remote.
|
|
.SS config/create: create the config for a remote.
|
|
.PP
|
|
This takes the following parameters
|
|
.IP \[bu] 2
|
|
name \- name of remote
|
|
.IP \[bu] 2
|
|
parameters \- a map of { \[dq]key\[dq]: \[dq]value\[dq] } pairs
|
|
.IP \[bu] 2
|
|
type \- type of the new remote
|
|
.IP \[bu] 2
|
|
obscure \- optional bool \- forces obscuring of passwords
|
|
.IP \[bu] 2
|
|
noObscure \- optional bool \- forces passwords not to be obscured
|
|
.PP
|
|
See the config create
|
|
command (https://rclone.org/commands/rclone_config_create/) command for
|
|
more information on the above.
|
|
.PP
|
|
\f[B]Authentication is required for this call.\f[R]
|
|
.SS config/delete: Delete a remote in the config file.
|
|
.PP
|
|
Parameters:
|
|
.IP \[bu] 2
|
|
name \- name of remote to delete
|
|
.PP
|
|
See the config delete
|
|
command (https://rclone.org/commands/rclone_config_delete/) command for
|
|
more information on the above.
|
|
.PP
|
|
\f[B]Authentication is required for this call.\f[R]
|
|
.SS config/dump: Dumps the config file.
|
|
.PP
|
|
Returns a JSON object: \- key: value
|
|
.PP
|
|
Where keys are remote names and values are the config parameters.
|
|
.PP
|
|
See the config dump
|
|
command (https://rclone.org/commands/rclone_config_dump/) command for
|
|
more information on the above.
|
|
.PP
|
|
\f[B]Authentication is required for this call.\f[R]
|
|
.SS config/get: Get a remote in the config file.
|
|
.PP
|
|
Parameters:
|
|
.IP \[bu] 2
|
|
name \- name of remote to get
|
|
.PP
|
|
See the config dump
|
|
command (https://rclone.org/commands/rclone_config_dump/) command for
|
|
more information on the above.
|
|
.PP
|
|
\f[B]Authentication is required for this call.\f[R]
|
|
.SS config/listremotes: Lists the remotes in the config file.
|
|
.PP
|
|
Returns \- remotes \- array of remote names
|
|
.PP
|
|
See the listremotes
|
|
command (https://rclone.org/commands/rclone_listremotes/) command for
|
|
more information on the above.
|
|
.PP
|
|
\f[B]Authentication is required for this call.\f[R]
|
|
.SS config/password: password the config for a remote.
|
|
.PP
|
|
This takes the following parameters
|
|
.IP \[bu] 2
|
|
name \- name of remote
|
|
.IP \[bu] 2
|
|
parameters \- a map of { \[dq]key\[dq]: \[dq]value\[dq] } pairs
|
|
.PP
|
|
See the config password
|
|
command (https://rclone.org/commands/rclone_config_password/) command
|
|
for more information on the above.
|
|
.PP
|
|
\f[B]Authentication is required for this call.\f[R]
|
|
.SS config/providers: Shows how providers are configured in the config file.
|
|
.PP
|
|
Returns a JSON object: \- providers \- array of objects
|
|
.PP
|
|
See the config providers
|
|
command (https://rclone.org/commands/rclone_config_providers/) command
|
|
for more information on the above.
|
|
.PP
|
|
\f[B]Authentication is required for this call.\f[R]
|
|
.SS config/update: update the config for a remote.
|
|
.PP
|
|
This takes the following parameters
|
|
.IP \[bu] 2
|
|
name \- name of remote
|
|
.IP \[bu] 2
|
|
parameters \- a map of { \[dq]key\[dq]: \[dq]value\[dq] } pairs
|
|
.IP \[bu] 2
|
|
obscure \- optional bool \- forces obscuring of passwords
|
|
.IP \[bu] 2
|
|
noObscure \- optional bool \- forces passwords not to be obscured
|
|
.PP
|
|
See the config update
|
|
command (https://rclone.org/commands/rclone_config_update/) command for
|
|
more information on the above.
|
|
.PP
|
|
\f[B]Authentication is required for this call.\f[R]
|
|
.SS core/bwlimit: Set the bandwidth limit.
|
|
.PP
|
|
This sets the bandwidth limit to the string passed in.
|
|
This should be a single bandwidth limit entry or a pair of
|
|
upload:download bandwidth.
|
|
.PP
|
|
Eg
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone rc core/bwlimit rate=off
|
|
{
|
|
\[dq]bytesPerSecond\[dq]: \-1,
|
|
\[dq]bytesPerSecondTx\[dq]: \-1,
|
|
\[dq]bytesPerSecondRx\[dq]: \-1,
|
|
\[dq]rate\[dq]: \[dq]off\[dq]
|
|
}
|
|
rclone rc core/bwlimit rate=1M
|
|
{
|
|
\[dq]bytesPerSecond\[dq]: 1048576,
|
|
\[dq]bytesPerSecondTx\[dq]: 1048576,
|
|
\[dq]bytesPerSecondRx\[dq]: 1048576,
|
|
\[dq]rate\[dq]: \[dq]1M\[dq]
|
|
}
|
|
rclone rc core/bwlimit rate=1M:100k
|
|
{
|
|
\[dq]bytesPerSecond\[dq]: 1048576,
|
|
\[dq]bytesPerSecondTx\[dq]: 1048576,
|
|
\[dq]bytesPerSecondRx\[dq]: 131072,
|
|
\[dq]rate\[dq]: \[dq]1M\[dq]
|
|
}
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
If the rate parameter is not supplied then the bandwidth is queried
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone rc core/bwlimit
|
|
{
|
|
\[dq]bytesPerSecond\[dq]: 1048576,
|
|
\[dq]bytesPerSecondTx\[dq]: 1048576,
|
|
\[dq]bytesPerSecondRx\[dq]: 1048576,
|
|
\[dq]rate\[dq]: \[dq]1M\[dq]
|
|
}
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
The format of the parameter is exactly the same as passed to \-\-bwlimit
|
|
except only one bandwidth may be specified.
|
|
.PP
|
|
In either case \[dq]rate\[dq] is returned as a human readable string,
|
|
and \[dq]bytesPerSecond\[dq] is returned as a number.
|
|
.SS core/command: Run a rclone terminal command over rc.
|
|
.PP
|
|
This takes the following parameters
|
|
.IP \[bu] 2
|
|
command \- a string with the command name
|
|
.IP \[bu] 2
|
|
arg \- a list of arguments for the backend command
|
|
.IP \[bu] 2
|
|
opt \- a map of string to string of options
|
|
.IP \[bu] 2
|
|
returnType \- one of (\[dq]COMBINED_OUTPUT\[dq], \[dq]STREAM\[dq],
|
|
\[dq]STREAM_ONLY_STDOUT\[dq], \[dq]STREAM_ONLY_STDERR\[dq])
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
defaults to \[dq]COMBINED_OUTPUT\[dq] if not set
|
|
.IP \[bu] 2
|
|
the STREAM returnTypes will write the output to the body of the HTTP
|
|
message
|
|
.IP \[bu] 2
|
|
the COMBINED_OUTPUT will write the output to the \[dq]result\[dq]
|
|
parameter
|
|
.RE
|
|
.PP
|
|
Returns
|
|
.IP \[bu] 2
|
|
result \- result from the backend command
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
only set when using returnType \[dq]COMBINED_OUTPUT\[dq]
|
|
.RE
|
|
.IP \[bu] 2
|
|
error \- set if rclone exits with an error code
|
|
.IP \[bu] 2
|
|
returnType \- one of (\[dq]COMBINED_OUTPUT\[dq], \[dq]STREAM\[dq],
|
|
\[dq]STREAM_ONLY_STDOUT\[dq], \[dq]STREAM_ONLY_STDERR\[dq])
|
|
.PP
|
|
For example
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone rc core/command command=ls \-a mydrive:/ \-o max\-depth=1
|
|
rclone rc core/command \-a ls \-a mydrive:/ \-o max\-depth=1
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
Returns
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
{
|
|
\[dq]error\[dq]: false,
|
|
\[dq]result\[dq]: \[dq]<Raw command line output>\[dq]
|
|
}
|
|
|
|
OR
|
|
{
|
|
\[dq]error\[dq]: true,
|
|
\[dq]result\[dq]: \[dq]<Raw command line output>\[dq]
|
|
}
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
\f[B]Authentication is required for this call.\f[R]
|
|
.SS core/gc: Runs a garbage collection.
|
|
.PP
|
|
This tells the go runtime to do a garbage collection run.
|
|
It isn\[aq]t necessary to call this normally, but it can be useful for
|
|
debugging memory problems.
|
|
.SS core/group\-list: Returns list of stats.
|
|
.PP
|
|
This returns list of stats groups currently in memory.
|
|
.PP
|
|
Returns the following values:
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
{
|
|
\[dq]groups\[dq]: an array of group names:
|
|
[
|
|
\[dq]group1\[dq],
|
|
\[dq]group2\[dq],
|
|
...
|
|
]
|
|
}
|
|
\f[R]
|
|
.fi
|
|
.SS core/memstats: Returns the memory statistics
|
|
.PP
|
|
This returns the memory statistics of the running program.
|
|
What the values mean are explained in the go docs:
|
|
https://golang.org/pkg/runtime/#MemStats
|
|
.PP
|
|
The most interesting values for most people are:
|
|
.IP \[bu] 2
|
|
HeapAlloc: This is the amount of memory rclone is actually using
|
|
.IP \[bu] 2
|
|
HeapSys: This is the amount of memory rclone has obtained from the OS
|
|
.IP \[bu] 2
|
|
Sys: this is the total amount of memory requested from the OS
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
It is virtual memory so may include unused memory
|
|
.RE
|
|
.SS core/obscure: Obscures a string passed in.
|
|
.PP
|
|
Pass a clear string and rclone will obscure it for the config file: \-
|
|
clear \- string
|
|
.PP
|
|
Returns \- obscured \- string
|
|
.SS core/pid: Return PID of current process
|
|
.PP
|
|
This returns PID of current process.
|
|
Useful for stopping rclone process.
|
|
.SS core/quit: Terminates the app.
|
|
.PP
|
|
(optional) Pass an exit code to be used for terminating the app: \-
|
|
exitCode \- int
|
|
.SS core/stats: Returns stats about current transfers.
|
|
.PP
|
|
This returns all available stats:
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone rc core/stats
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
If group is not provided then summed up stats for all groups will be
|
|
returned.
|
|
.PP
|
|
Parameters
|
|
.IP \[bu] 2
|
|
group \- name of the stats group (string)
|
|
.PP
|
|
Returns the following values:
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
{
|
|
\[dq]bytes\[dq]: total transferred bytes since the start of the group,
|
|
\[dq]checks\[dq]: number of files checked,
|
|
\[dq]deletes\[dq] : number of files deleted,
|
|
\[dq]elapsedTime\[dq]: time in floating point seconds since rclone was started,
|
|
\[dq]errors\[dq]: number of errors,
|
|
\[dq]eta\[dq]: estimated time in seconds until the group completes,
|
|
\[dq]fatalError\[dq]: boolean whether there has been at least one fatal error,
|
|
\[dq]lastError\[dq]: last error string,
|
|
\[dq]renames\[dq] : number of files renamed,
|
|
\[dq]retryError\[dq]: boolean showing whether there has been at least one non\-NoRetryError,
|
|
\[dq]speed\[dq]: average speed in bytes/sec since start of the group,
|
|
\[dq]totalBytes\[dq]: total number of bytes in the group,
|
|
\[dq]totalChecks\[dq]: total number of checks in the group,
|
|
\[dq]totalTransfers\[dq]: total number of transfers in the group,
|
|
\[dq]transferTime\[dq] : total time spent on running jobs,
|
|
\[dq]transfers\[dq]: number of transferred files,
|
|
\[dq]transferring\[dq]: an array of currently active file transfers:
|
|
[
|
|
{
|
|
\[dq]bytes\[dq]: total transferred bytes for this file,
|
|
\[dq]eta\[dq]: estimated time in seconds until file transfer completion
|
|
\[dq]name\[dq]: name of the file,
|
|
\[dq]percentage\[dq]: progress of the file transfer in percent,
|
|
\[dq]speed\[dq]: average speed over the whole transfer in bytes/sec,
|
|
\[dq]speedAvg\[dq]: current speed in bytes/sec as an exponentially weighted moving average,
|
|
\[dq]size\[dq]: size of the file in bytes
|
|
}
|
|
],
|
|
\[dq]checking\[dq]: an array of names of currently active file checks
|
|
[]
|
|
}
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
Values for \[dq]transferring\[dq], \[dq]checking\[dq] and
|
|
\[dq]lastError\[dq] are only assigned if data is available.
|
|
The value for \[dq]eta\[dq] is null if an eta cannot be determined.
|
|
.SS core/stats\-delete: Delete stats group.
|
|
.PP
|
|
This deletes entire stats group
|
|
.PP
|
|
Parameters
|
|
.IP \[bu] 2
|
|
group \- name of the stats group (string)
|
|
.SS core/stats\-reset: Reset stats.
|
|
.PP
|
|
This clears counters, errors and finished transfers for all stats or
|
|
specific stats group if group is provided.
|
|
.PP
|
|
Parameters
|
|
.IP \[bu] 2
|
|
group \- name of the stats group (string)
|
|
.SS core/transferred: Returns stats about completed transfers.
|
|
.PP
|
|
This returns stats about completed transfers:
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone rc core/transferred
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
If group is not provided then completed transfers for all groups will be
|
|
returned.
|
|
.PP
|
|
Note only the last 100 completed transfers are returned.
|
|
.PP
|
|
Parameters
|
|
.IP \[bu] 2
|
|
group \- name of the stats group (string)
|
|
.PP
|
|
Returns the following values:
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
{
|
|
\[dq]transferred\[dq]: an array of completed transfers (including failed ones):
|
|
[
|
|
{
|
|
\[dq]name\[dq]: name of the file,
|
|
\[dq]size\[dq]: size of the file in bytes,
|
|
\[dq]bytes\[dq]: total transferred bytes for this file,
|
|
\[dq]checked\[dq]: if the transfer is only checked (skipped, deleted),
|
|
\[dq]timestamp\[dq]: integer representing millisecond unix epoch,
|
|
\[dq]error\[dq]: string description of the error (empty if successful),
|
|
\[dq]jobid\[dq]: id of the job that this transfer belongs to
|
|
}
|
|
]
|
|
}
|
|
\f[R]
|
|
.fi
|
|
.SS core/version: Shows the current version of rclone and the go runtime.
|
|
.PP
|
|
This shows the current version of go and the go runtime
|
|
.IP \[bu] 2
|
|
version \- rclone version, e.g.
|
|
\[dq]v1.53.0\[dq]
|
|
.IP \[bu] 2
|
|
decomposed \- version number as [major, minor, patch]
|
|
.IP \[bu] 2
|
|
isGit \- boolean \- true if this was compiled from the git version
|
|
.IP \[bu] 2
|
|
isBeta \- boolean \- true if this is a beta version
|
|
.IP \[bu] 2
|
|
os \- OS in use as according to Go
|
|
.IP \[bu] 2
|
|
arch \- cpu architecture in use according to Go
|
|
.IP \[bu] 2
|
|
goVersion \- version of Go runtime in use
|
|
.IP \[bu] 2
|
|
linking \- type of rclone executable (static or dynamic)
|
|
.IP \[bu] 2
|
|
goTags \- space separated build tags or \[dq]none\[dq]
|
|
.SS debug/set\-block\-profile\-rate: Set runtime.SetBlockProfileRate for blocking profiling.
|
|
.PP
|
|
SetBlockProfileRate controls the fraction of goroutine blocking events
|
|
that are reported in the blocking profile.
|
|
The profiler aims to sample an average of one blocking event per rate
|
|
nanoseconds spent blocked.
|
|
.PP
|
|
To include every blocking event in the profile, pass rate = 1.
|
|
To turn off profiling entirely, pass rate <= 0.
|
|
.PP
|
|
After calling this you can use this to see the blocking profile:
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
go tool pprof http://localhost:5572/debug/pprof/block
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
Parameters
|
|
.IP \[bu] 2
|
|
rate \- int
|
|
.SS debug/set\-mutex\-profile\-fraction: Set runtime.SetMutexProfileFraction for mutex profiling.
|
|
.PP
|
|
SetMutexProfileFraction controls the fraction of mutex contention events
|
|
that are reported in the mutex profile.
|
|
On average 1/rate events are reported.
|
|
The previous rate is returned.
|
|
.PP
|
|
To turn off profiling entirely, pass rate 0.
|
|
To just read the current rate, pass rate < 0.
|
|
(For n>1 the details of sampling may change.)
|
|
.PP
|
|
Once this is set you can look use this to profile the mutex contention:
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
go tool pprof http://localhost:5572/debug/pprof/mutex
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
Parameters
|
|
.IP \[bu] 2
|
|
rate \- int
|
|
.PP
|
|
Results
|
|
.IP \[bu] 2
|
|
previousRate \- int
|
|
.SS fscache/clear: Clear the Fs cache.
|
|
.PP
|
|
This clears the fs cache.
|
|
This is where remotes created from backends are cached for a short while
|
|
to make repeated rc calls more efficient.
|
|
.PP
|
|
If you change the parameters of a backend then you may want to call this
|
|
to clear an existing remote out of the cache before re\-creating it.
|
|
.PP
|
|
\f[B]Authentication is required for this call.\f[R]
|
|
.SS fscache/entries: Returns the number of entries in the fs cache.
|
|
.PP
|
|
This returns the number of entries in the fs cache.
|
|
.PP
|
|
Returns \- entries \- number of items in the cache
|
|
.PP
|
|
\f[B]Authentication is required for this call.\f[R]
|
|
.SS job/list: Lists the IDs of the running jobs
|
|
.PP
|
|
Parameters \- None
|
|
.PP
|
|
Results
|
|
.IP \[bu] 2
|
|
jobids \- array of integer job ids
|
|
.SS job/status: Reads the status of the job ID
|
|
.PP
|
|
Parameters
|
|
.IP \[bu] 2
|
|
jobid \- id of the job (integer)
|
|
.PP
|
|
Results
|
|
.IP \[bu] 2
|
|
finished \- boolean
|
|
.IP \[bu] 2
|
|
duration \- time in seconds that the job ran for
|
|
.IP \[bu] 2
|
|
endTime \- time the job finished (e.g.
|
|
\[dq]2018\-10\-26T18:50:20.528746884+01:00\[dq])
|
|
.IP \[bu] 2
|
|
error \- error from the job or empty string for no error
|
|
.IP \[bu] 2
|
|
finished \- boolean whether the job has finished or not
|
|
.IP \[bu] 2
|
|
id \- as passed in above
|
|
.IP \[bu] 2
|
|
startTime \- time the job started (e.g.
|
|
\[dq]2018\-10\-26T18:50:20.528336039+01:00\[dq])
|
|
.IP \[bu] 2
|
|
success \- boolean \- true for success false otherwise
|
|
.IP \[bu] 2
|
|
output \- output of the job as would have been returned if called
|
|
synchronously
|
|
.IP \[bu] 2
|
|
progress \- output of the progress related to the underlying job
|
|
.SS job/stop: Stop the running job
|
|
.PP
|
|
Parameters
|
|
.IP \[bu] 2
|
|
jobid \- id of the job (integer)
|
|
.SS mount/listmounts: Show current mount points
|
|
.PP
|
|
This shows currently mounted points, which can be used for performing an
|
|
unmount
|
|
.PP
|
|
This takes no parameters and returns
|
|
.IP \[bu] 2
|
|
mountPoints: list of current mount points
|
|
.PP
|
|
Eg
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone rc mount/listmounts
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
\f[B]Authentication is required for this call.\f[R]
|
|
.SS mount/mount: Create a new mount point
|
|
.PP
|
|
rclone allows Linux, FreeBSD, macOS and Windows to mount any of
|
|
Rclone\[aq]s cloud storage systems as a file system with FUSE.
|
|
.PP
|
|
If no mountType is provided, the priority is given as follows: 1.
|
|
mount 2.cmount 3.mount2
|
|
.PP
|
|
This takes the following parameters
|
|
.IP \[bu] 2
|
|
fs \- a remote path to be mounted (required)
|
|
.IP \[bu] 2
|
|
mountPoint: valid path on the local machine (required)
|
|
.IP \[bu] 2
|
|
mountType: One of the values (mount, cmount, mount2) specifies the mount
|
|
implementation to use
|
|
.IP \[bu] 2
|
|
mountOpt: a JSON object with Mount options in.
|
|
.IP \[bu] 2
|
|
vfsOpt: a JSON object with VFS options in.
|
|
.PP
|
|
Eg
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone rc mount/mount fs=mydrive: mountPoint=/home/<user>/mountPoint
|
|
rclone rc mount/mount fs=mydrive: mountPoint=/home/<user>/mountPoint mountType=mount
|
|
rclone rc mount/mount fs=TestDrive: mountPoint=/mnt/tmp vfsOpt=\[aq]{\[dq]CacheMode\[dq]: 2}\[aq] mountOpt=\[aq]{\[dq]AllowOther\[dq]: true}\[aq]
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
The vfsOpt are as described in options/get and can be seen in the the
|
|
\[dq]vfs\[dq] section when running and the mountOpt can be seen in the
|
|
\[dq]mount\[dq] section.
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone rc options/get
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
\f[B]Authentication is required for this call.\f[R]
|
|
.SS mount/types: Show all possible mount types
|
|
.PP
|
|
This shows all possible mount types and returns them as a list.
|
|
.PP
|
|
This takes no parameters and returns
|
|
.IP \[bu] 2
|
|
mountTypes: list of mount types
|
|
.PP
|
|
The mount types are strings like \[dq]mount\[dq], \[dq]mount2\[dq],
|
|
\[dq]cmount\[dq] and can be passed to mount/mount as the mountType
|
|
parameter.
|
|
.PP
|
|
Eg
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone rc mount/types
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
\f[B]Authentication is required for this call.\f[R]
|
|
.SS mount/unmount: Unmount selected active mount
|
|
.PP
|
|
rclone allows Linux, FreeBSD, macOS and Windows to mount any of
|
|
Rclone\[aq]s cloud storage systems as a file system with FUSE.
|
|
.PP
|
|
This takes the following parameters
|
|
.IP \[bu] 2
|
|
mountPoint: valid path on the local machine where the mount was created
|
|
(required)
|
|
.PP
|
|
Eg
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone rc mount/unmount mountPoint=/home/<user>/mountPoint
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
\f[B]Authentication is required for this call.\f[R]
|
|
.SS mount/unmountall: Show current mount points
|
|
.PP
|
|
This shows currently mounted points, which can be used for performing an
|
|
unmount
|
|
.PP
|
|
This takes no parameters and returns error if unmount does not succeed.
|
|
.PP
|
|
Eg
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone rc mount/unmountall
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
\f[B]Authentication is required for this call.\f[R]
|
|
.SS operations/about: Return the space used on the remote
|
|
.PP
|
|
This takes the following parameters
|
|
.IP \[bu] 2
|
|
fs \- a remote name string e.g.
|
|
\[dq]drive:\[dq]
|
|
.PP
|
|
The result is as returned from rclone about \-\-json
|
|
.PP
|
|
See the about command (https://rclone.org/commands/rclone_size/) command
|
|
for more information on the above.
|
|
.PP
|
|
\f[B]Authentication is required for this call.\f[R]
|
|
.SS operations/cleanup: Remove trashed files in the remote or path
|
|
.PP
|
|
This takes the following parameters
|
|
.IP \[bu] 2
|
|
fs \- a remote name string e.g.
|
|
\[dq]drive:\[dq]
|
|
.PP
|
|
See the cleanup command (https://rclone.org/commands/rclone_cleanup/)
|
|
command for more information on the above.
|
|
.PP
|
|
\f[B]Authentication is required for this call.\f[R]
|
|
.SS operations/copyfile: Copy a file from source remote to destination remote
|
|
.PP
|
|
This takes the following parameters
|
|
.IP \[bu] 2
|
|
srcFs \- a remote name string e.g.
|
|
\[dq]drive:\[dq] for the source
|
|
.IP \[bu] 2
|
|
srcRemote \- a path within that remote e.g.
|
|
\[dq]file.txt\[dq] for the source
|
|
.IP \[bu] 2
|
|
dstFs \- a remote name string e.g.
|
|
\[dq]drive2:\[dq] for the destination
|
|
.IP \[bu] 2
|
|
dstRemote \- a path within that remote e.g.
|
|
\[dq]file2.txt\[dq] for the destination
|
|
.PP
|
|
\f[B]Authentication is required for this call.\f[R]
|
|
.SS operations/copyurl: Copy the URL to the object
|
|
.PP
|
|
This takes the following parameters
|
|
.IP \[bu] 2
|
|
fs \- a remote name string e.g.
|
|
\[dq]drive:\[dq]
|
|
.IP \[bu] 2
|
|
remote \- a path within that remote e.g.
|
|
\[dq]dir\[dq]
|
|
.IP \[bu] 2
|
|
url \- string, URL to read from
|
|
.IP \[bu] 2
|
|
autoFilename \- boolean, set to true to retrieve destination file name
|
|
from url See the copyurl
|
|
command (https://rclone.org/commands/rclone_copyurl/) command for more
|
|
information on the above.
|
|
.PP
|
|
\f[B]Authentication is required for this call.\f[R]
|
|
.SS operations/delete: Remove files in the path
|
|
.PP
|
|
This takes the following parameters
|
|
.IP \[bu] 2
|
|
fs \- a remote name string e.g.
|
|
\[dq]drive:\[dq]
|
|
.PP
|
|
See the delete command (https://rclone.org/commands/rclone_delete/)
|
|
command for more information on the above.
|
|
.PP
|
|
\f[B]Authentication is required for this call.\f[R]
|
|
.SS operations/deletefile: Remove the single file pointed to
|
|
.PP
|
|
This takes the following parameters
|
|
.IP \[bu] 2
|
|
fs \- a remote name string e.g.
|
|
\[dq]drive:\[dq]
|
|
.IP \[bu] 2
|
|
remote \- a path within that remote e.g.
|
|
\[dq]dir\[dq]
|
|
.PP
|
|
See the deletefile
|
|
command (https://rclone.org/commands/rclone_deletefile/) command for
|
|
more information on the above.
|
|
.PP
|
|
\f[B]Authentication is required for this call.\f[R]
|
|
.SS operations/fsinfo: Return information about the remote
|
|
.PP
|
|
This takes the following parameters
|
|
.IP \[bu] 2
|
|
fs \- a remote name string e.g.
|
|
\[dq]drive:\[dq]
|
|
.PP
|
|
This returns info about the remote passed in;
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
{
|
|
// optional features and whether they are available or not
|
|
\[dq]Features\[dq]: {
|
|
\[dq]About\[dq]: true,
|
|
\[dq]BucketBased\[dq]: false,
|
|
\[dq]CanHaveEmptyDirectories\[dq]: true,
|
|
\[dq]CaseInsensitive\[dq]: false,
|
|
\[dq]ChangeNotify\[dq]: false,
|
|
\[dq]CleanUp\[dq]: false,
|
|
\[dq]Copy\[dq]: false,
|
|
\[dq]DirCacheFlush\[dq]: false,
|
|
\[dq]DirMove\[dq]: true,
|
|
\[dq]DuplicateFiles\[dq]: false,
|
|
\[dq]GetTier\[dq]: false,
|
|
\[dq]ListR\[dq]: false,
|
|
\[dq]MergeDirs\[dq]: false,
|
|
\[dq]Move\[dq]: true,
|
|
\[dq]OpenWriterAt\[dq]: true,
|
|
\[dq]PublicLink\[dq]: false,
|
|
\[dq]Purge\[dq]: true,
|
|
\[dq]PutStream\[dq]: true,
|
|
\[dq]PutUnchecked\[dq]: false,
|
|
\[dq]ReadMimeType\[dq]: false,
|
|
\[dq]ServerSideAcrossConfigs\[dq]: false,
|
|
\[dq]SetTier\[dq]: false,
|
|
\[dq]SetWrapper\[dq]: false,
|
|
\[dq]UnWrap\[dq]: false,
|
|
\[dq]WrapFs\[dq]: false,
|
|
\[dq]WriteMimeType\[dq]: false
|
|
},
|
|
// Names of hashes available
|
|
\[dq]Hashes\[dq]: [
|
|
\[dq]MD5\[dq],
|
|
\[dq]SHA\-1\[dq],
|
|
\[dq]DropboxHash\[dq],
|
|
\[dq]QuickXorHash\[dq]
|
|
],
|
|
\[dq]Name\[dq]: \[dq]local\[dq], // Name as created
|
|
\[dq]Precision\[dq]: 1, // Precision of timestamps in ns
|
|
\[dq]Root\[dq]: \[dq]/\[dq], // Path as created
|
|
\[dq]String\[dq]: \[dq]Local file system at /\[dq] // how the remote will appear in logs
|
|
}
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
This command does not have a command line equivalent so use this
|
|
instead:
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone rc \-\-loopback operations/fsinfo fs=remote:
|
|
\f[R]
|
|
.fi
|
|
.SS operations/list: List the given remote and path in JSON format
|
|
.PP
|
|
This takes the following parameters
|
|
.IP \[bu] 2
|
|
fs \- a remote name string e.g.
|
|
\[dq]drive:\[dq]
|
|
.IP \[bu] 2
|
|
remote \- a path within that remote e.g.
|
|
\[dq]dir\[dq]
|
|
.IP \[bu] 2
|
|
opt \- a dictionary of options to control the listing (optional)
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
recurse \- If set recurse directories
|
|
.IP \[bu] 2
|
|
noModTime \- If set return modification time
|
|
.IP \[bu] 2
|
|
showEncrypted \- If set show decrypted names
|
|
.IP \[bu] 2
|
|
showOrigIDs \- If set show the IDs for each item if known
|
|
.IP \[bu] 2
|
|
showHash \- If set return a dictionary of hashes
|
|
.RE
|
|
.PP
|
|
The result is
|
|
.IP \[bu] 2
|
|
list
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
This is an array of objects as described in the lsjson command
|
|
.RE
|
|
.PP
|
|
See the lsjson command (https://rclone.org/commands/rclone_lsjson/) for
|
|
more information on the above and examples.
|
|
.PP
|
|
\f[B]Authentication is required for this call.\f[R]
|
|
.SS operations/mkdir: Make a destination directory or container
|
|
.PP
|
|
This takes the following parameters
|
|
.IP \[bu] 2
|
|
fs \- a remote name string e.g.
|
|
\[dq]drive:\[dq]
|
|
.IP \[bu] 2
|
|
remote \- a path within that remote e.g.
|
|
\[dq]dir\[dq]
|
|
.PP
|
|
See the mkdir command (https://rclone.org/commands/rclone_mkdir/)
|
|
command for more information on the above.
|
|
.PP
|
|
\f[B]Authentication is required for this call.\f[R]
|
|
.SS operations/movefile: Move a file from source remote to destination remote
|
|
.PP
|
|
This takes the following parameters
|
|
.IP \[bu] 2
|
|
srcFs \- a remote name string e.g.
|
|
\[dq]drive:\[dq] for the source
|
|
.IP \[bu] 2
|
|
srcRemote \- a path within that remote e.g.
|
|
\[dq]file.txt\[dq] for the source
|
|
.IP \[bu] 2
|
|
dstFs \- a remote name string e.g.
|
|
\[dq]drive2:\[dq] for the destination
|
|
.IP \[bu] 2
|
|
dstRemote \- a path within that remote e.g.
|
|
\[dq]file2.txt\[dq] for the destination
|
|
.PP
|
|
\f[B]Authentication is required for this call.\f[R]
|
|
.SS operations/publiclink: Create or retrieve a public link to the given file or folder.
|
|
.PP
|
|
This takes the following parameters
|
|
.IP \[bu] 2
|
|
fs \- a remote name string e.g.
|
|
\[dq]drive:\[dq]
|
|
.IP \[bu] 2
|
|
remote \- a path within that remote e.g.
|
|
\[dq]dir\[dq]
|
|
.IP \[bu] 2
|
|
unlink \- boolean \- if set removes the link rather than adding it
|
|
(optional)
|
|
.IP \[bu] 2
|
|
expire \- string \- the expiry time of the link e.g.
|
|
\[dq]1d\[dq] (optional)
|
|
.PP
|
|
Returns
|
|
.IP \[bu] 2
|
|
url \- URL of the resource
|
|
.PP
|
|
See the link command (https://rclone.org/commands/rclone_link/) command
|
|
for more information on the above.
|
|
.PP
|
|
\f[B]Authentication is required for this call.\f[R]
|
|
.SS operations/purge: Remove a directory or container and all of its contents
|
|
.PP
|
|
This takes the following parameters
|
|
.IP \[bu] 2
|
|
fs \- a remote name string e.g.
|
|
\[dq]drive:\[dq]
|
|
.IP \[bu] 2
|
|
remote \- a path within that remote e.g.
|
|
\[dq]dir\[dq]
|
|
.PP
|
|
See the purge command (https://rclone.org/commands/rclone_purge/)
|
|
command for more information on the above.
|
|
.PP
|
|
\f[B]Authentication is required for this call.\f[R]
|
|
.SS operations/rmdir: Remove an empty directory or container
|
|
.PP
|
|
This takes the following parameters
|
|
.IP \[bu] 2
|
|
fs \- a remote name string e.g.
|
|
\[dq]drive:\[dq]
|
|
.IP \[bu] 2
|
|
remote \- a path within that remote e.g.
|
|
\[dq]dir\[dq]
|
|
.PP
|
|
See the rmdir command (https://rclone.org/commands/rclone_rmdir/)
|
|
command for more information on the above.
|
|
.PP
|
|
\f[B]Authentication is required for this call.\f[R]
|
|
.SS operations/rmdirs: Remove all the empty directories in the path
|
|
.PP
|
|
This takes the following parameters
|
|
.IP \[bu] 2
|
|
fs \- a remote name string e.g.
|
|
\[dq]drive:\[dq]
|
|
.IP \[bu] 2
|
|
remote \- a path within that remote e.g.
|
|
\[dq]dir\[dq]
|
|
.IP \[bu] 2
|
|
leaveRoot \- boolean, set to true not to delete the root
|
|
.PP
|
|
See the rmdirs command (https://rclone.org/commands/rclone_rmdirs/)
|
|
command for more information on the above.
|
|
.PP
|
|
\f[B]Authentication is required for this call.\f[R]
|
|
.SS operations/size: Count the number of bytes and files in remote
|
|
.PP
|
|
This takes the following parameters
|
|
.IP \[bu] 2
|
|
fs \- a remote name string e.g.
|
|
\[dq]drive:path/to/dir\[dq]
|
|
.PP
|
|
Returns
|
|
.IP \[bu] 2
|
|
count \- number of files
|
|
.IP \[bu] 2
|
|
bytes \- number of bytes in those files
|
|
.PP
|
|
See the size command (https://rclone.org/commands/rclone_size/) command
|
|
for more information on the above.
|
|
.PP
|
|
\f[B]Authentication is required for this call.\f[R]
|
|
.SS operations/uploadfile: Upload file using multiform/form\-data
|
|
.PP
|
|
This takes the following parameters
|
|
.IP \[bu] 2
|
|
fs \- a remote name string e.g.
|
|
\[dq]drive:\[dq]
|
|
.IP \[bu] 2
|
|
remote \- a path within that remote e.g.
|
|
\[dq]dir\[dq]
|
|
.IP \[bu] 2
|
|
each part in body represents a file to be uploaded See the uploadfile
|
|
command (https://rclone.org/commands/rclone_uploadfile/) command for
|
|
more information on the above.
|
|
.PP
|
|
\f[B]Authentication is required for this call.\f[R]
|
|
.SS options/blocks: List all the option blocks
|
|
.PP
|
|
Returns \- options \- a list of the options block names
|
|
.SS options/get: Get all the global options
|
|
.PP
|
|
Returns an object where keys are option block names and values are an
|
|
object with the current option values in.
|
|
.PP
|
|
Note that these are the global options which are unaffected by use of
|
|
the _config and _filter parameters.
|
|
If you wish to read the parameters set in _config then use
|
|
options/config and for _filter use options/filter.
|
|
.PP
|
|
This shows the internal names of the option within rclone which should
|
|
map to the external options very easily with a few exceptions.
|
|
.SS options/local: Get the currently active config for this call
|
|
.PP
|
|
Returns an object with the keys \[dq]config\[dq] and \[dq]filter\[dq].
|
|
The \[dq]config\[dq] key contains the local config and the
|
|
\[dq]filter\[dq] key contains the local filters.
|
|
.PP
|
|
Note that these are the local options specific to this rc call.
|
|
If _config was not supplied then they will be the global options.
|
|
Likewise with \[dq]_filter\[dq].
|
|
.PP
|
|
This call is mostly useful for seeing if _config and _filter passing is
|
|
working.
|
|
.PP
|
|
This shows the internal names of the option within rclone which should
|
|
map to the external options very easily with a few exceptions.
|
|
.SS options/set: Set an option
|
|
.PP
|
|
Parameters
|
|
.IP \[bu] 2
|
|
option block name containing an object with
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
key: value
|
|
.RE
|
|
.PP
|
|
Repeated as often as required.
|
|
.PP
|
|
Only supply the options you wish to change.
|
|
If an option is unknown it will be silently ignored.
|
|
Not all options will have an effect when changed like this.
|
|
.PP
|
|
For example:
|
|
.PP
|
|
This sets DEBUG level logs (\-vv) (these can be set by number or string)
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone rc options/set \-\-json \[aq]{\[dq]main\[dq]: {\[dq]LogLevel\[dq]: \[dq]DEBUG\[dq]}}\[aq]
|
|
rclone rc options/set \-\-json \[aq]{\[dq]main\[dq]: {\[dq]LogLevel\[dq]: 8}}\[aq]
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
And this sets INFO level logs (\-v)
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone rc options/set \-\-json \[aq]{\[dq]main\[dq]: {\[dq]LogLevel\[dq]: \[dq]INFO\[dq]}}\[aq]
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
And this sets NOTICE level logs (normal without \-v)
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone rc options/set \-\-json \[aq]{\[dq]main\[dq]: {\[dq]LogLevel\[dq]: \[dq]NOTICE\[dq]}}\[aq]
|
|
\f[R]
|
|
.fi
|
|
.SS pluginsctl/addPlugin: Add a plugin using url
|
|
.PP
|
|
used for adding a plugin to the webgui
|
|
.PP
|
|
This takes the following parameters
|
|
.IP \[bu] 2
|
|
url: http url of the github repo where the plugin is hosted
|
|
(http://github.com/rclone/rclone\-webui\-react)
|
|
.PP
|
|
Eg
|
|
.PP
|
|
rclone rc pluginsctl/addPlugin
|
|
.PP
|
|
\f[B]Authentication is required for this call.\f[R]
|
|
.SS pluginsctl/getPluginsForType: Get plugins with type criteria
|
|
.PP
|
|
This shows all possible plugins by a mime type
|
|
.PP
|
|
This takes the following parameters
|
|
.IP \[bu] 2
|
|
type: supported mime type by a loaded plugin e.g.
|
|
(video/mp4, audio/mp3)
|
|
.IP \[bu] 2
|
|
pluginType: filter plugins based on their type e.g.
|
|
(DASHBOARD, FILE_HANDLER, TERMINAL)
|
|
.PP
|
|
and returns
|
|
.IP \[bu] 2
|
|
loadedPlugins: list of current production plugins
|
|
.IP \[bu] 2
|
|
testPlugins: list of temporarily loaded development plugins, usually
|
|
running on a different server.
|
|
.PP
|
|
Eg
|
|
.PP
|
|
rclone rc pluginsctl/getPluginsForType type=video/mp4
|
|
.PP
|
|
\f[B]Authentication is required for this call.\f[R]
|
|
.SS pluginsctl/listPlugins: Get the list of currently loaded plugins
|
|
.PP
|
|
This allows you to get the currently enabled plugins and their details.
|
|
.PP
|
|
This takes no parameters and returns
|
|
.IP \[bu] 2
|
|
loadedPlugins: list of current production plugins
|
|
.IP \[bu] 2
|
|
testPlugins: list of temporarily loaded development plugins, usually
|
|
running on a different server.
|
|
.PP
|
|
Eg
|
|
.PP
|
|
rclone rc pluginsctl/listPlugins
|
|
.PP
|
|
\f[B]Authentication is required for this call.\f[R]
|
|
.SS pluginsctl/listTestPlugins: Show currently loaded test plugins
|
|
.PP
|
|
allows listing of test plugins with the rclone.test set to true in
|
|
package.json of the plugin
|
|
.PP
|
|
This takes no parameters and returns
|
|
.IP \[bu] 2
|
|
loadedTestPlugins: list of currently available test plugins
|
|
.PP
|
|
Eg
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone rc pluginsctl/listTestPlugins
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
\f[B]Authentication is required for this call.\f[R]
|
|
.SS pluginsctl/removePlugin: Remove a loaded plugin
|
|
.PP
|
|
This allows you to remove a plugin using it\[aq]s name
|
|
.PP
|
|
This takes parameters
|
|
.IP \[bu] 2
|
|
name: name of the plugin in the format
|
|
\f[C]author\f[R]/\f[C]plugin_name\f[R]
|
|
.PP
|
|
Eg
|
|
.PP
|
|
rclone rc pluginsctl/removePlugin name=rclone/video\-plugin
|
|
.PP
|
|
\f[B]Authentication is required for this call.\f[R]
|
|
.SS pluginsctl/removeTestPlugin: Remove a test plugin
|
|
.PP
|
|
This allows you to remove a plugin using it\[aq]s name
|
|
.PP
|
|
This takes the following parameters
|
|
.IP \[bu] 2
|
|
name: name of the plugin in the format
|
|
\f[C]author\f[R]/\f[C]plugin_name\f[R]
|
|
.PP
|
|
Eg
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone rc pluginsctl/removeTestPlugin name=rclone/rclone\-webui\-react
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
\f[B]Authentication is required for this call.\f[R]
|
|
.SS rc/error: This returns an error
|
|
.PP
|
|
This returns an error with the input as part of its error string.
|
|
Useful for testing error handling.
|
|
.SS rc/list: List all the registered remote control commands
|
|
.PP
|
|
This lists all the registered remote control commands as a JSON map in
|
|
the commands response.
|
|
.SS rc/noop: Echo the input to the output parameters
|
|
.PP
|
|
This echoes the input parameters to the output parameters for testing
|
|
purposes.
|
|
It can be used to check that rclone is still alive and to check that
|
|
parameter passing is working properly.
|
|
.SS rc/noopauth: Echo the input to the output parameters requiring auth
|
|
.PP
|
|
This echoes the input parameters to the output parameters for testing
|
|
purposes.
|
|
It can be used to check that rclone is still alive and to check that
|
|
parameter passing is working properly.
|
|
.PP
|
|
\f[B]Authentication is required for this call.\f[R]
|
|
.SS sync/copy: copy a directory from source remote to destination remote
|
|
.PP
|
|
This takes the following parameters
|
|
.IP \[bu] 2
|
|
srcFs \- a remote name string e.g.
|
|
\[dq]drive:src\[dq] for the source
|
|
.IP \[bu] 2
|
|
dstFs \- a remote name string e.g.
|
|
\[dq]drive:dst\[dq] for the destination
|
|
.IP \[bu] 2
|
|
createEmptySrcDirs \- create empty src directories on destination if set
|
|
.PP
|
|
See the copy command (https://rclone.org/commands/rclone_copy/) command
|
|
for more information on the above.
|
|
.PP
|
|
\f[B]Authentication is required for this call.\f[R]
|
|
.SS sync/move: move a directory from source remote to destination remote
|
|
.PP
|
|
This takes the following parameters
|
|
.IP \[bu] 2
|
|
srcFs \- a remote name string e.g.
|
|
\[dq]drive:src\[dq] for the source
|
|
.IP \[bu] 2
|
|
dstFs \- a remote name string e.g.
|
|
\[dq]drive:dst\[dq] for the destination
|
|
.IP \[bu] 2
|
|
createEmptySrcDirs \- create empty src directories on destination if set
|
|
.IP \[bu] 2
|
|
deleteEmptySrcDirs \- delete empty src directories if set
|
|
.PP
|
|
See the move command (https://rclone.org/commands/rclone_move/) command
|
|
for more information on the above.
|
|
.PP
|
|
\f[B]Authentication is required for this call.\f[R]
|
|
.SS sync/sync: sync a directory from source remote to destination remote
|
|
.PP
|
|
This takes the following parameters
|
|
.IP \[bu] 2
|
|
srcFs \- a remote name string e.g.
|
|
\[dq]drive:src\[dq] for the source
|
|
.IP \[bu] 2
|
|
dstFs \- a remote name string e.g.
|
|
\[dq]drive:dst\[dq] for the destination
|
|
.IP \[bu] 2
|
|
createEmptySrcDirs \- create empty src directories on destination if set
|
|
.PP
|
|
See the sync command (https://rclone.org/commands/rclone_sync/) command
|
|
for more information on the above.
|
|
.PP
|
|
\f[B]Authentication is required for this call.\f[R]
|
|
.SS vfs/forget: Forget files or directories in the directory cache.
|
|
.PP
|
|
This forgets the paths in the directory cache causing them to be
|
|
re\-read from the remote when needed.
|
|
.PP
|
|
If no paths are passed in then it will forget all the paths in the
|
|
directory cache.
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone rc vfs/forget
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
Otherwise pass files or dirs in as file=path or dir=path.
|
|
Any parameter key starting with file will forget that file and any
|
|
starting with dir will forget that dir, e.g.
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone rc vfs/forget file=hello file2=goodbye dir=home/junk
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
This command takes an \[dq]fs\[dq] parameter.
|
|
If this parameter is not supplied and if there is only one VFS in use
|
|
then that VFS will be used.
|
|
If there is more than one VFS in use then the \[dq]fs\[dq] parameter
|
|
must be supplied.
|
|
.SS vfs/list: List active VFSes.
|
|
.PP
|
|
This lists the active VFSes.
|
|
.PP
|
|
It returns a list under the key \[dq]vfses\[dq] where the values are the
|
|
VFS names that could be passed to the other VFS commands in the
|
|
\[dq]fs\[dq] parameter.
|
|
.SS vfs/poll\-interval: Get the status or update the value of the poll\-interval option.
|
|
.PP
|
|
Without any parameter given this returns the current status of the
|
|
poll\-interval setting.
|
|
.PP
|
|
When the interval=duration parameter is set, the poll\-interval value is
|
|
updated and the polling function is notified.
|
|
Setting interval=0 disables poll\-interval.
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone rc vfs/poll\-interval interval=5m
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
The timeout=duration parameter can be used to specify a time to wait for
|
|
the current poll function to apply the new value.
|
|
If timeout is less or equal 0, which is the default, wait indefinitely.
|
|
.PP
|
|
The new poll\-interval value will only be active when the timeout is not
|
|
reached.
|
|
.PP
|
|
If poll\-interval is updated or disabled temporarily, some changes might
|
|
not get picked up by the polling function, depending on the used remote.
|
|
.PP
|
|
This command takes an \[dq]fs\[dq] parameter.
|
|
If this parameter is not supplied and if there is only one VFS in use
|
|
then that VFS will be used.
|
|
If there is more than one VFS in use then the \[dq]fs\[dq] parameter
|
|
must be supplied.
|
|
.SS vfs/refresh: Refresh the directory cache.
|
|
.PP
|
|
This reads the directories for the specified paths and freshens the
|
|
directory cache.
|
|
.PP
|
|
If no paths are passed in then it will refresh the root directory.
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone rc vfs/refresh
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
Otherwise pass directories in as dir=path.
|
|
Any parameter key starting with dir will refresh that directory, e.g.
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone rc vfs/refresh dir=home/junk dir2=data/misc
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
If the parameter recursive=true is given the whole directory tree will
|
|
get refreshed.
|
|
This refresh will use \-\-fast\-list if enabled.
|
|
.PP
|
|
This command takes an \[dq]fs\[dq] parameter.
|
|
If this parameter is not supplied and if there is only one VFS in use
|
|
then that VFS will be used.
|
|
If there is more than one VFS in use then the \[dq]fs\[dq] parameter
|
|
must be supplied.
|
|
.SS Accessing the remote control via HTTP
|
|
.PP
|
|
Rclone implements a simple HTTP based protocol.
|
|
.PP
|
|
Each endpoint takes an JSON object and returns a JSON object or an
|
|
error.
|
|
The JSON objects are essentially a map of string names to values.
|
|
.PP
|
|
All calls must made using POST.
|
|
.PP
|
|
The input objects can be supplied using URL parameters, POST parameters
|
|
or by supplying \[dq]Content\-Type: application/json\[dq] and a JSON
|
|
blob in the body.
|
|
There are examples of these below using \f[C]curl\f[R].
|
|
.PP
|
|
The response will be a JSON blob in the body of the response.
|
|
This is formatted to be reasonably human readable.
|
|
.SS Error returns
|
|
.PP
|
|
If an error occurs then there will be an HTTP error status (e.g.
|
|
500) and the body of the response will contain a JSON encoded error
|
|
object, e.g.
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
{
|
|
\[dq]error\[dq]: \[dq]Expecting string value for key \[rs]\[dq]remote\[rs]\[dq] (was float64)\[dq],
|
|
\[dq]input\[dq]: {
|
|
\[dq]fs\[dq]: \[dq]/tmp\[dq],
|
|
\[dq]remote\[dq]: 3
|
|
},
|
|
\[dq]status\[dq]: 400
|
|
\[dq]path\[dq]: \[dq]operations/rmdir\[dq],
|
|
}
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
The keys in the error response are \- error \- error string \- input \-
|
|
the input parameters to the call \- status \- the HTTP status code \-
|
|
path \- the path of the call
|
|
.SS CORS
|
|
.PP
|
|
The sever implements basic CORS support and allows all origins for that.
|
|
The response to a preflight OPTIONS request will echo the requested
|
|
\[dq]Access\-Control\-Request\-Headers\[dq] back.
|
|
.SS Using POST with URL parameters only
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
curl \-X POST \[aq]http://localhost:5572/rc/noop?potato=1&sausage=2\[aq]
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
Response
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
{
|
|
\[dq]potato\[dq]: \[dq]1\[dq],
|
|
\[dq]sausage\[dq]: \[dq]2\[dq]
|
|
}
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
Here is what an error response looks like:
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
curl \-X POST \[aq]http://localhost:5572/rc/error?potato=1&sausage=2\[aq]
|
|
\f[R]
|
|
.fi
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
{
|
|
\[dq]error\[dq]: \[dq]arbitrary error on input map[potato:1 sausage:2]\[dq],
|
|
\[dq]input\[dq]: {
|
|
\[dq]potato\[dq]: \[dq]1\[dq],
|
|
\[dq]sausage\[dq]: \[dq]2\[dq]
|
|
}
|
|
}
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
Note that curl doesn\[aq]t return errors to the shell unless you use the
|
|
\f[C]\-f\f[R] option
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
$ curl \-f \-X POST \[aq]http://localhost:5572/rc/error?potato=1&sausage=2\[aq]
|
|
curl: (22) The requested URL returned error: 400 Bad Request
|
|
$ echo $?
|
|
22
|
|
\f[R]
|
|
.fi
|
|
.SS Using POST with a form
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
curl \-\-data \[dq]potato=1\[dq] \-\-data \[dq]sausage=2\[dq] http://localhost:5572/rc/noop
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
Response
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
{
|
|
\[dq]potato\[dq]: \[dq]1\[dq],
|
|
\[dq]sausage\[dq]: \[dq]2\[dq]
|
|
}
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
Note that you can combine these with URL parameters too with the POST
|
|
parameters taking precedence.
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
curl \-\-data \[dq]potato=1\[dq] \-\-data \[dq]sausage=2\[dq] \[dq]http://localhost:5572/rc/noop?rutabaga=3&sausage=4\[dq]
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
Response
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
{
|
|
\[dq]potato\[dq]: \[dq]1\[dq],
|
|
\[dq]rutabaga\[dq]: \[dq]3\[dq],
|
|
\[dq]sausage\[dq]: \[dq]4\[dq]
|
|
}
|
|
\f[R]
|
|
.fi
|
|
.SS Using POST with a JSON blob
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
curl \-H \[dq]Content\-Type: application/json\[dq] \-X POST \-d \[aq]{\[dq]potato\[dq]:2,\[dq]sausage\[dq]:1}\[aq] http://localhost:5572/rc/noop
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
response
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
{
|
|
\[dq]password\[dq]: \[dq]xyz\[dq],
|
|
\[dq]username\[dq]: \[dq]xyz\[dq]
|
|
}
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
This can be combined with URL parameters too if required.
|
|
The JSON blob takes precedence.
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
curl \-H \[dq]Content\-Type: application/json\[dq] \-X POST \-d \[aq]{\[dq]potato\[dq]:2,\[dq]sausage\[dq]:1}\[aq] \[aq]http://localhost:5572/rc/noop?rutabaga=3&potato=4\[aq]
|
|
\f[R]
|
|
.fi
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
{
|
|
\[dq]potato\[dq]: 2,
|
|
\[dq]rutabaga\[dq]: \[dq]3\[dq],
|
|
\[dq]sausage\[dq]: 1
|
|
}
|
|
\f[R]
|
|
.fi
|
|
.SS Debugging rclone with pprof
|
|
.PP
|
|
If you use the \f[C]\-\-rc\f[R] flag this will also enable the use of
|
|
the go profiling tools on the same port.
|
|
.PP
|
|
To use these, first install go (https://golang.org/doc/install).
|
|
.SS Debugging memory use
|
|
.PP
|
|
To profile rclone\[aq]s memory use you can run:
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
go tool pprof \-web http://localhost:5572/debug/pprof/heap
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
This should open a page in your browser showing what is using what
|
|
memory.
|
|
.PP
|
|
You can also use the \f[C]\-text\f[R] flag to produce a textual summary
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
$ go tool pprof \-text http://localhost:5572/debug/pprof/heap
|
|
Showing nodes accounting for 1537.03kB, 100% of 1537.03kB total
|
|
flat flat% sum% cum cum%
|
|
1024.03kB 66.62% 66.62% 1024.03kB 66.62% github.com/rclone/rclone/vendor/golang.org/x/net/http2/hpack.addDecoderNode
|
|
513kB 33.38% 100% 513kB 33.38% net/http.newBufioWriterSize
|
|
0 0% 100% 1024.03kB 66.62% github.com/rclone/rclone/cmd/all.init
|
|
0 0% 100% 1024.03kB 66.62% github.com/rclone/rclone/cmd/serve.init
|
|
0 0% 100% 1024.03kB 66.62% github.com/rclone/rclone/cmd/serve/restic.init
|
|
0 0% 100% 1024.03kB 66.62% github.com/rclone/rclone/vendor/golang.org/x/net/http2.init
|
|
0 0% 100% 1024.03kB 66.62% github.com/rclone/rclone/vendor/golang.org/x/net/http2/hpack.init
|
|
0 0% 100% 1024.03kB 66.62% github.com/rclone/rclone/vendor/golang.org/x/net/http2/hpack.init.0
|
|
0 0% 100% 1024.03kB 66.62% main.init
|
|
0 0% 100% 513kB 33.38% net/http.(*conn).readRequest
|
|
0 0% 100% 513kB 33.38% net/http.(*conn).serve
|
|
0 0% 100% 1024.03kB 66.62% runtime.main
|
|
\f[R]
|
|
.fi
|
|
.SS Debugging go routine leaks
|
|
.PP
|
|
Memory leaks are most often caused by go routine leaks keeping memory
|
|
alive which should have been garbage collected.
|
|
.PP
|
|
See all active go routines using
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
curl http://localhost:5572/debug/pprof/goroutine?debug=1
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
Or go to http://localhost:5572/debug/pprof/goroutine?debug=1 in your
|
|
browser.
|
|
.SS Other profiles to look at
|
|
.PP
|
|
You can see a summary of profiles available at
|
|
http://localhost:5572/debug/pprof/
|
|
.PP
|
|
Here is how to use some of them:
|
|
.IP \[bu] 2
|
|
Memory: \f[C]go tool pprof http://localhost:5572/debug/pprof/heap\f[R]
|
|
.IP \[bu] 2
|
|
Go routines:
|
|
\f[C]curl http://localhost:5572/debug/pprof/goroutine?debug=1\f[R]
|
|
.IP \[bu] 2
|
|
30\-second CPU profile:
|
|
\f[C]go tool pprof http://localhost:5572/debug/pprof/profile\f[R]
|
|
.IP \[bu] 2
|
|
5\-second execution trace:
|
|
\f[C]wget http://localhost:5572/debug/pprof/trace?seconds=5\f[R]
|
|
.IP \[bu] 2
|
|
Goroutine blocking profile
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Enable first with:
|
|
\f[C]rclone rc debug/set\-block\-profile\-rate rate=1\f[R] (docs)
|
|
.IP \[bu] 2
|
|
\f[C]go tool pprof http://localhost:5572/debug/pprof/block\f[R]
|
|
.RE
|
|
.IP \[bu] 2
|
|
Contended mutexes:
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Enable first with:
|
|
\f[C]rclone rc debug/set\-mutex\-profile\-fraction rate=1\f[R] (docs)
|
|
.IP \[bu] 2
|
|
\f[C]go tool pprof http://localhost:5572/debug/pprof/mutex\f[R]
|
|
.RE
|
|
.PP
|
|
See the net/http/pprof docs (https://golang.org/pkg/net/http/pprof/) for
|
|
more info on how to use the profiling and for a general overview see the
|
|
Go team\[aq]s blog post on profiling go
|
|
programs (https://blog.golang.org/profiling-go-programs).
|
|
.PP
|
|
The profiling hook is zero overhead unless it is
|
|
used (https://stackoverflow.com/q/26545159/164234).
|
|
.SH Overview of cloud storage systems
|
|
.PP
|
|
Each cloud storage system is slightly different.
|
|
Rclone attempts to provide a unified interface to them, but some
|
|
underlying differences show through.
|
|
.SS Features
|
|
.PP
|
|
Here is an overview of the major features of each cloud storage system.
|
|
.PP
|
|
.TS
|
|
tab(@);
|
|
l c c c c c.
|
|
T{
|
|
Name
|
|
T}@T{
|
|
Hash
|
|
T}@T{
|
|
ModTime
|
|
T}@T{
|
|
Case Insensitive
|
|
T}@T{
|
|
Duplicate Files
|
|
T}@T{
|
|
MIME Type
|
|
T}
|
|
_
|
|
T{
|
|
1Fichier
|
|
T}@T{
|
|
Whirlpool
|
|
T}@T{
|
|
No
|
|
T}@T{
|
|
No
|
|
T}@T{
|
|
Yes
|
|
T}@T{
|
|
R
|
|
T}
|
|
T{
|
|
Amazon Drive
|
|
T}@T{
|
|
MD5
|
|
T}@T{
|
|
No
|
|
T}@T{
|
|
Yes
|
|
T}@T{
|
|
No
|
|
T}@T{
|
|
R
|
|
T}
|
|
T{
|
|
Amazon S3
|
|
T}@T{
|
|
MD5
|
|
T}@T{
|
|
Yes
|
|
T}@T{
|
|
No
|
|
T}@T{
|
|
No
|
|
T}@T{
|
|
R/W
|
|
T}
|
|
T{
|
|
Backblaze B2
|
|
T}@T{
|
|
SHA1
|
|
T}@T{
|
|
Yes
|
|
T}@T{
|
|
No
|
|
T}@T{
|
|
No
|
|
T}@T{
|
|
R/W
|
|
T}
|
|
T{
|
|
Box
|
|
T}@T{
|
|
SHA1
|
|
T}@T{
|
|
Yes
|
|
T}@T{
|
|
Yes
|
|
T}@T{
|
|
No
|
|
T}@T{
|
|
\-
|
|
T}
|
|
T{
|
|
Citrix ShareFile
|
|
T}@T{
|
|
MD5
|
|
T}@T{
|
|
Yes
|
|
T}@T{
|
|
Yes
|
|
T}@T{
|
|
No
|
|
T}@T{
|
|
\-
|
|
T}
|
|
T{
|
|
Dropbox
|
|
T}@T{
|
|
DBHASH \[S1]
|
|
T}@T{
|
|
Yes
|
|
T}@T{
|
|
Yes
|
|
T}@T{
|
|
No
|
|
T}@T{
|
|
\-
|
|
T}
|
|
T{
|
|
Enterprise File Fabric
|
|
T}@T{
|
|
\-
|
|
T}@T{
|
|
Yes
|
|
T}@T{
|
|
Yes
|
|
T}@T{
|
|
No
|
|
T}@T{
|
|
R/W
|
|
T}
|
|
T{
|
|
FTP
|
|
T}@T{
|
|
\-
|
|
T}@T{
|
|
No
|
|
T}@T{
|
|
No
|
|
T}@T{
|
|
No
|
|
T}@T{
|
|
\-
|
|
T}
|
|
T{
|
|
Google Cloud Storage
|
|
T}@T{
|
|
MD5
|
|
T}@T{
|
|
Yes
|
|
T}@T{
|
|
No
|
|
T}@T{
|
|
No
|
|
T}@T{
|
|
R/W
|
|
T}
|
|
T{
|
|
Google Drive
|
|
T}@T{
|
|
MD5
|
|
T}@T{
|
|
Yes
|
|
T}@T{
|
|
No
|
|
T}@T{
|
|
Yes
|
|
T}@T{
|
|
R/W
|
|
T}
|
|
T{
|
|
Google Photos
|
|
T}@T{
|
|
\-
|
|
T}@T{
|
|
No
|
|
T}@T{
|
|
No
|
|
T}@T{
|
|
Yes
|
|
T}@T{
|
|
R
|
|
T}
|
|
T{
|
|
HDFS
|
|
T}@T{
|
|
\-
|
|
T}@T{
|
|
Yes
|
|
T}@T{
|
|
No
|
|
T}@T{
|
|
No
|
|
T}@T{
|
|
\-
|
|
T}
|
|
T{
|
|
HTTP
|
|
T}@T{
|
|
\-
|
|
T}@T{
|
|
No
|
|
T}@T{
|
|
No
|
|
T}@T{
|
|
No
|
|
T}@T{
|
|
R
|
|
T}
|
|
T{
|
|
Hubic
|
|
T}@T{
|
|
MD5
|
|
T}@T{
|
|
Yes
|
|
T}@T{
|
|
No
|
|
T}@T{
|
|
No
|
|
T}@T{
|
|
R/W
|
|
T}
|
|
T{
|
|
Jottacloud
|
|
T}@T{
|
|
MD5
|
|
T}@T{
|
|
Yes
|
|
T}@T{
|
|
Yes
|
|
T}@T{
|
|
No
|
|
T}@T{
|
|
R
|
|
T}
|
|
T{
|
|
Koofr
|
|
T}@T{
|
|
MD5
|
|
T}@T{
|
|
No
|
|
T}@T{
|
|
Yes
|
|
T}@T{
|
|
No
|
|
T}@T{
|
|
\-
|
|
T}
|
|
T{
|
|
Mail.ru Cloud
|
|
T}@T{
|
|
Mailru \[u2076]
|
|
T}@T{
|
|
Yes
|
|
T}@T{
|
|
Yes
|
|
T}@T{
|
|
No
|
|
T}@T{
|
|
\-
|
|
T}
|
|
T{
|
|
Mega
|
|
T}@T{
|
|
\-
|
|
T}@T{
|
|
No
|
|
T}@T{
|
|
No
|
|
T}@T{
|
|
Yes
|
|
T}@T{
|
|
\-
|
|
T}
|
|
T{
|
|
Memory
|
|
T}@T{
|
|
MD5
|
|
T}@T{
|
|
Yes
|
|
T}@T{
|
|
No
|
|
T}@T{
|
|
No
|
|
T}@T{
|
|
\-
|
|
T}
|
|
T{
|
|
Microsoft Azure Blob Storage
|
|
T}@T{
|
|
MD5
|
|
T}@T{
|
|
Yes
|
|
T}@T{
|
|
No
|
|
T}@T{
|
|
No
|
|
T}@T{
|
|
R/W
|
|
T}
|
|
T{
|
|
Microsoft OneDrive
|
|
T}@T{
|
|
SHA1 \[u2075]
|
|
T}@T{
|
|
Yes
|
|
T}@T{
|
|
Yes
|
|
T}@T{
|
|
No
|
|
T}@T{
|
|
R
|
|
T}
|
|
T{
|
|
OpenDrive
|
|
T}@T{
|
|
MD5
|
|
T}@T{
|
|
Yes
|
|
T}@T{
|
|
Yes
|
|
T}@T{
|
|
Partial \[u2078]
|
|
T}@T{
|
|
\-
|
|
T}
|
|
T{
|
|
OpenStack Swift
|
|
T}@T{
|
|
MD5
|
|
T}@T{
|
|
Yes
|
|
T}@T{
|
|
No
|
|
T}@T{
|
|
No
|
|
T}@T{
|
|
R/W
|
|
T}
|
|
T{
|
|
pCloud
|
|
T}@T{
|
|
MD5, SHA1 \[u2077]
|
|
T}@T{
|
|
Yes
|
|
T}@T{
|
|
No
|
|
T}@T{
|
|
No
|
|
T}@T{
|
|
W
|
|
T}
|
|
T{
|
|
premiumize.me
|
|
T}@T{
|
|
\-
|
|
T}@T{
|
|
No
|
|
T}@T{
|
|
Yes
|
|
T}@T{
|
|
No
|
|
T}@T{
|
|
R
|
|
T}
|
|
T{
|
|
put.io
|
|
T}@T{
|
|
CRC\-32
|
|
T}@T{
|
|
Yes
|
|
T}@T{
|
|
No
|
|
T}@T{
|
|
Yes
|
|
T}@T{
|
|
R
|
|
T}
|
|
T{
|
|
QingStor
|
|
T}@T{
|
|
MD5
|
|
T}@T{
|
|
No
|
|
T}@T{
|
|
No
|
|
T}@T{
|
|
No
|
|
T}@T{
|
|
R/W
|
|
T}
|
|
T{
|
|
Seafile
|
|
T}@T{
|
|
\-
|
|
T}@T{
|
|
No
|
|
T}@T{
|
|
No
|
|
T}@T{
|
|
No
|
|
T}@T{
|
|
\-
|
|
T}
|
|
T{
|
|
SFTP
|
|
T}@T{
|
|
MD5, SHA1 \[S2]
|
|
T}@T{
|
|
Yes
|
|
T}@T{
|
|
Depends
|
|
T}@T{
|
|
No
|
|
T}@T{
|
|
\-
|
|
T}
|
|
T{
|
|
SugarSync
|
|
T}@T{
|
|
\-
|
|
T}@T{
|
|
No
|
|
T}@T{
|
|
No
|
|
T}@T{
|
|
No
|
|
T}@T{
|
|
\-
|
|
T}
|
|
T{
|
|
Tardigrade
|
|
T}@T{
|
|
\-
|
|
T}@T{
|
|
Yes
|
|
T}@T{
|
|
No
|
|
T}@T{
|
|
No
|
|
T}@T{
|
|
\-
|
|
T}
|
|
T{
|
|
WebDAV
|
|
T}@T{
|
|
MD5, SHA1 \[S3]
|
|
T}@T{
|
|
Yes \[u2074]
|
|
T}@T{
|
|
Depends
|
|
T}@T{
|
|
No
|
|
T}@T{
|
|
\-
|
|
T}
|
|
T{
|
|
Yandex Disk
|
|
T}@T{
|
|
MD5
|
|
T}@T{
|
|
Yes
|
|
T}@T{
|
|
No
|
|
T}@T{
|
|
No
|
|
T}@T{
|
|
R
|
|
T}
|
|
T{
|
|
Zoho WorkDrive
|
|
T}@T{
|
|
\-
|
|
T}@T{
|
|
No
|
|
T}@T{
|
|
No
|
|
T}@T{
|
|
No
|
|
T}@T{
|
|
\-
|
|
T}
|
|
T{
|
|
The local filesystem
|
|
T}@T{
|
|
All
|
|
T}@T{
|
|
Yes
|
|
T}@T{
|
|
Depends
|
|
T}@T{
|
|
No
|
|
T}@T{
|
|
\-
|
|
T}
|
|
.TE
|
|
.SS Notes
|
|
.PP
|
|
\[S1] Dropbox supports its own custom
|
|
hash (https://www.dropbox.com/developers/reference/content-hash).
|
|
This is an SHA256 sum of all the 4MB block SHA256s.
|
|
.PP
|
|
\[S2] SFTP supports checksums if the same login has shell access and
|
|
\f[C]md5sum\f[R] or \f[C]sha1sum\f[R] as well as \f[C]echo\f[R] are in
|
|
the remote\[aq]s PATH.
|
|
.PP
|
|
\[S3] WebDAV supports hashes when used with Owncloud and Nextcloud only.
|
|
.PP
|
|
\[u2074] WebDAV supports modtimes when used with Owncloud and Nextcloud
|
|
only.
|
|
.PP
|
|
\[u2075] Microsoft OneDrive Personal supports SHA1 hashes, whereas
|
|
OneDrive for business and SharePoint server support Microsoft\[aq]s own
|
|
QuickXorHash (https://docs.microsoft.com/en-us/onedrive/developer/code-snippets/quickxorhash).
|
|
.PP
|
|
\[u2076] Mail.ru uses its own modified SHA1 hash
|
|
.PP
|
|
\[u2077] pCloud only supports SHA1 (not MD5) in its EU region
|
|
.PP
|
|
\[u2078] Opendrive does not support creation of duplicate files using
|
|
their web client interface or other stock clients, but the underlying
|
|
storage platform has been determined to allow duplicate files, and it is
|
|
possible to create them with \f[C]rclone\f[R].
|
|
It may be that this is a mistake or an unsupported feature.
|
|
.SS Hash
|
|
.PP
|
|
The cloud storage system supports various hash types of the objects.
|
|
The hashes are used when transferring data as an integrity check and can
|
|
be specifically used with the \f[C]\-\-checksum\f[R] flag in syncs and
|
|
in the \f[C]check\f[R] command.
|
|
.PP
|
|
To use the verify checksums when transferring between cloud storage
|
|
systems they must support a common hash type.
|
|
.SS ModTime
|
|
.PP
|
|
The cloud storage system supports setting modification times on objects.
|
|
If it does then this enables a using the modification times as part of
|
|
the sync.
|
|
If not then only the size will be checked by default, though the MD5SUM
|
|
can be checked with the \f[C]\-\-checksum\f[R] flag.
|
|
.PP
|
|
All cloud storage systems support some kind of date on the object and
|
|
these will be set when transferring from the cloud storage system.
|
|
.SS Case Insensitive
|
|
.PP
|
|
If a cloud storage systems is case sensitive then it is possible to have
|
|
two files which differ only in case, e.g.
|
|
\f[C]file.txt\f[R] and \f[C]FILE.txt\f[R].
|
|
If a cloud storage system is case insensitive then that isn\[aq]t
|
|
possible.
|
|
.PP
|
|
This can cause problems when syncing between a case insensitive system
|
|
and a case sensitive system.
|
|
The symptom of this is that no matter how many times you run the sync it
|
|
never completes fully.
|
|
.PP
|
|
The local filesystem and SFTP may or may not be case sensitive depending
|
|
on OS.
|
|
.IP \[bu] 2
|
|
Windows \- usually case insensitive, though case is preserved
|
|
.IP \[bu] 2
|
|
OSX \- usually case insensitive, though it is possible to format case
|
|
sensitive
|
|
.IP \[bu] 2
|
|
Linux \- usually case sensitive, but there are case insensitive file
|
|
systems (e.g.
|
|
FAT formatted USB keys)
|
|
.PP
|
|
Most of the time this doesn\[aq]t cause any problems as people tend to
|
|
avoid files whose name differs only by case even on case sensitive
|
|
systems.
|
|
.SS Duplicate files
|
|
.PP
|
|
If a cloud storage system allows duplicate files then it can have two
|
|
objects with the same name.
|
|
.PP
|
|
This confuses rclone greatly when syncing \- use the
|
|
\f[C]rclone dedupe\f[R] command to rename or remove duplicates.
|
|
.SS Restricted filenames
|
|
.PP
|
|
Some cloud storage systems might have restrictions on the characters
|
|
that are usable in file or directory names.
|
|
When \f[C]rclone\f[R] detects such a name during a file upload, it will
|
|
transparently replace the restricted characters with similar looking
|
|
Unicode characters.
|
|
.PP
|
|
This process is designed to avoid ambiguous file names as much as
|
|
possible and allow to move files between many cloud storage systems
|
|
transparently.
|
|
.PP
|
|
The name shown by \f[C]rclone\f[R] to the user or during log output will
|
|
only contain a minimal set of replaced characters to ensure correct
|
|
formatting and not necessarily the actual name used on the cloud
|
|
storage.
|
|
.PP
|
|
This transformation is reversed when downloading a file or parsing
|
|
\f[C]rclone\f[R] arguments.
|
|
For example, when uploading a file named \f[C]my file?.txt\f[R] to
|
|
Onedrive will be displayed as \f[C]my file?.txt\f[R] on the console, but
|
|
stored as \f[C]my file\[uFF1F].txt\f[R] (the \f[C]?\f[R] gets replaced
|
|
by the similar looking \f[C]\[uFF1F]\f[R] character) to Onedrive.
|
|
The reverse transformation allows to read a
|
|
file\f[C]unusual/name.txt\f[R] from Google Drive, by passing the name
|
|
\f[C]unusual\[uFF0F]name.txt\f[R] (the \f[C]/\f[R] needs to be replaced
|
|
by the similar looking \f[C]\[uFF0F]\f[R] character) on the command
|
|
line.
|
|
.SS Default restricted characters
|
|
.PP
|
|
The table below shows the characters that are replaced by default.
|
|
.PP
|
|
When a replacement character is found in a filename, this character will
|
|
be escaped with the \f[C]\[u201B]\f[R] character to avoid ambiguous file
|
|
names.
|
|
(e.g.
|
|
a file named \f[C]\[u2400].txt\f[R] would shown as
|
|
\f[C]\[u201B]\[u2400].txt\f[R])
|
|
.PP
|
|
Each cloud storage backend can use a different set of characters, which
|
|
will be specified in the documentation for each backend.
|
|
.PP
|
|
.TS
|
|
tab(@);
|
|
l c c.
|
|
T{
|
|
Character
|
|
T}@T{
|
|
Value
|
|
T}@T{
|
|
Replacement
|
|
T}
|
|
_
|
|
T{
|
|
NUL
|
|
T}@T{
|
|
0x00
|
|
T}@T{
|
|
\[u2400]
|
|
T}
|
|
T{
|
|
SOH
|
|
T}@T{
|
|
0x01
|
|
T}@T{
|
|
\[u2401]
|
|
T}
|
|
T{
|
|
STX
|
|
T}@T{
|
|
0x02
|
|
T}@T{
|
|
\[u2402]
|
|
T}
|
|
T{
|
|
ETX
|
|
T}@T{
|
|
0x03
|
|
T}@T{
|
|
\[u2403]
|
|
T}
|
|
T{
|
|
EOT
|
|
T}@T{
|
|
0x04
|
|
T}@T{
|
|
\[u2404]
|
|
T}
|
|
T{
|
|
ENQ
|
|
T}@T{
|
|
0x05
|
|
T}@T{
|
|
\[u2405]
|
|
T}
|
|
T{
|
|
ACK
|
|
T}@T{
|
|
0x06
|
|
T}@T{
|
|
\[u2406]
|
|
T}
|
|
T{
|
|
BEL
|
|
T}@T{
|
|
0x07
|
|
T}@T{
|
|
\[u2407]
|
|
T}
|
|
T{
|
|
BS
|
|
T}@T{
|
|
0x08
|
|
T}@T{
|
|
\[u2408]
|
|
T}
|
|
T{
|
|
HT
|
|
T}@T{
|
|
0x09
|
|
T}@T{
|
|
\[u2409]
|
|
T}
|
|
T{
|
|
LF
|
|
T}@T{
|
|
0x0A
|
|
T}@T{
|
|
\[u240A]
|
|
T}
|
|
T{
|
|
VT
|
|
T}@T{
|
|
0x0B
|
|
T}@T{
|
|
\[u240B]
|
|
T}
|
|
T{
|
|
FF
|
|
T}@T{
|
|
0x0C
|
|
T}@T{
|
|
\[u240C]
|
|
T}
|
|
T{
|
|
CR
|
|
T}@T{
|
|
0x0D
|
|
T}@T{
|
|
\[u240D]
|
|
T}
|
|
T{
|
|
SO
|
|
T}@T{
|
|
0x0E
|
|
T}@T{
|
|
\[u240E]
|
|
T}
|
|
T{
|
|
SI
|
|
T}@T{
|
|
0x0F
|
|
T}@T{
|
|
\[u240F]
|
|
T}
|
|
T{
|
|
DLE
|
|
T}@T{
|
|
0x10
|
|
T}@T{
|
|
\[u2410]
|
|
T}
|
|
T{
|
|
DC1
|
|
T}@T{
|
|
0x11
|
|
T}@T{
|
|
\[u2411]
|
|
T}
|
|
T{
|
|
DC2
|
|
T}@T{
|
|
0x12
|
|
T}@T{
|
|
\[u2412]
|
|
T}
|
|
T{
|
|
DC3
|
|
T}@T{
|
|
0x13
|
|
T}@T{
|
|
\[u2413]
|
|
T}
|
|
T{
|
|
DC4
|
|
T}@T{
|
|
0x14
|
|
T}@T{
|
|
\[u2414]
|
|
T}
|
|
T{
|
|
NAK
|
|
T}@T{
|
|
0x15
|
|
T}@T{
|
|
\[u2415]
|
|
T}
|
|
T{
|
|
SYN
|
|
T}@T{
|
|
0x16
|
|
T}@T{
|
|
\[u2416]
|
|
T}
|
|
T{
|
|
ETB
|
|
T}@T{
|
|
0x17
|
|
T}@T{
|
|
\[u2417]
|
|
T}
|
|
T{
|
|
CAN
|
|
T}@T{
|
|
0x18
|
|
T}@T{
|
|
\[u2418]
|
|
T}
|
|
T{
|
|
EM
|
|
T}@T{
|
|
0x19
|
|
T}@T{
|
|
\[u2419]
|
|
T}
|
|
T{
|
|
SUB
|
|
T}@T{
|
|
0x1A
|
|
T}@T{
|
|
\[u241A]
|
|
T}
|
|
T{
|
|
ESC
|
|
T}@T{
|
|
0x1B
|
|
T}@T{
|
|
\[u241B]
|
|
T}
|
|
T{
|
|
FS
|
|
T}@T{
|
|
0x1C
|
|
T}@T{
|
|
\[u241C]
|
|
T}
|
|
T{
|
|
GS
|
|
T}@T{
|
|
0x1D
|
|
T}@T{
|
|
\[u241D]
|
|
T}
|
|
T{
|
|
RS
|
|
T}@T{
|
|
0x1E
|
|
T}@T{
|
|
\[u241E]
|
|
T}
|
|
T{
|
|
US
|
|
T}@T{
|
|
0x1F
|
|
T}@T{
|
|
\[u241F]
|
|
T}
|
|
T{
|
|
/
|
|
T}@T{
|
|
0x2F
|
|
T}@T{
|
|
\[uFF0F]
|
|
T}
|
|
T{
|
|
DEL
|
|
T}@T{
|
|
0x7F
|
|
T}@T{
|
|
\[u2421]
|
|
T}
|
|
.TE
|
|
.PP
|
|
The default encoding will also encode these file names as they are
|
|
problematic with many cloud storage systems.
|
|
.PP
|
|
.TS
|
|
tab(@);
|
|
l c.
|
|
T{
|
|
File name
|
|
T}@T{
|
|
Replacement
|
|
T}
|
|
_
|
|
T{
|
|
\&.
|
|
T}@T{
|
|
\[uFF0E]
|
|
T}
|
|
T{
|
|
\&..
|
|
T}@T{
|
|
\[uFF0E]\[uFF0E]
|
|
T}
|
|
.TE
|
|
.SS Invalid UTF\-8 bytes
|
|
.PP
|
|
Some backends only support a sequence of well formed UTF\-8 bytes as
|
|
file or directory names.
|
|
.PP
|
|
In this case all invalid UTF\-8 bytes will be replaced with a quoted
|
|
representation of the byte value to allow uploading a file to such a
|
|
backend.
|
|
For example, the invalid byte \f[C]0xFE\f[R] will be encoded as
|
|
\f[C]\[u201B]FE\f[R].
|
|
.PP
|
|
A common source of invalid UTF\-8 bytes are local filesystems, that
|
|
store names in a different encoding than UTF\-8 or UTF\-16, like latin1.
|
|
See the local filenames (https://rclone.org/local/#filenames) section
|
|
for details.
|
|
.SS Encoding option
|
|
.PP
|
|
Most backends have an encoding options, specified as a flag
|
|
\f[C]\-\-backend\-encoding\f[R] where \f[C]backend\f[R] is the name of
|
|
the backend, or as a config parameter \f[C]encoding\f[R] (you\[aq]ll
|
|
need to select the Advanced config in \f[C]rclone config\f[R] to see
|
|
it).
|
|
.PP
|
|
This will have default value which encodes and decodes characters in
|
|
such a way as to preserve the maximum number of characters (see above).
|
|
.PP
|
|
However this can be incorrect in some scenarios, for example if you have
|
|
a Windows file system with characters such as \f[C]\[uFF0A]\f[R] and
|
|
\f[C]\[uFF1F]\f[R] that you want to remain as those characters on the
|
|
remote rather than being translated to \f[C]*\f[R] and \f[C]?\f[R].
|
|
.PP
|
|
The \f[C]\-\-backend\-encoding\f[R] flags allow you to change that.
|
|
You can disable the encoding completely with
|
|
\f[C]\-\-backend\-encoding None\f[R] or set \f[C]encoding = None\f[R] in
|
|
the config file.
|
|
.PP
|
|
Encoding takes a comma separated list of encodings.
|
|
You can see the list of all available characters by passing an invalid
|
|
value to this flag, e.g.
|
|
\f[C]\-\-local\-encoding \[dq]help\[dq]\f[R] and
|
|
\f[C]rclone help flags encoding\f[R] will show you the defaults for the
|
|
backends.
|
|
.PP
|
|
.TS
|
|
tab(@);
|
|
l l.
|
|
T{
|
|
Encoding
|
|
T}@T{
|
|
Characters
|
|
T}
|
|
_
|
|
T{
|
|
Asterisk
|
|
T}@T{
|
|
\f[C]*\f[R]
|
|
T}
|
|
T{
|
|
BackQuote
|
|
T}@T{
|
|
\f[C]\[ga]\f[R]
|
|
T}
|
|
T{
|
|
BackSlash
|
|
T}@T{
|
|
\f[C]\[rs]\f[R]
|
|
T}
|
|
T{
|
|
Colon
|
|
T}@T{
|
|
\f[C]:\f[R]
|
|
T}
|
|
T{
|
|
CrLf
|
|
T}@T{
|
|
CR 0x0D, LF 0x0A
|
|
T}
|
|
T{
|
|
Ctl
|
|
T}@T{
|
|
All control characters 0x00\-0x1F
|
|
T}
|
|
T{
|
|
Del
|
|
T}@T{
|
|
DEL 0x7F
|
|
T}
|
|
T{
|
|
Dollar
|
|
T}@T{
|
|
\f[C]$\f[R]
|
|
T}
|
|
T{
|
|
Dot
|
|
T}@T{
|
|
\f[C].\f[R]
|
|
T}
|
|
T{
|
|
DoubleQuote
|
|
T}@T{
|
|
\f[C]\[dq]\f[R]
|
|
T}
|
|
T{
|
|
Hash
|
|
T}@T{
|
|
\f[C]#\f[R]
|
|
T}
|
|
T{
|
|
InvalidUtf8
|
|
T}@T{
|
|
An invalid UTF\-8 character (e.g.
|
|
latin1)
|
|
T}
|
|
T{
|
|
LeftCrLfHtVt
|
|
T}@T{
|
|
CR 0x0D, LF 0x0A,HT 0x09, VT 0x0B on the left of a string
|
|
T}
|
|
T{
|
|
LeftPeriod
|
|
T}@T{
|
|
\f[C].\f[R] on the left of a string
|
|
T}
|
|
T{
|
|
LeftSpace
|
|
T}@T{
|
|
SPACE on the left of a string
|
|
T}
|
|
T{
|
|
LeftTilde
|
|
T}@T{
|
|
\f[C]\[ti]\f[R] on the left of a string
|
|
T}
|
|
T{
|
|
LtGt
|
|
T}@T{
|
|
\f[C]<\f[R], \f[C]>\f[R]
|
|
T}
|
|
T{
|
|
None
|
|
T}@T{
|
|
No characters are encoded
|
|
T}
|
|
T{
|
|
Percent
|
|
T}@T{
|
|
\f[C]%\f[R]
|
|
T}
|
|
T{
|
|
Pipe
|
|
T}@T{
|
|
|
|
|
T}
|
|
T{
|
|
Question
|
|
T}@T{
|
|
\f[C]?\f[R]
|
|
T}
|
|
T{
|
|
RightCrLfHtVt
|
|
T}@T{
|
|
CR 0x0D, LF 0x0A, HT 0x09, VT 0x0B on the right of a string
|
|
T}
|
|
T{
|
|
RightPeriod
|
|
T}@T{
|
|
\f[C].\f[R] on the right of a string
|
|
T}
|
|
T{
|
|
RightSpace
|
|
T}@T{
|
|
SPACE on the right of a string
|
|
T}
|
|
T{
|
|
SingleQuote
|
|
T}@T{
|
|
\f[C]\[aq]\f[R]
|
|
T}
|
|
T{
|
|
Slash
|
|
T}@T{
|
|
\f[C]/\f[R]
|
|
T}
|
|
.TE
|
|
.PP
|
|
To take a specific example, the FTP backend\[aq]s default encoding is
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
\-\-ftp\-encoding \[dq]Slash,Del,Ctl,RightSpace,Dot\[dq]
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
However, let\[aq]s say the FTP server is running on Windows and
|
|
can\[aq]t have any of the invalid Windows characters in file names.
|
|
You are backing up Linux servers to this FTP server which do have those
|
|
characters in file names.
|
|
So you would add the Windows set which are
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
Slash,LtGt,DoubleQuote,Colon,Question,Asterisk,Pipe,BackSlash,Ctl,RightSpace,RightPeriod,InvalidUtf8,Dot
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
to the existing ones, giving:
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
Slash,LtGt,DoubleQuote,Colon,Question,Asterisk,Pipe,BackSlash,Ctl,RightSpace,RightPeriod,InvalidUtf8,Dot,Del,RightSpace
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
This can be specified using the \f[C]\-\-ftp\-encoding\f[R] flag or
|
|
using an \f[C]encoding\f[R] parameter in the config file.
|
|
.PP
|
|
Or let\[aq]s say you have a Windows server but you want to preserve
|
|
\f[C]\[uFF0A]\f[R] and \f[C]\[uFF1F]\f[R], you would then have this as
|
|
the encoding (the Windows encoding minus \f[C]Asterisk\f[R] and
|
|
\f[C]Question\f[R]).
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
Slash,LtGt,DoubleQuote,Colon,Pipe,BackSlash,Ctl,RightSpace,RightPeriod,InvalidUtf8,Dot
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
This can be specified using the \f[C]\-\-local\-encoding\f[R] flag or
|
|
using an \f[C]encoding\f[R] parameter in the config file.
|
|
.SS MIME Type
|
|
.PP
|
|
MIME types (also known as media types) classify types of documents using
|
|
a simple text classification, e.g.
|
|
\f[C]text/html\f[R] or \f[C]application/pdf\f[R].
|
|
.PP
|
|
Some cloud storage systems support reading (\f[C]R\f[R]) the MIME type
|
|
of objects and some support writing (\f[C]W\f[R]) the MIME type of
|
|
objects.
|
|
.PP
|
|
The MIME type can be important if you are serving files directly to HTTP
|
|
from the storage system.
|
|
.PP
|
|
If you are copying from a remote which supports reading (\f[C]R\f[R]) to
|
|
a remote which supports writing (\f[C]W\f[R]) then rclone will preserve
|
|
the MIME types.
|
|
Otherwise they will be guessed from the extension, or the remote itself
|
|
may assign the MIME type.
|
|
.SS Optional Features
|
|
.PP
|
|
All rclone remotes support a base command set.
|
|
Other features depend upon backend specific capabilities.
|
|
.PP
|
|
.TS
|
|
tab(@);
|
|
l c c c c c c c c c c.
|
|
T{
|
|
Name
|
|
T}@T{
|
|
Purge
|
|
T}@T{
|
|
Copy
|
|
T}@T{
|
|
Move
|
|
T}@T{
|
|
DirMove
|
|
T}@T{
|
|
CleanUp
|
|
T}@T{
|
|
ListR
|
|
T}@T{
|
|
StreamUpload
|
|
T}@T{
|
|
LinkSharing
|
|
T}@T{
|
|
About
|
|
T}@T{
|
|
EmptyDir
|
|
T}
|
|
_
|
|
T{
|
|
1Fichier
|
|
T}@T{
|
|
No
|
|
T}@T{
|
|
Yes
|
|
T}@T{
|
|
Yes
|
|
T}@T{
|
|
No
|
|
T}@T{
|
|
No
|
|
T}@T{
|
|
No
|
|
T}@T{
|
|
No
|
|
T}@T{
|
|
Yes
|
|
T}@T{
|
|
No
|
|
T}@T{
|
|
Yes
|
|
T}
|
|
T{
|
|
Amazon Drive
|
|
T}@T{
|
|
Yes
|
|
T}@T{
|
|
No
|
|
T}@T{
|
|
Yes
|
|
T}@T{
|
|
Yes
|
|
T}@T{
|
|
No
|
|
T}@T{
|
|
No
|
|
T}@T{
|
|
No
|
|
T}@T{
|
|
No
|
|
T}@T{
|
|
No
|
|
T}@T{
|
|
Yes
|
|
T}
|
|
T{
|
|
Amazon S3
|
|
T}@T{
|
|
No
|
|
T}@T{
|
|
Yes
|
|
T}@T{
|
|
No
|
|
T}@T{
|
|
No
|
|
T}@T{
|
|
Yes
|
|
T}@T{
|
|
Yes
|
|
T}@T{
|
|
Yes
|
|
T}@T{
|
|
Yes
|
|
T}@T{
|
|
No
|
|
T}@T{
|
|
No
|
|
T}
|
|
T{
|
|
Backblaze B2
|
|
T}@T{
|
|
No
|
|
T}@T{
|
|
Yes
|
|
T}@T{
|
|
No
|
|
T}@T{
|
|
No
|
|
T}@T{
|
|
Yes
|
|
T}@T{
|
|
Yes
|
|
T}@T{
|
|
Yes
|
|
T}@T{
|
|
Yes
|
|
T}@T{
|
|
No
|
|
T}@T{
|
|
No
|
|
T}
|
|
T{
|
|
Box
|
|
T}@T{
|
|
Yes
|
|
T}@T{
|
|
Yes
|
|
T}@T{
|
|
Yes
|
|
T}@T{
|
|
Yes
|
|
T}@T{
|
|
Yes \[dd]\[dd]
|
|
T}@T{
|
|
No
|
|
T}@T{
|
|
Yes
|
|
T}@T{
|
|
Yes
|
|
T}@T{
|
|
Yes
|
|
T}@T{
|
|
Yes
|
|
T}
|
|
T{
|
|
Citrix ShareFile
|
|
T}@T{
|
|
Yes
|
|
T}@T{
|
|
Yes
|
|
T}@T{
|
|
Yes
|
|
T}@T{
|
|
Yes
|
|
T}@T{
|
|
No
|
|
T}@T{
|
|
No
|
|
T}@T{
|
|
Yes
|
|
T}@T{
|
|
No
|
|
T}@T{
|
|
No
|
|
T}@T{
|
|
Yes
|
|
T}
|
|
T{
|
|
Dropbox
|
|
T}@T{
|
|
Yes
|
|
T}@T{
|
|
Yes
|
|
T}@T{
|
|
Yes
|
|
T}@T{
|
|
Yes
|
|
T}@T{
|
|
No
|
|
T}@T{
|
|
No
|
|
T}@T{
|
|
Yes
|
|
T}@T{
|
|
Yes
|
|
T}@T{
|
|
Yes
|
|
T}@T{
|
|
Yes
|
|
T}
|
|
T{
|
|
Enterprise File Fabric
|
|
T}@T{
|
|
Yes
|
|
T}@T{
|
|
Yes
|
|
T}@T{
|
|
Yes
|
|
T}@T{
|
|
Yes
|
|
T}@T{
|
|
Yes
|
|
T}@T{
|
|
No
|
|
T}@T{
|
|
No
|
|
T}@T{
|
|
No
|
|
T}@T{
|
|
No
|
|
T}@T{
|
|
Yes
|
|
T}
|
|
T{
|
|
FTP
|
|
T}@T{
|
|
No
|
|
T}@T{
|
|
No
|
|
T}@T{
|
|
Yes
|
|
T}@T{
|
|
Yes
|
|
T}@T{
|
|
No
|
|
T}@T{
|
|
No
|
|
T}@T{
|
|
Yes
|
|
T}@T{
|
|
No
|
|
T}@T{
|
|
No
|
|
T}@T{
|
|
Yes
|
|
T}
|
|
T{
|
|
Google Cloud Storage
|
|
T}@T{
|
|
Yes
|
|
T}@T{
|
|
Yes
|
|
T}@T{
|
|
No
|
|
T}@T{
|
|
No
|
|
T}@T{
|
|
No
|
|
T}@T{
|
|
Yes
|
|
T}@T{
|
|
Yes
|
|
T}@T{
|
|
No
|
|
T}@T{
|
|
No
|
|
T}@T{
|
|
No
|
|
T}
|
|
T{
|
|
Google Drive
|
|
T}@T{
|
|
Yes
|
|
T}@T{
|
|
Yes
|
|
T}@T{
|
|
Yes
|
|
T}@T{
|
|
Yes
|
|
T}@T{
|
|
Yes
|
|
T}@T{
|
|
Yes
|
|
T}@T{
|
|
Yes
|
|
T}@T{
|
|
Yes
|
|
T}@T{
|
|
Yes
|
|
T}@T{
|
|
Yes
|
|
T}
|
|
T{
|
|
Google Photos
|
|
T}@T{
|
|
No
|
|
T}@T{
|
|
No
|
|
T}@T{
|
|
No
|
|
T}@T{
|
|
No
|
|
T}@T{
|
|
No
|
|
T}@T{
|
|
No
|
|
T}@T{
|
|
No
|
|
T}@T{
|
|
No
|
|
T}@T{
|
|
No
|
|
T}@T{
|
|
No
|
|
T}
|
|
T{
|
|
HDFS
|
|
T}@T{
|
|
Yes
|
|
T}@T{
|
|
No
|
|
T}@T{
|
|
No
|
|
T}@T{
|
|
No
|
|
T}@T{
|
|
No
|
|
T}@T{
|
|
No
|
|
T}@T{
|
|
Yes
|
|
T}@T{
|
|
No
|
|
T}@T{
|
|
Yes
|
|
T}@T{
|
|
Yes
|
|
T}
|
|
T{
|
|
HTTP
|
|
T}@T{
|
|
No
|
|
T}@T{
|
|
No
|
|
T}@T{
|
|
No
|
|
T}@T{
|
|
No
|
|
T}@T{
|
|
No
|
|
T}@T{
|
|
No
|
|
T}@T{
|
|
No
|
|
T}@T{
|
|
No
|
|
T}@T{
|
|
No
|
|
T}@T{
|
|
Yes
|
|
T}
|
|
T{
|
|
Hubic
|
|
T}@T{
|
|
Yes \[dg]
|
|
T}@T{
|
|
Yes
|
|
T}@T{
|
|
No
|
|
T}@T{
|
|
No
|
|
T}@T{
|
|
No
|
|
T}@T{
|
|
Yes
|
|
T}@T{
|
|
Yes
|
|
T}@T{
|
|
No
|
|
T}@T{
|
|
Yes
|
|
T}@T{
|
|
No
|
|
T}
|
|
T{
|
|
Jottacloud
|
|
T}@T{
|
|
Yes
|
|
T}@T{
|
|
Yes
|
|
T}@T{
|
|
Yes
|
|
T}@T{
|
|
Yes
|
|
T}@T{
|
|
Yes
|
|
T}@T{
|
|
Yes
|
|
T}@T{
|
|
No
|
|
T}@T{
|
|
Yes
|
|
T}@T{
|
|
Yes
|
|
T}@T{
|
|
Yes
|
|
T}
|
|
T{
|
|
Mail.ru Cloud
|
|
T}@T{
|
|
Yes
|
|
T}@T{
|
|
Yes
|
|
T}@T{
|
|
Yes
|
|
T}@T{
|
|
Yes
|
|
T}@T{
|
|
Yes
|
|
T}@T{
|
|
No
|
|
T}@T{
|
|
No
|
|
T}@T{
|
|
Yes
|
|
T}@T{
|
|
Yes
|
|
T}@T{
|
|
Yes
|
|
T}
|
|
T{
|
|
Mega
|
|
T}@T{
|
|
Yes
|
|
T}@T{
|
|
No
|
|
T}@T{
|
|
Yes
|
|
T}@T{
|
|
Yes
|
|
T}@T{
|
|
Yes
|
|
T}@T{
|
|
No
|
|
T}@T{
|
|
No
|
|
T}@T{
|
|
Yes
|
|
T}@T{
|
|
Yes
|
|
T}@T{
|
|
Yes
|
|
T}
|
|
T{
|
|
Memory
|
|
T}@T{
|
|
No
|
|
T}@T{
|
|
Yes
|
|
T}@T{
|
|
No
|
|
T}@T{
|
|
No
|
|
T}@T{
|
|
No
|
|
T}@T{
|
|
Yes
|
|
T}@T{
|
|
Yes
|
|
T}@T{
|
|
No
|
|
T}@T{
|
|
No
|
|
T}@T{
|
|
No
|
|
T}
|
|
T{
|
|
Microsoft Azure Blob Storage
|
|
T}@T{
|
|
Yes
|
|
T}@T{
|
|
Yes
|
|
T}@T{
|
|
No
|
|
T}@T{
|
|
No
|
|
T}@T{
|
|
No
|
|
T}@T{
|
|
Yes
|
|
T}@T{
|
|
Yes
|
|
T}@T{
|
|
No
|
|
T}@T{
|
|
No
|
|
T}@T{
|
|
No
|
|
T}
|
|
T{
|
|
Microsoft OneDrive
|
|
T}@T{
|
|
Yes
|
|
T}@T{
|
|
Yes
|
|
T}@T{
|
|
Yes
|
|
T}@T{
|
|
Yes
|
|
T}@T{
|
|
Yes
|
|
T}@T{
|
|
No
|
|
T}@T{
|
|
No
|
|
T}@T{
|
|
Yes
|
|
T}@T{
|
|
Yes
|
|
T}@T{
|
|
Yes
|
|
T}
|
|
T{
|
|
OpenDrive
|
|
T}@T{
|
|
Yes
|
|
T}@T{
|
|
Yes
|
|
T}@T{
|
|
Yes
|
|
T}@T{
|
|
Yes
|
|
T}@T{
|
|
No
|
|
T}@T{
|
|
No
|
|
T}@T{
|
|
No
|
|
T}@T{
|
|
No
|
|
T}@T{
|
|
No
|
|
T}@T{
|
|
Yes
|
|
T}
|
|
T{
|
|
OpenStack Swift
|
|
T}@T{
|
|
Yes \[dg]
|
|
T}@T{
|
|
Yes
|
|
T}@T{
|
|
No
|
|
T}@T{
|
|
No
|
|
T}@T{
|
|
No
|
|
T}@T{
|
|
Yes
|
|
T}@T{
|
|
Yes
|
|
T}@T{
|
|
No
|
|
T}@T{
|
|
Yes
|
|
T}@T{
|
|
No
|
|
T}
|
|
T{
|
|
pCloud
|
|
T}@T{
|
|
Yes
|
|
T}@T{
|
|
Yes
|
|
T}@T{
|
|
Yes
|
|
T}@T{
|
|
Yes
|
|
T}@T{
|
|
Yes
|
|
T}@T{
|
|
No
|
|
T}@T{
|
|
No
|
|
T}@T{
|
|
Yes
|
|
T}@T{
|
|
Yes
|
|
T}@T{
|
|
Yes
|
|
T}
|
|
T{
|
|
premiumize.me
|
|
T}@T{
|
|
Yes
|
|
T}@T{
|
|
No
|
|
T}@T{
|
|
Yes
|
|
T}@T{
|
|
Yes
|
|
T}@T{
|
|
No
|
|
T}@T{
|
|
No
|
|
T}@T{
|
|
No
|
|
T}@T{
|
|
Yes
|
|
T}@T{
|
|
Yes
|
|
T}@T{
|
|
Yes
|
|
T}
|
|
T{
|
|
put.io
|
|
T}@T{
|
|
Yes
|
|
T}@T{
|
|
No
|
|
T}@T{
|
|
Yes
|
|
T}@T{
|
|
Yes
|
|
T}@T{
|
|
Yes
|
|
T}@T{
|
|
No
|
|
T}@T{
|
|
Yes
|
|
T}@T{
|
|
No
|
|
T}@T{
|
|
Yes
|
|
T}@T{
|
|
Yes
|
|
T}
|
|
T{
|
|
QingStor
|
|
T}@T{
|
|
No
|
|
T}@T{
|
|
Yes
|
|
T}@T{
|
|
No
|
|
T}@T{
|
|
No
|
|
T}@T{
|
|
Yes
|
|
T}@T{
|
|
Yes
|
|
T}@T{
|
|
No
|
|
T}@T{
|
|
No
|
|
T}@T{
|
|
No
|
|
T}@T{
|
|
No
|
|
T}
|
|
T{
|
|
Seafile
|
|
T}@T{
|
|
Yes
|
|
T}@T{
|
|
Yes
|
|
T}@T{
|
|
Yes
|
|
T}@T{
|
|
Yes
|
|
T}@T{
|
|
Yes
|
|
T}@T{
|
|
Yes
|
|
T}@T{
|
|
Yes
|
|
T}@T{
|
|
Yes
|
|
T}@T{
|
|
Yes
|
|
T}@T{
|
|
Yes
|
|
T}
|
|
T{
|
|
SFTP
|
|
T}@T{
|
|
No
|
|
T}@T{
|
|
No
|
|
T}@T{
|
|
Yes
|
|
T}@T{
|
|
Yes
|
|
T}@T{
|
|
No
|
|
T}@T{
|
|
No
|
|
T}@T{
|
|
Yes
|
|
T}@T{
|
|
No
|
|
T}@T{
|
|
Yes
|
|
T}@T{
|
|
Yes
|
|
T}
|
|
T{
|
|
SugarSync
|
|
T}@T{
|
|
Yes
|
|
T}@T{
|
|
Yes
|
|
T}@T{
|
|
Yes
|
|
T}@T{
|
|
Yes
|
|
T}@T{
|
|
No
|
|
T}@T{
|
|
No
|
|
T}@T{
|
|
Yes
|
|
T}@T{
|
|
Yes
|
|
T}@T{
|
|
No
|
|
T}@T{
|
|
Yes
|
|
T}
|
|
T{
|
|
Tardigrade
|
|
T}@T{
|
|
Yes \[dg]
|
|
T}@T{
|
|
No
|
|
T}@T{
|
|
No
|
|
T}@T{
|
|
No
|
|
T}@T{
|
|
No
|
|
T}@T{
|
|
Yes
|
|
T}@T{
|
|
Yes
|
|
T}@T{
|
|
No
|
|
T}@T{
|
|
No
|
|
T}@T{
|
|
No
|
|
T}
|
|
T{
|
|
WebDAV
|
|
T}@T{
|
|
Yes
|
|
T}@T{
|
|
Yes
|
|
T}@T{
|
|
Yes
|
|
T}@T{
|
|
Yes
|
|
T}@T{
|
|
No
|
|
T}@T{
|
|
No
|
|
T}@T{
|
|
Yes \[dd]
|
|
T}@T{
|
|
No
|
|
T}@T{
|
|
Yes
|
|
T}@T{
|
|
Yes
|
|
T}
|
|
T{
|
|
Yandex Disk
|
|
T}@T{
|
|
Yes
|
|
T}@T{
|
|
Yes
|
|
T}@T{
|
|
Yes
|
|
T}@T{
|
|
Yes
|
|
T}@T{
|
|
Yes
|
|
T}@T{
|
|
No
|
|
T}@T{
|
|
Yes
|
|
T}@T{
|
|
Yes
|
|
T}@T{
|
|
Yes
|
|
T}@T{
|
|
Yes
|
|
T}
|
|
T{
|
|
Zoho WorkDrive
|
|
T}@T{
|
|
Yes
|
|
T}@T{
|
|
Yes
|
|
T}@T{
|
|
Yes
|
|
T}@T{
|
|
Yes
|
|
T}@T{
|
|
No
|
|
T}@T{
|
|
No
|
|
T}@T{
|
|
No
|
|
T}@T{
|
|
No
|
|
T}@T{
|
|
Yes
|
|
T}@T{
|
|
Yes
|
|
T}
|
|
T{
|
|
The local filesystem
|
|
T}@T{
|
|
Yes
|
|
T}@T{
|
|
No
|
|
T}@T{
|
|
Yes
|
|
T}@T{
|
|
Yes
|
|
T}@T{
|
|
No
|
|
T}@T{
|
|
No
|
|
T}@T{
|
|
Yes
|
|
T}@T{
|
|
No
|
|
T}@T{
|
|
Yes
|
|
T}@T{
|
|
Yes
|
|
T}
|
|
.TE
|
|
.SS Purge
|
|
.PP
|
|
This deletes a directory quicker than just deleting all the files in the
|
|
directory.
|
|
.PP
|
|
\[dg] Note Swift, Hubic, and Tardigrade implement this in order to
|
|
delete directory markers but they don\[aq]t actually have a quicker way
|
|
of deleting files other than deleting them individually.
|
|
.PP
|
|
\[dd] StreamUpload is not supported with Nextcloud
|
|
.SS Copy
|
|
.PP
|
|
Used when copying an object to and from the same remote.
|
|
This known as a server\-side copy so you can copy a file without
|
|
downloading it and uploading it again.
|
|
It is used if you use \f[C]rclone copy\f[R] or \f[C]rclone move\f[R] if
|
|
the remote doesn\[aq]t support \f[C]Move\f[R] directly.
|
|
.PP
|
|
If the server doesn\[aq]t support \f[C]Copy\f[R] directly then for copy
|
|
operations the file is downloaded then re\-uploaded.
|
|
.SS Move
|
|
.PP
|
|
Used when moving/renaming an object on the same remote.
|
|
This is known as a server\-side move of a file.
|
|
This is used in \f[C]rclone move\f[R] if the server doesn\[aq]t support
|
|
\f[C]DirMove\f[R].
|
|
.PP
|
|
If the server isn\[aq]t capable of \f[C]Move\f[R] then rclone simulates
|
|
it with \f[C]Copy\f[R] then delete.
|
|
If the server doesn\[aq]t support \f[C]Copy\f[R] then rclone will
|
|
download the file and re\-upload it.
|
|
.SS DirMove
|
|
.PP
|
|
This is used to implement \f[C]rclone move\f[R] to move a directory if
|
|
possible.
|
|
If it isn\[aq]t then it will use \f[C]Move\f[R] on each file (which
|
|
falls back to \f[C]Copy\f[R] then download and upload \- see
|
|
\f[C]Move\f[R] section).
|
|
.SS CleanUp
|
|
.PP
|
|
This is used for emptying the trash for a remote by
|
|
\f[C]rclone cleanup\f[R].
|
|
.PP
|
|
If the server can\[aq]t do \f[C]CleanUp\f[R] then
|
|
\f[C]rclone cleanup\f[R] will return an error.
|
|
.PP
|
|
\[dd]\[dd] Note that while Box implements this it has to delete every
|
|
file individually so it will be slower than emptying the trash via the
|
|
WebUI
|
|
.SS ListR
|
|
.PP
|
|
The remote supports a recursive list to list all the contents beneath a
|
|
directory quickly.
|
|
This enables the \f[C]\-\-fast\-list\f[R] flag to work.
|
|
See the rclone docs (https://rclone.org/docs/#fast-list) for more
|
|
details.
|
|
.SS StreamUpload
|
|
.PP
|
|
Some remotes allow files to be uploaded without knowing the file size in
|
|
advance.
|
|
This allows certain operations to work without spooling the file to
|
|
local disk first, e.g.
|
|
\f[C]rclone rcat\f[R].
|
|
.SS LinkSharing
|
|
.PP
|
|
Sets the necessary permissions on a file or folder and prints a link
|
|
that allows others to access them, even if they don\[aq]t have an
|
|
account on the particular cloud provider.
|
|
.SS About
|
|
.PP
|
|
Rclone \f[C]about\f[R] prints quota information for a remote.
|
|
Typical output includes bytes used, free, quota and in trash.
|
|
.PP
|
|
If a remote lacks about capability \f[C]rclone about remote:\f[R]returns
|
|
an error.
|
|
.PP
|
|
Backends without about capability cannot determine free space for an
|
|
rclone mount, or use policy \f[C]mfs\f[R] (most free space) as a member
|
|
of an rclone union remote.
|
|
.PP
|
|
See rclone about command (https://rclone.org/commands/rclone_about/)
|
|
.SS EmptyDir
|
|
.PP
|
|
The remote supports empty directories.
|
|
See Limitations (https://rclone.org/bugs/#limitations) for details.
|
|
Most Object/Bucket based remotes do not support this.
|
|
.SH Global Flags
|
|
.PP
|
|
This describes the global flags available to every rclone command split
|
|
into two groups, non backend and backend flags.
|
|
.SS Non Backend Flags
|
|
.PP
|
|
These flags are available for every command.
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
\-\-ask\-password Allow prompt for password for encrypted configuration. (default true)
|
|
\-\-auto\-confirm If enabled, do not request console confirmation.
|
|
\-\-backup\-dir string Make backups into hierarchy based in DIR.
|
|
\-\-bind string Local address to bind to for outgoing connections, IPv4, IPv6 or name.
|
|
\-\-buffer\-size SizeSuffix In memory buffer size when reading files for each \-\-transfer. (default 16M)
|
|
\-\-bwlimit BwTimetable Bandwidth limit in kBytes/s, or use suffix b|k|M|G or a full timetable.
|
|
\-\-bwlimit\-file BwTimetable Bandwidth limit per file in kBytes/s, or use suffix b|k|M|G or a full timetable.
|
|
\-\-ca\-cert string CA certificate used to verify servers
|
|
\-\-cache\-dir string Directory rclone will use for caching. (default \[dq]$HOME/.cache/rclone\[dq])
|
|
\-\-check\-first Do all the checks before starting transfers.
|
|
\-\-checkers int Number of checkers to run in parallel. (default 8)
|
|
\-c, \-\-checksum Skip based on checksum (if available) & size, not mod\-time & size
|
|
\-\-client\-cert string Client SSL certificate (PEM) for mutual TLS auth
|
|
\-\-client\-key string Client SSL private key (PEM) for mutual TLS auth
|
|
\-\-compare\-dest stringArray Include additional comma separated server\-side paths during comparison.
|
|
\-\-config string Config file. (default \[dq]$HOME/.config/rclone/rclone.conf\[dq])
|
|
\-\-contimeout duration Connect timeout (default 1m0s)
|
|
\-\-copy\-dest stringArray Implies \-\-compare\-dest but also copies files from paths into destination.
|
|
\-\-cpuprofile string Write cpu profile to file
|
|
\-\-cutoff\-mode string Mode to stop transfers when reaching the max transfer limit HARD|SOFT|CAUTIOUS (default \[dq]HARD\[dq])
|
|
\-\-delete\-after When synchronizing, delete files on destination after transferring (default)
|
|
\-\-delete\-before When synchronizing, delete files on destination before transferring
|
|
\-\-delete\-during When synchronizing, delete files during transfer
|
|
\-\-delete\-excluded Delete files on dest excluded from sync
|
|
\-\-disable string Disable a comma separated list of features. Use help to see a list.
|
|
\-n, \-\-dry\-run Do a trial run with no permanent changes
|
|
\-\-dscp string Set DSCP value to connections. Can be value or names, eg. CS1, LE, DF, AF21.
|
|
\-\-dump DumpFlags List of items to dump from: headers,bodies,requests,responses,auth,filters,goroutines,openfiles
|
|
\-\-dump\-bodies Dump HTTP headers and bodies \- may contain sensitive info
|
|
\-\-dump\-headers Dump HTTP headers \- may contain sensitive info
|
|
\-\-error\-on\-no\-transfer Sets exit code 9 if no files are transferred, useful in scripts
|
|
\-\-exclude stringArray Exclude files matching pattern
|
|
\-\-exclude\-from stringArray Read exclude patterns from file (use \- to read from stdin)
|
|
\-\-exclude\-if\-present string Exclude directories if filename is present
|
|
\-\-expect\-continue\-timeout duration Timeout when using expect / 100\-continue in HTTP (default 1s)
|
|
\-\-fast\-list Use recursive list if available. Uses more memory but fewer transactions.
|
|
\-\-files\-from stringArray Read list of source\-file names from file (use \- to read from stdin)
|
|
\-\-files\-from\-raw stringArray Read list of source\-file names from file without any processing of lines (use \- to read from stdin)
|
|
\-f, \-\-filter stringArray Add a file\-filtering rule
|
|
\-\-filter\-from stringArray Read filtering patterns from a file (use \- to read from stdin)
|
|
\-\-fs\-cache\-expire\-duration duration cache remotes for this long (0 to disable caching) (default 5m0s)
|
|
\-\-fs\-cache\-expire\-interval duration interval to check for expired remotes (default 1m0s)
|
|
\-\-header stringArray Set HTTP header for all transactions
|
|
\-\-header\-download stringArray Set HTTP header for download transactions
|
|
\-\-header\-upload stringArray Set HTTP header for upload transactions
|
|
\-\-ignore\-case Ignore case in filters (case insensitive)
|
|
\-\-ignore\-case\-sync Ignore case when synchronizing
|
|
\-\-ignore\-checksum Skip post copy check of checksums.
|
|
\-\-ignore\-errors delete even if there are I/O errors
|
|
\-\-ignore\-existing Skip all files that exist on destination
|
|
\-\-ignore\-size Ignore size when skipping use mod\-time or checksum.
|
|
\-I, \-\-ignore\-times Don\[aq]t skip files that match size and time \- transfer all files
|
|
\-\-immutable Do not modify files. Fail if existing files have been modified.
|
|
\-\-include stringArray Include files matching pattern
|
|
\-\-include\-from stringArray Read include patterns from file (use \- to read from stdin)
|
|
\-i, \-\-interactive Enable interactive mode
|
|
\-\-log\-file string Log everything to this file
|
|
\-\-log\-format string Comma separated list of log format options (default \[dq]date,time\[dq])
|
|
\-\-log\-level string Log level DEBUG|INFO|NOTICE|ERROR (default \[dq]NOTICE\[dq])
|
|
\-\-log\-systemd Activate systemd integration for the logger.
|
|
\-\-low\-level\-retries int Number of low level retries to do. (default 10)
|
|
\-\-max\-age Duration Only transfer files younger than this in s or suffix ms|s|m|h|d|w|M|y (default off)
|
|
\-\-max\-backlog int Maximum number of objects in sync or check backlog. (default 10000)
|
|
\-\-max\-delete int When synchronizing, limit the number of deletes (default \-1)
|
|
\-\-max\-depth int If set limits the recursion depth to this. (default \-1)
|
|
\-\-max\-duration duration Maximum duration rclone will transfer data for.
|
|
\-\-max\-size SizeSuffix Only transfer files smaller than this in k or suffix b|k|M|G (default off)
|
|
\-\-max\-stats\-groups int Maximum number of stats groups to keep in memory. On max oldest is discarded. (default 1000)
|
|
\-\-max\-transfer SizeSuffix Maximum size of data to transfer. (default off)
|
|
\-\-memprofile string Write memory profile to file
|
|
\-\-min\-age Duration Only transfer files older than this in s or suffix ms|s|m|h|d|w|M|y (default off)
|
|
\-\-min\-size SizeSuffix Only transfer files bigger than this in k or suffix b|k|M|G (default off)
|
|
\-\-modify\-window duration Max time diff to be considered the same (default 1ns)
|
|
\-\-multi\-thread\-cutoff SizeSuffix Use multi\-thread downloads for files above this size. (default 250M)
|
|
\-\-multi\-thread\-streams int Max number of streams to use for multi\-thread downloads. (default 4)
|
|
\-\-no\-check\-certificate Do not verify the server SSL certificate. Insecure.
|
|
\-\-no\-check\-dest Don\[aq]t check the destination, copy regardless.
|
|
\-\-no\-console Hide console window. Supported on Windows only.
|
|
\-\-no\-gzip\-encoding Don\[aq]t set Accept\-Encoding: gzip.
|
|
\-\-no\-traverse Don\[aq]t traverse destination file system on copy.
|
|
\-\-no\-unicode\-normalization Don\[aq]t normalize unicode characters in filenames.
|
|
\-\-no\-update\-modtime Don\[aq]t update destination mod\-time if files identical.
|
|
\-\-order\-by string Instructions on how to order the transfers, e.g. \[aq]size,descending\[aq]
|
|
\-\-password\-command SpaceSepList Command for supplying password for encrypted configuration.
|
|
\-P, \-\-progress Show progress during transfer.
|
|
\-\-progress\-terminal\-title Show progress on the terminal title. Requires \-P/\-\-progress.
|
|
\-q, \-\-quiet Print as little stuff as possible
|
|
\-\-rc Enable the remote control server.
|
|
\-\-rc\-addr string IPaddress:Port or :Port to bind server to. (default \[dq]localhost:5572\[dq])
|
|
\-\-rc\-allow\-origin string Set the allowed origin for CORS.
|
|
\-\-rc\-baseurl string Prefix for URLs \- leave blank for root.
|
|
\-\-rc\-cert string SSL PEM key (concatenation of certificate and CA certificate)
|
|
\-\-rc\-client\-ca string Client certificate authority to verify clients with
|
|
\-\-rc\-enable\-metrics Enable prometheus metrics on /metrics
|
|
\-\-rc\-files string Path to local files to serve on the HTTP server.
|
|
\-\-rc\-htpasswd string htpasswd file \- if not provided no authentication is done
|
|
\-\-rc\-job\-expire\-duration duration expire finished async jobs older than this value (default 1m0s)
|
|
\-\-rc\-job\-expire\-interval duration interval to check for expired async jobs (default 10s)
|
|
\-\-rc\-key string SSL PEM Private key
|
|
\-\-rc\-max\-header\-bytes int Maximum size of request header (default 4096)
|
|
\-\-rc\-no\-auth Don\[aq]t require auth for certain methods.
|
|
\-\-rc\-pass string Password for authentication.
|
|
\-\-rc\-realm string realm for authentication (default \[dq]rclone\[dq])
|
|
\-\-rc\-serve Enable the serving of remote objects.
|
|
\-\-rc\-server\-read\-timeout duration Timeout for server reading data (default 1h0m0s)
|
|
\-\-rc\-server\-write\-timeout duration Timeout for server writing data (default 1h0m0s)
|
|
\-\-rc\-template string User Specified Template.
|
|
\-\-rc\-user string User name for authentication.
|
|
\-\-rc\-web\-fetch\-url string URL to fetch the releases for webgui. (default \[dq]https://api.github.com/repos/rclone/rclone\-webui\-react/releases/latest\[dq])
|
|
\-\-rc\-web\-gui Launch WebGUI on localhost
|
|
\-\-rc\-web\-gui\-force\-update Force update to latest version of web gui
|
|
\-\-rc\-web\-gui\-no\-open\-browser Don\[aq]t open the browser automatically
|
|
\-\-rc\-web\-gui\-update Check and update to latest version of web gui
|
|
\-\-refresh\-times Refresh the modtime of remote files.
|
|
\-\-retries int Retry operations this many times if they fail (default 3)
|
|
\-\-retries\-sleep duration Interval between retrying operations if they fail, e.g 500ms, 60s, 5m. (0 to disable)
|
|
\-\-size\-only Skip based on size only, not mod\-time or checksum
|
|
\-\-stats duration Interval between printing stats, e.g 500ms, 60s, 5m. (0 to disable) (default 1m0s)
|
|
\-\-stats\-file\-name\-length int Max file name length in stats. 0 for no limit (default 45)
|
|
\-\-stats\-log\-level string Log level to show \-\-stats output DEBUG|INFO|NOTICE|ERROR (default \[dq]INFO\[dq])
|
|
\-\-stats\-one\-line Make the stats fit on one line.
|
|
\-\-stats\-one\-line\-date Enables \-\-stats\-one\-line and add current date/time prefix.
|
|
\-\-stats\-one\-line\-date\-format string Enables \-\-stats\-one\-line\-date and uses custom formatted date. Enclose date string in double quotes (\[dq]). See https://golang.org/pkg/time/#Time.Format
|
|
\-\-stats\-unit string Show data rate in stats as either \[aq]bits\[aq] or \[aq]bytes\[aq]/s (default \[dq]bytes\[dq])
|
|
\-\-streaming\-upload\-cutoff SizeSuffix Cutoff for switching to chunked upload if file size is unknown. Upload starts after reaching cutoff or when file ends. (default 100k)
|
|
\-\-suffix string Suffix to add to changed files.
|
|
\-\-suffix\-keep\-extension Preserve the extension when using \-\-suffix.
|
|
\-\-syslog Use Syslog for logging
|
|
\-\-syslog\-facility string Facility for syslog, e.g. KERN,USER,... (default \[dq]DAEMON\[dq])
|
|
\-\-timeout duration IO idle timeout (default 5m0s)
|
|
\-\-tpslimit float Limit HTTP transactions per second to this.
|
|
\-\-tpslimit\-burst int Max burst of transactions for \-\-tpslimit. (default 1)
|
|
\-\-track\-renames When synchronizing, track file renames and do a server\-side move if possible
|
|
\-\-track\-renames\-strategy string Strategies to use when synchronizing using track\-renames hash|modtime|leaf (default \[dq]hash\[dq])
|
|
\-\-transfers int Number of file transfers to run in parallel. (default 4)
|
|
\-u, \-\-update Skip files that are newer on the destination.
|
|
\-\-use\-cookies Enable session cookiejar.
|
|
\-\-use\-json\-log Use json log format.
|
|
\-\-use\-mmap Use mmap allocator (see docs).
|
|
\-\-use\-server\-modtime Use server modified time instead of object metadata
|
|
\-\-user\-agent string Set the user\-agent to a specified string. The default is rclone/ version (default \[dq]rclone/v1.55.0\[dq])
|
|
\-v, \-\-verbose count Print lots more stuff (repeat for more)
|
|
\f[R]
|
|
.fi
|
|
.SS Backend Flags
|
|
.PP
|
|
These flags are available for every command.
|
|
They control the backends and may be set in the config file.
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
\-\-acd\-auth\-url string Auth server URL.
|
|
\-\-acd\-client\-id string OAuth Client Id
|
|
\-\-acd\-client\-secret string OAuth Client Secret
|
|
\-\-acd\-encoding MultiEncoder This sets the encoding for the backend. (default Slash,InvalidUtf8,Dot)
|
|
\-\-acd\-templink\-threshold SizeSuffix Files >= this size will be downloaded via their tempLink. (default 9G)
|
|
\-\-acd\-token string OAuth Access Token as a JSON blob.
|
|
\-\-acd\-token\-url string Token server url.
|
|
\-\-acd\-upload\-wait\-per\-gb Duration Additional time per GB to wait after a failed complete upload to see if it appears. (default 3m0s)
|
|
\-\-alias\-remote string Remote or path to alias.
|
|
\-\-azureblob\-access\-tier string Access tier of blob: hot, cool or archive.
|
|
\-\-azureblob\-account string Storage Account Name (leave blank to use SAS URL or Emulator)
|
|
\-\-azureblob\-archive\-tier\-delete Delete archive tier blobs before overwriting.
|
|
\-\-azureblob\-chunk\-size SizeSuffix Upload chunk size (<= 100MB). (default 4M)
|
|
\-\-azureblob\-disable\-checksum Don\[aq]t store MD5 checksum with object metadata.
|
|
\-\-azureblob\-encoding MultiEncoder This sets the encoding for the backend. (default Slash,BackSlash,Del,Ctl,RightPeriod,InvalidUtf8)
|
|
\-\-azureblob\-endpoint string Endpoint for the service
|
|
\-\-azureblob\-key string Storage Account Key (leave blank to use SAS URL or Emulator)
|
|
\-\-azureblob\-list\-chunk int Size of blob list. (default 5000)
|
|
\-\-azureblob\-memory\-pool\-flush\-time Duration How often internal memory buffer pools will be flushed. (default 1m0s)
|
|
\-\-azureblob\-memory\-pool\-use\-mmap Whether to use mmap buffers in internal memory pool.
|
|
\-\-azureblob\-msi\-client\-id string Object ID of the user\-assigned MSI to use, if any. Leave blank if msi_object_id or msi_mi_res_id specified.
|
|
\-\-azureblob\-msi\-mi\-res\-id string Azure resource ID of the user\-assigned MSI to use, if any. Leave blank if msi_client_id or msi_object_id specified.
|
|
\-\-azureblob\-msi\-object\-id string Object ID of the user\-assigned MSI to use, if any. Leave blank if msi_client_id or msi_mi_res_id specified.
|
|
\-\-azureblob\-public\-access string Public access level of a container: blob, container.
|
|
\-\-azureblob\-sas\-url string SAS URL for container level access only
|
|
\-\-azureblob\-service\-principal\-file string Path to file containing credentials for use with a service principal.
|
|
\-\-azureblob\-upload\-cutoff string Cutoff for switching to chunked upload (<= 256MB). (Deprecated)
|
|
\-\-azureblob\-use\-emulator Uses local storage emulator if provided as \[aq]true\[aq] (leave blank if using real azure storage endpoint)
|
|
\-\-azureblob\-use\-msi Use a managed service identity to authenticate (only works in Azure)
|
|
\-\-b2\-account string Account ID or Application Key ID
|
|
\-\-b2\-chunk\-size SizeSuffix Upload chunk size. Must fit in memory. (default 96M)
|
|
\-\-b2\-copy\-cutoff SizeSuffix Cutoff for switching to multipart copy (default 4G)
|
|
\-\-b2\-disable\-checksum Disable checksums for large (> upload cutoff) files
|
|
\-\-b2\-download\-auth\-duration Duration Time before the authorization token will expire in s or suffix ms|s|m|h|d. (default 1w)
|
|
\-\-b2\-download\-url string Custom endpoint for downloads.
|
|
\-\-b2\-encoding MultiEncoder This sets the encoding for the backend. (default Slash,BackSlash,Del,Ctl,InvalidUtf8,Dot)
|
|
\-\-b2\-endpoint string Endpoint for the service.
|
|
\-\-b2\-hard\-delete Permanently delete files on remote removal, otherwise hide files.
|
|
\-\-b2\-key string Application Key
|
|
\-\-b2\-memory\-pool\-flush\-time Duration How often internal memory buffer pools will be flushed. (default 1m0s)
|
|
\-\-b2\-memory\-pool\-use\-mmap Whether to use mmap buffers in internal memory pool.
|
|
\-\-b2\-test\-mode string A flag string for X\-Bz\-Test\-Mode header for debugging.
|
|
\-\-b2\-upload\-cutoff SizeSuffix Cutoff for switching to chunked upload. (default 200M)
|
|
\-\-b2\-versions Include old versions in directory listings.
|
|
\-\-box\-access\-token string Box App Primary Access Token
|
|
\-\-box\-auth\-url string Auth server URL.
|
|
\-\-box\-box\-config\-file string Box App config.json location
|
|
\-\-box\-box\-sub\-type string (default \[dq]user\[dq])
|
|
\-\-box\-client\-id string OAuth Client Id
|
|
\-\-box\-client\-secret string OAuth Client Secret
|
|
\-\-box\-commit\-retries int Max number of times to try committing a multipart file. (default 100)
|
|
\-\-box\-encoding MultiEncoder This sets the encoding for the backend. (default Slash,BackSlash,Del,Ctl,RightSpace,InvalidUtf8,Dot)
|
|
\-\-box\-root\-folder\-id string Fill in for rclone to use a non root folder as its starting point.
|
|
\-\-box\-token string OAuth Access Token as a JSON blob.
|
|
\-\-box\-token\-url string Token server url.
|
|
\-\-box\-upload\-cutoff SizeSuffix Cutoff for switching to multipart upload (>= 50MB). (default 50M)
|
|
\-\-cache\-chunk\-clean\-interval Duration How often should the cache perform cleanups of the chunk storage. (default 1m0s)
|
|
\-\-cache\-chunk\-no\-memory Disable the in\-memory cache for storing chunks during streaming.
|
|
\-\-cache\-chunk\-path string Directory to cache chunk files. (default \[dq]$HOME/.cache/rclone/cache\-backend\[dq])
|
|
\-\-cache\-chunk\-size SizeSuffix The size of a chunk (partial file data). (default 5M)
|
|
\-\-cache\-chunk\-total\-size SizeSuffix The total size that the chunks can take up on the local disk. (default 10G)
|
|
\-\-cache\-db\-path string Directory to store file structure metadata DB. (default \[dq]$HOME/.cache/rclone/cache\-backend\[dq])
|
|
\-\-cache\-db\-purge Clear all the cached data for this remote on start.
|
|
\-\-cache\-db\-wait\-time Duration How long to wait for the DB to be available \- 0 is unlimited (default 1s)
|
|
\-\-cache\-info\-age Duration How long to cache file structure information (directory listings, file size, times, etc.). (default 6h0m0s)
|
|
\-\-cache\-plex\-insecure string Skip all certificate verification when connecting to the Plex server
|
|
\-\-cache\-plex\-password string The password of the Plex user (obscured)
|
|
\-\-cache\-plex\-url string The URL of the Plex server
|
|
\-\-cache\-plex\-username string The username of the Plex user
|
|
\-\-cache\-read\-retries int How many times to retry a read from a cache storage. (default 10)
|
|
\-\-cache\-remote string Remote to cache.
|
|
\-\-cache\-rps int Limits the number of requests per second to the source FS (\-1 to disable) (default \-1)
|
|
\-\-cache\-tmp\-upload\-path string Directory to keep temporary files until they are uploaded.
|
|
\-\-cache\-tmp\-wait\-time Duration How long should files be stored in local cache before being uploaded (default 15s)
|
|
\-\-cache\-workers int How many workers should run in parallel to download chunks. (default 4)
|
|
\-\-cache\-writes Cache file data on writes through the FS
|
|
\-\-chunker\-chunk\-size SizeSuffix Files larger than chunk size will be split in chunks. (default 2G)
|
|
\-\-chunker\-fail\-hard Choose how chunker should handle files with missing or invalid chunks.
|
|
\-\-chunker\-hash\-type string Choose how chunker handles hash sums. All modes but \[dq]none\[dq] require metadata. (default \[dq]md5\[dq])
|
|
\-\-chunker\-remote string Remote to chunk/unchunk.
|
|
\-\-compress\-level int GZIP compression level (\-2 to 9). (default \-1)
|
|
\-\-compress\-mode string Compression mode. (default \[dq]gzip\[dq])
|
|
\-\-compress\-ram\-cache\-limit SizeSuffix Some remotes don\[aq]t allow the upload of files with unknown size. (default 20M)
|
|
\-\-compress\-remote string Remote to compress.
|
|
\-L, \-\-copy\-links Follow symlinks and copy the pointed to item.
|
|
\-\-crypt\-directory\-name\-encryption Option to either encrypt directory names or leave them intact. (default true)
|
|
\-\-crypt\-filename\-encryption string How to encrypt the filenames. (default \[dq]standard\[dq])
|
|
\-\-crypt\-no\-data\-encryption Option to either encrypt file data or leave it unencrypted.
|
|
\-\-crypt\-password string Password or pass phrase for encryption. (obscured)
|
|
\-\-crypt\-password2 string Password or pass phrase for salt. Optional but recommended. (obscured)
|
|
\-\-crypt\-remote string Remote to encrypt/decrypt.
|
|
\-\-crypt\-server\-side\-across\-configs Allow server\-side operations (e.g. copy) to work across different crypt configs.
|
|
\-\-crypt\-show\-mapping For all files listed show how the names encrypt.
|
|
\-\-drive\-acknowledge\-abuse Set to allow files which return cannotDownloadAbusiveFile to be downloaded.
|
|
\-\-drive\-allow\-import\-name\-change Allow the filetype to change when uploading Google docs (e.g. file.doc to file.docx). This will confuse sync and reupload every time.
|
|
\-\-drive\-auth\-owner\-only Only consider files owned by the authenticated user.
|
|
\-\-drive\-auth\-url string Auth server URL.
|
|
\-\-drive\-chunk\-size SizeSuffix Upload chunk size. Must a power of 2 >= 256k. (default 8M)
|
|
\-\-drive\-client\-id string Google Application Client Id
|
|
\-\-drive\-client\-secret string OAuth Client Secret
|
|
\-\-drive\-disable\-http2 Disable drive using http2 (default true)
|
|
\-\-drive\-encoding MultiEncoder This sets the encoding for the backend. (default InvalidUtf8)
|
|
\-\-drive\-export\-formats string Comma separated list of preferred formats for downloading Google docs. (default \[dq]docx,xlsx,pptx,svg\[dq])
|
|
\-\-drive\-formats string Deprecated: see export_formats
|
|
\-\-drive\-impersonate string Impersonate this user when using a service account.
|
|
\-\-drive\-import\-formats string Comma separated list of preferred formats for uploading Google docs.
|
|
\-\-drive\-keep\-revision\-forever Keep new head revision of each file forever.
|
|
\-\-drive\-list\-chunk int Size of listing chunk 100\-1000. 0 to disable. (default 1000)
|
|
\-\-drive\-pacer\-burst int Number of API calls to allow without sleeping. (default 100)
|
|
\-\-drive\-pacer\-min\-sleep Duration Minimum time to sleep between API calls. (default 100ms)
|
|
\-\-drive\-root\-folder\-id string ID of the root folder
|
|
\-\-drive\-scope string Scope that rclone should use when requesting access from drive.
|
|
\-\-drive\-server\-side\-across\-configs Allow server\-side operations (e.g. copy) to work across different drive configs.
|
|
\-\-drive\-service\-account\-credentials string Service Account Credentials JSON blob
|
|
\-\-drive\-service\-account\-file string Service Account Credentials JSON file path
|
|
\-\-drive\-shared\-with\-me Only show files that are shared with me.
|
|
\-\-drive\-size\-as\-quota Show sizes as storage quota usage, not actual size.
|
|
\-\-drive\-skip\-checksum\-gphotos Skip MD5 checksum on Google photos and videos only.
|
|
\-\-drive\-skip\-gdocs Skip google documents in all listings.
|
|
\-\-drive\-skip\-shortcuts If set skip shortcut files
|
|
\-\-drive\-starred\-only Only show files that are starred.
|
|
\-\-drive\-stop\-on\-download\-limit Make download limit errors be fatal
|
|
\-\-drive\-stop\-on\-upload\-limit Make upload limit errors be fatal
|
|
\-\-drive\-team\-drive string ID of the Shared Drive (Team Drive)
|
|
\-\-drive\-token string OAuth Access Token as a JSON blob.
|
|
\-\-drive\-token\-url string Token server url.
|
|
\-\-drive\-trashed\-only Only show files that are in the trash.
|
|
\-\-drive\-upload\-cutoff SizeSuffix Cutoff for switching to chunked upload (default 8M)
|
|
\-\-drive\-use\-created\-date Use file created date instead of modified date.,
|
|
\-\-drive\-use\-shared\-date Use date file was shared instead of modified date.
|
|
\-\-drive\-use\-trash Send files to the trash instead of deleting permanently. (default true)
|
|
\-\-drive\-v2\-download\-min\-size SizeSuffix If Object\[aq]s are greater, use drive v2 API to download. (default off)
|
|
\-\-dropbox\-auth\-url string Auth server URL.
|
|
\-\-dropbox\-chunk\-size SizeSuffix Upload chunk size. (< 150M). (default 48M)
|
|
\-\-dropbox\-client\-id string OAuth Client Id
|
|
\-\-dropbox\-client\-secret string OAuth Client Secret
|
|
\-\-dropbox\-encoding MultiEncoder This sets the encoding for the backend. (default Slash,BackSlash,Del,RightSpace,InvalidUtf8,Dot)
|
|
\-\-dropbox\-impersonate string Impersonate this user when using a business account.
|
|
\-\-dropbox\-shared\-files Instructs rclone to work on individual shared files.
|
|
\-\-dropbox\-shared\-folders Instructs rclone to work on shared folders.
|
|
\-\-dropbox\-token string OAuth Access Token as a JSON blob.
|
|
\-\-dropbox\-token\-url string Token server url.
|
|
\-\-fichier\-api\-key string Your API Key, get it from https://1fichier.com/console/params.pl
|
|
\-\-fichier\-encoding MultiEncoder This sets the encoding for the backend. (default Slash,LtGt,DoubleQuote,SingleQuote,BackQuote,Dollar,BackSlash,Del,Ctl,LeftSpace,RightSpace,InvalidUtf8,Dot)
|
|
\-\-fichier\-shared\-folder string If you want to download a shared folder, add this parameter
|
|
\-\-filefabric\-encoding MultiEncoder This sets the encoding for the backend. (default Slash,Del,Ctl,InvalidUtf8,Dot)
|
|
\-\-filefabric\-permanent\-token string Permanent Authentication Token
|
|
\-\-filefabric\-root\-folder\-id string ID of the root folder
|
|
\-\-filefabric\-token string Session Token
|
|
\-\-filefabric\-token\-expiry string Token expiry time
|
|
\-\-filefabric\-url string URL of the Enterprise File Fabric to connect to
|
|
\-\-filefabric\-version string Version read from the file fabric
|
|
\-\-ftp\-close\-timeout Duration Maximum time to wait for a response to close. (default 1m0s)
|
|
\-\-ftp\-concurrency int Maximum number of FTP simultaneous connections, 0 for unlimited
|
|
\-\-ftp\-disable\-epsv Disable using EPSV even if server advertises support
|
|
\-\-ftp\-disable\-mlsd Disable using MLSD even if server advertises support
|
|
\-\-ftp\-encoding MultiEncoder This sets the encoding for the backend. (default Slash,Del,Ctl,RightSpace,Dot)
|
|
\-\-ftp\-explicit\-tls Use Explicit FTPS (FTP over TLS)
|
|
\-\-ftp\-host string FTP host to connect to
|
|
\-\-ftp\-idle\-timeout Duration Max time before closing idle connections (default 1m0s)
|
|
\-\-ftp\-no\-check\-certificate Do not verify the TLS certificate of the server
|
|
\-\-ftp\-pass string FTP password (obscured)
|
|
\-\-ftp\-port string FTP port, leave blank to use default (21)
|
|
\-\-ftp\-tls Use Implicit FTPS (FTP over TLS)
|
|
\-\-ftp\-user string FTP username, leave blank for current username, $USER
|
|
\-\-gcs\-anonymous Access public buckets and objects without credentials
|
|
\-\-gcs\-auth\-url string Auth server URL.
|
|
\-\-gcs\-bucket\-acl string Access Control List for new buckets.
|
|
\-\-gcs\-bucket\-policy\-only Access checks should use bucket\-level IAM policies.
|
|
\-\-gcs\-client\-id string OAuth Client Id
|
|
\-\-gcs\-client\-secret string OAuth Client Secret
|
|
\-\-gcs\-encoding MultiEncoder This sets the encoding for the backend. (default Slash,CrLf,InvalidUtf8,Dot)
|
|
\-\-gcs\-location string Location for the newly created buckets.
|
|
\-\-gcs\-object\-acl string Access Control List for new objects.
|
|
\-\-gcs\-project\-number string Project number.
|
|
\-\-gcs\-service\-account\-file string Service Account Credentials JSON file path
|
|
\-\-gcs\-storage\-class string The storage class to use when storing objects in Google Cloud Storage.
|
|
\-\-gcs\-token string OAuth Access Token as a JSON blob.
|
|
\-\-gcs\-token\-url string Token server url.
|
|
\-\-gphotos\-auth\-url string Auth server URL.
|
|
\-\-gphotos\-client\-id string OAuth Client Id
|
|
\-\-gphotos\-client\-secret string OAuth Client Secret
|
|
\-\-gphotos\-include\-archived Also view and download archived media.
|
|
\-\-gphotos\-read\-only Set to make the Google Photos backend read only.
|
|
\-\-gphotos\-read\-size Set to read the size of media items.
|
|
\-\-gphotos\-start\-year int Year limits the photos to be downloaded to those which are uploaded after the given year (default 2000)
|
|
\-\-gphotos\-token string OAuth Access Token as a JSON blob.
|
|
\-\-gphotos\-token\-url string Token server url.
|
|
\-\-hdfs\-data\-transfer\-protection string Kerberos data transfer protection: authentication|integrity|privacy
|
|
\-\-hdfs\-encoding MultiEncoder This sets the encoding for the backend. (default Slash,Colon,Del,Ctl,InvalidUtf8,Dot)
|
|
\-\-hdfs\-namenode string hadoop name node and port
|
|
\-\-hdfs\-service\-principal\-name string Kerberos service principal name for the namenode
|
|
\-\-hdfs\-username string hadoop user name
|
|
\-\-http\-headers CommaSepList Set HTTP headers for all transactions
|
|
\-\-http\-no\-head Don\[aq]t use HEAD requests to find file sizes in dir listing
|
|
\-\-http\-no\-slash Set this if the site doesn\[aq]t end directories with /
|
|
\-\-http\-url string URL of http host to connect to
|
|
\-\-hubic\-auth\-url string Auth server URL.
|
|
\-\-hubic\-chunk\-size SizeSuffix Above this size files will be chunked into a _segments container. (default 5G)
|
|
\-\-hubic\-client\-id string OAuth Client Id
|
|
\-\-hubic\-client\-secret string OAuth Client Secret
|
|
\-\-hubic\-encoding MultiEncoder This sets the encoding for the backend. (default Slash,InvalidUtf8)
|
|
\-\-hubic\-no\-chunk Don\[aq]t chunk files during streaming upload.
|
|
\-\-hubic\-token string OAuth Access Token as a JSON blob.
|
|
\-\-hubic\-token\-url string Token server url.
|
|
\-\-jottacloud\-encoding MultiEncoder This sets the encoding for the backend. (default Slash,LtGt,DoubleQuote,Colon,Question,Asterisk,Pipe,Del,Ctl,InvalidUtf8,Dot)
|
|
\-\-jottacloud\-hard\-delete Delete files permanently rather than putting them into the trash.
|
|
\-\-jottacloud\-md5\-memory\-limit SizeSuffix Files bigger than this will be cached on disk to calculate the MD5 if required. (default 10M)
|
|
\-\-jottacloud\-trashed\-only Only show files that are in the trash.
|
|
\-\-jottacloud\-upload\-resume\-limit SizeSuffix Files bigger than this can be resumed if the upload fail\[aq]s. (default 10M)
|
|
\-\-koofr\-encoding MultiEncoder This sets the encoding for the backend. (default Slash,BackSlash,Del,Ctl,InvalidUtf8,Dot)
|
|
\-\-koofr\-endpoint string The Koofr API endpoint to use (default \[dq]https://app.koofr.net\[dq])
|
|
\-\-koofr\-mountid string Mount ID of the mount to use. If omitted, the primary mount is used.
|
|
\-\-koofr\-password string Your Koofr password for rclone (generate one at https://app.koofr.net/app/admin/preferences/password) (obscured)
|
|
\-\-koofr\-setmtime Does the backend support setting modification time. Set this to false if you use a mount ID that points to a Dropbox or Amazon Drive backend. (default true)
|
|
\-\-koofr\-user string Your Koofr user name
|
|
\-l, \-\-links Translate symlinks to/from regular files with a \[aq].rclonelink\[aq] extension
|
|
\-\-local\-case\-insensitive Force the filesystem to report itself as case insensitive
|
|
\-\-local\-case\-sensitive Force the filesystem to report itself as case sensitive.
|
|
\-\-local\-encoding MultiEncoder This sets the encoding for the backend. (default Slash,Dot)
|
|
\-\-local\-no\-check\-updated Don\[aq]t check to see if the files change during upload
|
|
\-\-local\-no\-preallocate Disable preallocation of disk space for transferred files
|
|
\-\-local\-no\-set\-modtime Disable setting modtime
|
|
\-\-local\-no\-sparse Disable sparse files for multi\-thread downloads
|
|
\-\-local\-no\-unicode\-normalization Don\[aq]t apply unicode normalization to paths and filenames (Deprecated)
|
|
\-\-local\-nounc string Disable UNC (long path names) conversion on Windows
|
|
\-\-local\-zero\-size\-links Assume the Stat size of links is zero (and read them instead)
|
|
\-\-mailru\-check\-hash What should copy do if file checksum is mismatched or invalid (default true)
|
|
\-\-mailru\-encoding MultiEncoder This sets the encoding for the backend. (default Slash,LtGt,DoubleQuote,Colon,Question,Asterisk,Pipe,BackSlash,Del,Ctl,InvalidUtf8,Dot)
|
|
\-\-mailru\-pass string Password (obscured)
|
|
\-\-mailru\-speedup\-enable Skip full upload if there is another file with same data hash. (default true)
|
|
\-\-mailru\-speedup\-file\-patterns string Comma separated list of file name patterns eligible for speedup (put by hash). (default \[dq]*.mkv,*.avi,*.mp4,*.mp3,*.zip,*.gz,*.rar,*.pdf\[dq])
|
|
\-\-mailru\-speedup\-max\-disk SizeSuffix This option allows you to disable speedup (put by hash) for large files (default 3G)
|
|
\-\-mailru\-speedup\-max\-memory SizeSuffix Files larger than the size given below will always be hashed on disk. (default 32M)
|
|
\-\-mailru\-user string User name (usually email)
|
|
\-\-mega\-debug Output more debug from Mega.
|
|
\-\-mega\-encoding MultiEncoder This sets the encoding for the backend. (default Slash,InvalidUtf8,Dot)
|
|
\-\-mega\-hard\-delete Delete files permanently rather than putting them into the trash.
|
|
\-\-mega\-pass string Password. (obscured)
|
|
\-\-mega\-user string User name
|
|
\-x, \-\-one\-file\-system Don\[aq]t cross filesystem boundaries (unix/macOS only).
|
|
\-\-onedrive\-auth\-url string Auth server URL.
|
|
\-\-onedrive\-chunk\-size SizeSuffix Chunk size to upload files with \- must be multiple of 320k (327,680 bytes). (default 10M)
|
|
\-\-onedrive\-client\-id string OAuth Client Id
|
|
\-\-onedrive\-client\-secret string OAuth Client Secret
|
|
\-\-onedrive\-drive\-id string The ID of the drive to use
|
|
\-\-onedrive\-drive\-type string The type of the drive ( personal | business | documentLibrary )
|
|
\-\-onedrive\-encoding MultiEncoder This sets the encoding for the backend. (default Slash,LtGt,DoubleQuote,Colon,Question,Asterisk,Pipe,BackSlash,Del,Ctl,LeftSpace,LeftTilde,RightSpace,RightPeriod,InvalidUtf8,Dot)
|
|
\-\-onedrive\-expose\-onenote\-files Set to make OneNote files show up in directory listings.
|
|
\-\-onedrive\-link\-password string Set the password for links created by the link command.
|
|
\-\-onedrive\-link\-scope string Set the scope of the links created by the link command. (default \[dq]anonymous\[dq])
|
|
\-\-onedrive\-link\-type string Set the type of the links created by the link command. (default \[dq]view\[dq])
|
|
\-\-onedrive\-no\-versions Remove all versions on modifying operations
|
|
\-\-onedrive\-region string Choose national cloud region for OneDrive. (default \[dq]global\[dq])
|
|
\-\-onedrive\-server\-side\-across\-configs Allow server\-side operations (e.g. copy) to work across different onedrive configs.
|
|
\-\-onedrive\-token string OAuth Access Token as a JSON blob.
|
|
\-\-onedrive\-token\-url string Token server url.
|
|
\-\-opendrive\-chunk\-size SizeSuffix Files will be uploaded in chunks this size. (default 10M)
|
|
\-\-opendrive\-encoding MultiEncoder This sets the encoding for the backend. (default Slash,LtGt,DoubleQuote,Colon,Question,Asterisk,Pipe,BackSlash,LeftSpace,LeftCrLfHtVt,RightSpace,RightCrLfHtVt,InvalidUtf8,Dot)
|
|
\-\-opendrive\-password string Password. (obscured)
|
|
\-\-opendrive\-username string Username
|
|
\-\-pcloud\-auth\-url string Auth server URL.
|
|
\-\-pcloud\-client\-id string OAuth Client Id
|
|
\-\-pcloud\-client\-secret string OAuth Client Secret
|
|
\-\-pcloud\-encoding MultiEncoder This sets the encoding for the backend. (default Slash,BackSlash,Del,Ctl,InvalidUtf8,Dot)
|
|
\-\-pcloud\-hostname string Hostname to connect to. (default \[dq]api.pcloud.com\[dq])
|
|
\-\-pcloud\-root\-folder\-id string Fill in for rclone to use a non root folder as its starting point. (default \[dq]d0\[dq])
|
|
\-\-pcloud\-token string OAuth Access Token as a JSON blob.
|
|
\-\-pcloud\-token\-url string Token server url.
|
|
\-\-premiumizeme\-encoding MultiEncoder This sets the encoding for the backend. (default Slash,DoubleQuote,BackSlash,Del,Ctl,InvalidUtf8,Dot)
|
|
\-\-putio\-encoding MultiEncoder This sets the encoding for the backend. (default Slash,BackSlash,Del,Ctl,InvalidUtf8,Dot)
|
|
\-\-qingstor\-access\-key\-id string QingStor Access Key ID
|
|
\-\-qingstor\-chunk\-size SizeSuffix Chunk size to use for uploading. (default 4M)
|
|
\-\-qingstor\-connection\-retries int Number of connection retries. (default 3)
|
|
\-\-qingstor\-encoding MultiEncoder This sets the encoding for the backend. (default Slash,Ctl,InvalidUtf8)
|
|
\-\-qingstor\-endpoint string Enter an endpoint URL to connection QingStor API.
|
|
\-\-qingstor\-env\-auth Get QingStor credentials from runtime. Only applies if access_key_id and secret_access_key is blank.
|
|
\-\-qingstor\-secret\-access\-key string QingStor Secret Access Key (password)
|
|
\-\-qingstor\-upload\-concurrency int Concurrency for multipart uploads. (default 1)
|
|
\-\-qingstor\-upload\-cutoff SizeSuffix Cutoff for switching to chunked upload (default 200M)
|
|
\-\-qingstor\-zone string Zone to connect to.
|
|
\-\-s3\-access\-key\-id string AWS Access Key ID.
|
|
\-\-s3\-acl string Canned ACL used when creating buckets and storing or copying objects.
|
|
\-\-s3\-bucket\-acl string Canned ACL used when creating buckets.
|
|
\-\-s3\-chunk\-size SizeSuffix Chunk size to use for uploading. (default 5M)
|
|
\-\-s3\-copy\-cutoff SizeSuffix Cutoff for switching to multipart copy (default 4.656G)
|
|
\-\-s3\-disable\-checksum Don\[aq]t store MD5 checksum with object metadata
|
|
\-\-s3\-disable\-http2 Disable usage of http2 for S3 backends
|
|
\-\-s3\-encoding MultiEncoder This sets the encoding for the backend. (default Slash,InvalidUtf8,Dot)
|
|
\-\-s3\-endpoint string Endpoint for S3 API.
|
|
\-\-s3\-env\-auth Get AWS credentials from runtime (environment variables or EC2/ECS meta data if no env vars).
|
|
\-\-s3\-force\-path\-style If true use path style access if false use virtual hosted style. (default true)
|
|
\-\-s3\-leave\-parts\-on\-error If true avoid calling abort upload on a failure, leaving all successfully uploaded parts on S3 for manual recovery.
|
|
\-\-s3\-list\-chunk int Size of listing chunk (response list for each ListObject S3 request). (default 1000)
|
|
\-\-s3\-location\-constraint string Location constraint \- must be set to match the Region.
|
|
\-\-s3\-max\-upload\-parts int Maximum number of parts in a multipart upload. (default 10000)
|
|
\-\-s3\-memory\-pool\-flush\-time Duration How often internal memory buffer pools will be flushed. (default 1m0s)
|
|
\-\-s3\-memory\-pool\-use\-mmap Whether to use mmap buffers in internal memory pool.
|
|
\-\-s3\-no\-check\-bucket If set, don\[aq]t attempt to check the bucket exists or create it
|
|
\-\-s3\-no\-head If set, don\[aq]t HEAD uploaded objects to check integrity
|
|
\-\-s3\-profile string Profile to use in the shared credentials file
|
|
\-\-s3\-provider string Choose your S3 provider.
|
|
\-\-s3\-region string Region to connect to.
|
|
\-\-s3\-requester\-pays Enables requester pays option when interacting with S3 bucket.
|
|
\-\-s3\-secret\-access\-key string AWS Secret Access Key (password)
|
|
\-\-s3\-server\-side\-encryption string The server\-side encryption algorithm used when storing this object in S3.
|
|
\-\-s3\-session\-token string An AWS session token
|
|
\-\-s3\-shared\-credentials\-file string Path to the shared credentials file
|
|
\-\-s3\-sse\-customer\-algorithm string If using SSE\-C, the server\-side encryption algorithm used when storing this object in S3.
|
|
\-\-s3\-sse\-customer\-key string If using SSE\-C you must provide the secret encryption key used to encrypt/decrypt your data.
|
|
\-\-s3\-sse\-customer\-key\-md5 string If using SSE\-C you may provide the secret encryption key MD5 checksum (optional).
|
|
\-\-s3\-sse\-kms\-key\-id string If using KMS ID you must provide the ARN of Key.
|
|
\-\-s3\-storage\-class string The storage class to use when storing new objects in S3.
|
|
\-\-s3\-upload\-concurrency int Concurrency for multipart uploads. (default 4)
|
|
\-\-s3\-upload\-cutoff SizeSuffix Cutoff for switching to chunked upload (default 200M)
|
|
\-\-s3\-use\-accelerate\-endpoint If true use the AWS S3 accelerated endpoint.
|
|
\-\-s3\-v2\-auth If true use v2 authentication.
|
|
\-\-seafile\-2fa Two\-factor authentication (\[aq]true\[aq] if the account has 2FA enabled)
|
|
\-\-seafile\-create\-library Should rclone create a library if it doesn\[aq]t exist
|
|
\-\-seafile\-encoding MultiEncoder This sets the encoding for the backend. (default Slash,DoubleQuote,BackSlash,Ctl,InvalidUtf8)
|
|
\-\-seafile\-library string Name of the library. Leave blank to access all non\-encrypted libraries.
|
|
\-\-seafile\-library\-key string Library password (for encrypted libraries only). Leave blank if you pass it through the command line. (obscured)
|
|
\-\-seafile\-pass string Password (obscured)
|
|
\-\-seafile\-url string URL of seafile host to connect to
|
|
\-\-seafile\-user string User name (usually email address)
|
|
\-\-sftp\-ask\-password Allow asking for SFTP password when needed.
|
|
\-\-sftp\-disable\-concurrent\-reads If set don\[aq]t use concurrent reads
|
|
\-\-sftp\-disable\-hashcheck Disable the execution of SSH commands to determine if remote file hashing is available.
|
|
\-\-sftp\-host string SSH host to connect to
|
|
\-\-sftp\-idle\-timeout Duration Max time before closing idle connections (default 1m0s)
|
|
\-\-sftp\-key\-file string Path to PEM\-encoded private key file, leave blank or set key\-use\-agent to use ssh\-agent.
|
|
\-\-sftp\-key\-file\-pass string The passphrase to decrypt the PEM\-encoded private key file. (obscured)
|
|
\-\-sftp\-key\-pem string Raw PEM\-encoded private key, If specified, will override key_file parameter.
|
|
\-\-sftp\-key\-use\-agent When set forces the usage of the ssh\-agent.
|
|
\-\-sftp\-known\-hosts\-file string Optional path to known_hosts file.
|
|
\-\-sftp\-md5sum\-command string The command used to read md5 hashes. Leave blank for autodetect.
|
|
\-\-sftp\-pass string SSH password, leave blank to use ssh\-agent. (obscured)
|
|
\-\-sftp\-path\-override string Override path used by SSH connection.
|
|
\-\-sftp\-port string SSH port, leave blank to use default (22)
|
|
\-\-sftp\-pubkey\-file string Optional path to public key file.
|
|
\-\-sftp\-server\-command string Specifies the path or command to run a sftp server on the remote host.
|
|
\-\-sftp\-set\-modtime Set the modified time on the remote if set. (default true)
|
|
\-\-sftp\-sha1sum\-command string The command used to read sha1 hashes. Leave blank for autodetect.
|
|
\-\-sftp\-skip\-links Set to skip any symlinks and any other non regular files.
|
|
\-\-sftp\-subsystem string Specifies the SSH2 subsystem on the remote host. (default \[dq]sftp\[dq])
|
|
\-\-sftp\-use\-fstat If set use fstat instead of stat
|
|
\-\-sftp\-use\-insecure\-cipher Enable the use of insecure ciphers and key exchange methods.
|
|
\-\-sftp\-user string SSH username, leave blank for current username, $USER
|
|
\-\-sharefile\-chunk\-size SizeSuffix Upload chunk size. Must a power of 2 >= 256k. (default 64M)
|
|
\-\-sharefile\-encoding MultiEncoder This sets the encoding for the backend. (default Slash,LtGt,DoubleQuote,Colon,Question,Asterisk,Pipe,BackSlash,Ctl,LeftSpace,LeftPeriod,RightSpace,RightPeriod,InvalidUtf8,Dot)
|
|
\-\-sharefile\-endpoint string Endpoint for API calls.
|
|
\-\-sharefile\-root\-folder\-id string ID of the root folder
|
|
\-\-sharefile\-upload\-cutoff SizeSuffix Cutoff for switching to multipart upload. (default 128M)
|
|
\-\-skip\-links Don\[aq]t warn about skipped symlinks.
|
|
\-\-sugarsync\-access\-key\-id string Sugarsync Access Key ID.
|
|
\-\-sugarsync\-app\-id string Sugarsync App ID.
|
|
\-\-sugarsync\-authorization string Sugarsync authorization
|
|
\-\-sugarsync\-authorization\-expiry string Sugarsync authorization expiry
|
|
\-\-sugarsync\-deleted\-id string Sugarsync deleted folder id
|
|
\-\-sugarsync\-encoding MultiEncoder This sets the encoding for the backend. (default Slash,Ctl,InvalidUtf8,Dot)
|
|
\-\-sugarsync\-hard\-delete Permanently delete files if true
|
|
\-\-sugarsync\-private\-access\-key string Sugarsync Private Access Key
|
|
\-\-sugarsync\-refresh\-token string Sugarsync refresh token
|
|
\-\-sugarsync\-root\-id string Sugarsync root id
|
|
\-\-sugarsync\-user string Sugarsync user
|
|
\-\-swift\-application\-credential\-id string Application Credential ID (OS_APPLICATION_CREDENTIAL_ID)
|
|
\-\-swift\-application\-credential\-name string Application Credential Name (OS_APPLICATION_CREDENTIAL_NAME)
|
|
\-\-swift\-application\-credential\-secret string Application Credential Secret (OS_APPLICATION_CREDENTIAL_SECRET)
|
|
\-\-swift\-auth string Authentication URL for server (OS_AUTH_URL).
|
|
\-\-swift\-auth\-token string Auth Token from alternate authentication \- optional (OS_AUTH_TOKEN)
|
|
\-\-swift\-auth\-version int AuthVersion \- optional \- set to (1,2,3) if your auth URL has no version (ST_AUTH_VERSION)
|
|
\-\-swift\-chunk\-size SizeSuffix Above this size files will be chunked into a _segments container. (default 5G)
|
|
\-\-swift\-domain string User domain \- optional (v3 auth) (OS_USER_DOMAIN_NAME)
|
|
\-\-swift\-encoding MultiEncoder This sets the encoding for the backend. (default Slash,InvalidUtf8)
|
|
\-\-swift\-endpoint\-type string Endpoint type to choose from the service catalogue (OS_ENDPOINT_TYPE) (default \[dq]public\[dq])
|
|
\-\-swift\-env\-auth Get swift credentials from environment variables in standard OpenStack form.
|
|
\-\-swift\-key string API key or password (OS_PASSWORD).
|
|
\-\-swift\-leave\-parts\-on\-error If true avoid calling abort upload on a failure. It should be set to true for resuming uploads across different sessions.
|
|
\-\-swift\-no\-chunk Don\[aq]t chunk files during streaming upload.
|
|
\-\-swift\-region string Region name \- optional (OS_REGION_NAME)
|
|
\-\-swift\-storage\-policy string The storage policy to use when creating a new container
|
|
\-\-swift\-storage\-url string Storage URL \- optional (OS_STORAGE_URL)
|
|
\-\-swift\-tenant string Tenant name \- optional for v1 auth, this or tenant_id required otherwise (OS_TENANT_NAME or OS_PROJECT_NAME)
|
|
\-\-swift\-tenant\-domain string Tenant domain \- optional (v3 auth) (OS_PROJECT_DOMAIN_NAME)
|
|
\-\-swift\-tenant\-id string Tenant ID \- optional for v1 auth, this or tenant required otherwise (OS_TENANT_ID)
|
|
\-\-swift\-user string User name to log in (OS_USERNAME).
|
|
\-\-swift\-user\-id string User ID to log in \- optional \- most swift systems use user and leave this blank (v3 auth) (OS_USER_ID).
|
|
\-\-tardigrade\-access\-grant string Access Grant.
|
|
\-\-tardigrade\-api\-key string API Key.
|
|
\-\-tardigrade\-passphrase string Encryption Passphrase. To access existing objects enter passphrase used for uploading.
|
|
\-\-tardigrade\-provider string Choose an authentication method. (default \[dq]existing\[dq])
|
|
\-\-tardigrade\-satellite\-address <nodeid>\[at]<address>:<port> Satellite Address. Custom satellite address should match the format: <nodeid>\[at]<address>:<port>. (default \[dq]us\-central\-1.tardigrade.io\[dq])
|
|
\-\-union\-action\-policy string Policy to choose upstream on ACTION category. (default \[dq]epall\[dq])
|
|
\-\-union\-cache\-time int Cache time of usage and free space (in seconds). This option is only useful when a path preserving policy is used. (default 120)
|
|
\-\-union\-create\-policy string Policy to choose upstream on CREATE category. (default \[dq]epmfs\[dq])
|
|
\-\-union\-search\-policy string Policy to choose upstream on SEARCH category. (default \[dq]ff\[dq])
|
|
\-\-union\-upstreams string List of space separated upstreams.
|
|
\-\-webdav\-bearer\-token string Bearer token instead of user/pass (e.g. a Macaroon)
|
|
\-\-webdav\-bearer\-token\-command string Command to run to get a bearer token
|
|
\-\-webdav\-encoding string This sets the encoding for the backend.
|
|
\-\-webdav\-pass string Password. (obscured)
|
|
\-\-webdav\-url string URL of http host to connect to
|
|
\-\-webdav\-user string User name. In case NTLM authentication is used, the username should be in the format \[aq]Domain\[rs]User\[aq].
|
|
\-\-webdav\-vendor string Name of the Webdav site/service/software you are using
|
|
\-\-yandex\-auth\-url string Auth server URL.
|
|
\-\-yandex\-client\-id string OAuth Client Id
|
|
\-\-yandex\-client\-secret string OAuth Client Secret
|
|
\-\-yandex\-encoding MultiEncoder This sets the encoding for the backend. (default Slash,Del,Ctl,InvalidUtf8,Dot)
|
|
\-\-yandex\-token string OAuth Access Token as a JSON blob.
|
|
\-\-yandex\-token\-url string Token server url.
|
|
\-\-zoho\-auth\-url string Auth server URL.
|
|
\-\-zoho\-client\-id string OAuth Client Id
|
|
\-\-zoho\-client\-secret string OAuth Client Secret
|
|
\-\-zoho\-encoding MultiEncoder This sets the encoding for the backend. (default Del,Ctl,InvalidUtf8)
|
|
\-\-zoho\-region string Zoho region to connect to. You\[aq]ll have to use the region you organization is registered in.
|
|
\-\-zoho\-token string OAuth Access Token as a JSON blob.
|
|
\-\-zoho\-token\-url string Token server url.
|
|
\f[R]
|
|
.fi
|
|
.SS 1Fichier
|
|
.PP
|
|
This is a backend for the 1fichier (https://1fichier.com) cloud storage
|
|
service.
|
|
Note that a Premium subscription is required to use the API.
|
|
.PP
|
|
Paths are specified as \f[C]remote:path\f[R]
|
|
.PP
|
|
Paths may be as deep as required, e.g.
|
|
\f[C]remote:directory/subdirectory\f[R].
|
|
.PP
|
|
The initial setup for 1Fichier involves getting the API key from the
|
|
website which you need to do in your browser.
|
|
.PP
|
|
Here is an example of how to make a remote called \f[C]remote\f[R].
|
|
First run:
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone config
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
This will guide you through an interactive setup process:
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
No remotes found \- make a new one
|
|
n) New remote
|
|
s) Set configuration password
|
|
q) Quit config
|
|
n/s/q> n
|
|
name> remote
|
|
Type of storage to configure.
|
|
Enter a string value. Press Enter for the default (\[dq]\[dq]).
|
|
Choose a number from below, or type in your own value
|
|
[snip]
|
|
XX / 1Fichier
|
|
\[rs] \[dq]fichier\[dq]
|
|
[snip]
|
|
Storage> fichier
|
|
** See help for fichier backend at: https://rclone.org/fichier/ **
|
|
|
|
Your API Key, get it from https://1fichier.com/console/params.pl
|
|
Enter a string value. Press Enter for the default (\[dq]\[dq]).
|
|
api_key> example_key
|
|
|
|
Edit advanced config? (y/n)
|
|
y) Yes
|
|
n) No
|
|
y/n>
|
|
Remote config
|
|
\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-
|
|
[remote]
|
|
type = fichier
|
|
api_key = example_key
|
|
\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-
|
|
y) Yes this is OK
|
|
e) Edit this remote
|
|
d) Delete this remote
|
|
y/e/d> y
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
Once configured you can then use \f[C]rclone\f[R] like this,
|
|
.PP
|
|
List directories in top level of your 1Fichier account
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone lsd remote:
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
List all the files in your 1Fichier account
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone ls remote:
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
To copy a local directory to a 1Fichier directory called backup
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone copy /home/source remote:backup
|
|
\f[R]
|
|
.fi
|
|
.SS Modified time and hashes
|
|
.PP
|
|
1Fichier does not support modification times.
|
|
It supports the Whirlpool hash algorithm.
|
|
.SS Duplicated files
|
|
.PP
|
|
1Fichier can have two files with exactly the same name and path (unlike
|
|
a normal file system).
|
|
.PP
|
|
Duplicated files cause problems with the syncing and you will see
|
|
messages in the log about duplicates.
|
|
.SS Restricted filename characters
|
|
.PP
|
|
In addition to the default restricted characters
|
|
set (https://rclone.org/overview/#restricted-characters) the following
|
|
characters are also replaced:
|
|
.PP
|
|
.TS
|
|
tab(@);
|
|
l c c.
|
|
T{
|
|
Character
|
|
T}@T{
|
|
Value
|
|
T}@T{
|
|
Replacement
|
|
T}
|
|
_
|
|
T{
|
|
\[rs]
|
|
T}@T{
|
|
0x5C
|
|
T}@T{
|
|
\[uFF3C]
|
|
T}
|
|
T{
|
|
<
|
|
T}@T{
|
|
0x3C
|
|
T}@T{
|
|
\[uFF1C]
|
|
T}
|
|
T{
|
|
>
|
|
T}@T{
|
|
0x3E
|
|
T}@T{
|
|
\[uFF1E]
|
|
T}
|
|
T{
|
|
\[dq]
|
|
T}@T{
|
|
0x22
|
|
T}@T{
|
|
\[uFF02]
|
|
T}
|
|
T{
|
|
$
|
|
T}@T{
|
|
0x24
|
|
T}@T{
|
|
\[uFF04]
|
|
T}
|
|
T{
|
|
\[ga]
|
|
T}@T{
|
|
0x60
|
|
T}@T{
|
|
\[uFF40]
|
|
T}
|
|
T{
|
|
\[aq]
|
|
T}@T{
|
|
0x27
|
|
T}@T{
|
|
\[uFF07]
|
|
T}
|
|
.TE
|
|
.PP
|
|
File names can also not start or end with the following characters.
|
|
These only get replaced if they are the first or last character in the
|
|
name:
|
|
.PP
|
|
.TS
|
|
tab(@);
|
|
l c c.
|
|
T{
|
|
Character
|
|
T}@T{
|
|
Value
|
|
T}@T{
|
|
Replacement
|
|
T}
|
|
_
|
|
T{
|
|
SP
|
|
T}@T{
|
|
0x20
|
|
T}@T{
|
|
\[u2420]
|
|
T}
|
|
.TE
|
|
.PP
|
|
Invalid UTF\-8 bytes will also be
|
|
replaced (https://rclone.org/overview/#invalid-utf8), as they can\[aq]t
|
|
be used in JSON strings.
|
|
.SS Standard Options
|
|
.PP
|
|
Here are the standard options specific to fichier (1Fichier).
|
|
.SS \-\-fichier\-api\-key
|
|
.PP
|
|
Your API Key, get it from https://1fichier.com/console/params.pl
|
|
.IP \[bu] 2
|
|
Config: api_key
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_FICHIER_API_KEY
|
|
.IP \[bu] 2
|
|
Type: string
|
|
.IP \[bu] 2
|
|
Default: \[dq]\[dq]
|
|
.SS Advanced Options
|
|
.PP
|
|
Here are the advanced options specific to fichier (1Fichier).
|
|
.SS \-\-fichier\-shared\-folder
|
|
.PP
|
|
If you want to download a shared folder, add this parameter
|
|
.IP \[bu] 2
|
|
Config: shared_folder
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_FICHIER_SHARED_FOLDER
|
|
.IP \[bu] 2
|
|
Type: string
|
|
.IP \[bu] 2
|
|
Default: \[dq]\[dq]
|
|
.SS \-\-fichier\-encoding
|
|
.PP
|
|
This sets the encoding for the backend.
|
|
.PP
|
|
See: the encoding section in the
|
|
overview (https://rclone.org/overview/#encoding) for more info.
|
|
.IP \[bu] 2
|
|
Config: encoding
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_FICHIER_ENCODING
|
|
.IP \[bu] 2
|
|
Type: MultiEncoder
|
|
.IP \[bu] 2
|
|
Default:
|
|
Slash,LtGt,DoubleQuote,SingleQuote,BackQuote,Dollar,BackSlash,Del,Ctl,LeftSpace,RightSpace,InvalidUtf8,Dot
|
|
.SS Limitations
|
|
.PP
|
|
\f[C]rclone about\f[R] is not supported by the 1Fichier backend.
|
|
Backends without this capability cannot determine free space for an
|
|
rclone mount or use policy \f[C]mfs\f[R] (most free space) as a member
|
|
of an rclone union remote.
|
|
.PP
|
|
See List of backends that do not support rclone
|
|
about (https://rclone.org/overview/#optional-features) See rclone
|
|
about (https://rclone.org/commands/rclone_about/)
|
|
.SS Alias
|
|
.PP
|
|
The \f[C]alias\f[R] remote provides a new name for another remote.
|
|
.PP
|
|
Paths may be as deep as required or a local path, e.g.
|
|
\f[C]remote:directory/subdirectory\f[R] or
|
|
\f[C]/directory/subdirectory\f[R].
|
|
.PP
|
|
During the initial setup with \f[C]rclone config\f[R] you will specify
|
|
the target remote.
|
|
The target remote can either be a local path or another remote.
|
|
.PP
|
|
Subfolders can be used in target remote.
|
|
Assume an alias remote named \f[C]backup\f[R] with the target
|
|
\f[C]mydrive:private/backup\f[R].
|
|
Invoking \f[C]rclone mkdir backup:desktop\f[R] is exactly the same as
|
|
invoking \f[C]rclone mkdir mydrive:private/backup/desktop\f[R].
|
|
.PP
|
|
There will be no special handling of paths containing \f[C]..\f[R]
|
|
segments.
|
|
Invoking \f[C]rclone mkdir backup:../desktop\f[R] is exactly the same as
|
|
invoking \f[C]rclone mkdir mydrive:private/backup/../desktop\f[R].
|
|
The empty path is not allowed as a remote.
|
|
To alias the current directory use \f[C].\f[R] instead.
|
|
.PP
|
|
Here is an example of how to make an alias called \f[C]remote\f[R] for
|
|
local folder.
|
|
First run:
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone config
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
This will guide you through an interactive setup process:
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
No remotes found \- make a new one
|
|
n) New remote
|
|
s) Set configuration password
|
|
q) Quit config
|
|
n/s/q> n
|
|
name> remote
|
|
Type of storage to configure.
|
|
Choose a number from below, or type in your own value
|
|
[snip]
|
|
XX / Alias for an existing remote
|
|
\[rs] \[dq]alias\[dq]
|
|
[snip]
|
|
Storage> alias
|
|
Remote or path to alias.
|
|
Can be \[dq]myremote:path/to/dir\[dq], \[dq]myremote:bucket\[dq], \[dq]myremote:\[dq] or \[dq]/local/path\[dq].
|
|
remote> /mnt/storage/backup
|
|
Remote config
|
|
\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-
|
|
[remote]
|
|
remote = /mnt/storage/backup
|
|
\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-
|
|
y) Yes this is OK
|
|
e) Edit this remote
|
|
d) Delete this remote
|
|
y/e/d> y
|
|
Current remotes:
|
|
|
|
Name Type
|
|
==== ====
|
|
remote alias
|
|
|
|
e) Edit existing remote
|
|
n) New remote
|
|
d) Delete remote
|
|
r) Rename remote
|
|
c) Copy remote
|
|
s) Set configuration password
|
|
q) Quit config
|
|
e/n/d/r/c/s/q> q
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
Once configured you can then use \f[C]rclone\f[R] like this,
|
|
.PP
|
|
List directories in top level in \f[C]/mnt/storage/backup\f[R]
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone lsd remote:
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
List all the files in \f[C]/mnt/storage/backup\f[R]
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone ls remote:
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
Copy another local directory to the alias directory called source
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone copy /home/source remote:source
|
|
\f[R]
|
|
.fi
|
|
.SS Standard Options
|
|
.PP
|
|
Here are the standard options specific to alias (Alias for an existing
|
|
remote).
|
|
.SS \-\-alias\-remote
|
|
.PP
|
|
Remote or path to alias.
|
|
Can be \[dq]myremote:path/to/dir\[dq], \[dq]myremote:bucket\[dq],
|
|
\[dq]myremote:\[dq] or \[dq]/local/path\[dq].
|
|
.IP \[bu] 2
|
|
Config: remote
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_ALIAS_REMOTE
|
|
.IP \[bu] 2
|
|
Type: string
|
|
.IP \[bu] 2
|
|
Default: \[dq]\[dq]
|
|
.SS Amazon Drive
|
|
.PP
|
|
Amazon Drive, formerly known as Amazon Cloud Drive, is a cloud storage
|
|
service run by Amazon for consumers.
|
|
.SS Status
|
|
.PP
|
|
\f[B]Important:\f[R] rclone supports Amazon Drive only if you have your
|
|
own set of API keys.
|
|
Unfortunately the Amazon Drive developer
|
|
program (https://developer.amazon.com/amazon-drive) is now closed to new
|
|
entries so if you don\[aq]t already have your own set of keys you will
|
|
not be able to use rclone with Amazon Drive.
|
|
.PP
|
|
For the history on why rclone no longer has a set of Amazon Drive API
|
|
keys see the
|
|
forum (https://forum.rclone.org/t/rclone-has-been-banned-from-amazon-drive/2314).
|
|
.PP
|
|
If you happen to know anyone who works at Amazon then please ask them to
|
|
re\-instate rclone into the Amazon Drive developer program \- thanks!
|
|
.SS Setup
|
|
.PP
|
|
The initial setup for Amazon Drive involves getting a token from Amazon
|
|
which you need to do in your browser.
|
|
\f[C]rclone config\f[R] walks you through it.
|
|
.PP
|
|
The configuration process for Amazon Drive may involve using an oauth
|
|
proxy (https://github.com/ncw/oauthproxy).
|
|
This is used to keep the Amazon credentials out of the source code.
|
|
The proxy runs in Google\[aq]s very secure App Engine environment and
|
|
doesn\[aq]t store any credentials which pass through it.
|
|
.PP
|
|
Since rclone doesn\[aq]t currently have its own Amazon Drive credentials
|
|
so you will either need to have your own \f[C]client_id\f[R] and
|
|
\f[C]client_secret\f[R] with Amazon Drive, or use a third party oauth
|
|
proxy in which case you will need to enter \f[C]client_id\f[R],
|
|
\f[C]client_secret\f[R], \f[C]auth_url\f[R] and \f[C]token_url\f[R].
|
|
.PP
|
|
Note also if you are not using Amazon\[aq]s \f[C]auth_url\f[R] and
|
|
\f[C]token_url\f[R], (ie you filled in something for those) then if
|
|
setting up on a remote machine you can only use the copying the config
|
|
method of
|
|
configuration (https://rclone.org/remote_setup/#configuring-by-copying-the-config-file)
|
|
\- \f[C]rclone authorize\f[R] will not work.
|
|
.PP
|
|
Here is an example of how to make a remote called \f[C]remote\f[R].
|
|
First run:
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone config
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
This will guide you through an interactive setup process:
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
No remotes found \- make a new one
|
|
n) New remote
|
|
r) Rename remote
|
|
c) Copy remote
|
|
s) Set configuration password
|
|
q) Quit config
|
|
n/r/c/s/q> n
|
|
name> remote
|
|
Type of storage to configure.
|
|
Choose a number from below, or type in your own value
|
|
[snip]
|
|
XX / Amazon Drive
|
|
\[rs] \[dq]amazon cloud drive\[dq]
|
|
[snip]
|
|
Storage> amazon cloud drive
|
|
Amazon Application Client Id \- required.
|
|
client_id> your client ID goes here
|
|
Amazon Application Client Secret \- required.
|
|
client_secret> your client secret goes here
|
|
Auth server URL \- leave blank to use Amazon\[aq]s.
|
|
auth_url> Optional auth URL
|
|
Token server url \- leave blank to use Amazon\[aq]s.
|
|
token_url> Optional token URL
|
|
Remote config
|
|
Make sure your Redirect URL is set to \[dq]http://127.0.0.1:53682/\[dq] in your custom config.
|
|
Use auto config?
|
|
* Say Y if not sure
|
|
* Say N if you are working on a remote or headless machine
|
|
y) Yes
|
|
n) No
|
|
y/n> y
|
|
If your browser doesn\[aq]t open automatically go to the following link: http://127.0.0.1:53682/auth
|
|
Log in and authorize rclone for access
|
|
Waiting for code...
|
|
Got code
|
|
\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-
|
|
[remote]
|
|
client_id = your client ID goes here
|
|
client_secret = your client secret goes here
|
|
auth_url = Optional auth URL
|
|
token_url = Optional token URL
|
|
token = {\[dq]access_token\[dq]:\[dq]xxxxxxxxxxxxxxxxxxxxxxx\[dq],\[dq]token_type\[dq]:\[dq]bearer\[dq],\[dq]refresh_token\[dq]:\[dq]xxxxxxxxxxxxxxxxxx\[dq],\[dq]expiry\[dq]:\[dq]2015\-09\-06T16:07:39.658438471+01:00\[dq]}
|
|
\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-
|
|
y) Yes this is OK
|
|
e) Edit this remote
|
|
d) Delete this remote
|
|
y/e/d> y
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
See the remote setup docs (https://rclone.org/remote_setup/) for how to
|
|
set it up on a machine with no Internet browser available.
|
|
.PP
|
|
Note that rclone runs a webserver on your local machine to collect the
|
|
token as returned from Amazon.
|
|
This only runs from the moment it opens your browser to the moment you
|
|
get back the verification code.
|
|
This is on \f[C]http://127.0.0.1:53682/\f[R] and this it may require you
|
|
to unblock it temporarily if you are running a host firewall.
|
|
.PP
|
|
Once configured you can then use \f[C]rclone\f[R] like this,
|
|
.PP
|
|
List directories in top level of your Amazon Drive
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone lsd remote:
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
List all the files in your Amazon Drive
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone ls remote:
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
To copy a local directory to an Amazon Drive directory called backup
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone copy /home/source remote:backup
|
|
\f[R]
|
|
.fi
|
|
.SS Modified time and MD5SUMs
|
|
.PP
|
|
Amazon Drive doesn\[aq]t allow modification times to be changed via the
|
|
API so these won\[aq]t be accurate or used for syncing.
|
|
.PP
|
|
It does store MD5SUMs so for a more accurate sync, you can use the
|
|
\f[C]\-\-checksum\f[R] flag.
|
|
.SS Restricted filename characters
|
|
.PP
|
|
.TS
|
|
tab(@);
|
|
l c c.
|
|
T{
|
|
Character
|
|
T}@T{
|
|
Value
|
|
T}@T{
|
|
Replacement
|
|
T}
|
|
_
|
|
T{
|
|
NUL
|
|
T}@T{
|
|
0x00
|
|
T}@T{
|
|
\[u2400]
|
|
T}
|
|
T{
|
|
/
|
|
T}@T{
|
|
0x2F
|
|
T}@T{
|
|
\[uFF0F]
|
|
T}
|
|
.TE
|
|
.PP
|
|
Invalid UTF\-8 bytes will also be
|
|
replaced (https://rclone.org/overview/#invalid-utf8), as they can\[aq]t
|
|
be used in JSON strings.
|
|
.SS Deleting files
|
|
.PP
|
|
Any files you delete with rclone will end up in the trash.
|
|
Amazon don\[aq]t provide an API to permanently delete files, nor to
|
|
empty the trash, so you will have to do that with one of Amazon\[aq]s
|
|
apps or via the Amazon Drive website.
|
|
As of November 17, 2016, files are automatically deleted by Amazon from
|
|
the trash after 30 days.
|
|
.SS Using with non \f[C].com\f[R] Amazon accounts
|
|
.PP
|
|
Let\[aq]s say you usually use \f[C]amazon.co.uk\f[R].
|
|
When you authenticate with rclone it will take you to an
|
|
\f[C]amazon.com\f[R] page to log in.
|
|
Your \f[C]amazon.co.uk\f[R] email and password should work here just
|
|
fine.
|
|
.SS Standard Options
|
|
.PP
|
|
Here are the standard options specific to amazon cloud drive (Amazon
|
|
Drive).
|
|
.SS \-\-acd\-client\-id
|
|
.PP
|
|
OAuth Client Id Leave blank normally.
|
|
.IP \[bu] 2
|
|
Config: client_id
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_ACD_CLIENT_ID
|
|
.IP \[bu] 2
|
|
Type: string
|
|
.IP \[bu] 2
|
|
Default: \[dq]\[dq]
|
|
.SS \-\-acd\-client\-secret
|
|
.PP
|
|
OAuth Client Secret Leave blank normally.
|
|
.IP \[bu] 2
|
|
Config: client_secret
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_ACD_CLIENT_SECRET
|
|
.IP \[bu] 2
|
|
Type: string
|
|
.IP \[bu] 2
|
|
Default: \[dq]\[dq]
|
|
.SS Advanced Options
|
|
.PP
|
|
Here are the advanced options specific to amazon cloud drive (Amazon
|
|
Drive).
|
|
.SS \-\-acd\-token
|
|
.PP
|
|
OAuth Access Token as a JSON blob.
|
|
.IP \[bu] 2
|
|
Config: token
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_ACD_TOKEN
|
|
.IP \[bu] 2
|
|
Type: string
|
|
.IP \[bu] 2
|
|
Default: \[dq]\[dq]
|
|
.SS \-\-acd\-auth\-url
|
|
.PP
|
|
Auth server URL.
|
|
Leave blank to use the provider defaults.
|
|
.IP \[bu] 2
|
|
Config: auth_url
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_ACD_AUTH_URL
|
|
.IP \[bu] 2
|
|
Type: string
|
|
.IP \[bu] 2
|
|
Default: \[dq]\[dq]
|
|
.SS \-\-acd\-token\-url
|
|
.PP
|
|
Token server url.
|
|
Leave blank to use the provider defaults.
|
|
.IP \[bu] 2
|
|
Config: token_url
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_ACD_TOKEN_URL
|
|
.IP \[bu] 2
|
|
Type: string
|
|
.IP \[bu] 2
|
|
Default: \[dq]\[dq]
|
|
.SS \-\-acd\-checkpoint
|
|
.PP
|
|
Checkpoint for internal polling (debug).
|
|
.IP \[bu] 2
|
|
Config: checkpoint
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_ACD_CHECKPOINT
|
|
.IP \[bu] 2
|
|
Type: string
|
|
.IP \[bu] 2
|
|
Default: \[dq]\[dq]
|
|
.SS \-\-acd\-upload\-wait\-per\-gb
|
|
.PP
|
|
Additional time per GB to wait after a failed complete upload to see if
|
|
it appears.
|
|
.PP
|
|
Sometimes Amazon Drive gives an error when a file has been fully
|
|
uploaded but the file appears anyway after a little while.
|
|
This happens sometimes for files over 1GB in size and nearly every time
|
|
for files bigger than 10GB.
|
|
This parameter controls the time rclone waits for the file to appear.
|
|
.PP
|
|
The default value for this parameter is 3 minutes per GB, so by default
|
|
it will wait 3 minutes for every GB uploaded to see if the file appears.
|
|
.PP
|
|
You can disable this feature by setting it to 0.
|
|
This may cause conflict errors as rclone retries the failed upload but
|
|
the file will most likely appear correctly eventually.
|
|
.PP
|
|
These values were determined empirically by observing lots of uploads of
|
|
big files for a range of file sizes.
|
|
.PP
|
|
Upload with the \[dq]\-v\[dq] flag to see more info about what rclone is
|
|
doing in this situation.
|
|
.IP \[bu] 2
|
|
Config: upload_wait_per_gb
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_ACD_UPLOAD_WAIT_PER_GB
|
|
.IP \[bu] 2
|
|
Type: Duration
|
|
.IP \[bu] 2
|
|
Default: 3m0s
|
|
.SS \-\-acd\-templink\-threshold
|
|
.PP
|
|
Files >= this size will be downloaded via their tempLink.
|
|
.PP
|
|
Files this size or more will be downloaded via their \[dq]tempLink\[dq].
|
|
This is to work around a problem with Amazon Drive which blocks
|
|
downloads of files bigger than about 10GB.
|
|
The default for this is 9GB which shouldn\[aq]t need to be changed.
|
|
.PP
|
|
To download files above this threshold, rclone requests a
|
|
\[dq]tempLink\[dq] which downloads the file through a temporary URL
|
|
directly from the underlying S3 storage.
|
|
.IP \[bu] 2
|
|
Config: templink_threshold
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_ACD_TEMPLINK_THRESHOLD
|
|
.IP \[bu] 2
|
|
Type: SizeSuffix
|
|
.IP \[bu] 2
|
|
Default: 9G
|
|
.SS \-\-acd\-encoding
|
|
.PP
|
|
This sets the encoding for the backend.
|
|
.PP
|
|
See: the encoding section in the
|
|
overview (https://rclone.org/overview/#encoding) for more info.
|
|
.IP \[bu] 2
|
|
Config: encoding
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_ACD_ENCODING
|
|
.IP \[bu] 2
|
|
Type: MultiEncoder
|
|
.IP \[bu] 2
|
|
Default: Slash,InvalidUtf8,Dot
|
|
.SS Limitations
|
|
.PP
|
|
Note that Amazon Drive is case insensitive so you can\[aq]t have a file
|
|
called \[dq]Hello.doc\[dq] and one called \[dq]hello.doc\[dq].
|
|
.PP
|
|
Amazon Drive has rate limiting so you may notice errors in the sync (429
|
|
errors).
|
|
rclone will automatically retry the sync up to 3 times by default (see
|
|
\f[C]\-\-retries\f[R] flag) which should hopefully work around this
|
|
problem.
|
|
.PP
|
|
Amazon Drive has an internal limit of file sizes that can be uploaded to
|
|
the service.
|
|
This limit is not officially published, but all files larger than this
|
|
will fail.
|
|
.PP
|
|
At the time of writing (Jan 2016) is in the area of 50GB per file.
|
|
This means that larger files are likely to fail.
|
|
.PP
|
|
Unfortunately there is no way for rclone to see that this failure is
|
|
because of file size, so it will retry the operation, as any other
|
|
failure.
|
|
To avoid this problem, use \f[C]\-\-max\-size 50000M\f[R] option to
|
|
limit the maximum size of uploaded files.
|
|
Note that \f[C]\-\-max\-size\f[R] does not split files into segments, it
|
|
only ignores files over this size.
|
|
.PP
|
|
\f[C]rclone about\f[R] is not supported by the Amazon Drive backend.
|
|
Backends without this capability cannot determine free space for an
|
|
rclone mount or use policy \f[C]mfs\f[R] (most free space) as a member
|
|
of an rclone union remote.
|
|
.PP
|
|
See List of backends that do not support rclone
|
|
about (https://rclone.org/overview/#optional-features) See rclone
|
|
about (https://rclone.org/commands/rclone_about/)
|
|
.SS Amazon S3 Storage Providers
|
|
.PP
|
|
The S3 backend can be used with a number of different providers:
|
|
.IP \[bu] 2
|
|
AWS S3
|
|
.IP \[bu] 2
|
|
Alibaba Cloud (Aliyun) Object Storage System (OSS)
|
|
.IP \[bu] 2
|
|
Ceph
|
|
.IP \[bu] 2
|
|
DigitalOcean Spaces
|
|
.IP \[bu] 2
|
|
Dreamhost
|
|
.IP \[bu] 2
|
|
IBM COS S3
|
|
.IP \[bu] 2
|
|
Minio
|
|
.IP \[bu] 2
|
|
Scaleway
|
|
.IP \[bu] 2
|
|
StackPath
|
|
.IP \[bu] 2
|
|
Tencent Cloud Object Storage (COS)
|
|
.IP \[bu] 2
|
|
Wasabi
|
|
.PP
|
|
Paths are specified as \f[C]remote:bucket\f[R] (or \f[C]remote:\f[R] for
|
|
the \f[C]lsd\f[R] command.) You may put subdirectories in too, e.g.
|
|
\f[C]remote:bucket/path/to/dir\f[R].
|
|
.PP
|
|
Once you have made a remote (see the provider specific section above)
|
|
you can use it like this:
|
|
.PP
|
|
See all buckets
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone lsd remote:
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
Make a new bucket
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone mkdir remote:bucket
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
List the contents of a bucket
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone ls remote:bucket
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
Sync \f[C]/home/local/directory\f[R] to the remote bucket, deleting any
|
|
excess files in the bucket.
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone sync \-i /home/local/directory remote:bucket
|
|
\f[R]
|
|
.fi
|
|
.SS AWS S3
|
|
.PP
|
|
Here is an example of making an s3 configuration.
|
|
First run
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone config
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
This will guide you through an interactive setup process.
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
No remotes found \- make a new one
|
|
n) New remote
|
|
s) Set configuration password
|
|
q) Quit config
|
|
n/s/q> n
|
|
name> remote
|
|
Type of storage to configure.
|
|
Choose a number from below, or type in your own value
|
|
[snip]
|
|
XX / Amazon S3 Compliant Storage Providers including AWS, Ceph, Dreamhost, IBM COS, Minio, and Tencent COS
|
|
\[rs] \[dq]s3\[dq]
|
|
[snip]
|
|
Storage> s3
|
|
Choose your S3 provider.
|
|
Choose a number from below, or type in your own value
|
|
1 / Amazon Web Services (AWS) S3
|
|
\[rs] \[dq]AWS\[dq]
|
|
2 / Ceph Object Storage
|
|
\[rs] \[dq]Ceph\[dq]
|
|
3 / Digital Ocean Spaces
|
|
\[rs] \[dq]DigitalOcean\[dq]
|
|
4 / Dreamhost DreamObjects
|
|
\[rs] \[dq]Dreamhost\[dq]
|
|
5 / IBM COS S3
|
|
\[rs] \[dq]IBMCOS\[dq]
|
|
6 / Minio Object Storage
|
|
\[rs] \[dq]Minio\[dq]
|
|
7 / Wasabi Object Storage
|
|
\[rs] \[dq]Wasabi\[dq]
|
|
8 / Any other S3 compatible provider
|
|
\[rs] \[dq]Other\[dq]
|
|
provider> 1
|
|
Get AWS credentials from runtime (environment variables or EC2/ECS meta data if no env vars). Only applies if access_key_id and secret_access_key is blank.
|
|
Choose a number from below, or type in your own value
|
|
1 / Enter AWS credentials in the next step
|
|
\[rs] \[dq]false\[dq]
|
|
2 / Get AWS credentials from the environment (env vars or IAM)
|
|
\[rs] \[dq]true\[dq]
|
|
env_auth> 1
|
|
AWS Access Key ID \- leave blank for anonymous access or runtime credentials.
|
|
access_key_id> XXX
|
|
AWS Secret Access Key (password) \- leave blank for anonymous access or runtime credentials.
|
|
secret_access_key> YYY
|
|
Region to connect to.
|
|
Choose a number from below, or type in your own value
|
|
/ The default endpoint \- a good choice if you are unsure.
|
|
1 | US Region, Northern Virginia, or Pacific Northwest.
|
|
| Leave location constraint empty.
|
|
\[rs] \[dq]us\-east\-1\[dq]
|
|
/ US East (Ohio) Region
|
|
2 | Needs location constraint us\-east\-2.
|
|
\[rs] \[dq]us\-east\-2\[dq]
|
|
/ US West (Oregon) Region
|
|
3 | Needs location constraint us\-west\-2.
|
|
\[rs] \[dq]us\-west\-2\[dq]
|
|
/ US West (Northern California) Region
|
|
4 | Needs location constraint us\-west\-1.
|
|
\[rs] \[dq]us\-west\-1\[dq]
|
|
/ Canada (Central) Region
|
|
5 | Needs location constraint ca\-central\-1.
|
|
\[rs] \[dq]ca\-central\-1\[dq]
|
|
/ EU (Ireland) Region
|
|
6 | Needs location constraint EU or eu\-west\-1.
|
|
\[rs] \[dq]eu\-west\-1\[dq]
|
|
/ EU (London) Region
|
|
7 | Needs location constraint eu\-west\-2.
|
|
\[rs] \[dq]eu\-west\-2\[dq]
|
|
/ EU (Frankfurt) Region
|
|
8 | Needs location constraint eu\-central\-1.
|
|
\[rs] \[dq]eu\-central\-1\[dq]
|
|
/ Asia Pacific (Singapore) Region
|
|
9 | Needs location constraint ap\-southeast\-1.
|
|
\[rs] \[dq]ap\-southeast\-1\[dq]
|
|
/ Asia Pacific (Sydney) Region
|
|
10 | Needs location constraint ap\-southeast\-2.
|
|
\[rs] \[dq]ap\-southeast\-2\[dq]
|
|
/ Asia Pacific (Tokyo) Region
|
|
11 | Needs location constraint ap\-northeast\-1.
|
|
\[rs] \[dq]ap\-northeast\-1\[dq]
|
|
/ Asia Pacific (Seoul)
|
|
12 | Needs location constraint ap\-northeast\-2.
|
|
\[rs] \[dq]ap\-northeast\-2\[dq]
|
|
/ Asia Pacific (Mumbai)
|
|
13 | Needs location constraint ap\-south\-1.
|
|
\[rs] \[dq]ap\-south\-1\[dq]
|
|
/ Asia Pacific (Hong Kong) Region
|
|
14 | Needs location constraint ap\-east\-1.
|
|
\[rs] \[dq]ap\-east\-1\[dq]
|
|
/ South America (Sao Paulo) Region
|
|
15 | Needs location constraint sa\-east\-1.
|
|
\[rs] \[dq]sa\-east\-1\[dq]
|
|
region> 1
|
|
Endpoint for S3 API.
|
|
Leave blank if using AWS to use the default endpoint for the region.
|
|
endpoint>
|
|
Location constraint \- must be set to match the Region. Used when creating buckets only.
|
|
Choose a number from below, or type in your own value
|
|
1 / Empty for US Region, Northern Virginia, or Pacific Northwest.
|
|
\[rs] \[dq]\[dq]
|
|
2 / US East (Ohio) Region.
|
|
\[rs] \[dq]us\-east\-2\[dq]
|
|
3 / US West (Oregon) Region.
|
|
\[rs] \[dq]us\-west\-2\[dq]
|
|
4 / US West (Northern California) Region.
|
|
\[rs] \[dq]us\-west\-1\[dq]
|
|
5 / Canada (Central) Region.
|
|
\[rs] \[dq]ca\-central\-1\[dq]
|
|
6 / EU (Ireland) Region.
|
|
\[rs] \[dq]eu\-west\-1\[dq]
|
|
7 / EU (London) Region.
|
|
\[rs] \[dq]eu\-west\-2\[dq]
|
|
8 / EU Region.
|
|
\[rs] \[dq]EU\[dq]
|
|
9 / Asia Pacific (Singapore) Region.
|
|
\[rs] \[dq]ap\-southeast\-1\[dq]
|
|
10 / Asia Pacific (Sydney) Region.
|
|
\[rs] \[dq]ap\-southeast\-2\[dq]
|
|
11 / Asia Pacific (Tokyo) Region.
|
|
\[rs] \[dq]ap\-northeast\-1\[dq]
|
|
12 / Asia Pacific (Seoul)
|
|
\[rs] \[dq]ap\-northeast\-2\[dq]
|
|
13 / Asia Pacific (Mumbai)
|
|
\[rs] \[dq]ap\-south\-1\[dq]
|
|
14 / Asia Pacific (Hong Kong)
|
|
\[rs] \[dq]ap\-east\-1\[dq]
|
|
15 / South America (Sao Paulo) Region.
|
|
\[rs] \[dq]sa\-east\-1\[dq]
|
|
location_constraint> 1
|
|
Canned ACL used when creating buckets and/or storing objects in S3.
|
|
For more info visit https://docs.aws.amazon.com/AmazonS3/latest/dev/acl\-overview.html#canned\-acl
|
|
Choose a number from below, or type in your own value
|
|
1 / Owner gets FULL_CONTROL. No one else has access rights (default).
|
|
\[rs] \[dq]private\[dq]
|
|
2 / Owner gets FULL_CONTROL. The AllUsers group gets READ access.
|
|
\[rs] \[dq]public\-read\[dq]
|
|
/ Owner gets FULL_CONTROL. The AllUsers group gets READ and WRITE access.
|
|
3 | Granting this on a bucket is generally not recommended.
|
|
\[rs] \[dq]public\-read\-write\[dq]
|
|
4 / Owner gets FULL_CONTROL. The AuthenticatedUsers group gets READ access.
|
|
\[rs] \[dq]authenticated\-read\[dq]
|
|
/ Object owner gets FULL_CONTROL. Bucket owner gets READ access.
|
|
5 | If you specify this canned ACL when creating a bucket, Amazon S3 ignores it.
|
|
\[rs] \[dq]bucket\-owner\-read\[dq]
|
|
/ Both the object owner and the bucket owner get FULL_CONTROL over the object.
|
|
6 | If you specify this canned ACL when creating a bucket, Amazon S3 ignores it.
|
|
\[rs] \[dq]bucket\-owner\-full\-control\[dq]
|
|
acl> 1
|
|
The server\-side encryption algorithm used when storing this object in S3.
|
|
Choose a number from below, or type in your own value
|
|
1 / None
|
|
\[rs] \[dq]\[dq]
|
|
2 / AES256
|
|
\[rs] \[dq]AES256\[dq]
|
|
server_side_encryption> 1
|
|
The storage class to use when storing objects in S3.
|
|
Choose a number from below, or type in your own value
|
|
1 / Default
|
|
\[rs] \[dq]\[dq]
|
|
2 / Standard storage class
|
|
\[rs] \[dq]STANDARD\[dq]
|
|
3 / Reduced redundancy storage class
|
|
\[rs] \[dq]REDUCED_REDUNDANCY\[dq]
|
|
4 / Standard Infrequent Access storage class
|
|
\[rs] \[dq]STANDARD_IA\[dq]
|
|
5 / One Zone Infrequent Access storage class
|
|
\[rs] \[dq]ONEZONE_IA\[dq]
|
|
6 / Glacier storage class
|
|
\[rs] \[dq]GLACIER\[dq]
|
|
7 / Glacier Deep Archive storage class
|
|
\[rs] \[dq]DEEP_ARCHIVE\[dq]
|
|
8 / Intelligent\-Tiering storage class
|
|
\[rs] \[dq]INTELLIGENT_TIERING\[dq]
|
|
storage_class> 1
|
|
Remote config
|
|
\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-
|
|
[remote]
|
|
type = s3
|
|
provider = AWS
|
|
env_auth = false
|
|
access_key_id = XXX
|
|
secret_access_key = YYY
|
|
region = us\-east\-1
|
|
endpoint =
|
|
location_constraint =
|
|
acl = private
|
|
server_side_encryption =
|
|
storage_class =
|
|
\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-
|
|
y) Yes this is OK
|
|
e) Edit this remote
|
|
d) Delete this remote
|
|
y/e/d>
|
|
\f[R]
|
|
.fi
|
|
.SS Modified time
|
|
.PP
|
|
The modified time is stored as metadata on the object as
|
|
\f[C]X\-Amz\-Meta\-Mtime\f[R] as floating point since the epoch accurate
|
|
to 1 ns.
|
|
.PP
|
|
If the modification time needs to be updated rclone will attempt to
|
|
perform a server side copy to update the modification if the object can
|
|
be copied in a single part.
|
|
In the case the object is larger than 5Gb or is in Glacier or Glacier
|
|
Deep Archive storage the object will be uploaded rather than copied.
|
|
.PP
|
|
Note that reading this from the object takes an additional
|
|
\f[C]HEAD\f[R] request as the metadata isn\[aq]t returned in object
|
|
listings.
|
|
.SS Reducing costs
|
|
.SS Avoiding HEAD requests to read the modification time
|
|
.PP
|
|
By default rclone will use the modification time of objects stored in S3
|
|
for syncing.
|
|
This is stored in object metadata which unfortunately takes an extra
|
|
HEAD request to read which can be expensive (in time and money).
|
|
.PP
|
|
The modification time is used by default for all operations that require
|
|
checking the time a file was last updated.
|
|
It allows rclone to treat the remote more like a true filesystem, but it
|
|
is inefficient on S3 because it requires an extra API call to retrieve
|
|
the metadata.
|
|
.PP
|
|
The extra API calls can be avoided when syncing (using
|
|
\f[C]rclone sync\f[R] or \f[C]rclone copy\f[R]) in a few different ways,
|
|
each with its own tradeoffs.
|
|
.IP \[bu] 2
|
|
\f[C]\-\-size\-only\f[R]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Only checks the size of files.
|
|
.IP \[bu] 2
|
|
Uses no extra transactions.
|
|
.IP \[bu] 2
|
|
If the file doesn\[aq]t change size then rclone won\[aq]t detect it has
|
|
changed.
|
|
.IP \[bu] 2
|
|
\f[C]rclone sync \-\-size\-only /path/to/source s3:bucket\f[R]
|
|
.RE
|
|
.IP \[bu] 2
|
|
\f[C]\-\-checksum\f[R]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Checks the size and MD5 checksum of files.
|
|
.IP \[bu] 2
|
|
Uses no extra transactions.
|
|
.IP \[bu] 2
|
|
The most accurate detection of changes possible.
|
|
.IP \[bu] 2
|
|
Will cause the source to read an MD5 checksum which, if it is a local
|
|
disk, will cause lots of disk activity.
|
|
.IP \[bu] 2
|
|
If the source and destination are both S3 this is the
|
|
\f[B]recommended\f[R] flag to use for maximum efficiency.
|
|
.IP \[bu] 2
|
|
\f[C]rclone sync \-\-checksum /path/to/source s3:bucket\f[R]
|
|
.RE
|
|
.IP \[bu] 2
|
|
\f[C]\-\-update \-\-use\-server\-modtime\f[R]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Uses no extra transactions.
|
|
.IP \[bu] 2
|
|
Modification time becomes the time the object was uploaded.
|
|
.IP \[bu] 2
|
|
For many operations this is sufficient to determine if it needs
|
|
uploading.
|
|
.IP \[bu] 2
|
|
Using \f[C]\-\-update\f[R] along with
|
|
\f[C]\-\-use\-server\-modtime\f[R], avoids the extra API call and
|
|
uploads files whose local modification time is newer than the time it
|
|
was last uploaded.
|
|
.IP \[bu] 2
|
|
Files created with timestamps in the past will be missed by the sync.
|
|
.IP \[bu] 2
|
|
\f[C]rclone sync \-\-update \-\-use\-server\-modtime /path/to/source s3:bucket\f[R]
|
|
.RE
|
|
.PP
|
|
These flags can and should be used in combination with
|
|
\f[C]\-\-fast\-list\f[R] \- see below.
|
|
.PP
|
|
If using \f[C]rclone mount\f[R] or any command using the VFS (eg
|
|
\f[C]rclone serve\f[R]) commands then you might want to consider using
|
|
the VFS flag \f[C]\-\-no\-modtime\f[R] which will stop rclone reading
|
|
the modification time for every object.
|
|
You could also use \f[C]\-\-use\-server\-modtime\f[R] if you are happy
|
|
with the modification times of the objects being the time of upload.
|
|
.SS Avoiding GET requests to read directory listings
|
|
.PP
|
|
Rclone\[aq]s default directory traversal is to process each directory
|
|
individually.
|
|
This takes one API call per directory.
|
|
Using the \f[C]\-\-fast\-list\f[R] flag will read all info about the the
|
|
objects into memory first using a smaller number of API calls (one per
|
|
1000 objects).
|
|
See the rclone docs (https://rclone.org/docs/#fast-list) for more
|
|
details.
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone sync \-\-fast\-list \-\-checksum /path/to/source s3:bucket
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
\f[C]\-\-fast\-list\f[R] trades off API transactions for memory use.
|
|
As a rough guide rclone uses 1k of memory per object stored, so using
|
|
\f[C]\-\-fast\-list\f[R] on a sync of a million objects will use roughly
|
|
1 GB of RAM.
|
|
.PP
|
|
If you are only copying a small number of files into a big repository
|
|
then using \f[C]\-\-no\-traverse\f[R] is a good idea.
|
|
This finds objects directly instead of through directory listings.
|
|
You can do a \[dq]top\-up\[dq] sync very cheaply by using
|
|
\f[C]\-\-max\-age\f[R] and \f[C]\-\-no\-traverse\f[R] to copy only
|
|
recent files, eg
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone copy \-\-min\-age 24h \-\-no\-traverse /path/to/source s3:bucket
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
You\[aq]d then do a full \f[C]rclone sync\f[R] less often.
|
|
.PP
|
|
Note that \f[C]\-\-fast\-list\f[R] isn\[aq]t required in the top\-up
|
|
sync.
|
|
.SS Avoiding HEAD requests after PUT
|
|
.PP
|
|
By default rclone will HEAD every object it uploads.
|
|
It does this to check the object got uploaded correctly.
|
|
.PP
|
|
You can disable this with the \-\-s3\-no\-head option \- see there for
|
|
more details.
|
|
.PP
|
|
Setting this flag increases the chance for undetected upload failures.
|
|
.SS Hashes
|
|
.PP
|
|
For small objects which weren\[aq]t uploaded as multipart uploads
|
|
(objects sized below \f[C]\-\-s3\-upload\-cutoff\f[R] if uploaded with
|
|
rclone) rclone uses the \f[C]ETag:\f[R] header as an MD5 checksum.
|
|
.PP
|
|
However for objects which were uploaded as multipart uploads or with
|
|
server side encryption (SSE\-AWS or SSE\-C) the \f[C]ETag\f[R] header is
|
|
no longer the MD5 sum of the data, so rclone adds an additional piece of
|
|
metadata \f[C]X\-Amz\-Meta\-Md5chksum\f[R] which is a base64 encoded MD5
|
|
hash (in the same format as is required for \f[C]Content\-MD5\f[R]).
|
|
.PP
|
|
For large objects, calculating this hash can take some time so the
|
|
addition of this hash can be disabled with
|
|
\f[C]\-\-s3\-disable\-checksum\f[R].
|
|
This will mean that these objects do not have an MD5 checksum.
|
|
.PP
|
|
Note that reading this from the object takes an additional
|
|
\f[C]HEAD\f[R] request as the metadata isn\[aq]t returned in object
|
|
listings.
|
|
.SS Cleanup
|
|
.PP
|
|
If you run \f[C]rclone cleanup s3:bucket\f[R] then it will remove all
|
|
pending multipart uploads older than 24 hours.
|
|
You can use the \f[C]\-i\f[R] flag to see exactly what it will do.
|
|
If you want more control over the expiry date then run
|
|
\f[C]rclone backend cleanup s3:bucket \-o max\-age=1h\f[R] to expire all
|
|
uploads older than one hour.
|
|
You can use \f[C]rclone backend list\-multipart\-uploads s3:bucket\f[R]
|
|
to see the pending multipart uploads.
|
|
.SS Restricted filename characters
|
|
.PP
|
|
S3 allows any valid UTF\-8 string as a key.
|
|
.PP
|
|
Invalid UTF\-8 bytes will be
|
|
replaced (https://rclone.org/overview/#invalid-utf8), as they can\[aq]t
|
|
be used in XML.
|
|
.PP
|
|
The following characters are replaced since these are problematic when
|
|
dealing with the REST API:
|
|
.PP
|
|
.TS
|
|
tab(@);
|
|
l c c.
|
|
T{
|
|
Character
|
|
T}@T{
|
|
Value
|
|
T}@T{
|
|
Replacement
|
|
T}
|
|
_
|
|
T{
|
|
NUL
|
|
T}@T{
|
|
0x00
|
|
T}@T{
|
|
\[u2400]
|
|
T}
|
|
T{
|
|
/
|
|
T}@T{
|
|
0x2F
|
|
T}@T{
|
|
\[uFF0F]
|
|
T}
|
|
.TE
|
|
.PP
|
|
The encoding will also encode these file names as they don\[aq]t seem to
|
|
work with the SDK properly:
|
|
.PP
|
|
.TS
|
|
tab(@);
|
|
l c.
|
|
T{
|
|
File name
|
|
T}@T{
|
|
Replacement
|
|
T}
|
|
_
|
|
T{
|
|
\&.
|
|
T}@T{
|
|
\[uFF0E]
|
|
T}
|
|
T{
|
|
\&..
|
|
T}@T{
|
|
\[uFF0E]\[uFF0E]
|
|
T}
|
|
.TE
|
|
.SS Multipart uploads
|
|
.PP
|
|
rclone supports multipart uploads with S3 which means that it can upload
|
|
files bigger than 5GB.
|
|
.PP
|
|
Note that files uploaded \f[I]both\f[R] with multipart upload
|
|
\f[I]and\f[R] through crypt remotes do not have MD5 sums.
|
|
.PP
|
|
rclone switches from single part uploads to multipart uploads at the
|
|
point specified by \f[C]\-\-s3\-upload\-cutoff\f[R].
|
|
This can be a maximum of 5GB and a minimum of 0 (ie always upload
|
|
multipart files).
|
|
.PP
|
|
The chunk sizes used in the multipart upload are specified by
|
|
\f[C]\-\-s3\-chunk\-size\f[R] and the number of chunks uploaded
|
|
concurrently is specified by \f[C]\-\-s3\-upload\-concurrency\f[R].
|
|
.PP
|
|
Multipart uploads will use \f[C]\-\-transfers\f[R] *
|
|
\f[C]\-\-s3\-upload\-concurrency\f[R] * \f[C]\-\-s3\-chunk\-size\f[R]
|
|
extra memory.
|
|
Single part uploads to not use extra memory.
|
|
.PP
|
|
Single part transfers can be faster than multipart transfers or slower
|
|
depending on your latency from S3 \- the more latency, the more likely
|
|
single part transfers will be faster.
|
|
.PP
|
|
Increasing \f[C]\-\-s3\-upload\-concurrency\f[R] will increase
|
|
throughput (8 would be a sensible value) and increasing
|
|
\f[C]\-\-s3\-chunk\-size\f[R] also increases throughput (16M would be
|
|
sensible).
|
|
Increasing either of these will use more memory.
|
|
The default values are high enough to gain most of the possible
|
|
performance without using too much memory.
|
|
.SS Buckets and Regions
|
|
.PP
|
|
With Amazon S3 you can list buckets (\f[C]rclone lsd\f[R]) using any
|
|
region, but you can only access the content of a bucket from the region
|
|
it was created in.
|
|
If you attempt to access a bucket from the wrong region, you will get an
|
|
error,
|
|
\f[C]incorrect region, the bucket is not in \[aq]XXX\[aq] region\f[R].
|
|
.SS Authentication
|
|
.PP
|
|
There are a number of ways to supply \f[C]rclone\f[R] with a set of AWS
|
|
credentials, with and without using the environment.
|
|
.PP
|
|
The different authentication methods are tried in this order:
|
|
.IP \[bu] 2
|
|
Directly in the rclone configuration file (\f[C]env_auth = false\f[R] in
|
|
the config file):
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
\f[C]access_key_id\f[R] and \f[C]secret_access_key\f[R] are required.
|
|
.IP \[bu] 2
|
|
\f[C]session_token\f[R] can be optionally set when using AWS STS.
|
|
.RE
|
|
.IP \[bu] 2
|
|
Runtime configuration (\f[C]env_auth = true\f[R] in the config file):
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Export the following environment variables before running
|
|
\f[C]rclone\f[R]:
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Access Key ID: \f[C]AWS_ACCESS_KEY_ID\f[R] or \f[C]AWS_ACCESS_KEY\f[R]
|
|
.IP \[bu] 2
|
|
Secret Access Key: \f[C]AWS_SECRET_ACCESS_KEY\f[R] or
|
|
\f[C]AWS_SECRET_KEY\f[R]
|
|
.IP \[bu] 2
|
|
Session Token: \f[C]AWS_SESSION_TOKEN\f[R] (optional)
|
|
.RE
|
|
.IP \[bu] 2
|
|
Or, use a named
|
|
profile (https://docs.aws.amazon.com/cli/latest/userguide/cli-multiple-profiles.html):
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Profile files are standard files used by AWS CLI tools
|
|
.IP \[bu] 2
|
|
By default it will use the profile in your home directory (e.g.
|
|
\f[C]\[ti]/.aws/credentials\f[R] on unix based systems) file and the
|
|
\[dq]default\[dq] profile, to change set these environment variables:
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
\f[C]AWS_SHARED_CREDENTIALS_FILE\f[R] to control which file.
|
|
.IP \[bu] 2
|
|
\f[C]AWS_PROFILE\f[R] to control which profile to use.
|
|
.RE
|
|
.RE
|
|
.IP \[bu] 2
|
|
Or, run \f[C]rclone\f[R] in an ECS task with an IAM role (AWS only).
|
|
.IP \[bu] 2
|
|
Or, run \f[C]rclone\f[R] on an EC2 instance with an IAM role (AWS only).
|
|
.IP \[bu] 2
|
|
Or, run \f[C]rclone\f[R] in an EKS pod with an IAM role that is
|
|
associated with a service account (AWS only).
|
|
.RE
|
|
.PP
|
|
If none of these option actually end up providing \f[C]rclone\f[R] with
|
|
AWS credentials then S3 interaction will be non\-authenticated (see
|
|
below).
|
|
.SS S3 Permissions
|
|
.PP
|
|
When using the \f[C]sync\f[R] subcommand of \f[C]rclone\f[R] the
|
|
following minimum permissions are required to be available on the bucket
|
|
being written to:
|
|
.IP \[bu] 2
|
|
\f[C]ListBucket\f[R]
|
|
.IP \[bu] 2
|
|
\f[C]DeleteObject\f[R]
|
|
.IP \[bu] 2
|
|
\f[C]GetObject\f[R]
|
|
.IP \[bu] 2
|
|
\f[C]PutObject\f[R]
|
|
.IP \[bu] 2
|
|
\f[C]PutObjectACL\f[R]
|
|
.PP
|
|
When using the \f[C]lsd\f[R] subcommand, the \f[C]ListAllMyBuckets\f[R]
|
|
permission is required.
|
|
.PP
|
|
Example policy:
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
{
|
|
\[dq]Version\[dq]: \[dq]2012\-10\-17\[dq],
|
|
\[dq]Statement\[dq]: [
|
|
{
|
|
\[dq]Effect\[dq]: \[dq]Allow\[dq],
|
|
\[dq]Principal\[dq]: {
|
|
\[dq]AWS\[dq]: \[dq]arn:aws:iam::USER_SID:user/USER_NAME\[dq]
|
|
},
|
|
\[dq]Action\[dq]: [
|
|
\[dq]s3:ListBucket\[dq],
|
|
\[dq]s3:DeleteObject\[dq],
|
|
\[dq]s3:GetObject\[dq],
|
|
\[dq]s3:PutObject\[dq],
|
|
\[dq]s3:PutObjectAcl\[dq]
|
|
],
|
|
\[dq]Resource\[dq]: [
|
|
\[dq]arn:aws:s3:::BUCKET_NAME/*\[dq],
|
|
\[dq]arn:aws:s3:::BUCKET_NAME\[dq]
|
|
]
|
|
},
|
|
{
|
|
\[dq]Effect\[dq]: \[dq]Allow\[dq],
|
|
\[dq]Action\[dq]: \[dq]s3:ListAllMyBuckets\[dq],
|
|
\[dq]Resource\[dq]: \[dq]arn:aws:s3:::*\[dq]
|
|
}
|
|
]
|
|
}
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
Notes on above:
|
|
.IP "1." 3
|
|
This is a policy that can be used when creating bucket.
|
|
It assumes that \f[C]USER_NAME\f[R] has been created.
|
|
.IP "2." 3
|
|
The Resource entry must include both resource ARNs, as one implies the
|
|
bucket and the other implies the bucket\[aq]s objects.
|
|
.PP
|
|
For reference, here\[aq]s an Ansible
|
|
script (https://gist.github.com/ebridges/ebfc9042dd7c756cd101cfa807b7ae2b)
|
|
that will generate one or more buckets that will work with
|
|
\f[C]rclone sync\f[R].
|
|
.SS Key Management System (KMS)
|
|
.PP
|
|
If you are using server\-side encryption with KMS then you must make
|
|
sure rclone is configured with
|
|
\f[C]server_side_encryption = aws:kms\f[R] otherwise you will find you
|
|
can\[aq]t transfer small objects \- these will create checksum errors.
|
|
.SS Glacier and Glacier Deep Archive
|
|
.PP
|
|
You can upload objects using the glacier storage class or transition
|
|
them to glacier using a lifecycle
|
|
policy (http://docs.aws.amazon.com/AmazonS3/latest/user-guide/create-lifecycle.html).
|
|
The bucket can still be synced or copied into normally, but if rclone
|
|
tries to access data from the glacier storage class you will see an
|
|
error like below.
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
2017/09/11 19:07:43 Failed to sync: failed to open source object: Object in GLACIER, restore first: path/to/file
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
In this case you need to
|
|
restore (http://docs.aws.amazon.com/AmazonS3/latest/user-guide/restore-archived-objects.html)
|
|
the object(s) in question before using rclone.
|
|
.PP
|
|
Note that rclone only speaks the S3 API it does not speak the Glacier
|
|
Vault API, so rclone cannot directly access Glacier Vaults.
|
|
.SS Standard Options
|
|
.PP
|
|
Here are the standard options specific to s3 (Amazon S3 Compliant
|
|
Storage Providers including AWS, Alibaba, Ceph, Digital Ocean,
|
|
Dreamhost, IBM COS, Minio, and Tencent COS).
|
|
.SS \-\-s3\-provider
|
|
.PP
|
|
Choose your S3 provider.
|
|
.IP \[bu] 2
|
|
Config: provider
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_S3_PROVIDER
|
|
.IP \[bu] 2
|
|
Type: string
|
|
.IP \[bu] 2
|
|
Default: \[dq]\[dq]
|
|
.IP \[bu] 2
|
|
Examples:
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
\[dq]AWS\[dq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Amazon Web Services (AWS) S3
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[dq]Alibaba\[dq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Alibaba Cloud Object Storage System (OSS) formerly Aliyun
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[dq]Ceph\[dq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Ceph Object Storage
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[dq]DigitalOcean\[dq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Digital Ocean Spaces
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[dq]Dreamhost\[dq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Dreamhost DreamObjects
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[dq]IBMCOS\[dq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
IBM COS S3
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[dq]Minio\[dq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Minio Object Storage
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[dq]Netease\[dq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Netease Object Storage (NOS)
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[dq]Scaleway\[dq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Scaleway Object Storage
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[dq]StackPath\[dq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
StackPath Object Storage
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[dq]TencentCOS\[dq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Tencent Cloud Object Storage (COS)
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[dq]Wasabi\[dq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Wasabi Object Storage
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[dq]Other\[dq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Any other S3 compatible provider
|
|
.RE
|
|
.RE
|
|
.SS \-\-s3\-env\-auth
|
|
.PP
|
|
Get AWS credentials from runtime (environment variables or EC2/ECS meta
|
|
data if no env vars).
|
|
Only applies if access_key_id and secret_access_key is blank.
|
|
.IP \[bu] 2
|
|
Config: env_auth
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_S3_ENV_AUTH
|
|
.IP \[bu] 2
|
|
Type: bool
|
|
.IP \[bu] 2
|
|
Default: false
|
|
.IP \[bu] 2
|
|
Examples:
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
\[dq]false\[dq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Enter AWS credentials in the next step
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[dq]true\[dq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Get AWS credentials from the environment (env vars or IAM)
|
|
.RE
|
|
.RE
|
|
.SS \-\-s3\-access\-key\-id
|
|
.PP
|
|
AWS Access Key ID.
|
|
Leave blank for anonymous access or runtime credentials.
|
|
.IP \[bu] 2
|
|
Config: access_key_id
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_S3_ACCESS_KEY_ID
|
|
.IP \[bu] 2
|
|
Type: string
|
|
.IP \[bu] 2
|
|
Default: \[dq]\[dq]
|
|
.SS \-\-s3\-secret\-access\-key
|
|
.PP
|
|
AWS Secret Access Key (password) Leave blank for anonymous access or
|
|
runtime credentials.
|
|
.IP \[bu] 2
|
|
Config: secret_access_key
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_S3_SECRET_ACCESS_KEY
|
|
.IP \[bu] 2
|
|
Type: string
|
|
.IP \[bu] 2
|
|
Default: \[dq]\[dq]
|
|
.SS \-\-s3\-region
|
|
.PP
|
|
Region to connect to.
|
|
.IP \[bu] 2
|
|
Config: region
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_S3_REGION
|
|
.IP \[bu] 2
|
|
Type: string
|
|
.IP \[bu] 2
|
|
Default: \[dq]\[dq]
|
|
.IP \[bu] 2
|
|
Examples:
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
\[dq]us\-east\-1\[dq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
The default endpoint \- a good choice if you are unsure.
|
|
.IP \[bu] 2
|
|
US Region, Northern Virginia, or Pacific Northwest.
|
|
.IP \[bu] 2
|
|
Leave location constraint empty.
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[dq]us\-east\-2\[dq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
US East (Ohio) Region
|
|
.IP \[bu] 2
|
|
Needs location constraint us\-east\-2.
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[dq]us\-west\-1\[dq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
US West (Northern California) Region
|
|
.IP \[bu] 2
|
|
Needs location constraint us\-west\-1.
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[dq]us\-west\-2\[dq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
US West (Oregon) Region
|
|
.IP \[bu] 2
|
|
Needs location constraint us\-west\-2.
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[dq]ca\-central\-1\[dq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Canada (Central) Region
|
|
.IP \[bu] 2
|
|
Needs location constraint ca\-central\-1.
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[dq]eu\-west\-1\[dq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
EU (Ireland) Region
|
|
.IP \[bu] 2
|
|
Needs location constraint EU or eu\-west\-1.
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[dq]eu\-west\-2\[dq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
EU (London) Region
|
|
.IP \[bu] 2
|
|
Needs location constraint eu\-west\-2.
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[dq]eu\-west\-3\[dq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
EU (Paris) Region
|
|
.IP \[bu] 2
|
|
Needs location constraint eu\-west\-3.
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[dq]eu\-north\-1\[dq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
EU (Stockholm) Region
|
|
.IP \[bu] 2
|
|
Needs location constraint eu\-north\-1.
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[dq]eu\-south\-1\[dq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
EU (Milan) Region
|
|
.IP \[bu] 2
|
|
Needs location constraint eu\-south\-1.
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[dq]eu\-central\-1\[dq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
EU (Frankfurt) Region
|
|
.IP \[bu] 2
|
|
Needs location constraint eu\-central\-1.
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[dq]ap\-southeast\-1\[dq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Asia Pacific (Singapore) Region
|
|
.IP \[bu] 2
|
|
Needs location constraint ap\-southeast\-1.
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[dq]ap\-southeast\-2\[dq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Asia Pacific (Sydney) Region
|
|
.IP \[bu] 2
|
|
Needs location constraint ap\-southeast\-2.
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[dq]ap\-northeast\-1\[dq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Asia Pacific (Tokyo) Region
|
|
.IP \[bu] 2
|
|
Needs location constraint ap\-northeast\-1.
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[dq]ap\-northeast\-2\[dq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Asia Pacific (Seoul)
|
|
.IP \[bu] 2
|
|
Needs location constraint ap\-northeast\-2.
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[dq]ap\-northeast\-3\[dq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Asia Pacific (Osaka\-Local)
|
|
.IP \[bu] 2
|
|
Needs location constraint ap\-northeast\-3.
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[dq]ap\-south\-1\[dq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Asia Pacific (Mumbai)
|
|
.IP \[bu] 2
|
|
Needs location constraint ap\-south\-1.
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[dq]ap\-east\-1\[dq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Asia Pacific (Hong Kong) Region
|
|
.IP \[bu] 2
|
|
Needs location constraint ap\-east\-1.
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[dq]sa\-east\-1\[dq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
South America (Sao Paulo) Region
|
|
.IP \[bu] 2
|
|
Needs location constraint sa\-east\-1.
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[dq]me\-south\-1\[dq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Middle East (Bahrain) Region
|
|
.IP \[bu] 2
|
|
Needs location constraint me\-south\-1.
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[dq]af\-south\-1\[dq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Africa (Cape Town) Region
|
|
.IP \[bu] 2
|
|
Needs location constraint af\-south\-1.
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[dq]cn\-north\-1\[dq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
China (Beijing) Region
|
|
.IP \[bu] 2
|
|
Needs location constraint cn\-north\-1.
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[dq]cn\-northwest\-1\[dq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
China (Ningxia) Region
|
|
.IP \[bu] 2
|
|
Needs location constraint cn\-northwest\-1.
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[dq]us\-gov\-east\-1\[dq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
AWS GovCloud (US\-East) Region
|
|
.IP \[bu] 2
|
|
Needs location constraint us\-gov\-east\-1.
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[dq]us\-gov\-west\-1\[dq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
AWS GovCloud (US) Region
|
|
.IP \[bu] 2
|
|
Needs location constraint us\-gov\-west\-1.
|
|
.RE
|
|
.RE
|
|
.SS \-\-s3\-region
|
|
.PP
|
|
Region to connect to.
|
|
.IP \[bu] 2
|
|
Config: region
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_S3_REGION
|
|
.IP \[bu] 2
|
|
Type: string
|
|
.IP \[bu] 2
|
|
Default: \[dq]\[dq]
|
|
.IP \[bu] 2
|
|
Examples:
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
\[dq]nl\-ams\[dq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Amsterdam, The Netherlands
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[dq]fr\-par\[dq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Paris, France
|
|
.RE
|
|
.RE
|
|
.SS \-\-s3\-region
|
|
.PP
|
|
Region to connect to.
|
|
Leave blank if you are using an S3 clone and you don\[aq]t have a
|
|
region.
|
|
.IP \[bu] 2
|
|
Config: region
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_S3_REGION
|
|
.IP \[bu] 2
|
|
Type: string
|
|
.IP \[bu] 2
|
|
Default: \[dq]\[dq]
|
|
.IP \[bu] 2
|
|
Examples:
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
\[dq]\[dq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Use this if unsure.
|
|
Will use v4 signatures and an empty region.
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[dq]other\-v2\-signature\[dq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Use this only if v4 signatures don\[aq]t work, e.g.
|
|
pre Jewel/v10 CEPH.
|
|
.RE
|
|
.RE
|
|
.SS \-\-s3\-endpoint
|
|
.PP
|
|
Endpoint for S3 API.
|
|
Leave blank if using AWS to use the default endpoint for the region.
|
|
.IP \[bu] 2
|
|
Config: endpoint
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_S3_ENDPOINT
|
|
.IP \[bu] 2
|
|
Type: string
|
|
.IP \[bu] 2
|
|
Default: \[dq]\[dq]
|
|
.SS \-\-s3\-endpoint
|
|
.PP
|
|
Endpoint for IBM COS S3 API.
|
|
Specify if using an IBM COS On Premise.
|
|
.IP \[bu] 2
|
|
Config: endpoint
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_S3_ENDPOINT
|
|
.IP \[bu] 2
|
|
Type: string
|
|
.IP \[bu] 2
|
|
Default: \[dq]\[dq]
|
|
.IP \[bu] 2
|
|
Examples:
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
\[dq]s3.us.cloud\-object\-storage.appdomain.cloud\[dq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
US Cross Region Endpoint
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[dq]s3.dal.us.cloud\-object\-storage.appdomain.cloud\[dq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
US Cross Region Dallas Endpoint
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[dq]s3.wdc.us.cloud\-object\-storage.appdomain.cloud\[dq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
US Cross Region Washington DC Endpoint
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[dq]s3.sjc.us.cloud\-object\-storage.appdomain.cloud\[dq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
US Cross Region San Jose Endpoint
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[dq]s3.private.us.cloud\-object\-storage.appdomain.cloud\[dq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
US Cross Region Private Endpoint
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[dq]s3.private.dal.us.cloud\-object\-storage.appdomain.cloud\[dq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
US Cross Region Dallas Private Endpoint
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[dq]s3.private.wdc.us.cloud\-object\-storage.appdomain.cloud\[dq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
US Cross Region Washington DC Private Endpoint
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[dq]s3.private.sjc.us.cloud\-object\-storage.appdomain.cloud\[dq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
US Cross Region San Jose Private Endpoint
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[dq]s3.us\-east.cloud\-object\-storage.appdomain.cloud\[dq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
US Region East Endpoint
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[dq]s3.private.us\-east.cloud\-object\-storage.appdomain.cloud\[dq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
US Region East Private Endpoint
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[dq]s3.us\-south.cloud\-object\-storage.appdomain.cloud\[dq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
US Region South Endpoint
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[dq]s3.private.us\-south.cloud\-object\-storage.appdomain.cloud\[dq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
US Region South Private Endpoint
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[dq]s3.eu.cloud\-object\-storage.appdomain.cloud\[dq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
EU Cross Region Endpoint
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[dq]s3.fra.eu.cloud\-object\-storage.appdomain.cloud\[dq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
EU Cross Region Frankfurt Endpoint
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[dq]s3.mil.eu.cloud\-object\-storage.appdomain.cloud\[dq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
EU Cross Region Milan Endpoint
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[dq]s3.ams.eu.cloud\-object\-storage.appdomain.cloud\[dq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
EU Cross Region Amsterdam Endpoint
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[dq]s3.private.eu.cloud\-object\-storage.appdomain.cloud\[dq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
EU Cross Region Private Endpoint
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[dq]s3.private.fra.eu.cloud\-object\-storage.appdomain.cloud\[dq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
EU Cross Region Frankfurt Private Endpoint
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[dq]s3.private.mil.eu.cloud\-object\-storage.appdomain.cloud\[dq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
EU Cross Region Milan Private Endpoint
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[dq]s3.private.ams.eu.cloud\-object\-storage.appdomain.cloud\[dq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
EU Cross Region Amsterdam Private Endpoint
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[dq]s3.eu\-gb.cloud\-object\-storage.appdomain.cloud\[dq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Great Britain Endpoint
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[dq]s3.private.eu\-gb.cloud\-object\-storage.appdomain.cloud\[dq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Great Britain Private Endpoint
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[dq]s3.eu\-de.cloud\-object\-storage.appdomain.cloud\[dq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
EU Region DE Endpoint
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[dq]s3.private.eu\-de.cloud\-object\-storage.appdomain.cloud\[dq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
EU Region DE Private Endpoint
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[dq]s3.ap.cloud\-object\-storage.appdomain.cloud\[dq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
APAC Cross Regional Endpoint
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[dq]s3.tok.ap.cloud\-object\-storage.appdomain.cloud\[dq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
APAC Cross Regional Tokyo Endpoint
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[dq]s3.hkg.ap.cloud\-object\-storage.appdomain.cloud\[dq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
APAC Cross Regional HongKong Endpoint
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[dq]s3.seo.ap.cloud\-object\-storage.appdomain.cloud\[dq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
APAC Cross Regional Seoul Endpoint
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[dq]s3.private.ap.cloud\-object\-storage.appdomain.cloud\[dq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
APAC Cross Regional Private Endpoint
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[dq]s3.private.tok.ap.cloud\-object\-storage.appdomain.cloud\[dq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
APAC Cross Regional Tokyo Private Endpoint
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[dq]s3.private.hkg.ap.cloud\-object\-storage.appdomain.cloud\[dq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
APAC Cross Regional HongKong Private Endpoint
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[dq]s3.private.seo.ap.cloud\-object\-storage.appdomain.cloud\[dq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
APAC Cross Regional Seoul Private Endpoint
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[dq]s3.jp\-tok.cloud\-object\-storage.appdomain.cloud\[dq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
APAC Region Japan Endpoint
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[dq]s3.private.jp\-tok.cloud\-object\-storage.appdomain.cloud\[dq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
APAC Region Japan Private Endpoint
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[dq]s3.au\-syd.cloud\-object\-storage.appdomain.cloud\[dq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
APAC Region Australia Endpoint
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[dq]s3.private.au\-syd.cloud\-object\-storage.appdomain.cloud\[dq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
APAC Region Australia Private Endpoint
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[dq]s3.ams03.cloud\-object\-storage.appdomain.cloud\[dq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Amsterdam Single Site Endpoint
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[dq]s3.private.ams03.cloud\-object\-storage.appdomain.cloud\[dq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Amsterdam Single Site Private Endpoint
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[dq]s3.che01.cloud\-object\-storage.appdomain.cloud\[dq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Chennai Single Site Endpoint
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[dq]s3.private.che01.cloud\-object\-storage.appdomain.cloud\[dq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Chennai Single Site Private Endpoint
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[dq]s3.mel01.cloud\-object\-storage.appdomain.cloud\[dq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Melbourne Single Site Endpoint
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[dq]s3.private.mel01.cloud\-object\-storage.appdomain.cloud\[dq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Melbourne Single Site Private Endpoint
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[dq]s3.osl01.cloud\-object\-storage.appdomain.cloud\[dq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Oslo Single Site Endpoint
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[dq]s3.private.osl01.cloud\-object\-storage.appdomain.cloud\[dq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Oslo Single Site Private Endpoint
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[dq]s3.tor01.cloud\-object\-storage.appdomain.cloud\[dq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Toronto Single Site Endpoint
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[dq]s3.private.tor01.cloud\-object\-storage.appdomain.cloud\[dq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Toronto Single Site Private Endpoint
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[dq]s3.seo01.cloud\-object\-storage.appdomain.cloud\[dq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Seoul Single Site Endpoint
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[dq]s3.private.seo01.cloud\-object\-storage.appdomain.cloud\[dq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Seoul Single Site Private Endpoint
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[dq]s3.mon01.cloud\-object\-storage.appdomain.cloud\[dq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Montreal Single Site Endpoint
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[dq]s3.private.mon01.cloud\-object\-storage.appdomain.cloud\[dq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Montreal Single Site Private Endpoint
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[dq]s3.mex01.cloud\-object\-storage.appdomain.cloud\[dq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Mexico Single Site Endpoint
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[dq]s3.private.mex01.cloud\-object\-storage.appdomain.cloud\[dq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Mexico Single Site Private Endpoint
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[dq]s3.sjc04.cloud\-object\-storage.appdomain.cloud\[dq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
San Jose Single Site Endpoint
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[dq]s3.private.sjc04.cloud\-object\-storage.appdomain.cloud\[dq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
San Jose Single Site Private Endpoint
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[dq]s3.mil01.cloud\-object\-storage.appdomain.cloud\[dq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Milan Single Site Endpoint
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[dq]s3.private.mil01.cloud\-object\-storage.appdomain.cloud\[dq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Milan Single Site Private Endpoint
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[dq]s3.hkg02.cloud\-object\-storage.appdomain.cloud\[dq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Hong Kong Single Site Endpoint
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[dq]s3.private.hkg02.cloud\-object\-storage.appdomain.cloud\[dq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Hong Kong Single Site Private Endpoint
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[dq]s3.par01.cloud\-object\-storage.appdomain.cloud\[dq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Paris Single Site Endpoint
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[dq]s3.private.par01.cloud\-object\-storage.appdomain.cloud\[dq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Paris Single Site Private Endpoint
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[dq]s3.sng01.cloud\-object\-storage.appdomain.cloud\[dq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Singapore Single Site Endpoint
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[dq]s3.private.sng01.cloud\-object\-storage.appdomain.cloud\[dq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Singapore Single Site Private Endpoint
|
|
.RE
|
|
.RE
|
|
.SS \-\-s3\-endpoint
|
|
.PP
|
|
Endpoint for OSS API.
|
|
.IP \[bu] 2
|
|
Config: endpoint
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_S3_ENDPOINT
|
|
.IP \[bu] 2
|
|
Type: string
|
|
.IP \[bu] 2
|
|
Default: \[dq]\[dq]
|
|
.IP \[bu] 2
|
|
Examples:
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
\[dq]oss\-cn\-hangzhou.aliyuncs.com\[dq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
East China 1 (Hangzhou)
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[dq]oss\-cn\-shanghai.aliyuncs.com\[dq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
East China 2 (Shanghai)
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[dq]oss\-cn\-qingdao.aliyuncs.com\[dq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
North China 1 (Qingdao)
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[dq]oss\-cn\-beijing.aliyuncs.com\[dq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
North China 2 (Beijing)
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[dq]oss\-cn\-zhangjiakou.aliyuncs.com\[dq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
North China 3 (Zhangjiakou)
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[dq]oss\-cn\-huhehaote.aliyuncs.com\[dq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
North China 5 (Huhehaote)
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[dq]oss\-cn\-shenzhen.aliyuncs.com\[dq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
South China 1 (Shenzhen)
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[dq]oss\-cn\-hongkong.aliyuncs.com\[dq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Hong Kong (Hong Kong)
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[dq]oss\-us\-west\-1.aliyuncs.com\[dq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
US West 1 (Silicon Valley)
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[dq]oss\-us\-east\-1.aliyuncs.com\[dq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
US East 1 (Virginia)
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[dq]oss\-ap\-southeast\-1.aliyuncs.com\[dq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Southeast Asia Southeast 1 (Singapore)
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[dq]oss\-ap\-southeast\-2.aliyuncs.com\[dq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Asia Pacific Southeast 2 (Sydney)
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[dq]oss\-ap\-southeast\-3.aliyuncs.com\[dq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Southeast Asia Southeast 3 (Kuala Lumpur)
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[dq]oss\-ap\-southeast\-5.aliyuncs.com\[dq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Asia Pacific Southeast 5 (Jakarta)
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[dq]oss\-ap\-northeast\-1.aliyuncs.com\[dq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Asia Pacific Northeast 1 (Japan)
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[dq]oss\-ap\-south\-1.aliyuncs.com\[dq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Asia Pacific South 1 (Mumbai)
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[dq]oss\-eu\-central\-1.aliyuncs.com\[dq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Central Europe 1 (Frankfurt)
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[dq]oss\-eu\-west\-1.aliyuncs.com\[dq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
West Europe (London)
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[dq]oss\-me\-east\-1.aliyuncs.com\[dq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Middle East 1 (Dubai)
|
|
.RE
|
|
.RE
|
|
.SS \-\-s3\-endpoint
|
|
.PP
|
|
Endpoint for Scaleway Object Storage.
|
|
.IP \[bu] 2
|
|
Config: endpoint
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_S3_ENDPOINT
|
|
.IP \[bu] 2
|
|
Type: string
|
|
.IP \[bu] 2
|
|
Default: \[dq]\[dq]
|
|
.IP \[bu] 2
|
|
Examples:
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
\[dq]s3.nl\-ams.scw.cloud\[dq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Amsterdam Endpoint
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[dq]s3.fr\-par.scw.cloud\[dq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Paris Endpoint
|
|
.RE
|
|
.RE
|
|
.SS \-\-s3\-endpoint
|
|
.PP
|
|
Endpoint for StackPath Object Storage.
|
|
.IP \[bu] 2
|
|
Config: endpoint
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_S3_ENDPOINT
|
|
.IP \[bu] 2
|
|
Type: string
|
|
.IP \[bu] 2
|
|
Default: \[dq]\[dq]
|
|
.IP \[bu] 2
|
|
Examples:
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
\[dq]s3.us\-east\-2.stackpathstorage.com\[dq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
US East Endpoint
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[dq]s3.us\-west\-1.stackpathstorage.com\[dq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
US West Endpoint
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[dq]s3.eu\-central\-1.stackpathstorage.com\[dq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
EU Endpoint
|
|
.RE
|
|
.RE
|
|
.SS \-\-s3\-endpoint
|
|
.PP
|
|
Endpoint for Tencent COS API.
|
|
.IP \[bu] 2
|
|
Config: endpoint
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_S3_ENDPOINT
|
|
.IP \[bu] 2
|
|
Type: string
|
|
.IP \[bu] 2
|
|
Default: \[dq]\[dq]
|
|
.IP \[bu] 2
|
|
Examples:
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
\[dq]cos.ap\-beijing.myqcloud.com\[dq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Beijing Region.
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[dq]cos.ap\-nanjing.myqcloud.com\[dq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Nanjing Region.
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[dq]cos.ap\-shanghai.myqcloud.com\[dq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Shanghai Region.
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[dq]cos.ap\-guangzhou.myqcloud.com\[dq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Guangzhou Region.
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[dq]cos.ap\-nanjing.myqcloud.com\[dq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Nanjing Region.
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[dq]cos.ap\-chengdu.myqcloud.com\[dq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Chengdu Region.
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[dq]cos.ap\-chongqing.myqcloud.com\[dq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Chongqing Region.
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[dq]cos.ap\-hongkong.myqcloud.com\[dq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Hong Kong (China) Region.
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[dq]cos.ap\-singapore.myqcloud.com\[dq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Singapore Region.
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[dq]cos.ap\-mumbai.myqcloud.com\[dq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Mumbai Region.
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[dq]cos.ap\-seoul.myqcloud.com\[dq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Seoul Region.
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[dq]cos.ap\-bangkok.myqcloud.com\[dq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Bangkok Region.
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[dq]cos.ap\-tokyo.myqcloud.com\[dq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Tokyo Region.
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[dq]cos.na\-siliconvalley.myqcloud.com\[dq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Silicon Valley Region.
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[dq]cos.na\-ashburn.myqcloud.com\[dq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Virginia Region.
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[dq]cos.na\-toronto.myqcloud.com\[dq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Toronto Region.
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[dq]cos.eu\-frankfurt.myqcloud.com\[dq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Frankfurt Region.
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[dq]cos.eu\-moscow.myqcloud.com\[dq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Moscow Region.
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[dq]cos.accelerate.myqcloud.com\[dq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Use Tencent COS Accelerate Endpoint.
|
|
.RE
|
|
.RE
|
|
.SS \-\-s3\-endpoint
|
|
.PP
|
|
Endpoint for S3 API.
|
|
Required when using an S3 clone.
|
|
.IP \[bu] 2
|
|
Config: endpoint
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_S3_ENDPOINT
|
|
.IP \[bu] 2
|
|
Type: string
|
|
.IP \[bu] 2
|
|
Default: \[dq]\[dq]
|
|
.IP \[bu] 2
|
|
Examples:
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
\[dq]objects\-us\-east\-1.dream.io\[dq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Dream Objects endpoint
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[dq]nyc3.digitaloceanspaces.com\[dq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Digital Ocean Spaces New York 3
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[dq]ams3.digitaloceanspaces.com\[dq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Digital Ocean Spaces Amsterdam 3
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[dq]sgp1.digitaloceanspaces.com\[dq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Digital Ocean Spaces Singapore 1
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[dq]s3.wasabisys.com\[dq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Wasabi US East endpoint
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[dq]s3.us\-west\-1.wasabisys.com\[dq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Wasabi US West endpoint
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[dq]s3.eu\-central\-1.wasabisys.com\[dq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Wasabi EU Central endpoint
|
|
.RE
|
|
.RE
|
|
.SS \-\-s3\-location\-constraint
|
|
.PP
|
|
Location constraint \- must be set to match the Region.
|
|
Used when creating buckets only.
|
|
.IP \[bu] 2
|
|
Config: location_constraint
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_S3_LOCATION_CONSTRAINT
|
|
.IP \[bu] 2
|
|
Type: string
|
|
.IP \[bu] 2
|
|
Default: \[dq]\[dq]
|
|
.IP \[bu] 2
|
|
Examples:
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
\[dq]\[dq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Empty for US Region, Northern Virginia, or Pacific Northwest.
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[dq]us\-east\-2\[dq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
US East (Ohio) Region.
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[dq]us\-west\-1\[dq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
US West (Northern California) Region.
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[dq]us\-west\-2\[dq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
US West (Oregon) Region.
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[dq]ca\-central\-1\[dq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Canada (Central) Region.
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[dq]eu\-west\-1\[dq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
EU (Ireland) Region.
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[dq]eu\-west\-2\[dq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
EU (London) Region.
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[dq]eu\-west\-3\[dq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
EU (Paris) Region.
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[dq]eu\-north\-1\[dq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
EU (Stockholm) Region.
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[dq]eu\-south\-1\[dq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
EU (Milan) Region.
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[dq]EU\[dq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
EU Region.
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[dq]ap\-southeast\-1\[dq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Asia Pacific (Singapore) Region.
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[dq]ap\-southeast\-2\[dq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Asia Pacific (Sydney) Region.
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[dq]ap\-northeast\-1\[dq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Asia Pacific (Tokyo) Region.
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[dq]ap\-northeast\-2\[dq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Asia Pacific (Seoul) Region.
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[dq]ap\-northeast\-3\[dq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Asia Pacific (Osaka\-Local) Region.
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[dq]ap\-south\-1\[dq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Asia Pacific (Mumbai) Region.
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[dq]ap\-east\-1\[dq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Asia Pacific (Hong Kong) Region.
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[dq]sa\-east\-1\[dq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
South America (Sao Paulo) Region.
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[dq]me\-south\-1\[dq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Middle East (Bahrain) Region.
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[dq]af\-south\-1\[dq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Africa (Cape Town) Region.
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[dq]cn\-north\-1\[dq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
China (Beijing) Region
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[dq]cn\-northwest\-1\[dq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
China (Ningxia) Region.
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[dq]us\-gov\-east\-1\[dq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
AWS GovCloud (US\-East) Region.
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[dq]us\-gov\-west\-1\[dq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
AWS GovCloud (US) Region.
|
|
.RE
|
|
.RE
|
|
.SS \-\-s3\-location\-constraint
|
|
.PP
|
|
Location constraint \- must match endpoint when using IBM Cloud Public.
|
|
For on\-prem COS, do not make a selection from this list, hit enter
|
|
.IP \[bu] 2
|
|
Config: location_constraint
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_S3_LOCATION_CONSTRAINT
|
|
.IP \[bu] 2
|
|
Type: string
|
|
.IP \[bu] 2
|
|
Default: \[dq]\[dq]
|
|
.IP \[bu] 2
|
|
Examples:
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
\[dq]us\-standard\[dq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
US Cross Region Standard
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[dq]us\-vault\[dq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
US Cross Region Vault
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[dq]us\-cold\[dq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
US Cross Region Cold
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[dq]us\-flex\[dq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
US Cross Region Flex
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[dq]us\-east\-standard\[dq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
US East Region Standard
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[dq]us\-east\-vault\[dq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
US East Region Vault
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[dq]us\-east\-cold\[dq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
US East Region Cold
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[dq]us\-east\-flex\[dq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
US East Region Flex
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[dq]us\-south\-standard\[dq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
US South Region Standard
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[dq]us\-south\-vault\[dq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
US South Region Vault
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[dq]us\-south\-cold\[dq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
US South Region Cold
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[dq]us\-south\-flex\[dq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
US South Region Flex
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[dq]eu\-standard\[dq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
EU Cross Region Standard
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[dq]eu\-vault\[dq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
EU Cross Region Vault
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[dq]eu\-cold\[dq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
EU Cross Region Cold
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[dq]eu\-flex\[dq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
EU Cross Region Flex
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[dq]eu\-gb\-standard\[dq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Great Britain Standard
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[dq]eu\-gb\-vault\[dq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Great Britain Vault
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[dq]eu\-gb\-cold\[dq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Great Britain Cold
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[dq]eu\-gb\-flex\[dq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Great Britain Flex
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[dq]ap\-standard\[dq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
APAC Standard
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[dq]ap\-vault\[dq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
APAC Vault
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[dq]ap\-cold\[dq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
APAC Cold
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[dq]ap\-flex\[dq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
APAC Flex
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[dq]mel01\-standard\[dq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Melbourne Standard
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[dq]mel01\-vault\[dq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Melbourne Vault
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[dq]mel01\-cold\[dq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Melbourne Cold
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[dq]mel01\-flex\[dq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Melbourne Flex
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[dq]tor01\-standard\[dq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Toronto Standard
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[dq]tor01\-vault\[dq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Toronto Vault
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[dq]tor01\-cold\[dq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Toronto Cold
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[dq]tor01\-flex\[dq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Toronto Flex
|
|
.RE
|
|
.RE
|
|
.SS \-\-s3\-location\-constraint
|
|
.PP
|
|
Location constraint \- must be set to match the Region.
|
|
Leave blank if not sure.
|
|
Used when creating buckets only.
|
|
.IP \[bu] 2
|
|
Config: location_constraint
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_S3_LOCATION_CONSTRAINT
|
|
.IP \[bu] 2
|
|
Type: string
|
|
.IP \[bu] 2
|
|
Default: \[dq]\[dq]
|
|
.SS \-\-s3\-acl
|
|
.PP
|
|
Canned ACL used when creating buckets and storing or copying objects.
|
|
.PP
|
|
This ACL is used for creating objects and if bucket_acl isn\[aq]t set,
|
|
for creating buckets too.
|
|
.PP
|
|
For more info visit
|
|
https://docs.aws.amazon.com/AmazonS3/latest/dev/acl\-overview.html#canned\-acl
|
|
.PP
|
|
Note that this ACL is applied when server\-side copying objects as S3
|
|
doesn\[aq]t copy the ACL from the source but rather writes a fresh one.
|
|
.IP \[bu] 2
|
|
Config: acl
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_S3_ACL
|
|
.IP \[bu] 2
|
|
Type: string
|
|
.IP \[bu] 2
|
|
Default: \[dq]\[dq]
|
|
.IP \[bu] 2
|
|
Examples:
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
\[dq]default\[dq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Owner gets Full_CONTROL.
|
|
No one else has access rights (default).
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[dq]private\[dq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Owner gets FULL_CONTROL.
|
|
No one else has access rights (default).
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[dq]public\-read\[dq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Owner gets FULL_CONTROL.
|
|
The AllUsers group gets READ access.
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[dq]public\-read\-write\[dq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Owner gets FULL_CONTROL.
|
|
The AllUsers group gets READ and WRITE access.
|
|
.IP \[bu] 2
|
|
Granting this on a bucket is generally not recommended.
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[dq]authenticated\-read\[dq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Owner gets FULL_CONTROL.
|
|
The AuthenticatedUsers group gets READ access.
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[dq]bucket\-owner\-read\[dq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Object owner gets FULL_CONTROL.
|
|
Bucket owner gets READ access.
|
|
.IP \[bu] 2
|
|
If you specify this canned ACL when creating a bucket, Amazon S3 ignores
|
|
it.
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[dq]bucket\-owner\-full\-control\[dq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Both the object owner and the bucket owner get FULL_CONTROL over the
|
|
object.
|
|
.IP \[bu] 2
|
|
If you specify this canned ACL when creating a bucket, Amazon S3 ignores
|
|
it.
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[dq]private\[dq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Owner gets FULL_CONTROL.
|
|
No one else has access rights (default).
|
|
This acl is available on IBM Cloud (Infra), IBM Cloud (Storage),
|
|
On\-Premise COS
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[dq]public\-read\[dq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Owner gets FULL_CONTROL.
|
|
The AllUsers group gets READ access.
|
|
This acl is available on IBM Cloud (Infra), IBM Cloud (Storage),
|
|
On\-Premise IBM COS
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[dq]public\-read\-write\[dq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Owner gets FULL_CONTROL.
|
|
The AllUsers group gets READ and WRITE access.
|
|
This acl is available on IBM Cloud (Infra), On\-Premise IBM COS
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[dq]authenticated\-read\[dq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Owner gets FULL_CONTROL.
|
|
The AuthenticatedUsers group gets READ access.
|
|
Not supported on Buckets.
|
|
This acl is available on IBM Cloud (Infra) and On\-Premise IBM COS
|
|
.RE
|
|
.RE
|
|
.SS \-\-s3\-server\-side\-encryption
|
|
.PP
|
|
The server\-side encryption algorithm used when storing this object in
|
|
S3.
|
|
.IP \[bu] 2
|
|
Config: server_side_encryption
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_S3_SERVER_SIDE_ENCRYPTION
|
|
.IP \[bu] 2
|
|
Type: string
|
|
.IP \[bu] 2
|
|
Default: \[dq]\[dq]
|
|
.IP \[bu] 2
|
|
Examples:
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
\[dq]\[dq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
None
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[dq]AES256\[dq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
AES256
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[dq]aws:kms\[dq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
aws:kms
|
|
.RE
|
|
.RE
|
|
.SS \-\-s3\-sse\-kms\-key\-id
|
|
.PP
|
|
If using KMS ID you must provide the ARN of Key.
|
|
.IP \[bu] 2
|
|
Config: sse_kms_key_id
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_S3_SSE_KMS_KEY_ID
|
|
.IP \[bu] 2
|
|
Type: string
|
|
.IP \[bu] 2
|
|
Default: \[dq]\[dq]
|
|
.IP \[bu] 2
|
|
Examples:
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
\[dq]\[dq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
None
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[dq]arn:aws:kms:us\-east\-1:*\[dq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
arn:aws:kms:*
|
|
.RE
|
|
.RE
|
|
.SS \-\-s3\-storage\-class
|
|
.PP
|
|
The storage class to use when storing new objects in S3.
|
|
.IP \[bu] 2
|
|
Config: storage_class
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_S3_STORAGE_CLASS
|
|
.IP \[bu] 2
|
|
Type: string
|
|
.IP \[bu] 2
|
|
Default: \[dq]\[dq]
|
|
.IP \[bu] 2
|
|
Examples:
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
\[dq]\[dq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Default
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[dq]STANDARD\[dq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Standard storage class
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[dq]REDUCED_REDUNDANCY\[dq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Reduced redundancy storage class
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[dq]STANDARD_IA\[dq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Standard Infrequent Access storage class
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[dq]ONEZONE_IA\[dq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
One Zone Infrequent Access storage class
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[dq]GLACIER\[dq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Glacier storage class
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[dq]DEEP_ARCHIVE\[dq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Glacier Deep Archive storage class
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[dq]INTELLIGENT_TIERING\[dq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Intelligent\-Tiering storage class
|
|
.RE
|
|
.RE
|
|
.SS \-\-s3\-storage\-class
|
|
.PP
|
|
The storage class to use when storing new objects in OSS.
|
|
.IP \[bu] 2
|
|
Config: storage_class
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_S3_STORAGE_CLASS
|
|
.IP \[bu] 2
|
|
Type: string
|
|
.IP \[bu] 2
|
|
Default: \[dq]\[dq]
|
|
.IP \[bu] 2
|
|
Examples:
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
\[dq]\[dq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Default
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[dq]STANDARD\[dq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Standard storage class
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[dq]GLACIER\[dq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Archive storage mode.
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[dq]STANDARD_IA\[dq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Infrequent access storage mode.
|
|
.RE
|
|
.RE
|
|
.SS \-\-s3\-storage\-class
|
|
.PP
|
|
The storage class to use when storing new objects in Tencent COS.
|
|
.IP \[bu] 2
|
|
Config: storage_class
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_S3_STORAGE_CLASS
|
|
.IP \[bu] 2
|
|
Type: string
|
|
.IP \[bu] 2
|
|
Default: \[dq]\[dq]
|
|
.IP \[bu] 2
|
|
Examples:
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
\[dq]\[dq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Default
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[dq]STANDARD\[dq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Standard storage class
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[dq]ARCHIVE\[dq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Archive storage mode.
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[dq]STANDARD_IA\[dq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Infrequent access storage mode.
|
|
.RE
|
|
.RE
|
|
.SS \-\-s3\-storage\-class
|
|
.PP
|
|
The storage class to use when storing new objects in S3.
|
|
.IP \[bu] 2
|
|
Config: storage_class
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_S3_STORAGE_CLASS
|
|
.IP \[bu] 2
|
|
Type: string
|
|
.IP \[bu] 2
|
|
Default: \[dq]\[dq]
|
|
.IP \[bu] 2
|
|
Examples:
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
\[dq]\[dq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Default
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[dq]STANDARD\[dq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
The Standard class for any upload; suitable for on\-demand content like
|
|
streaming or CDN.
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[dq]GLACIER\[dq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Archived storage; prices are lower, but it needs to be restored first to
|
|
be accessed.
|
|
.RE
|
|
.RE
|
|
.SS Advanced Options
|
|
.PP
|
|
Here are the advanced options specific to s3 (Amazon S3 Compliant
|
|
Storage Providers including AWS, Alibaba, Ceph, Digital Ocean,
|
|
Dreamhost, IBM COS, Minio, and Tencent COS).
|
|
.SS \-\-s3\-bucket\-acl
|
|
.PP
|
|
Canned ACL used when creating buckets.
|
|
.PP
|
|
For more info visit
|
|
https://docs.aws.amazon.com/AmazonS3/latest/dev/acl\-overview.html#canned\-acl
|
|
.PP
|
|
Note that this ACL is applied when only when creating buckets.
|
|
If it isn\[aq]t set then \[dq]acl\[dq] is used instead.
|
|
.IP \[bu] 2
|
|
Config: bucket_acl
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_S3_BUCKET_ACL
|
|
.IP \[bu] 2
|
|
Type: string
|
|
.IP \[bu] 2
|
|
Default: \[dq]\[dq]
|
|
.IP \[bu] 2
|
|
Examples:
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
\[dq]private\[dq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Owner gets FULL_CONTROL.
|
|
No one else has access rights (default).
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[dq]public\-read\[dq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Owner gets FULL_CONTROL.
|
|
The AllUsers group gets READ access.
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[dq]public\-read\-write\[dq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Owner gets FULL_CONTROL.
|
|
The AllUsers group gets READ and WRITE access.
|
|
.IP \[bu] 2
|
|
Granting this on a bucket is generally not recommended.
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[dq]authenticated\-read\[dq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Owner gets FULL_CONTROL.
|
|
The AuthenticatedUsers group gets READ access.
|
|
.RE
|
|
.RE
|
|
.SS \-\-s3\-requester\-pays
|
|
.PP
|
|
Enables requester pays option when interacting with S3 bucket.
|
|
.IP \[bu] 2
|
|
Config: requester_pays
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_S3_REQUESTER_PAYS
|
|
.IP \[bu] 2
|
|
Type: bool
|
|
.IP \[bu] 2
|
|
Default: false
|
|
.SS \-\-s3\-sse\-customer\-algorithm
|
|
.PP
|
|
If using SSE\-C, the server\-side encryption algorithm used when storing
|
|
this object in S3.
|
|
.IP \[bu] 2
|
|
Config: sse_customer_algorithm
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_S3_SSE_CUSTOMER_ALGORITHM
|
|
.IP \[bu] 2
|
|
Type: string
|
|
.IP \[bu] 2
|
|
Default: \[dq]\[dq]
|
|
.IP \[bu] 2
|
|
Examples:
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
\[dq]\[dq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
None
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[dq]AES256\[dq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
AES256
|
|
.RE
|
|
.RE
|
|
.SS \-\-s3\-sse\-customer\-key
|
|
.PP
|
|
If using SSE\-C you must provide the secret encryption key used to
|
|
encrypt/decrypt your data.
|
|
.IP \[bu] 2
|
|
Config: sse_customer_key
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_S3_SSE_CUSTOMER_KEY
|
|
.IP \[bu] 2
|
|
Type: string
|
|
.IP \[bu] 2
|
|
Default: \[dq]\[dq]
|
|
.IP \[bu] 2
|
|
Examples:
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
\[dq]\[dq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
None
|
|
.RE
|
|
.RE
|
|
.SS \-\-s3\-sse\-customer\-key\-md5
|
|
.PP
|
|
If using SSE\-C you may provide the secret encryption key MD5 checksum
|
|
(optional).
|
|
.PP
|
|
If you leave it blank, this is calculated automatically from the
|
|
sse_customer_key provided.
|
|
.IP \[bu] 2
|
|
Config: sse_customer_key_md5
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_S3_SSE_CUSTOMER_KEY_MD5
|
|
.IP \[bu] 2
|
|
Type: string
|
|
.IP \[bu] 2
|
|
Default: \[dq]\[dq]
|
|
.IP \[bu] 2
|
|
Examples:
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
\[dq]\[dq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
None
|
|
.RE
|
|
.RE
|
|
.SS \-\-s3\-upload\-cutoff
|
|
.PP
|
|
Cutoff for switching to chunked upload
|
|
.PP
|
|
Any files larger than this will be uploaded in chunks of chunk_size.
|
|
The minimum is 0 and the maximum is 5GB.
|
|
.IP \[bu] 2
|
|
Config: upload_cutoff
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_S3_UPLOAD_CUTOFF
|
|
.IP \[bu] 2
|
|
Type: SizeSuffix
|
|
.IP \[bu] 2
|
|
Default: 200M
|
|
.SS \-\-s3\-chunk\-size
|
|
.PP
|
|
Chunk size to use for uploading.
|
|
.PP
|
|
When uploading files larger than upload_cutoff or files with unknown
|
|
size (e.g.
|
|
from \[dq]rclone rcat\[dq] or uploaded with \[dq]rclone mount\[dq] or
|
|
google photos or google docs) they will be uploaded as multipart uploads
|
|
using this chunk size.
|
|
.PP
|
|
Note that \[dq]\-\-s3\-upload\-concurrency\[dq] chunks of this size are
|
|
buffered in memory per transfer.
|
|
.PP
|
|
If you are transferring large files over high\-speed links and you have
|
|
enough memory, then increasing this will speed up the transfers.
|
|
.PP
|
|
Rclone will automatically increase the chunk size when uploading a large
|
|
file of known size to stay below the 10,000 chunks limit.
|
|
.PP
|
|
Files of unknown size are uploaded with the configured chunk_size.
|
|
Since the default chunk size is 5MB and there can be at most 10,000
|
|
chunks, this means that by default the maximum size of a file you can
|
|
stream upload is 48GB.
|
|
If you wish to stream upload larger files then you will need to increase
|
|
chunk_size.
|
|
.IP \[bu] 2
|
|
Config: chunk_size
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_S3_CHUNK_SIZE
|
|
.IP \[bu] 2
|
|
Type: SizeSuffix
|
|
.IP \[bu] 2
|
|
Default: 5M
|
|
.SS \-\-s3\-max\-upload\-parts
|
|
.PP
|
|
Maximum number of parts in a multipart upload.
|
|
.PP
|
|
This option defines the maximum number of multipart chunks to use when
|
|
doing a multipart upload.
|
|
.PP
|
|
This can be useful if a service does not support the AWS S3
|
|
specification of 10,000 chunks.
|
|
.PP
|
|
Rclone will automatically increase the chunk size when uploading a large
|
|
file of a known size to stay below this number of chunks limit.
|
|
.IP \[bu] 2
|
|
Config: max_upload_parts
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_S3_MAX_UPLOAD_PARTS
|
|
.IP \[bu] 2
|
|
Type: int
|
|
.IP \[bu] 2
|
|
Default: 10000
|
|
.SS \-\-s3\-copy\-cutoff
|
|
.PP
|
|
Cutoff for switching to multipart copy
|
|
.PP
|
|
Any files larger than this that need to be server\-side copied will be
|
|
copied in chunks of this size.
|
|
.PP
|
|
The minimum is 0 and the maximum is 5GB.
|
|
.IP \[bu] 2
|
|
Config: copy_cutoff
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_S3_COPY_CUTOFF
|
|
.IP \[bu] 2
|
|
Type: SizeSuffix
|
|
.IP \[bu] 2
|
|
Default: 4.656G
|
|
.SS \-\-s3\-disable\-checksum
|
|
.PP
|
|
Don\[aq]t store MD5 checksum with object metadata
|
|
.PP
|
|
Normally rclone will calculate the MD5 checksum of the input before
|
|
uploading it so it can add it to metadata on the object.
|
|
This is great for data integrity checking but can cause long delays for
|
|
large files to start uploading.
|
|
.IP \[bu] 2
|
|
Config: disable_checksum
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_S3_DISABLE_CHECKSUM
|
|
.IP \[bu] 2
|
|
Type: bool
|
|
.IP \[bu] 2
|
|
Default: false
|
|
.SS \-\-s3\-shared\-credentials\-file
|
|
.PP
|
|
Path to the shared credentials file
|
|
.PP
|
|
If env_auth = true then rclone can use a shared credentials file.
|
|
.PP
|
|
If this variable is empty rclone will look for the
|
|
\[dq]AWS_SHARED_CREDENTIALS_FILE\[dq] env variable.
|
|
If the env value is empty it will default to the current user\[aq]s home
|
|
directory.
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
Linux/OSX: \[dq]$HOME/.aws/credentials\[dq]
|
|
Windows: \[dq]%USERPROFILE%\[rs].aws\[rs]credentials\[dq]
|
|
\f[R]
|
|
.fi
|
|
.IP \[bu] 2
|
|
Config: shared_credentials_file
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_S3_SHARED_CREDENTIALS_FILE
|
|
.IP \[bu] 2
|
|
Type: string
|
|
.IP \[bu] 2
|
|
Default: \[dq]\[dq]
|
|
.SS \-\-s3\-profile
|
|
.PP
|
|
Profile to use in the shared credentials file
|
|
.PP
|
|
If env_auth = true then rclone can use a shared credentials file.
|
|
This variable controls which profile is used in that file.
|
|
.PP
|
|
If empty it will default to the environment variable
|
|
\[dq]AWS_PROFILE\[dq] or \[dq]default\[dq] if that environment variable
|
|
is also not set.
|
|
.IP \[bu] 2
|
|
Config: profile
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_S3_PROFILE
|
|
.IP \[bu] 2
|
|
Type: string
|
|
.IP \[bu] 2
|
|
Default: \[dq]\[dq]
|
|
.SS \-\-s3\-session\-token
|
|
.PP
|
|
An AWS session token
|
|
.IP \[bu] 2
|
|
Config: session_token
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_S3_SESSION_TOKEN
|
|
.IP \[bu] 2
|
|
Type: string
|
|
.IP \[bu] 2
|
|
Default: \[dq]\[dq]
|
|
.SS \-\-s3\-upload\-concurrency
|
|
.PP
|
|
Concurrency for multipart uploads.
|
|
.PP
|
|
This is the number of chunks of the same file that are uploaded
|
|
concurrently.
|
|
.PP
|
|
If you are uploading small numbers of large files over high\-speed links
|
|
and these uploads do not fully utilize your bandwidth, then increasing
|
|
this may help to speed up the transfers.
|
|
.IP \[bu] 2
|
|
Config: upload_concurrency
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_S3_UPLOAD_CONCURRENCY
|
|
.IP \[bu] 2
|
|
Type: int
|
|
.IP \[bu] 2
|
|
Default: 4
|
|
.SS \-\-s3\-force\-path\-style
|
|
.PP
|
|
If true use path style access if false use virtual hosted style.
|
|
.PP
|
|
If this is true (the default) then rclone will use path style access, if
|
|
false then rclone will use virtual path style.
|
|
See the AWS S3
|
|
docs (https://docs.aws.amazon.com/AmazonS3/latest/dev/UsingBucket.html#access-bucket-intro)
|
|
for more info.
|
|
.PP
|
|
Some providers (e.g.
|
|
AWS, Aliyun OSS, Netease COS, or Tencent COS) require this set to false
|
|
\- rclone will do this automatically based on the provider setting.
|
|
.IP \[bu] 2
|
|
Config: force_path_style
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_S3_FORCE_PATH_STYLE
|
|
.IP \[bu] 2
|
|
Type: bool
|
|
.IP \[bu] 2
|
|
Default: true
|
|
.SS \-\-s3\-v2\-auth
|
|
.PP
|
|
If true use v2 authentication.
|
|
.PP
|
|
If this is false (the default) then rclone will use v4 authentication.
|
|
If it is set then rclone will use v2 authentication.
|
|
.PP
|
|
Use this only if v4 signatures don\[aq]t work, e.g.
|
|
pre Jewel/v10 CEPH.
|
|
.IP \[bu] 2
|
|
Config: v2_auth
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_S3_V2_AUTH
|
|
.IP \[bu] 2
|
|
Type: bool
|
|
.IP \[bu] 2
|
|
Default: false
|
|
.SS \-\-s3\-use\-accelerate\-endpoint
|
|
.PP
|
|
If true use the AWS S3 accelerated endpoint.
|
|
.PP
|
|
See: AWS S3 Transfer
|
|
acceleration (https://docs.aws.amazon.com/AmazonS3/latest/dev/transfer-acceleration-examples.html)
|
|
.IP \[bu] 2
|
|
Config: use_accelerate_endpoint
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_S3_USE_ACCELERATE_ENDPOINT
|
|
.IP \[bu] 2
|
|
Type: bool
|
|
.IP \[bu] 2
|
|
Default: false
|
|
.SS \-\-s3\-leave\-parts\-on\-error
|
|
.PP
|
|
If true avoid calling abort upload on a failure, leaving all
|
|
successfully uploaded parts on S3 for manual recovery.
|
|
.PP
|
|
It should be set to true for resuming uploads across different sessions.
|
|
.PP
|
|
WARNING: Storing parts of an incomplete multipart upload counts towards
|
|
space usage on S3 and will add additional costs if not cleaned up.
|
|
.IP \[bu] 2
|
|
Config: leave_parts_on_error
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_S3_LEAVE_PARTS_ON_ERROR
|
|
.IP \[bu] 2
|
|
Type: bool
|
|
.IP \[bu] 2
|
|
Default: false
|
|
.SS \-\-s3\-list\-chunk
|
|
.PP
|
|
Size of listing chunk (response list for each ListObject S3 request).
|
|
.PP
|
|
This option is also known as \[dq]MaxKeys\[dq], \[dq]max\-items\[dq], or
|
|
\[dq]page\-size\[dq] from the AWS S3 specification.
|
|
Most services truncate the response list to 1000 objects even if
|
|
requested more than that.
|
|
In AWS S3 this is a global maximum and cannot be changed, see AWS
|
|
S3 (https://docs.aws.amazon.com/cli/latest/reference/s3/ls.html).
|
|
In Ceph, this can be increased with the \[dq]rgw list buckets max
|
|
chunk\[dq] option.
|
|
.IP \[bu] 2
|
|
Config: list_chunk
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_S3_LIST_CHUNK
|
|
.IP \[bu] 2
|
|
Type: int
|
|
.IP \[bu] 2
|
|
Default: 1000
|
|
.SS \-\-s3\-no\-check\-bucket
|
|
.PP
|
|
If set, don\[aq]t attempt to check the bucket exists or create it
|
|
.PP
|
|
This can be useful when trying to minimise the number of transactions
|
|
rclone does if you know the bucket exists already.
|
|
.PP
|
|
It can also be needed if the user you are using does not have bucket
|
|
creation permissions.
|
|
Before v1.52.0 this would have passed silently due to a bug.
|
|
.IP \[bu] 2
|
|
Config: no_check_bucket
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_S3_NO_CHECK_BUCKET
|
|
.IP \[bu] 2
|
|
Type: bool
|
|
.IP \[bu] 2
|
|
Default: false
|
|
.SS \-\-s3\-no\-head
|
|
.PP
|
|
If set, don\[aq]t HEAD uploaded objects to check integrity
|
|
.PP
|
|
This can be useful when trying to minimise the number of transactions
|
|
rclone does.
|
|
.PP
|
|
Setting it means that if rclone receives a 200 OK message after
|
|
uploading an object with PUT then it will assume that it got uploaded
|
|
properly.
|
|
.PP
|
|
In particular it will assume:
|
|
.IP \[bu] 2
|
|
the metadata, including modtime, storage class and content type was as
|
|
uploaded
|
|
.IP \[bu] 2
|
|
the size was as uploaded
|
|
.PP
|
|
It reads the following items from the response for a single part PUT:
|
|
.IP \[bu] 2
|
|
the MD5SUM
|
|
.IP \[bu] 2
|
|
The uploaded date
|
|
.PP
|
|
For multipart uploads these items aren\[aq]t read.
|
|
.PP
|
|
If an source object of unknown length is uploaded then rclone
|
|
\f[B]will\f[R] do a HEAD request.
|
|
.PP
|
|
Setting this flag increases the chance for undetected upload failures,
|
|
in particular an incorrect size, so it isn\[aq]t recommended for normal
|
|
operation.
|
|
In practice the chance of an undetected upload failure is very small
|
|
even with this flag.
|
|
.IP \[bu] 2
|
|
Config: no_head
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_S3_NO_HEAD
|
|
.IP \[bu] 2
|
|
Type: bool
|
|
.IP \[bu] 2
|
|
Default: false
|
|
.SS \-\-s3\-encoding
|
|
.PP
|
|
This sets the encoding for the backend.
|
|
.PP
|
|
See: the encoding section in the
|
|
overview (https://rclone.org/overview/#encoding) for more info.
|
|
.IP \[bu] 2
|
|
Config: encoding
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_S3_ENCODING
|
|
.IP \[bu] 2
|
|
Type: MultiEncoder
|
|
.IP \[bu] 2
|
|
Default: Slash,InvalidUtf8,Dot
|
|
.SS \-\-s3\-memory\-pool\-flush\-time
|
|
.PP
|
|
How often internal memory buffer pools will be flushed.
|
|
Uploads which requires additional buffers (f.e multipart) will use
|
|
memory pool for allocations.
|
|
This option controls how often unused buffers will be removed from the
|
|
pool.
|
|
.IP \[bu] 2
|
|
Config: memory_pool_flush_time
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_S3_MEMORY_POOL_FLUSH_TIME
|
|
.IP \[bu] 2
|
|
Type: Duration
|
|
.IP \[bu] 2
|
|
Default: 1m0s
|
|
.SS \-\-s3\-memory\-pool\-use\-mmap
|
|
.PP
|
|
Whether to use mmap buffers in internal memory pool.
|
|
.IP \[bu] 2
|
|
Config: memory_pool_use_mmap
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_S3_MEMORY_POOL_USE_MMAP
|
|
.IP \[bu] 2
|
|
Type: bool
|
|
.IP \[bu] 2
|
|
Default: false
|
|
.SS \-\-s3\-disable\-http2
|
|
.PP
|
|
Disable usage of http2 for S3 backends
|
|
.PP
|
|
There is currently an unsolved issue with the s3 (specifically minio)
|
|
backend and HTTP/2.
|
|
HTTP/2 is enabled by default for the s3 backend but can be disabled
|
|
here.
|
|
When the issue is solved this flag will be removed.
|
|
.PP
|
|
See: https://github.com/rclone/rclone/issues/4673,
|
|
https://github.com/rclone/rclone/issues/3631
|
|
.IP \[bu] 2
|
|
Config: disable_http2
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_S3_DISABLE_HTTP2
|
|
.IP \[bu] 2
|
|
Type: bool
|
|
.IP \[bu] 2
|
|
Default: false
|
|
.SS Backend commands
|
|
.PP
|
|
Here are the commands specific to the s3 backend.
|
|
.PP
|
|
Run them with
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone backend COMMAND remote:
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
The help below will explain what arguments each command takes.
|
|
.PP
|
|
See the \[dq]rclone backend\[dq]
|
|
command (https://rclone.org/commands/rclone_backend/) for more info on
|
|
how to pass options and arguments.
|
|
.PP
|
|
These can be run on a running backend using the rc command
|
|
backend/command (https://rclone.org/rc/#backend/command).
|
|
.SS restore
|
|
.PP
|
|
Restore objects from GLACIER to normal storage
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone backend restore remote: [options] [<arguments>+]
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
This command can be used to restore one or more objects from GLACIER to
|
|
normal storage.
|
|
.PP
|
|
Usage Examples:
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone backend restore s3:bucket/path/to/object [\-o priority=PRIORITY] [\-o lifetime=DAYS]
|
|
rclone backend restore s3:bucket/path/to/directory [\-o priority=PRIORITY] [\-o lifetime=DAYS]
|
|
rclone backend restore s3:bucket [\-o priority=PRIORITY] [\-o lifetime=DAYS]
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
This flag also obeys the filters.
|
|
Test first with \-i/\-\-interactive or \-\-dry\-run flags
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone \-i backend restore \-\-include \[dq]*.txt\[dq] s3:bucket/path \-o priority=Standard
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
All the objects shown will be marked for restore, then
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone backend restore \-\-include \[dq]*.txt\[dq] s3:bucket/path \-o priority=Standard
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
It returns a list of status dictionaries with Remote and Status keys.
|
|
The Status will be OK if it was successful or an error message if not.
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
[
|
|
{
|
|
\[dq]Status\[dq]: \[dq]OK\[dq],
|
|
\[dq]Path\[dq]: \[dq]test.txt\[dq]
|
|
},
|
|
{
|
|
\[dq]Status\[dq]: \[dq]OK\[dq],
|
|
\[dq]Path\[dq]: \[dq]test/file4.txt\[dq]
|
|
}
|
|
]
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
Options:
|
|
.IP \[bu] 2
|
|
\[dq]description\[dq]: The optional description for the job.
|
|
.IP \[bu] 2
|
|
\[dq]lifetime\[dq]: Lifetime of the active copy in days
|
|
.IP \[bu] 2
|
|
\[dq]priority\[dq]: Priority of restore: Standard|Expedited|Bulk
|
|
.SS list\-multipart\-uploads
|
|
.PP
|
|
List the unfinished multipart uploads
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone backend list\-multipart\-uploads remote: [options] [<arguments>+]
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
This command lists the unfinished multipart uploads in JSON format.
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone backend list\-multipart s3:bucket/path/to/object
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
It returns a dictionary of buckets with values as lists of unfinished
|
|
multipart uploads.
|
|
.PP
|
|
You can call it with no bucket in which case it lists all bucket, with a
|
|
bucket or with a bucket and path.
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
{
|
|
\[dq]rclone\[dq]: [
|
|
{
|
|
\[dq]Initiated\[dq]: \[dq]2020\-06\-26T14:20:36Z\[dq],
|
|
\[dq]Initiator\[dq]: {
|
|
\[dq]DisplayName\[dq]: \[dq]XXX\[dq],
|
|
\[dq]ID\[dq]: \[dq]arn:aws:iam::XXX:user/XXX\[dq]
|
|
},
|
|
\[dq]Key\[dq]: \[dq]KEY\[dq],
|
|
\[dq]Owner\[dq]: {
|
|
\[dq]DisplayName\[dq]: null,
|
|
\[dq]ID\[dq]: \[dq]XXX\[dq]
|
|
},
|
|
\[dq]StorageClass\[dq]: \[dq]STANDARD\[dq],
|
|
\[dq]UploadId\[dq]: \[dq]XXX\[dq]
|
|
}
|
|
],
|
|
\[dq]rclone\-1000files\[dq]: [],
|
|
\[dq]rclone\-dst\[dq]: []
|
|
}
|
|
\f[R]
|
|
.fi
|
|
.SS cleanup
|
|
.PP
|
|
Remove unfinished multipart uploads.
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone backend cleanup remote: [options] [<arguments>+]
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
This command removes unfinished multipart uploads of age greater than
|
|
max\-age which defaults to 24 hours.
|
|
.PP
|
|
Note that you can use \-i/\-\-dry\-run with this command to see what it
|
|
would do.
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone backend cleanup s3:bucket/path/to/object
|
|
rclone backend cleanup \-o max\-age=7w s3:bucket/path/to/object
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
Durations are parsed as per the rest of rclone, 2h, 7d, 7w etc.
|
|
.PP
|
|
Options:
|
|
.IP \[bu] 2
|
|
\[dq]max\-age\[dq]: Max age of upload to delete
|
|
.SS Anonymous access to public buckets
|
|
.PP
|
|
If you want to use rclone to access a public bucket, configure with a
|
|
blank \f[C]access_key_id\f[R] and \f[C]secret_access_key\f[R].
|
|
Your config should end up looking like this:
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
[anons3]
|
|
type = s3
|
|
provider = AWS
|
|
env_auth = false
|
|
access_key_id =
|
|
secret_access_key =
|
|
region = us\-east\-1
|
|
endpoint =
|
|
location_constraint =
|
|
acl = private
|
|
server_side_encryption =
|
|
storage_class =
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
Then use it as normal with the name of the public bucket, e.g.
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone lsd anons3:1000genomes
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
You will be able to list and copy data but not upload it.
|
|
.SS Ceph
|
|
.PP
|
|
Ceph (https://ceph.com/) is an open source unified, distributed storage
|
|
system designed for excellent performance, reliability and scalability.
|
|
It has an S3 compatible object storage interface.
|
|
.PP
|
|
To use rclone with Ceph, configure as above but leave the region blank
|
|
and set the endpoint.
|
|
You should end up with something like this in your config:
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
[ceph]
|
|
type = s3
|
|
provider = Ceph
|
|
env_auth = false
|
|
access_key_id = XXX
|
|
secret_access_key = YYY
|
|
region =
|
|
endpoint = https://ceph.endpoint.example.com
|
|
location_constraint =
|
|
acl =
|
|
server_side_encryption =
|
|
storage_class =
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
If you are using an older version of CEPH, e.g.
|
|
10.2.x Jewel, then you may need to supply the parameter
|
|
\f[C]\-\-s3\-upload\-cutoff 0\f[R] or put this in the config file as
|
|
\f[C]upload_cutoff 0\f[R] to work around a bug which causes uploading of
|
|
small files to fail.
|
|
.PP
|
|
Note also that Ceph sometimes puts \f[C]/\f[R] in the passwords it gives
|
|
users.
|
|
If you read the secret access key using the command line tools you will
|
|
get a JSON blob with the \f[C]/\f[R] escaped as \f[C]\[rs]/\f[R].
|
|
Make sure you only write \f[C]/\f[R] in the secret access key.
|
|
.PP
|
|
Eg the dump from Ceph looks something like this (irrelevant keys
|
|
removed).
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
{
|
|
\[dq]user_id\[dq]: \[dq]xxx\[dq],
|
|
\[dq]display_name\[dq]: \[dq]xxxx\[dq],
|
|
\[dq]keys\[dq]: [
|
|
{
|
|
\[dq]user\[dq]: \[dq]xxx\[dq],
|
|
\[dq]access_key\[dq]: \[dq]xxxxxx\[dq],
|
|
\[dq]secret_key\[dq]: \[dq]xxxxxx\[rs]/xxxx\[dq]
|
|
}
|
|
],
|
|
}
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
Because this is a json dump, it is encoding the \f[C]/\f[R] as
|
|
\f[C]\[rs]/\f[R], so if you use the secret key as \f[C]xxxxxx/xxxx\f[R]
|
|
it will work fine.
|
|
.SS Dreamhost
|
|
.PP
|
|
Dreamhost DreamObjects (https://www.dreamhost.com/cloud/storage/) is an
|
|
object storage system based on CEPH.
|
|
.PP
|
|
To use rclone with Dreamhost, configure as above but leave the region
|
|
blank and set the endpoint.
|
|
You should end up with something like this in your config:
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
[dreamobjects]
|
|
type = s3
|
|
provider = DreamHost
|
|
env_auth = false
|
|
access_key_id = your_access_key
|
|
secret_access_key = your_secret_key
|
|
region =
|
|
endpoint = objects\-us\-west\-1.dream.io
|
|
location_constraint =
|
|
acl = private
|
|
server_side_encryption =
|
|
storage_class =
|
|
\f[R]
|
|
.fi
|
|
.SS DigitalOcean Spaces
|
|
.PP
|
|
Spaces (https://www.digitalocean.com/products/object-storage/) is an
|
|
S3\-interoperable (https://developers.digitalocean.com/documentation/spaces/)
|
|
object storage service from cloud provider DigitalOcean.
|
|
.PP
|
|
To connect to DigitalOcean Spaces you will need an access key and secret
|
|
key.
|
|
These can be retrieved on the \[dq]Applications &
|
|
API (https://cloud.digitalocean.com/settings/api/tokens)\[dq] page of
|
|
the DigitalOcean control panel.
|
|
They will be needed when prompted by \f[C]rclone config\f[R] for your
|
|
\f[C]access_key_id\f[R] and \f[C]secret_access_key\f[R].
|
|
.PP
|
|
When prompted for a \f[C]region\f[R] or \f[C]location_constraint\f[R],
|
|
press enter to use the default value.
|
|
The region must be included in the \f[C]endpoint\f[R] setting (e.g.
|
|
\f[C]nyc3.digitaloceanspaces.com\f[R]).
|
|
The default values can be used for other settings.
|
|
.PP
|
|
Going through the whole process of creating a new remote by running
|
|
\f[C]rclone config\f[R], each prompt should be answered as shown below:
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
Storage> s3
|
|
env_auth> 1
|
|
access_key_id> YOUR_ACCESS_KEY
|
|
secret_access_key> YOUR_SECRET_KEY
|
|
region>
|
|
endpoint> nyc3.digitaloceanspaces.com
|
|
location_constraint>
|
|
acl>
|
|
storage_class>
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
The resulting configuration file should look like:
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
[spaces]
|
|
type = s3
|
|
provider = DigitalOcean
|
|
env_auth = false
|
|
access_key_id = YOUR_ACCESS_KEY
|
|
secret_access_key = YOUR_SECRET_KEY
|
|
region =
|
|
endpoint = nyc3.digitaloceanspaces.com
|
|
location_constraint =
|
|
acl =
|
|
server_side_encryption =
|
|
storage_class =
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
Once configured, you can create a new Space and begin copying files.
|
|
For example:
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone mkdir spaces:my\-new\-space
|
|
rclone copy /path/to/files spaces:my\-new\-space
|
|
\f[R]
|
|
.fi
|
|
.SS IBM COS (S3)
|
|
.PP
|
|
Information stored with IBM Cloud Object Storage is encrypted and
|
|
dispersed across multiple geographic locations, and accessed through an
|
|
implementation of the S3 API.
|
|
This service makes use of the distributed storage technologies provided
|
|
by IBM\[cq]s Cloud Object Storage System (formerly Cleversafe).
|
|
For more information visit: (http://www.ibm.com/cloud/object\-storage)
|
|
.PP
|
|
To configure access to IBM COS S3, follow the steps below:
|
|
.IP "1." 3
|
|
Run rclone config and select n for a new remote.
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
2018/02/14 14:13:11 NOTICE: Config file \[dq]C:\[rs]\[rs]Users\[rs]\[rs]a\[rs]\[rs].config\[rs]\[rs]rclone\[rs]\[rs]rclone.conf\[dq] not found \- using defaults
|
|
No remotes found \- make a new one
|
|
n) New remote
|
|
s) Set configuration password
|
|
q) Quit config
|
|
n/s/q> n
|
|
\f[R]
|
|
.fi
|
|
.IP "2." 3
|
|
Enter the name for the configuration
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
name> <YOUR NAME>
|
|
\f[R]
|
|
.fi
|
|
.IP "3." 3
|
|
Select \[dq]s3\[dq] storage.
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
Choose a number from below, or type in your own value
|
|
1 / Alias for an existing remote
|
|
\[rs] \[dq]alias\[dq]
|
|
2 / Amazon Drive
|
|
\[rs] \[dq]amazon cloud drive\[dq]
|
|
3 / Amazon S3 Complaint Storage Providers (Dreamhost, Ceph, Minio, IBM COS)
|
|
\[rs] \[dq]s3\[dq]
|
|
4 / Backblaze B2
|
|
\[rs] \[dq]b2\[dq]
|
|
[snip]
|
|
23 / http Connection
|
|
\[rs] \[dq]http\[dq]
|
|
Storage> 3
|
|
\f[R]
|
|
.fi
|
|
.IP "4." 3
|
|
Select IBM COS as the S3 Storage Provider.
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
Choose the S3 provider.
|
|
Choose a number from below, or type in your own value
|
|
1 / Choose this option to configure Storage to AWS S3
|
|
\[rs] \[dq]AWS\[dq]
|
|
2 / Choose this option to configure Storage to Ceph Systems
|
|
\[rs] \[dq]Ceph\[dq]
|
|
3 / Choose this option to configure Storage to Dreamhost
|
|
\[rs] \[dq]Dreamhost\[dq]
|
|
4 / Choose this option to the configure Storage to IBM COS S3
|
|
\[rs] \[dq]IBMCOS\[dq]
|
|
5 / Choose this option to the configure Storage to Minio
|
|
\[rs] \[dq]Minio\[dq]
|
|
Provider>4
|
|
\f[R]
|
|
.fi
|
|
.IP "5." 3
|
|
Enter the Access Key and Secret.
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
AWS Access Key ID \- leave blank for anonymous access or runtime credentials.
|
|
access_key_id> <>
|
|
AWS Secret Access Key (password) \- leave blank for anonymous access or runtime credentials.
|
|
secret_access_key> <>
|
|
\f[R]
|
|
.fi
|
|
.IP "6." 3
|
|
Specify the endpoint for IBM COS.
|
|
For Public IBM COS, choose from the option below.
|
|
For On Premise IBM COS, enter an endpoint address.
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
Endpoint for IBM COS S3 API.
|
|
Specify if using an IBM COS On Premise.
|
|
Choose a number from below, or type in your own value
|
|
1 / US Cross Region Endpoint
|
|
\[rs] \[dq]s3\-api.us\-geo.objectstorage.softlayer.net\[dq]
|
|
2 / US Cross Region Dallas Endpoint
|
|
\[rs] \[dq]s3\-api.dal.us\-geo.objectstorage.softlayer.net\[dq]
|
|
3 / US Cross Region Washington DC Endpoint
|
|
\[rs] \[dq]s3\-api.wdc\-us\-geo.objectstorage.softlayer.net\[dq]
|
|
4 / US Cross Region San Jose Endpoint
|
|
\[rs] \[dq]s3\-api.sjc\-us\-geo.objectstorage.softlayer.net\[dq]
|
|
5 / US Cross Region Private Endpoint
|
|
\[rs] \[dq]s3\-api.us\-geo.objectstorage.service.networklayer.com\[dq]
|
|
6 / US Cross Region Dallas Private Endpoint
|
|
\[rs] \[dq]s3\-api.dal\-us\-geo.objectstorage.service.networklayer.com\[dq]
|
|
7 / US Cross Region Washington DC Private Endpoint
|
|
\[rs] \[dq]s3\-api.wdc\-us\-geo.objectstorage.service.networklayer.com\[dq]
|
|
8 / US Cross Region San Jose Private Endpoint
|
|
\[rs] \[dq]s3\-api.sjc\-us\-geo.objectstorage.service.networklayer.com\[dq]
|
|
9 / US Region East Endpoint
|
|
\[rs] \[dq]s3.us\-east.objectstorage.softlayer.net\[dq]
|
|
10 / US Region East Private Endpoint
|
|
\[rs] \[dq]s3.us\-east.objectstorage.service.networklayer.com\[dq]
|
|
11 / US Region South Endpoint
|
|
[snip]
|
|
34 / Toronto Single Site Private Endpoint
|
|
\[rs] \[dq]s3.tor01.objectstorage.service.networklayer.com\[dq]
|
|
endpoint>1
|
|
\f[R]
|
|
.fi
|
|
.IP "7." 3
|
|
Specify a IBM COS Location Constraint.
|
|
The location constraint must match endpoint when using IBM Cloud Public.
|
|
For on\-prem COS, do not make a selection from this list, hit enter
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
1 / US Cross Region Standard
|
|
\[rs] \[dq]us\-standard\[dq]
|
|
2 / US Cross Region Vault
|
|
\[rs] \[dq]us\-vault\[dq]
|
|
3 / US Cross Region Cold
|
|
\[rs] \[dq]us\-cold\[dq]
|
|
4 / US Cross Region Flex
|
|
\[rs] \[dq]us\-flex\[dq]
|
|
5 / US East Region Standard
|
|
\[rs] \[dq]us\-east\-standard\[dq]
|
|
6 / US East Region Vault
|
|
\[rs] \[dq]us\-east\-vault\[dq]
|
|
7 / US East Region Cold
|
|
\[rs] \[dq]us\-east\-cold\[dq]
|
|
8 / US East Region Flex
|
|
\[rs] \[dq]us\-east\-flex\[dq]
|
|
9 / US South Region Standard
|
|
\[rs] \[dq]us\-south\-standard\[dq]
|
|
10 / US South Region Vault
|
|
\[rs] \[dq]us\-south\-vault\[dq]
|
|
[snip]
|
|
32 / Toronto Flex
|
|
\[rs] \[dq]tor01\-flex\[dq]
|
|
location_constraint>1
|
|
\f[R]
|
|
.fi
|
|
.IP "9." 3
|
|
Specify a canned ACL.
|
|
IBM Cloud (Storage) supports \[dq]public\-read\[dq] and
|
|
\[dq]private\[dq].
|
|
IBM Cloud(Infra) supports all the canned ACLs.
|
|
On\-Premise COS supports all the canned ACLs.
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
Canned ACL used when creating buckets and/or storing objects in S3.
|
|
For more info visit https://docs.aws.amazon.com/AmazonS3/latest/dev/acl\-overview.html#canned\-acl
|
|
Choose a number from below, or type in your own value
|
|
1 / Owner gets FULL_CONTROL. No one else has access rights (default). This acl is available on IBM Cloud (Infra), IBM Cloud (Storage), On\-Premise COS
|
|
\[rs] \[dq]private\[dq]
|
|
2 / Owner gets FULL_CONTROL. The AllUsers group gets READ access. This acl is available on IBM Cloud (Infra), IBM Cloud (Storage), On\-Premise IBM COS
|
|
\[rs] \[dq]public\-read\[dq]
|
|
3 / Owner gets FULL_CONTROL. The AllUsers group gets READ and WRITE access. This acl is available on IBM Cloud (Infra), On\-Premise IBM COS
|
|
\[rs] \[dq]public\-read\-write\[dq]
|
|
4 / Owner gets FULL_CONTROL. The AuthenticatedUsers group gets READ access. Not supported on Buckets. This acl is available on IBM Cloud (Infra) and On\-Premise IBM COS
|
|
\[rs] \[dq]authenticated\-read\[dq]
|
|
acl> 1
|
|
\f[R]
|
|
.fi
|
|
.IP "12." 4
|
|
Review the displayed configuration and accept to save the
|
|
\[dq]remote\[dq] then quit.
|
|
The config file should look like this
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
[xxx]
|
|
type = s3
|
|
Provider = IBMCOS
|
|
access_key_id = xxx
|
|
secret_access_key = yyy
|
|
endpoint = s3\-api.us\-geo.objectstorage.softlayer.net
|
|
location_constraint = us\-standard
|
|
acl = private
|
|
\f[R]
|
|
.fi
|
|
.IP "13." 4
|
|
Execute rclone commands
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
1) Create a bucket.
|
|
rclone mkdir IBM\-COS\-XREGION:newbucket
|
|
2) List available buckets.
|
|
rclone lsd IBM\-COS\-XREGION:
|
|
\-1 2017\-11\-08 21:16:22 \-1 test
|
|
\-1 2018\-02\-14 20:16:39 \-1 newbucket
|
|
3) List contents of a bucket.
|
|
rclone ls IBM\-COS\-XREGION:newbucket
|
|
18685952 test.exe
|
|
4) Copy a file from local to remote.
|
|
rclone copy /Users/file.txt IBM\-COS\-XREGION:newbucket
|
|
5) Copy a file from remote to local.
|
|
rclone copy IBM\-COS\-XREGION:newbucket/file.txt .
|
|
6) Delete a file on remote.
|
|
rclone delete IBM\-COS\-XREGION:newbucket/file.txt
|
|
\f[R]
|
|
.fi
|
|
.SS Minio
|
|
.PP
|
|
Minio (https://minio.io/) is an object storage server built for cloud
|
|
application developers and devops.
|
|
.PP
|
|
It is very easy to install and provides an S3 compatible server which
|
|
can be used by rclone.
|
|
.PP
|
|
To use it, install Minio following the instructions
|
|
here (https://docs.minio.io/docs/minio-quickstart-guide).
|
|
.PP
|
|
When it configures itself Minio will print something like this
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
Endpoint: http://192.168.1.106:9000 http://172.23.0.1:9000
|
|
AccessKey: USWUXHGYZQYFYFFIT3RE
|
|
SecretKey: MOJRH0mkL1IPauahWITSVvyDrQbEEIwljvmxdq03
|
|
Region: us\-east\-1
|
|
SQS ARNs: arn:minio:sqs:us\-east\-1:1:redis arn:minio:sqs:us\-east\-1:2:redis
|
|
|
|
Browser Access:
|
|
http://192.168.1.106:9000 http://172.23.0.1:9000
|
|
|
|
Command\-line Access: https://docs.minio.io/docs/minio\-client\-quickstart\-guide
|
|
$ mc config host add myminio http://192.168.1.106:9000 USWUXHGYZQYFYFFIT3RE MOJRH0mkL1IPauahWITSVvyDrQbEEIwljvmxdq03
|
|
|
|
Object API (Amazon S3 compatible):
|
|
Go: https://docs.minio.io/docs/golang\-client\-quickstart\-guide
|
|
Java: https://docs.minio.io/docs/java\-client\-quickstart\-guide
|
|
Python: https://docs.minio.io/docs/python\-client\-quickstart\-guide
|
|
JavaScript: https://docs.minio.io/docs/javascript\-client\-quickstart\-guide
|
|
.NET: https://docs.minio.io/docs/dotnet\-client\-quickstart\-guide
|
|
|
|
Drive Capacity: 26 GiB Free, 165 GiB Total
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
These details need to go into \f[C]rclone config\f[R] like this.
|
|
Note that it is important to put the region in as stated above.
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
env_auth> 1
|
|
access_key_id> USWUXHGYZQYFYFFIT3RE
|
|
secret_access_key> MOJRH0mkL1IPauahWITSVvyDrQbEEIwljvmxdq03
|
|
region> us\-east\-1
|
|
endpoint> http://192.168.1.106:9000
|
|
location_constraint>
|
|
server_side_encryption>
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
Which makes the config file look like this
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
[minio]
|
|
type = s3
|
|
provider = Minio
|
|
env_auth = false
|
|
access_key_id = USWUXHGYZQYFYFFIT3RE
|
|
secret_access_key = MOJRH0mkL1IPauahWITSVvyDrQbEEIwljvmxdq03
|
|
region = us\-east\-1
|
|
endpoint = http://192.168.1.106:9000
|
|
location_constraint =
|
|
server_side_encryption =
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
So once set up, for example to copy files into a bucket
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone copy /path/to/files minio:bucket
|
|
\f[R]
|
|
.fi
|
|
.SS Scaleway
|
|
.PP
|
|
Scaleway (https://www.scaleway.com/object-storage/) The Object Storage
|
|
platform allows you to store anything from backups, logs and web assets
|
|
to documents and photos.
|
|
Files can be dropped from the Scaleway console or transferred through
|
|
our API and CLI or using any S3\-compatible tool.
|
|
.PP
|
|
Scaleway provides an S3 interface which can be configured for use with
|
|
rclone like this:
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
[scaleway]
|
|
type = s3
|
|
provider = Scaleway
|
|
env_auth = false
|
|
endpoint = s3.nl\-ams.scw.cloud
|
|
access_key_id = SCWXXXXXXXXXXXXXX
|
|
secret_access_key = 1111111\-2222\-3333\-44444\-55555555555555
|
|
region = nl\-ams
|
|
location_constraint =
|
|
acl = private
|
|
server_side_encryption =
|
|
storage_class =
|
|
\f[R]
|
|
.fi
|
|
.SS Wasabi
|
|
.PP
|
|
Wasabi (https://wasabi.com) is a cloud\-based object storage service for
|
|
a broad range of applications and use cases.
|
|
Wasabi is designed for individuals and organizations that require a
|
|
high\-performance, reliable, and secure data storage infrastructure at
|
|
minimal cost.
|
|
.PP
|
|
Wasabi provides an S3 interface which can be configured for use with
|
|
rclone like this.
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
No remotes found \- make a new one
|
|
n) New remote
|
|
s) Set configuration password
|
|
n/s> n
|
|
name> wasabi
|
|
Type of storage to configure.
|
|
Choose a number from below, or type in your own value
|
|
[snip]
|
|
XX / Amazon S3 (also Dreamhost, Ceph, Minio)
|
|
\[rs] \[dq]s3\[dq]
|
|
[snip]
|
|
Storage> s3
|
|
Get AWS credentials from runtime (environment variables or EC2/ECS meta data if no env vars). Only applies if access_key_id and secret_access_key is blank.
|
|
Choose a number from below, or type in your own value
|
|
1 / Enter AWS credentials in the next step
|
|
\[rs] \[dq]false\[dq]
|
|
2 / Get AWS credentials from the environment (env vars or IAM)
|
|
\[rs] \[dq]true\[dq]
|
|
env_auth> 1
|
|
AWS Access Key ID \- leave blank for anonymous access or runtime credentials.
|
|
access_key_id> YOURACCESSKEY
|
|
AWS Secret Access Key (password) \- leave blank for anonymous access or runtime credentials.
|
|
secret_access_key> YOURSECRETACCESSKEY
|
|
Region to connect to.
|
|
Choose a number from below, or type in your own value
|
|
/ The default endpoint \- a good choice if you are unsure.
|
|
1 | US Region, Northern Virginia, or Pacific Northwest.
|
|
| Leave location constraint empty.
|
|
\[rs] \[dq]us\-east\-1\[dq]
|
|
[snip]
|
|
region> us\-east\-1
|
|
Endpoint for S3 API.
|
|
Leave blank if using AWS to use the default endpoint for the region.
|
|
Specify if using an S3 clone such as Ceph.
|
|
endpoint> s3.wasabisys.com
|
|
Location constraint \- must be set to match the Region. Used when creating buckets only.
|
|
Choose a number from below, or type in your own value
|
|
1 / Empty for US Region, Northern Virginia, or Pacific Northwest.
|
|
\[rs] \[dq]\[dq]
|
|
[snip]
|
|
location_constraint>
|
|
Canned ACL used when creating buckets and/or storing objects in S3.
|
|
For more info visit https://docs.aws.amazon.com/AmazonS3/latest/dev/acl\-overview.html#canned\-acl
|
|
Choose a number from below, or type in your own value
|
|
1 / Owner gets FULL_CONTROL. No one else has access rights (default).
|
|
\[rs] \[dq]private\[dq]
|
|
[snip]
|
|
acl>
|
|
The server\-side encryption algorithm used when storing this object in S3.
|
|
Choose a number from below, or type in your own value
|
|
1 / None
|
|
\[rs] \[dq]\[dq]
|
|
2 / AES256
|
|
\[rs] \[dq]AES256\[dq]
|
|
server_side_encryption>
|
|
The storage class to use when storing objects in S3.
|
|
Choose a number from below, or type in your own value
|
|
1 / Default
|
|
\[rs] \[dq]\[dq]
|
|
2 / Standard storage class
|
|
\[rs] \[dq]STANDARD\[dq]
|
|
3 / Reduced redundancy storage class
|
|
\[rs] \[dq]REDUCED_REDUNDANCY\[dq]
|
|
4 / Standard Infrequent Access storage class
|
|
\[rs] \[dq]STANDARD_IA\[dq]
|
|
storage_class>
|
|
Remote config
|
|
\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-
|
|
[wasabi]
|
|
env_auth = false
|
|
access_key_id = YOURACCESSKEY
|
|
secret_access_key = YOURSECRETACCESSKEY
|
|
region = us\-east\-1
|
|
endpoint = s3.wasabisys.com
|
|
location_constraint =
|
|
acl =
|
|
server_side_encryption =
|
|
storage_class =
|
|
\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-
|
|
y) Yes this is OK
|
|
e) Edit this remote
|
|
d) Delete this remote
|
|
y/e/d> y
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
This will leave the config file looking like this.
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
[wasabi]
|
|
type = s3
|
|
provider = Wasabi
|
|
env_auth = false
|
|
access_key_id = YOURACCESSKEY
|
|
secret_access_key = YOURSECRETACCESSKEY
|
|
region =
|
|
endpoint = s3.wasabisys.com
|
|
location_constraint =
|
|
acl =
|
|
server_side_encryption =
|
|
storage_class =
|
|
\f[R]
|
|
.fi
|
|
.SS Alibaba OSS
|
|
.PP
|
|
Here is an example of making an Alibaba Cloud (Aliyun)
|
|
OSS (https://www.alibabacloud.com/product/oss/) configuration.
|
|
First run:
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone config
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
This will guide you through an interactive setup process.
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
No remotes found \- make a new one
|
|
n) New remote
|
|
s) Set configuration password
|
|
q) Quit config
|
|
n/s/q> n
|
|
name> oss
|
|
Type of storage to configure.
|
|
Enter a string value. Press Enter for the default (\[dq]\[dq]).
|
|
Choose a number from below, or type in your own value
|
|
[snip]
|
|
4 / Amazon S3 Compliant Storage Providers including AWS, Alibaba, Ceph, Digital Ocean, Dreamhost, IBM COS, Minio, and Tencent COS
|
|
\[rs] \[dq]s3\[dq]
|
|
[snip]
|
|
Storage> s3
|
|
Choose your S3 provider.
|
|
Enter a string value. Press Enter for the default (\[dq]\[dq]).
|
|
Choose a number from below, or type in your own value
|
|
1 / Amazon Web Services (AWS) S3
|
|
\[rs] \[dq]AWS\[dq]
|
|
2 / Alibaba Cloud Object Storage System (OSS) formerly Aliyun
|
|
\[rs] \[dq]Alibaba\[dq]
|
|
3 / Ceph Object Storage
|
|
\[rs] \[dq]Ceph\[dq]
|
|
[snip]
|
|
provider> Alibaba
|
|
Get AWS credentials from runtime (environment variables or EC2/ECS meta data if no env vars).
|
|
Only applies if access_key_id and secret_access_key is blank.
|
|
Enter a boolean value (true or false). Press Enter for the default (\[dq]false\[dq]).
|
|
Choose a number from below, or type in your own value
|
|
1 / Enter AWS credentials in the next step
|
|
\[rs] \[dq]false\[dq]
|
|
2 / Get AWS credentials from the environment (env vars or IAM)
|
|
\[rs] \[dq]true\[dq]
|
|
env_auth> 1
|
|
AWS Access Key ID.
|
|
Leave blank for anonymous access or runtime credentials.
|
|
Enter a string value. Press Enter for the default (\[dq]\[dq]).
|
|
access_key_id> accesskeyid
|
|
AWS Secret Access Key (password)
|
|
Leave blank for anonymous access or runtime credentials.
|
|
Enter a string value. Press Enter for the default (\[dq]\[dq]).
|
|
secret_access_key> secretaccesskey
|
|
Endpoint for OSS API.
|
|
Enter a string value. Press Enter for the default (\[dq]\[dq]).
|
|
Choose a number from below, or type in your own value
|
|
1 / East China 1 (Hangzhou)
|
|
\[rs] \[dq]oss\-cn\-hangzhou.aliyuncs.com\[dq]
|
|
2 / East China 2 (Shanghai)
|
|
\[rs] \[dq]oss\-cn\-shanghai.aliyuncs.com\[dq]
|
|
3 / North China 1 (Qingdao)
|
|
\[rs] \[dq]oss\-cn\-qingdao.aliyuncs.com\[dq]
|
|
[snip]
|
|
endpoint> 1
|
|
Canned ACL used when creating buckets and storing or copying objects.
|
|
|
|
Note that this ACL is applied when server\-side copying objects as S3
|
|
doesn\[aq]t copy the ACL from the source but rather writes a fresh one.
|
|
Enter a string value. Press Enter for the default (\[dq]\[dq]).
|
|
Choose a number from below, or type in your own value
|
|
1 / Owner gets FULL_CONTROL. No one else has access rights (default).
|
|
\[rs] \[dq]private\[dq]
|
|
2 / Owner gets FULL_CONTROL. The AllUsers group gets READ access.
|
|
\[rs] \[dq]public\-read\[dq]
|
|
/ Owner gets FULL_CONTROL. The AllUsers group gets READ and WRITE access.
|
|
[snip]
|
|
acl> 1
|
|
The storage class to use when storing new objects in OSS.
|
|
Enter a string value. Press Enter for the default (\[dq]\[dq]).
|
|
Choose a number from below, or type in your own value
|
|
1 / Default
|
|
\[rs] \[dq]\[dq]
|
|
2 / Standard storage class
|
|
\[rs] \[dq]STANDARD\[dq]
|
|
3 / Archive storage mode.
|
|
\[rs] \[dq]GLACIER\[dq]
|
|
4 / Infrequent access storage mode.
|
|
\[rs] \[dq]STANDARD_IA\[dq]
|
|
storage_class> 1
|
|
Edit advanced config? (y/n)
|
|
y) Yes
|
|
n) No
|
|
y/n> n
|
|
Remote config
|
|
\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-
|
|
[oss]
|
|
type = s3
|
|
provider = Alibaba
|
|
env_auth = false
|
|
access_key_id = accesskeyid
|
|
secret_access_key = secretaccesskey
|
|
endpoint = oss\-cn\-hangzhou.aliyuncs.com
|
|
acl = private
|
|
storage_class = Standard
|
|
\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-
|
|
y) Yes this is OK
|
|
e) Edit this remote
|
|
d) Delete this remote
|
|
y/e/d> y
|
|
\f[R]
|
|
.fi
|
|
.SS Tencent COS
|
|
.PP
|
|
Tencent Cloud Object Storage
|
|
(COS) (https://intl.cloud.tencent.com/product/cos) is a distributed
|
|
storage service offered by Tencent Cloud for unstructured data.
|
|
It is secure, stable, massive, convenient, low\-delay and low\-cost.
|
|
.PP
|
|
To configure access to Tencent COS, follow the steps below:
|
|
.IP "1." 3
|
|
Run \f[C]rclone config\f[R] and select \f[C]n\f[R] for a new remote.
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone config
|
|
No remotes found \- make a new one
|
|
n) New remote
|
|
s) Set configuration password
|
|
q) Quit config
|
|
n/s/q> n
|
|
\f[R]
|
|
.fi
|
|
.IP "2." 3
|
|
Give the name of the configuration.
|
|
For example, name it \[aq]cos\[aq].
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
name> cos
|
|
\f[R]
|
|
.fi
|
|
.IP "3." 3
|
|
Select \f[C]s3\f[R] storage.
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
Choose a number from below, or type in your own value
|
|
1 / 1Fichier
|
|
\[rs] \[dq]fichier\[dq]
|
|
2 / Alias for an existing remote
|
|
\[rs] \[dq]alias\[dq]
|
|
3 / Amazon Drive
|
|
\[rs] \[dq]amazon cloud drive\[dq]
|
|
4 / Amazon S3 Compliant Storage Providers including AWS, Alibaba, Ceph, Digital Ocean, Dreamhost, IBM COS, Minio, and Tencent COS
|
|
\[rs] \[dq]s3\[dq]
|
|
[snip]
|
|
Storage> s3
|
|
\f[R]
|
|
.fi
|
|
.IP "4." 3
|
|
Select \f[C]TencentCOS\f[R] provider.
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
Choose a number from below, or type in your own value
|
|
1 / Amazon Web Services (AWS) S3
|
|
\[rs] \[dq]AWS\[dq]
|
|
[snip]
|
|
11 / Tencent Cloud Object Storage (COS)
|
|
\[rs] \[dq]TencentCOS\[dq]
|
|
[snip]
|
|
provider> TencentCOS
|
|
\f[R]
|
|
.fi
|
|
.IP "5." 3
|
|
Enter your SecretId and SecretKey of Tencent Cloud.
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
Get AWS credentials from runtime (environment variables or EC2/ECS meta data if no env vars).
|
|
Only applies if access_key_id and secret_access_key is blank.
|
|
Enter a boolean value (true or false). Press Enter for the default (\[dq]false\[dq]).
|
|
Choose a number from below, or type in your own value
|
|
1 / Enter AWS credentials in the next step
|
|
\[rs] \[dq]false\[dq]
|
|
2 / Get AWS credentials from the environment (env vars or IAM)
|
|
\[rs] \[dq]true\[dq]
|
|
env_auth> 1
|
|
AWS Access Key ID.
|
|
Leave blank for anonymous access or runtime credentials.
|
|
Enter a string value. Press Enter for the default (\[dq]\[dq]).
|
|
access_key_id> AKIDxxxxxxxxxx
|
|
AWS Secret Access Key (password)
|
|
Leave blank for anonymous access or runtime credentials.
|
|
Enter a string value. Press Enter for the default (\[dq]\[dq]).
|
|
secret_access_key> xxxxxxxxxxx
|
|
\f[R]
|
|
.fi
|
|
.IP "6." 3
|
|
Select endpoint for Tencent COS.
|
|
This is the standard endpoint for different region.
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
1 / Beijing Region.
|
|
\[rs] \[dq]cos.ap\-beijing.myqcloud.com\[dq]
|
|
2 / Nanjing Region.
|
|
\[rs] \[dq]cos.ap\-nanjing.myqcloud.com\[dq]
|
|
3 / Shanghai Region.
|
|
\[rs] \[dq]cos.ap\-shanghai.myqcloud.com\[dq]
|
|
4 / Guangzhou Region.
|
|
\[rs] \[dq]cos.ap\-guangzhou.myqcloud.com\[dq]
|
|
[snip]
|
|
endpoint> 4
|
|
\f[R]
|
|
.fi
|
|
.IP "7." 3
|
|
Choose acl and storage class.
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
Note that this ACL is applied when server\-side copying objects as S3
|
|
doesn\[aq]t copy the ACL from the source but rather writes a fresh one.
|
|
Enter a string value. Press Enter for the default (\[dq]\[dq]).
|
|
Choose a number from below, or type in your own value
|
|
1 / Owner gets Full_CONTROL. No one else has access rights (default).
|
|
\[rs] \[dq]default\[dq]
|
|
[snip]
|
|
acl> 1
|
|
The storage class to use when storing new objects in Tencent COS.
|
|
Enter a string value. Press Enter for the default (\[dq]\[dq]).
|
|
Choose a number from below, or type in your own value
|
|
1 / Default
|
|
\[rs] \[dq]\[dq]
|
|
[snip]
|
|
storage_class> 1
|
|
Edit advanced config? (y/n)
|
|
y) Yes
|
|
n) No (default)
|
|
y/n> n
|
|
Remote config
|
|
\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-
|
|
[cos]
|
|
type = s3
|
|
provider = TencentCOS
|
|
env_auth = false
|
|
access_key_id = xxx
|
|
secret_access_key = xxx
|
|
endpoint = cos.ap\-guangzhou.myqcloud.com
|
|
acl = default
|
|
\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-
|
|
y) Yes this is OK (default)
|
|
e) Edit this remote
|
|
d) Delete this remote
|
|
y/e/d> y
|
|
Current remotes:
|
|
|
|
Name Type
|
|
==== ====
|
|
cos s3
|
|
\f[R]
|
|
.fi
|
|
.SS Netease NOS
|
|
.PP
|
|
For Netease NOS configure as per the configurator
|
|
\f[C]rclone config\f[R] setting the provider \f[C]Netease\f[R].
|
|
This will automatically set \f[C]force_path_style = false\f[R] which is
|
|
necessary for it to run properly.
|
|
.SS Limitations
|
|
.PP
|
|
\f[C]rclone about\f[R] is not supported by the S3 backend.
|
|
Backends without this capability cannot determine free space for an
|
|
rclone mount or use policy \f[C]mfs\f[R] (most free space) as a member
|
|
of an rclone union remote.
|
|
.PP
|
|
See List of backends that do not support rclone
|
|
about (https://rclone.org/overview/#optional-features) See rclone
|
|
about (https://rclone.org/commands/rclone_about/)
|
|
.SS Backblaze B2
|
|
.PP
|
|
B2 is Backblaze\[aq]s cloud storage
|
|
system (https://www.backblaze.com/b2/).
|
|
.PP
|
|
Paths are specified as \f[C]remote:bucket\f[R] (or \f[C]remote:\f[R] for
|
|
the \f[C]lsd\f[R] command.) You may put subdirectories in too, e.g.
|
|
\f[C]remote:bucket/path/to/dir\f[R].
|
|
.PP
|
|
Here is an example of making a b2 configuration.
|
|
First run
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone config
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
This will guide you through an interactive setup process.
|
|
To authenticate you will either need your Account ID (a short hex
|
|
number) and Master Application Key (a long hex number) OR an Application
|
|
Key, which is the recommended method.
|
|
See below for further details on generating and using an Application
|
|
Key.
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
No remotes found \- make a new one
|
|
n) New remote
|
|
q) Quit config
|
|
n/q> n
|
|
name> remote
|
|
Type of storage to configure.
|
|
Choose a number from below, or type in your own value
|
|
[snip]
|
|
XX / Backblaze B2
|
|
\[rs] \[dq]b2\[dq]
|
|
[snip]
|
|
Storage> b2
|
|
Account ID or Application Key ID
|
|
account> 123456789abc
|
|
Application Key
|
|
key> 0123456789abcdef0123456789abcdef0123456789
|
|
Endpoint for the service \- leave blank normally.
|
|
endpoint>
|
|
Remote config
|
|
\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-
|
|
[remote]
|
|
account = 123456789abc
|
|
key = 0123456789abcdef0123456789abcdef0123456789
|
|
endpoint =
|
|
\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-
|
|
y) Yes this is OK
|
|
e) Edit this remote
|
|
d) Delete this remote
|
|
y/e/d> y
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
This remote is called \f[C]remote\f[R] and can now be used like this
|
|
.PP
|
|
See all buckets
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone lsd remote:
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
Create a new bucket
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone mkdir remote:bucket
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
List the contents of a bucket
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone ls remote:bucket
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
Sync \f[C]/home/local/directory\f[R] to the remote bucket, deleting any
|
|
excess files in the bucket.
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone sync \-i /home/local/directory remote:bucket
|
|
\f[R]
|
|
.fi
|
|
.SS Application Keys
|
|
.PP
|
|
B2 supports multiple Application Keys for different access permission to
|
|
B2 Buckets (https://www.backblaze.com/b2/docs/application_keys.html).
|
|
.PP
|
|
You can use these with rclone too; you will need to use rclone version
|
|
1.43 or later.
|
|
.PP
|
|
Follow Backblaze\[aq]s docs to create an Application Key with the
|
|
required permission and add the \f[C]applicationKeyId\f[R] as the
|
|
\f[C]account\f[R] and the \f[C]Application Key\f[R] itself as the
|
|
\f[C]key\f[R].
|
|
.PP
|
|
Note that you must put the \f[I]applicationKeyId\f[R] as the
|
|
\f[C]account\f[R] \[en] you can\[aq]t use the master Account ID.
|
|
If you try then B2 will return 401 errors.
|
|
.SS \-\-fast\-list
|
|
.PP
|
|
This remote supports \f[C]\-\-fast\-list\f[R] which allows you to use
|
|
fewer transactions in exchange for more memory.
|
|
See the rclone docs (https://rclone.org/docs/#fast-list) for more
|
|
details.
|
|
.SS Modified time
|
|
.PP
|
|
The modified time is stored as metadata on the object as
|
|
\f[C]X\-Bz\-Info\-src_last_modified_millis\f[R] as milliseconds since
|
|
1970\-01\-01 in the Backblaze standard.
|
|
Other tools should be able to use this as a modified time.
|
|
.PP
|
|
Modified times are used in syncing and are fully supported.
|
|
Note that if a modification time needs to be updated on an object then
|
|
it will create a new version of the object.
|
|
.SS Restricted filename characters
|
|
.PP
|
|
In addition to the default restricted characters
|
|
set (https://rclone.org/overview/#restricted-characters) the following
|
|
characters are also replaced:
|
|
.PP
|
|
.TS
|
|
tab(@);
|
|
l c c.
|
|
T{
|
|
Character
|
|
T}@T{
|
|
Value
|
|
T}@T{
|
|
Replacement
|
|
T}
|
|
_
|
|
T{
|
|
\[rs]
|
|
T}@T{
|
|
0x5C
|
|
T}@T{
|
|
\[uFF3C]
|
|
T}
|
|
.TE
|
|
.PP
|
|
Invalid UTF\-8 bytes will also be
|
|
replaced (https://rclone.org/overview/#invalid-utf8), as they can\[aq]t
|
|
be used in JSON strings.
|
|
.PP
|
|
Note that in 2020\-05 Backblaze started allowing \ characters in file
|
|
names.
|
|
Rclone hasn\[aq]t changed its encoding as this could cause syncs to
|
|
re\-transfer files.
|
|
If you want rclone not to replace \ then see the
|
|
\f[C]\-\-b2\-encoding\f[R] flag below and remove the \f[C]BackSlash\f[R]
|
|
from the string.
|
|
This can be set in the config.
|
|
.SS SHA1 checksums
|
|
.PP
|
|
The SHA1 checksums of the files are checked on upload and download and
|
|
will be used in the syncing process.
|
|
.PP
|
|
Large files (bigger than the limit in \f[C]\-\-b2\-upload\-cutoff\f[R])
|
|
which are uploaded in chunks will store their SHA1 on the object as
|
|
\f[C]X\-Bz\-Info\-large_file_sha1\f[R] as recommended by Backblaze.
|
|
.PP
|
|
For a large file to be uploaded with an SHA1 checksum, the source needs
|
|
to support SHA1 checksums.
|
|
The local disk supports SHA1 checksums so large file transfers from
|
|
local disk will have an SHA1.
|
|
See the overview (https://rclone.org/overview/#features) for exactly
|
|
which remotes support SHA1.
|
|
.PP
|
|
Sources which don\[aq]t support SHA1, in particular \f[C]crypt\f[R] will
|
|
upload large files without SHA1 checksums.
|
|
This may be fixed in the future (see
|
|
#1767 (https://github.com/rclone/rclone/issues/1767)).
|
|
.PP
|
|
Files sizes below \f[C]\-\-b2\-upload\-cutoff\f[R] will always have an
|
|
SHA1 regardless of the source.
|
|
.SS Transfers
|
|
.PP
|
|
Backblaze recommends that you do lots of transfers simultaneously for
|
|
maximum speed.
|
|
In tests from my SSD equipped laptop the optimum setting is about
|
|
\f[C]\-\-transfers 32\f[R] though higher numbers may be used for a
|
|
slight speed improvement.
|
|
The optimum number for you may vary depending on your hardware, how big
|
|
the files are, how much you want to load your computer, etc.
|
|
The default of \f[C]\-\-transfers 4\f[R] is definitely too low for
|
|
Backblaze B2 though.
|
|
.PP
|
|
Note that uploading big files (bigger than 200 MB by default) will use a
|
|
96 MB RAM buffer by default.
|
|
There can be at most \f[C]\-\-transfers\f[R] of these in use at any
|
|
moment, so this sets the upper limit on the memory used.
|
|
.SS Versions
|
|
.PP
|
|
When rclone uploads a new version of a file it creates a new version of
|
|
it (https://www.backblaze.com/b2/docs/file_versions.html).
|
|
Likewise when you delete a file, the old version will be marked hidden
|
|
and still be available.
|
|
Conversely, you may opt in to a \[dq]hard delete\[dq] of files with the
|
|
\f[C]\-\-b2\-hard\-delete\f[R] flag which would permanently remove the
|
|
file instead of hiding it.
|
|
.PP
|
|
Old versions of files, where available, are visible using the
|
|
\f[C]\-\-b2\-versions\f[R] flag.
|
|
.PP
|
|
\f[B]NB\f[R] Note that \f[C]\-\-b2\-versions\f[R] does not work with
|
|
crypt at the moment
|
|
#1627 (https://github.com/rclone/rclone/issues/1627).
|
|
Using \-\-backup\-dir (https://rclone.org/docs/#backup-dir-dir) with
|
|
rclone is the recommended way of working around this.
|
|
.PP
|
|
If you wish to remove all the old versions then you can use the
|
|
\f[C]rclone cleanup remote:bucket\f[R] command which will delete all the
|
|
old versions of files, leaving the current ones intact.
|
|
You can also supply a path and only old versions under that path will be
|
|
deleted, e.g.
|
|
\f[C]rclone cleanup remote:bucket/path/to/stuff\f[R].
|
|
.PP
|
|
Note that \f[C]cleanup\f[R] will remove partially uploaded files from
|
|
the bucket if they are more than a day old.
|
|
.PP
|
|
When you \f[C]purge\f[R] a bucket, the current and the old versions will
|
|
be deleted then the bucket will be deleted.
|
|
.PP
|
|
However \f[C]delete\f[R] will cause the current versions of the files to
|
|
become hidden old versions.
|
|
.PP
|
|
Here is a session showing the listing and retrieval of an old version
|
|
followed by a \f[C]cleanup\f[R] of the old versions.
|
|
.PP
|
|
Show current version and all the versions with
|
|
\f[C]\-\-b2\-versions\f[R] flag.
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
$ rclone \-q ls b2:cleanup\-test
|
|
9 one.txt
|
|
|
|
$ rclone \-q \-\-b2\-versions ls b2:cleanup\-test
|
|
9 one.txt
|
|
8 one\-v2016\-07\-04\-141032\-000.txt
|
|
16 one\-v2016\-07\-04\-141003\-000.txt
|
|
15 one\-v2016\-07\-02\-155621\-000.txt
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
Retrieve an old version
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
$ rclone \-q \-\-b2\-versions copy b2:cleanup\-test/one\-v2016\-07\-04\-141003\-000.txt /tmp
|
|
|
|
$ ls \-l /tmp/one\-v2016\-07\-04\-141003\-000.txt
|
|
\-rw\-rw\-r\-\- 1 ncw ncw 16 Jul 2 17:46 /tmp/one\-v2016\-07\-04\-141003\-000.txt
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
Clean up all the old versions and show that they\[aq]ve gone.
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
$ rclone \-q cleanup b2:cleanup\-test
|
|
|
|
$ rclone \-q ls b2:cleanup\-test
|
|
9 one.txt
|
|
|
|
$ rclone \-q \-\-b2\-versions ls b2:cleanup\-test
|
|
9 one.txt
|
|
\f[R]
|
|
.fi
|
|
.SS Data usage
|
|
.PP
|
|
It is useful to know how many requests are sent to the server in
|
|
different scenarios.
|
|
.PP
|
|
All copy commands send the following 4 requests:
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
/b2api/v1/b2_authorize_account
|
|
/b2api/v1/b2_create_bucket
|
|
/b2api/v1/b2_list_buckets
|
|
/b2api/v1/b2_list_file_names
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
The \f[C]b2_list_file_names\f[R] request will be sent once for every 1k
|
|
files in the remote path, providing the checksum and modification time
|
|
of the listed files.
|
|
As of version 1.33 issue
|
|
#818 (https://github.com/rclone/rclone/issues/818) causes extra requests
|
|
to be sent when using B2 with Crypt.
|
|
When a copy operation does not require any files to be uploaded, no more
|
|
requests will be sent.
|
|
.PP
|
|
Uploading files that do not require chunking, will send 2 requests per
|
|
file upload:
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
/b2api/v1/b2_get_upload_url
|
|
/b2api/v1/b2_upload_file/
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
Uploading files requiring chunking, will send 2 requests (one each to
|
|
start and finish the upload) and another 2 requests for each chunk:
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
/b2api/v1/b2_start_large_file
|
|
/b2api/v1/b2_get_upload_part_url
|
|
/b2api/v1/b2_upload_part/
|
|
/b2api/v1/b2_finish_large_file
|
|
\f[R]
|
|
.fi
|
|
.SS Versions
|
|
.PP
|
|
Versions can be viewed with the \f[C]\-\-b2\-versions\f[R] flag.
|
|
When it is set rclone will show and act on older versions of files.
|
|
For example
|
|
.PP
|
|
Listing without \f[C]\-\-b2\-versions\f[R]
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
$ rclone \-q ls b2:cleanup\-test
|
|
9 one.txt
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
And with
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
$ rclone \-q \-\-b2\-versions ls b2:cleanup\-test
|
|
9 one.txt
|
|
8 one\-v2016\-07\-04\-141032\-000.txt
|
|
16 one\-v2016\-07\-04\-141003\-000.txt
|
|
15 one\-v2016\-07\-02\-155621\-000.txt
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
Showing that the current version is unchanged but older versions can be
|
|
seen.
|
|
These have the UTC date that they were uploaded to the server to the
|
|
nearest millisecond appended to them.
|
|
.PP
|
|
Note that when using \f[C]\-\-b2\-versions\f[R] no file write operations
|
|
are permitted, so you can\[aq]t upload files or delete them.
|
|
.SS B2 and rclone link
|
|
.PP
|
|
Rclone supports generating file share links for private B2 buckets.
|
|
They can either be for a file for example:
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
\&./rclone link B2:bucket/path/to/file.txt
|
|
https://f002.backblazeb2.com/file/bucket/path/to/file.txt?Authorization=xxxxxxxx
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
or if run on a directory you will get:
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
\&./rclone link B2:bucket/path
|
|
https://f002.backblazeb2.com/file/bucket/path?Authorization=xxxxxxxx
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
you can then use the authorization token (the part of the url from the
|
|
\f[C]?Authorization=\f[R] on) on any file path under that directory.
|
|
For example:
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
https://f002.backblazeb2.com/file/bucket/path/to/file1?Authorization=xxxxxxxx
|
|
https://f002.backblazeb2.com/file/bucket/path/file2?Authorization=xxxxxxxx
|
|
https://f002.backblazeb2.com/file/bucket/path/folder/file3?Authorization=xxxxxxxx
|
|
\f[R]
|
|
.fi
|
|
.SS Standard Options
|
|
.PP
|
|
Here are the standard options specific to b2 (Backblaze B2).
|
|
.SS \-\-b2\-account
|
|
.PP
|
|
Account ID or Application Key ID
|
|
.IP \[bu] 2
|
|
Config: account
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_B2_ACCOUNT
|
|
.IP \[bu] 2
|
|
Type: string
|
|
.IP \[bu] 2
|
|
Default: \[dq]\[dq]
|
|
.SS \-\-b2\-key
|
|
.PP
|
|
Application Key
|
|
.IP \[bu] 2
|
|
Config: key
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_B2_KEY
|
|
.IP \[bu] 2
|
|
Type: string
|
|
.IP \[bu] 2
|
|
Default: \[dq]\[dq]
|
|
.SS \-\-b2\-hard\-delete
|
|
.PP
|
|
Permanently delete files on remote removal, otherwise hide files.
|
|
.IP \[bu] 2
|
|
Config: hard_delete
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_B2_HARD_DELETE
|
|
.IP \[bu] 2
|
|
Type: bool
|
|
.IP \[bu] 2
|
|
Default: false
|
|
.SS Advanced Options
|
|
.PP
|
|
Here are the advanced options specific to b2 (Backblaze B2).
|
|
.SS \-\-b2\-endpoint
|
|
.PP
|
|
Endpoint for the service.
|
|
Leave blank normally.
|
|
.IP \[bu] 2
|
|
Config: endpoint
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_B2_ENDPOINT
|
|
.IP \[bu] 2
|
|
Type: string
|
|
.IP \[bu] 2
|
|
Default: \[dq]\[dq]
|
|
.SS \-\-b2\-test\-mode
|
|
.PP
|
|
A flag string for X\-Bz\-Test\-Mode header for debugging.
|
|
.PP
|
|
This is for debugging purposes only.
|
|
Setting it to one of the strings below will cause b2 to return specific
|
|
errors:
|
|
.IP \[bu] 2
|
|
\[dq]fail_some_uploads\[dq]
|
|
.IP \[bu] 2
|
|
\[dq]expire_some_account_authorization_tokens\[dq]
|
|
.IP \[bu] 2
|
|
\[dq]force_cap_exceeded\[dq]
|
|
.PP
|
|
These will be set in the \[dq]X\-Bz\-Test\-Mode\[dq] header which is
|
|
documented in the b2 integrations
|
|
checklist (https://www.backblaze.com/b2/docs/integration_checklist.html).
|
|
.IP \[bu] 2
|
|
Config: test_mode
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_B2_TEST_MODE
|
|
.IP \[bu] 2
|
|
Type: string
|
|
.IP \[bu] 2
|
|
Default: \[dq]\[dq]
|
|
.SS \-\-b2\-versions
|
|
.PP
|
|
Include old versions in directory listings.
|
|
Note that when using this no file write operations are permitted, so you
|
|
can\[aq]t upload files or delete them.
|
|
.IP \[bu] 2
|
|
Config: versions
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_B2_VERSIONS
|
|
.IP \[bu] 2
|
|
Type: bool
|
|
.IP \[bu] 2
|
|
Default: false
|
|
.SS \-\-b2\-upload\-cutoff
|
|
.PP
|
|
Cutoff for switching to chunked upload.
|
|
.PP
|
|
Files above this size will be uploaded in chunks of
|
|
\[dq]\-\-b2\-chunk\-size\[dq].
|
|
.PP
|
|
This value should be set no larger than 4.657GiB (== 5GB).
|
|
.IP \[bu] 2
|
|
Config: upload_cutoff
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_B2_UPLOAD_CUTOFF
|
|
.IP \[bu] 2
|
|
Type: SizeSuffix
|
|
.IP \[bu] 2
|
|
Default: 200M
|
|
.SS \-\-b2\-copy\-cutoff
|
|
.PP
|
|
Cutoff for switching to multipart copy
|
|
.PP
|
|
Any files larger than this that need to be server\-side copied will be
|
|
copied in chunks of this size.
|
|
.PP
|
|
The minimum is 0 and the maximum is 4.6GB.
|
|
.IP \[bu] 2
|
|
Config: copy_cutoff
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_B2_COPY_CUTOFF
|
|
.IP \[bu] 2
|
|
Type: SizeSuffix
|
|
.IP \[bu] 2
|
|
Default: 4G
|
|
.SS \-\-b2\-chunk\-size
|
|
.PP
|
|
Upload chunk size.
|
|
Must fit in memory.
|
|
.PP
|
|
When uploading large files, chunk the file into this size.
|
|
Note that these chunks are buffered in memory and there might a maximum
|
|
of \[dq]\-\-transfers\[dq] chunks in progress at once.
|
|
5,000,000 Bytes is the minimum size.
|
|
.IP \[bu] 2
|
|
Config: chunk_size
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_B2_CHUNK_SIZE
|
|
.IP \[bu] 2
|
|
Type: SizeSuffix
|
|
.IP \[bu] 2
|
|
Default: 96M
|
|
.SS \-\-b2\-disable\-checksum
|
|
.PP
|
|
Disable checksums for large (> upload cutoff) files
|
|
.PP
|
|
Normally rclone will calculate the SHA1 checksum of the input before
|
|
uploading it so it can add it to metadata on the object.
|
|
This is great for data integrity checking but can cause long delays for
|
|
large files to start uploading.
|
|
.IP \[bu] 2
|
|
Config: disable_checksum
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_B2_DISABLE_CHECKSUM
|
|
.IP \[bu] 2
|
|
Type: bool
|
|
.IP \[bu] 2
|
|
Default: false
|
|
.SS \-\-b2\-download\-url
|
|
.PP
|
|
Custom endpoint for downloads.
|
|
.PP
|
|
This is usually set to a Cloudflare CDN URL as Backblaze offers free
|
|
egress for data downloaded through the Cloudflare network.
|
|
Rclone works with private buckets by sending an \[dq]Authorization\[dq]
|
|
header.
|
|
If the custom endpoint rewrites the requests for authentication, e.g.,
|
|
in Cloudflare Workers, this header needs to be handled properly.
|
|
Leave blank if you want to use the endpoint provided by Backblaze.
|
|
.IP \[bu] 2
|
|
Config: download_url
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_B2_DOWNLOAD_URL
|
|
.IP \[bu] 2
|
|
Type: string
|
|
.IP \[bu] 2
|
|
Default: \[dq]\[dq]
|
|
.SS \-\-b2\-download\-auth\-duration
|
|
.PP
|
|
Time before the authorization token will expire in s or suffix
|
|
ms|s|m|h|d.
|
|
.PP
|
|
The duration before the download authorization token will expire.
|
|
The minimum value is 1 second.
|
|
The maximum value is one week.
|
|
.IP \[bu] 2
|
|
Config: download_auth_duration
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_B2_DOWNLOAD_AUTH_DURATION
|
|
.IP \[bu] 2
|
|
Type: Duration
|
|
.IP \[bu] 2
|
|
Default: 1w
|
|
.SS \-\-b2\-memory\-pool\-flush\-time
|
|
.PP
|
|
How often internal memory buffer pools will be flushed.
|
|
Uploads which requires additional buffers (f.e multipart) will use
|
|
memory pool for allocations.
|
|
This option controls how often unused buffers will be removed from the
|
|
pool.
|
|
.IP \[bu] 2
|
|
Config: memory_pool_flush_time
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_B2_MEMORY_POOL_FLUSH_TIME
|
|
.IP \[bu] 2
|
|
Type: Duration
|
|
.IP \[bu] 2
|
|
Default: 1m0s
|
|
.SS \-\-b2\-memory\-pool\-use\-mmap
|
|
.PP
|
|
Whether to use mmap buffers in internal memory pool.
|
|
.IP \[bu] 2
|
|
Config: memory_pool_use_mmap
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_B2_MEMORY_POOL_USE_MMAP
|
|
.IP \[bu] 2
|
|
Type: bool
|
|
.IP \[bu] 2
|
|
Default: false
|
|
.SS \-\-b2\-encoding
|
|
.PP
|
|
This sets the encoding for the backend.
|
|
.PP
|
|
See: the encoding section in the
|
|
overview (https://rclone.org/overview/#encoding) for more info.
|
|
.IP \[bu] 2
|
|
Config: encoding
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_B2_ENCODING
|
|
.IP \[bu] 2
|
|
Type: MultiEncoder
|
|
.IP \[bu] 2
|
|
Default: Slash,BackSlash,Del,Ctl,InvalidUtf8,Dot
|
|
.SS Limitations
|
|
.PP
|
|
\f[C]rclone about\f[R] is not supported by the B2 backend.
|
|
Backends without this capability cannot determine free space for an
|
|
rclone mount or use policy \f[C]mfs\f[R] (most free space) as a member
|
|
of an rclone union remote.
|
|
.PP
|
|
See List of backends that do not support rclone
|
|
about (https://rclone.org/overview/#optional-features) See rclone
|
|
about (https://rclone.org/commands/rclone_about/)
|
|
.SS Box
|
|
.PP
|
|
Paths are specified as \f[C]remote:path\f[R]
|
|
.PP
|
|
Paths may be as deep as required, e.g.
|
|
\f[C]remote:directory/subdirectory\f[R].
|
|
.PP
|
|
The initial setup for Box involves getting a token from Box which you
|
|
can do either in your browser, or with a config.json downloaded from Box
|
|
to use JWT authentication.
|
|
\f[C]rclone config\f[R] walks you through it.
|
|
.PP
|
|
Here is an example of how to make a remote called \f[C]remote\f[R].
|
|
First run:
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone config
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
This will guide you through an interactive setup process:
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
No remotes found \- make a new one
|
|
n) New remote
|
|
s) Set configuration password
|
|
q) Quit config
|
|
n/s/q> n
|
|
name> remote
|
|
Type of storage to configure.
|
|
Choose a number from below, or type in your own value
|
|
[snip]
|
|
XX / Box
|
|
\[rs] \[dq]box\[dq]
|
|
[snip]
|
|
Storage> box
|
|
Box App Client Id \- leave blank normally.
|
|
client_id>
|
|
Box App Client Secret \- leave blank normally.
|
|
client_secret>
|
|
Box App config.json location
|
|
Leave blank normally.
|
|
Enter a string value. Press Enter for the default (\[dq]\[dq]).
|
|
box_config_file>
|
|
Box App Primary Access Token
|
|
Leave blank normally.
|
|
Enter a string value. Press Enter for the default (\[dq]\[dq]).
|
|
access_token>
|
|
|
|
Enter a string value. Press Enter for the default (\[dq]user\[dq]).
|
|
Choose a number from below, or type in your own value
|
|
1 / Rclone should act on behalf of a user
|
|
\[rs] \[dq]user\[dq]
|
|
2 / Rclone should act on behalf of a service account
|
|
\[rs] \[dq]enterprise\[dq]
|
|
box_sub_type>
|
|
Remote config
|
|
Use auto config?
|
|
* Say Y if not sure
|
|
* Say N if you are working on a remote or headless machine
|
|
y) Yes
|
|
n) No
|
|
y/n> y
|
|
If your browser doesn\[aq]t open automatically go to the following link: http://127.0.0.1:53682/auth
|
|
Log in and authorize rclone for access
|
|
Waiting for code...
|
|
Got code
|
|
\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-
|
|
[remote]
|
|
client_id =
|
|
client_secret =
|
|
token = {\[dq]access_token\[dq]:\[dq]XXX\[dq],\[dq]token_type\[dq]:\[dq]bearer\[dq],\[dq]refresh_token\[dq]:\[dq]XXX\[dq],\[dq]expiry\[dq]:\[dq]XXX\[dq]}
|
|
\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-
|
|
y) Yes this is OK
|
|
e) Edit this remote
|
|
d) Delete this remote
|
|
y/e/d> y
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
See the remote setup docs (https://rclone.org/remote_setup/) for how to
|
|
set it up on a machine with no Internet browser available.
|
|
.PP
|
|
Note that rclone runs a webserver on your local machine to collect the
|
|
token as returned from Box.
|
|
This only runs from the moment it opens your browser to the moment you
|
|
get back the verification code.
|
|
This is on \f[C]http://127.0.0.1:53682/\f[R] and this it may require you
|
|
to unblock it temporarily if you are running a host firewall.
|
|
.PP
|
|
Once configured you can then use \f[C]rclone\f[R] like this,
|
|
.PP
|
|
List directories in top level of your Box
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone lsd remote:
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
List all the files in your Box
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone ls remote:
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
To copy a local directory to an Box directory called backup
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone copy /home/source remote:backup
|
|
\f[R]
|
|
.fi
|
|
.SS Using rclone with an Enterprise account with SSO
|
|
.PP
|
|
If you have an \[dq]Enterprise\[dq] account type with Box with single
|
|
sign on (SSO), you need to create a password to use Box with rclone.
|
|
This can be done at your Enterprise Box account by going to Settings,
|
|
\[dq]Account\[dq] Tab, and then set the password in the
|
|
\[dq]Authentication\[dq] field.
|
|
.PP
|
|
Once you have done this, you can setup your Enterprise Box account using
|
|
the same procedure detailed above in the, using the password you have
|
|
just set.
|
|
.SS Invalid refresh token
|
|
.PP
|
|
According to the box
|
|
docs (https://developer.box.com/v2.0/docs/oauth-20#section-6-using-the-access-and-refresh-tokens):
|
|
.RS
|
|
.PP
|
|
Each refresh_token is valid for one use in 60 days.
|
|
.RE
|
|
.PP
|
|
This means that if you
|
|
.IP \[bu] 2
|
|
Don\[aq]t use the box remote for 60 days
|
|
.IP \[bu] 2
|
|
Copy the config file with a box refresh token in and use it in two
|
|
places
|
|
.IP \[bu] 2
|
|
Get an error on a token refresh
|
|
.PP
|
|
then rclone will return an error which includes the text
|
|
\f[C]Invalid refresh token\f[R].
|
|
.PP
|
|
To fix this you will need to use oauth2 again to update the refresh
|
|
token.
|
|
You can use the methods in the remote setup
|
|
docs (https://rclone.org/remote_setup/), bearing in mind that if you use
|
|
the copy the config file method, you should not use that remote on the
|
|
computer you did the authentication on.
|
|
.PP
|
|
Here is how to do it.
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
$ rclone config
|
|
Current remotes:
|
|
|
|
Name Type
|
|
==== ====
|
|
remote box
|
|
|
|
e) Edit existing remote
|
|
n) New remote
|
|
d) Delete remote
|
|
r) Rename remote
|
|
c) Copy remote
|
|
s) Set configuration password
|
|
q) Quit config
|
|
e/n/d/r/c/s/q> e
|
|
Choose a number from below, or type in an existing value
|
|
1 > remote
|
|
remote> remote
|
|
\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-
|
|
[remote]
|
|
type = box
|
|
token = {\[dq]access_token\[dq]:\[dq]XXX\[dq],\[dq]token_type\[dq]:\[dq]bearer\[dq],\[dq]refresh_token\[dq]:\[dq]XXX\[dq],\[dq]expiry\[dq]:\[dq]2017\-07\-08T23:40:08.059167677+01:00\[dq]}
|
|
\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-
|
|
Edit remote
|
|
Value \[dq]client_id\[dq] = \[dq]\[dq]
|
|
Edit? (y/n)>
|
|
y) Yes
|
|
n) No
|
|
y/n> n
|
|
Value \[dq]client_secret\[dq] = \[dq]\[dq]
|
|
Edit? (y/n)>
|
|
y) Yes
|
|
n) No
|
|
y/n> n
|
|
Remote config
|
|
Already have a token \- refresh?
|
|
y) Yes
|
|
n) No
|
|
y/n> y
|
|
Use auto config?
|
|
* Say Y if not sure
|
|
* Say N if you are working on a remote or headless machine
|
|
y) Yes
|
|
n) No
|
|
y/n> y
|
|
If your browser doesn\[aq]t open automatically go to the following link: http://127.0.0.1:53682/auth
|
|
Log in and authorize rclone for access
|
|
Waiting for code...
|
|
Got code
|
|
\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-
|
|
[remote]
|
|
type = box
|
|
token = {\[dq]access_token\[dq]:\[dq]YYY\[dq],\[dq]token_type\[dq]:\[dq]bearer\[dq],\[dq]refresh_token\[dq]:\[dq]YYY\[dq],\[dq]expiry\[dq]:\[dq]2017\-07\-23T12:22:29.259137901+01:00\[dq]}
|
|
\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-
|
|
y) Yes this is OK
|
|
e) Edit this remote
|
|
d) Delete this remote
|
|
y/e/d> y
|
|
\f[R]
|
|
.fi
|
|
.SS Modified time and hashes
|
|
.PP
|
|
Box allows modification times to be set on objects accurate to 1 second.
|
|
These will be used to detect whether objects need syncing or not.
|
|
.PP
|
|
Box supports SHA1 type hashes, so you can use the \f[C]\-\-checksum\f[R]
|
|
flag.
|
|
.SS Restricted filename characters
|
|
.PP
|
|
In addition to the default restricted characters
|
|
set (https://rclone.org/overview/#restricted-characters) the following
|
|
characters are also replaced:
|
|
.PP
|
|
.TS
|
|
tab(@);
|
|
l c c.
|
|
T{
|
|
Character
|
|
T}@T{
|
|
Value
|
|
T}@T{
|
|
Replacement
|
|
T}
|
|
_
|
|
T{
|
|
\[rs]
|
|
T}@T{
|
|
0x5C
|
|
T}@T{
|
|
\[uFF3C]
|
|
T}
|
|
.TE
|
|
.PP
|
|
File names can also not end with the following characters.
|
|
These only get replaced if they are the last character in the name:
|
|
.PP
|
|
.TS
|
|
tab(@);
|
|
l c c.
|
|
T{
|
|
Character
|
|
T}@T{
|
|
Value
|
|
T}@T{
|
|
Replacement
|
|
T}
|
|
_
|
|
T{
|
|
SP
|
|
T}@T{
|
|
0x20
|
|
T}@T{
|
|
\[u2420]
|
|
T}
|
|
.TE
|
|
.PP
|
|
Invalid UTF\-8 bytes will also be
|
|
replaced (https://rclone.org/overview/#invalid-utf8), as they can\[aq]t
|
|
be used in JSON strings.
|
|
.SS Transfers
|
|
.PP
|
|
For files above 50MB rclone will use a chunked transfer.
|
|
Rclone will upload up to \f[C]\-\-transfers\f[R] chunks at the same time
|
|
(shared among all the multipart uploads).
|
|
Chunks are buffered in memory and are normally 8MB so increasing
|
|
\f[C]\-\-transfers\f[R] will increase memory use.
|
|
.SS Deleting files
|
|
.PP
|
|
Depending on the enterprise settings for your user, the item will either
|
|
be actually deleted from Box or moved to the trash.
|
|
.PP
|
|
Emptying the trash is supported via the rclone however cleanup command
|
|
however this deletes every trashed file and folder individually so it
|
|
may take a very long time.
|
|
Emptying the trash via the WebUI does not have this limitation so it is
|
|
advised to empty the trash via the WebUI.
|
|
.SS Root folder ID
|
|
.PP
|
|
You can set the \f[C]root_folder_id\f[R] for rclone.
|
|
This is the directory (identified by its \f[C]Folder ID\f[R]) that
|
|
rclone considers to be the root of your Box drive.
|
|
.PP
|
|
Normally you will leave this blank and rclone will determine the correct
|
|
root to use itself.
|
|
.PP
|
|
However you can set this to restrict rclone to a specific folder
|
|
hierarchy.
|
|
.PP
|
|
In order to do this you will have to find the \f[C]Folder ID\f[R] of the
|
|
directory you wish rclone to display.
|
|
This will be the last segment of the URL when you open the relevant
|
|
folder in the Box web interface.
|
|
.PP
|
|
So if the folder you want rclone to use has a URL which looks like
|
|
\f[C]https://app.box.com/folder/11xxxxxxxxx8\f[R] in the browser, then
|
|
you use \f[C]11xxxxxxxxx8\f[R] as the \f[C]root_folder_id\f[R] in the
|
|
config.
|
|
.SS Standard Options
|
|
.PP
|
|
Here are the standard options specific to box (Box).
|
|
.SS \-\-box\-client\-id
|
|
.PP
|
|
OAuth Client Id Leave blank normally.
|
|
.IP \[bu] 2
|
|
Config: client_id
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_BOX_CLIENT_ID
|
|
.IP \[bu] 2
|
|
Type: string
|
|
.IP \[bu] 2
|
|
Default: \[dq]\[dq]
|
|
.SS \-\-box\-client\-secret
|
|
.PP
|
|
OAuth Client Secret Leave blank normally.
|
|
.IP \[bu] 2
|
|
Config: client_secret
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_BOX_CLIENT_SECRET
|
|
.IP \[bu] 2
|
|
Type: string
|
|
.IP \[bu] 2
|
|
Default: \[dq]\[dq]
|
|
.SS \-\-box\-box\-config\-file
|
|
.PP
|
|
Box App config.json location Leave blank normally.
|
|
.PP
|
|
Leading \f[C]\[ti]\f[R] will be expanded in the file name as will
|
|
environment variables such as \f[C]${RCLONE_CONFIG_DIR}\f[R].
|
|
.IP \[bu] 2
|
|
Config: box_config_file
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_BOX_BOX_CONFIG_FILE
|
|
.IP \[bu] 2
|
|
Type: string
|
|
.IP \[bu] 2
|
|
Default: \[dq]\[dq]
|
|
.SS \-\-box\-access\-token
|
|
.PP
|
|
Box App Primary Access Token Leave blank normally.
|
|
.IP \[bu] 2
|
|
Config: access_token
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_BOX_ACCESS_TOKEN
|
|
.IP \[bu] 2
|
|
Type: string
|
|
.IP \[bu] 2
|
|
Default: \[dq]\[dq]
|
|
.SS \-\-box\-box\-sub\-type
|
|
.IP \[bu] 2
|
|
Config: box_sub_type
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_BOX_BOX_SUB_TYPE
|
|
.IP \[bu] 2
|
|
Type: string
|
|
.IP \[bu] 2
|
|
Default: \[dq]user\[dq]
|
|
.IP \[bu] 2
|
|
Examples:
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
\[dq]user\[dq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Rclone should act on behalf of a user
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[dq]enterprise\[dq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Rclone should act on behalf of a service account
|
|
.RE
|
|
.RE
|
|
.SS Advanced Options
|
|
.PP
|
|
Here are the advanced options specific to box (Box).
|
|
.SS \-\-box\-token
|
|
.PP
|
|
OAuth Access Token as a JSON blob.
|
|
.IP \[bu] 2
|
|
Config: token
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_BOX_TOKEN
|
|
.IP \[bu] 2
|
|
Type: string
|
|
.IP \[bu] 2
|
|
Default: \[dq]\[dq]
|
|
.SS \-\-box\-auth\-url
|
|
.PP
|
|
Auth server URL.
|
|
Leave blank to use the provider defaults.
|
|
.IP \[bu] 2
|
|
Config: auth_url
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_BOX_AUTH_URL
|
|
.IP \[bu] 2
|
|
Type: string
|
|
.IP \[bu] 2
|
|
Default: \[dq]\[dq]
|
|
.SS \-\-box\-token\-url
|
|
.PP
|
|
Token server url.
|
|
Leave blank to use the provider defaults.
|
|
.IP \[bu] 2
|
|
Config: token_url
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_BOX_TOKEN_URL
|
|
.IP \[bu] 2
|
|
Type: string
|
|
.IP \[bu] 2
|
|
Default: \[dq]\[dq]
|
|
.SS \-\-box\-root\-folder\-id
|
|
.PP
|
|
Fill in for rclone to use a non root folder as its starting point.
|
|
.IP \[bu] 2
|
|
Config: root_folder_id
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_BOX_ROOT_FOLDER_ID
|
|
.IP \[bu] 2
|
|
Type: string
|
|
.IP \[bu] 2
|
|
Default: \[dq]0\[dq]
|
|
.SS \-\-box\-upload\-cutoff
|
|
.PP
|
|
Cutoff for switching to multipart upload (>= 50MB).
|
|
.IP \[bu] 2
|
|
Config: upload_cutoff
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_BOX_UPLOAD_CUTOFF
|
|
.IP \[bu] 2
|
|
Type: SizeSuffix
|
|
.IP \[bu] 2
|
|
Default: 50M
|
|
.SS \-\-box\-commit\-retries
|
|
.PP
|
|
Max number of times to try committing a multipart file.
|
|
.IP \[bu] 2
|
|
Config: commit_retries
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_BOX_COMMIT_RETRIES
|
|
.IP \[bu] 2
|
|
Type: int
|
|
.IP \[bu] 2
|
|
Default: 100
|
|
.SS \-\-box\-encoding
|
|
.PP
|
|
This sets the encoding for the backend.
|
|
.PP
|
|
See: the encoding section in the
|
|
overview (https://rclone.org/overview/#encoding) for more info.
|
|
.IP \[bu] 2
|
|
Config: encoding
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_BOX_ENCODING
|
|
.IP \[bu] 2
|
|
Type: MultiEncoder
|
|
.IP \[bu] 2
|
|
Default: Slash,BackSlash,Del,Ctl,RightSpace,InvalidUtf8,Dot
|
|
.SS Limitations
|
|
.PP
|
|
Note that Box is case insensitive so you can\[aq]t have a file called
|
|
\[dq]Hello.doc\[dq] and one called \[dq]hello.doc\[dq].
|
|
.PP
|
|
Box file names can\[aq]t have the \f[C]\[rs]\f[R] character in.
|
|
rclone maps this to and from an identical looking unicode equivalent
|
|
\f[C]\[uFF3C]\f[R] (U+FF3C Fullwidth Reverse Solidus).
|
|
.PP
|
|
Box only supports filenames up to 255 characters in length.
|
|
.PP
|
|
\f[C]rclone about\f[R] is not supported by the Box backend.
|
|
Backends without this capability cannot determine free space for an
|
|
rclone mount or use policy \f[C]mfs\f[R] (most free space) as a member
|
|
of an rclone union remote.
|
|
.PP
|
|
See List of backends that do not support rclone
|
|
about (https://rclone.org/overview/#optional-features) See rclone
|
|
about (https://rclone.org/commands/rclone_about/)
|
|
.SS Cache (BETA)
|
|
.PP
|
|
The \f[C]cache\f[R] remote wraps another existing remote and stores file
|
|
structure and its data for long running tasks like
|
|
\f[C]rclone mount\f[R].
|
|
.SS Status
|
|
.PP
|
|
The cache backend code is working but it currently doesn\[aq]t have a
|
|
maintainer so there are outstanding
|
|
bugs (https://github.com/rclone/rclone/issues?q=is%3Aopen+is%3Aissue+label%3Abug+label%3A%22Remote%3A+Cache%22)
|
|
which aren\[aq]t getting fixed.
|
|
.PP
|
|
The cache backend is due to be phased out in favour of the VFS caching
|
|
layer eventually which is more tightly integrated into rclone.
|
|
.PP
|
|
Until this happens we recommend only using the cache backend if you find
|
|
you can\[aq]t work without it.
|
|
There are many docs online describing the use of the cache backend to
|
|
minimize API hits and by\-and\-large these are out of date and the cache
|
|
backend isn\[aq]t needed in those scenarios any more.
|
|
.SS Setup
|
|
.PP
|
|
To get started you just need to have an existing remote which can be
|
|
configured with \f[C]cache\f[R].
|
|
.PP
|
|
Here is an example of how to make a remote called \f[C]test\-cache\f[R].
|
|
First run:
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone config
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
This will guide you through an interactive setup process:
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
No remotes found \- make a new one
|
|
n) New remote
|
|
r) Rename remote
|
|
c) Copy remote
|
|
s) Set configuration password
|
|
q) Quit config
|
|
n/r/c/s/q> n
|
|
name> test\-cache
|
|
Type of storage to configure.
|
|
Choose a number from below, or type in your own value
|
|
[snip]
|
|
XX / Cache a remote
|
|
\[rs] \[dq]cache\[dq]
|
|
[snip]
|
|
Storage> cache
|
|
Remote to cache.
|
|
Normally should contain a \[aq]:\[aq] and a path, e.g. \[dq]myremote:path/to/dir\[dq],
|
|
\[dq]myremote:bucket\[dq] or maybe \[dq]myremote:\[dq] (not recommended).
|
|
remote> local:/test
|
|
Optional: The URL of the Plex server
|
|
plex_url> http://127.0.0.1:32400
|
|
Optional: The username of the Plex user
|
|
plex_username> dummyusername
|
|
Optional: The password of the Plex user
|
|
y) Yes type in my own password
|
|
g) Generate random password
|
|
n) No leave this optional password blank
|
|
y/g/n> y
|
|
Enter the password:
|
|
password:
|
|
Confirm the password:
|
|
password:
|
|
The size of a chunk. Lower value good for slow connections but can affect seamless reading.
|
|
Default: 5M
|
|
Choose a number from below, or type in your own value
|
|
1 / 1MB
|
|
\[rs] \[dq]1m\[dq]
|
|
2 / 5 MB
|
|
\[rs] \[dq]5M\[dq]
|
|
3 / 10 MB
|
|
\[rs] \[dq]10M\[dq]
|
|
chunk_size> 2
|
|
How much time should object info (file size, file hashes, etc.) be stored in cache. Use a very high value if you don\[aq]t plan on changing the source FS from outside the cache.
|
|
Accepted units are: \[dq]s\[dq], \[dq]m\[dq], \[dq]h\[dq].
|
|
Default: 5m
|
|
Choose a number from below, or type in your own value
|
|
1 / 1 hour
|
|
\[rs] \[dq]1h\[dq]
|
|
2 / 24 hours
|
|
\[rs] \[dq]24h\[dq]
|
|
3 / 24 hours
|
|
\[rs] \[dq]48h\[dq]
|
|
info_age> 2
|
|
The maximum size of stored chunks. When the storage grows beyond this size, the oldest chunks will be deleted.
|
|
Default: 10G
|
|
Choose a number from below, or type in your own value
|
|
1 / 500 MB
|
|
\[rs] \[dq]500M\[dq]
|
|
2 / 1 GB
|
|
\[rs] \[dq]1G\[dq]
|
|
3 / 10 GB
|
|
\[rs] \[dq]10G\[dq]
|
|
chunk_total_size> 3
|
|
Remote config
|
|
\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-
|
|
[test\-cache]
|
|
remote = local:/test
|
|
plex_url = http://127.0.0.1:32400
|
|
plex_username = dummyusername
|
|
plex_password = *** ENCRYPTED ***
|
|
chunk_size = 5M
|
|
info_age = 48h
|
|
chunk_total_size = 10G
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
You can then use it like this,
|
|
.PP
|
|
List directories in top level of your drive
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone lsd test\-cache:
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
List all the files in your drive
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone ls test\-cache:
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
To start a cached mount
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone mount \-\-allow\-other test\-cache: /var/tmp/test\-cache
|
|
\f[R]
|
|
.fi
|
|
.SS Write Features
|
|
.SS Offline uploading
|
|
.PP
|
|
In an effort to make writing through cache more reliable, the backend
|
|
now supports this feature which can be activated by specifying a
|
|
\f[C]cache\-tmp\-upload\-path\f[R].
|
|
.PP
|
|
A files goes through these states when using this feature:
|
|
.IP "1." 3
|
|
An upload is started (usually by copying a file on the cache remote)
|
|
.IP "2." 3
|
|
When the copy to the temporary location is complete the file is part of
|
|
the cached remote and looks and behaves like any other file (reading
|
|
included)
|
|
.IP "3." 3
|
|
After \f[C]cache\-tmp\-wait\-time\f[R] passes and the file is next in
|
|
line, \f[C]rclone move\f[R] is used to move the file to the cloud
|
|
provider
|
|
.IP "4." 3
|
|
Reading the file still works during the upload but most modifications on
|
|
it will be prohibited
|
|
.IP "5." 3
|
|
Once the move is complete the file is unlocked for modifications as it
|
|
becomes as any other regular file
|
|
.IP "6." 3
|
|
If the file is being read through \f[C]cache\f[R] when it\[aq]s actually
|
|
deleted from the temporary path then \f[C]cache\f[R] will simply swap
|
|
the source to the cloud provider without interrupting the reading (small
|
|
blip can happen though)
|
|
.PP
|
|
Files are uploaded in sequence and only one file is uploaded at a time.
|
|
Uploads will be stored in a queue and be processed based on the order
|
|
they were added.
|
|
The queue and the temporary storage is persistent across restarts but
|
|
can be cleared on startup with the \f[C]\-\-cache\-db\-purge\f[R] flag.
|
|
.SS Write Support
|
|
.PP
|
|
Writes are supported through \f[C]cache\f[R].
|
|
One caveat is that a mounted cache remote does not add any retry or
|
|
fallback mechanism to the upload operation.
|
|
This will depend on the implementation of the wrapped remote.
|
|
Consider using \f[C]Offline uploading\f[R] for reliable writes.
|
|
.PP
|
|
One special case is covered with \f[C]cache\-writes\f[R] which will
|
|
cache the file data at the same time as the upload when it is enabled
|
|
making it available from the cache store immediately once the upload is
|
|
finished.
|
|
.SS Read Features
|
|
.SS Multiple connections
|
|
.PP
|
|
To counter the high latency between a local PC where rclone is running
|
|
and cloud providers, the cache remote can split multiple requests to the
|
|
cloud provider for smaller file chunks and combines them together
|
|
locally where they can be available almost immediately before the reader
|
|
usually needs them.
|
|
.PP
|
|
This is similar to buffering when media files are played online.
|
|
Rclone will stay around the current marker but always try its best to
|
|
stay ahead and prepare the data before.
|
|
.SS Plex Integration
|
|
.PP
|
|
There is a direct integration with Plex which allows cache to detect
|
|
during reading if the file is in playback or not.
|
|
This helps cache to adapt how it queries the cloud provider depending on
|
|
what is needed for.
|
|
.PP
|
|
Scans will have a minimum amount of workers (1) while in a confirmed
|
|
playback cache will deploy the configured number of workers.
|
|
.PP
|
|
This integration opens the doorway to additional performance
|
|
improvements which will be explored in the near future.
|
|
.PP
|
|
\f[B]Note:\f[R] If Plex options are not configured, \f[C]cache\f[R] will
|
|
function with its configured options without adapting any of its
|
|
settings.
|
|
.PP
|
|
How to enable?
|
|
Run \f[C]rclone config\f[R] and add all the Plex options (endpoint,
|
|
username and password) in your remote and it will be automatically
|
|
enabled.
|
|
.PP
|
|
Affected settings: \- \f[C]cache\-workers\f[R]: \f[I]Configured
|
|
value\f[R] during confirmed playback or \f[I]1\f[R] all the other times
|
|
.SS Certificate Validation
|
|
.PP
|
|
When the Plex server is configured to only accept secure connections, it
|
|
is possible to use \f[C].plex.direct\f[R] URLs to ensure certificate
|
|
validation succeeds.
|
|
These URLs are used by Plex internally to connect to the Plex server
|
|
securely.
|
|
.PP
|
|
The format for these URLs is the following:
|
|
.PP
|
|
\f[C]https://ip\-with\-dots\-replaced.server\-hash.plex.direct:32400/\f[R]
|
|
.PP
|
|
The \f[C]ip\-with\-dots\-replaced\f[R] part can be any IPv4 address,
|
|
where the dots have been replaced with dashes, e.g.
|
|
\f[C]127.0.0.1\f[R] becomes \f[C]127\-0\-0\-1\f[R].
|
|
.PP
|
|
To get the \f[C]server\-hash\f[R] part, the easiest way is to visit
|
|
.PP
|
|
https://plex.tv/api/resources?includeHttps=1&X\-Plex\-Token=your\-plex\-token
|
|
.PP
|
|
This page will list all the available Plex servers for your account with
|
|
at least one \f[C].plex.direct\f[R] link for each.
|
|
Copy one URL and replace the IP address with the desired address.
|
|
This can be used as the \f[C]plex_url\f[R] value.
|
|
.SS Known issues
|
|
.SS Mount and \-\-dir\-cache\-time
|
|
.PP
|
|
\-\-dir\-cache\-time controls the first layer of directory caching which
|
|
works at the mount layer.
|
|
Being an independent caching mechanism from the \f[C]cache\f[R] backend,
|
|
it will manage its own entries based on the configured time.
|
|
.PP
|
|
To avoid getting in a scenario where dir cache has obsolete data and
|
|
cache would have the correct one, try to set
|
|
\f[C]\-\-dir\-cache\-time\f[R] to a lower time than
|
|
\f[C]\-\-cache\-info\-age\f[R].
|
|
Default values are already configured in this way.
|
|
.SS Windows support \- Experimental
|
|
.PP
|
|
There are a couple of issues with Windows \f[C]mount\f[R] functionality
|
|
that still require some investigations.
|
|
It should be considered as experimental thus far as fixes come in for
|
|
this OS.
|
|
.PP
|
|
Most of the issues seem to be related to the difference between
|
|
filesystems on Linux flavors and Windows as cache is heavily dependent
|
|
on them.
|
|
.PP
|
|
Any reports or feedback on how cache behaves on this OS is greatly
|
|
appreciated.
|
|
.IP \[bu] 2
|
|
https://github.com/rclone/rclone/issues/1935
|
|
.IP \[bu] 2
|
|
https://github.com/rclone/rclone/issues/1907
|
|
.IP \[bu] 2
|
|
https://github.com/rclone/rclone/issues/1834
|
|
.SS Risk of throttling
|
|
.PP
|
|
Future iterations of the cache backend will make use of the pooling
|
|
functionality of the cloud provider to synchronize and at the same time
|
|
make writing through it more tolerant to failures.
|
|
.PP
|
|
There are a couple of enhancements in track to add these but in the
|
|
meantime there is a valid concern that the expiring cache listings can
|
|
lead to cloud provider throttles or bans due to repeated queries on it
|
|
for very large mounts.
|
|
.PP
|
|
Some recommendations: \- don\[aq]t use a very small interval for entry
|
|
information (\f[C]\-\-cache\-info\-age\f[R]) \- while writes aren\[aq]t
|
|
yet optimised, you can still write through \f[C]cache\f[R] which gives
|
|
you the advantage of adding the file in the cache at the same time if
|
|
configured to do so.
|
|
.PP
|
|
Future enhancements:
|
|
.IP \[bu] 2
|
|
https://github.com/rclone/rclone/issues/1937
|
|
.IP \[bu] 2
|
|
https://github.com/rclone/rclone/issues/1936
|
|
.SS cache and crypt
|
|
.PP
|
|
One common scenario is to keep your data encrypted in the cloud provider
|
|
using the \f[C]crypt\f[R] remote.
|
|
\f[C]crypt\f[R] uses a similar technique to wrap around an existing
|
|
remote and handles this translation in a seamless way.
|
|
.PP
|
|
There is an issue with wrapping the remotes in this order: \f[B]cloud
|
|
remote\f[R] \-> \f[B]crypt\f[R] \-> \f[B]cache\f[R]
|
|
.PP
|
|
During testing, I experienced a lot of bans with the remotes in this
|
|
order.
|
|
I suspect it might be related to how crypt opens files on the cloud
|
|
provider which makes it think we\[aq]re downloading the full file
|
|
instead of small chunks.
|
|
Organizing the remotes in this order yields better results: \f[B]cloud
|
|
remote\f[R] \-> \f[B]cache\f[R] \-> \f[B]crypt\f[R]
|
|
.SS absolute remote paths
|
|
.PP
|
|
\f[C]cache\f[R] can not differentiate between relative and absolute
|
|
paths for the wrapped remote.
|
|
Any path given in the \f[C]remote\f[R] config setting and on the command
|
|
line will be passed to the wrapped remote as is, but for storing the
|
|
chunks on disk the path will be made relative by removing any leading
|
|
\f[C]/\f[R] character.
|
|
.PP
|
|
This behavior is irrelevant for most backend types, but there are
|
|
backends where a leading \f[C]/\f[R] changes the effective directory,
|
|
e.g.
|
|
in the \f[C]sftp\f[R] backend paths starting with a \f[C]/\f[R] are
|
|
relative to the root of the SSH server and paths without are relative to
|
|
the user home directory.
|
|
As a result \f[C]sftp:bin\f[R] and \f[C]sftp:/bin\f[R] will share the
|
|
same cache folder, even if they represent a different directory on the
|
|
SSH server.
|
|
.SS Cache and Remote Control (\-\-rc)
|
|
.PP
|
|
Cache supports the new \f[C]\-\-rc\f[R] mode in rclone and can be remote
|
|
controlled through the following end points: By default, the listener is
|
|
disabled if you do not add the flag.
|
|
.SS rc cache/expire
|
|
.PP
|
|
Purge a remote from the cache backend.
|
|
Supports either a directory or a file.
|
|
It supports both encrypted and unencrypted file names if cache is
|
|
wrapped by crypt.
|
|
.PP
|
|
Params: \- \f[B]remote\f[R] = path to remote \f[B](required)\f[R] \-
|
|
\f[B]withData\f[R] = true/false to delete cached data (chunks) as well
|
|
\f[I](optional, false by default)\f[R]
|
|
.SS Standard Options
|
|
.PP
|
|
Here are the standard options specific to cache (Cache a remote).
|
|
.SS \-\-cache\-remote
|
|
.PP
|
|
Remote to cache.
|
|
Normally should contain a \[aq]:\[aq] and a path, e.g.
|
|
\[dq]myremote:path/to/dir\[dq], \[dq]myremote:bucket\[dq] or maybe
|
|
\[dq]myremote:\[dq] (not recommended).
|
|
.IP \[bu] 2
|
|
Config: remote
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_CACHE_REMOTE
|
|
.IP \[bu] 2
|
|
Type: string
|
|
.IP \[bu] 2
|
|
Default: \[dq]\[dq]
|
|
.SS \-\-cache\-plex\-url
|
|
.PP
|
|
The URL of the Plex server
|
|
.IP \[bu] 2
|
|
Config: plex_url
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_CACHE_PLEX_URL
|
|
.IP \[bu] 2
|
|
Type: string
|
|
.IP \[bu] 2
|
|
Default: \[dq]\[dq]
|
|
.SS \-\-cache\-plex\-username
|
|
.PP
|
|
The username of the Plex user
|
|
.IP \[bu] 2
|
|
Config: plex_username
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_CACHE_PLEX_USERNAME
|
|
.IP \[bu] 2
|
|
Type: string
|
|
.IP \[bu] 2
|
|
Default: \[dq]\[dq]
|
|
.SS \-\-cache\-plex\-password
|
|
.PP
|
|
The password of the Plex user
|
|
.PP
|
|
\f[B]NB\f[R] Input to this must be obscured \- see rclone
|
|
obscure (https://rclone.org/commands/rclone_obscure/).
|
|
.IP \[bu] 2
|
|
Config: plex_password
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_CACHE_PLEX_PASSWORD
|
|
.IP \[bu] 2
|
|
Type: string
|
|
.IP \[bu] 2
|
|
Default: \[dq]\[dq]
|
|
.SS \-\-cache\-chunk\-size
|
|
.PP
|
|
The size of a chunk (partial file data).
|
|
.PP
|
|
Use lower numbers for slower connections.
|
|
If the chunk size is changed, any downloaded chunks will be invalid and
|
|
cache\-chunk\-path will need to be cleared or unexpected EOF errors will
|
|
occur.
|
|
.IP \[bu] 2
|
|
Config: chunk_size
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_CACHE_CHUNK_SIZE
|
|
.IP \[bu] 2
|
|
Type: SizeSuffix
|
|
.IP \[bu] 2
|
|
Default: 5M
|
|
.IP \[bu] 2
|
|
Examples:
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
\[dq]1m\[dq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
1MB
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[dq]5M\[dq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
5 MB
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[dq]10M\[dq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
10 MB
|
|
.RE
|
|
.RE
|
|
.SS \-\-cache\-info\-age
|
|
.PP
|
|
How long to cache file structure information (directory listings, file
|
|
size, times, etc.).
|
|
If all write operations are done through the cache then you can safely
|
|
make this value very large as the cache store will also be updated in
|
|
real time.
|
|
.IP \[bu] 2
|
|
Config: info_age
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_CACHE_INFO_AGE
|
|
.IP \[bu] 2
|
|
Type: Duration
|
|
.IP \[bu] 2
|
|
Default: 6h0m0s
|
|
.IP \[bu] 2
|
|
Examples:
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
\[dq]1h\[dq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
1 hour
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[dq]24h\[dq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
24 hours
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[dq]48h\[dq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
48 hours
|
|
.RE
|
|
.RE
|
|
.SS \-\-cache\-chunk\-total\-size
|
|
.PP
|
|
The total size that the chunks can take up on the local disk.
|
|
.PP
|
|
If the cache exceeds this value then it will start to delete the oldest
|
|
chunks until it goes under this value.
|
|
.IP \[bu] 2
|
|
Config: chunk_total_size
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_CACHE_CHUNK_TOTAL_SIZE
|
|
.IP \[bu] 2
|
|
Type: SizeSuffix
|
|
.IP \[bu] 2
|
|
Default: 10G
|
|
.IP \[bu] 2
|
|
Examples:
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
\[dq]500M\[dq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
500 MB
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[dq]1G\[dq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
1 GB
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[dq]10G\[dq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
10 GB
|
|
.RE
|
|
.RE
|
|
.SS Advanced Options
|
|
.PP
|
|
Here are the advanced options specific to cache (Cache a remote).
|
|
.SS \-\-cache\-plex\-token
|
|
.PP
|
|
The plex token for authentication \- auto set normally
|
|
.IP \[bu] 2
|
|
Config: plex_token
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_CACHE_PLEX_TOKEN
|
|
.IP \[bu] 2
|
|
Type: string
|
|
.IP \[bu] 2
|
|
Default: \[dq]\[dq]
|
|
.SS \-\-cache\-plex\-insecure
|
|
.PP
|
|
Skip all certificate verification when connecting to the Plex server
|
|
.IP \[bu] 2
|
|
Config: plex_insecure
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_CACHE_PLEX_INSECURE
|
|
.IP \[bu] 2
|
|
Type: string
|
|
.IP \[bu] 2
|
|
Default: \[dq]\[dq]
|
|
.SS \-\-cache\-db\-path
|
|
.PP
|
|
Directory to store file structure metadata DB.
|
|
The remote name is used as the DB file name.
|
|
.IP \[bu] 2
|
|
Config: db_path
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_CACHE_DB_PATH
|
|
.IP \[bu] 2
|
|
Type: string
|
|
.IP \[bu] 2
|
|
Default: \[dq]$HOME/.cache/rclone/cache\-backend\[dq]
|
|
.SS \-\-cache\-chunk\-path
|
|
.PP
|
|
Directory to cache chunk files.
|
|
.PP
|
|
Path to where partial file data (chunks) are stored locally.
|
|
The remote name is appended to the final path.
|
|
.PP
|
|
This config follows the \[dq]\-\-cache\-db\-path\[dq].
|
|
If you specify a custom location for \[dq]\-\-cache\-db\-path\[dq] and
|
|
don\[aq]t specify one for \[dq]\-\-cache\-chunk\-path\[dq] then
|
|
\[dq]\-\-cache\-chunk\-path\[dq] will use the same path as
|
|
\[dq]\-\-cache\-db\-path\[dq].
|
|
.IP \[bu] 2
|
|
Config: chunk_path
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_CACHE_CHUNK_PATH
|
|
.IP \[bu] 2
|
|
Type: string
|
|
.IP \[bu] 2
|
|
Default: \[dq]$HOME/.cache/rclone/cache\-backend\[dq]
|
|
.SS \-\-cache\-db\-purge
|
|
.PP
|
|
Clear all the cached data for this remote on start.
|
|
.IP \[bu] 2
|
|
Config: db_purge
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_CACHE_DB_PURGE
|
|
.IP \[bu] 2
|
|
Type: bool
|
|
.IP \[bu] 2
|
|
Default: false
|
|
.SS \-\-cache\-chunk\-clean\-interval
|
|
.PP
|
|
How often should the cache perform cleanups of the chunk storage.
|
|
The default value should be ok for most people.
|
|
If you find that the cache goes over \[dq]cache\-chunk\-total\-size\[dq]
|
|
too often then try to lower this value to force it to perform cleanups
|
|
more often.
|
|
.IP \[bu] 2
|
|
Config: chunk_clean_interval
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_CACHE_CHUNK_CLEAN_INTERVAL
|
|
.IP \[bu] 2
|
|
Type: Duration
|
|
.IP \[bu] 2
|
|
Default: 1m0s
|
|
.SS \-\-cache\-read\-retries
|
|
.PP
|
|
How many times to retry a read from a cache storage.
|
|
.PP
|
|
Since reading from a cache stream is independent from downloading file
|
|
data, readers can get to a point where there\[aq]s no more data in the
|
|
cache.
|
|
Most of the times this can indicate a connectivity issue if cache
|
|
isn\[aq]t able to provide file data anymore.
|
|
.PP
|
|
For really slow connections, increase this to a point where the stream
|
|
is able to provide data but your experience will be very stuttering.
|
|
.IP \[bu] 2
|
|
Config: read_retries
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_CACHE_READ_RETRIES
|
|
.IP \[bu] 2
|
|
Type: int
|
|
.IP \[bu] 2
|
|
Default: 10
|
|
.SS \-\-cache\-workers
|
|
.PP
|
|
How many workers should run in parallel to download chunks.
|
|
.PP
|
|
Higher values will mean more parallel processing (better CPU needed) and
|
|
more concurrent requests on the cloud provider.
|
|
This impacts several aspects like the cloud provider API limits, more
|
|
stress on the hardware that rclone runs on but it also means that
|
|
streams will be more fluid and data will be available much more faster
|
|
to readers.
|
|
.PP
|
|
\f[B]Note\f[R]: If the optional Plex integration is enabled then this
|
|
setting will adapt to the type of reading performed and the value
|
|
specified here will be used as a maximum number of workers to use.
|
|
.IP \[bu] 2
|
|
Config: workers
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_CACHE_WORKERS
|
|
.IP \[bu] 2
|
|
Type: int
|
|
.IP \[bu] 2
|
|
Default: 4
|
|
.SS \-\-cache\-chunk\-no\-memory
|
|
.PP
|
|
Disable the in\-memory cache for storing chunks during streaming.
|
|
.PP
|
|
By default, cache will keep file data during streaming in RAM as well to
|
|
provide it to readers as fast as possible.
|
|
.PP
|
|
This transient data is evicted as soon as it is read and the number of
|
|
chunks stored doesn\[aq]t exceed the number of workers.
|
|
However, depending on other settings like \[dq]cache\-chunk\-size\[dq]
|
|
and \[dq]cache\-workers\[dq] this footprint can increase if there are
|
|
parallel streams too (multiple files being read at the same time).
|
|
.PP
|
|
If the hardware permits it, use this feature to provide an overall
|
|
better performance during streaming but it can also be disabled if RAM
|
|
is not available on the local machine.
|
|
.IP \[bu] 2
|
|
Config: chunk_no_memory
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_CACHE_CHUNK_NO_MEMORY
|
|
.IP \[bu] 2
|
|
Type: bool
|
|
.IP \[bu] 2
|
|
Default: false
|
|
.SS \-\-cache\-rps
|
|
.PP
|
|
Limits the number of requests per second to the source FS (\-1 to
|
|
disable)
|
|
.PP
|
|
This setting places a hard limit on the number of requests per second
|
|
that cache will be doing to the cloud provider remote and try to respect
|
|
that value by setting waits between reads.
|
|
.PP
|
|
If you find that you\[aq]re getting banned or limited on the cloud
|
|
provider through cache and know that a smaller number of requests per
|
|
second will allow you to work with it then you can use this setting for
|
|
that.
|
|
.PP
|
|
A good balance of all the other settings should make this setting
|
|
useless but it is available to set for more special cases.
|
|
.PP
|
|
\f[B]NOTE\f[R]: This will limit the number of requests during streams
|
|
but other API calls to the cloud provider like directory listings will
|
|
still pass.
|
|
.IP \[bu] 2
|
|
Config: rps
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_CACHE_RPS
|
|
.IP \[bu] 2
|
|
Type: int
|
|
.IP \[bu] 2
|
|
Default: \-1
|
|
.SS \-\-cache\-writes
|
|
.PP
|
|
Cache file data on writes through the FS
|
|
.PP
|
|
If you need to read files immediately after you upload them through
|
|
cache you can enable this flag to have their data stored in the cache
|
|
store at the same time during upload.
|
|
.IP \[bu] 2
|
|
Config: writes
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_CACHE_WRITES
|
|
.IP \[bu] 2
|
|
Type: bool
|
|
.IP \[bu] 2
|
|
Default: false
|
|
.SS \-\-cache\-tmp\-upload\-path
|
|
.PP
|
|
Directory to keep temporary files until they are uploaded.
|
|
.PP
|
|
This is the path where cache will use as a temporary storage for new
|
|
files that need to be uploaded to the cloud provider.
|
|
.PP
|
|
Specifying a value will enable this feature.
|
|
Without it, it is completely disabled and files will be uploaded
|
|
directly to the cloud provider
|
|
.IP \[bu] 2
|
|
Config: tmp_upload_path
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_CACHE_TMP_UPLOAD_PATH
|
|
.IP \[bu] 2
|
|
Type: string
|
|
.IP \[bu] 2
|
|
Default: \[dq]\[dq]
|
|
.SS \-\-cache\-tmp\-wait\-time
|
|
.PP
|
|
How long should files be stored in local cache before being uploaded
|
|
.PP
|
|
This is the duration that a file must wait in the temporary location
|
|
\f[I]cache\-tmp\-upload\-path\f[R] before it is selected for upload.
|
|
.PP
|
|
Note that only one file is uploaded at a time and it can take longer to
|
|
start the upload if a queue formed for this purpose.
|
|
.IP \[bu] 2
|
|
Config: tmp_wait_time
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_CACHE_TMP_WAIT_TIME
|
|
.IP \[bu] 2
|
|
Type: Duration
|
|
.IP \[bu] 2
|
|
Default: 15s
|
|
.SS \-\-cache\-db\-wait\-time
|
|
.PP
|
|
How long to wait for the DB to be available \- 0 is unlimited
|
|
.PP
|
|
Only one process can have the DB open at any one time, so rclone waits
|
|
for this duration for the DB to become available before it gives an
|
|
error.
|
|
.PP
|
|
If you set it to 0 then it will wait forever.
|
|
.IP \[bu] 2
|
|
Config: db_wait_time
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_CACHE_DB_WAIT_TIME
|
|
.IP \[bu] 2
|
|
Type: Duration
|
|
.IP \[bu] 2
|
|
Default: 1s
|
|
.SS Backend commands
|
|
.PP
|
|
Here are the commands specific to the cache backend.
|
|
.PP
|
|
Run them with
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone backend COMMAND remote:
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
The help below will explain what arguments each command takes.
|
|
.PP
|
|
See the \[dq]rclone backend\[dq]
|
|
command (https://rclone.org/commands/rclone_backend/) for more info on
|
|
how to pass options and arguments.
|
|
.PP
|
|
These can be run on a running backend using the rc command
|
|
backend/command (https://rclone.org/rc/#backend/command).
|
|
.SS stats
|
|
.PP
|
|
Print stats on the cache backend in JSON format.
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone backend stats remote: [options] [<arguments>+]
|
|
\f[R]
|
|
.fi
|
|
.SS Chunker (BETA)
|
|
.PP
|
|
The \f[C]chunker\f[R] overlay transparently splits large files into
|
|
smaller chunks during upload to wrapped remote and transparently
|
|
assembles them back when the file is downloaded.
|
|
This allows to effectively overcome size limits imposed by storage
|
|
providers.
|
|
.PP
|
|
To use it, first set up the underlying remote following the
|
|
configuration instructions for that remote.
|
|
You can also use a local pathname instead of a remote.
|
|
.PP
|
|
First check your chosen remote is working \- we\[aq]ll call it
|
|
\f[C]remote:path\f[R] here.
|
|
Note that anything inside \f[C]remote:path\f[R] will be chunked and
|
|
anything outside won\[aq]t.
|
|
This means that if you are using a bucket based remote (e.g.
|
|
S3, B2, swift) then you should probably put the bucket in the remote
|
|
\f[C]s3:bucket\f[R].
|
|
.PP
|
|
Now configure \f[C]chunker\f[R] using \f[C]rclone config\f[R].
|
|
We will call this one \f[C]overlay\f[R] to separate it from the
|
|
\f[C]remote\f[R] itself.
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
No remotes found \- make a new one
|
|
n) New remote
|
|
s) Set configuration password
|
|
q) Quit config
|
|
n/s/q> n
|
|
name> overlay
|
|
Type of storage to configure.
|
|
Choose a number from below, or type in your own value
|
|
[snip]
|
|
XX / Transparently chunk/split large files
|
|
\[rs] \[dq]chunker\[dq]
|
|
[snip]
|
|
Storage> chunker
|
|
Remote to chunk/unchunk.
|
|
Normally should contain a \[aq]:\[aq] and a path, e.g. \[dq]myremote:path/to/dir\[dq],
|
|
\[dq]myremote:bucket\[dq] or maybe \[dq]myremote:\[dq] (not recommended).
|
|
Enter a string value. Press Enter for the default (\[dq]\[dq]).
|
|
remote> remote:path
|
|
Files larger than chunk size will be split in chunks.
|
|
Enter a size with suffix k,M,G,T. Press Enter for the default (\[dq]2G\[dq]).
|
|
chunk_size> 100M
|
|
Choose how chunker handles hash sums. All modes but \[dq]none\[dq] require metadata.
|
|
Enter a string value. Press Enter for the default (\[dq]md5\[dq]).
|
|
Choose a number from below, or type in your own value
|
|
1 / Pass any hash supported by wrapped remote for non\-chunked files, return nothing otherwise
|
|
\[rs] \[dq]none\[dq]
|
|
2 / MD5 for composite files
|
|
\[rs] \[dq]md5\[dq]
|
|
3 / SHA1 for composite files
|
|
\[rs] \[dq]sha1\[dq]
|
|
4 / MD5 for all files
|
|
\[rs] \[dq]md5all\[dq]
|
|
5 / SHA1 for all files
|
|
\[rs] \[dq]sha1all\[dq]
|
|
6 / Copying a file to chunker will request MD5 from the source falling back to SHA1 if unsupported
|
|
\[rs] \[dq]md5quick\[dq]
|
|
7 / Similar to \[dq]md5quick\[dq] but prefers SHA1 over MD5
|
|
\[rs] \[dq]sha1quick\[dq]
|
|
hash_type> md5
|
|
Edit advanced config? (y/n)
|
|
y) Yes
|
|
n) No
|
|
y/n> n
|
|
Remote config
|
|
\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-
|
|
[overlay]
|
|
type = chunker
|
|
remote = remote:bucket
|
|
chunk_size = 100M
|
|
hash_type = md5
|
|
\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-
|
|
y) Yes this is OK
|
|
e) Edit this remote
|
|
d) Delete this remote
|
|
y/e/d> y
|
|
\f[R]
|
|
.fi
|
|
.SS Specifying the remote
|
|
.PP
|
|
In normal use, make sure the remote has a \f[C]:\f[R] in.
|
|
If you specify the remote without a \f[C]:\f[R] then rclone will use a
|
|
local directory of that name.
|
|
So if you use a remote of \f[C]/path/to/secret/files\f[R] then rclone
|
|
will chunk stuff in that directory.
|
|
If you use a remote of \f[C]name\f[R] then rclone will put files in a
|
|
directory called \f[C]name\f[R] in the current directory.
|
|
.SS Chunking
|
|
.PP
|
|
When rclone starts a file upload, chunker checks the file size.
|
|
If it doesn\[aq]t exceed the configured chunk size, chunker will just
|
|
pass the file to the wrapped remote.
|
|
If a file is large, chunker will transparently cut data in pieces with
|
|
temporary names and stream them one by one, on the fly.
|
|
Each data chunk will contain the specified number of bytes, except for
|
|
the last one which may have less data.
|
|
If file size is unknown in advance (this is called a streaming upload),
|
|
chunker will internally create a temporary copy, record its size and
|
|
repeat the above process.
|
|
.PP
|
|
When upload completes, temporary chunk files are finally renamed.
|
|
This scheme guarantees that operations can be run in parallel and look
|
|
from outside as atomic.
|
|
A similar method with hidden temporary chunks is used for other
|
|
operations (copy/move/rename, etc.).
|
|
If an operation fails, hidden chunks are normally destroyed, and the
|
|
target composite file stays intact.
|
|
.PP
|
|
When a composite file download is requested, chunker transparently
|
|
assembles it by concatenating data chunks in order.
|
|
As the split is trivial one could even manually concatenate data chunks
|
|
together to obtain the original content.
|
|
.PP
|
|
When the \f[C]list\f[R] rclone command scans a directory on wrapped
|
|
remote, the potential chunk files are accounted for, grouped and
|
|
assembled into composite directory entries.
|
|
Any temporary chunks are hidden.
|
|
.PP
|
|
List and other commands can sometimes come across composite files with
|
|
missing or invalid chunks, e.g.
|
|
shadowed by like\-named directory or another file.
|
|
This usually means that wrapped file system has been directly tampered
|
|
with or damaged.
|
|
If chunker detects a missing chunk it will by default print warning,
|
|
skip the whole incomplete group of chunks but proceed with current
|
|
command.
|
|
You can set the \f[C]\-\-chunker\-fail\-hard\f[R] flag to have commands
|
|
abort with error message in such cases.
|
|
.SS Chunk names
|
|
.PP
|
|
The default chunk name format is \f[C]*.rclone_chunk.###\f[R], hence by
|
|
default chunk names are \f[C]BIG_FILE_NAME.rclone_chunk.001\f[R],
|
|
\f[C]BIG_FILE_NAME.rclone_chunk.002\f[R] etc.
|
|
You can configure another name format using the \f[C]name_format\f[R]
|
|
configuration file option.
|
|
The format uses asterisk \f[C]*\f[R] as a placeholder for the base file
|
|
name and one or more consecutive hash characters \f[C]#\f[R] as a
|
|
placeholder for sequential chunk number.
|
|
There must be one and only one asterisk.
|
|
The number of consecutive hash characters defines the minimum length of
|
|
a string representing a chunk number.
|
|
If decimal chunk number has less digits than the number of hashes, it is
|
|
left\-padded by zeros.
|
|
If the decimal string is longer, it is left intact.
|
|
By default numbering starts from 1 but there is another option that
|
|
allows user to start from 0, e.g.
|
|
for compatibility with legacy software.
|
|
.PP
|
|
For example, if name format is \f[C]big_*\-##.part\f[R] and original
|
|
file name is \f[C]data.txt\f[R] and numbering starts from 0, then the
|
|
first chunk will be named \f[C]big_data.txt\-00.part\f[R], the 99th
|
|
chunk will be \f[C]big_data.txt\-98.part\f[R] and the 302nd chunk will
|
|
become \f[C]big_data.txt\-301.part\f[R].
|
|
.PP
|
|
Note that \f[C]list\f[R] assembles composite directory entries only when
|
|
chunk names match the configured format and treats non\-conforming file
|
|
names as normal non\-chunked files.
|
|
.PP
|
|
When using \f[C]norename\f[R] transactions, chunk names will
|
|
additionally have a unique file version suffix.
|
|
For example, \f[C]BIG_FILE_NAME.rclone_chunk.001_bp562k\f[R].
|
|
.SS Metadata
|
|
.PP
|
|
Besides data chunks chunker will by default create metadata object for a
|
|
composite file.
|
|
The object is named after the original file.
|
|
Chunker allows user to disable metadata completely (the \f[C]none\f[R]
|
|
format).
|
|
Note that metadata is normally not created for files smaller than the
|
|
configured chunk size.
|
|
This may change in future rclone releases.
|
|
.SS Simple JSON metadata format
|
|
.PP
|
|
This is the default format.
|
|
It supports hash sums and chunk validation for composite files.
|
|
Meta objects carry the following fields:
|
|
.IP \[bu] 2
|
|
\f[C]ver\f[R] \- version of format, currently \f[C]1\f[R]
|
|
.IP \[bu] 2
|
|
\f[C]size\f[R] \- total size of composite file
|
|
.IP \[bu] 2
|
|
\f[C]nchunks\f[R] \- number of data chunks in file
|
|
.IP \[bu] 2
|
|
\f[C]md5\f[R] \- MD5 hashsum of composite file (if present)
|
|
.IP \[bu] 2
|
|
\f[C]sha1\f[R] \- SHA1 hashsum (if present)
|
|
.IP \[bu] 2
|
|
\f[C]txn\f[R] \- identifies current version of the file
|
|
.PP
|
|
There is no field for composite file name as it\[aq]s simply equal to
|
|
the name of meta object on the wrapped remote.
|
|
Please refer to respective sections for details on hashsums and modified
|
|
time handling.
|
|
.SS No metadata
|
|
.PP
|
|
You can disable meta objects by setting the meta format option to
|
|
\f[C]none\f[R].
|
|
In this mode chunker will scan directory for all files that follow
|
|
configured chunk name format, group them by detecting chunks with the
|
|
same base name and show group names as virtual composite files.
|
|
This method is more prone to missing chunk errors (especially missing
|
|
last chunk) than format with metadata enabled.
|
|
.SS Hashsums
|
|
.PP
|
|
Chunker supports hashsums only when a compatible metadata is present.
|
|
Hence, if you choose metadata format of \f[C]none\f[R], chunker will
|
|
report hashsum as \f[C]UNSUPPORTED\f[R].
|
|
.PP
|
|
Please note that by default metadata is stored only for composite files.
|
|
If a file is smaller than configured chunk size, chunker will
|
|
transparently redirect hash requests to wrapped remote, so support
|
|
depends on that.
|
|
You will see the empty string as a hashsum of requested type for small
|
|
files if the wrapped remote doesn\[aq]t support it.
|
|
.PP
|
|
Many storage backends support MD5 and SHA1 hash types, so does chunker.
|
|
With chunker you can choose one or another but not both.
|
|
MD5 is set by default as the most supported type.
|
|
Since chunker keeps hashes for composite files and falls back to the
|
|
wrapped remote hash for non\-chunked ones, we advise you to choose the
|
|
same hash type as supported by wrapped remote so that your file listings
|
|
look coherent.
|
|
.PP
|
|
If your storage backend does not support MD5 or SHA1 but you need
|
|
consistent file hashing, configure chunker with \f[C]md5all\f[R] or
|
|
\f[C]sha1all\f[R].
|
|
These two modes guarantee given hash for all files.
|
|
If wrapped remote doesn\[aq]t support it, chunker will then add metadata
|
|
to all files, even small.
|
|
However, this can double the amount of small files in storage and incur
|
|
additional service charges.
|
|
You can even use chunker to force md5/sha1 support in any other remote
|
|
at expense of sidecar meta objects by setting e.g.
|
|
\f[C]chunk_type=sha1all\f[R] to force hashsums and
|
|
\f[C]chunk_size=1P\f[R] to effectively disable chunking.
|
|
.PP
|
|
Normally, when a file is copied to chunker controlled remote, chunker
|
|
will ask the file source for compatible file hash and revert to
|
|
on\-the\-fly calculation if none is found.
|
|
This involves some CPU overhead but provides a guarantee that given
|
|
hashsum is available.
|
|
Also, chunker will reject a server\-side copy or move operation if
|
|
source and destination hashsum types are different resulting in the
|
|
extra network bandwidth, too.
|
|
In some rare cases this may be undesired, so chunker provides two
|
|
optional choices: \f[C]sha1quick\f[R] and \f[C]md5quick\f[R].
|
|
If the source does not support primary hash type and the quick mode is
|
|
enabled, chunker will try to fall back to the secondary type.
|
|
This will save CPU and bandwidth but can result in empty hashsums at
|
|
destination.
|
|
Beware of consequences: the \f[C]sync\f[R] command will revert
|
|
(sometimes silently) to time/size comparison if compatible hashsums
|
|
between source and target are not found.
|
|
.SS Modified time
|
|
.PP
|
|
Chunker stores modification times using the wrapped remote so support
|
|
depends on that.
|
|
For a small non\-chunked file the chunker overlay simply manipulates
|
|
modification time of the wrapped remote file.
|
|
For a composite file with metadata chunker will get and set modification
|
|
time of the metadata object on the wrapped remote.
|
|
If file is chunked but metadata format is \f[C]none\f[R] then chunker
|
|
will use modification time of the first data chunk.
|
|
.SS Migrations
|
|
.PP
|
|
The idiomatic way to migrate to a different chunk size, hash type,
|
|
transaction style or chunk naming scheme is to:
|
|
.IP \[bu] 2
|
|
Collect all your chunked files under a directory and have your chunker
|
|
remote point to it.
|
|
.IP \[bu] 2
|
|
Create another directory (most probably on the same cloud storage) and
|
|
configure a new remote with desired metadata format, hash type, chunk
|
|
naming etc.
|
|
.IP \[bu] 2
|
|
Now run \f[C]rclone sync \-i oldchunks: newchunks:\f[R] and all your
|
|
data will be transparently converted in transfer.
|
|
This may take some time, yet chunker will try server\-side copy if
|
|
possible.
|
|
.IP \[bu] 2
|
|
After checking data integrity you may remove configuration section of
|
|
the old remote.
|
|
.PP
|
|
If rclone gets killed during a long operation on a big composite file,
|
|
hidden temporary chunks may stay in the directory.
|
|
They will not be shown by the \f[C]list\f[R] command but will eat up
|
|
your account quota.
|
|
Please note that the \f[C]deletefile\f[R] command deletes only active
|
|
chunks of a file.
|
|
As a workaround, you can use remote of the wrapped file system to see
|
|
them.
|
|
An easy way to get rid of hidden garbage is to copy littered directory
|
|
somewhere using the chunker remote and purge the original directory.
|
|
The \f[C]copy\f[R] command will copy only active chunks while the
|
|
\f[C]purge\f[R] will remove everything including garbage.
|
|
.SS Caveats and Limitations
|
|
.PP
|
|
Chunker requires wrapped remote to support server\-side \f[C]move\f[R]
|
|
(or \f[C]copy\f[R] + \f[C]delete\f[R]) operations, otherwise it will
|
|
explicitly refuse to start.
|
|
This is because it internally renames temporary chunk files to their
|
|
final names when an operation completes successfully.
|
|
.PP
|
|
Chunker encodes chunk number in file name, so with default
|
|
\f[C]name_format\f[R] setting it adds 17 characters.
|
|
Also chunker adds 7 characters of temporary suffix during operations.
|
|
Many file systems limit base file name without path by 255 characters.
|
|
Using rclone\[aq]s crypt remote as a base file system limits file name
|
|
by 143 characters.
|
|
Thus, maximum name length is 231 for most files and 119 for
|
|
chunker\-over\-crypt.
|
|
A user in need can change name format to e.g.
|
|
\f[C]*.rcc##\f[R] and save 10 characters (provided at most 99 chunks per
|
|
file).
|
|
.PP
|
|
Note that a move implemented using the copy\-and\-delete method may
|
|
incur double charging with some cloud storage providers.
|
|
.PP
|
|
Chunker will not automatically rename existing chunks when you run
|
|
\f[C]rclone config\f[R] on a live remote and change the chunk name
|
|
format.
|
|
Beware that in result of this some files which have been treated as
|
|
chunks before the change can pop up in directory listings as normal
|
|
files and vice versa.
|
|
The same warning holds for the chunk size.
|
|
If you desperately need to change critical chunking settings, you should
|
|
run data migration as described above.
|
|
.PP
|
|
If wrapped remote is case insensitive, the chunker overlay will inherit
|
|
that property (so you can\[aq]t have a file called \[dq]Hello.doc\[dq]
|
|
and \[dq]hello.doc\[dq] in the same directory).
|
|
.PP
|
|
Chunker included in rclone releases up to \f[C]v1.54\f[R] can sometimes
|
|
fail to detect metadata produced by recent versions of rclone.
|
|
We recommend users to keep rclone up\-to\-date to avoid data corruption.
|
|
.PP
|
|
Changing \f[C]transactions\f[R] is dangerous and requires explicit
|
|
migration.
|
|
.SS Standard Options
|
|
.PP
|
|
Here are the standard options specific to chunker (Transparently
|
|
chunk/split large files).
|
|
.SS \-\-chunker\-remote
|
|
.PP
|
|
Remote to chunk/unchunk.
|
|
Normally should contain a \[aq]:\[aq] and a path, e.g.
|
|
\[dq]myremote:path/to/dir\[dq], \[dq]myremote:bucket\[dq] or maybe
|
|
\[dq]myremote:\[dq] (not recommended).
|
|
.IP \[bu] 2
|
|
Config: remote
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_CHUNKER_REMOTE
|
|
.IP \[bu] 2
|
|
Type: string
|
|
.IP \[bu] 2
|
|
Default: \[dq]\[dq]
|
|
.SS \-\-chunker\-chunk\-size
|
|
.PP
|
|
Files larger than chunk size will be split in chunks.
|
|
.IP \[bu] 2
|
|
Config: chunk_size
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_CHUNKER_CHUNK_SIZE
|
|
.IP \[bu] 2
|
|
Type: SizeSuffix
|
|
.IP \[bu] 2
|
|
Default: 2G
|
|
.SS \-\-chunker\-hash\-type
|
|
.PP
|
|
Choose how chunker handles hash sums.
|
|
All modes but \[dq]none\[dq] require metadata.
|
|
.IP \[bu] 2
|
|
Config: hash_type
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_CHUNKER_HASH_TYPE
|
|
.IP \[bu] 2
|
|
Type: string
|
|
.IP \[bu] 2
|
|
Default: \[dq]md5\[dq]
|
|
.IP \[bu] 2
|
|
Examples:
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
\[dq]none\[dq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Pass any hash supported by wrapped remote for non\-chunked files, return
|
|
nothing otherwise
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[dq]md5\[dq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
MD5 for composite files
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[dq]sha1\[dq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
SHA1 for composite files
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[dq]md5all\[dq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
MD5 for all files
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[dq]sha1all\[dq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
SHA1 for all files
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[dq]md5quick\[dq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Copying a file to chunker will request MD5 from the source falling back
|
|
to SHA1 if unsupported
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[dq]sha1quick\[dq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Similar to \[dq]md5quick\[dq] but prefers SHA1 over MD5
|
|
.RE
|
|
.RE
|
|
.SS Advanced Options
|
|
.PP
|
|
Here are the advanced options specific to chunker (Transparently
|
|
chunk/split large files).
|
|
.SS \-\-chunker\-name\-format
|
|
.PP
|
|
String format of chunk file names.
|
|
The two placeholders are: base file name (*) and chunk number (#...).
|
|
There must be one and only one asterisk and one or more consecutive hash
|
|
characters.
|
|
If chunk number has less digits than the number of hashes, it is
|
|
left\-padded by zeros.
|
|
If there are more digits in the number, they are left as is.
|
|
Possible chunk files are ignored if their name does not match given
|
|
format.
|
|
.IP \[bu] 2
|
|
Config: name_format
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_CHUNKER_NAME_FORMAT
|
|
.IP \[bu] 2
|
|
Type: string
|
|
.IP \[bu] 2
|
|
Default: \[dq]*.rclone_chunk.###\[dq]
|
|
.SS \-\-chunker\-start\-from
|
|
.PP
|
|
Minimum valid chunk number.
|
|
Usually 0 or 1.
|
|
By default chunk numbers start from 1.
|
|
.IP \[bu] 2
|
|
Config: start_from
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_CHUNKER_START_FROM
|
|
.IP \[bu] 2
|
|
Type: int
|
|
.IP \[bu] 2
|
|
Default: 1
|
|
.SS \-\-chunker\-meta\-format
|
|
.PP
|
|
Format of the metadata object or \[dq]none\[dq].
|
|
By default \[dq]simplejson\[dq].
|
|
Metadata is a small JSON file named after the composite file.
|
|
.IP \[bu] 2
|
|
Config: meta_format
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_CHUNKER_META_FORMAT
|
|
.IP \[bu] 2
|
|
Type: string
|
|
.IP \[bu] 2
|
|
Default: \[dq]simplejson\[dq]
|
|
.IP \[bu] 2
|
|
Examples:
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
\[dq]none\[dq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Do not use metadata files at all.
|
|
Requires hash type \[dq]none\[dq].
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[dq]simplejson\[dq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Simple JSON supports hash sums and chunk validation.
|
|
.IP \[bu] 2
|
|
It has the following fields: ver, size, nchunks, md5, sha1.
|
|
.RE
|
|
.RE
|
|
.SS \-\-chunker\-fail\-hard
|
|
.PP
|
|
Choose how chunker should handle files with missing or invalid chunks.
|
|
.IP \[bu] 2
|
|
Config: fail_hard
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_CHUNKER_FAIL_HARD
|
|
.IP \[bu] 2
|
|
Type: bool
|
|
.IP \[bu] 2
|
|
Default: false
|
|
.IP \[bu] 2
|
|
Examples:
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
\[dq]true\[dq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Report errors and abort current command.
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[dq]false\[dq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Warn user, skip incomplete file and proceed.
|
|
.RE
|
|
.RE
|
|
.SS \-\-chunker\-transactions
|
|
.PP
|
|
Choose how chunker should handle temporary files during transactions.
|
|
.IP \[bu] 2
|
|
Config: transactions
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_CHUNKER_TRANSACTIONS
|
|
.IP \[bu] 2
|
|
Type: string
|
|
.IP \[bu] 2
|
|
Default: \[dq]rename\[dq]
|
|
.IP \[bu] 2
|
|
Examples:
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
\[dq]rename\[dq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Rename temporary files after a successful transaction.
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[dq]norename\[dq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Leave temporary file names and write transaction ID to metadata file.
|
|
.IP \[bu] 2
|
|
Metadata is required for no rename transactions (meta format cannot be
|
|
\[dq]none\[dq]).
|
|
.IP \[bu] 2
|
|
If you are using norename transactions you should be careful not to
|
|
downgrade Rclone
|
|
.IP \[bu] 2
|
|
as older versions of Rclone don\[aq]t support this transaction style and
|
|
will misinterpret
|
|
.IP \[bu] 2
|
|
files manipulated by norename transactions.
|
|
.IP \[bu] 2
|
|
This method is EXPERIMENTAL, don\[aq]t use on production systems.
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[dq]auto\[dq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Rename or norename will be used depending on capabilities of the
|
|
backend.
|
|
.IP \[bu] 2
|
|
If meta format is set to \[dq]none\[dq], rename transactions will always
|
|
be used.
|
|
.IP \[bu] 2
|
|
This method is EXPERIMENTAL, don\[aq]t use on production systems.
|
|
.RE
|
|
.RE
|
|
.SS Citrix ShareFile
|
|
.PP
|
|
Citrix ShareFile (https://sharefile.com) is a secure file sharing and
|
|
transfer service aimed as business.
|
|
.PP
|
|
The initial setup for Citrix ShareFile involves getting a token from
|
|
Citrix ShareFile which you can in your browser.
|
|
\f[C]rclone config\f[R] walks you through it.
|
|
.PP
|
|
Here is an example of how to make a remote called \f[C]remote\f[R].
|
|
First run:
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone config
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
This will guide you through an interactive setup process:
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
No remotes found \- make a new one
|
|
n) New remote
|
|
s) Set configuration password
|
|
q) Quit config
|
|
n/s/q> n
|
|
name> remote
|
|
Type of storage to configure.
|
|
Enter a string value. Press Enter for the default (\[dq]\[dq]).
|
|
Choose a number from below, or type in your own value
|
|
XX / Citrix Sharefile
|
|
\[rs] \[dq]sharefile\[dq]
|
|
Storage> sharefile
|
|
** See help for sharefile backend at: https://rclone.org/sharefile/ **
|
|
|
|
ID of the root folder
|
|
|
|
Leave blank to access \[dq]Personal Folders\[dq]. You can use one of the
|
|
standard values here or any folder ID (long hex number ID).
|
|
Enter a string value. Press Enter for the default (\[dq]\[dq]).
|
|
Choose a number from below, or type in your own value
|
|
1 / Access the Personal Folders. (Default)
|
|
\[rs] \[dq]\[dq]
|
|
2 / Access the Favorites folder.
|
|
\[rs] \[dq]favorites\[dq]
|
|
3 / Access all the shared folders.
|
|
\[rs] \[dq]allshared\[dq]
|
|
4 / Access all the individual connectors.
|
|
\[rs] \[dq]connectors\[dq]
|
|
5 / Access the home, favorites, and shared folders as well as the connectors.
|
|
\[rs] \[dq]top\[dq]
|
|
root_folder_id>
|
|
Edit advanced config? (y/n)
|
|
y) Yes
|
|
n) No
|
|
y/n> n
|
|
Remote config
|
|
Use auto config?
|
|
* Say Y if not sure
|
|
* Say N if you are working on a remote or headless machine
|
|
y) Yes
|
|
n) No
|
|
y/n> y
|
|
If your browser doesn\[aq]t open automatically go to the following link: http://127.0.0.1:53682/auth?state=XXX
|
|
Log in and authorize rclone for access
|
|
Waiting for code...
|
|
Got code
|
|
\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-
|
|
[remote]
|
|
type = sharefile
|
|
endpoint = https://XXX.sharefile.com
|
|
token = {\[dq]access_token\[dq]:\[dq]XXX\[dq],\[dq]token_type\[dq]:\[dq]bearer\[dq],\[dq]refresh_token\[dq]:\[dq]XXX\[dq],\[dq]expiry\[dq]:\[dq]2019\-09\-30T19:41:45.878561877+01:00\[dq]}
|
|
\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-
|
|
y) Yes this is OK
|
|
e) Edit this remote
|
|
d) Delete this remote
|
|
y/e/d> y
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
See the remote setup docs (https://rclone.org/remote_setup/) for how to
|
|
set it up on a machine with no Internet browser available.
|
|
.PP
|
|
Note that rclone runs a webserver on your local machine to collect the
|
|
token as returned from Citrix ShareFile.
|
|
This only runs from the moment it opens your browser to the moment you
|
|
get back the verification code.
|
|
This is on \f[C]http://127.0.0.1:53682/\f[R] and this it may require you
|
|
to unblock it temporarily if you are running a host firewall.
|
|
.PP
|
|
Once configured you can then use \f[C]rclone\f[R] like this,
|
|
.PP
|
|
List directories in top level of your ShareFile
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone lsd remote:
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
List all the files in your ShareFile
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone ls remote:
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
To copy a local directory to an ShareFile directory called backup
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone copy /home/source remote:backup
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
Paths may be as deep as required, e.g.
|
|
\f[C]remote:directory/subdirectory\f[R].
|
|
.SS Modified time and hashes
|
|
.PP
|
|
ShareFile allows modification times to be set on objects accurate to 1
|
|
second.
|
|
These will be used to detect whether objects need syncing or not.
|
|
.PP
|
|
ShareFile supports MD5 type hashes, so you can use the
|
|
\f[C]\-\-checksum\f[R] flag.
|
|
.SS Transfers
|
|
.PP
|
|
For files above 128MB rclone will use a chunked transfer.
|
|
Rclone will upload up to \f[C]\-\-transfers\f[R] chunks at the same time
|
|
(shared among all the multipart uploads).
|
|
Chunks are buffered in memory and are normally 64MB so increasing
|
|
\f[C]\-\-transfers\f[R] will increase memory use.
|
|
.SS Limitations
|
|
.PP
|
|
Note that ShareFile is case insensitive so you can\[aq]t have a file
|
|
called \[dq]Hello.doc\[dq] and one called \[dq]hello.doc\[dq].
|
|
.PP
|
|
ShareFile only supports filenames up to 256 characters in length.
|
|
.SS Restricted filename characters
|
|
.PP
|
|
In addition to the default restricted characters
|
|
set (https://rclone.org/overview/#restricted-characters) the following
|
|
characters are also replaced:
|
|
.PP
|
|
.TS
|
|
tab(@);
|
|
l c c.
|
|
T{
|
|
Character
|
|
T}@T{
|
|
Value
|
|
T}@T{
|
|
Replacement
|
|
T}
|
|
_
|
|
T{
|
|
\[rs]
|
|
T}@T{
|
|
0x5C
|
|
T}@T{
|
|
\[uFF3C]
|
|
T}
|
|
T{
|
|
*
|
|
T}@T{
|
|
0x2A
|
|
T}@T{
|
|
\[uFF0A]
|
|
T}
|
|
T{
|
|
<
|
|
T}@T{
|
|
0x3C
|
|
T}@T{
|
|
\[uFF1C]
|
|
T}
|
|
T{
|
|
>
|
|
T}@T{
|
|
0x3E
|
|
T}@T{
|
|
\[uFF1E]
|
|
T}
|
|
T{
|
|
?
|
|
T}@T{
|
|
0x3F
|
|
T}@T{
|
|
\[uFF1F]
|
|
T}
|
|
T{
|
|
:
|
|
T}@T{
|
|
0x3A
|
|
T}@T{
|
|
\[uFF1A]
|
|
T}
|
|
T{
|
|
|
|
|
T}@T{
|
|
0x7C
|
|
T}@T{
|
|
\[uFF5C]
|
|
T}
|
|
T{
|
|
\[dq]
|
|
T}@T{
|
|
0x22
|
|
T}@T{
|
|
\[uFF02]
|
|
T}
|
|
.TE
|
|
.PP
|
|
File names can also not start or end with the following characters.
|
|
These only get replaced if they are the first or last character in the
|
|
name:
|
|
.PP
|
|
.TS
|
|
tab(@);
|
|
l c c.
|
|
T{
|
|
Character
|
|
T}@T{
|
|
Value
|
|
T}@T{
|
|
Replacement
|
|
T}
|
|
_
|
|
T{
|
|
SP
|
|
T}@T{
|
|
0x20
|
|
T}@T{
|
|
\[u2420]
|
|
T}
|
|
T{
|
|
\&.
|
|
T}@T{
|
|
0x2E
|
|
T}@T{
|
|
\[uFF0E]
|
|
T}
|
|
.TE
|
|
.PP
|
|
Invalid UTF\-8 bytes will also be
|
|
replaced (https://rclone.org/overview/#invalid-utf8), as they can\[aq]t
|
|
be used in JSON strings.
|
|
.SS Standard Options
|
|
.PP
|
|
Here are the standard options specific to sharefile (Citrix Sharefile).
|
|
.SS \-\-sharefile\-root\-folder\-id
|
|
.PP
|
|
ID of the root folder
|
|
.PP
|
|
Leave blank to access \[dq]Personal Folders\[dq].
|
|
You can use one of the standard values here or any folder ID (long hex
|
|
number ID).
|
|
.IP \[bu] 2
|
|
Config: root_folder_id
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_SHAREFILE_ROOT_FOLDER_ID
|
|
.IP \[bu] 2
|
|
Type: string
|
|
.IP \[bu] 2
|
|
Default: \[dq]\[dq]
|
|
.IP \[bu] 2
|
|
Examples:
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
\[dq]\[dq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Access the Personal Folders.
|
|
(Default)
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[dq]favorites\[dq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Access the Favorites folder.
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[dq]allshared\[dq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Access all the shared folders.
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[dq]connectors\[dq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Access all the individual connectors.
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[dq]top\[dq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Access the home, favorites, and shared folders as well as the
|
|
connectors.
|
|
.RE
|
|
.RE
|
|
.SS Advanced Options
|
|
.PP
|
|
Here are the advanced options specific to sharefile (Citrix Sharefile).
|
|
.SS \-\-sharefile\-upload\-cutoff
|
|
.PP
|
|
Cutoff for switching to multipart upload.
|
|
.IP \[bu] 2
|
|
Config: upload_cutoff
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_SHAREFILE_UPLOAD_CUTOFF
|
|
.IP \[bu] 2
|
|
Type: SizeSuffix
|
|
.IP \[bu] 2
|
|
Default: 128M
|
|
.SS \-\-sharefile\-chunk\-size
|
|
.PP
|
|
Upload chunk size.
|
|
Must a power of 2 >= 256k.
|
|
.PP
|
|
Making this larger will improve performance, but note that each chunk is
|
|
buffered in memory one per transfer.
|
|
.PP
|
|
Reducing this will reduce memory usage but decrease performance.
|
|
.IP \[bu] 2
|
|
Config: chunk_size
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_SHAREFILE_CHUNK_SIZE
|
|
.IP \[bu] 2
|
|
Type: SizeSuffix
|
|
.IP \[bu] 2
|
|
Default: 64M
|
|
.SS \-\-sharefile\-endpoint
|
|
.PP
|
|
Endpoint for API calls.
|
|
.PP
|
|
This is usually auto discovered as part of the oauth process, but can be
|
|
set manually to something like: https://XXX.sharefile.com
|
|
.IP \[bu] 2
|
|
Config: endpoint
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_SHAREFILE_ENDPOINT
|
|
.IP \[bu] 2
|
|
Type: string
|
|
.IP \[bu] 2
|
|
Default: \[dq]\[dq]
|
|
.SS \-\-sharefile\-encoding
|
|
.PP
|
|
This sets the encoding for the backend.
|
|
.PP
|
|
See: the encoding section in the
|
|
overview (https://rclone.org/overview/#encoding) for more info.
|
|
.IP \[bu] 2
|
|
Config: encoding
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_SHAREFILE_ENCODING
|
|
.IP \[bu] 2
|
|
Type: MultiEncoder
|
|
.IP \[bu] 2
|
|
Default:
|
|
Slash,LtGt,DoubleQuote,Colon,Question,Asterisk,Pipe,BackSlash,Ctl,LeftSpace,LeftPeriod,RightSpace,RightPeriod,InvalidUtf8,Dot
|
|
.SS Limitations
|
|
.PP
|
|
\f[C]rclone about\f[R] is not supported by the Citrix ShareFile backend.
|
|
Backends without this capability cannot determine free space for an
|
|
rclone mount or use policy \f[C]mfs\f[R] (most free space) as a member
|
|
of an rclone union remote.
|
|
.PP
|
|
See List of backends that do not support rclone
|
|
about (https://rclone.org/overview/#optional-features) See rclone
|
|
about (https://rclone.org/commands/rclone_about/)
|
|
.SS Crypt
|
|
.PP
|
|
Rclone \f[C]crypt\f[R] remotes encrypt and decrypt other remotes.
|
|
.PP
|
|
A remote of type \f[C]crypt\f[R] does not access a storage
|
|
system (https://rclone.org/overview/) directly, but instead wraps
|
|
another remote, which in turn accesses the storage system.
|
|
This is similar to how alias (https://rclone.org/alias/),
|
|
union (https://rclone.org/union/), chunker (https://rclone.org/chunker/)
|
|
and a few others work.
|
|
It makes the usage very flexible, as you can add a layer, in this case
|
|
an encryption layer, on top of any other backend, even in multiple
|
|
layers.
|
|
Rclone\[aq]s functionality can be used as with any other remote, for
|
|
example you can mount (https://rclone.org/commands/rclone_mount/) a
|
|
crypt remote.
|
|
.PP
|
|
Accessing a storage system through a crypt remote realizes client\-side
|
|
encryption, which makes it safe to keep your data in a location you do
|
|
not trust will not get compromised.
|
|
When working against the \f[C]crypt\f[R] remote, rclone will
|
|
automatically encrypt (before uploading) and decrypt (after downloading)
|
|
on your local system as needed on the fly, leaving the data encrypted at
|
|
rest in the wrapped remote.
|
|
If you access the storage system using an application other than rclone,
|
|
or access the wrapped remote directly using rclone, there will not be
|
|
any encryption/decryption: Downloading existing content will just give
|
|
you the encrypted (scrambled) format, and anything you upload will
|
|
\f[I]not\f[R] become encrypted.
|
|
.PP
|
|
The encryption is a secret\-key encryption (also called symmetric key
|
|
encryption) algorithm, where a password (or pass phrase) is used to
|
|
generate real encryption key.
|
|
The password can be supplied by user, or you may chose to let rclone
|
|
generate one.
|
|
It will be stored in the configuration file, in a lightly obscured form.
|
|
If you are in an environment where you are not able to keep your
|
|
configuration secured, you should add configuration
|
|
encryption (https://rclone.org/docs/#configuration-encryption) as
|
|
protection.
|
|
As long as you have this configuration file, you will be able to decrypt
|
|
your data.
|
|
Without the configuration file, as long as you remember the password (or
|
|
keep it in a safe place), you can re\-create the configuration and gain
|
|
access to the existing data.
|
|
You may also configure a corresponding remote in a different
|
|
installation to access the same data.
|
|
See below for guidance to changing password.
|
|
.PP
|
|
Encryption uses cryptographic
|
|
salt (https://en.wikipedia.org/wiki/Salt_(cryptography)), to permute the
|
|
encryption key so that the same string may be encrypted in different
|
|
ways.
|
|
When configuring the crypt remote it is optional to enter a salt, or to
|
|
let rclone generate a unique salt.
|
|
If omitted, rclone uses a built\-in unique string.
|
|
Normally in cryptography, the salt is stored together with the encrypted
|
|
content, and do not have to be memorized by the user.
|
|
This is not the case in rclone, because rclone does not store any
|
|
additional information on the remotes.
|
|
Use of custom salt is effectively a second password that must be
|
|
memorized.
|
|
.PP
|
|
File content encryption is performed using NaCl
|
|
SecretBox (https://godoc.org/golang.org/x/crypto/nacl/secretbox), based
|
|
on XSalsa20 cipher and Poly1305 for integrity.
|
|
Names (file\- and directory names) are also encrypted by default, but
|
|
this has some implications and is therefore possible to turned off.
|
|
.SS Configuration
|
|
.PP
|
|
Here is an example of how to make a remote called \f[C]secret\f[R].
|
|
.PP
|
|
To use \f[C]crypt\f[R], first set up the underlying remote.
|
|
Follow the \f[C]rclone config\f[R] instructions for the specific
|
|
backend.
|
|
.PP
|
|
Before configuring the crypt remote, check the underlying remote is
|
|
working.
|
|
In this example the underlying remote is called \f[C]remote\f[R].
|
|
We will configure a path \f[C]path\f[R] within this remote to contain
|
|
the encrypted content.
|
|
Anything inside \f[C]remote:path\f[R] will be encrypted and anything
|
|
outside will not.
|
|
.PP
|
|
Configure \f[C]crypt\f[R] using \f[C]rclone config\f[R].
|
|
In this example the \f[C]crypt\f[R] remote is called \f[C]secret\f[R],
|
|
to differentiate it from the underlying \f[C]remote\f[R].
|
|
.PP
|
|
When you are done you can use the crypt remote named \f[C]secret\f[R]
|
|
just as you would with any other remote, e.g.
|
|
\f[C]rclone copy D:\[rs]docs secret:\[rs]docs\f[R], and rclone will
|
|
encrypt and decrypt as needed on the fly.
|
|
If you access the wrapped remote \f[C]remote:path\f[R] directly you will
|
|
bypass the encryption, and anything you read will be in encrypted form,
|
|
and anything you write will be unencrypted.
|
|
To avoid issues it is best to configure a dedicated path for encrypted
|
|
content, and access it exclusively through a crypt remote.
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
No remotes found \- make a new one
|
|
n) New remote
|
|
s) Set configuration password
|
|
q) Quit config
|
|
n/s/q> n
|
|
name> secret
|
|
Type of storage to configure.
|
|
Enter a string value. Press Enter for the default (\[dq]\[dq]).
|
|
Choose a number from below, or type in your own value
|
|
[snip]
|
|
XX / Encrypt/Decrypt a remote
|
|
\[rs] \[dq]crypt\[dq]
|
|
[snip]
|
|
Storage> crypt
|
|
** See help for crypt backend at: https://rclone.org/crypt/ **
|
|
|
|
Remote to encrypt/decrypt.
|
|
Normally should contain a \[aq]:\[aq] and a path, eg \[dq]myremote:path/to/dir\[dq],
|
|
\[dq]myremote:bucket\[dq] or maybe \[dq]myremote:\[dq] (not recommended).
|
|
Enter a string value. Press Enter for the default (\[dq]\[dq]).
|
|
remote> remote:path
|
|
How to encrypt the filenames.
|
|
Enter a string value. Press Enter for the default (\[dq]standard\[dq]).
|
|
Choose a number from below, or type in your own value
|
|
1 / Encrypt the filenames see the docs for the details.
|
|
\[rs] \[dq]standard\[dq]
|
|
2 / Very simple filename obfuscation.
|
|
\[rs] \[dq]obfuscate\[dq]
|
|
3 / Don\[aq]t encrypt the file names. Adds a \[dq].bin\[dq] extension only.
|
|
\[rs] \[dq]off\[dq]
|
|
filename_encryption>
|
|
Option to either encrypt directory names or leave them intact.
|
|
|
|
NB If filename_encryption is \[dq]off\[dq] then this option will do nothing.
|
|
Enter a boolean value (true or false). Press Enter for the default (\[dq]true\[dq]).
|
|
Choose a number from below, or type in your own value
|
|
1 / Encrypt directory names.
|
|
\[rs] \[dq]true\[dq]
|
|
2 / Don\[aq]t encrypt directory names, leave them intact.
|
|
\[rs] \[dq]false\[dq]
|
|
directory_name_encryption>
|
|
Password or pass phrase for encryption.
|
|
y) Yes type in my own password
|
|
g) Generate random password
|
|
y/g> y
|
|
Enter the password:
|
|
password:
|
|
Confirm the password:
|
|
password:
|
|
Password or pass phrase for salt. Optional but recommended.
|
|
Should be different to the previous password.
|
|
y) Yes type in my own password
|
|
g) Generate random password
|
|
n) No leave this optional password blank (default)
|
|
y/g/n> g
|
|
Password strength in bits.
|
|
64 is just about memorable
|
|
128 is secure
|
|
1024 is the maximum
|
|
Bits> 128
|
|
Your password is: JAsJvRcgR\-_veXNfy_sGmQ
|
|
Use this password? Please note that an obscured version of this
|
|
password (and not the password itself) will be stored under your
|
|
configuration file, so keep this generated password in a safe place.
|
|
y) Yes (default)
|
|
n) No
|
|
y/n>
|
|
Edit advanced config? (y/n)
|
|
y) Yes
|
|
n) No (default)
|
|
y/n>
|
|
Remote config
|
|
\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-
|
|
[secret]
|
|
type = crypt
|
|
remote = remote:path
|
|
password = *** ENCRYPTED ***
|
|
password2 = *** ENCRYPTED ***
|
|
\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-
|
|
y) Yes this is OK (default)
|
|
e) Edit this remote
|
|
d) Delete this remote
|
|
y/e/d>
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
\f[B]Important\f[R] The crypt password stored in \f[C]rclone.conf\f[R]
|
|
is lightly obscured.
|
|
That only protects it from cursory inspection.
|
|
It is not secure unless configuration
|
|
encryption (https://rclone.org/docs/#configuration-encryption) of
|
|
\f[C]rclone.conf\f[R] is specified.
|
|
.PP
|
|
A long passphrase is recommended, or \f[C]rclone config\f[R] can
|
|
generate a random one.
|
|
.PP
|
|
The obscured password is created using AES\-CTR with a static key.
|
|
The salt is stored verbatim at the beginning of the obscured password.
|
|
This static key is shared between all versions of rclone.
|
|
.PP
|
|
If you reconfigure rclone with the same passwords/passphrases elsewhere
|
|
it will be compatible, but the obscured version will be different due to
|
|
the different salt.
|
|
.PP
|
|
Rclone does not encrypt
|
|
.IP \[bu] 2
|
|
file length \- this can be calculated within 16 bytes
|
|
.IP \[bu] 2
|
|
modification time \- used for syncing
|
|
.SS Specifying the remote
|
|
.PP
|
|
When configuring the remote to encrypt/decrypt, you may specify any
|
|
string that rclone accepts as a source/destination of other commands.
|
|
.PP
|
|
The primary use case is to specify the path into an already configured
|
|
remote (e.g.
|
|
\f[C]remote:path/to/dir\f[R] or \f[C]remote:bucket\f[R]), such that data
|
|
in a remote untrusted location can be stored encrypted.
|
|
.PP
|
|
You may also specify a local filesystem path, such as
|
|
\f[C]/path/to/dir\f[R] on Linux, \f[C]C:\[rs]path\[rs]to\[rs]dir\f[R] on
|
|
Windows.
|
|
By creating a crypt remote pointing to such a local filesystem path, you
|
|
can use rclone as a utility for pure local file encryption, for example
|
|
to keep encrypted files on a removable USB drive.
|
|
.PP
|
|
\f[B]Note\f[R]: A string which do not contain a \f[C]:\f[R] will by
|
|
rclone be treated as a relative path in the local filesystem.
|
|
For example, if you enter the name \f[C]remote\f[R] without the trailing
|
|
\f[C]:\f[R], it will be treated as a subdirectory of the current
|
|
directory with name \[dq]remote\[dq].
|
|
.PP
|
|
If a path \f[C]remote:path/to/dir\f[R] is specified, rclone stores
|
|
encrypted files in \f[C]path/to/dir\f[R] on the remote.
|
|
With file name encryption, files saved to
|
|
\f[C]secret:subdir/subfile\f[R] are stored in the unencrypted path
|
|
\f[C]path/to/dir\f[R] but the \f[C]subdir/subpath\f[R] element is
|
|
encrypted.
|
|
.PP
|
|
The path you specify does not have to exist, rclone will create it when
|
|
needed.
|
|
.PP
|
|
If you intend to use the wrapped remote both directly for keeping
|
|
unencrypted content, as well as through a crypt remote for encrypted
|
|
content, it is recommended to point the crypt remote to a separate
|
|
directory within the wrapped remote.
|
|
If you use a bucket based storage system (e.g.
|
|
Swift, S3, Google Compute Storage, B2, Hubic) it is generally advisable
|
|
to wrap the crypt remote around a specific bucket (\f[C]s3:bucket\f[R]).
|
|
If wrapping around the entire root of the storage (\f[C]s3:\f[R]), and
|
|
use the optional file name encryption, rclone will encrypt the bucket
|
|
name.
|
|
.SS Changing password
|
|
.PP
|
|
Should the password, or the configuration file containing a lightly
|
|
obscured form of the password, be compromised, you need to re\-encrypt
|
|
your data with a new password.
|
|
Since rclone uses secret\-key encryption, where the encryption key is
|
|
generated directly from the password kept on the client, it is not
|
|
possible to change the password/key of already encrypted content.
|
|
Just changing the password configured for an existing crypt remote means
|
|
you will no longer able to decrypt any of the previously encrypted
|
|
content.
|
|
The only possibility is to re\-upload everything via a crypt remote
|
|
configured with your new password.
|
|
.PP
|
|
Depending on the size of your data, your bandwith, storage quota etc,
|
|
there are different approaches you can take: \- If you have everything
|
|
in a different location, for example on your local system, you could
|
|
remove all of the prior encrypted files, change the password for your
|
|
configured crypt remote (or delete and re\-create the crypt
|
|
configuration), and then re\-upload everything from the alternative
|
|
location.
|
|
\- If you have enough space on the storage system you can create a new
|
|
crypt remote pointing to a separate directory on the same backend, and
|
|
then use rclone to copy everything from the original crypt remote to the
|
|
new, effectively decrypting everything on the fly using the old password
|
|
and re\-encrypting using the new password.
|
|
When done, delete the original crypt remote directory and finally the
|
|
rclone crypt configuration with the old password.
|
|
All data will be streamed from the storage system and back, so you will
|
|
get half the bandwith and be charged twice if you have upload and
|
|
download quota on the storage system.
|
|
.PP
|
|
\f[B]Note\f[R]: A security problem related to the random password
|
|
generator was fixed in rclone version 1.53.3 (released 2020\-11\-19).
|
|
Passwords generated by rclone config in version 1.49.0 (released
|
|
2019\-08\-26) to 1.53.2 (released 2020\-10\-26) are not considered
|
|
secure and should be changed.
|
|
If you made up your own password, or used rclone version older than
|
|
1.49.0 or newer than 1.53.2 to generate it, you are \f[I]not\f[R]
|
|
affected by this issue.
|
|
See issue #4783 (https://github.com/rclone/rclone/issues/4783) for more
|
|
details, and a tool you can use to check if you are affected.
|
|
.SS Example
|
|
.PP
|
|
Create the following file structure using \[dq]standard\[dq] file name
|
|
encryption.
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
plaintext/
|
|
\[u251C]\[u2500]\[u2500] file0.txt
|
|
\[u251C]\[u2500]\[u2500] file1.txt
|
|
\[u2514]\[u2500]\[u2500] subdir
|
|
\[u251C]\[u2500]\[u2500] file2.txt
|
|
\[u251C]\[u2500]\[u2500] file3.txt
|
|
\[u2514]\[u2500]\[u2500] subsubdir
|
|
\[u2514]\[u2500]\[u2500] file4.txt
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
Copy these to the remote, and list them
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
$ rclone \-q copy plaintext secret:
|
|
$ rclone \-q ls secret:
|
|
7 file1.txt
|
|
6 file0.txt
|
|
8 subdir/file2.txt
|
|
10 subdir/subsubdir/file4.txt
|
|
9 subdir/file3.txt
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
The crypt remote looks like
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
$ rclone \-q ls remote:path
|
|
55 hagjclgavj2mbiqm6u6cnjjqcg
|
|
54 v05749mltvv1tf4onltun46gls
|
|
57 86vhrsv86mpbtd3a0akjuqslj8/dlj7fkq4kdq72emafg7a7s41uo
|
|
58 86vhrsv86mpbtd3a0akjuqslj8/7uu829995du6o42n32otfhjqp4/b9pausrfansjth5ob3jkdqd4lc
|
|
56 86vhrsv86mpbtd3a0akjuqslj8/8njh1sk437gttmep3p70g81aps
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
The directory structure is preserved
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
$ rclone \-q ls secret:subdir
|
|
8 file2.txt
|
|
9 file3.txt
|
|
10 subsubdir/file4.txt
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
Without file name encryption \f[C].bin\f[R] extensions are added to
|
|
underlying names.
|
|
This prevents the cloud provider attempting to interpret file content.
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
$ rclone \-q ls remote:path
|
|
54 file0.txt.bin
|
|
57 subdir/file3.txt.bin
|
|
56 subdir/file2.txt.bin
|
|
58 subdir/subsubdir/file4.txt.bin
|
|
55 file1.txt.bin
|
|
\f[R]
|
|
.fi
|
|
.SS File name encryption modes
|
|
.PP
|
|
Off
|
|
.IP \[bu] 2
|
|
doesn\[aq]t hide file names or directory structure
|
|
.IP \[bu] 2
|
|
allows for longer file names (\[ti]246 characters)
|
|
.IP \[bu] 2
|
|
can use sub paths and copy single files
|
|
.PP
|
|
Standard
|
|
.IP \[bu] 2
|
|
file names encrypted
|
|
.IP \[bu] 2
|
|
file names can\[aq]t be as long (\[ti]143 characters)
|
|
.IP \[bu] 2
|
|
can use sub paths and copy single files
|
|
.IP \[bu] 2
|
|
directory structure visible
|
|
.IP \[bu] 2
|
|
identical files names will have identical uploaded names
|
|
.IP \[bu] 2
|
|
can use shortcuts to shorten the directory recursion
|
|
.PP
|
|
Obfuscation
|
|
.PP
|
|
This is a simple \[dq]rotate\[dq] of the filename, with each file having
|
|
a rot distance based on the filename.
|
|
Rclone stores the distance at the beginning of the filename.
|
|
A file called \[dq]hello\[dq] may become \[dq]53.jgnnq\[dq].
|
|
.PP
|
|
Obfuscation is not a strong encryption of filenames, but hinders
|
|
automated scanning tools picking up on filename patterns.
|
|
It is an intermediate between \[dq]off\[dq] and \[dq]standard\[dq] which
|
|
allows for longer path segment names.
|
|
.PP
|
|
There is a possibility with some unicode based filenames that the
|
|
obfuscation is weak and may map lower case characters to upper case
|
|
equivalents.
|
|
.PP
|
|
Obfuscation cannot be relied upon for strong protection.
|
|
.IP \[bu] 2
|
|
file names very lightly obfuscated
|
|
.IP \[bu] 2
|
|
file names can be longer than standard encryption
|
|
.IP \[bu] 2
|
|
can use sub paths and copy single files
|
|
.IP \[bu] 2
|
|
directory structure visible
|
|
.IP \[bu] 2
|
|
identical files names will have identical uploaded names
|
|
.PP
|
|
Cloud storage systems have limits on file name length and total path
|
|
length which rclone is more likely to breach using \[dq]Standard\[dq]
|
|
file name encryption.
|
|
Where file names are less thn 156 characters in length issues should not
|
|
be encountered, irrespective of cloud storage provider.
|
|
.PP
|
|
An alternative, future rclone file name encryption mode may tolerate
|
|
backend provider path length limits.
|
|
.SS Directory name encryption
|
|
.PP
|
|
Crypt offers the option of encrypting dir names or leaving them intact.
|
|
There are two options:
|
|
.PP
|
|
True
|
|
.PP
|
|
Encrypts the whole file path including directory names Example:
|
|
\f[C]1/12/123.txt\f[R] is encrypted to
|
|
\f[C]p0e52nreeaj0a5ea7s64m4j72s/l42g6771hnv3an9cgc8cr2n1ng/qgm4avr35m5loi1th53ato71v0\f[R]
|
|
.PP
|
|
False
|
|
.PP
|
|
Only encrypts file names, skips directory names Example:
|
|
\f[C]1/12/123.txt\f[R] is encrypted to
|
|
\f[C]1/12/qgm4avr35m5loi1th53ato71v0\f[R]
|
|
.SS Modified time and hashes
|
|
.PP
|
|
Crypt stores modification times using the underlying remote so support
|
|
depends on that.
|
|
.PP
|
|
Hashes are not stored for crypt.
|
|
However the data integrity is protected by an extremely strong crypto
|
|
authenticator.
|
|
.PP
|
|
Use the \f[C]rclone cryptcheck\f[R] command to check the integrity of a
|
|
crypted remote instead of \f[C]rclone check\f[R] which can\[aq]t check
|
|
the checksums properly.
|
|
.SS Standard Options
|
|
.PP
|
|
Here are the standard options specific to crypt (Encrypt/Decrypt a
|
|
remote).
|
|
.SS \-\-crypt\-remote
|
|
.PP
|
|
Remote to encrypt/decrypt.
|
|
Normally should contain a \[aq]:\[aq] and a path, e.g.
|
|
\[dq]myremote:path/to/dir\[dq], \[dq]myremote:bucket\[dq] or maybe
|
|
\[dq]myremote:\[dq] (not recommended).
|
|
.IP \[bu] 2
|
|
Config: remote
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_CRYPT_REMOTE
|
|
.IP \[bu] 2
|
|
Type: string
|
|
.IP \[bu] 2
|
|
Default: \[dq]\[dq]
|
|
.SS \-\-crypt\-filename\-encryption
|
|
.PP
|
|
How to encrypt the filenames.
|
|
.IP \[bu] 2
|
|
Config: filename_encryption
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_CRYPT_FILENAME_ENCRYPTION
|
|
.IP \[bu] 2
|
|
Type: string
|
|
.IP \[bu] 2
|
|
Default: \[dq]standard\[dq]
|
|
.IP \[bu] 2
|
|
Examples:
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
\[dq]standard\[dq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Encrypt the filenames see the docs for the details.
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[dq]obfuscate\[dq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Very simple filename obfuscation.
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[dq]off\[dq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Don\[aq]t encrypt the file names.
|
|
Adds a \[dq].bin\[dq] extension only.
|
|
.RE
|
|
.RE
|
|
.SS \-\-crypt\-directory\-name\-encryption
|
|
.PP
|
|
Option to either encrypt directory names or leave them intact.
|
|
.PP
|
|
NB If filename_encryption is \[dq]off\[dq] then this option will do
|
|
nothing.
|
|
.IP \[bu] 2
|
|
Config: directory_name_encryption
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_CRYPT_DIRECTORY_NAME_ENCRYPTION
|
|
.IP \[bu] 2
|
|
Type: bool
|
|
.IP \[bu] 2
|
|
Default: true
|
|
.IP \[bu] 2
|
|
Examples:
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
\[dq]true\[dq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Encrypt directory names.
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[dq]false\[dq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Don\[aq]t encrypt directory names, leave them intact.
|
|
.RE
|
|
.RE
|
|
.SS \-\-crypt\-password
|
|
.PP
|
|
Password or pass phrase for encryption.
|
|
.PP
|
|
\f[B]NB\f[R] Input to this must be obscured \- see rclone
|
|
obscure (https://rclone.org/commands/rclone_obscure/).
|
|
.IP \[bu] 2
|
|
Config: password
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_CRYPT_PASSWORD
|
|
.IP \[bu] 2
|
|
Type: string
|
|
.IP \[bu] 2
|
|
Default: \[dq]\[dq]
|
|
.SS \-\-crypt\-password2
|
|
.PP
|
|
Password or pass phrase for salt.
|
|
Optional but recommended.
|
|
Should be different to the previous password.
|
|
.PP
|
|
\f[B]NB\f[R] Input to this must be obscured \- see rclone
|
|
obscure (https://rclone.org/commands/rclone_obscure/).
|
|
.IP \[bu] 2
|
|
Config: password2
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_CRYPT_PASSWORD2
|
|
.IP \[bu] 2
|
|
Type: string
|
|
.IP \[bu] 2
|
|
Default: \[dq]\[dq]
|
|
.SS Advanced Options
|
|
.PP
|
|
Here are the advanced options specific to crypt (Encrypt/Decrypt a
|
|
remote).
|
|
.SS \-\-crypt\-server\-side\-across\-configs
|
|
.PP
|
|
Allow server\-side operations (e.g.
|
|
copy) to work across different crypt configs.
|
|
.PP
|
|
Normally this option is not what you want, but if you have two crypts
|
|
pointing to the same backend you can use it.
|
|
.PP
|
|
This can be used, for example, to change file name encryption type
|
|
without re\-uploading all the data.
|
|
Just make two crypt backends pointing to two different directories with
|
|
the single changed parameter and use rclone move to move the files
|
|
between the crypt remotes.
|
|
.IP \[bu] 2
|
|
Config: server_side_across_configs
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_CRYPT_SERVER_SIDE_ACROSS_CONFIGS
|
|
.IP \[bu] 2
|
|
Type: bool
|
|
.IP \[bu] 2
|
|
Default: false
|
|
.SS \-\-crypt\-show\-mapping
|
|
.PP
|
|
For all files listed show how the names encrypt.
|
|
.PP
|
|
If this flag is set then for each file that the remote is asked to list,
|
|
it will log (at level INFO) a line stating the decrypted file name and
|
|
the encrypted file name.
|
|
.PP
|
|
This is so you can work out which encrypted names are which decrypted
|
|
names just in case you need to do something with the encrypted file
|
|
names, or for debugging purposes.
|
|
.IP \[bu] 2
|
|
Config: show_mapping
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_CRYPT_SHOW_MAPPING
|
|
.IP \[bu] 2
|
|
Type: bool
|
|
.IP \[bu] 2
|
|
Default: false
|
|
.SS \-\-crypt\-no\-data\-encryption
|
|
.PP
|
|
Option to either encrypt file data or leave it unencrypted.
|
|
.IP \[bu] 2
|
|
Config: no_data_encryption
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_CRYPT_NO_DATA_ENCRYPTION
|
|
.IP \[bu] 2
|
|
Type: bool
|
|
.IP \[bu] 2
|
|
Default: false
|
|
.IP \[bu] 2
|
|
Examples:
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
\[dq]true\[dq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Don\[aq]t encrypt file data, leave it unencrypted.
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[dq]false\[dq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Encrypt file data.
|
|
.RE
|
|
.RE
|
|
.SS Backend commands
|
|
.PP
|
|
Here are the commands specific to the crypt backend.
|
|
.PP
|
|
Run them with
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone backend COMMAND remote:
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
The help below will explain what arguments each command takes.
|
|
.PP
|
|
See the \[dq]rclone backend\[dq]
|
|
command (https://rclone.org/commands/rclone_backend/) for more info on
|
|
how to pass options and arguments.
|
|
.PP
|
|
These can be run on a running backend using the rc command
|
|
backend/command (https://rclone.org/rc/#backend/command).
|
|
.SS encode
|
|
.PP
|
|
Encode the given filename(s)
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone backend encode remote: [options] [<arguments>+]
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
This encodes the filenames given as arguments returning a list of
|
|
strings of the encoded results.
|
|
.PP
|
|
Usage Example:
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone backend encode crypt: file1 [file2...]
|
|
rclone rc backend/command command=encode fs=crypt: file1 [file2...]
|
|
\f[R]
|
|
.fi
|
|
.SS decode
|
|
.PP
|
|
Decode the given filename(s)
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone backend decode remote: [options] [<arguments>+]
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
This decodes the filenames given as arguments returning a list of
|
|
strings of the decoded results.
|
|
It will return an error if any of the inputs are invalid.
|
|
.PP
|
|
Usage Example:
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone backend decode crypt: encryptedfile1 [encryptedfile2...]
|
|
rclone rc backend/command command=decode fs=crypt: encryptedfile1 [encryptedfile2...]
|
|
\f[R]
|
|
.fi
|
|
.SS Backing up a crypted remote
|
|
.PP
|
|
If you wish to backup a crypted remote, it is recommended that you use
|
|
\f[C]rclone sync\f[R] on the encrypted files, and make sure the
|
|
passwords are the same in the new encrypted remote.
|
|
.PP
|
|
This will have the following advantages
|
|
.IP \[bu] 2
|
|
\f[C]rclone sync\f[R] will check the checksums while copying
|
|
.IP \[bu] 2
|
|
you can use \f[C]rclone check\f[R] between the encrypted remotes
|
|
.IP \[bu] 2
|
|
you don\[aq]t decrypt and encrypt unnecessarily
|
|
.PP
|
|
For example, let\[aq]s say you have your original remote at
|
|
\f[C]remote:\f[R] with the encrypted version at \f[C]eremote:\f[R] with
|
|
path \f[C]remote:crypt\f[R].
|
|
You would then set up the new remote \f[C]remote2:\f[R] and then the
|
|
encrypted version \f[C]eremote2:\f[R] with path \f[C]remote2:crypt\f[R]
|
|
using the same passwords as \f[C]eremote:\f[R].
|
|
.PP
|
|
To sync the two remotes you would do
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone sync \-i remote:crypt remote2:crypt
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
And to check the integrity you would do
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone check remote:crypt remote2:crypt
|
|
\f[R]
|
|
.fi
|
|
.SS File formats
|
|
.SS File encryption
|
|
.PP
|
|
Files are encrypted 1:1 source file to destination object.
|
|
The file has a header and is divided into chunks.
|
|
.SS Header
|
|
.IP \[bu] 2
|
|
8 bytes magic string \f[C]RCLONE\[rs]x00\[rs]x00\f[R]
|
|
.IP \[bu] 2
|
|
24 bytes Nonce (IV)
|
|
.PP
|
|
The initial nonce is generated from the operating systems crypto strong
|
|
random number generator.
|
|
The nonce is incremented for each chunk read making sure each nonce is
|
|
unique for each block written.
|
|
The chance of a nonce being re\-used is minuscule.
|
|
If you wrote an exabyte of data (10\[S1]\[u2078] bytes) you would have a
|
|
probability of approximately 2\[tmu]10\[u207B]\[S3]\[S2] of re\-using a
|
|
nonce.
|
|
.SS Chunk
|
|
.PP
|
|
Each chunk will contain 64kB of data, except for the last one which may
|
|
have less data.
|
|
The data chunk is in standard NaCl SecretBox format.
|
|
SecretBox uses XSalsa20 and Poly1305 to encrypt and authenticate
|
|
messages.
|
|
.PP
|
|
Each chunk contains:
|
|
.IP \[bu] 2
|
|
16 Bytes of Poly1305 authenticator
|
|
.IP \[bu] 2
|
|
1 \- 65536 bytes XSalsa20 encrypted data
|
|
.PP
|
|
64k chunk size was chosen as the best performing chunk size (the
|
|
authenticator takes too much time below this and the performance drops
|
|
off due to cache effects above this).
|
|
Note that these chunks are buffered in memory so they can\[aq]t be too
|
|
big.
|
|
.PP
|
|
This uses a 32 byte (256 bit key) key derived from the user password.
|
|
.SS Examples
|
|
.PP
|
|
1 byte file will encrypt to
|
|
.IP \[bu] 2
|
|
32 bytes header
|
|
.IP \[bu] 2
|
|
17 bytes data chunk
|
|
.PP
|
|
49 bytes total
|
|
.PP
|
|
1MB (1048576 bytes) file will encrypt to
|
|
.IP \[bu] 2
|
|
32 bytes header
|
|
.IP \[bu] 2
|
|
16 chunks of 65568 bytes
|
|
.PP
|
|
1049120 bytes total (a 0.05% overhead).
|
|
This is the overhead for big files.
|
|
.SS Name encryption
|
|
.PP
|
|
File names are encrypted segment by segment \- the path is broken up
|
|
into \f[C]/\f[R] separated strings and these are encrypted individually.
|
|
.PP
|
|
File segments are padded using PKCS#7 to a multiple of 16 bytes before
|
|
encryption.
|
|
.PP
|
|
They are then encrypted with EME using AES with 256 bit key.
|
|
EME (ECB\-Mix\-ECB) is a wide\-block encryption mode presented in the
|
|
2003 paper \[dq]A Parallelizable Enciphering Mode\[dq] by Halevi and
|
|
Rogaway.
|
|
.PP
|
|
This makes for deterministic encryption which is what we want \- the
|
|
same filename must encrypt to the same thing otherwise we can\[aq]t find
|
|
it on the cloud storage system.
|
|
.PP
|
|
This means that
|
|
.IP \[bu] 2
|
|
filenames with the same name will encrypt the same
|
|
.IP \[bu] 2
|
|
filenames which start the same won\[aq]t have a common prefix
|
|
.PP
|
|
This uses a 32 byte key (256 bits) and a 16 byte (128 bits) IV both of
|
|
which are derived from the user password.
|
|
.PP
|
|
After encryption they are written out using a modified version of
|
|
standard \f[C]base32\f[R] encoding as described in RFC4648.
|
|
The standard encoding is modified in two ways:
|
|
.IP \[bu] 2
|
|
it becomes lower case (no\-one likes upper case filenames!)
|
|
.IP \[bu] 2
|
|
we strip the padding character \f[C]=\f[R]
|
|
.PP
|
|
\f[C]base32\f[R] is used rather than the more efficient \f[C]base64\f[R]
|
|
so rclone can be used on case insensitive remotes (e.g.
|
|
Windows, Amazon Drive).
|
|
.SS Key derivation
|
|
.PP
|
|
Rclone uses \f[C]scrypt\f[R] with parameters \f[C]N=16384, r=8, p=1\f[R]
|
|
with an optional user supplied salt (password2) to derive the 32+32+16 =
|
|
80 bytes of key material required.
|
|
If the user doesn\[aq]t supply a salt then rclone uses an internal one.
|
|
.PP
|
|
\f[C]scrypt\f[R] makes it impractical to mount a dictionary attack on
|
|
rclone encrypted data.
|
|
For full protection against this you should always use a salt.
|
|
.SS SEE ALSO
|
|
.IP \[bu] 2
|
|
rclone cryptdecode (https://rclone.org/commands/rclone_cryptdecode/) \-
|
|
Show forward/reverse mapping of encrypted filenames
|
|
.SS Compress (Experimental)
|
|
.SS Warning
|
|
.PP
|
|
This remote is currently \f[B]experimental\f[R].
|
|
Things may break and data may be lost.
|
|
Anything you do with this remote is at your own risk.
|
|
Please understand the risks associated with using experimental code and
|
|
don\[aq]t use this remote in critical applications.
|
|
.PP
|
|
The \f[C]Compress\f[R] remote adds compression to another remote.
|
|
It is best used with remotes containing many large compressible files.
|
|
.PP
|
|
To use this remote, all you need to do is specify another remote and a
|
|
compression mode to use:
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
Current remotes:
|
|
|
|
Name Type
|
|
==== ====
|
|
remote_to_press sometype
|
|
|
|
e) Edit existing remote
|
|
$ rclone config
|
|
n) New remote
|
|
d) Delete remote
|
|
r) Rename remote
|
|
c) Copy remote
|
|
s) Set configuration password
|
|
q) Quit config
|
|
e/n/d/r/c/s/q> n
|
|
name> compress
|
|
\&...
|
|
8 / Compress a remote
|
|
\[rs] \[dq]compress\[dq]
|
|
\&...
|
|
Storage> compress
|
|
** See help for compress backend at: https://rclone.org/compress/ **
|
|
|
|
Remote to compress.
|
|
Enter a string value. Press Enter for the default (\[dq]\[dq]).
|
|
remote> remote_to_press:subdir
|
|
Compression mode.
|
|
Enter a string value. Press Enter for the default (\[dq]gzip\[dq]).
|
|
Choose a number from below, or type in your own value
|
|
1 / Gzip compression balanced for speed and compression strength.
|
|
\[rs] \[dq]gzip\[dq]
|
|
compression_mode> gzip
|
|
Edit advanced config? (y/n)
|
|
y) Yes
|
|
n) No (default)
|
|
y/n> n
|
|
Remote config
|
|
\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-
|
|
[compress]
|
|
type = compress
|
|
remote = remote_to_press:subdir
|
|
compression_mode = gzip
|
|
\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-
|
|
y) Yes this is OK (default)
|
|
e) Edit this remote
|
|
d) Delete this remote
|
|
y/e/d> y
|
|
\f[R]
|
|
.fi
|
|
.SS Compression Modes
|
|
.PP
|
|
Currently only gzip compression is supported, it provides a decent
|
|
balance between speed and strength and is well supported by other
|
|
application.
|
|
Compression strength can further be configured via an advanced setting
|
|
where 0 is no compression and 9 is strongest compression.
|
|
.SS Filetype
|
|
.PP
|
|
If you open a remote wrapped by press, you will see that there are many
|
|
files with an extension corresponding to the compression algorithm you
|
|
chose.
|
|
These files are standard files that can be opened by various archive
|
|
programs, but they have some hidden metadata that allows them to be used
|
|
by rclone.
|
|
While you may download and decompress these files at will, do
|
|
\f[B]not\f[R] manually delete or rename files.
|
|
Files without correct metadata files will not be recognized by rclone.
|
|
.SS File names
|
|
.PP
|
|
The compressed files will be named \f[C]*.###########.gz\f[R] where
|
|
\f[C]*\f[R] is the base file and the \f[C]#\f[R] part is base64 encoded
|
|
size of the uncompressed file.
|
|
The file names should not be changed by anything other than the rclone
|
|
compression backend.
|
|
.SS Standard Options
|
|
.PP
|
|
Here are the standard options specific to compress (Compress a remote).
|
|
.SS \-\-compress\-remote
|
|
.PP
|
|
Remote to compress.
|
|
.IP \[bu] 2
|
|
Config: remote
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_COMPRESS_REMOTE
|
|
.IP \[bu] 2
|
|
Type: string
|
|
.IP \[bu] 2
|
|
Default: \[dq]\[dq]
|
|
.SS \-\-compress\-mode
|
|
.PP
|
|
Compression mode.
|
|
.IP \[bu] 2
|
|
Config: mode
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_COMPRESS_MODE
|
|
.IP \[bu] 2
|
|
Type: string
|
|
.IP \[bu] 2
|
|
Default: \[dq]gzip\[dq]
|
|
.IP \[bu] 2
|
|
Examples:
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
\[dq]gzip\[dq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Standard gzip compression with fastest parameters.
|
|
.RE
|
|
.RE
|
|
.SS Advanced Options
|
|
.PP
|
|
Here are the advanced options specific to compress (Compress a remote).
|
|
.SS \-\-compress\-level
|
|
.PP
|
|
GZIP compression level (\-2 to 9).
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
Generally \-1 (default, equivalent to 5) is recommended.
|
|
Levels 1 to 9 increase compressiong at the cost of speed.. Going past 6
|
|
generally offers very little return.
|
|
|
|
Level \-2 uses Huffmann encoding only. Only use if you now what you
|
|
are doing
|
|
Level 0 turns off compression.
|
|
\f[R]
|
|
.fi
|
|
.IP \[bu] 2
|
|
Config: level
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_COMPRESS_LEVEL
|
|
.IP \[bu] 2
|
|
Type: int
|
|
.IP \[bu] 2
|
|
Default: \-1
|
|
.SS \-\-compress\-ram\-cache\-limit
|
|
.PP
|
|
Some remotes don\[aq]t allow the upload of files with unknown size.
|
|
In this case the compressed file will need to be cached to determine
|
|
it\[aq]s size.
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
Files smaller than this limit will be cached in RAM, file larger than
|
|
this limit will be cached on disk
|
|
\f[R]
|
|
.fi
|
|
.IP \[bu] 2
|
|
Config: ram_cache_limit
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_COMPRESS_RAM_CACHE_LIMIT
|
|
.IP \[bu] 2
|
|
Type: SizeSuffix
|
|
.IP \[bu] 2
|
|
Default: 20M
|
|
.SS Dropbox
|
|
.PP
|
|
Paths are specified as \f[C]remote:path\f[R]
|
|
.PP
|
|
Dropbox paths may be as deep as required, e.g.
|
|
\f[C]remote:directory/subdirectory\f[R].
|
|
.PP
|
|
The initial setup for dropbox involves getting a token from Dropbox
|
|
which you need to do in your browser.
|
|
\f[C]rclone config\f[R] walks you through it.
|
|
.PP
|
|
Here is an example of how to make a remote called \f[C]remote\f[R].
|
|
First run:
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone config
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
This will guide you through an interactive setup process:
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
n) New remote
|
|
d) Delete remote
|
|
q) Quit config
|
|
e/n/d/q> n
|
|
name> remote
|
|
Type of storage to configure.
|
|
Choose a number from below, or type in your own value
|
|
[snip]
|
|
XX / Dropbox
|
|
\[rs] \[dq]dropbox\[dq]
|
|
[snip]
|
|
Storage> dropbox
|
|
Dropbox App Key \- leave blank normally.
|
|
app_key>
|
|
Dropbox App Secret \- leave blank normally.
|
|
app_secret>
|
|
Remote config
|
|
Please visit:
|
|
https://www.dropbox.com/1/oauth2/authorize?client_id=XXXXXXXXXXXXXXX&response_type=code
|
|
Enter the code: XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX_XXXXXXXXXX
|
|
\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-
|
|
[remote]
|
|
app_key =
|
|
app_secret =
|
|
token = XXXXXXXXXXXXXXXXXXXXXXXXXXXXX_XXXX_XXXXXXXXXXXXXXXXXXXXXXXXXXXXX
|
|
\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-
|
|
y) Yes this is OK
|
|
e) Edit this remote
|
|
d) Delete this remote
|
|
y/e/d> y
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
You can then use it like this,
|
|
.PP
|
|
List directories in top level of your dropbox
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone lsd remote:
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
List all the files in your dropbox
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone ls remote:
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
To copy a local directory to a dropbox directory called backup
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone copy /home/source remote:backup
|
|
\f[R]
|
|
.fi
|
|
.SS Dropbox for business
|
|
.PP
|
|
Rclone supports Dropbox for business and Team Folders.
|
|
.PP
|
|
When using Dropbox for business \f[C]remote:\f[R] and
|
|
\f[C]remote:path/to/file\f[R] will refer to your personal folder.
|
|
.PP
|
|
If you wish to see Team Folders you must use a leading \f[C]/\f[R] in
|
|
the path, so \f[C]rclone lsd remote:/\f[R] will refer to the root and
|
|
show you all Team Folders and your User Folder.
|
|
.PP
|
|
You can then use team folders like this \f[C]remote:/TeamFolder\f[R] and
|
|
\f[C]remote:/TeamFolder/path/to/file\f[R].
|
|
.PP
|
|
A leading \f[C]/\f[R] for a Dropbox personal account will do nothing,
|
|
but it will take an extra HTTP transaction so it should be avoided.
|
|
.SS Modified time and Hashes
|
|
.PP
|
|
Dropbox supports modified times, but the only way to set a modification
|
|
time is to re\-upload the file.
|
|
.PP
|
|
This means that if you uploaded your data with an older version of
|
|
rclone which didn\[aq]t support the v2 API and modified times, rclone
|
|
will decide to upload all your old data to fix the modification times.
|
|
If you don\[aq]t want this to happen use \f[C]\-\-size\-only\f[R] or
|
|
\f[C]\-\-checksum\f[R] flag to stop it.
|
|
.PP
|
|
Dropbox supports its own hash
|
|
type (https://www.dropbox.com/developers/reference/content-hash) which
|
|
is checked for all transfers.
|
|
.SS Restricted filename characters
|
|
.PP
|
|
.TS
|
|
tab(@);
|
|
l c c.
|
|
T{
|
|
Character
|
|
T}@T{
|
|
Value
|
|
T}@T{
|
|
Replacement
|
|
T}
|
|
_
|
|
T{
|
|
NUL
|
|
T}@T{
|
|
0x00
|
|
T}@T{
|
|
\[u2400]
|
|
T}
|
|
T{
|
|
/
|
|
T}@T{
|
|
0x2F
|
|
T}@T{
|
|
\[uFF0F]
|
|
T}
|
|
T{
|
|
DEL
|
|
T}@T{
|
|
0x7F
|
|
T}@T{
|
|
\[u2421]
|
|
T}
|
|
T{
|
|
\[rs]
|
|
T}@T{
|
|
0x5C
|
|
T}@T{
|
|
\[uFF3C]
|
|
T}
|
|
.TE
|
|
.PP
|
|
File names can also not end with the following characters.
|
|
These only get replaced if they are the last character in the name:
|
|
.PP
|
|
.TS
|
|
tab(@);
|
|
l c c.
|
|
T{
|
|
Character
|
|
T}@T{
|
|
Value
|
|
T}@T{
|
|
Replacement
|
|
T}
|
|
_
|
|
T{
|
|
SP
|
|
T}@T{
|
|
0x20
|
|
T}@T{
|
|
\[u2420]
|
|
T}
|
|
.TE
|
|
.PP
|
|
Invalid UTF\-8 bytes will also be
|
|
replaced (https://rclone.org/overview/#invalid-utf8), as they can\[aq]t
|
|
be used in JSON strings.
|
|
.SS Standard Options
|
|
.PP
|
|
Here are the standard options specific to dropbox (Dropbox).
|
|
.SS \-\-dropbox\-client\-id
|
|
.PP
|
|
OAuth Client Id Leave blank normally.
|
|
.IP \[bu] 2
|
|
Config: client_id
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_DROPBOX_CLIENT_ID
|
|
.IP \[bu] 2
|
|
Type: string
|
|
.IP \[bu] 2
|
|
Default: \[dq]\[dq]
|
|
.SS \-\-dropbox\-client\-secret
|
|
.PP
|
|
OAuth Client Secret Leave blank normally.
|
|
.IP \[bu] 2
|
|
Config: client_secret
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_DROPBOX_CLIENT_SECRET
|
|
.IP \[bu] 2
|
|
Type: string
|
|
.IP \[bu] 2
|
|
Default: \[dq]\[dq]
|
|
.SS Advanced Options
|
|
.PP
|
|
Here are the advanced options specific to dropbox (Dropbox).
|
|
.SS \-\-dropbox\-token
|
|
.PP
|
|
OAuth Access Token as a JSON blob.
|
|
.IP \[bu] 2
|
|
Config: token
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_DROPBOX_TOKEN
|
|
.IP \[bu] 2
|
|
Type: string
|
|
.IP \[bu] 2
|
|
Default: \[dq]\[dq]
|
|
.SS \-\-dropbox\-auth\-url
|
|
.PP
|
|
Auth server URL.
|
|
Leave blank to use the provider defaults.
|
|
.IP \[bu] 2
|
|
Config: auth_url
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_DROPBOX_AUTH_URL
|
|
.IP \[bu] 2
|
|
Type: string
|
|
.IP \[bu] 2
|
|
Default: \[dq]\[dq]
|
|
.SS \-\-dropbox\-token\-url
|
|
.PP
|
|
Token server url.
|
|
Leave blank to use the provider defaults.
|
|
.IP \[bu] 2
|
|
Config: token_url
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_DROPBOX_TOKEN_URL
|
|
.IP \[bu] 2
|
|
Type: string
|
|
.IP \[bu] 2
|
|
Default: \[dq]\[dq]
|
|
.SS \-\-dropbox\-chunk\-size
|
|
.PP
|
|
Upload chunk size.
|
|
(< 150M).
|
|
.PP
|
|
Any files larger than this will be uploaded in chunks of this size.
|
|
.PP
|
|
Note that chunks are buffered in memory (one at a time) so rclone can
|
|
deal with retries.
|
|
Setting this larger will increase the speed slightly (at most 10% for
|
|
128MB in tests) at the cost of using more memory.
|
|
It can be set smaller if you are tight on memory.
|
|
.IP \[bu] 2
|
|
Config: chunk_size
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_DROPBOX_CHUNK_SIZE
|
|
.IP \[bu] 2
|
|
Type: SizeSuffix
|
|
.IP \[bu] 2
|
|
Default: 48M
|
|
.SS \-\-dropbox\-impersonate
|
|
.PP
|
|
Impersonate this user when using a business account.
|
|
.PP
|
|
Note that if you want to use impersonate, you should make sure this flag
|
|
is set when running \[dq]rclone config\[dq] as this will cause rclone to
|
|
request the \[dq]members.read\[dq] scope which it won\[aq]t normally.
|
|
This is needed to lookup a members email address into the internal ID
|
|
that dropbox uses in the API.
|
|
.PP
|
|
Using the \[dq]members.read\[dq] scope will require a Dropbox Team Admin
|
|
to approve during the OAuth flow.
|
|
.PP
|
|
You will have to use your own App (setting your own client_id and
|
|
client_secret) to use this option as currently rclone\[aq]s default set
|
|
of permissions doesn\[aq]t include \[dq]members.read\[dq].
|
|
This can be added once v1.55 or later is in use everywhere.
|
|
.IP \[bu] 2
|
|
Config: impersonate
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_DROPBOX_IMPERSONATE
|
|
.IP \[bu] 2
|
|
Type: string
|
|
.IP \[bu] 2
|
|
Default: \[dq]\[dq]
|
|
.SS \-\-dropbox\-shared\-files
|
|
.PP
|
|
Instructs rclone to work on individual shared files.
|
|
.PP
|
|
In this mode rclone\[aq]s features are extremely limited \- only list
|
|
(ls, lsl, etc.) operations and read operations (e.g.
|
|
downloading) are supported in this mode.
|
|
All other operations will be disabled.
|
|
.IP \[bu] 2
|
|
Config: shared_files
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_DROPBOX_SHARED_FILES
|
|
.IP \[bu] 2
|
|
Type: bool
|
|
.IP \[bu] 2
|
|
Default: false
|
|
.SS \-\-dropbox\-shared\-folders
|
|
.PP
|
|
Instructs rclone to work on shared folders.
|
|
.PP
|
|
When this flag is used with no path only the List operation is supported
|
|
and all available shared folders will be listed.
|
|
If you specify a path the first part will be interpreted as the name of
|
|
shared folder.
|
|
Rclone will then try to mount this shared to the root namespace.
|
|
On success shared folder rclone proceeds normally.
|
|
The shared folder is now pretty much a normal folder and all normal
|
|
operations are supported.
|
|
.PP
|
|
Note that we don\[aq]t unmount the shared folder afterwards so the
|
|
\-\-dropbox\-shared\-folders can be omitted after the first use of a
|
|
particular shared folder.
|
|
.IP \[bu] 2
|
|
Config: shared_folders
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_DROPBOX_SHARED_FOLDERS
|
|
.IP \[bu] 2
|
|
Type: bool
|
|
.IP \[bu] 2
|
|
Default: false
|
|
.SS \-\-dropbox\-encoding
|
|
.PP
|
|
This sets the encoding for the backend.
|
|
.PP
|
|
See: the encoding section in the
|
|
overview (https://rclone.org/overview/#encoding) for more info.
|
|
.IP \[bu] 2
|
|
Config: encoding
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_DROPBOX_ENCODING
|
|
.IP \[bu] 2
|
|
Type: MultiEncoder
|
|
.IP \[bu] 2
|
|
Default: Slash,BackSlash,Del,RightSpace,InvalidUtf8,Dot
|
|
.SS Limitations
|
|
.PP
|
|
Note that Dropbox is case insensitive so you can\[aq]t have a file
|
|
called \[dq]Hello.doc\[dq] and one called \[dq]hello.doc\[dq].
|
|
.PP
|
|
There are some file names such as \f[C]thumbs.db\f[R] which Dropbox
|
|
can\[aq]t store.
|
|
There is a full list of them in the \[dq]Ignored Files\[dq] section of
|
|
this document (https://www.dropbox.com/en/help/145).
|
|
Rclone will issue an error message
|
|
\f[C]File name disallowed \- not uploading\f[R] if it attempts to upload
|
|
one of those file names, but the sync won\[aq]t fail.
|
|
.PP
|
|
Some errors may occur if you try to sync copyright\-protected files
|
|
because Dropbox has its own copyright
|
|
detector (https://techcrunch.com/2014/03/30/how-dropbox-knows-when-youre-sharing-copyrighted-stuff-without-actually-looking-at-your-stuff/)
|
|
that prevents this sort of file being downloaded.
|
|
This will return the error
|
|
\f[C]ERROR : /path/to/your/file: Failed to copy: failed to open source object: path/restricted_content/.\f[R]
|
|
.PP
|
|
If you have more than 10,000 files in a directory then
|
|
\f[C]rclone purge dropbox:dir\f[R] will return the error
|
|
\f[C]Failed to purge: There are too many files involved in this operation\f[R].
|
|
As a work\-around do an \f[C]rclone delete dropbox:dir\f[R] followed by
|
|
an \f[C]rclone rmdir dropbox:dir\f[R].
|
|
.SS Get your own Dropbox App ID
|
|
.PP
|
|
When you use rclone with Dropbox in its default configuration you are
|
|
using rclone\[aq]s App ID.
|
|
This is shared between all the rclone users.
|
|
.PP
|
|
Here is how to create your own Dropbox App ID for rclone:
|
|
.IP "1." 3
|
|
Log into the Dropbox App
|
|
console (https://www.dropbox.com/developers/apps/create) with your
|
|
Dropbox Account (It need not to be the same account as the Dropbox you
|
|
want to access)
|
|
.IP "2." 3
|
|
Choose an API => Usually this should be \f[C]Dropbox API\f[R]
|
|
.IP "3." 3
|
|
Choose the type of access you want to use => \f[C]Full Dropbox\f[R] or
|
|
\f[C]App Folder\f[R]
|
|
.IP "4." 3
|
|
Name your App.
|
|
The app name is global, so you can\[aq]t use \f[C]rclone\f[R] for
|
|
example
|
|
.IP "5." 3
|
|
Click the button \f[C]Create App\f[R]
|
|
.IP "6." 3
|
|
Fill \f[C]Redirect URIs\f[R] as \f[C]http://localhost:53682/\f[R]
|
|
.IP "7." 3
|
|
Find the \f[C]App key\f[R] and \f[C]App secret\f[R] Use these values in
|
|
rclone config to add a new remote or edit an existing remote.
|
|
.SS Enterprise File Fabric
|
|
.PP
|
|
This backend supports Storage Made Easy\[aq]s Enterprise File
|
|
Fabric\[tm] (https://storagemadeeasy.com/about/) which provides a
|
|
software solution to integrate and unify File and Object Storage
|
|
accessible through a global file system.
|
|
.PP
|
|
The initial setup for the Enterprise File Fabric backend involves
|
|
getting a token from the the Enterprise File Fabric which you need to do
|
|
in your browser.
|
|
\f[C]rclone config\f[R] walks you through it.
|
|
.PP
|
|
Here is an example of how to make a remote called \f[C]remote\f[R].
|
|
First run:
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone config
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
This will guide you through an interactive setup process:
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
No remotes found \- make a new one
|
|
n) New remote
|
|
s) Set configuration password
|
|
q) Quit config
|
|
n/s/q> n
|
|
name> remote
|
|
Type of storage to configure.
|
|
Enter a string value. Press Enter for the default (\[dq]\[dq]).
|
|
Choose a number from below, or type in your own value
|
|
[snip]
|
|
XX / Enterprise File Fabric
|
|
\[rs] \[dq]filefabric\[dq]
|
|
[snip]
|
|
Storage> filefabric
|
|
** See help for filefabric backend at: https://rclone.org/filefabric/ **
|
|
|
|
URL of the Enterprise File Fabric to connect to
|
|
Enter a string value. Press Enter for the default (\[dq]\[dq]).
|
|
Choose a number from below, or type in your own value
|
|
1 / Storage Made Easy US
|
|
\[rs] \[dq]https://storagemadeeasy.com\[dq]
|
|
2 / Storage Made Easy EU
|
|
\[rs] \[dq]https://eu.storagemadeeasy.com\[dq]
|
|
3 / Connect to your Enterprise File Fabric
|
|
\[rs] \[dq]https://yourfabric.smestorage.com\[dq]
|
|
url> https://yourfabric.smestorage.com/
|
|
ID of the root folder
|
|
Leave blank normally.
|
|
|
|
Fill in to make rclone start with directory of a given ID.
|
|
|
|
Enter a string value. Press Enter for the default (\[dq]\[dq]).
|
|
root_folder_id>
|
|
Permanent Authentication Token
|
|
|
|
A Permanent Authentication Token can be created in the Enterprise File
|
|
Fabric, on the users Dashboard under Security, there is an entry
|
|
you\[aq]ll see called \[dq]My Authentication Tokens\[dq]. Click the Manage button
|
|
to create one.
|
|
|
|
These tokens are normally valid for several years.
|
|
|
|
For more info see: https://docs.storagemadeeasy.com/organisationcloud/api\-tokens
|
|
|
|
Enter a string value. Press Enter for the default (\[dq]\[dq]).
|
|
permanent_token> xxxxxxxxxxxxxxx\-xxxxxxxxxxxxxxxx
|
|
Edit advanced config? (y/n)
|
|
y) Yes
|
|
n) No (default)
|
|
y/n> n
|
|
Remote config
|
|
\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-
|
|
[remote]
|
|
type = filefabric
|
|
url = https://yourfabric.smestorage.com/
|
|
permanent_token = xxxxxxxxxxxxxxx\-xxxxxxxxxxxxxxxx
|
|
\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-
|
|
y) Yes this is OK (default)
|
|
e) Edit this remote
|
|
d) Delete this remote
|
|
y/e/d> y
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
Once configured you can then use \f[C]rclone\f[R] like this,
|
|
.PP
|
|
List directories in top level of your Enterprise File Fabric
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone lsd remote:
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
List all the files in your Enterprise File Fabric
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone ls remote:
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
To copy a local directory to an Enterprise File Fabric directory called
|
|
backup
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone copy /home/source remote:backup
|
|
\f[R]
|
|
.fi
|
|
.SS Modified time and hashes
|
|
.PP
|
|
The Enterprise File Fabric allows modification times to be set on files
|
|
accurate to 1 second.
|
|
These will be used to detect whether objects need syncing or not.
|
|
.PP
|
|
The Enterprise File Fabric does not support any data hashes at this
|
|
time.
|
|
.SS Restricted filename characters
|
|
.PP
|
|
The default restricted characters
|
|
set (https://rclone.org/overview/#restricted-characters) will be
|
|
replaced.
|
|
.PP
|
|
Invalid UTF\-8 bytes will also be
|
|
replaced (https://rclone.org/overview/#invalid-utf8), as they can\[aq]t
|
|
be used in JSON strings.
|
|
.SS Empty files
|
|
.PP
|
|
Empty files aren\[aq]t supported by the Enterprise File Fabric.
|
|
Rclone will therefore upload an empty file as a single space with a mime
|
|
type of \f[C]application/vnd.rclone.empty.file\f[R] and files with that
|
|
mime type are treated as empty.
|
|
.SS Root folder ID
|
|
.PP
|
|
You can set the \f[C]root_folder_id\f[R] for rclone.
|
|
This is the directory (identified by its \f[C]Folder ID\f[R]) that
|
|
rclone considers to be the root of your Enterprise File Fabric.
|
|
.PP
|
|
Normally you will leave this blank and rclone will determine the correct
|
|
root to use itself.
|
|
.PP
|
|
However you can set this to restrict rclone to a specific folder
|
|
hierarchy.
|
|
.PP
|
|
In order to do this you will have to find the \f[C]Folder ID\f[R] of the
|
|
directory you wish rclone to display.
|
|
These aren\[aq]t displayed in the web interface, but you can use
|
|
\f[C]rclone lsf\f[R] to find them, for example
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
$ rclone lsf \-\-dirs\-only \-Fip \-\-csv filefabric:
|
|
120673758,Burnt PDFs/
|
|
120673759,My Quick Uploads/
|
|
120673755,My Syncs/
|
|
120673756,My backups/
|
|
120673757,My contacts/
|
|
120673761,S3 Storage/
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
The ID for \[dq]S3 Storage\[dq] would be \f[C]120673761\f[R].
|
|
.SS Standard Options
|
|
.PP
|
|
Here are the standard options specific to filefabric (Enterprise File
|
|
Fabric).
|
|
.SS \-\-filefabric\-url
|
|
.PP
|
|
URL of the Enterprise File Fabric to connect to
|
|
.IP \[bu] 2
|
|
Config: url
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_FILEFABRIC_URL
|
|
.IP \[bu] 2
|
|
Type: string
|
|
.IP \[bu] 2
|
|
Default: \[dq]\[dq]
|
|
.IP \[bu] 2
|
|
Examples:
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
\[dq]https://storagemadeeasy.com\[dq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Storage Made Easy US
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[dq]https://eu.storagemadeeasy.com\[dq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Storage Made Easy EU
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[dq]https://yourfabric.smestorage.com\[dq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Connect to your Enterprise File Fabric
|
|
.RE
|
|
.RE
|
|
.SS \-\-filefabric\-root\-folder\-id
|
|
.PP
|
|
ID of the root folder Leave blank normally.
|
|
.PP
|
|
Fill in to make rclone start with directory of a given ID.
|
|
.IP \[bu] 2
|
|
Config: root_folder_id
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_FILEFABRIC_ROOT_FOLDER_ID
|
|
.IP \[bu] 2
|
|
Type: string
|
|
.IP \[bu] 2
|
|
Default: \[dq]\[dq]
|
|
.SS \-\-filefabric\-permanent\-token
|
|
.PP
|
|
Permanent Authentication Token
|
|
.PP
|
|
A Permanent Authentication Token can be created in the Enterprise File
|
|
Fabric, on the users Dashboard under Security, there is an entry
|
|
you\[aq]ll see called \[dq]My Authentication Tokens\[dq].
|
|
Click the Manage button to create one.
|
|
.PP
|
|
These tokens are normally valid for several years.
|
|
.PP
|
|
For more info see:
|
|
https://docs.storagemadeeasy.com/organisationcloud/api\-tokens
|
|
.IP \[bu] 2
|
|
Config: permanent_token
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_FILEFABRIC_PERMANENT_TOKEN
|
|
.IP \[bu] 2
|
|
Type: string
|
|
.IP \[bu] 2
|
|
Default: \[dq]\[dq]
|
|
.SS Advanced Options
|
|
.PP
|
|
Here are the advanced options specific to filefabric (Enterprise File
|
|
Fabric).
|
|
.SS \-\-filefabric\-token
|
|
.PP
|
|
Session Token
|
|
.PP
|
|
This is a session token which rclone caches in the config file.
|
|
It is usually valid for 1 hour.
|
|
.PP
|
|
Don\[aq]t set this value \- rclone will set it automatically.
|
|
.IP \[bu] 2
|
|
Config: token
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_FILEFABRIC_TOKEN
|
|
.IP \[bu] 2
|
|
Type: string
|
|
.IP \[bu] 2
|
|
Default: \[dq]\[dq]
|
|
.SS \-\-filefabric\-token\-expiry
|
|
.PP
|
|
Token expiry time
|
|
.PP
|
|
Don\[aq]t set this value \- rclone will set it automatically.
|
|
.IP \[bu] 2
|
|
Config: token_expiry
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_FILEFABRIC_TOKEN_EXPIRY
|
|
.IP \[bu] 2
|
|
Type: string
|
|
.IP \[bu] 2
|
|
Default: \[dq]\[dq]
|
|
.SS \-\-filefabric\-version
|
|
.PP
|
|
Version read from the file fabric
|
|
.PP
|
|
Don\[aq]t set this value \- rclone will set it automatically.
|
|
.IP \[bu] 2
|
|
Config: version
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_FILEFABRIC_VERSION
|
|
.IP \[bu] 2
|
|
Type: string
|
|
.IP \[bu] 2
|
|
Default: \[dq]\[dq]
|
|
.SS \-\-filefabric\-encoding
|
|
.PP
|
|
This sets the encoding for the backend.
|
|
.PP
|
|
See: the encoding section in the
|
|
overview (https://rclone.org/overview/#encoding) for more info.
|
|
.IP \[bu] 2
|
|
Config: encoding
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_FILEFABRIC_ENCODING
|
|
.IP \[bu] 2
|
|
Type: MultiEncoder
|
|
.IP \[bu] 2
|
|
Default: Slash,Del,Ctl,InvalidUtf8,Dot
|
|
.SS FTP
|
|
.PP
|
|
FTP is the File Transfer Protocol.
|
|
Rclone FTP support is provided using the
|
|
github.com/jlaffaye/ftp (https://godoc.org/github.com/jlaffaye/ftp)
|
|
package.
|
|
.PP
|
|
Limitations of Rclone\[aq]s FTP backend
|
|
.PP
|
|
Paths are specified as \f[C]remote:path\f[R].
|
|
If the path does not begin with a \f[C]/\f[R] it is relative to the home
|
|
directory of the user.
|
|
An empty path \f[C]remote:\f[R] refers to the user\[aq]s home directory.
|
|
.PP
|
|
To create an FTP configuration named \f[C]remote\f[R], run
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone config
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
Rclone config guides you through an interactive setup process.
|
|
A minimal rclone FTP remote definition only requires host, username and
|
|
password.
|
|
For an anonymous FTP server, use \f[C]anonymous\f[R] as username and
|
|
your email address as password.
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
No remotes found \- make a new one
|
|
n) New remote
|
|
r) Rename remote
|
|
c) Copy remote
|
|
s) Set configuration password
|
|
q) Quit config
|
|
n/r/c/s/q> n
|
|
name> remote
|
|
Type of storage to configure.
|
|
Enter a string value. Press Enter for the default (\[dq]\[dq]).
|
|
Choose a number from below, or type in your own value
|
|
[snip]
|
|
XX / FTP Connection
|
|
\[rs] \[dq]ftp\[dq]
|
|
[snip]
|
|
Storage> ftp
|
|
** See help for ftp backend at: https://rclone.org/ftp/ **
|
|
|
|
FTP host to connect to
|
|
Enter a string value. Press Enter for the default (\[dq]\[dq]).
|
|
Choose a number from below, or type in your own value
|
|
1 / Connect to ftp.example.com
|
|
\[rs] \[dq]ftp.example.com\[dq]
|
|
host> ftp.example.com
|
|
FTP username, leave blank for current username, $USER
|
|
Enter a string value. Press Enter for the default (\[dq]\[dq]).
|
|
user>
|
|
FTP port, leave blank to use default (21)
|
|
Enter a string value. Press Enter for the default (\[dq]\[dq]).
|
|
port>
|
|
FTP password
|
|
y) Yes type in my own password
|
|
g) Generate random password
|
|
y/g> y
|
|
Enter the password:
|
|
password:
|
|
Confirm the password:
|
|
password:
|
|
Use FTP over TLS (Implicit)
|
|
Enter a boolean value (true or false). Press Enter for the default (\[dq]false\[dq]).
|
|
tls>
|
|
Use FTP over TLS (Explicit)
|
|
Enter a boolean value (true or false). Press Enter for the default (\[dq]false\[dq]).
|
|
explicit_tls>
|
|
Remote config
|
|
\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-
|
|
[remote]
|
|
type = ftp
|
|
host = ftp.example.com
|
|
pass = *** ENCRYPTED ***
|
|
\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-
|
|
y) Yes this is OK
|
|
e) Edit this remote
|
|
d) Delete this remote
|
|
y/e/d> y
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
To see all directories in the home directory of \f[C]remote\f[R]
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone lsd remote:
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
Make a new directory
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone mkdir remote:path/to/directory
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
List the contents of a directory
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone ls remote:path/to/directory
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
Sync \f[C]/home/local/directory\f[R] to the remote directory, deleting
|
|
any excess files in the directory.
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone sync \-i /home/local/directory remote:directory
|
|
\f[R]
|
|
.fi
|
|
.SS Example without a config file
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone lsf :ftp: \-\-ftp\-host=speedtest.tele2.net \-\-ftp\-user=anonymous \-\-ftp\-pass=\[ga]rclone obscure dummy\[ga]
|
|
\f[R]
|
|
.fi
|
|
.SS Implicit TLS
|
|
.PP
|
|
Rlone FTP supports implicit FTP over TLS servers (FTPS).
|
|
This has to be enabled in the FTP backend config for the remote, or with
|
|
\f[C]\-\-ftp\-tls\f[R].
|
|
The default FTPS port is \f[C]990\f[R], not \f[C]21\f[R] and can be set
|
|
with \f[C]\-\-ftp\-port\f[R].
|
|
.SS Standard Options
|
|
.PP
|
|
Here are the standard options specific to ftp (FTP Connection).
|
|
.SS \-\-ftp\-host
|
|
.PP
|
|
FTP host to connect to
|
|
.IP \[bu] 2
|
|
Config: host
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_FTP_HOST
|
|
.IP \[bu] 2
|
|
Type: string
|
|
.IP \[bu] 2
|
|
Default: \[dq]\[dq]
|
|
.IP \[bu] 2
|
|
Examples:
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
\[dq]ftp.example.com\[dq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Connect to ftp.example.com
|
|
.RE
|
|
.RE
|
|
.SS \-\-ftp\-user
|
|
.PP
|
|
FTP username, leave blank for current username, $USER
|
|
.IP \[bu] 2
|
|
Config: user
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_FTP_USER
|
|
.IP \[bu] 2
|
|
Type: string
|
|
.IP \[bu] 2
|
|
Default: \[dq]\[dq]
|
|
.SS \-\-ftp\-port
|
|
.PP
|
|
FTP port, leave blank to use default (21)
|
|
.IP \[bu] 2
|
|
Config: port
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_FTP_PORT
|
|
.IP \[bu] 2
|
|
Type: string
|
|
.IP \[bu] 2
|
|
Default: \[dq]\[dq]
|
|
.SS \-\-ftp\-pass
|
|
.PP
|
|
FTP password
|
|
.PP
|
|
\f[B]NB\f[R] Input to this must be obscured \- see rclone
|
|
obscure (https://rclone.org/commands/rclone_obscure/).
|
|
.IP \[bu] 2
|
|
Config: pass
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_FTP_PASS
|
|
.IP \[bu] 2
|
|
Type: string
|
|
.IP \[bu] 2
|
|
Default: \[dq]\[dq]
|
|
.SS \-\-ftp\-tls
|
|
.PP
|
|
Use Implicit FTPS (FTP over TLS) When using implicit FTP over TLS the
|
|
client connects using TLS right from the start which breaks
|
|
compatibility with non\-TLS\-aware servers.
|
|
This is usually served over port 990 rather than port 21.
|
|
Cannot be used in combination with explicit FTP.
|
|
.IP \[bu] 2
|
|
Config: tls
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_FTP_TLS
|
|
.IP \[bu] 2
|
|
Type: bool
|
|
.IP \[bu] 2
|
|
Default: false
|
|
.SS \-\-ftp\-explicit\-tls
|
|
.PP
|
|
Use Explicit FTPS (FTP over TLS) When using explicit FTP over TLS the
|
|
client explicitly requests security from the server in order to upgrade
|
|
a plain text connection to an encrypted one.
|
|
Cannot be used in combination with implicit FTP.
|
|
.IP \[bu] 2
|
|
Config: explicit_tls
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_FTP_EXPLICIT_TLS
|
|
.IP \[bu] 2
|
|
Type: bool
|
|
.IP \[bu] 2
|
|
Default: false
|
|
.SS Advanced Options
|
|
.PP
|
|
Here are the advanced options specific to ftp (FTP Connection).
|
|
.SS \-\-ftp\-concurrency
|
|
.PP
|
|
Maximum number of FTP simultaneous connections, 0 for unlimited
|
|
.IP \[bu] 2
|
|
Config: concurrency
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_FTP_CONCURRENCY
|
|
.IP \[bu] 2
|
|
Type: int
|
|
.IP \[bu] 2
|
|
Default: 0
|
|
.SS \-\-ftp\-no\-check\-certificate
|
|
.PP
|
|
Do not verify the TLS certificate of the server
|
|
.IP \[bu] 2
|
|
Config: no_check_certificate
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_FTP_NO_CHECK_CERTIFICATE
|
|
.IP \[bu] 2
|
|
Type: bool
|
|
.IP \[bu] 2
|
|
Default: false
|
|
.SS \-\-ftp\-disable\-epsv
|
|
.PP
|
|
Disable using EPSV even if server advertises support
|
|
.IP \[bu] 2
|
|
Config: disable_epsv
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_FTP_DISABLE_EPSV
|
|
.IP \[bu] 2
|
|
Type: bool
|
|
.IP \[bu] 2
|
|
Default: false
|
|
.SS \-\-ftp\-disable\-mlsd
|
|
.PP
|
|
Disable using MLSD even if server advertises support
|
|
.IP \[bu] 2
|
|
Config: disable_mlsd
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_FTP_DISABLE_MLSD
|
|
.IP \[bu] 2
|
|
Type: bool
|
|
.IP \[bu] 2
|
|
Default: false
|
|
.SS \-\-ftp\-idle\-timeout
|
|
.PP
|
|
Max time before closing idle connections
|
|
.PP
|
|
If no connections have been returned to the connection pool in the time
|
|
given, rclone will empty the connection pool.
|
|
.PP
|
|
Set to 0 to keep connections indefinitely.
|
|
.IP \[bu] 2
|
|
Config: idle_timeout
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_FTP_IDLE_TIMEOUT
|
|
.IP \[bu] 2
|
|
Type: Duration
|
|
.IP \[bu] 2
|
|
Default: 1m0s
|
|
.SS \-\-ftp\-close\-timeout
|
|
.PP
|
|
Maximum time to wait for a response to close.
|
|
.IP \[bu] 2
|
|
Config: close_timeout
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_FTP_CLOSE_TIMEOUT
|
|
.IP \[bu] 2
|
|
Type: Duration
|
|
.IP \[bu] 2
|
|
Default: 1m0s
|
|
.SS \-\-ftp\-encoding
|
|
.PP
|
|
This sets the encoding for the backend.
|
|
.PP
|
|
See: the encoding section in the
|
|
overview (https://rclone.org/overview/#encoding) for more info.
|
|
.IP \[bu] 2
|
|
Config: encoding
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_FTP_ENCODING
|
|
.IP \[bu] 2
|
|
Type: MultiEncoder
|
|
.IP \[bu] 2
|
|
Default: Slash,Del,Ctl,RightSpace,Dot
|
|
.SS Limitations
|
|
.PP
|
|
Modified times are not supported.
|
|
Times you see on the FTP server through rclone are those of upload.
|
|
.PP
|
|
Rclone\[aq]s FTP backend does not support any checksums but can compare
|
|
file sizes.
|
|
.PP
|
|
\f[C]rclone about\f[R] is not supported by the FTP backend.
|
|
Backends without this capability cannot determine free space for an
|
|
rclone mount or use policy \f[C]mfs\f[R] (most free space) as a member
|
|
of an rclone union remote.
|
|
.PP
|
|
See List of backends that do not support rclone
|
|
about (https://rclone.org/overview/#optional-features) See rclone
|
|
about (https://rclone.org/commands/rclone_about/)
|
|
.PP
|
|
The implementation of : \f[C]\-\-dump headers\f[R],
|
|
\f[C]\-\-dump bodies\f[R], \f[C]\-\-dump auth\f[R] for debugging
|
|
isn\[aq]t the same as for rclone HTTP based backends \- it has less fine
|
|
grained control.
|
|
.PP
|
|
\f[C]\-\-timeout\f[R] isn\[aq]t supported (but \f[C]\-\-contimeout\f[R]
|
|
is).
|
|
.PP
|
|
\f[C]\-\-bind\f[R] isn\[aq]t supported.
|
|
.PP
|
|
Rclone\[aq]s FTP backend could support server\-side move but does not at
|
|
present.
|
|
.PP
|
|
The \f[C]ftp_proxy\f[R] environment variable is not currently supported.
|
|
.PP
|
|
FTP servers acting as rclone remotes must support \[aq]passive\[aq]
|
|
mode.
|
|
Rclone\[aq]s FTP implementation is not compatible with \[aq]active\[aq]
|
|
mode.
|
|
.SS Restricted filename characters
|
|
.PP
|
|
In addition to the default restricted characters
|
|
set (https://rclone.org/overview/#restricted-characters) the following
|
|
characters are also replaced:
|
|
.PP
|
|
File names cannot end with the following characters.
|
|
Repacement is limited to the last character in a file name:
|
|
.PP
|
|
.TS
|
|
tab(@);
|
|
l c c.
|
|
T{
|
|
Character
|
|
T}@T{
|
|
Value
|
|
T}@T{
|
|
Replacement
|
|
T}
|
|
_
|
|
T{
|
|
SP
|
|
T}@T{
|
|
0x20
|
|
T}@T{
|
|
\[u2420]
|
|
T}
|
|
.TE
|
|
.PP
|
|
Not all FTP servers can have all characters in file names, for example:
|
|
.PP
|
|
.TS
|
|
tab(@);
|
|
l c.
|
|
T{
|
|
FTP Server
|
|
T}@T{
|
|
Forbidden characters
|
|
T}
|
|
_
|
|
T{
|
|
proftpd
|
|
T}@T{
|
|
\f[C]*\f[R]
|
|
T}
|
|
T{
|
|
pureftpd
|
|
T}@T{
|
|
\f[C]\[rs] [ ]\f[R]
|
|
T}
|
|
.TE
|
|
.SS Google Cloud Storage
|
|
.PP
|
|
Paths are specified as \f[C]remote:bucket\f[R] (or \f[C]remote:\f[R] for
|
|
the \f[C]lsd\f[R] command.) You may put subdirectories in too, e.g.
|
|
\f[C]remote:bucket/path/to/dir\f[R].
|
|
.PP
|
|
The initial setup for google cloud storage involves getting a token from
|
|
Google Cloud Storage which you need to do in your browser.
|
|
\f[C]rclone config\f[R] walks you through it.
|
|
.PP
|
|
Here is an example of how to make a remote called \f[C]remote\f[R].
|
|
First run:
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone config
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
This will guide you through an interactive setup process:
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
n) New remote
|
|
d) Delete remote
|
|
q) Quit config
|
|
e/n/d/q> n
|
|
name> remote
|
|
Type of storage to configure.
|
|
Choose a number from below, or type in your own value
|
|
[snip]
|
|
XX / Google Cloud Storage (this is not Google Drive)
|
|
\[rs] \[dq]google cloud storage\[dq]
|
|
[snip]
|
|
Storage> google cloud storage
|
|
Google Application Client Id \- leave blank normally.
|
|
client_id>
|
|
Google Application Client Secret \- leave blank normally.
|
|
client_secret>
|
|
Project number optional \- needed only for list/create/delete buckets \- see your developer console.
|
|
project_number> 12345678
|
|
Service Account Credentials JSON file path \- needed only if you want use SA instead of interactive login.
|
|
service_account_file>
|
|
Access Control List for new objects.
|
|
Choose a number from below, or type in your own value
|
|
1 / Object owner gets OWNER access, and all Authenticated Users get READER access.
|
|
\[rs] \[dq]authenticatedRead\[dq]
|
|
2 / Object owner gets OWNER access, and project team owners get OWNER access.
|
|
\[rs] \[dq]bucketOwnerFullControl\[dq]
|
|
3 / Object owner gets OWNER access, and project team owners get READER access.
|
|
\[rs] \[dq]bucketOwnerRead\[dq]
|
|
4 / Object owner gets OWNER access [default if left blank].
|
|
\[rs] \[dq]private\[dq]
|
|
5 / Object owner gets OWNER access, and project team members get access according to their roles.
|
|
\[rs] \[dq]projectPrivate\[dq]
|
|
6 / Object owner gets OWNER access, and all Users get READER access.
|
|
\[rs] \[dq]publicRead\[dq]
|
|
object_acl> 4
|
|
Access Control List for new buckets.
|
|
Choose a number from below, or type in your own value
|
|
1 / Project team owners get OWNER access, and all Authenticated Users get READER access.
|
|
\[rs] \[dq]authenticatedRead\[dq]
|
|
2 / Project team owners get OWNER access [default if left blank].
|
|
\[rs] \[dq]private\[dq]
|
|
3 / Project team members get access according to their roles.
|
|
\[rs] \[dq]projectPrivate\[dq]
|
|
4 / Project team owners get OWNER access, and all Users get READER access.
|
|
\[rs] \[dq]publicRead\[dq]
|
|
5 / Project team owners get OWNER access, and all Users get WRITER access.
|
|
\[rs] \[dq]publicReadWrite\[dq]
|
|
bucket_acl> 2
|
|
Location for the newly created buckets.
|
|
Choose a number from below, or type in your own value
|
|
1 / Empty for default location (US).
|
|
\[rs] \[dq]\[dq]
|
|
2 / Multi\-regional location for Asia.
|
|
\[rs] \[dq]asia\[dq]
|
|
3 / Multi\-regional location for Europe.
|
|
\[rs] \[dq]eu\[dq]
|
|
4 / Multi\-regional location for United States.
|
|
\[rs] \[dq]us\[dq]
|
|
5 / Taiwan.
|
|
\[rs] \[dq]asia\-east1\[dq]
|
|
6 / Tokyo.
|
|
\[rs] \[dq]asia\-northeast1\[dq]
|
|
7 / Singapore.
|
|
\[rs] \[dq]asia\-southeast1\[dq]
|
|
8 / Sydney.
|
|
\[rs] \[dq]australia\-southeast1\[dq]
|
|
9 / Belgium.
|
|
\[rs] \[dq]europe\-west1\[dq]
|
|
10 / London.
|
|
\[rs] \[dq]europe\-west2\[dq]
|
|
11 / Iowa.
|
|
\[rs] \[dq]us\-central1\[dq]
|
|
12 / South Carolina.
|
|
\[rs] \[dq]us\-east1\[dq]
|
|
13 / Northern Virginia.
|
|
\[rs] \[dq]us\-east4\[dq]
|
|
14 / Oregon.
|
|
\[rs] \[dq]us\-west1\[dq]
|
|
location> 12
|
|
The storage class to use when storing objects in Google Cloud Storage.
|
|
Choose a number from below, or type in your own value
|
|
1 / Default
|
|
\[rs] \[dq]\[dq]
|
|
2 / Multi\-regional storage class
|
|
\[rs] \[dq]MULTI_REGIONAL\[dq]
|
|
3 / Regional storage class
|
|
\[rs] \[dq]REGIONAL\[dq]
|
|
4 / Nearline storage class
|
|
\[rs] \[dq]NEARLINE\[dq]
|
|
5 / Coldline storage class
|
|
\[rs] \[dq]COLDLINE\[dq]
|
|
6 / Durable reduced availability storage class
|
|
\[rs] \[dq]DURABLE_REDUCED_AVAILABILITY\[dq]
|
|
storage_class> 5
|
|
Remote config
|
|
Use auto config?
|
|
* Say Y if not sure
|
|
* Say N if you are working on a remote or headless machine or Y didn\[aq]t work
|
|
y) Yes
|
|
n) No
|
|
y/n> y
|
|
If your browser doesn\[aq]t open automatically go to the following link: http://127.0.0.1:53682/auth
|
|
Log in and authorize rclone for access
|
|
Waiting for code...
|
|
Got code
|
|
\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-
|
|
[remote]
|
|
type = google cloud storage
|
|
client_id =
|
|
client_secret =
|
|
token = {\[dq]AccessToken\[dq]:\[dq]xxxx.xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx\-xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx\[dq],\[dq]RefreshToken\[dq]:\[dq]x/xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx_xxxxxxxxx\[dq],\[dq]Expiry\[dq]:\[dq]2014\-07\-17T20:49:14.929208288+01:00\[dq],\[dq]Extra\[dq]:null}
|
|
project_number = 12345678
|
|
object_acl = private
|
|
bucket_acl = private
|
|
\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-
|
|
y) Yes this is OK
|
|
e) Edit this remote
|
|
d) Delete this remote
|
|
y/e/d> y
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
Note that rclone runs a webserver on your local machine to collect the
|
|
token as returned from Google if you use auto config mode.
|
|
This only runs from the moment it opens your browser to the moment you
|
|
get back the verification code.
|
|
This is on \f[C]http://127.0.0.1:53682/\f[R] and this it may require you
|
|
to unblock it temporarily if you are running a host firewall, or use
|
|
manual mode.
|
|
.PP
|
|
This remote is called \f[C]remote\f[R] and can now be used like this
|
|
.PP
|
|
See all the buckets in your project
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone lsd remote:
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
Make a new bucket
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone mkdir remote:bucket
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
List the contents of a bucket
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone ls remote:bucket
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
Sync \f[C]/home/local/directory\f[R] to the remote bucket, deleting any
|
|
excess files in the bucket.
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone sync \-i /home/local/directory remote:bucket
|
|
\f[R]
|
|
.fi
|
|
.SS Service Account support
|
|
.PP
|
|
You can set up rclone with Google Cloud Storage in an unattended mode,
|
|
i.e.
|
|
not tied to a specific end\-user Google account.
|
|
This is useful when you want to synchronise files onto machines that
|
|
don\[aq]t have actively logged\-in users, for example build machines.
|
|
.PP
|
|
To get credentials for Google Cloud Platform IAM Service
|
|
Accounts (https://cloud.google.com/iam/docs/service-accounts), please
|
|
head to the Service
|
|
Account (https://console.cloud.google.com/permissions/serviceaccounts)
|
|
section of the Google Developer Console.
|
|
Service Accounts behave just like normal \f[C]User\f[R] permissions in
|
|
Google Cloud Storage
|
|
ACLs (https://cloud.google.com/storage/docs/access-control), so you can
|
|
limit their access (e.g.
|
|
make them read only).
|
|
After creating an account, a JSON file containing the Service
|
|
Account\[aq]s credentials will be downloaded onto your machines.
|
|
These credentials are what rclone will use for authentication.
|
|
.PP
|
|
To use a Service Account instead of OAuth2 token flow, enter the path to
|
|
your Service Account credentials at the \f[C]service_account_file\f[R]
|
|
prompt and rclone won\[aq]t use the browser based authentication flow.
|
|
If you\[aq]d rather stuff the contents of the credentials file into the
|
|
rclone config file, you can set \f[C]service_account_credentials\f[R]
|
|
with the actual contents of the file instead, or set the equivalent
|
|
environment variable.
|
|
.SS Anonymous Access
|
|
.PP
|
|
For downloads of objects that permit public access you can configure
|
|
rclone to use anonymous access by setting \f[C]anonymous\f[R] to
|
|
\f[C]true\f[R].
|
|
With unauthorized access you can\[aq]t write or create files but only
|
|
read or list those buckets and objects that have public read access.
|
|
.SS Application Default Credentials
|
|
.PP
|
|
If no other source of credentials is provided, rclone will fall back to
|
|
Application Default
|
|
Credentials (https://cloud.google.com/video-intelligence/docs/common/auth#authenticating_with_application_default_credentials)
|
|
this is useful both when you already have configured authentication for
|
|
your developer account, or in production when running on a google
|
|
compute host.
|
|
Note that if running in docker, you may need to run additional commands
|
|
on your google compute machine \- see this
|
|
page (https://cloud.google.com/container-registry/docs/advanced-authentication#gcloud_as_a_docker_credential_helper).
|
|
.PP
|
|
Note that in the case application default credentials are used, there is
|
|
no need to explicitly configure a project number.
|
|
.SS \-\-fast\-list
|
|
.PP
|
|
This remote supports \f[C]\-\-fast\-list\f[R] which allows you to use
|
|
fewer transactions in exchange for more memory.
|
|
See the rclone docs (https://rclone.org/docs/#fast-list) for more
|
|
details.
|
|
.SS Custom upload headers
|
|
.PP
|
|
You can set custom upload headers with the \f[C]\-\-header\-upload\f[R]
|
|
flag.
|
|
Google Cloud Storage supports the headers as described in the working
|
|
with metadata
|
|
documentation (https://cloud.google.com/storage/docs/gsutil/addlhelp/WorkingWithObjectMetadata)
|
|
.IP \[bu] 2
|
|
Cache\-Control
|
|
.IP \[bu] 2
|
|
Content\-Disposition
|
|
.IP \[bu] 2
|
|
Content\-Encoding
|
|
.IP \[bu] 2
|
|
Content\-Language
|
|
.IP \[bu] 2
|
|
Content\-Type
|
|
.IP \[bu] 2
|
|
X\-Goog\-Storage\-Class
|
|
.IP \[bu] 2
|
|
X\-Goog\-Meta\-
|
|
.PP
|
|
Eg \f[C]\-\-header\-upload \[dq]Content\-Type text/potato\[dq]\f[R]
|
|
.PP
|
|
Note that the last of these is for setting custom metadata in the form
|
|
\f[C]\-\-header\-upload \[dq]x\-goog\-meta\-key: value\[dq]\f[R]
|
|
.SS Modified time
|
|
.PP
|
|
Google google cloud storage stores md5sums natively and rclone stores
|
|
modification times as metadata on the object, under the \[dq]mtime\[dq]
|
|
key in RFC3339 format accurate to 1ns.
|
|
.SS Restricted filename characters
|
|
.PP
|
|
.TS
|
|
tab(@);
|
|
l c c.
|
|
T{
|
|
Character
|
|
T}@T{
|
|
Value
|
|
T}@T{
|
|
Replacement
|
|
T}
|
|
_
|
|
T{
|
|
NUL
|
|
T}@T{
|
|
0x00
|
|
T}@T{
|
|
\[u2400]
|
|
T}
|
|
T{
|
|
LF
|
|
T}@T{
|
|
0x0A
|
|
T}@T{
|
|
\[u240A]
|
|
T}
|
|
T{
|
|
CR
|
|
T}@T{
|
|
0x0D
|
|
T}@T{
|
|
\[u240D]
|
|
T}
|
|
T{
|
|
/
|
|
T}@T{
|
|
0x2F
|
|
T}@T{
|
|
\[uFF0F]
|
|
T}
|
|
.TE
|
|
.PP
|
|
Invalid UTF\-8 bytes will also be
|
|
replaced (https://rclone.org/overview/#invalid-utf8), as they can\[aq]t
|
|
be used in JSON strings.
|
|
.SS Standard Options
|
|
.PP
|
|
Here are the standard options specific to google cloud storage (Google
|
|
Cloud Storage (this is not Google Drive)).
|
|
.SS \-\-gcs\-client\-id
|
|
.PP
|
|
OAuth Client Id Leave blank normally.
|
|
.IP \[bu] 2
|
|
Config: client_id
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_GCS_CLIENT_ID
|
|
.IP \[bu] 2
|
|
Type: string
|
|
.IP \[bu] 2
|
|
Default: \[dq]\[dq]
|
|
.SS \-\-gcs\-client\-secret
|
|
.PP
|
|
OAuth Client Secret Leave blank normally.
|
|
.IP \[bu] 2
|
|
Config: client_secret
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_GCS_CLIENT_SECRET
|
|
.IP \[bu] 2
|
|
Type: string
|
|
.IP \[bu] 2
|
|
Default: \[dq]\[dq]
|
|
.SS \-\-gcs\-project\-number
|
|
.PP
|
|
Project number.
|
|
Optional \- needed only for list/create/delete buckets \- see your
|
|
developer console.
|
|
.IP \[bu] 2
|
|
Config: project_number
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_GCS_PROJECT_NUMBER
|
|
.IP \[bu] 2
|
|
Type: string
|
|
.IP \[bu] 2
|
|
Default: \[dq]\[dq]
|
|
.SS \-\-gcs\-service\-account\-file
|
|
.PP
|
|
Service Account Credentials JSON file path Leave blank normally.
|
|
Needed only if you want use SA instead of interactive login.
|
|
.PP
|
|
Leading \f[C]\[ti]\f[R] will be expanded in the file name as will
|
|
environment variables such as \f[C]${RCLONE_CONFIG_DIR}\f[R].
|
|
.IP \[bu] 2
|
|
Config: service_account_file
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_GCS_SERVICE_ACCOUNT_FILE
|
|
.IP \[bu] 2
|
|
Type: string
|
|
.IP \[bu] 2
|
|
Default: \[dq]\[dq]
|
|
.SS \-\-gcs\-service\-account\-credentials
|
|
.PP
|
|
Service Account Credentials JSON blob Leave blank normally.
|
|
Needed only if you want use SA instead of interactive login.
|
|
.IP \[bu] 2
|
|
Config: service_account_credentials
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_GCS_SERVICE_ACCOUNT_CREDENTIALS
|
|
.IP \[bu] 2
|
|
Type: string
|
|
.IP \[bu] 2
|
|
Default: \[dq]\[dq]
|
|
.SS \-\-gcs\-anonymous
|
|
.PP
|
|
Access public buckets and objects without credentials Set to
|
|
\[aq]true\[aq] if you just want to download files and don\[aq]t
|
|
configure credentials.
|
|
.IP \[bu] 2
|
|
Config: anonymous
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_GCS_ANONYMOUS
|
|
.IP \[bu] 2
|
|
Type: bool
|
|
.IP \[bu] 2
|
|
Default: false
|
|
.SS \-\-gcs\-object\-acl
|
|
.PP
|
|
Access Control List for new objects.
|
|
.IP \[bu] 2
|
|
Config: object_acl
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_GCS_OBJECT_ACL
|
|
.IP \[bu] 2
|
|
Type: string
|
|
.IP \[bu] 2
|
|
Default: \[dq]\[dq]
|
|
.IP \[bu] 2
|
|
Examples:
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
\[dq]authenticatedRead\[dq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Object owner gets OWNER access, and all Authenticated Users get READER
|
|
access.
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[dq]bucketOwnerFullControl\[dq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Object owner gets OWNER access, and project team owners get OWNER
|
|
access.
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[dq]bucketOwnerRead\[dq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Object owner gets OWNER access, and project team owners get READER
|
|
access.
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[dq]private\[dq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Object owner gets OWNER access [default if left blank].
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[dq]projectPrivate\[dq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Object owner gets OWNER access, and project team members get access
|
|
according to their roles.
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[dq]publicRead\[dq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Object owner gets OWNER access, and all Users get READER access.
|
|
.RE
|
|
.RE
|
|
.SS \-\-gcs\-bucket\-acl
|
|
.PP
|
|
Access Control List for new buckets.
|
|
.IP \[bu] 2
|
|
Config: bucket_acl
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_GCS_BUCKET_ACL
|
|
.IP \[bu] 2
|
|
Type: string
|
|
.IP \[bu] 2
|
|
Default: \[dq]\[dq]
|
|
.IP \[bu] 2
|
|
Examples:
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
\[dq]authenticatedRead\[dq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Project team owners get OWNER access, and all Authenticated Users get
|
|
READER access.
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[dq]private\[dq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Project team owners get OWNER access [default if left blank].
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[dq]projectPrivate\[dq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Project team members get access according to their roles.
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[dq]publicRead\[dq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Project team owners get OWNER access, and all Users get READER access.
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[dq]publicReadWrite\[dq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Project team owners get OWNER access, and all Users get WRITER access.
|
|
.RE
|
|
.RE
|
|
.SS \-\-gcs\-bucket\-policy\-only
|
|
.PP
|
|
Access checks should use bucket\-level IAM policies.
|
|
.PP
|
|
If you want to upload objects to a bucket with Bucket Policy Only set
|
|
then you will need to set this.
|
|
.PP
|
|
When it is set, rclone:
|
|
.IP \[bu] 2
|
|
ignores ACLs set on buckets
|
|
.IP \[bu] 2
|
|
ignores ACLs set on objects
|
|
.IP \[bu] 2
|
|
creates buckets with Bucket Policy Only set
|
|
.PP
|
|
Docs: https://cloud.google.com/storage/docs/bucket\-policy\-only
|
|
.IP \[bu] 2
|
|
Config: bucket_policy_only
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_GCS_BUCKET_POLICY_ONLY
|
|
.IP \[bu] 2
|
|
Type: bool
|
|
.IP \[bu] 2
|
|
Default: false
|
|
.SS \-\-gcs\-location
|
|
.PP
|
|
Location for the newly created buckets.
|
|
.IP \[bu] 2
|
|
Config: location
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_GCS_LOCATION
|
|
.IP \[bu] 2
|
|
Type: string
|
|
.IP \[bu] 2
|
|
Default: \[dq]\[dq]
|
|
.IP \[bu] 2
|
|
Examples:
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
\[dq]\[dq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Empty for default location (US).
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[dq]asia\[dq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Multi\-regional location for Asia.
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[dq]eu\[dq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Multi\-regional location for Europe.
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[dq]us\[dq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Multi\-regional location for United States.
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[dq]asia\-east1\[dq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Taiwan.
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[dq]asia\-east2\[dq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Hong Kong.
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[dq]asia\-northeast1\[dq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Tokyo.
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[dq]asia\-south1\[dq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Mumbai.
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[dq]asia\-southeast1\[dq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Singapore.
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[dq]australia\-southeast1\[dq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Sydney.
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[dq]europe\-north1\[dq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Finland.
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[dq]europe\-west1\[dq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Belgium.
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[dq]europe\-west2\[dq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
London.
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[dq]europe\-west3\[dq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Frankfurt.
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[dq]europe\-west4\[dq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Netherlands.
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[dq]us\-central1\[dq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Iowa.
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[dq]us\-east1\[dq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
South Carolina.
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[dq]us\-east4\[dq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Northern Virginia.
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[dq]us\-west1\[dq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Oregon.
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[dq]us\-west2\[dq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
California.
|
|
.RE
|
|
.RE
|
|
.SS \-\-gcs\-storage\-class
|
|
.PP
|
|
The storage class to use when storing objects in Google Cloud Storage.
|
|
.IP \[bu] 2
|
|
Config: storage_class
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_GCS_STORAGE_CLASS
|
|
.IP \[bu] 2
|
|
Type: string
|
|
.IP \[bu] 2
|
|
Default: \[dq]\[dq]
|
|
.IP \[bu] 2
|
|
Examples:
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
\[dq]\[dq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Default
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[dq]MULTI_REGIONAL\[dq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Multi\-regional storage class
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[dq]REGIONAL\[dq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Regional storage class
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[dq]NEARLINE\[dq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Nearline storage class
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[dq]COLDLINE\[dq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Coldline storage class
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[dq]ARCHIVE\[dq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Archive storage class
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[dq]DURABLE_REDUCED_AVAILABILITY\[dq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Durable reduced availability storage class
|
|
.RE
|
|
.RE
|
|
.SS Advanced Options
|
|
.PP
|
|
Here are the advanced options specific to google cloud storage (Google
|
|
Cloud Storage (this is not Google Drive)).
|
|
.SS \-\-gcs\-token
|
|
.PP
|
|
OAuth Access Token as a JSON blob.
|
|
.IP \[bu] 2
|
|
Config: token
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_GCS_TOKEN
|
|
.IP \[bu] 2
|
|
Type: string
|
|
.IP \[bu] 2
|
|
Default: \[dq]\[dq]
|
|
.SS \-\-gcs\-auth\-url
|
|
.PP
|
|
Auth server URL.
|
|
Leave blank to use the provider defaults.
|
|
.IP \[bu] 2
|
|
Config: auth_url
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_GCS_AUTH_URL
|
|
.IP \[bu] 2
|
|
Type: string
|
|
.IP \[bu] 2
|
|
Default: \[dq]\[dq]
|
|
.SS \-\-gcs\-token\-url
|
|
.PP
|
|
Token server url.
|
|
Leave blank to use the provider defaults.
|
|
.IP \[bu] 2
|
|
Config: token_url
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_GCS_TOKEN_URL
|
|
.IP \[bu] 2
|
|
Type: string
|
|
.IP \[bu] 2
|
|
Default: \[dq]\[dq]
|
|
.SS \-\-gcs\-encoding
|
|
.PP
|
|
This sets the encoding for the backend.
|
|
.PP
|
|
See: the encoding section in the
|
|
overview (https://rclone.org/overview/#encoding) for more info.
|
|
.IP \[bu] 2
|
|
Config: encoding
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_GCS_ENCODING
|
|
.IP \[bu] 2
|
|
Type: MultiEncoder
|
|
.IP \[bu] 2
|
|
Default: Slash,CrLf,InvalidUtf8,Dot
|
|
.SS Limitations
|
|
.PP
|
|
\f[C]rclone about\f[R] is not supported by the Google Cloud Storage
|
|
backend.
|
|
Backends without this capability cannot determine free space for an
|
|
rclone mount or use policy \f[C]mfs\f[R] (most free space) as a member
|
|
of an rclone union remote.
|
|
.PP
|
|
See List of backends that do not support rclone
|
|
about (https://rclone.org/overview/#optional-features) See rclone
|
|
about (https://rclone.org/commands/rclone_about/)
|
|
.SS Google Drive
|
|
.PP
|
|
Paths are specified as \f[C]drive:path\f[R]
|
|
.PP
|
|
Drive paths may be as deep as required, e.g.
|
|
\f[C]drive:directory/subdirectory\f[R].
|
|
.PP
|
|
The initial setup for drive involves getting a token from Google drive
|
|
which you need to do in your browser.
|
|
\f[C]rclone config\f[R] walks you through it.
|
|
.PP
|
|
Here is an example of how to make a remote called \f[C]remote\f[R].
|
|
First run:
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone config
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
This will guide you through an interactive setup process:
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
No remotes found \- make a new one
|
|
n) New remote
|
|
r) Rename remote
|
|
c) Copy remote
|
|
s) Set configuration password
|
|
q) Quit config
|
|
n/r/c/s/q> n
|
|
name> remote
|
|
Type of storage to configure.
|
|
Choose a number from below, or type in your own value
|
|
[snip]
|
|
XX / Google Drive
|
|
\[rs] \[dq]drive\[dq]
|
|
[snip]
|
|
Storage> drive
|
|
Google Application Client Id \- leave blank normally.
|
|
client_id>
|
|
Google Application Client Secret \- leave blank normally.
|
|
client_secret>
|
|
Scope that rclone should use when requesting access from drive.
|
|
Choose a number from below, or type in your own value
|
|
1 / Full access all files, excluding Application Data Folder.
|
|
\[rs] \[dq]drive\[dq]
|
|
2 / Read\-only access to file metadata and file contents.
|
|
\[rs] \[dq]drive.readonly\[dq]
|
|
/ Access to files created by rclone only.
|
|
3 | These are visible in the drive website.
|
|
| File authorization is revoked when the user deauthorizes the app.
|
|
\[rs] \[dq]drive.file\[dq]
|
|
/ Allows read and write access to the Application Data folder.
|
|
4 | This is not visible in the drive website.
|
|
\[rs] \[dq]drive.appfolder\[dq]
|
|
/ Allows read\-only access to file metadata but
|
|
5 | does not allow any access to read or download file content.
|
|
\[rs] \[dq]drive.metadata.readonly\[dq]
|
|
scope> 1
|
|
ID of the root folder \- leave blank normally. Fill in to access \[dq]Computers\[dq] folders. (see docs).
|
|
root_folder_id>
|
|
Service Account Credentials JSON file path \- needed only if you want use SA instead of interactive login.
|
|
service_account_file>
|
|
Remote config
|
|
Use auto config?
|
|
* Say Y if not sure
|
|
* Say N if you are working on a remote or headless machine or Y didn\[aq]t work
|
|
y) Yes
|
|
n) No
|
|
y/n> y
|
|
If your browser doesn\[aq]t open automatically go to the following link: http://127.0.0.1:53682/auth
|
|
Log in and authorize rclone for access
|
|
Waiting for code...
|
|
Got code
|
|
Configure this as a Shared Drive (Team Drive)?
|
|
y) Yes
|
|
n) No
|
|
y/n> n
|
|
\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-
|
|
[remote]
|
|
client_id =
|
|
client_secret =
|
|
scope = drive
|
|
root_folder_id =
|
|
service_account_file =
|
|
token = {\[dq]access_token\[dq]:\[dq]XXX\[dq],\[dq]token_type\[dq]:\[dq]Bearer\[dq],\[dq]refresh_token\[dq]:\[dq]XXX\[dq],\[dq]expiry\[dq]:\[dq]2014\-03\-16T13:57:58.955387075Z\[dq]}
|
|
\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-
|
|
y) Yes this is OK
|
|
e) Edit this remote
|
|
d) Delete this remote
|
|
y/e/d> y
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
Note that rclone runs a webserver on your local machine to collect the
|
|
token as returned from Google if you use auto config mode.
|
|
This only runs from the moment it opens your browser to the moment you
|
|
get back the verification code.
|
|
This is on \f[C]http://127.0.0.1:53682/\f[R] and this it may require you
|
|
to unblock it temporarily if you are running a host firewall, or use
|
|
manual mode.
|
|
.PP
|
|
You can then use it like this,
|
|
.PP
|
|
List directories in top level of your drive
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone lsd remote:
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
List all the files in your drive
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone ls remote:
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
To copy a local directory to a drive directory called backup
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone copy /home/source remote:backup
|
|
\f[R]
|
|
.fi
|
|
.SS Scopes
|
|
.PP
|
|
Rclone allows you to select which scope you would like for rclone to
|
|
use.
|
|
This changes what type of token is granted to rclone.
|
|
The scopes are defined
|
|
here (https://developers.google.com/drive/v3/web/about-auth).
|
|
.PP
|
|
The scope are
|
|
.SS drive
|
|
.PP
|
|
This is the default scope and allows full access to all files, except
|
|
for the Application Data Folder (see below).
|
|
.PP
|
|
Choose this one if you aren\[aq]t sure.
|
|
.SS drive.readonly
|
|
.PP
|
|
This allows read only access to all files.
|
|
Files may be listed and downloaded but not uploaded, renamed or deleted.
|
|
.SS drive.file
|
|
.PP
|
|
With this scope rclone can read/view/modify only those files and folders
|
|
it creates.
|
|
.PP
|
|
So if you uploaded files to drive via the web interface (or any other
|
|
means) they will not be visible to rclone.
|
|
.PP
|
|
This can be useful if you are using rclone to backup data and you want
|
|
to be sure confidential data on your drive is not visible to rclone.
|
|
.PP
|
|
Files created with this scope are visible in the web interface.
|
|
.SS drive.appfolder
|
|
.PP
|
|
This gives rclone its own private area to store files.
|
|
Rclone will not be able to see any other files on your drive and you
|
|
won\[aq]t be able to see rclone\[aq]s files from the web interface
|
|
either.
|
|
.SS drive.metadata.readonly
|
|
.PP
|
|
This allows read only access to file names only.
|
|
It does not allow rclone to download or upload data, or rename or delete
|
|
files or directories.
|
|
.SS Root folder ID
|
|
.PP
|
|
You can set the \f[C]root_folder_id\f[R] for rclone.
|
|
This is the directory (identified by its \f[C]Folder ID\f[R]) that
|
|
rclone considers to be the root of your drive.
|
|
.PP
|
|
Normally you will leave this blank and rclone will determine the correct
|
|
root to use itself.
|
|
.PP
|
|
However you can set this to restrict rclone to a specific folder
|
|
hierarchy or to access data within the \[dq]Computers\[dq] tab on the
|
|
drive web interface (where files from Google\[aq]s Backup and Sync
|
|
desktop program go).
|
|
.PP
|
|
In order to do this you will have to find the \f[C]Folder ID\f[R] of the
|
|
directory you wish rclone to display.
|
|
This will be the last segment of the URL when you open the relevant
|
|
folder in the drive web interface.
|
|
.PP
|
|
So if the folder you want rclone to use has a URL which looks like
|
|
\f[C]https://drive.google.com/drive/folders/1XyfxxxxxxxxxxxxxxxxxxxxxxxxxKHCh\f[R]
|
|
in the browser, then you use \f[C]1XyfxxxxxxxxxxxxxxxxxxxxxxxxxKHCh\f[R]
|
|
as the \f[C]root_folder_id\f[R] in the config.
|
|
.PP
|
|
\f[B]NB\f[R] folders under the \[dq]Computers\[dq] tab seem to be read
|
|
only (drive gives a 500 error) when using rclone.
|
|
.PP
|
|
There doesn\[aq]t appear to be an API to discover the folder IDs of the
|
|
\[dq]Computers\[dq] tab \- please contact us if you know otherwise!
|
|
.PP
|
|
Note also that rclone can\[aq]t access any data under the
|
|
\[dq]Backups\[dq] tab on the google drive web interface yet.
|
|
.SS Service Account support
|
|
.PP
|
|
You can set up rclone with Google Drive in an unattended mode, i.e.
|
|
not tied to a specific end\-user Google account.
|
|
This is useful when you want to synchronise files onto machines that
|
|
don\[aq]t have actively logged\-in users, for example build machines.
|
|
.PP
|
|
To use a Service Account instead of OAuth2 token flow, enter the path to
|
|
your Service Account credentials at the \f[C]service_account_file\f[R]
|
|
prompt during \f[C]rclone config\f[R] and rclone won\[aq]t use the
|
|
browser based authentication flow.
|
|
If you\[aq]d rather stuff the contents of the credentials file into the
|
|
rclone config file, you can set \f[C]service_account_credentials\f[R]
|
|
with the actual contents of the file instead, or set the equivalent
|
|
environment variable.
|
|
.SS Use case \- Google Apps/G\-suite account and individual Drive
|
|
.PP
|
|
Let\[aq]s say that you are the administrator of a Google Apps (old) or
|
|
G\-suite account.
|
|
The goal is to store data on an individual\[aq]s Drive account, who IS a
|
|
member of the domain.
|
|
We\[aq]ll call the domain \f[B]example.com\f[R], and the user
|
|
\f[B]foo\[at]example.com\f[R].
|
|
.PP
|
|
There\[aq]s a few steps we need to go through to accomplish this:
|
|
.SS 1. Create a service account for example.com
|
|
.IP \[bu] 2
|
|
To create a service account and obtain its credentials, go to the Google
|
|
Developer Console (https://console.developers.google.com).
|
|
.IP \[bu] 2
|
|
You must have a project \- create one if you don\[aq]t.
|
|
.IP \[bu] 2
|
|
Then go to \[dq]IAM & admin\[dq] \-> \[dq]Service Accounts\[dq].
|
|
.IP \[bu] 2
|
|
Use the \[dq]Create Credentials\[dq] button.
|
|
Fill in \[dq]Service account name\[dq] with something that identifies
|
|
your client.
|
|
\[dq]Role\[dq] can be empty.
|
|
.IP \[bu] 2
|
|
Tick \[dq]Furnish a new private key\[dq] \- select \[dq]Key type
|
|
JSON\[dq].
|
|
.IP \[bu] 2
|
|
Tick \[dq]Enable G Suite Domain\-wide Delegation\[dq].
|
|
This option makes \[dq]impersonation\[dq] possible, as documented here:
|
|
Delegating domain\-wide authority to the service
|
|
account (https://developers.google.com/identity/protocols/OAuth2ServiceAccount#delegatingauthority)
|
|
.IP \[bu] 2
|
|
These credentials are what rclone will use for authentication.
|
|
If you ever need to remove access, press the \[dq]Delete service account
|
|
key\[dq] button.
|
|
.SS 2. Allowing API access to example.com Google Drive
|
|
.IP \[bu] 2
|
|
Go to example.com\[aq]s admin console
|
|
.IP \[bu] 2
|
|
Go into \[dq]Security\[dq] (or use the search bar)
|
|
.IP \[bu] 2
|
|
Select \[dq]Show more\[dq] and then \[dq]Advanced settings\[dq]
|
|
.IP \[bu] 2
|
|
Select \[dq]Manage API client access\[dq] in the
|
|
\[dq]Authentication\[dq] section
|
|
.IP \[bu] 2
|
|
In the \[dq]Client Name\[dq] field enter the service account\[aq]s
|
|
\[dq]Client ID\[dq] \- this can be found in the Developer Console under
|
|
\[dq]IAM & Admin\[dq] \-> \[dq]Service Accounts\[dq], then \[dq]View
|
|
Client ID\[dq] for the newly created service account.
|
|
It is a \[ti]21 character numerical string.
|
|
.IP \[bu] 2
|
|
In the next field, \[dq]One or More API Scopes\[dq], enter
|
|
\f[C]https://www.googleapis.com/auth/drive\f[R] to grant access to
|
|
Google Drive specifically.
|
|
.SS 3. Configure rclone, assuming a new install
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone config
|
|
|
|
n/s/q> n # New
|
|
name>gdrive # Gdrive is an example name
|
|
Storage> # Select the number shown for Google Drive
|
|
client_id> # Can be left blank
|
|
client_secret> # Can be left blank
|
|
scope> # Select your scope, 1 for example
|
|
root_folder_id> # Can be left blank
|
|
service_account_file> /home/foo/myJSONfile.json # This is where the JSON file goes!
|
|
y/n> # Auto config, y
|
|
\f[R]
|
|
.fi
|
|
.SS 4. Verify that it\[aq]s working
|
|
.IP \[bu] 2
|
|
\f[C]rclone \-v \-\-drive\-impersonate foo\[at]example.com lsf gdrive:backup\f[R]
|
|
.IP \[bu] 2
|
|
The arguments do:
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
\f[C]\-v\f[R] \- verbose logging
|
|
.IP \[bu] 2
|
|
\f[C]\-\-drive\-impersonate foo\[at]example.com\f[R] \- this is what
|
|
does the magic, pretending to be user foo.
|
|
.IP \[bu] 2
|
|
\f[C]lsf\f[R] \- list files in a parsing friendly way
|
|
.IP \[bu] 2
|
|
\f[C]gdrive:backup\f[R] \- use the remote called gdrive, work in the
|
|
folder named backup.
|
|
.RE
|
|
.PP
|
|
Note: in case you configured a specific root folder on gdrive and rclone
|
|
is unable to access the contents of that folder when using
|
|
\f[C]\-\-drive\-impersonate\f[R], do this instead: \- in the gdrive web
|
|
interface, share your root folder with the user/email of the new Service
|
|
Account you created/selected at step #1 \- use rclone without specifying
|
|
the \f[C]\-\-drive\-impersonate\f[R] option, like this:
|
|
\f[C]rclone \-v foo\[at]example.com lsf gdrive:backup\f[R]
|
|
.SS Shared drives (team drives)
|
|
.PP
|
|
If you want to configure the remote to point to a Google Shared Drive
|
|
(previously known as Team Drives) then answer \f[C]y\f[R] to the
|
|
question \f[C]Configure this as a Shared Drive (Team Drive)?\f[R].
|
|
.PP
|
|
This will fetch the list of Shared Drives from google and allow you to
|
|
configure which one you want to use.
|
|
You can also type in a Shared Drive ID if you prefer.
|
|
.PP
|
|
For example:
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
Configure this as a Shared Drive (Team Drive)?
|
|
y) Yes
|
|
n) No
|
|
y/n> y
|
|
Fetching Shared Drive list...
|
|
Choose a number from below, or type in your own value
|
|
1 / Rclone Test
|
|
\[rs] \[dq]xxxxxxxxxxxxxxxxxxxx\[dq]
|
|
2 / Rclone Test 2
|
|
\[rs] \[dq]yyyyyyyyyyyyyyyyyyyy\[dq]
|
|
3 / Rclone Test 3
|
|
\[rs] \[dq]zzzzzzzzzzzzzzzzzzzz\[dq]
|
|
Enter a Shared Drive ID> 1
|
|
\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-
|
|
[remote]
|
|
client_id =
|
|
client_secret =
|
|
token = {\[dq]AccessToken\[dq]:\[dq]xxxx.x.xxxxx_xxxxxxxxxxx_xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx\[dq],\[dq]RefreshToken\[dq]:\[dq]1/xxxxxxxxxxxxxxxx_xxxxxxxxxxxxxxxxxxxxxxxxxx\[dq],\[dq]Expiry\[dq]:\[dq]2014\-03\-16T13:57:58.955387075Z\[dq],\[dq]Extra\[dq]:null}
|
|
team_drive = xxxxxxxxxxxxxxxxxxxx
|
|
\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-
|
|
y) Yes this is OK
|
|
e) Edit this remote
|
|
d) Delete this remote
|
|
y/e/d> y
|
|
\f[R]
|
|
.fi
|
|
.SS \-\-fast\-list
|
|
.PP
|
|
This remote supports \f[C]\-\-fast\-list\f[R] which allows you to use
|
|
fewer transactions in exchange for more memory.
|
|
See the rclone docs (https://rclone.org/docs/#fast-list) for more
|
|
details.
|
|
.PP
|
|
It does this by combining multiple \f[C]list\f[R] calls into a single
|
|
API request.
|
|
.PP
|
|
This works by combining many \f[C]\[aq]%s\[aq] in parents\f[R] filters
|
|
into one expression.
|
|
To list the contents of directories a, b and c, the following requests
|
|
will be send by the regular \f[C]List\f[R] function:
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
trashed=false and \[aq]a\[aq] in parents
|
|
trashed=false and \[aq]b\[aq] in parents
|
|
trashed=false and \[aq]c\[aq] in parents
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
These can now be combined into a single request:
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
trashed=false and (\[aq]a\[aq] in parents or \[aq]b\[aq] in parents or \[aq]c\[aq] in parents)
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
The implementation of \f[C]ListR\f[R] will put up to 50
|
|
\f[C]parents\f[R] filters into one request.
|
|
It will use the \f[C]\-\-checkers\f[R] value to specify the number of
|
|
requests to run in parallel.
|
|
.PP
|
|
In tests, these batch requests were up to 20x faster than the regular
|
|
method.
|
|
Running the following command against different sized folders gives:
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone lsjson \-vv \-R \-\-checkers=6 gdrive:folder
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
small folder (220 directories, 700 files):
|
|
.IP \[bu] 2
|
|
without \f[C]\-\-fast\-list\f[R]: 38s
|
|
.IP \[bu] 2
|
|
with \f[C]\-\-fast\-list\f[R]: 10s
|
|
.PP
|
|
large folder (10600 directories, 39000 files):
|
|
.IP \[bu] 2
|
|
without \f[C]\-\-fast\-list\f[R]: 22:05 min
|
|
.IP \[bu] 2
|
|
with \f[C]\-\-fast\-list\f[R]: 58s
|
|
.SS Modified time
|
|
.PP
|
|
Google drive stores modification times accurate to 1 ms.
|
|
.SS Restricted filename characters
|
|
.PP
|
|
Only Invalid UTF\-8 bytes will be
|
|
replaced (https://rclone.org/overview/#invalid-utf8), as they can\[aq]t
|
|
be used in JSON strings.
|
|
.PP
|
|
In contrast to other backends, \f[C]/\f[R] can also be used in names and
|
|
\f[C].\f[R] or \f[C]..\f[R] are valid names.
|
|
.SS Revisions
|
|
.PP
|
|
Google drive stores revisions of files.
|
|
When you upload a change to an existing file to google drive using
|
|
rclone it will create a new revision of that file.
|
|
.PP
|
|
Revisions follow the standard google policy which at time of writing was
|
|
.IP \[bu] 2
|
|
They are deleted after 30 days or 100 revisions (whatever comes first).
|
|
.IP \[bu] 2
|
|
They do not count towards a user storage quota.
|
|
.SS Deleting files
|
|
.PP
|
|
By default rclone will send all files to the trash when deleting files.
|
|
If deleting them permanently is required then use the
|
|
\f[C]\-\-drive\-use\-trash=false\f[R] flag, or set the equivalent
|
|
environment variable.
|
|
.SS Shortcuts
|
|
.PP
|
|
In March 2020 Google introduced a new feature in Google Drive called
|
|
drive shortcuts (https://support.google.com/drive/answer/9700156)
|
|
(API (https://developers.google.com/drive/api/v3/shortcuts)).
|
|
These will (by September 2020) replace the ability for files or folders
|
|
to be in multiple folders at
|
|
once (https://cloud.google.com/blog/products/g-suite/simplifying-google-drives-folder-structure-and-sharing-models).
|
|
.PP
|
|
Shortcuts are files that link to other files on Google Drive somewhat
|
|
like a symlink in unix, except they point to the underlying file data
|
|
(e.g.
|
|
the inode in unix terms) so they don\[aq]t break if the source is
|
|
renamed or moved about.
|
|
.PP
|
|
Be default rclone treats these as follows.
|
|
.PP
|
|
For shortcuts pointing to files:
|
|
.IP \[bu] 2
|
|
When listing a file shortcut appears as the destination file.
|
|
.IP \[bu] 2
|
|
When downloading the contents of the destination file is downloaded.
|
|
.IP \[bu] 2
|
|
When updating shortcut file with a non shortcut file, the shortcut is
|
|
removed then a new file is uploaded in place of the shortcut.
|
|
.IP \[bu] 2
|
|
When server\-side moving (renaming) the shortcut is renamed, not the
|
|
destination file.
|
|
.IP \[bu] 2
|
|
When server\-side copying the shortcut is copied, not the contents of
|
|
the shortcut.
|
|
.IP \[bu] 2
|
|
When deleting the shortcut is deleted not the linked file.
|
|
.IP \[bu] 2
|
|
When setting the modification time, the modification time of the linked
|
|
file will be set.
|
|
.PP
|
|
For shortcuts pointing to folders:
|
|
.IP \[bu] 2
|
|
When listing the shortcut appears as a folder and that folder will
|
|
contain the contents of the linked folder appear (including any sub
|
|
folders)
|
|
.IP \[bu] 2
|
|
When downloading the contents of the linked folder and sub contents are
|
|
downloaded
|
|
.IP \[bu] 2
|
|
When uploading to a shortcut folder the file will be placed in the
|
|
linked folder
|
|
.IP \[bu] 2
|
|
When server\-side moving (renaming) the shortcut is renamed, not the
|
|
destination folder
|
|
.IP \[bu] 2
|
|
When server\-side copying the contents of the linked folder is copied,
|
|
not the shortcut.
|
|
.IP \[bu] 2
|
|
When deleting with \f[C]rclone rmdir\f[R] or \f[C]rclone purge\f[R] the
|
|
shortcut is deleted not the linked folder.
|
|
.IP \[bu] 2
|
|
\f[B]NB\f[R] When deleting with \f[C]rclone remove\f[R] or
|
|
\f[C]rclone mount\f[R] the contents of the linked folder will be
|
|
deleted.
|
|
.PP
|
|
The rclone backend (https://rclone.org/commands/rclone_backend/) command
|
|
can be used to create shortcuts.
|
|
.PP
|
|
Shortcuts can be completely ignored with the
|
|
\f[C]\-\-drive\-skip\-shortcuts\f[R] flag or the corresponding
|
|
\f[C]skip_shortcuts\f[R] configuration setting.
|
|
.SS Emptying trash
|
|
.PP
|
|
If you wish to empty your trash you can use the
|
|
\f[C]rclone cleanup remote:\f[R] command which will permanently delete
|
|
all your trashed files.
|
|
This command does not take any path arguments.
|
|
.PP
|
|
Note that Google Drive takes some time (minutes to days) to empty the
|
|
trash even though the command returns within a few seconds.
|
|
No output is echoed, so there will be no confirmation even using \-v or
|
|
\-vv.
|
|
.SS Quota information
|
|
.PP
|
|
To view your current quota you can use the
|
|
\f[C]rclone about remote:\f[R] command which will display your usage
|
|
limit (quota), the usage in Google Drive, the size of all files in the
|
|
Trash and the space used by other Google services such as Gmail.
|
|
This command does not take any path arguments.
|
|
.SS Import/Export of google documents
|
|
.PP
|
|
Google documents can be exported from and uploaded to Google Drive.
|
|
.PP
|
|
When rclone downloads a Google doc it chooses a format to download
|
|
depending upon the \f[C]\-\-drive\-export\-formats\f[R] setting.
|
|
By default the export formats are \f[C]docx,xlsx,pptx,svg\f[R] which are
|
|
a sensible default for an editable document.
|
|
.PP
|
|
When choosing a format, rclone runs down the list provided in order and
|
|
chooses the first file format the doc can be exported as from the list.
|
|
If the file can\[aq]t be exported to a format on the formats list, then
|
|
rclone will choose a format from the default list.
|
|
.PP
|
|
If you prefer an archive copy then you might use
|
|
\f[C]\-\-drive\-export\-formats pdf\f[R], or if you prefer
|
|
openoffice/libreoffice formats you might use
|
|
\f[C]\-\-drive\-export\-formats ods,odt,odp\f[R].
|
|
.PP
|
|
Note that rclone adds the extension to the google doc, so if it is
|
|
called \f[C]My Spreadsheet\f[R] on google docs, it will be exported as
|
|
\f[C]My Spreadsheet.xlsx\f[R] or \f[C]My Spreadsheet.pdf\f[R] etc.
|
|
.PP
|
|
When importing files into Google Drive, rclone will convert all files
|
|
with an extension in \f[C]\-\-drive\-import\-formats\f[R] to their
|
|
associated document type.
|
|
rclone will not convert any files by default, since the conversion is
|
|
lossy process.
|
|
.PP
|
|
The conversion must result in a file with the same extension when the
|
|
\f[C]\-\-drive\-export\-formats\f[R] rules are applied to the uploaded
|
|
document.
|
|
.PP
|
|
Here are some examples for allowed and prohibited conversions.
|
|
.PP
|
|
.TS
|
|
tab(@);
|
|
l l l l l.
|
|
T{
|
|
export\-formats
|
|
T}@T{
|
|
import\-formats
|
|
T}@T{
|
|
Upload Ext
|
|
T}@T{
|
|
Document Ext
|
|
T}@T{
|
|
Allowed
|
|
T}
|
|
_
|
|
T{
|
|
odt
|
|
T}@T{
|
|
odt
|
|
T}@T{
|
|
odt
|
|
T}@T{
|
|
odt
|
|
T}@T{
|
|
Yes
|
|
T}
|
|
T{
|
|
odt
|
|
T}@T{
|
|
docx,odt
|
|
T}@T{
|
|
odt
|
|
T}@T{
|
|
odt
|
|
T}@T{
|
|
Yes
|
|
T}
|
|
T{
|
|
T}@T{
|
|
docx
|
|
T}@T{
|
|
docx
|
|
T}@T{
|
|
docx
|
|
T}@T{
|
|
Yes
|
|
T}
|
|
T{
|
|
T}@T{
|
|
odt
|
|
T}@T{
|
|
odt
|
|
T}@T{
|
|
docx
|
|
T}@T{
|
|
No
|
|
T}
|
|
T{
|
|
odt,docx
|
|
T}@T{
|
|
docx,odt
|
|
T}@T{
|
|
docx
|
|
T}@T{
|
|
odt
|
|
T}@T{
|
|
No
|
|
T}
|
|
T{
|
|
docx,odt
|
|
T}@T{
|
|
docx,odt
|
|
T}@T{
|
|
docx
|
|
T}@T{
|
|
docx
|
|
T}@T{
|
|
Yes
|
|
T}
|
|
T{
|
|
docx,odt
|
|
T}@T{
|
|
docx,odt
|
|
T}@T{
|
|
odt
|
|
T}@T{
|
|
docx
|
|
T}@T{
|
|
No
|
|
T}
|
|
.TE
|
|
.PP
|
|
This limitation can be disabled by specifying
|
|
\f[C]\-\-drive\-allow\-import\-name\-change\f[R].
|
|
When using this flag, rclone can convert multiple files types resulting
|
|
in the same document type at once, e.g.
|
|
with \f[C]\-\-drive\-import\-formats docx,odt,txt\f[R], all files having
|
|
these extension would result in a document represented as a docx file.
|
|
This brings the additional risk of overwriting a document, if multiple
|
|
files have the same stem.
|
|
Many rclone operations will not handle this name change in any way.
|
|
They assume an equal name when copying files and might copy the file
|
|
again or delete them when the name changes.
|
|
.PP
|
|
Here are the possible export extensions with their corresponding mime
|
|
types.
|
|
Most of these can also be used for importing, but there more that are
|
|
not listed here.
|
|
Some of these additional ones might only be available when the operating
|
|
system provides the correct MIME type entries.
|
|
.PP
|
|
This list can be changed by Google Drive at any time and might not
|
|
represent the currently available conversions.
|
|
.PP
|
|
.TS
|
|
tab(@);
|
|
lw(19.7n) lw(24.1n) lw(26.2n).
|
|
T{
|
|
Extension
|
|
T}@T{
|
|
Mime Type
|
|
T}@T{
|
|
Description
|
|
T}
|
|
_
|
|
T{
|
|
csv
|
|
T}@T{
|
|
text/csv
|
|
T}@T{
|
|
Standard CSV format for Spreadsheets
|
|
T}
|
|
T{
|
|
docx
|
|
T}@T{
|
|
application/vnd.openxmlformats\-officedocument.wordprocessingml.document
|
|
T}@T{
|
|
Microsoft Office Document
|
|
T}
|
|
T{
|
|
epub
|
|
T}@T{
|
|
application/epub+zip
|
|
T}@T{
|
|
E\-book format
|
|
T}
|
|
T{
|
|
html
|
|
T}@T{
|
|
text/html
|
|
T}@T{
|
|
An HTML Document
|
|
T}
|
|
T{
|
|
jpg
|
|
T}@T{
|
|
image/jpeg
|
|
T}@T{
|
|
A JPEG Image File
|
|
T}
|
|
T{
|
|
json
|
|
T}@T{
|
|
application/vnd.google\-apps.script+json
|
|
T}@T{
|
|
JSON Text Format
|
|
T}
|
|
T{
|
|
odp
|
|
T}@T{
|
|
application/vnd.oasis.opendocument.presentation
|
|
T}@T{
|
|
Openoffice Presentation
|
|
T}
|
|
T{
|
|
ods
|
|
T}@T{
|
|
application/vnd.oasis.opendocument.spreadsheet
|
|
T}@T{
|
|
Openoffice Spreadsheet
|
|
T}
|
|
T{
|
|
ods
|
|
T}@T{
|
|
application/x\-vnd.oasis.opendocument.spreadsheet
|
|
T}@T{
|
|
Openoffice Spreadsheet
|
|
T}
|
|
T{
|
|
odt
|
|
T}@T{
|
|
application/vnd.oasis.opendocument.text
|
|
T}@T{
|
|
Openoffice Document
|
|
T}
|
|
T{
|
|
pdf
|
|
T}@T{
|
|
application/pdf
|
|
T}@T{
|
|
Adobe PDF Format
|
|
T}
|
|
T{
|
|
png
|
|
T}@T{
|
|
image/png
|
|
T}@T{
|
|
PNG Image Format
|
|
T}
|
|
T{
|
|
pptx
|
|
T}@T{
|
|
application/vnd.openxmlformats\-officedocument.presentationml.presentation
|
|
T}@T{
|
|
Microsoft Office Powerpoint
|
|
T}
|
|
T{
|
|
rtf
|
|
T}@T{
|
|
application/rtf
|
|
T}@T{
|
|
Rich Text Format
|
|
T}
|
|
T{
|
|
svg
|
|
T}@T{
|
|
image/svg+xml
|
|
T}@T{
|
|
Scalable Vector Graphics Format
|
|
T}
|
|
T{
|
|
tsv
|
|
T}@T{
|
|
text/tab\-separated\-values
|
|
T}@T{
|
|
Standard TSV format for spreadsheets
|
|
T}
|
|
T{
|
|
txt
|
|
T}@T{
|
|
text/plain
|
|
T}@T{
|
|
Plain Text
|
|
T}
|
|
T{
|
|
xlsx
|
|
T}@T{
|
|
application/vnd.openxmlformats\-officedocument.spreadsheetml.sheet
|
|
T}@T{
|
|
Microsoft Office Spreadsheet
|
|
T}
|
|
T{
|
|
zip
|
|
T}@T{
|
|
application/zip
|
|
T}@T{
|
|
A ZIP file of HTML, Images CSS
|
|
T}
|
|
.TE
|
|
.PP
|
|
Google documents can also be exported as link files.
|
|
These files will open a browser window for the Google Docs website of
|
|
that document when opened.
|
|
The link file extension has to be specified as a
|
|
\f[C]\-\-drive\-export\-formats\f[R] parameter.
|
|
They will match all available Google Documents.
|
|
.PP
|
|
.TS
|
|
tab(@);
|
|
l l l.
|
|
T{
|
|
Extension
|
|
T}@T{
|
|
Description
|
|
T}@T{
|
|
OS Support
|
|
T}
|
|
_
|
|
T{
|
|
desktop
|
|
T}@T{
|
|
freedesktop.org specified desktop entry
|
|
T}@T{
|
|
Linux
|
|
T}
|
|
T{
|
|
link.html
|
|
T}@T{
|
|
An HTML Document with a redirect
|
|
T}@T{
|
|
All
|
|
T}
|
|
T{
|
|
url
|
|
T}@T{
|
|
INI style link file
|
|
T}@T{
|
|
macOS, Windows
|
|
T}
|
|
T{
|
|
webloc
|
|
T}@T{
|
|
macOS specific XML format
|
|
T}@T{
|
|
macOS
|
|
T}
|
|
.TE
|
|
.SS Standard Options
|
|
.PP
|
|
Here are the standard options specific to drive (Google Drive).
|
|
.SS \-\-drive\-client\-id
|
|
.PP
|
|
Google Application Client Id Setting your own is recommended.
|
|
See https://rclone.org/drive/#making\-your\-own\-client\-id for how to
|
|
create your own.
|
|
If you leave this blank, it will use an internal key which is low
|
|
performance.
|
|
.IP \[bu] 2
|
|
Config: client_id
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_DRIVE_CLIENT_ID
|
|
.IP \[bu] 2
|
|
Type: string
|
|
.IP \[bu] 2
|
|
Default: \[dq]\[dq]
|
|
.SS \-\-drive\-client\-secret
|
|
.PP
|
|
OAuth Client Secret Leave blank normally.
|
|
.IP \[bu] 2
|
|
Config: client_secret
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_DRIVE_CLIENT_SECRET
|
|
.IP \[bu] 2
|
|
Type: string
|
|
.IP \[bu] 2
|
|
Default: \[dq]\[dq]
|
|
.SS \-\-drive\-scope
|
|
.PP
|
|
Scope that rclone should use when requesting access from drive.
|
|
.IP \[bu] 2
|
|
Config: scope
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_DRIVE_SCOPE
|
|
.IP \[bu] 2
|
|
Type: string
|
|
.IP \[bu] 2
|
|
Default: \[dq]\[dq]
|
|
.IP \[bu] 2
|
|
Examples:
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
\[dq]drive\[dq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Full access all files, excluding Application Data Folder.
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[dq]drive.readonly\[dq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Read\-only access to file metadata and file contents.
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[dq]drive.file\[dq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Access to files created by rclone only.
|
|
.IP \[bu] 2
|
|
These are visible in the drive website.
|
|
.IP \[bu] 2
|
|
File authorization is revoked when the user deauthorizes the app.
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[dq]drive.appfolder\[dq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Allows read and write access to the Application Data folder.
|
|
.IP \[bu] 2
|
|
This is not visible in the drive website.
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[dq]drive.metadata.readonly\[dq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Allows read\-only access to file metadata but
|
|
.IP \[bu] 2
|
|
does not allow any access to read or download file content.
|
|
.RE
|
|
.RE
|
|
.SS \-\-drive\-root\-folder\-id
|
|
.PP
|
|
ID of the root folder Leave blank normally.
|
|
.PP
|
|
Fill in to access \[dq]Computers\[dq] folders (see docs), or for rclone
|
|
to use a non root folder as its starting point.
|
|
.IP \[bu] 2
|
|
Config: root_folder_id
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_DRIVE_ROOT_FOLDER_ID
|
|
.IP \[bu] 2
|
|
Type: string
|
|
.IP \[bu] 2
|
|
Default: \[dq]\[dq]
|
|
.SS \-\-drive\-service\-account\-file
|
|
.PP
|
|
Service Account Credentials JSON file path Leave blank normally.
|
|
Needed only if you want use SA instead of interactive login.
|
|
.PP
|
|
Leading \f[C]\[ti]\f[R] will be expanded in the file name as will
|
|
environment variables such as \f[C]${RCLONE_CONFIG_DIR}\f[R].
|
|
.IP \[bu] 2
|
|
Config: service_account_file
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_DRIVE_SERVICE_ACCOUNT_FILE
|
|
.IP \[bu] 2
|
|
Type: string
|
|
.IP \[bu] 2
|
|
Default: \[dq]\[dq]
|
|
.SS \-\-drive\-alternate\-export
|
|
.PP
|
|
Deprecated: no longer needed
|
|
.IP \[bu] 2
|
|
Config: alternate_export
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_DRIVE_ALTERNATE_EXPORT
|
|
.IP \[bu] 2
|
|
Type: bool
|
|
.IP \[bu] 2
|
|
Default: false
|
|
.SS Advanced Options
|
|
.PP
|
|
Here are the advanced options specific to drive (Google Drive).
|
|
.SS \-\-drive\-token
|
|
.PP
|
|
OAuth Access Token as a JSON blob.
|
|
.IP \[bu] 2
|
|
Config: token
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_DRIVE_TOKEN
|
|
.IP \[bu] 2
|
|
Type: string
|
|
.IP \[bu] 2
|
|
Default: \[dq]\[dq]
|
|
.SS \-\-drive\-auth\-url
|
|
.PP
|
|
Auth server URL.
|
|
Leave blank to use the provider defaults.
|
|
.IP \[bu] 2
|
|
Config: auth_url
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_DRIVE_AUTH_URL
|
|
.IP \[bu] 2
|
|
Type: string
|
|
.IP \[bu] 2
|
|
Default: \[dq]\[dq]
|
|
.SS \-\-drive\-token\-url
|
|
.PP
|
|
Token server url.
|
|
Leave blank to use the provider defaults.
|
|
.IP \[bu] 2
|
|
Config: token_url
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_DRIVE_TOKEN_URL
|
|
.IP \[bu] 2
|
|
Type: string
|
|
.IP \[bu] 2
|
|
Default: \[dq]\[dq]
|
|
.SS \-\-drive\-service\-account\-credentials
|
|
.PP
|
|
Service Account Credentials JSON blob Leave blank normally.
|
|
Needed only if you want use SA instead of interactive login.
|
|
.IP \[bu] 2
|
|
Config: service_account_credentials
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_DRIVE_SERVICE_ACCOUNT_CREDENTIALS
|
|
.IP \[bu] 2
|
|
Type: string
|
|
.IP \[bu] 2
|
|
Default: \[dq]\[dq]
|
|
.SS \-\-drive\-team\-drive
|
|
.PP
|
|
ID of the Shared Drive (Team Drive)
|
|
.IP \[bu] 2
|
|
Config: team_drive
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_DRIVE_TEAM_DRIVE
|
|
.IP \[bu] 2
|
|
Type: string
|
|
.IP \[bu] 2
|
|
Default: \[dq]\[dq]
|
|
.SS \-\-drive\-auth\-owner\-only
|
|
.PP
|
|
Only consider files owned by the authenticated user.
|
|
.IP \[bu] 2
|
|
Config: auth_owner_only
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_DRIVE_AUTH_OWNER_ONLY
|
|
.IP \[bu] 2
|
|
Type: bool
|
|
.IP \[bu] 2
|
|
Default: false
|
|
.SS \-\-drive\-use\-trash
|
|
.PP
|
|
Send files to the trash instead of deleting permanently.
|
|
Defaults to true, namely sending files to the trash.
|
|
Use \f[C]\-\-drive\-use\-trash=false\f[R] to delete files permanently
|
|
instead.
|
|
.IP \[bu] 2
|
|
Config: use_trash
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_DRIVE_USE_TRASH
|
|
.IP \[bu] 2
|
|
Type: bool
|
|
.IP \[bu] 2
|
|
Default: true
|
|
.SS \-\-drive\-skip\-gdocs
|
|
.PP
|
|
Skip google documents in all listings.
|
|
If given, gdocs practically become invisible to rclone.
|
|
.IP \[bu] 2
|
|
Config: skip_gdocs
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_DRIVE_SKIP_GDOCS
|
|
.IP \[bu] 2
|
|
Type: bool
|
|
.IP \[bu] 2
|
|
Default: false
|
|
.SS \-\-drive\-skip\-checksum\-gphotos
|
|
.PP
|
|
Skip MD5 checksum on Google photos and videos only.
|
|
.PP
|
|
Use this if you get checksum errors when transferring Google photos or
|
|
videos.
|
|
.PP
|
|
Setting this flag will cause Google photos and videos to return a blank
|
|
MD5 checksum.
|
|
.PP
|
|
Google photos are identified by being in the \[dq]photos\[dq] space.
|
|
.PP
|
|
Corrupted checksums are caused by Google modifying the image/video but
|
|
not updating the checksum.
|
|
.IP \[bu] 2
|
|
Config: skip_checksum_gphotos
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_DRIVE_SKIP_CHECKSUM_GPHOTOS
|
|
.IP \[bu] 2
|
|
Type: bool
|
|
.IP \[bu] 2
|
|
Default: false
|
|
.SS \-\-drive\-shared\-with\-me
|
|
.PP
|
|
Only show files that are shared with me.
|
|
.PP
|
|
Instructs rclone to operate on your \[dq]Shared with me\[dq] folder
|
|
(where Google Drive lets you access the files and folders others have
|
|
shared with you).
|
|
.PP
|
|
This works both with the \[dq]list\[dq] (lsd, lsl, etc.) and the
|
|
\[dq]copy\[dq] commands (copy, sync, etc.), and with all other commands
|
|
too.
|
|
.IP \[bu] 2
|
|
Config: shared_with_me
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_DRIVE_SHARED_WITH_ME
|
|
.IP \[bu] 2
|
|
Type: bool
|
|
.IP \[bu] 2
|
|
Default: false
|
|
.SS \-\-drive\-trashed\-only
|
|
.PP
|
|
Only show files that are in the trash.
|
|
This will show trashed files in their original directory structure.
|
|
.IP \[bu] 2
|
|
Config: trashed_only
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_DRIVE_TRASHED_ONLY
|
|
.IP \[bu] 2
|
|
Type: bool
|
|
.IP \[bu] 2
|
|
Default: false
|
|
.SS \-\-drive\-starred\-only
|
|
.PP
|
|
Only show files that are starred.
|
|
.IP \[bu] 2
|
|
Config: starred_only
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_DRIVE_STARRED_ONLY
|
|
.IP \[bu] 2
|
|
Type: bool
|
|
.IP \[bu] 2
|
|
Default: false
|
|
.SS \-\-drive\-formats
|
|
.PP
|
|
Deprecated: see export_formats
|
|
.IP \[bu] 2
|
|
Config: formats
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_DRIVE_FORMATS
|
|
.IP \[bu] 2
|
|
Type: string
|
|
.IP \[bu] 2
|
|
Default: \[dq]\[dq]
|
|
.SS \-\-drive\-export\-formats
|
|
.PP
|
|
Comma separated list of preferred formats for downloading Google docs.
|
|
.IP \[bu] 2
|
|
Config: export_formats
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_DRIVE_EXPORT_FORMATS
|
|
.IP \[bu] 2
|
|
Type: string
|
|
.IP \[bu] 2
|
|
Default: \[dq]docx,xlsx,pptx,svg\[dq]
|
|
.SS \-\-drive\-import\-formats
|
|
.PP
|
|
Comma separated list of preferred formats for uploading Google docs.
|
|
.IP \[bu] 2
|
|
Config: import_formats
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_DRIVE_IMPORT_FORMATS
|
|
.IP \[bu] 2
|
|
Type: string
|
|
.IP \[bu] 2
|
|
Default: \[dq]\[dq]
|
|
.SS \-\-drive\-allow\-import\-name\-change
|
|
.PP
|
|
Allow the filetype to change when uploading Google docs (e.g.
|
|
file.doc to file.docx).
|
|
This will confuse sync and reupload every time.
|
|
.IP \[bu] 2
|
|
Config: allow_import_name_change
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_DRIVE_ALLOW_IMPORT_NAME_CHANGE
|
|
.IP \[bu] 2
|
|
Type: bool
|
|
.IP \[bu] 2
|
|
Default: false
|
|
.SS \-\-drive\-use\-created\-date
|
|
.PP
|
|
Use file created date instead of modified date.,
|
|
.PP
|
|
Useful when downloading data and you want the creation date used in
|
|
place of the last modified date.
|
|
.PP
|
|
\f[B]WARNING\f[R]: This flag may have some unexpected consequences.
|
|
.PP
|
|
When uploading to your drive all files will be overwritten unless they
|
|
haven\[aq]t been modified since their creation.
|
|
And the inverse will occur while downloading.
|
|
This side effect can be avoided by using the \[dq]\-\-checksum\[dq]
|
|
flag.
|
|
.PP
|
|
This feature was implemented to retain photos capture date as recorded
|
|
by google photos.
|
|
You will first need to check the \[dq]Create a Google Photos folder\[dq]
|
|
option in your google drive settings.
|
|
You can then copy or move the photos locally and use the date the image
|
|
was taken (created) set as the modification date.
|
|
.IP \[bu] 2
|
|
Config: use_created_date
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_DRIVE_USE_CREATED_DATE
|
|
.IP \[bu] 2
|
|
Type: bool
|
|
.IP \[bu] 2
|
|
Default: false
|
|
.SS \-\-drive\-use\-shared\-date
|
|
.PP
|
|
Use date file was shared instead of modified date.
|
|
.PP
|
|
Note that, as with \[dq]\-\-drive\-use\-created\-date\[dq], this flag
|
|
may have unexpected consequences when uploading/downloading files.
|
|
.PP
|
|
If both this flag and \[dq]\-\-drive\-use\-created\-date\[dq] are set,
|
|
the created date is used.
|
|
.IP \[bu] 2
|
|
Config: use_shared_date
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_DRIVE_USE_SHARED_DATE
|
|
.IP \[bu] 2
|
|
Type: bool
|
|
.IP \[bu] 2
|
|
Default: false
|
|
.SS \-\-drive\-list\-chunk
|
|
.PP
|
|
Size of listing chunk 100\-1000.
|
|
0 to disable.
|
|
.IP \[bu] 2
|
|
Config: list_chunk
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_DRIVE_LIST_CHUNK
|
|
.IP \[bu] 2
|
|
Type: int
|
|
.IP \[bu] 2
|
|
Default: 1000
|
|
.SS \-\-drive\-impersonate
|
|
.PP
|
|
Impersonate this user when using a service account.
|
|
.IP \[bu] 2
|
|
Config: impersonate
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_DRIVE_IMPERSONATE
|
|
.IP \[bu] 2
|
|
Type: string
|
|
.IP \[bu] 2
|
|
Default: \[dq]\[dq]
|
|
.SS \-\-drive\-upload\-cutoff
|
|
.PP
|
|
Cutoff for switching to chunked upload
|
|
.IP \[bu] 2
|
|
Config: upload_cutoff
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_DRIVE_UPLOAD_CUTOFF
|
|
.IP \[bu] 2
|
|
Type: SizeSuffix
|
|
.IP \[bu] 2
|
|
Default: 8M
|
|
.SS \-\-drive\-chunk\-size
|
|
.PP
|
|
Upload chunk size.
|
|
Must a power of 2 >= 256k.
|
|
.PP
|
|
Making this larger will improve performance, but note that each chunk is
|
|
buffered in memory one per transfer.
|
|
.PP
|
|
Reducing this will reduce memory usage but decrease performance.
|
|
.IP \[bu] 2
|
|
Config: chunk_size
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_DRIVE_CHUNK_SIZE
|
|
.IP \[bu] 2
|
|
Type: SizeSuffix
|
|
.IP \[bu] 2
|
|
Default: 8M
|
|
.SS \-\-drive\-acknowledge\-abuse
|
|
.PP
|
|
Set to allow files which return cannotDownloadAbusiveFile to be
|
|
downloaded.
|
|
.PP
|
|
If downloading a file returns the error \[dq]This file has been
|
|
identified as malware or spam and cannot be downloaded\[dq] with the
|
|
error code \[dq]cannotDownloadAbusiveFile\[dq] then supply this flag to
|
|
rclone to indicate you acknowledge the risks of downloading the file and
|
|
rclone will download it anyway.
|
|
.IP \[bu] 2
|
|
Config: acknowledge_abuse
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_DRIVE_ACKNOWLEDGE_ABUSE
|
|
.IP \[bu] 2
|
|
Type: bool
|
|
.IP \[bu] 2
|
|
Default: false
|
|
.SS \-\-drive\-keep\-revision\-forever
|
|
.PP
|
|
Keep new head revision of each file forever.
|
|
.IP \[bu] 2
|
|
Config: keep_revision_forever
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_DRIVE_KEEP_REVISION_FOREVER
|
|
.IP \[bu] 2
|
|
Type: bool
|
|
.IP \[bu] 2
|
|
Default: false
|
|
.SS \-\-drive\-size\-as\-quota
|
|
.PP
|
|
Show sizes as storage quota usage, not actual size.
|
|
.PP
|
|
Show the size of a file as the storage quota used.
|
|
This is the current version plus any older versions that have been set
|
|
to keep forever.
|
|
.PP
|
|
\f[B]WARNING\f[R]: This flag may have some unexpected consequences.
|
|
.PP
|
|
It is not recommended to set this flag in your config \- the recommended
|
|
usage is using the flag form \-\-drive\-size\-as\-quota when doing
|
|
rclone ls/lsl/lsf/lsjson/etc only.
|
|
.PP
|
|
If you do use this flag for syncing (not recommended) then you will need
|
|
to use \-\-ignore size also.
|
|
.IP \[bu] 2
|
|
Config: size_as_quota
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_DRIVE_SIZE_AS_QUOTA
|
|
.IP \[bu] 2
|
|
Type: bool
|
|
.IP \[bu] 2
|
|
Default: false
|
|
.SS \-\-drive\-v2\-download\-min\-size
|
|
.PP
|
|
If Object\[aq]s are greater, use drive v2 API to download.
|
|
.IP \[bu] 2
|
|
Config: v2_download_min_size
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_DRIVE_V2_DOWNLOAD_MIN_SIZE
|
|
.IP \[bu] 2
|
|
Type: SizeSuffix
|
|
.IP \[bu] 2
|
|
Default: off
|
|
.SS \-\-drive\-pacer\-min\-sleep
|
|
.PP
|
|
Minimum time to sleep between API calls.
|
|
.IP \[bu] 2
|
|
Config: pacer_min_sleep
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_DRIVE_PACER_MIN_SLEEP
|
|
.IP \[bu] 2
|
|
Type: Duration
|
|
.IP \[bu] 2
|
|
Default: 100ms
|
|
.SS \-\-drive\-pacer\-burst
|
|
.PP
|
|
Number of API calls to allow without sleeping.
|
|
.IP \[bu] 2
|
|
Config: pacer_burst
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_DRIVE_PACER_BURST
|
|
.IP \[bu] 2
|
|
Type: int
|
|
.IP \[bu] 2
|
|
Default: 100
|
|
.SS \-\-drive\-server\-side\-across\-configs
|
|
.PP
|
|
Allow server\-side operations (e.g.
|
|
copy) to work across different drive configs.
|
|
.PP
|
|
This can be useful if you wish to do a server\-side copy between two
|
|
different Google drives.
|
|
Note that this isn\[aq]t enabled by default because it isn\[aq]t easy to
|
|
tell if it will work between any two configurations.
|
|
.IP \[bu] 2
|
|
Config: server_side_across_configs
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_DRIVE_SERVER_SIDE_ACROSS_CONFIGS
|
|
.IP \[bu] 2
|
|
Type: bool
|
|
.IP \[bu] 2
|
|
Default: false
|
|
.SS \-\-drive\-disable\-http2
|
|
.PP
|
|
Disable drive using http2
|
|
.PP
|
|
There is currently an unsolved issue with the google drive backend and
|
|
HTTP/2.
|
|
HTTP/2 is therefore disabled by default for the drive backend but can be
|
|
re\-enabled here.
|
|
When the issue is solved this flag will be removed.
|
|
.PP
|
|
See: https://github.com/rclone/rclone/issues/3631
|
|
.IP \[bu] 2
|
|
Config: disable_http2
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_DRIVE_DISABLE_HTTP2
|
|
.IP \[bu] 2
|
|
Type: bool
|
|
.IP \[bu] 2
|
|
Default: true
|
|
.SS \-\-drive\-stop\-on\-upload\-limit
|
|
.PP
|
|
Make upload limit errors be fatal
|
|
.PP
|
|
At the time of writing it is only possible to upload 750GB of data to
|
|
Google Drive a day (this is an undocumented limit).
|
|
When this limit is reached Google Drive produces a slightly different
|
|
error message.
|
|
When this flag is set it causes these errors to be fatal.
|
|
These will stop the in\-progress sync.
|
|
.PP
|
|
Note that this detection is relying on error message strings which
|
|
Google don\[aq]t document so it may break in the future.
|
|
.PP
|
|
See: https://github.com/rclone/rclone/issues/3857
|
|
.IP \[bu] 2
|
|
Config: stop_on_upload_limit
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_DRIVE_STOP_ON_UPLOAD_LIMIT
|
|
.IP \[bu] 2
|
|
Type: bool
|
|
.IP \[bu] 2
|
|
Default: false
|
|
.SS \-\-drive\-stop\-on\-download\-limit
|
|
.PP
|
|
Make download limit errors be fatal
|
|
.PP
|
|
At the time of writing it is only possible to download 10TB of data from
|
|
Google Drive a day (this is an undocumented limit).
|
|
When this limit is reached Google Drive produces a slightly different
|
|
error message.
|
|
When this flag is set it causes these errors to be fatal.
|
|
These will stop the in\-progress sync.
|
|
.PP
|
|
Note that this detection is relying on error message strings which
|
|
Google don\[aq]t document so it may break in the future.
|
|
.IP \[bu] 2
|
|
Config: stop_on_download_limit
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_DRIVE_STOP_ON_DOWNLOAD_LIMIT
|
|
.IP \[bu] 2
|
|
Type: bool
|
|
.IP \[bu] 2
|
|
Default: false
|
|
.SS \-\-drive\-skip\-shortcuts
|
|
.PP
|
|
If set skip shortcut files
|
|
.PP
|
|
Normally rclone dereferences shortcut files making them appear as if
|
|
they are the original file (see the shortcuts section).
|
|
If this flag is set then rclone will ignore shortcut files completely.
|
|
.IP \[bu] 2
|
|
Config: skip_shortcuts
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_DRIVE_SKIP_SHORTCUTS
|
|
.IP \[bu] 2
|
|
Type: bool
|
|
.IP \[bu] 2
|
|
Default: false
|
|
.SS \-\-drive\-encoding
|
|
.PP
|
|
This sets the encoding for the backend.
|
|
.PP
|
|
See: the encoding section in the
|
|
overview (https://rclone.org/overview/#encoding) for more info.
|
|
.IP \[bu] 2
|
|
Config: encoding
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_DRIVE_ENCODING
|
|
.IP \[bu] 2
|
|
Type: MultiEncoder
|
|
.IP \[bu] 2
|
|
Default: InvalidUtf8
|
|
.SS Backend commands
|
|
.PP
|
|
Here are the commands specific to the drive backend.
|
|
.PP
|
|
Run them with
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone backend COMMAND remote:
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
The help below will explain what arguments each command takes.
|
|
.PP
|
|
See the \[dq]rclone backend\[dq]
|
|
command (https://rclone.org/commands/rclone_backend/) for more info on
|
|
how to pass options and arguments.
|
|
.PP
|
|
These can be run on a running backend using the rc command
|
|
backend/command (https://rclone.org/rc/#backend/command).
|
|
.SS get
|
|
.PP
|
|
Get command for fetching the drive config parameters
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone backend get remote: [options] [<arguments>+]
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
This is a get command which will be used to fetch the various drive
|
|
config parameters
|
|
.PP
|
|
Usage Examples:
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone backend get drive: [\-o service_account_file] [\-o chunk_size]
|
|
rclone rc backend/command command=get fs=drive: [\-o service_account_file] [\-o chunk_size]
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
Options:
|
|
.IP \[bu] 2
|
|
\[dq]chunk_size\[dq]: show the current upload chunk size
|
|
.IP \[bu] 2
|
|
\[dq]service_account_file\[dq]: show the current service account file
|
|
.SS set
|
|
.PP
|
|
Set command for updating the drive config parameters
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone backend set remote: [options] [<arguments>+]
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
This is a set command which will be used to update the various drive
|
|
config parameters
|
|
.PP
|
|
Usage Examples:
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone backend set drive: [\-o service_account_file=sa.json] [\-o chunk_size=67108864]
|
|
rclone rc backend/command command=set fs=drive: [\-o service_account_file=sa.json] [\-o chunk_size=67108864]
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
Options:
|
|
.IP \[bu] 2
|
|
\[dq]chunk_size\[dq]: update the current upload chunk size
|
|
.IP \[bu] 2
|
|
\[dq]service_account_file\[dq]: update the current service account file
|
|
.SS shortcut
|
|
.PP
|
|
Create shortcuts from files or directories
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone backend shortcut remote: [options] [<arguments>+]
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
This command creates shortcuts from files or directories.
|
|
.PP
|
|
Usage:
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone backend shortcut drive: source_item destination_shortcut
|
|
rclone backend shortcut drive: source_item \-o target=drive2: destination_shortcut
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
In the first example this creates a shortcut from the
|
|
\[dq]source_item\[dq] which can be a file or a directory to the
|
|
\[dq]destination_shortcut\[dq].
|
|
The \[dq]source_item\[dq] and the \[dq]destination_shortcut\[dq] should
|
|
be relative paths from \[dq]drive:\[dq]
|
|
.PP
|
|
In the second example this creates a shortcut from the
|
|
\[dq]source_item\[dq] relative to \[dq]drive:\[dq] to the
|
|
\[dq]destination_shortcut\[dq] relative to \[dq]drive2:\[dq].
|
|
This may fail with a permission error if the user authenticated with
|
|
\[dq]drive2:\[dq] can\[aq]t read files from \[dq]drive:\[dq].
|
|
.PP
|
|
Options:
|
|
.IP \[bu] 2
|
|
\[dq]target\[dq]: optional target remote for the shortcut destination
|
|
.SS drives
|
|
.PP
|
|
List the Shared Drives available to this account
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone backend drives remote: [options] [<arguments>+]
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
This command lists the Shared Drives (Team Drives) available to this
|
|
account.
|
|
.PP
|
|
Usage:
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone backend drives drive:
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
This will return a JSON list of objects like this
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
[
|
|
{
|
|
\[dq]id\[dq]: \[dq]0ABCDEF\-01234567890\[dq],
|
|
\[dq]kind\[dq]: \[dq]drive#teamDrive\[dq],
|
|
\[dq]name\[dq]: \[dq]My Drive\[dq]
|
|
},
|
|
{
|
|
\[dq]id\[dq]: \[dq]0ABCDEFabcdefghijkl\[dq],
|
|
\[dq]kind\[dq]: \[dq]drive#teamDrive\[dq],
|
|
\[dq]name\[dq]: \[dq]Test Drive\[dq]
|
|
}
|
|
]
|
|
\f[R]
|
|
.fi
|
|
.SS untrash
|
|
.PP
|
|
Untrash files and directories
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone backend untrash remote: [options] [<arguments>+]
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
This command untrashes all the files and directories in the directory
|
|
passed in recursively.
|
|
.PP
|
|
Usage:
|
|
.PP
|
|
This takes an optional directory to trash which make this easier to use
|
|
via the API.
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone backend untrash drive:directory
|
|
rclone backend \-i untrash drive:directory subdir
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
Use the \-i flag to see what would be restored before restoring it.
|
|
.PP
|
|
Result:
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
{
|
|
\[dq]Untrashed\[dq]: 17,
|
|
\[dq]Errors\[dq]: 0
|
|
}
|
|
\f[R]
|
|
.fi
|
|
.SS copyid
|
|
.PP
|
|
Copy files by ID
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone backend copyid remote: [options] [<arguments>+]
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
This command copies files by ID
|
|
.PP
|
|
Usage:
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone backend copyid drive: ID path
|
|
rclone backend copyid drive: ID1 path1 ID2 path2
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
It copies the drive file with ID given to the path (an rclone path which
|
|
will be passed internally to rclone copyto).
|
|
The ID and path pairs can be repeated.
|
|
.PP
|
|
The path should end with a / to indicate copy the file as named to this
|
|
directory.
|
|
If it doesn\[aq]t end with a / then the last path component will be used
|
|
as the file name.
|
|
.PP
|
|
If the destination is a drive backend then server\-side copying will be
|
|
attempted if possible.
|
|
.PP
|
|
Use the \-i flag to see what would be copied before copying.
|
|
.SS Limitations
|
|
.PP
|
|
Drive has quite a lot of rate limiting.
|
|
This causes rclone to be limited to transferring about 2 files per
|
|
second only.
|
|
Individual files may be transferred much faster at 100s of MBytes/s but
|
|
lots of small files can take a long time.
|
|
.PP
|
|
Server side copies are also subject to a separate rate limit.
|
|
If you see User rate limit exceeded errors, wait at least 24 hours and
|
|
retry.
|
|
You can disable server\-side copies with \f[C]\-\-disable copy\f[R] to
|
|
download and upload the files if you prefer.
|
|
.SS Limitations of Google Docs
|
|
.PP
|
|
Google docs will appear as size \-1 in \f[C]rclone ls\f[R] and as size 0
|
|
in anything which uses the VFS layer, e.g.
|
|
\f[C]rclone mount\f[R], \f[C]rclone serve\f[R].
|
|
.PP
|
|
This is because rclone can\[aq]t find out the size of the Google docs
|
|
without downloading them.
|
|
.PP
|
|
Google docs will transfer correctly with \f[C]rclone sync\f[R],
|
|
\f[C]rclone copy\f[R] etc as rclone knows to ignore the size when doing
|
|
the transfer.
|
|
.PP
|
|
However an unfortunate consequence of this is that you may not be able
|
|
to download Google docs using \f[C]rclone mount\f[R].
|
|
If it doesn\[aq]t work you will get a 0 sized file.
|
|
If you try again the doc may gain its correct size and be downloadable.
|
|
Whether it will work on not depends on the application accessing the
|
|
mount and the OS you are running \- experiment to find out if it does
|
|
work for you!
|
|
.SS Duplicated files
|
|
.PP
|
|
Sometimes, for no reason I\[aq]ve been able to track down, drive will
|
|
duplicate a file that rclone uploads.
|
|
Drive unlike all the other remotes can have duplicated files.
|
|
.PP
|
|
Duplicated files cause problems with the syncing and you will see
|
|
messages in the log about duplicates.
|
|
.PP
|
|
Use \f[C]rclone dedupe\f[R] to fix duplicated files.
|
|
.PP
|
|
Note that this isn\[aq]t just a problem with rclone, even Google Photos
|
|
on Android duplicates files on drive sometimes.
|
|
.SS Rclone appears to be re\-copying files it shouldn\[aq]t
|
|
.PP
|
|
The most likely cause of this is the duplicated file issue above \- run
|
|
\f[C]rclone dedupe\f[R] and check your logs for duplicate object or
|
|
directory messages.
|
|
.PP
|
|
This can also be caused by a delay/caching on google drive\[aq]s end
|
|
when comparing directory listings.
|
|
Specifically with team drives used in combination with \-\-fast\-list.
|
|
Files that were uploaded recently may not appear on the directory list
|
|
sent to rclone when using \-\-fast\-list.
|
|
.PP
|
|
Waiting a moderate period of time between attempts (estimated to be
|
|
approximately 1 hour) and/or not using \-\-fast\-list both seem to be
|
|
effective in preventing the problem.
|
|
.SS Making your own client_id
|
|
.PP
|
|
When you use rclone with Google drive in its default configuration you
|
|
are using rclone\[aq]s client_id.
|
|
This is shared between all the rclone users.
|
|
There is a global rate limit on the number of queries per second that
|
|
each client_id can do set by Google.
|
|
rclone already has a high quota and I will continue to make sure it is
|
|
high enough by contacting Google.
|
|
.PP
|
|
It is strongly recommended to use your own client ID as the default
|
|
rclone ID is heavily used.
|
|
If you have multiple services running, it is recommended to use an API
|
|
key for each service.
|
|
The default Google quota is 10 transactions per second so it is
|
|
recommended to stay under that number as if you use more than that, it
|
|
will cause rclone to rate limit and make things slower.
|
|
.PP
|
|
Here is how to create your own Google Drive client ID for rclone:
|
|
.IP "1." 3
|
|
Log into the Google API Console (https://console.developers.google.com/)
|
|
with your Google account.
|
|
It doesn\[aq]t matter what Google account you use.
|
|
(It need not be the same account as the Google Drive you want to access)
|
|
.IP "2." 3
|
|
Select a project or create a new project.
|
|
.IP "3." 3
|
|
Under \[dq]ENABLE APIS AND SERVICES\[dq] search for \[dq]Drive\[dq], and
|
|
enable the \[dq]Google Drive API\[dq].
|
|
.IP "4." 3
|
|
Click \[dq]Credentials\[dq] in the left\-side panel (not \[dq]Create
|
|
credentials\[dq], which opens the wizard), then \[dq]Create
|
|
credentials\[dq]
|
|
.IP "5." 3
|
|
If you already configured an \[dq]Oauth Consent Screen\[dq], then skip
|
|
to the next step; if not, click on \[dq]CONFIGURE CONSENT SCREEN\[dq]
|
|
button (near the top right corner of the right panel), then select
|
|
\[dq]External\[dq] and click on \[dq]CREATE\[dq]; on the next screen,
|
|
enter an \[dq]Application name\[dq] (\[dq]rclone\[dq] is OK) then click
|
|
on \[dq]Save\[dq] (all other data is optional).
|
|
Click again on \[dq]Credentials\[dq] on the left panel to go back to the
|
|
\[dq]Credentials\[dq] screen.
|
|
.PP
|
|
(PS: if you are a GSuite user, you could also select \[dq]Internal\[dq]
|
|
instead of \[dq]External\[dq] above, but this has not been
|
|
tested/documented so far).
|
|
.IP "6." 3
|
|
Click on the \[dq]+ CREATE CREDENTIALS\[dq] button at the top of the
|
|
screen, then select \[dq]OAuth client ID\[dq].
|
|
.IP "7." 3
|
|
Choose an application type of \[dq]Desktop app\[dq] if you using a
|
|
Google account or \[dq]Other\[dq] if you using a GSuite account and
|
|
click \[dq]Create\[dq].
|
|
(the default name is fine)
|
|
.IP "8." 3
|
|
It will show you a client ID and client secret.
|
|
Use these values in rclone config to add a new remote or edit an
|
|
existing remote.
|
|
.PP
|
|
Be aware that, due to the \[dq]enhanced security\[dq] recently
|
|
introduced by Google, you are theoretically expected to \[dq]submit your
|
|
app for verification\[dq] and then wait a few weeks(!) for their
|
|
response; in practice, you can go right ahead and use the client ID and
|
|
client secret with rclone, the only issue will be a very scary
|
|
confirmation screen shown when you connect via your browser for rclone
|
|
to be able to get its token\-id (but as this only happens during the
|
|
remote configuration, it\[aq]s not such a big deal).
|
|
.PP
|
|
(Thanks to \[at]balazer on github for these instructions.)
|
|
.PP
|
|
Sometimes, creation of an OAuth consent in Google API Console fails due
|
|
to an error message \[lq]The request failed because changes to one of
|
|
the field of the resource is not supported\[rq].
|
|
As a convenient workaround, the necessary Google Drive API key can be
|
|
created on the Python
|
|
Quickstart (https://developers.google.com/drive/api/v3/quickstart/python)
|
|
page.
|
|
Just push the Enable the Drive API button to receive the Client ID and
|
|
Secret.
|
|
Note that it will automatically create a new project in the API Console.
|
|
.SS Google Photos
|
|
.PP
|
|
The rclone backend for Google
|
|
Photos (https://www.google.com/photos/about/) is a specialized backend
|
|
for transferring photos and videos to and from Google Photos.
|
|
.PP
|
|
\f[B]NB\f[R] The Google Photos API which rclone uses has quite a few
|
|
limitations, so please read the limitations section carefully to make
|
|
sure it is suitable for your use.
|
|
.SS Configuring Google Photos
|
|
.PP
|
|
The initial setup for google cloud storage involves getting a token from
|
|
Google Photos which you need to do in your browser.
|
|
\f[C]rclone config\f[R] walks you through it.
|
|
.PP
|
|
Here is an example of how to make a remote called \f[C]remote\f[R].
|
|
First run:
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone config
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
This will guide you through an interactive setup process:
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
No remotes found \- make a new one
|
|
n) New remote
|
|
s) Set configuration password
|
|
q) Quit config
|
|
n/s/q> n
|
|
name> remote
|
|
Type of storage to configure.
|
|
Enter a string value. Press Enter for the default (\[dq]\[dq]).
|
|
Choose a number from below, or type in your own value
|
|
[snip]
|
|
XX / Google Photos
|
|
\[rs] \[dq]google photos\[dq]
|
|
[snip]
|
|
Storage> google photos
|
|
** See help for google photos backend at: https://rclone.org/googlephotos/ **
|
|
|
|
Google Application Client Id
|
|
Leave blank normally.
|
|
Enter a string value. Press Enter for the default (\[dq]\[dq]).
|
|
client_id>
|
|
Google Application Client Secret
|
|
Leave blank normally.
|
|
Enter a string value. Press Enter for the default (\[dq]\[dq]).
|
|
client_secret>
|
|
Set to make the Google Photos backend read only.
|
|
|
|
If you choose read only then rclone will only request read only access
|
|
to your photos, otherwise rclone will request full access.
|
|
Enter a boolean value (true or false). Press Enter for the default (\[dq]false\[dq]).
|
|
read_only>
|
|
Edit advanced config? (y/n)
|
|
y) Yes
|
|
n) No
|
|
y/n> n
|
|
Remote config
|
|
Use auto config?
|
|
* Say Y if not sure
|
|
* Say N if you are working on a remote or headless machine
|
|
y) Yes
|
|
n) No
|
|
y/n> y
|
|
If your browser doesn\[aq]t open automatically go to the following link: http://127.0.0.1:53682/auth
|
|
Log in and authorize rclone for access
|
|
Waiting for code...
|
|
Got code
|
|
|
|
*** IMPORTANT: All media items uploaded to Google Photos with rclone
|
|
*** are stored in full resolution at original quality. These uploads
|
|
*** will count towards storage in your Google Account.
|
|
|
|
\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-
|
|
[remote]
|
|
type = google photos
|
|
token = {\[dq]access_token\[dq]:\[dq]XXX\[dq],\[dq]token_type\[dq]:\[dq]Bearer\[dq],\[dq]refresh_token\[dq]:\[dq]XXX\[dq],\[dq]expiry\[dq]:\[dq]2019\-06\-28T17:38:04.644930156+01:00\[dq]}
|
|
\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-
|
|
y) Yes this is OK
|
|
e) Edit this remote
|
|
d) Delete this remote
|
|
y/e/d> y
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
Note that rclone runs a webserver on your local machine to collect the
|
|
token as returned from Google if you use auto config mode.
|
|
This only runs from the moment it opens your browser to the moment you
|
|
get back the verification code.
|
|
This is on \f[C]http://127.0.0.1:53682/\f[R] and this may require you to
|
|
unblock it temporarily if you are running a host firewall, or use manual
|
|
mode.
|
|
.PP
|
|
This remote is called \f[C]remote\f[R] and can now be used like this
|
|
.PP
|
|
See all the albums in your photos
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone lsd remote:album
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
Make a new album
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone mkdir remote:album/newAlbum
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
List the contents of an album
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone ls remote:album/newAlbum
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
Sync \f[C]/home/local/images\f[R] to the Google Photos, removing any
|
|
excess files in the album.
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone sync \-i /home/local/image remote:album/newAlbum
|
|
\f[R]
|
|
.fi
|
|
.SS Layout
|
|
.PP
|
|
As Google Photos is not a general purpose cloud storage system the
|
|
backend is laid out to help you navigate it.
|
|
.PP
|
|
The directories under \f[C]media\f[R] show different ways of
|
|
categorizing the media.
|
|
Each file will appear multiple times.
|
|
So if you want to make a backup of your google photos you might choose
|
|
to backup \f[C]remote:media/by\-month\f[R].
|
|
(\f[B]NB\f[R] \f[C]remote:media/by\-day\f[R] is rather slow at the
|
|
moment so avoid for syncing.)
|
|
.PP
|
|
Note that all your photos and videos will appear somewhere under
|
|
\f[C]media\f[R], but they may not appear under \f[C]album\f[R] unless
|
|
you\[aq]ve put them into albums.
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
/
|
|
\- upload
|
|
\- file1.jpg
|
|
\- file2.jpg
|
|
\- ...
|
|
\- media
|
|
\- all
|
|
\- file1.jpg
|
|
\- file2.jpg
|
|
\- ...
|
|
\- by\-year
|
|
\- 2000
|
|
\- file1.jpg
|
|
\- ...
|
|
\- 2001
|
|
\- file2.jpg
|
|
\- ...
|
|
\- ...
|
|
\- by\-month
|
|
\- 2000
|
|
\- 2000\-01
|
|
\- file1.jpg
|
|
\- ...
|
|
\- 2000\-02
|
|
\- file2.jpg
|
|
\- ...
|
|
\- ...
|
|
\- by\-day
|
|
\- 2000
|
|
\- 2000\-01\-01
|
|
\- file1.jpg
|
|
\- ...
|
|
\- 2000\-01\-02
|
|
\- file2.jpg
|
|
\- ...
|
|
\- ...
|
|
\- album
|
|
\- album name
|
|
\- album name/sub
|
|
\- shared\-album
|
|
\- album name
|
|
\- album name/sub
|
|
\- feature
|
|
\- favorites
|
|
\- file1.jpg
|
|
\- file2.jpg
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
There are two writable parts of the tree, the \f[C]upload\f[R] directory
|
|
and sub directories of the \f[C]album\f[R] directory.
|
|
.PP
|
|
The \f[C]upload\f[R] directory is for uploading files you don\[aq]t want
|
|
to put into albums.
|
|
This will be empty to start with and will contain the files you\[aq]ve
|
|
uploaded for one rclone session only, becoming empty again when you
|
|
restart rclone.
|
|
The use case for this would be if you have a load of files you just want
|
|
to once off dump into Google Photos.
|
|
For repeated syncing, uploading to \f[C]album\f[R] will work better.
|
|
.PP
|
|
Directories within the \f[C]album\f[R] directory are also writeable and
|
|
you may create new directories (albums) under \f[C]album\f[R].
|
|
If you copy files with a directory hierarchy in there then rclone will
|
|
create albums with the \f[C]/\f[R] character in them.
|
|
For example if you do
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone copy /path/to/images remote:album/images
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
and the images directory contains
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
images
|
|
\- file1.jpg
|
|
dir
|
|
file2.jpg
|
|
dir2
|
|
dir3
|
|
file3.jpg
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
Then rclone will create the following albums with the following files in
|
|
.IP \[bu] 2
|
|
images
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
file1.jpg
|
|
.RE
|
|
.IP \[bu] 2
|
|
images/dir
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
file2.jpg
|
|
.RE
|
|
.IP \[bu] 2
|
|
images/dir2/dir3
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
file3.jpg
|
|
.RE
|
|
.PP
|
|
This means that you can use the \f[C]album\f[R] path pretty much like a
|
|
normal filesystem and it is a good target for repeated syncing.
|
|
.PP
|
|
The \f[C]shared\-album\f[R] directory shows albums shared with you or by
|
|
you.
|
|
This is similar to the Sharing tab in the Google Photos web interface.
|
|
.SS Limitations
|
|
.PP
|
|
Only images and videos can be uploaded.
|
|
If you attempt to upload non videos or images or formats that Google
|
|
Photos doesn\[aq]t understand, rclone will upload the file, then Google
|
|
Photos will give an error when it is put turned into a media item.
|
|
.PP
|
|
Note that all media items uploaded to Google Photos through the API are
|
|
stored in full resolution at \[dq]original quality\[dq] and
|
|
\f[B]will\f[R] count towards your storage quota in your Google Account.
|
|
The API does \f[B]not\f[R] offer a way to upload in \[dq]high
|
|
quality\[dq] mode..
|
|
.PP
|
|
\f[C]rclone about\f[R] is not supported by the Google Photos backend.
|
|
Backends without this capability cannot determine free space for an
|
|
rclone mount or use policy \f[C]mfs\f[R] (most free space) as a member
|
|
of an rclone union remote.
|
|
.PP
|
|
See List of backends that do not support rclone
|
|
about (https://rclone.org/overview/#optional-features) See rclone
|
|
about (https://rclone.org/commands/rclone_about/)
|
|
.SS Downloading Images
|
|
.PP
|
|
When Images are downloaded this strips EXIF location (according to the
|
|
docs and my tests).
|
|
This is a limitation of the Google Photos API and is covered by bug
|
|
#112096115 (https://issuetracker.google.com/issues/112096115).
|
|
.PP
|
|
\f[B]The current google API does not allow photos to be downloaded at
|
|
original resolution. This is very important if you are, for example,
|
|
relying on \[dq]Google Photos\[dq] as a backup of your photos. You will
|
|
not be able to use rclone to redownload original images. You could use
|
|
\[aq]google takeout\[aq] to recover the original photos as a last
|
|
resort\f[R]
|
|
.SS Downloading Videos
|
|
.PP
|
|
When videos are downloaded they are downloaded in a really compressed
|
|
version of the video compared to downloading it via the Google Photos
|
|
web interface.
|
|
This is covered by bug
|
|
#113672044 (https://issuetracker.google.com/issues/113672044).
|
|
.SS Duplicates
|
|
.PP
|
|
If a file name is duplicated in a directory then rclone will add the
|
|
file ID into its name.
|
|
So two files called \f[C]file.jpg\f[R] would then appear as
|
|
\f[C]file {123456}.jpg\f[R] and \f[C]file {ABCDEF}.jpg\f[R] (the actual
|
|
IDs are a lot longer alas!).
|
|
.PP
|
|
If you upload the same image (with the same binary data) twice then
|
|
Google Photos will deduplicate it.
|
|
However it will retain the filename from the first upload which may
|
|
confuse rclone.
|
|
For example if you uploaded an image to \f[C]upload\f[R] then uploaded
|
|
the same image to \f[C]album/my_album\f[R] the filename of the image in
|
|
\f[C]album/my_album\f[R] will be what it was uploaded with initially,
|
|
not what you uploaded it with to \f[C]album\f[R].
|
|
In practise this shouldn\[aq]t cause too many problems.
|
|
.SS Modified time
|
|
.PP
|
|
The date shown of media in Google Photos is the creation date as
|
|
determined by the EXIF information, or the upload date if that is not
|
|
known.
|
|
.PP
|
|
This is not changeable by rclone and is not the modification date of the
|
|
media on local disk.
|
|
This means that rclone cannot use the dates from Google Photos for
|
|
syncing purposes.
|
|
.SS Size
|
|
.PP
|
|
The Google Photos API does not return the size of media.
|
|
This means that when syncing to Google Photos, rclone can only do a file
|
|
existence check.
|
|
.PP
|
|
It is possible to read the size of the media, but this needs an extra
|
|
HTTP HEAD request per media item so is \f[B]very slow\f[R] and uses up a
|
|
lot of transactions.
|
|
This can be enabled with the \f[C]\-\-gphotos\-read\-size\f[R] option or
|
|
the \f[C]read_size = true\f[R] config parameter.
|
|
.PP
|
|
If you want to use the backend with \f[C]rclone mount\f[R] you may need
|
|
to enable this flag (depending on your OS and application using the
|
|
photos) otherwise you may not be able to read media off the mount.
|
|
You\[aq]ll need to experiment to see if it works for you without the
|
|
flag.
|
|
.SS Albums
|
|
.PP
|
|
Rclone can only upload files to albums it created.
|
|
This is a limitation of the Google Photos
|
|
API (https://developers.google.com/photos/library/guides/manage-albums).
|
|
.PP
|
|
Rclone can remove files it uploaded from albums it created only.
|
|
.SS Deleting files
|
|
.PP
|
|
Rclone can remove files from albums it created, but note that the Google
|
|
Photos API does not allow media to be deleted permanently so this media
|
|
will still remain.
|
|
See bug #109759781 (https://issuetracker.google.com/issues/109759781).
|
|
.PP
|
|
Rclone cannot delete files anywhere except under \f[C]album\f[R].
|
|
.SS Deleting albums
|
|
.PP
|
|
The Google Photos API does not support deleting albums \- see bug
|
|
#135714733 (https://issuetracker.google.com/issues/135714733).
|
|
.SS Standard Options
|
|
.PP
|
|
Here are the standard options specific to google photos (Google Photos).
|
|
.SS \-\-gphotos\-client\-id
|
|
.PP
|
|
OAuth Client Id Leave blank normally.
|
|
.IP \[bu] 2
|
|
Config: client_id
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_GPHOTOS_CLIENT_ID
|
|
.IP \[bu] 2
|
|
Type: string
|
|
.IP \[bu] 2
|
|
Default: \[dq]\[dq]
|
|
.SS \-\-gphotos\-client\-secret
|
|
.PP
|
|
OAuth Client Secret Leave blank normally.
|
|
.IP \[bu] 2
|
|
Config: client_secret
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_GPHOTOS_CLIENT_SECRET
|
|
.IP \[bu] 2
|
|
Type: string
|
|
.IP \[bu] 2
|
|
Default: \[dq]\[dq]
|
|
.SS \-\-gphotos\-read\-only
|
|
.PP
|
|
Set to make the Google Photos backend read only.
|
|
.PP
|
|
If you choose read only then rclone will only request read only access
|
|
to your photos, otherwise rclone will request full access.
|
|
.IP \[bu] 2
|
|
Config: read_only
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_GPHOTOS_READ_ONLY
|
|
.IP \[bu] 2
|
|
Type: bool
|
|
.IP \[bu] 2
|
|
Default: false
|
|
.SS Advanced Options
|
|
.PP
|
|
Here are the advanced options specific to google photos (Google Photos).
|
|
.SS \-\-gphotos\-token
|
|
.PP
|
|
OAuth Access Token as a JSON blob.
|
|
.IP \[bu] 2
|
|
Config: token
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_GPHOTOS_TOKEN
|
|
.IP \[bu] 2
|
|
Type: string
|
|
.IP \[bu] 2
|
|
Default: \[dq]\[dq]
|
|
.SS \-\-gphotos\-auth\-url
|
|
.PP
|
|
Auth server URL.
|
|
Leave blank to use the provider defaults.
|
|
.IP \[bu] 2
|
|
Config: auth_url
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_GPHOTOS_AUTH_URL
|
|
.IP \[bu] 2
|
|
Type: string
|
|
.IP \[bu] 2
|
|
Default: \[dq]\[dq]
|
|
.SS \-\-gphotos\-token\-url
|
|
.PP
|
|
Token server url.
|
|
Leave blank to use the provider defaults.
|
|
.IP \[bu] 2
|
|
Config: token_url
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_GPHOTOS_TOKEN_URL
|
|
.IP \[bu] 2
|
|
Type: string
|
|
.IP \[bu] 2
|
|
Default: \[dq]\[dq]
|
|
.SS \-\-gphotos\-read\-size
|
|
.PP
|
|
Set to read the size of media items.
|
|
.PP
|
|
Normally rclone does not read the size of media items since this takes
|
|
another transaction.
|
|
This isn\[aq]t necessary for syncing.
|
|
However rclone mount needs to know the size of files in advance of
|
|
reading them, so setting this flag when using rclone mount is
|
|
recommended if you want to read the media.
|
|
.IP \[bu] 2
|
|
Config: read_size
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_GPHOTOS_READ_SIZE
|
|
.IP \[bu] 2
|
|
Type: bool
|
|
.IP \[bu] 2
|
|
Default: false
|
|
.SS \-\-gphotos\-start\-year
|
|
.PP
|
|
Year limits the photos to be downloaded to those which are uploaded
|
|
after the given year
|
|
.IP \[bu] 2
|
|
Config: start_year
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_GPHOTOS_START_YEAR
|
|
.IP \[bu] 2
|
|
Type: int
|
|
.IP \[bu] 2
|
|
Default: 2000
|
|
.SS \-\-gphotos\-include\-archived
|
|
.PP
|
|
Also view and download archived media.
|
|
.PP
|
|
By default rclone does not request archived media.
|
|
Thus, when syncing, archived media is not visible in directory listings
|
|
or transferred.
|
|
.PP
|
|
Note that media in albums is always visible and synced, no matter their
|
|
archive status.
|
|
.PP
|
|
With this flag, archived media are always visible in directory listings
|
|
and transferred.
|
|
.PP
|
|
Without this flag, archived media will not be visible in directory
|
|
listings and won\[aq]t be transferred.
|
|
.IP \[bu] 2
|
|
Config: include_archived
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_GPHOTOS_INCLUDE_ARCHIVED
|
|
.IP \[bu] 2
|
|
Type: bool
|
|
.IP \[bu] 2
|
|
Default: false
|
|
.SS HDFS
|
|
.PP
|
|
HDFS (https://hadoop.apache.org/docs/stable/hadoop-project-dist/hadoop-hdfs/HdfsDesign.html)
|
|
is a distributed file\-system, part of the Apache
|
|
Hadoop (https://hadoop.apache.org/) framework.
|
|
.PP
|
|
Paths are specified as \f[C]remote:\f[R] or
|
|
\f[C]remote:path/to/dir\f[R].
|
|
.PP
|
|
Here is an example of how to make a remote called \f[C]remote\f[R].
|
|
First run:
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone config
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
This will guide you through an interactive setup process:
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
No remotes found \- make a new one
|
|
n) New remote
|
|
s) Set configuration password
|
|
q) Quit config
|
|
n/s/q> n
|
|
name> remote
|
|
Type of storage to configure.
|
|
Enter a string value. Press Enter for the default (\[dq]\[dq]).
|
|
Choose a number from below, or type in your own value
|
|
[skip]
|
|
XX / Hadoop distributed file system
|
|
\[rs] \[dq]hdfs\[dq]
|
|
[skip]
|
|
Storage> hdfs
|
|
** See help for hdfs backend at: https://rclone.org/hdfs/ **
|
|
|
|
hadoop name node and port
|
|
Enter a string value. Press Enter for the default (\[dq]\[dq]).
|
|
Choose a number from below, or type in your own value
|
|
1 / Connect to host namenode at port 8020
|
|
\[rs] \[dq]namenode:8020\[dq]
|
|
namenode> namenode.hadoop:8020
|
|
hadoop user name
|
|
Enter a string value. Press Enter for the default (\[dq]\[dq]).
|
|
Choose a number from below, or type in your own value
|
|
1 / Connect to hdfs as root
|
|
\[rs] \[dq]root\[dq]
|
|
username> root
|
|
Edit advanced config? (y/n)
|
|
y) Yes
|
|
n) No (default)
|
|
y/n> n
|
|
Remote config
|
|
\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-
|
|
[remote]
|
|
type = hdfs
|
|
namenode = namenode.hadoop:8020
|
|
username = root
|
|
\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-
|
|
y) Yes this is OK (default)
|
|
e) Edit this remote
|
|
d) Delete this remote
|
|
y/e/d> y
|
|
Current remotes:
|
|
|
|
Name Type
|
|
==== ====
|
|
hadoop hdfs
|
|
|
|
e) Edit existing remote
|
|
n) New remote
|
|
d) Delete remote
|
|
r) Rename remote
|
|
c) Copy remote
|
|
s) Set configuration password
|
|
q) Quit config
|
|
e/n/d/r/c/s/q> q
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
This remote is called \f[C]remote\f[R] and can now be used like this
|
|
.PP
|
|
See all the top level directories
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone lsd remote:
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
List the contents of a directory
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone ls remote:directory
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
Sync the remote \f[C]directory\f[R] to \f[C]/home/local/directory\f[R],
|
|
deleting any excess files.
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone sync \-i remote:directory /home/local/directory
|
|
\f[R]
|
|
.fi
|
|
.SS Setting up your own HDFS instance for testing
|
|
.PP
|
|
You may start with a manual
|
|
setup (https://hadoop.apache.org/docs/stable/hadoop-project-dist/hadoop-common/SingleCluster.html)
|
|
or use the docker image from the tests:
|
|
.PP
|
|
If you want to build the docker image
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
git clone https://github.com/rclone/rclone.git
|
|
cd rclone/fstest/testserver/images/test\-hdfs
|
|
docker build \-\-rm \-t rclone/test\-hdfs .
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
Or you can just use the latest one pushed
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
docker run \-\-rm \-\-name \[dq]rclone\-hdfs\[dq] \-p 127.0.0.1:9866:9866 \-p 127.0.0.1:8020:8020 \-\-hostname \[dq]rclone\-hdfs\[dq] rclone/test\-hdfs
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
\f[B]NB\f[R] it need few seconds to startup.
|
|
.PP
|
|
For this docker image the remote needs to be configured like this:
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
[remote]
|
|
type = hdfs
|
|
namenode = 127.0.0.1:8020
|
|
username = root
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
You can stop this image with \f[C]docker kill rclone\-hdfs\f[R]
|
|
(\f[B]NB\f[R] it does not use volumes, so all data uploaded will be
|
|
lost.)
|
|
.SS Modified time
|
|
.PP
|
|
Time accurate to 1 second is stored.
|
|
.SS Checksum
|
|
.PP
|
|
No checksums are implemented.
|
|
.SS Usage information
|
|
.PP
|
|
You can use the \f[C]rclone about remote:\f[R] command which will
|
|
display filesystem size and current usage.
|
|
.SS Restricted filename characters
|
|
.PP
|
|
In addition to the default restricted characters
|
|
set (https://rclone.org/overview/#restricted-characters) the following
|
|
characters are also replaced:
|
|
.PP
|
|
.TS
|
|
tab(@);
|
|
l c c.
|
|
T{
|
|
Character
|
|
T}@T{
|
|
Value
|
|
T}@T{
|
|
Replacement
|
|
T}
|
|
_
|
|
T{
|
|
:
|
|
T}@T{
|
|
0x3A
|
|
T}@T{
|
|
\[uFF1A]
|
|
T}
|
|
.TE
|
|
.PP
|
|
Invalid UTF\-8 bytes will also be
|
|
replaced (https://rclone.org/overview/#invalid-utf8).
|
|
.SS Limitations
|
|
.IP \[bu] 2
|
|
No server\-side \f[C]Move\f[R] or \f[C]DirMove\f[R].
|
|
.IP \[bu] 2
|
|
Checksums not implemented.
|
|
.SS Standard Options
|
|
.PP
|
|
Here are the standard options specific to hdfs (Hadoop distributed file
|
|
system).
|
|
.SS \-\-hdfs\-namenode
|
|
.PP
|
|
hadoop name node and port
|
|
.IP \[bu] 2
|
|
Config: namenode
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_HDFS_NAMENODE
|
|
.IP \[bu] 2
|
|
Type: string
|
|
.IP \[bu] 2
|
|
Default: \[dq]\[dq]
|
|
.IP \[bu] 2
|
|
Examples:
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
\[dq]namenode:8020\[dq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Connect to host namenode at port 8020
|
|
.RE
|
|
.RE
|
|
.SS \-\-hdfs\-username
|
|
.PP
|
|
hadoop user name
|
|
.IP \[bu] 2
|
|
Config: username
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_HDFS_USERNAME
|
|
.IP \[bu] 2
|
|
Type: string
|
|
.IP \[bu] 2
|
|
Default: \[dq]\[dq]
|
|
.IP \[bu] 2
|
|
Examples:
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
\[dq]root\[dq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Connect to hdfs as root
|
|
.RE
|
|
.RE
|
|
.SS Advanced Options
|
|
.PP
|
|
Here are the advanced options specific to hdfs (Hadoop distributed file
|
|
system).
|
|
.SS \-\-hdfs\-service\-principal\-name
|
|
.PP
|
|
Kerberos service principal name for the namenode
|
|
.PP
|
|
Enables KERBEROS authentication.
|
|
Specifies the Service Principal Name (/) for the namenode.
|
|
.IP \[bu] 2
|
|
Config: service_principal_name
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_HDFS_SERVICE_PRINCIPAL_NAME
|
|
.IP \[bu] 2
|
|
Type: string
|
|
.IP \[bu] 2
|
|
Default: \[dq]\[dq]
|
|
.IP \[bu] 2
|
|
Examples:
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
\[dq]hdfs/namenode.hadoop.docker\[dq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Namenode running as service \[aq]hdfs\[aq] with FQDN
|
|
\[aq]namenode.hadoop.docker\[aq].
|
|
.RE
|
|
.RE
|
|
.SS \-\-hdfs\-data\-transfer\-protection
|
|
.PP
|
|
Kerberos data transfer protection: authentication|integrity|privacy
|
|
.PP
|
|
Specifies whether or not authentication, data signature integrity
|
|
checks, and wire encryption is required when communicating the the
|
|
datanodes.
|
|
Possible values are \[aq]authentication\[aq], \[aq]integrity\[aq] and
|
|
\[aq]privacy\[aq].
|
|
Used only with KERBEROS enabled.
|
|
.IP \[bu] 2
|
|
Config: data_transfer_protection
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_HDFS_DATA_TRANSFER_PROTECTION
|
|
.IP \[bu] 2
|
|
Type: string
|
|
.IP \[bu] 2
|
|
Default: \[dq]\[dq]
|
|
.IP \[bu] 2
|
|
Examples:
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
\[dq]privacy\[dq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Ensure authentication, integrity and encryption enabled.
|
|
.RE
|
|
.RE
|
|
.SS \-\-hdfs\-encoding
|
|
.PP
|
|
This sets the encoding for the backend.
|
|
.PP
|
|
See: the encoding section in the
|
|
overview (https://rclone.org/overview/#encoding) for more info.
|
|
.IP \[bu] 2
|
|
Config: encoding
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_HDFS_ENCODING
|
|
.IP \[bu] 2
|
|
Type: MultiEncoder
|
|
.IP \[bu] 2
|
|
Default: Slash,Colon,Del,Ctl,InvalidUtf8,Dot
|
|
.SS HTTP
|
|
.PP
|
|
The HTTP remote is a read only remote for reading files of a webserver.
|
|
The webserver should provide file listings which rclone will read and
|
|
turn into a remote.
|
|
This has been tested with common webservers such as Apache/Nginx/Caddy
|
|
and will likely work with file listings from most web servers.
|
|
(If it doesn\[aq]t then please file an issue, or send a pull request!)
|
|
.PP
|
|
Paths are specified as \f[C]remote:\f[R] or
|
|
\f[C]remote:path/to/dir\f[R].
|
|
.PP
|
|
Here is an example of how to make a remote called \f[C]remote\f[R].
|
|
First run:
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone config
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
This will guide you through an interactive setup process:
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
No remotes found \- make a new one
|
|
n) New remote
|
|
s) Set configuration password
|
|
q) Quit config
|
|
n/s/q> n
|
|
name> remote
|
|
Type of storage to configure.
|
|
Choose a number from below, or type in your own value
|
|
[snip]
|
|
XX / http Connection
|
|
\[rs] \[dq]http\[dq]
|
|
[snip]
|
|
Storage> http
|
|
URL of http host to connect to
|
|
Choose a number from below, or type in your own value
|
|
1 / Connect to example.com
|
|
\[rs] \[dq]https://example.com\[dq]
|
|
url> https://beta.rclone.org
|
|
Remote config
|
|
\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-
|
|
[remote]
|
|
url = https://beta.rclone.org
|
|
\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-
|
|
y) Yes this is OK
|
|
e) Edit this remote
|
|
d) Delete this remote
|
|
y/e/d> y
|
|
Current remotes:
|
|
|
|
Name Type
|
|
==== ====
|
|
remote http
|
|
|
|
e) Edit existing remote
|
|
n) New remote
|
|
d) Delete remote
|
|
r) Rename remote
|
|
c) Copy remote
|
|
s) Set configuration password
|
|
q) Quit config
|
|
e/n/d/r/c/s/q> q
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
This remote is called \f[C]remote\f[R] and can now be used like this
|
|
.PP
|
|
See all the top level directories
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone lsd remote:
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
List the contents of a directory
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone ls remote:directory
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
Sync the remote \f[C]directory\f[R] to \f[C]/home/local/directory\f[R],
|
|
deleting any excess files.
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone sync \-i remote:directory /home/local/directory
|
|
\f[R]
|
|
.fi
|
|
.SS Read only
|
|
.PP
|
|
This remote is read only \- you can\[aq]t upload files to an HTTP
|
|
server.
|
|
.SS Modified time
|
|
.PP
|
|
Most HTTP servers store time accurate to 1 second.
|
|
.SS Checksum
|
|
.PP
|
|
No checksums are stored.
|
|
.SS Usage without a config file
|
|
.PP
|
|
Since the http remote only has one config parameter it is easy to use
|
|
without a config file:
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone lsd \-\-http\-url https://beta.rclone.org :http:
|
|
\f[R]
|
|
.fi
|
|
.SS Standard Options
|
|
.PP
|
|
Here are the standard options specific to http (http Connection).
|
|
.SS \-\-http\-url
|
|
.PP
|
|
URL of http host to connect to
|
|
.IP \[bu] 2
|
|
Config: url
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_HTTP_URL
|
|
.IP \[bu] 2
|
|
Type: string
|
|
.IP \[bu] 2
|
|
Default: \[dq]\[dq]
|
|
.IP \[bu] 2
|
|
Examples:
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
\[dq]https://example.com\[dq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Connect to example.com
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[dq]https://user:pass\[at]example.com\[dq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Connect to example.com using a username and password
|
|
.RE
|
|
.RE
|
|
.SS Advanced Options
|
|
.PP
|
|
Here are the advanced options specific to http (http Connection).
|
|
.SS \-\-http\-headers
|
|
.PP
|
|
Set HTTP headers for all transactions
|
|
.PP
|
|
Use this to set additional HTTP headers for all transactions
|
|
.PP
|
|
The input format is comma separated list of key,value pairs.
|
|
Standard CSV encoding (https://godoc.org/encoding/csv) may be used.
|
|
.PP
|
|
For example to set a Cookie use \[aq]Cookie,name=value\[aq], or
|
|
\[aq]\[dq]Cookie\[dq],\[dq]name=value\[dq]\[aq].
|
|
.PP
|
|
You can set multiple headers, e.g.
|
|
\[aq]\[dq]Cookie\[dq],\[dq]name=value\[dq],\[dq]Authorization\[dq],\[dq]xxx\[dq]\[aq].
|
|
.IP \[bu] 2
|
|
Config: headers
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_HTTP_HEADERS
|
|
.IP \[bu] 2
|
|
Type: CommaSepList
|
|
.IP \[bu] 2
|
|
Default:
|
|
.SS \-\-http\-no\-slash
|
|
.PP
|
|
Set this if the site doesn\[aq]t end directories with /
|
|
.PP
|
|
Use this if your target website does not use / on the end of
|
|
directories.
|
|
.PP
|
|
A / on the end of a path is how rclone normally tells the difference
|
|
between files and directories.
|
|
If this flag is set, then rclone will treat all files with
|
|
Content\-Type: text/html as directories and read URLs from them rather
|
|
than downloading them.
|
|
.PP
|
|
Note that this may cause rclone to confuse genuine HTML files with
|
|
directories.
|
|
.IP \[bu] 2
|
|
Config: no_slash
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_HTTP_NO_SLASH
|
|
.IP \[bu] 2
|
|
Type: bool
|
|
.IP \[bu] 2
|
|
Default: false
|
|
.SS \-\-http\-no\-head
|
|
.PP
|
|
Don\[aq]t use HEAD requests to find file sizes in dir listing
|
|
.PP
|
|
If your site is being very slow to load then you can try this option.
|
|
Normally rclone does a HEAD request for each potential file in a
|
|
directory listing to:
|
|
.IP \[bu] 2
|
|
find its size
|
|
.IP \[bu] 2
|
|
check it really exists
|
|
.IP \[bu] 2
|
|
check to see if it is a directory
|
|
.PP
|
|
If you set this option, rclone will not do the HEAD request.
|
|
This will mean
|
|
.IP \[bu] 2
|
|
directory listings are much quicker
|
|
.IP \[bu] 2
|
|
rclone won\[aq]t have the times or sizes of any files
|
|
.IP \[bu] 2
|
|
some files that don\[aq]t exist may be in the listing
|
|
.IP \[bu] 2
|
|
Config: no_head
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_HTTP_NO_HEAD
|
|
.IP \[bu] 2
|
|
Type: bool
|
|
.IP \[bu] 2
|
|
Default: false
|
|
.SS Limitations
|
|
.PP
|
|
\f[C]rclone about\f[R] is not supported by the HTTP backend.
|
|
Backends without this capability cannot determine free space for an
|
|
rclone mount or use policy \f[C]mfs\f[R] (most free space) as a member
|
|
of an rclone union remote.
|
|
.PP
|
|
See List of backends that do not support rclone
|
|
about (https://rclone.org/overview/#optional-features) See rclone
|
|
about (https://rclone.org/commands/rclone_about/)
|
|
.SS Hubic
|
|
.PP
|
|
Paths are specified as \f[C]remote:path\f[R]
|
|
.PP
|
|
Paths are specified as \f[C]remote:container\f[R] (or \f[C]remote:\f[R]
|
|
for the \f[C]lsd\f[R] command.) You may put subdirectories in too, e.g.
|
|
\f[C]remote:container/path/to/dir\f[R].
|
|
.PP
|
|
The initial setup for Hubic involves getting a token from Hubic which
|
|
you need to do in your browser.
|
|
\f[C]rclone config\f[R] walks you through it.
|
|
.PP
|
|
Here is an example of how to make a remote called \f[C]remote\f[R].
|
|
First run:
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone config
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
This will guide you through an interactive setup process:
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
n) New remote
|
|
s) Set configuration password
|
|
n/s> n
|
|
name> remote
|
|
Type of storage to configure.
|
|
Choose a number from below, or type in your own value
|
|
[snip]
|
|
XX / Hubic
|
|
\[rs] \[dq]hubic\[dq]
|
|
[snip]
|
|
Storage> hubic
|
|
Hubic Client Id \- leave blank normally.
|
|
client_id>
|
|
Hubic Client Secret \- leave blank normally.
|
|
client_secret>
|
|
Remote config
|
|
Use auto config?
|
|
* Say Y if not sure
|
|
* Say N if you are working on a remote or headless machine
|
|
y) Yes
|
|
n) No
|
|
y/n> y
|
|
If your browser doesn\[aq]t open automatically go to the following link: http://127.0.0.1:53682/auth
|
|
Log in and authorize rclone for access
|
|
Waiting for code...
|
|
Got code
|
|
\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-
|
|
[remote]
|
|
client_id =
|
|
client_secret =
|
|
token = {\[dq]access_token\[dq]:\[dq]XXXXXX\[dq]}
|
|
\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-
|
|
y) Yes this is OK
|
|
e) Edit this remote
|
|
d) Delete this remote
|
|
y/e/d> y
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
See the remote setup docs (https://rclone.org/remote_setup/) for how to
|
|
set it up on a machine with no Internet browser available.
|
|
.PP
|
|
Note that rclone runs a webserver on your local machine to collect the
|
|
token as returned from Hubic.
|
|
This only runs from the moment it opens your browser to the moment you
|
|
get back the verification code.
|
|
This is on \f[C]http://127.0.0.1:53682/\f[R] and this it may require you
|
|
to unblock it temporarily if you are running a host firewall.
|
|
.PP
|
|
Once configured you can then use \f[C]rclone\f[R] like this,
|
|
.PP
|
|
List containers in the top level of your Hubic
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone lsd remote:
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
List all the files in your Hubic
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone ls remote:
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
To copy a local directory to an Hubic directory called backup
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone copy /home/source remote:backup
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
If you want the directory to be visible in the official \f[I]Hubic
|
|
browser\f[R], you need to copy your files to the \f[C]default\f[R]
|
|
directory
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone copy /home/source remote:default/backup
|
|
\f[R]
|
|
.fi
|
|
.SS \-\-fast\-list
|
|
.PP
|
|
This remote supports \f[C]\-\-fast\-list\f[R] which allows you to use
|
|
fewer transactions in exchange for more memory.
|
|
See the rclone docs (https://rclone.org/docs/#fast-list) for more
|
|
details.
|
|
.SS Modified time
|
|
.PP
|
|
The modified time is stored as metadata on the object as
|
|
\f[C]X\-Object\-Meta\-Mtime\f[R] as floating point since the epoch
|
|
accurate to 1 ns.
|
|
.PP
|
|
This is a de facto standard (used in the official python\-swiftclient
|
|
amongst others) for storing the modification time for an object.
|
|
.PP
|
|
Note that Hubic wraps the Swift backend, so most of the properties of
|
|
are the same.
|
|
.SS Standard Options
|
|
.PP
|
|
Here are the standard options specific to hubic (Hubic).
|
|
.SS \-\-hubic\-client\-id
|
|
.PP
|
|
OAuth Client Id Leave blank normally.
|
|
.IP \[bu] 2
|
|
Config: client_id
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_HUBIC_CLIENT_ID
|
|
.IP \[bu] 2
|
|
Type: string
|
|
.IP \[bu] 2
|
|
Default: \[dq]\[dq]
|
|
.SS \-\-hubic\-client\-secret
|
|
.PP
|
|
OAuth Client Secret Leave blank normally.
|
|
.IP \[bu] 2
|
|
Config: client_secret
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_HUBIC_CLIENT_SECRET
|
|
.IP \[bu] 2
|
|
Type: string
|
|
.IP \[bu] 2
|
|
Default: \[dq]\[dq]
|
|
.SS Advanced Options
|
|
.PP
|
|
Here are the advanced options specific to hubic (Hubic).
|
|
.SS \-\-hubic\-token
|
|
.PP
|
|
OAuth Access Token as a JSON blob.
|
|
.IP \[bu] 2
|
|
Config: token
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_HUBIC_TOKEN
|
|
.IP \[bu] 2
|
|
Type: string
|
|
.IP \[bu] 2
|
|
Default: \[dq]\[dq]
|
|
.SS \-\-hubic\-auth\-url
|
|
.PP
|
|
Auth server URL.
|
|
Leave blank to use the provider defaults.
|
|
.IP \[bu] 2
|
|
Config: auth_url
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_HUBIC_AUTH_URL
|
|
.IP \[bu] 2
|
|
Type: string
|
|
.IP \[bu] 2
|
|
Default: \[dq]\[dq]
|
|
.SS \-\-hubic\-token\-url
|
|
.PP
|
|
Token server url.
|
|
Leave blank to use the provider defaults.
|
|
.IP \[bu] 2
|
|
Config: token_url
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_HUBIC_TOKEN_URL
|
|
.IP \[bu] 2
|
|
Type: string
|
|
.IP \[bu] 2
|
|
Default: \[dq]\[dq]
|
|
.SS \-\-hubic\-chunk\-size
|
|
.PP
|
|
Above this size files will be chunked into a _segments container.
|
|
.PP
|
|
Above this size files will be chunked into a _segments container.
|
|
The default for this is 5GB which is its maximum value.
|
|
.IP \[bu] 2
|
|
Config: chunk_size
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_HUBIC_CHUNK_SIZE
|
|
.IP \[bu] 2
|
|
Type: SizeSuffix
|
|
.IP \[bu] 2
|
|
Default: 5G
|
|
.SS \-\-hubic\-no\-chunk
|
|
.PP
|
|
Don\[aq]t chunk files during streaming upload.
|
|
.PP
|
|
When doing streaming uploads (e.g.
|
|
using rcat or mount) setting this flag will cause the swift backend to
|
|
not upload chunked files.
|
|
.PP
|
|
This will limit the maximum upload size to 5GB.
|
|
However non chunked files are easier to deal with and have an MD5SUM.
|
|
.PP
|
|
Rclone will still chunk files bigger than chunk_size when doing normal
|
|
copy operations.
|
|
.IP \[bu] 2
|
|
Config: no_chunk
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_HUBIC_NO_CHUNK
|
|
.IP \[bu] 2
|
|
Type: bool
|
|
.IP \[bu] 2
|
|
Default: false
|
|
.SS \-\-hubic\-encoding
|
|
.PP
|
|
This sets the encoding for the backend.
|
|
.PP
|
|
See: the encoding section in the
|
|
overview (https://rclone.org/overview/#encoding) for more info.
|
|
.IP \[bu] 2
|
|
Config: encoding
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_HUBIC_ENCODING
|
|
.IP \[bu] 2
|
|
Type: MultiEncoder
|
|
.IP \[bu] 2
|
|
Default: Slash,InvalidUtf8
|
|
.SS Limitations
|
|
.PP
|
|
This uses the normal OpenStack Swift mechanism to refresh the Swift API
|
|
credentials and ignores the expires field returned by the Hubic API.
|
|
.PP
|
|
The Swift API doesn\[aq]t return a correct MD5SUM for segmented files
|
|
(Dynamic or Static Large Objects) so rclone won\[aq]t check or use the
|
|
MD5SUM for these.
|
|
.SS Jottacloud
|
|
.PP
|
|
Jottacloud is a cloud storage service provider from a Norwegian company,
|
|
using its own datacenters in Norway.
|
|
.PP
|
|
In addition to the official service at
|
|
jottacloud.com (https://www.jottacloud.com/), there are also several
|
|
whitelabel versions which should work with this backend.
|
|
.PP
|
|
Paths are specified as \f[C]remote:path\f[R]
|
|
.PP
|
|
Paths may be as deep as required, e.g.
|
|
\f[C]remote:directory/subdirectory\f[R].
|
|
.SS Setup
|
|
.SS Default Setup
|
|
.PP
|
|
To configure Jottacloud you will need to generate a personal security
|
|
token in the Jottacloud web interface.
|
|
You will the option to do in your account security
|
|
settings (https://www.jottacloud.com/web/secure) (for whitelabel version
|
|
you need to find this page in its web interface).
|
|
Note that the web interface may refer to this token as a JottaCli token.
|
|
.SS Legacy Setup
|
|
.PP
|
|
If you are using one of the whitelabel versions (Elgiganten, Com Hem
|
|
Cloud) you may not have the option to generate a CLI token.
|
|
In this case you\[aq]ll have to use the legacy authentication.
|
|
To to this select yes when the setup asks for legacy authentication and
|
|
enter your username and password.
|
|
The rest of the setup is identical to the default setup.
|
|
.SS Telia Cloud Setup
|
|
.PP
|
|
Similar to other whitelabel versions Telia Cloud doesn\[aq]t offer the
|
|
option of creating a CLI token, and additionally uses a separate
|
|
authentication flow where the username is generated internally.
|
|
To setup rclone to use Telia Cloud, choose Telia Cloud authentication in
|
|
the setup.
|
|
The rest of the setup is identical to the default setup.
|
|
.SS Example
|
|
.PP
|
|
Here is an example of how to make a remote called \f[C]remote\f[R] with
|
|
the default setup.
|
|
First run:
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone config
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
This will guide you through an interactive setup process:
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
No remotes found \- make a new one
|
|
n) New remote
|
|
s) Set configuration password
|
|
q) Quit config
|
|
n/s/q> n
|
|
name> remote
|
|
Type of storage to configure.
|
|
Enter a string value. Press Enter for the default (\[dq]\[dq]).
|
|
Choose a number from below, or type in your own value
|
|
[snip]
|
|
XX / Jottacloud
|
|
\[rs] \[dq]jottacloud\[dq]
|
|
[snip]
|
|
Storage> jottacloud
|
|
** See help for jottacloud backend at: https://rclone.org/jottacloud/ **
|
|
|
|
Edit advanced config? (y/n)
|
|
y) Yes
|
|
n) No
|
|
y/n> n
|
|
Remote config
|
|
Use legacy authentication?.
|
|
This is only required for certain whitelabel versions of Jottacloud and not recommended for normal users.
|
|
y) Yes
|
|
n) No (default)
|
|
y/n> n
|
|
|
|
Generate a personal login token here: https://www.jottacloud.com/web/secure
|
|
Login Token> <your token here>
|
|
|
|
Do you want to use a non standard device/mountpoint e.g. for accessing files uploaded using the official Jottacloud client?
|
|
|
|
y) Yes
|
|
n) No
|
|
y/n> y
|
|
Please select the device to use. Normally this will be Jotta
|
|
Choose a number from below, or type in an existing value
|
|
1 > DESKTOP\-3H31129
|
|
2 > Jotta
|
|
Devices> 2
|
|
Please select the mountpoint to user. Normally this will be Archive
|
|
Choose a number from below, or type in an existing value
|
|
1 > Archive
|
|
2 > Links
|
|
3 > Sync
|
|
|
|
Mountpoints> 1
|
|
\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-
|
|
[jotta]
|
|
type = jottacloud
|
|
token = {........}
|
|
device = Jotta
|
|
mountpoint = Archive
|
|
configVersion = 1
|
|
\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-
|
|
y) Yes this is OK
|
|
e) Edit this remote
|
|
d) Delete this remote
|
|
y/e/d> y
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
Once configured you can then use \f[C]rclone\f[R] like this,
|
|
.PP
|
|
List directories in top level of your Jottacloud
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone lsd remote:
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
List all the files in your Jottacloud
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone ls remote:
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
To copy a local directory to an Jottacloud directory called backup
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone copy /home/source remote:backup
|
|
\f[R]
|
|
.fi
|
|
.SS Devices and Mountpoints
|
|
.PP
|
|
The official Jottacloud client registers a device for each computer you
|
|
install it on, and then creates a mountpoint for each folder you select
|
|
for Backup.
|
|
The web interface uses a special device called Jotta for the Archive and
|
|
Sync mountpoints.
|
|
In most cases you\[aq]ll want to use the Jotta/Archive
|
|
device/mountpoint, however if you want to access files uploaded by any
|
|
of the official clients rclone provides the option to select other
|
|
devices and mountpoints during config.
|
|
.PP
|
|
The built\-in Jotta device may also contain several other mountpoints,
|
|
such as: Latest, Links, Shared and Trash.
|
|
These are special mountpoints with a different internal representation
|
|
than the \[dq]regular\[dq] mountpoints.
|
|
Rclone will only to a very limited degree support them.
|
|
Generally you should avoid these, unless you know what you are doing.
|
|
.SS \-\-fast\-list
|
|
.PP
|
|
This remote supports \f[C]\-\-fast\-list\f[R] which allows you to use
|
|
fewer transactions in exchange for more memory.
|
|
See the rclone docs (https://rclone.org/docs/#fast-list) for more
|
|
details.
|
|
.PP
|
|
Note that the implementation in Jottacloud always uses only a single API
|
|
request to get the entire list, so for large folders this could lead to
|
|
long wait time before the first results are shown.
|
|
.SS Modified time and hashes
|
|
.PP
|
|
Jottacloud allows modification times to be set on objects accurate to 1
|
|
second.
|
|
These will be used to detect whether objects need syncing or not.
|
|
.PP
|
|
Jottacloud supports MD5 type hashes, so you can use the
|
|
\f[C]\-\-checksum\f[R] flag.
|
|
.PP
|
|
Note that Jottacloud requires the MD5 hash before upload so if the
|
|
source does not have an MD5 checksum then the file will be cached
|
|
temporarily on disk (wherever the \f[C]TMPDIR\f[R] environment variable
|
|
points to) before it is uploaded.
|
|
Small files will be cached in memory \- see the
|
|
\-\-jottacloud\-md5\-memory\-limit flag.
|
|
When uploading from local disk the source checksum is always available,
|
|
so this does not apply.
|
|
Starting with rclone version 1.52 the same is true for crypted remotes
|
|
(in older versions the crypt backend would not calculate hashes for
|
|
uploads from local disk, so the Jottacloud backend had to do it as
|
|
described above).
|
|
.SS Restricted filename characters
|
|
.PP
|
|
In addition to the default restricted characters
|
|
set (https://rclone.org/overview/#restricted-characters) the following
|
|
characters are also replaced:
|
|
.PP
|
|
.TS
|
|
tab(@);
|
|
l c c.
|
|
T{
|
|
Character
|
|
T}@T{
|
|
Value
|
|
T}@T{
|
|
Replacement
|
|
T}
|
|
_
|
|
T{
|
|
\[dq]
|
|
T}@T{
|
|
0x22
|
|
T}@T{
|
|
\[uFF02]
|
|
T}
|
|
T{
|
|
*
|
|
T}@T{
|
|
0x2A
|
|
T}@T{
|
|
\[uFF0A]
|
|
T}
|
|
T{
|
|
:
|
|
T}@T{
|
|
0x3A
|
|
T}@T{
|
|
\[uFF1A]
|
|
T}
|
|
T{
|
|
<
|
|
T}@T{
|
|
0x3C
|
|
T}@T{
|
|
\[uFF1C]
|
|
T}
|
|
T{
|
|
>
|
|
T}@T{
|
|
0x3E
|
|
T}@T{
|
|
\[uFF1E]
|
|
T}
|
|
T{
|
|
?
|
|
T}@T{
|
|
0x3F
|
|
T}@T{
|
|
\[uFF1F]
|
|
T}
|
|
T{
|
|
|
|
|
T}@T{
|
|
0x7C
|
|
T}@T{
|
|
\[uFF5C]
|
|
T}
|
|
.TE
|
|
.PP
|
|
Invalid UTF\-8 bytes will also be
|
|
replaced (https://rclone.org/overview/#invalid-utf8), as they can\[aq]t
|
|
be used in XML strings.
|
|
.SS Deleting files
|
|
.PP
|
|
By default rclone will send all files to the trash when deleting files.
|
|
They will be permanently deleted automatically after 30 days.
|
|
You may bypass the trash and permanently delete files immediately by
|
|
using the \-\-jottacloud\-hard\-delete flag, or set the equivalent
|
|
environment variable.
|
|
Emptying the trash is supported by the
|
|
cleanup (https://rclone.org/commands/rclone_cleanup/) command.
|
|
.SS Versions
|
|
.PP
|
|
Jottacloud supports file versioning.
|
|
When rclone uploads a new version of a file it creates a new version of
|
|
it.
|
|
Currently rclone only supports retrieving the current version but older
|
|
versions can be accessed via the Jottacloud Website.
|
|
.SS Quota information
|
|
.PP
|
|
To view your current quota you can use the
|
|
\f[C]rclone about remote:\f[R] command which will display your usage
|
|
limit (unless it is unlimited) and the current usage.
|
|
.SS Advanced Options
|
|
.PP
|
|
Here are the advanced options specific to jottacloud (Jottacloud).
|
|
.SS \-\-jottacloud\-md5\-memory\-limit
|
|
.PP
|
|
Files bigger than this will be cached on disk to calculate the MD5 if
|
|
required.
|
|
.IP \[bu] 2
|
|
Config: md5_memory_limit
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_JOTTACLOUD_MD5_MEMORY_LIMIT
|
|
.IP \[bu] 2
|
|
Type: SizeSuffix
|
|
.IP \[bu] 2
|
|
Default: 10M
|
|
.SS \-\-jottacloud\-trashed\-only
|
|
.PP
|
|
Only show files that are in the trash.
|
|
This will show trashed files in their original directory structure.
|
|
.IP \[bu] 2
|
|
Config: trashed_only
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_JOTTACLOUD_TRASHED_ONLY
|
|
.IP \[bu] 2
|
|
Type: bool
|
|
.IP \[bu] 2
|
|
Default: false
|
|
.SS \-\-jottacloud\-hard\-delete
|
|
.PP
|
|
Delete files permanently rather than putting them into the trash.
|
|
.IP \[bu] 2
|
|
Config: hard_delete
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_JOTTACLOUD_HARD_DELETE
|
|
.IP \[bu] 2
|
|
Type: bool
|
|
.IP \[bu] 2
|
|
Default: false
|
|
.SS \-\-jottacloud\-upload\-resume\-limit
|
|
.PP
|
|
Files bigger than this can be resumed if the upload fail\[aq]s.
|
|
.IP \[bu] 2
|
|
Config: upload_resume_limit
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_JOTTACLOUD_UPLOAD_RESUME_LIMIT
|
|
.IP \[bu] 2
|
|
Type: SizeSuffix
|
|
.IP \[bu] 2
|
|
Default: 10M
|
|
.SS \-\-jottacloud\-encoding
|
|
.PP
|
|
This sets the encoding for the backend.
|
|
.PP
|
|
See: the encoding section in the
|
|
overview (https://rclone.org/overview/#encoding) for more info.
|
|
.IP \[bu] 2
|
|
Config: encoding
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_JOTTACLOUD_ENCODING
|
|
.IP \[bu] 2
|
|
Type: MultiEncoder
|
|
.IP \[bu] 2
|
|
Default:
|
|
Slash,LtGt,DoubleQuote,Colon,Question,Asterisk,Pipe,Del,Ctl,InvalidUtf8,Dot
|
|
.SS Limitations
|
|
.PP
|
|
Note that Jottacloud is case insensitive so you can\[aq]t have a file
|
|
called \[dq]Hello.doc\[dq] and one called \[dq]hello.doc\[dq].
|
|
.PP
|
|
There are quite a few characters that can\[aq]t be in Jottacloud file
|
|
names.
|
|
Rclone will map these names to and from an identical looking unicode
|
|
equivalent.
|
|
For example if a file has a ?
|
|
in it will be mapped to \[uFF1F] instead.
|
|
.PP
|
|
Jottacloud only supports filenames up to 255 characters in length.
|
|
.SS Troubleshooting
|
|
.PP
|
|
Jottacloud exhibits some inconsistent behaviours regarding deleted files
|
|
and folders which may cause Copy, Move and DirMove operations to
|
|
previously deleted paths to fail.
|
|
Emptying the trash should help in such cases.
|
|
.SS Koofr
|
|
.PP
|
|
Paths are specified as \f[C]remote:path\f[R]
|
|
.PP
|
|
Paths may be as deep as required, e.g.
|
|
\f[C]remote:directory/subdirectory\f[R].
|
|
.PP
|
|
The initial setup for Koofr involves creating an application password
|
|
for rclone.
|
|
You can do that by opening the Koofr web
|
|
application (https://app.koofr.net/app/admin/preferences/password),
|
|
giving the password a nice name like \f[C]rclone\f[R] and clicking on
|
|
generate.
|
|
.PP
|
|
Here is an example of how to make a remote called \f[C]koofr\f[R].
|
|
First run:
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone config
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
This will guide you through an interactive setup process:
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
No remotes found \- make a new one
|
|
n) New remote
|
|
s) Set configuration password
|
|
q) Quit config
|
|
n/s/q> n
|
|
name> koofr
|
|
Type of storage to configure.
|
|
Enter a string value. Press Enter for the default (\[dq]\[dq]).
|
|
Choose a number from below, or type in your own value
|
|
[snip]
|
|
XX / Koofr
|
|
\[rs] \[dq]koofr\[dq]
|
|
[snip]
|
|
Storage> koofr
|
|
** See help for koofr backend at: https://rclone.org/koofr/ **
|
|
|
|
Your Koofr user name
|
|
Enter a string value. Press Enter for the default (\[dq]\[dq]).
|
|
user> USER\[at]NAME
|
|
Your Koofr password for rclone (generate one at https://app.koofr.net/app/admin/preferences/password)
|
|
y) Yes type in my own password
|
|
g) Generate random password
|
|
y/g> y
|
|
Enter the password:
|
|
password:
|
|
Confirm the password:
|
|
password:
|
|
Edit advanced config? (y/n)
|
|
y) Yes
|
|
n) No
|
|
y/n> n
|
|
Remote config
|
|
\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-
|
|
[koofr]
|
|
type = koofr
|
|
baseurl = https://app.koofr.net
|
|
user = USER\[at]NAME
|
|
password = *** ENCRYPTED ***
|
|
\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-
|
|
y) Yes this is OK
|
|
e) Edit this remote
|
|
d) Delete this remote
|
|
y/e/d> y
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
You can choose to edit advanced config in order to enter your own
|
|
service URL if you use an on\-premise or white label Koofr instance, or
|
|
choose an alternative mount instead of your primary storage.
|
|
.PP
|
|
Once configured you can then use \f[C]rclone\f[R] like this,
|
|
.PP
|
|
List directories in top level of your Koofr
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone lsd koofr:
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
List all the files in your Koofr
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone ls koofr:
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
To copy a local directory to an Koofr directory called backup
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone copy /home/source remote:backup
|
|
\f[R]
|
|
.fi
|
|
.SS Restricted filename characters
|
|
.PP
|
|
In addition to the default restricted characters
|
|
set (https://rclone.org/overview/#restricted-characters) the following
|
|
characters are also replaced:
|
|
.PP
|
|
.TS
|
|
tab(@);
|
|
l c c.
|
|
T{
|
|
Character
|
|
T}@T{
|
|
Value
|
|
T}@T{
|
|
Replacement
|
|
T}
|
|
_
|
|
T{
|
|
\[rs]
|
|
T}@T{
|
|
0x5C
|
|
T}@T{
|
|
\[uFF3C]
|
|
T}
|
|
.TE
|
|
.PP
|
|
Invalid UTF\-8 bytes will also be
|
|
replaced (https://rclone.org/overview/#invalid-utf8), as they can\[aq]t
|
|
be used in XML strings.
|
|
.SS Standard Options
|
|
.PP
|
|
Here are the standard options specific to koofr (Koofr).
|
|
.SS \-\-koofr\-user
|
|
.PP
|
|
Your Koofr user name
|
|
.IP \[bu] 2
|
|
Config: user
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_KOOFR_USER
|
|
.IP \[bu] 2
|
|
Type: string
|
|
.IP \[bu] 2
|
|
Default: \[dq]\[dq]
|
|
.SS \-\-koofr\-password
|
|
.PP
|
|
Your Koofr password for rclone (generate one at
|
|
https://app.koofr.net/app/admin/preferences/password)
|
|
.PP
|
|
\f[B]NB\f[R] Input to this must be obscured \- see rclone
|
|
obscure (https://rclone.org/commands/rclone_obscure/).
|
|
.IP \[bu] 2
|
|
Config: password
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_KOOFR_PASSWORD
|
|
.IP \[bu] 2
|
|
Type: string
|
|
.IP \[bu] 2
|
|
Default: \[dq]\[dq]
|
|
.SS Advanced Options
|
|
.PP
|
|
Here are the advanced options specific to koofr (Koofr).
|
|
.SS \-\-koofr\-endpoint
|
|
.PP
|
|
The Koofr API endpoint to use
|
|
.IP \[bu] 2
|
|
Config: endpoint
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_KOOFR_ENDPOINT
|
|
.IP \[bu] 2
|
|
Type: string
|
|
.IP \[bu] 2
|
|
Default: \[dq]https://app.koofr.net\[dq]
|
|
.SS \-\-koofr\-mountid
|
|
.PP
|
|
Mount ID of the mount to use.
|
|
If omitted, the primary mount is used.
|
|
.IP \[bu] 2
|
|
Config: mountid
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_KOOFR_MOUNTID
|
|
.IP \[bu] 2
|
|
Type: string
|
|
.IP \[bu] 2
|
|
Default: \[dq]\[dq]
|
|
.SS \-\-koofr\-setmtime
|
|
.PP
|
|
Does the backend support setting modification time.
|
|
Set this to false if you use a mount ID that points to a Dropbox or
|
|
Amazon Drive backend.
|
|
.IP \[bu] 2
|
|
Config: setmtime
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_KOOFR_SETMTIME
|
|
.IP \[bu] 2
|
|
Type: bool
|
|
.IP \[bu] 2
|
|
Default: true
|
|
.SS \-\-koofr\-encoding
|
|
.PP
|
|
This sets the encoding for the backend.
|
|
.PP
|
|
See: the encoding section in the
|
|
overview (https://rclone.org/overview/#encoding) for more info.
|
|
.IP \[bu] 2
|
|
Config: encoding
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_KOOFR_ENCODING
|
|
.IP \[bu] 2
|
|
Type: MultiEncoder
|
|
.IP \[bu] 2
|
|
Default: Slash,BackSlash,Del,Ctl,InvalidUtf8,Dot
|
|
.SS Limitations
|
|
.PP
|
|
Note that Koofr is case insensitive so you can\[aq]t have a file called
|
|
\[dq]Hello.doc\[dq] and one called \[dq]hello.doc\[dq].
|
|
.SS Mail.ru Cloud
|
|
.PP
|
|
Mail.ru Cloud (https://cloud.mail.ru/) is a cloud storage provided by a
|
|
Russian internet company Mail.Ru Group (https://mail.ru).
|
|
The official desktop client is Disk\-O: (https://disk-o.cloud/en),
|
|
available on Windows and Mac OS.
|
|
.PP
|
|
Currently it is recommended to disable 2FA on Mail.ru accounts intended
|
|
for rclone until it gets eventually implemented.
|
|
.SS Features highlights
|
|
.IP \[bu] 2
|
|
Paths may be as deep as required, e.g.
|
|
\f[C]remote:directory/subdirectory\f[R]
|
|
.IP \[bu] 2
|
|
Files have a \f[C]last modified time\f[R] property, directories
|
|
don\[aq]t
|
|
.IP \[bu] 2
|
|
Deleted files are by default moved to the trash
|
|
.IP \[bu] 2
|
|
Files and directories can be shared via public links
|
|
.IP \[bu] 2
|
|
Partial uploads or streaming are not supported, file size must be known
|
|
before upload
|
|
.IP \[bu] 2
|
|
Maximum file size is limited to 2G for a free account, unlimited for
|
|
paid accounts
|
|
.IP \[bu] 2
|
|
Storage keeps hash for all files and performs transparent deduplication,
|
|
the hash algorithm is a modified SHA1
|
|
.IP \[bu] 2
|
|
If a particular file is already present in storage, one can quickly
|
|
submit file hash instead of long file upload (this optimization is
|
|
supported by rclone)
|
|
.SS Configuration
|
|
.PP
|
|
Here is an example of making a mailru configuration.
|
|
First create a Mail.ru Cloud account and choose a tariff, then run
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone config
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
This will guide you through an interactive setup process:
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
No remotes found \- make a new one
|
|
n) New remote
|
|
s) Set configuration password
|
|
q) Quit config
|
|
n/s/q> n
|
|
name> remote
|
|
Type of storage to configure.
|
|
Type of storage to configure.
|
|
Enter a string value. Press Enter for the default (\[dq]\[dq]).
|
|
Choose a number from below, or type in your own value
|
|
[snip]
|
|
XX / Mail.ru Cloud
|
|
\[rs] \[dq]mailru\[dq]
|
|
[snip]
|
|
Storage> mailru
|
|
User name (usually email)
|
|
Enter a string value. Press Enter for the default (\[dq]\[dq]).
|
|
user> username\[at]mail.ru
|
|
Password
|
|
y) Yes type in my own password
|
|
g) Generate random password
|
|
y/g> y
|
|
Enter the password:
|
|
password:
|
|
Confirm the password:
|
|
password:
|
|
Skip full upload if there is another file with same data hash.
|
|
This feature is called \[dq]speedup\[dq] or \[dq]put by hash\[dq]. It is especially efficient
|
|
in case of generally available files like popular books, video or audio clips
|
|
[snip]
|
|
Enter a boolean value (true or false). Press Enter for the default (\[dq]true\[dq]).
|
|
Choose a number from below, or type in your own value
|
|
1 / Enable
|
|
\[rs] \[dq]true\[dq]
|
|
2 / Disable
|
|
\[rs] \[dq]false\[dq]
|
|
speedup_enable> 1
|
|
Edit advanced config? (y/n)
|
|
y) Yes
|
|
n) No
|
|
y/n> n
|
|
Remote config
|
|
\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-
|
|
[remote]
|
|
type = mailru
|
|
user = username\[at]mail.ru
|
|
pass = *** ENCRYPTED ***
|
|
speedup_enable = true
|
|
\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-
|
|
y) Yes this is OK
|
|
e) Edit this remote
|
|
d) Delete this remote
|
|
y/e/d> y
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
Configuration of this backend does not require a local web browser.
|
|
You can use the configured backend as shown below:
|
|
.PP
|
|
See top level directories
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone lsd remote:
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
Make a new directory
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone mkdir remote:directory
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
List the contents of a directory
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone ls remote:directory
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
Sync \f[C]/home/local/directory\f[R] to the remote path, deleting any
|
|
excess files in the path.
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone sync \-i /home/local/directory remote:directory
|
|
\f[R]
|
|
.fi
|
|
.SS Modified time
|
|
.PP
|
|
Files support a modification time attribute with up to 1 second
|
|
precision.
|
|
Directories do not have a modification time, which is shown as \[dq]Jan
|
|
1 1970\[dq].
|
|
.SS Hash checksums
|
|
.PP
|
|
Hash sums use a custom Mail.ru algorithm based on SHA1.
|
|
If file size is less than or equal to the SHA1 block size (20 bytes),
|
|
its hash is simply its data right\-padded with zero bytes.
|
|
Hash sum of a larger file is computed as a SHA1 sum of the file data
|
|
bytes concatenated with a decimal representation of the data length.
|
|
.SS Emptying Trash
|
|
.PP
|
|
Removing a file or directory actually moves it to the trash, which is
|
|
not visible to rclone but can be seen in a web browser.
|
|
The trashed file still occupies part of total quota.
|
|
If you wish to empty your trash and free some quota, you can use the
|
|
\f[C]rclone cleanup remote:\f[R] command, which will permanently delete
|
|
all your trashed files.
|
|
This command does not take any path arguments.
|
|
.SS Quota information
|
|
.PP
|
|
To view your current quota you can use the
|
|
\f[C]rclone about remote:\f[R] command which will display your usage
|
|
limit (quota) and the current usage.
|
|
.SS Restricted filename characters
|
|
.PP
|
|
In addition to the default restricted characters
|
|
set (https://rclone.org/overview/#restricted-characters) the following
|
|
characters are also replaced:
|
|
.PP
|
|
.TS
|
|
tab(@);
|
|
l c c.
|
|
T{
|
|
Character
|
|
T}@T{
|
|
Value
|
|
T}@T{
|
|
Replacement
|
|
T}
|
|
_
|
|
T{
|
|
\[dq]
|
|
T}@T{
|
|
0x22
|
|
T}@T{
|
|
\[uFF02]
|
|
T}
|
|
T{
|
|
*
|
|
T}@T{
|
|
0x2A
|
|
T}@T{
|
|
\[uFF0A]
|
|
T}
|
|
T{
|
|
:
|
|
T}@T{
|
|
0x3A
|
|
T}@T{
|
|
\[uFF1A]
|
|
T}
|
|
T{
|
|
<
|
|
T}@T{
|
|
0x3C
|
|
T}@T{
|
|
\[uFF1C]
|
|
T}
|
|
T{
|
|
>
|
|
T}@T{
|
|
0x3E
|
|
T}@T{
|
|
\[uFF1E]
|
|
T}
|
|
T{
|
|
?
|
|
T}@T{
|
|
0x3F
|
|
T}@T{
|
|
\[uFF1F]
|
|
T}
|
|
T{
|
|
\[rs]
|
|
T}@T{
|
|
0x5C
|
|
T}@T{
|
|
\[uFF3C]
|
|
T}
|
|
T{
|
|
|
|
|
T}@T{
|
|
0x7C
|
|
T}@T{
|
|
\[uFF5C]
|
|
T}
|
|
.TE
|
|
.PP
|
|
Invalid UTF\-8 bytes will also be
|
|
replaced (https://rclone.org/overview/#invalid-utf8), as they can\[aq]t
|
|
be used in JSON strings.
|
|
.SS Limitations
|
|
.PP
|
|
File size limits depend on your account.
|
|
A single file size is limited by 2G for a free account and unlimited for
|
|
paid tariffs.
|
|
Please refer to the Mail.ru site for the total uploaded size limits.
|
|
.PP
|
|
Note that Mailru is case insensitive so you can\[aq]t have a file called
|
|
\[dq]Hello.doc\[dq] and one called \[dq]hello.doc\[dq].
|
|
.SS Standard Options
|
|
.PP
|
|
Here are the standard options specific to mailru (Mail.ru Cloud).
|
|
.SS \-\-mailru\-user
|
|
.PP
|
|
User name (usually email)
|
|
.IP \[bu] 2
|
|
Config: user
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_MAILRU_USER
|
|
.IP \[bu] 2
|
|
Type: string
|
|
.IP \[bu] 2
|
|
Default: \[dq]\[dq]
|
|
.SS \-\-mailru\-pass
|
|
.PP
|
|
Password
|
|
.PP
|
|
\f[B]NB\f[R] Input to this must be obscured \- see rclone
|
|
obscure (https://rclone.org/commands/rclone_obscure/).
|
|
.IP \[bu] 2
|
|
Config: pass
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_MAILRU_PASS
|
|
.IP \[bu] 2
|
|
Type: string
|
|
.IP \[bu] 2
|
|
Default: \[dq]\[dq]
|
|
.SS \-\-mailru\-speedup\-enable
|
|
.PP
|
|
Skip full upload if there is another file with same data hash.
|
|
This feature is called \[dq]speedup\[dq] or \[dq]put by hash\[dq].
|
|
It is especially efficient in case of generally available files like
|
|
popular books, video or audio clips, because files are searched by hash
|
|
in all accounts of all mailru users.
|
|
It is meaningless and ineffective if source file is unique or encrypted.
|
|
Please note that rclone may need local memory and disk space to
|
|
calculate content hash in advance and decide whether full upload is
|
|
required.
|
|
Also, if rclone does not know file size in advance (e.g.
|
|
in case of streaming or partial uploads), it will not even try this
|
|
optimization.
|
|
.IP \[bu] 2
|
|
Config: speedup_enable
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_MAILRU_SPEEDUP_ENABLE
|
|
.IP \[bu] 2
|
|
Type: bool
|
|
.IP \[bu] 2
|
|
Default: true
|
|
.IP \[bu] 2
|
|
Examples:
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
\[dq]true\[dq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Enable
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[dq]false\[dq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Disable
|
|
.RE
|
|
.RE
|
|
.SS Advanced Options
|
|
.PP
|
|
Here are the advanced options specific to mailru (Mail.ru Cloud).
|
|
.SS \-\-mailru\-speedup\-file\-patterns
|
|
.PP
|
|
Comma separated list of file name patterns eligible for speedup (put by
|
|
hash).
|
|
Patterns are case insensitive and can contain \[aq]*\[aq] or \[aq]?\[aq]
|
|
meta characters.
|
|
.IP \[bu] 2
|
|
Config: speedup_file_patterns
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_MAILRU_SPEEDUP_FILE_PATTERNS
|
|
.IP \[bu] 2
|
|
Type: string
|
|
.IP \[bu] 2
|
|
Default:
|
|
\[dq]\f[I].mkv,\f[R].avi,\f[I].mp4,\f[R].mp3,\f[I].zip,\f[R].gz,\f[I].rar,\f[R].pdf\[dq]
|
|
.IP \[bu] 2
|
|
Examples:
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
\[dq]\[dq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Empty list completely disables speedup (put by hash).
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[dq]*\[dq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
All files will be attempted for speedup.
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[dq]\f[I].mkv,\f[R].avi,\f[I].mp4,\f[R].mp3\[dq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Only common audio/video files will be tried for put by hash.
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[dq]\f[I].zip,\f[R].gz,\f[I].rar,\f[R].pdf\[dq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Only common archives or PDF books will be tried for speedup.
|
|
.RE
|
|
.RE
|
|
.SS \-\-mailru\-speedup\-max\-disk
|
|
.PP
|
|
This option allows you to disable speedup (put by hash) for large files
|
|
(because preliminary hashing can exhaust you RAM or disk space)
|
|
.IP \[bu] 2
|
|
Config: speedup_max_disk
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_MAILRU_SPEEDUP_MAX_DISK
|
|
.IP \[bu] 2
|
|
Type: SizeSuffix
|
|
.IP \[bu] 2
|
|
Default: 3G
|
|
.IP \[bu] 2
|
|
Examples:
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
\[dq]0\[dq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Completely disable speedup (put by hash).
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[dq]1G\[dq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Files larger than 1Gb will be uploaded directly.
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[dq]3G\[dq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Choose this option if you have less than 3Gb free on local disk.
|
|
.RE
|
|
.RE
|
|
.SS \-\-mailru\-speedup\-max\-memory
|
|
.PP
|
|
Files larger than the size given below will always be hashed on disk.
|
|
.IP \[bu] 2
|
|
Config: speedup_max_memory
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_MAILRU_SPEEDUP_MAX_MEMORY
|
|
.IP \[bu] 2
|
|
Type: SizeSuffix
|
|
.IP \[bu] 2
|
|
Default: 32M
|
|
.IP \[bu] 2
|
|
Examples:
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
\[dq]0\[dq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Preliminary hashing will always be done in a temporary disk location.
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[dq]32M\[dq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Do not dedicate more than 32Mb RAM for preliminary hashing.
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[dq]256M\[dq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
You have at most 256Mb RAM free for hash calculations.
|
|
.RE
|
|
.RE
|
|
.SS \-\-mailru\-check\-hash
|
|
.PP
|
|
What should copy do if file checksum is mismatched or invalid
|
|
.IP \[bu] 2
|
|
Config: check_hash
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_MAILRU_CHECK_HASH
|
|
.IP \[bu] 2
|
|
Type: bool
|
|
.IP \[bu] 2
|
|
Default: true
|
|
.IP \[bu] 2
|
|
Examples:
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
\[dq]true\[dq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Fail with error.
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[dq]false\[dq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Ignore and continue.
|
|
.RE
|
|
.RE
|
|
.SS \-\-mailru\-user\-agent
|
|
.PP
|
|
HTTP user agent used internally by client.
|
|
Defaults to \[dq]rclone/VERSION\[dq] or \[dq]\-\-user\-agent\[dq]
|
|
provided on command line.
|
|
.IP \[bu] 2
|
|
Config: user_agent
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_MAILRU_USER_AGENT
|
|
.IP \[bu] 2
|
|
Type: string
|
|
.IP \[bu] 2
|
|
Default: \[dq]\[dq]
|
|
.SS \-\-mailru\-quirks
|
|
.PP
|
|
Comma separated list of internal maintenance flags.
|
|
This option must not be used by an ordinary user.
|
|
It is intended only to facilitate remote troubleshooting of backend
|
|
issues.
|
|
Strict meaning of flags is not documented and not guaranteed to persist
|
|
between releases.
|
|
Quirks will be removed when the backend grows stable.
|
|
Supported quirks: atomicmkdir binlist unknowndirs
|
|
.IP \[bu] 2
|
|
Config: quirks
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_MAILRU_QUIRKS
|
|
.IP \[bu] 2
|
|
Type: string
|
|
.IP \[bu] 2
|
|
Default: \[dq]\[dq]
|
|
.SS \-\-mailru\-encoding
|
|
.PP
|
|
This sets the encoding for the backend.
|
|
.PP
|
|
See: the encoding section in the
|
|
overview (https://rclone.org/overview/#encoding) for more info.
|
|
.IP \[bu] 2
|
|
Config: encoding
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_MAILRU_ENCODING
|
|
.IP \[bu] 2
|
|
Type: MultiEncoder
|
|
.IP \[bu] 2
|
|
Default:
|
|
Slash,LtGt,DoubleQuote,Colon,Question,Asterisk,Pipe,BackSlash,Del,Ctl,InvalidUtf8,Dot
|
|
.SS Mega
|
|
.PP
|
|
Mega (https://mega.nz/) is a cloud storage and file hosting service
|
|
known for its security feature where all files are encrypted locally
|
|
before they are uploaded.
|
|
This prevents anyone (including employees of Mega) from accessing the
|
|
files without knowledge of the key used for encryption.
|
|
.PP
|
|
This is an rclone backend for Mega which supports the file transfer
|
|
features of Mega using the same client side encryption.
|
|
.PP
|
|
Paths are specified as \f[C]remote:path\f[R]
|
|
.PP
|
|
Paths may be as deep as required, e.g.
|
|
\f[C]remote:directory/subdirectory\f[R].
|
|
.PP
|
|
Here is an example of how to make a remote called \f[C]remote\f[R].
|
|
First run:
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone config
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
This will guide you through an interactive setup process:
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
No remotes found \- make a new one
|
|
n) New remote
|
|
s) Set configuration password
|
|
q) Quit config
|
|
n/s/q> n
|
|
name> remote
|
|
Type of storage to configure.
|
|
Choose a number from below, or type in your own value
|
|
[snip]
|
|
XX / Mega
|
|
\[rs] \[dq]mega\[dq]
|
|
[snip]
|
|
Storage> mega
|
|
User name
|
|
user> you\[at]example.com
|
|
Password.
|
|
y) Yes type in my own password
|
|
g) Generate random password
|
|
n) No leave this optional password blank
|
|
y/g/n> y
|
|
Enter the password:
|
|
password:
|
|
Confirm the password:
|
|
password:
|
|
Remote config
|
|
\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-
|
|
[remote]
|
|
type = mega
|
|
user = you\[at]example.com
|
|
pass = *** ENCRYPTED ***
|
|
\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-
|
|
y) Yes this is OK
|
|
e) Edit this remote
|
|
d) Delete this remote
|
|
y/e/d> y
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
\f[B]NOTE:\f[R] The encryption keys need to have been already generated
|
|
after a regular login via the browser, otherwise attempting to use the
|
|
credentials in \f[C]rclone\f[R] will fail.
|
|
.PP
|
|
Once configured you can then use \f[C]rclone\f[R] like this,
|
|
.PP
|
|
List directories in top level of your Mega
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone lsd remote:
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
List all the files in your Mega
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone ls remote:
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
To copy a local directory to an Mega directory called backup
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone copy /home/source remote:backup
|
|
\f[R]
|
|
.fi
|
|
.SS Modified time and hashes
|
|
.PP
|
|
Mega does not support modification times or hashes yet.
|
|
.SS Restricted filename characters
|
|
.PP
|
|
.TS
|
|
tab(@);
|
|
l c c.
|
|
T{
|
|
Character
|
|
T}@T{
|
|
Value
|
|
T}@T{
|
|
Replacement
|
|
T}
|
|
_
|
|
T{
|
|
NUL
|
|
T}@T{
|
|
0x00
|
|
T}@T{
|
|
\[u2400]
|
|
T}
|
|
T{
|
|
/
|
|
T}@T{
|
|
0x2F
|
|
T}@T{
|
|
\[uFF0F]
|
|
T}
|
|
.TE
|
|
.PP
|
|
Invalid UTF\-8 bytes will also be
|
|
replaced (https://rclone.org/overview/#invalid-utf8), as they can\[aq]t
|
|
be used in JSON strings.
|
|
.SS Duplicated files
|
|
.PP
|
|
Mega can have two files with exactly the same name and path (unlike a
|
|
normal file system).
|
|
.PP
|
|
Duplicated files cause problems with the syncing and you will see
|
|
messages in the log about duplicates.
|
|
.PP
|
|
Use \f[C]rclone dedupe\f[R] to fix duplicated files.
|
|
.SS Failure to log\-in
|
|
.PP
|
|
Mega remotes seem to get blocked (reject logins) under \[dq]heavy
|
|
use\[dq].
|
|
We haven\[aq]t worked out the exact blocking rules but it seems to be
|
|
related to fast paced, successive rclone commands.
|
|
.PP
|
|
For example, executing this command 90 times in a row
|
|
\f[C]rclone link remote:file\f[R] will cause the remote to become
|
|
\[dq]blocked\[dq].
|
|
This is not an abnormal situation, for example if you wish to get the
|
|
public links of a directory with hundred of files...
|
|
After more or less a week, the remote will remote accept rclone logins
|
|
normally again.
|
|
.PP
|
|
You can mitigate this issue by mounting the remote it with
|
|
\f[C]rclone mount\f[R].
|
|
This will log\-in when mounting and a log\-out when unmounting only.
|
|
You can also run \f[C]rclone rcd\f[R] and then use \f[C]rclone rc\f[R]
|
|
to run the commands over the API to avoid logging in each time.
|
|
.PP
|
|
Rclone does not currently close mega sessions (you can see them in the
|
|
web interface), however closing the sessions does not solve the issue.
|
|
.PP
|
|
If you space rclone commands by 3 seconds it will avoid blocking the
|
|
remote.
|
|
We haven\[aq]t identified the exact blocking rules, so perhaps one could
|
|
execute the command 80 times without waiting and avoid blocking by
|
|
waiting 3 seconds, then continuing...
|
|
.PP
|
|
Note that this has been observed by trial and error and might not be set
|
|
in stone.
|
|
.PP
|
|
Other tools seem not to produce this blocking effect, as they use a
|
|
different working approach (state\-based, using sessionIDs instead of
|
|
log\-in) which isn\[aq]t compatible with the current stateless rclone
|
|
approach.
|
|
.PP
|
|
Note that once blocked, the use of other tools (such as megacmd) is not
|
|
a sure workaround: following megacmd login times have been observed in
|
|
succession for blocked remote: 7 minutes, 20 min, 30min, 30 min, 30min.
|
|
Web access looks unaffected though.
|
|
.PP
|
|
Investigation is continuing in relation to workarounds based on
|
|
timeouts, pacers, retrials and tpslimits \- if you discover something
|
|
relevant, please post on the forum.
|
|
.PP
|
|
So, if rclone was working nicely and suddenly you are unable to log\-in
|
|
and you are sure the user and the password are correct, likely you have
|
|
got the remote blocked for a while.
|
|
.SS Standard Options
|
|
.PP
|
|
Here are the standard options specific to mega (Mega).
|
|
.SS \-\-mega\-user
|
|
.PP
|
|
User name
|
|
.IP \[bu] 2
|
|
Config: user
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_MEGA_USER
|
|
.IP \[bu] 2
|
|
Type: string
|
|
.IP \[bu] 2
|
|
Default: \[dq]\[dq]
|
|
.SS \-\-mega\-pass
|
|
.PP
|
|
Password.
|
|
.PP
|
|
\f[B]NB\f[R] Input to this must be obscured \- see rclone
|
|
obscure (https://rclone.org/commands/rclone_obscure/).
|
|
.IP \[bu] 2
|
|
Config: pass
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_MEGA_PASS
|
|
.IP \[bu] 2
|
|
Type: string
|
|
.IP \[bu] 2
|
|
Default: \[dq]\[dq]
|
|
.SS Advanced Options
|
|
.PP
|
|
Here are the advanced options specific to mega (Mega).
|
|
.SS \-\-mega\-debug
|
|
.PP
|
|
Output more debug from Mega.
|
|
.PP
|
|
If this flag is set (along with \-vv) it will print further debugging
|
|
information from the mega backend.
|
|
.IP \[bu] 2
|
|
Config: debug
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_MEGA_DEBUG
|
|
.IP \[bu] 2
|
|
Type: bool
|
|
.IP \[bu] 2
|
|
Default: false
|
|
.SS \-\-mega\-hard\-delete
|
|
.PP
|
|
Delete files permanently rather than putting them into the trash.
|
|
.PP
|
|
Normally the mega backend will put all deletions into the trash rather
|
|
than permanently deleting them.
|
|
If you specify this then rclone will permanently delete objects instead.
|
|
.IP \[bu] 2
|
|
Config: hard_delete
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_MEGA_HARD_DELETE
|
|
.IP \[bu] 2
|
|
Type: bool
|
|
.IP \[bu] 2
|
|
Default: false
|
|
.SS \-\-mega\-encoding
|
|
.PP
|
|
This sets the encoding for the backend.
|
|
.PP
|
|
See: the encoding section in the
|
|
overview (https://rclone.org/overview/#encoding) for more info.
|
|
.IP \[bu] 2
|
|
Config: encoding
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_MEGA_ENCODING
|
|
.IP \[bu] 2
|
|
Type: MultiEncoder
|
|
.IP \[bu] 2
|
|
Default: Slash,InvalidUtf8,Dot
|
|
.SS Limitations
|
|
.PP
|
|
This backend uses the go\-mega go
|
|
library (https://github.com/t3rm1n4l/go-mega) which is an opensource go
|
|
library implementing the Mega API.
|
|
There doesn\[aq]t appear to be any documentation for the mega protocol
|
|
beyond the mega C++ SDK (https://github.com/meganz/sdk) source code so
|
|
there are likely quite a few errors still remaining in this library.
|
|
.PP
|
|
Mega allows duplicate files which may confuse rclone.
|
|
.SS Memory
|
|
.PP
|
|
The memory backend is an in RAM backend.
|
|
It does not persist its data \- use the local backend for that.
|
|
.PP
|
|
The memory backend behaves like a bucket based remote (e.g.
|
|
like s3).
|
|
Because it has no parameters you can just use it with the
|
|
\f[C]:memory:\f[R] remote name.
|
|
.PP
|
|
You can configure it as a remote like this with \f[C]rclone config\f[R]
|
|
too if you want to:
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
No remotes found \- make a new one
|
|
n) New remote
|
|
s) Set configuration password
|
|
q) Quit config
|
|
n/s/q> n
|
|
name> remote
|
|
Type of storage to configure.
|
|
Enter a string value. Press Enter for the default (\[dq]\[dq]).
|
|
Choose a number from below, or type in your own value
|
|
[snip]
|
|
XX / Memory
|
|
\[rs] \[dq]memory\[dq]
|
|
[snip]
|
|
Storage> memory
|
|
** See help for memory backend at: https://rclone.org/memory/ **
|
|
|
|
Remote config
|
|
|
|
\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-
|
|
[remote]
|
|
type = memory
|
|
\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-
|
|
y) Yes this is OK (default)
|
|
e) Edit this remote
|
|
d) Delete this remote
|
|
y/e/d> y
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
Because the memory backend isn\[aq]t persistent it is most useful for
|
|
testing or with an rclone server or rclone mount, e.g.
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone mount :memory: /mnt/tmp
|
|
rclone serve webdav :memory:
|
|
rclone serve sftp :memory:
|
|
\f[R]
|
|
.fi
|
|
.SS Modified time and hashes
|
|
.PP
|
|
The memory backend supports MD5 hashes and modification times accurate
|
|
to 1 nS.
|
|
.SS Restricted filename characters
|
|
.PP
|
|
The memory backend replaces the default restricted characters
|
|
set (https://rclone.org/overview/#restricted-characters).
|
|
.SS Microsoft Azure Blob Storage
|
|
.PP
|
|
Paths are specified as \f[C]remote:container\f[R] (or \f[C]remote:\f[R]
|
|
for the \f[C]lsd\f[R] command.) You may put subdirectories in too, e.g.
|
|
\f[C]remote:container/path/to/dir\f[R].
|
|
.PP
|
|
Here is an example of making a Microsoft Azure Blob Storage
|
|
configuration.
|
|
For a remote called \f[C]remote\f[R].
|
|
First run:
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone config
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
This will guide you through an interactive setup process:
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
No remotes found \- make a new one
|
|
n) New remote
|
|
s) Set configuration password
|
|
q) Quit config
|
|
n/s/q> n
|
|
name> remote
|
|
Type of storage to configure.
|
|
Choose a number from below, or type in your own value
|
|
[snip]
|
|
XX / Microsoft Azure Blob Storage
|
|
\[rs] \[dq]azureblob\[dq]
|
|
[snip]
|
|
Storage> azureblob
|
|
Storage Account Name
|
|
account> account_name
|
|
Storage Account Key
|
|
key> base64encodedkey==
|
|
Endpoint for the service \- leave blank normally.
|
|
endpoint>
|
|
Remote config
|
|
\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-
|
|
[remote]
|
|
account = account_name
|
|
key = base64encodedkey==
|
|
endpoint =
|
|
\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-
|
|
y) Yes this is OK
|
|
e) Edit this remote
|
|
d) Delete this remote
|
|
y/e/d> y
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
See all containers
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone lsd remote:
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
Make a new container
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone mkdir remote:container
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
List the contents of a container
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone ls remote:container
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
Sync \f[C]/home/local/directory\f[R] to the remote container, deleting
|
|
any excess files in the container.
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone sync \-i /home/local/directory remote:container
|
|
\f[R]
|
|
.fi
|
|
.SS \-\-fast\-list
|
|
.PP
|
|
This remote supports \f[C]\-\-fast\-list\f[R] which allows you to use
|
|
fewer transactions in exchange for more memory.
|
|
See the rclone docs (https://rclone.org/docs/#fast-list) for more
|
|
details.
|
|
.SS Modified time
|
|
.PP
|
|
The modified time is stored as metadata on the object with the
|
|
\f[C]mtime\f[R] key.
|
|
It is stored using RFC3339 Format time with nanosecond precision.
|
|
The metadata is supplied during directory listings so there is no
|
|
overhead to using it.
|
|
.SS Restricted filename characters
|
|
.PP
|
|
In addition to the default restricted characters
|
|
set (https://rclone.org/overview/#restricted-characters) the following
|
|
characters are also replaced:
|
|
.PP
|
|
.TS
|
|
tab(@);
|
|
l c c.
|
|
T{
|
|
Character
|
|
T}@T{
|
|
Value
|
|
T}@T{
|
|
Replacement
|
|
T}
|
|
_
|
|
T{
|
|
/
|
|
T}@T{
|
|
0x2F
|
|
T}@T{
|
|
\[uFF0F]
|
|
T}
|
|
T{
|
|
\[rs]
|
|
T}@T{
|
|
0x5C
|
|
T}@T{
|
|
\[uFF3C]
|
|
T}
|
|
.TE
|
|
.PP
|
|
File names can also not end with the following characters.
|
|
These only get replaced if they are the last character in the name:
|
|
.PP
|
|
.TS
|
|
tab(@);
|
|
l c c.
|
|
T{
|
|
Character
|
|
T}@T{
|
|
Value
|
|
T}@T{
|
|
Replacement
|
|
T}
|
|
_
|
|
T{
|
|
\&.
|
|
T}@T{
|
|
0x2E
|
|
T}@T{
|
|
\[uFF0E]
|
|
T}
|
|
.TE
|
|
.PP
|
|
Invalid UTF\-8 bytes will also be
|
|
replaced (https://rclone.org/overview/#invalid-utf8), as they can\[aq]t
|
|
be used in JSON strings.
|
|
.SS Hashes
|
|
.PP
|
|
MD5 hashes are stored with blobs.
|
|
However blobs that were uploaded in chunks only have an MD5 if the
|
|
source remote was capable of MD5 hashes, e.g.
|
|
the local disk.
|
|
.SS Authenticating with Azure Blob Storage
|
|
.PP
|
|
Rclone has 3 ways of authenticating with Azure Blob Storage:
|
|
.SS Account and Key
|
|
.PP
|
|
This is the most straight forward and least flexible way.
|
|
Just fill in the \f[C]account\f[R] and \f[C]key\f[R] lines and leave the
|
|
rest blank.
|
|
.SS SAS URL
|
|
.PP
|
|
This can be an account level SAS URL or container level SAS URL.
|
|
.PP
|
|
To use it leave \f[C]account\f[R], \f[C]key\f[R] blank and fill in
|
|
\f[C]sas_url\f[R].
|
|
.PP
|
|
An account level SAS URL or container level SAS URL can be obtained from
|
|
the Azure portal or the Azure Storage Explorer.
|
|
To get a container level SAS URL right click on a container in the Azure
|
|
Blob explorer in the Azure portal.
|
|
.PP
|
|
If you use a container level SAS URL, rclone operations are permitted
|
|
only on a particular container, e.g.
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone ls azureblob:container
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
You can also list the single container from the root.
|
|
This will only show the container specified by the SAS URL.
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
$ rclone lsd azureblob:
|
|
container/
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
Note that you can\[aq]t see or access any other containers \- this will
|
|
fail
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone ls azureblob:othercontainer
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
Container level SAS URLs are useful for temporarily allowing third
|
|
parties access to a single container or putting credentials into an
|
|
untrusted environment such as a CI build server.
|
|
.SS Standard Options
|
|
.PP
|
|
Here are the standard options specific to azureblob (Microsoft Azure
|
|
Blob Storage).
|
|
.SS \-\-azureblob\-account
|
|
.PP
|
|
Storage Account Name (leave blank to use SAS URL or Emulator)
|
|
.IP \[bu] 2
|
|
Config: account
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_AZUREBLOB_ACCOUNT
|
|
.IP \[bu] 2
|
|
Type: string
|
|
.IP \[bu] 2
|
|
Default: \[dq]\[dq]
|
|
.SS \-\-azureblob\-service\-principal\-file
|
|
.PP
|
|
Path to file containing credentials for use with a service principal.
|
|
.PP
|
|
Leave blank normally.
|
|
Needed only if you want to use a service principal instead of
|
|
interactive login.
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
$ az sp create\-for\-rbac \-\-name \[dq]<name>\[dq] \[rs]
|
|
\-\-role \[dq]Storage Blob Data Owner\[dq] \[rs]
|
|
\-\-scopes \[dq]/subscriptions/<subscription>/resourceGroups/<resource\-group>/providers/Microsoft.Storage/storageAccounts/<storage\-account>/blobServices/default/containers/<container>\[dq] \[rs]
|
|
> azure\-principal.json
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
See Use Azure CLI to assign an Azure role for access to blob and queue
|
|
data (https://docs.microsoft.com/en-us/azure/storage/common/storage-auth-aad-rbac-cli)
|
|
for more details.
|
|
.IP \[bu] 2
|
|
Config: service_principal_file
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_AZUREBLOB_SERVICE_PRINCIPAL_FILE
|
|
.IP \[bu] 2
|
|
Type: string
|
|
.IP \[bu] 2
|
|
Default: \[dq]\[dq]
|
|
.SS \-\-azureblob\-key
|
|
.PP
|
|
Storage Account Key (leave blank to use SAS URL or Emulator)
|
|
.IP \[bu] 2
|
|
Config: key
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_AZUREBLOB_KEY
|
|
.IP \[bu] 2
|
|
Type: string
|
|
.IP \[bu] 2
|
|
Default: \[dq]\[dq]
|
|
.SS \-\-azureblob\-sas\-url
|
|
.PP
|
|
SAS URL for container level access only (leave blank if using
|
|
account/key or Emulator)
|
|
.IP \[bu] 2
|
|
Config: sas_url
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_AZUREBLOB_SAS_URL
|
|
.IP \[bu] 2
|
|
Type: string
|
|
.IP \[bu] 2
|
|
Default: \[dq]\[dq]
|
|
.SS \-\-azureblob\-use\-msi
|
|
.PP
|
|
Use a managed service identity to authenticate (only works in Azure)
|
|
.PP
|
|
When true, use a managed service
|
|
identity (https://docs.microsoft.com/en-us/azure/active-directory/managed-identities-azure-resources/)
|
|
to authenticate to Azure Storage instead of a SAS token or account key.
|
|
.PP
|
|
If the VM(SS) on which this program is running has a system\-assigned
|
|
identity, it will be used by default.
|
|
If the resource has no system\-assigned but exactly one user\-assigned
|
|
identity, the user\-assigned identity will be used by default.
|
|
If the resource has multiple user\-assigned identities, the identity to
|
|
use must be explicitly specified using exactly one of the msi_object_id,
|
|
msi_client_id, or msi_mi_res_id parameters.
|
|
.IP \[bu] 2
|
|
Config: use_msi
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_AZUREBLOB_USE_MSI
|
|
.IP \[bu] 2
|
|
Type: bool
|
|
.IP \[bu] 2
|
|
Default: false
|
|
.SS \-\-azureblob\-use\-emulator
|
|
.PP
|
|
Uses local storage emulator if provided as \[aq]true\[aq] (leave blank
|
|
if using real azure storage endpoint)
|
|
.IP \[bu] 2
|
|
Config: use_emulator
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_AZUREBLOB_USE_EMULATOR
|
|
.IP \[bu] 2
|
|
Type: bool
|
|
.IP \[bu] 2
|
|
Default: false
|
|
.SS Advanced Options
|
|
.PP
|
|
Here are the advanced options specific to azureblob (Microsoft Azure
|
|
Blob Storage).
|
|
.SS \-\-azureblob\-msi\-object\-id
|
|
.PP
|
|
Object ID of the user\-assigned MSI to use, if any.
|
|
Leave blank if msi_client_id or msi_mi_res_id specified.
|
|
.IP \[bu] 2
|
|
Config: msi_object_id
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_AZUREBLOB_MSI_OBJECT_ID
|
|
.IP \[bu] 2
|
|
Type: string
|
|
.IP \[bu] 2
|
|
Default: \[dq]\[dq]
|
|
.SS \-\-azureblob\-msi\-client\-id
|
|
.PP
|
|
Object ID of the user\-assigned MSI to use, if any.
|
|
Leave blank if msi_object_id or msi_mi_res_id specified.
|
|
.IP \[bu] 2
|
|
Config: msi_client_id
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_AZUREBLOB_MSI_CLIENT_ID
|
|
.IP \[bu] 2
|
|
Type: string
|
|
.IP \[bu] 2
|
|
Default: \[dq]\[dq]
|
|
.SS \-\-azureblob\-msi\-mi\-res\-id
|
|
.PP
|
|
Azure resource ID of the user\-assigned MSI to use, if any.
|
|
Leave blank if msi_client_id or msi_object_id specified.
|
|
.IP \[bu] 2
|
|
Config: msi_mi_res_id
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_AZUREBLOB_MSI_MI_RES_ID
|
|
.IP \[bu] 2
|
|
Type: string
|
|
.IP \[bu] 2
|
|
Default: \[dq]\[dq]
|
|
.SS \-\-azureblob\-endpoint
|
|
.PP
|
|
Endpoint for the service Leave blank normally.
|
|
.IP \[bu] 2
|
|
Config: endpoint
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_AZUREBLOB_ENDPOINT
|
|
.IP \[bu] 2
|
|
Type: string
|
|
.IP \[bu] 2
|
|
Default: \[dq]\[dq]
|
|
.SS \-\-azureblob\-upload\-cutoff
|
|
.PP
|
|
Cutoff for switching to chunked upload (<= 256MB).
|
|
(Deprecated)
|
|
.IP \[bu] 2
|
|
Config: upload_cutoff
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_AZUREBLOB_UPLOAD_CUTOFF
|
|
.IP \[bu] 2
|
|
Type: string
|
|
.IP \[bu] 2
|
|
Default: \[dq]\[dq]
|
|
.SS \-\-azureblob\-chunk\-size
|
|
.PP
|
|
Upload chunk size (<= 100MB).
|
|
.PP
|
|
Note that this is stored in memory and there may be up to
|
|
\[dq]\-\-transfers\[dq] chunks stored at once in memory.
|
|
.IP \[bu] 2
|
|
Config: chunk_size
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_AZUREBLOB_CHUNK_SIZE
|
|
.IP \[bu] 2
|
|
Type: SizeSuffix
|
|
.IP \[bu] 2
|
|
Default: 4M
|
|
.SS \-\-azureblob\-list\-chunk
|
|
.PP
|
|
Size of blob list.
|
|
.PP
|
|
This sets the number of blobs requested in each listing chunk.
|
|
Default is the maximum, 5000.
|
|
\[dq]List blobs\[dq] requests are permitted 2 minutes per megabyte to
|
|
complete.
|
|
If an operation is taking longer than 2 minutes per megabyte on average,
|
|
it will time out (
|
|
source (https://docs.microsoft.com/en-us/rest/api/storageservices/setting-timeouts-for-blob-service-operations#exceptions-to-default-timeout-interval)
|
|
).
|
|
This can be used to limit the number of blobs items to return, to avoid
|
|
the time out.
|
|
.IP \[bu] 2
|
|
Config: list_chunk
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_AZUREBLOB_LIST_CHUNK
|
|
.IP \[bu] 2
|
|
Type: int
|
|
.IP \[bu] 2
|
|
Default: 5000
|
|
.SS \-\-azureblob\-access\-tier
|
|
.PP
|
|
Access tier of blob: hot, cool or archive.
|
|
.PP
|
|
Archived blobs can be restored by setting access tier to hot or cool.
|
|
Leave blank if you intend to use default access tier, which is set at
|
|
account level
|
|
.PP
|
|
If there is no \[dq]access tier\[dq] specified, rclone doesn\[aq]t apply
|
|
any tier.
|
|
rclone performs \[dq]Set Tier\[dq] operation on blobs while uploading,
|
|
if objects are not modified, specifying \[dq]access tier\[dq] to new one
|
|
will have no effect.
|
|
If blobs are in \[dq]archive tier\[dq] at remote, trying to perform data
|
|
transfer operations from remote will not be allowed.
|
|
User should first restore by tiering blob to \[dq]Hot\[dq] or
|
|
\[dq]Cool\[dq].
|
|
.IP \[bu] 2
|
|
Config: access_tier
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_AZUREBLOB_ACCESS_TIER
|
|
.IP \[bu] 2
|
|
Type: string
|
|
.IP \[bu] 2
|
|
Default: \[dq]\[dq]
|
|
.SS \-\-azureblob\-archive\-tier\-delete
|
|
.PP
|
|
Delete archive tier blobs before overwriting.
|
|
.PP
|
|
Archive tier blobs cannot be updated.
|
|
So without this flag, if you attempt to update an archive tier blob,
|
|
then rclone will produce the error:
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
can\[aq]t update archive tier blob without \-\-azureblob\-archive\-tier\-delete
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
With this flag set then before rclone attempts to overwrite an archive
|
|
tier blob, it will delete the existing blob before uploading its
|
|
replacement.
|
|
This has the potential for data loss if the upload fails (unlike
|
|
updating a normal blob) and also may cost more since deleting archive
|
|
tier blobs early may be chargable.
|
|
.IP \[bu] 2
|
|
Config: archive_tier_delete
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_AZUREBLOB_ARCHIVE_TIER_DELETE
|
|
.IP \[bu] 2
|
|
Type: bool
|
|
.IP \[bu] 2
|
|
Default: false
|
|
.SS \-\-azureblob\-disable\-checksum
|
|
.PP
|
|
Don\[aq]t store MD5 checksum with object metadata.
|
|
.PP
|
|
Normally rclone will calculate the MD5 checksum of the input before
|
|
uploading it so it can add it to metadata on the object.
|
|
This is great for data integrity checking but can cause long delays for
|
|
large files to start uploading.
|
|
.IP \[bu] 2
|
|
Config: disable_checksum
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_AZUREBLOB_DISABLE_CHECKSUM
|
|
.IP \[bu] 2
|
|
Type: bool
|
|
.IP \[bu] 2
|
|
Default: false
|
|
.SS \-\-azureblob\-memory\-pool\-flush\-time
|
|
.PP
|
|
How often internal memory buffer pools will be flushed.
|
|
Uploads which requires additional buffers (f.e multipart) will use
|
|
memory pool for allocations.
|
|
This option controls how often unused buffers will be removed from the
|
|
pool.
|
|
.IP \[bu] 2
|
|
Config: memory_pool_flush_time
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_AZUREBLOB_MEMORY_POOL_FLUSH_TIME
|
|
.IP \[bu] 2
|
|
Type: Duration
|
|
.IP \[bu] 2
|
|
Default: 1m0s
|
|
.SS \-\-azureblob\-memory\-pool\-use\-mmap
|
|
.PP
|
|
Whether to use mmap buffers in internal memory pool.
|
|
.IP \[bu] 2
|
|
Config: memory_pool_use_mmap
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_AZUREBLOB_MEMORY_POOL_USE_MMAP
|
|
.IP \[bu] 2
|
|
Type: bool
|
|
.IP \[bu] 2
|
|
Default: false
|
|
.SS \-\-azureblob\-encoding
|
|
.PP
|
|
This sets the encoding for the backend.
|
|
.PP
|
|
See: the encoding section in the
|
|
overview (https://rclone.org/overview/#encoding) for more info.
|
|
.IP \[bu] 2
|
|
Config: encoding
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_AZUREBLOB_ENCODING
|
|
.IP \[bu] 2
|
|
Type: MultiEncoder
|
|
.IP \[bu] 2
|
|
Default: Slash,BackSlash,Del,Ctl,RightPeriod,InvalidUtf8
|
|
.SS \-\-azureblob\-public\-access
|
|
.PP
|
|
Public access level of a container: blob, container.
|
|
.IP \[bu] 2
|
|
Config: public_access
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_AZUREBLOB_PUBLIC_ACCESS
|
|
.IP \[bu] 2
|
|
Type: string
|
|
.IP \[bu] 2
|
|
Default: \[dq]\[dq]
|
|
.IP \[bu] 2
|
|
Examples:
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
\[dq]\[dq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
The container and its blobs can be accessed only with an authorized
|
|
request.
|
|
It\[aq]s a default value
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[dq]blob\[dq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Blob data within this container can be read via anonymous request.
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[dq]container\[dq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Allow full public read access for container and blob data.
|
|
.RE
|
|
.RE
|
|
.SS Limitations
|
|
.PP
|
|
MD5 sums are only uploaded with chunked files if the source has an MD5
|
|
sum.
|
|
This will always be the case for a local to azure copy.
|
|
.PP
|
|
\f[C]rclone about\f[R] is not supported by the Microsoft Azure Blob
|
|
storage backend.
|
|
Backends without this capability cannot determine free space for an
|
|
rclone mount or use policy \f[C]mfs\f[R] (most free space) as a member
|
|
of an rclone union remote.
|
|
.PP
|
|
See List of backends that do not support rclone
|
|
about (https://rclone.org/overview/#optional-features) See rclone
|
|
about (https://rclone.org/commands/rclone_about/)
|
|
.SS Azure Storage Emulator Support
|
|
.PP
|
|
You can test rclone with storage emulator locally, to do this make sure
|
|
azure storage emulator installed locally and set up a new remote with
|
|
\f[C]rclone config\f[R] follow instructions described in introduction,
|
|
set \f[C]use_emulator\f[R] config as \f[C]true\f[R], you do not need to
|
|
provide default account name or key if using emulator.
|
|
.SS Microsoft OneDrive
|
|
.PP
|
|
Paths are specified as \f[C]remote:path\f[R]
|
|
.PP
|
|
Paths may be as deep as required, e.g.
|
|
\f[C]remote:directory/subdirectory\f[R].
|
|
.PP
|
|
The initial setup for OneDrive involves getting a token from Microsoft
|
|
which you need to do in your browser.
|
|
\f[C]rclone config\f[R] walks you through it.
|
|
.PP
|
|
Here is an example of how to make a remote called \f[C]remote\f[R].
|
|
First run:
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone config
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
This will guide you through an interactive setup process:
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
e) Edit existing remote
|
|
n) New remote
|
|
d) Delete remote
|
|
r) Rename remote
|
|
c) Copy remote
|
|
s) Set configuration password
|
|
q) Quit config
|
|
e/n/d/r/c/s/q> n
|
|
name> remote
|
|
Type of storage to configure.
|
|
Enter a string value. Press Enter for the default (\[dq]\[dq]).
|
|
Choose a number from below, or type in your own value
|
|
[snip]
|
|
XX / Microsoft OneDrive
|
|
\[rs] \[dq]onedrive\[dq]
|
|
[snip]
|
|
Storage> onedrive
|
|
Microsoft App Client Id
|
|
Leave blank normally.
|
|
Enter a string value. Press Enter for the default (\[dq]\[dq]).
|
|
client_id>
|
|
Microsoft App Client Secret
|
|
Leave blank normally.
|
|
Enter a string value. Press Enter for the default (\[dq]\[dq]).
|
|
client_secret>
|
|
Edit advanced config? (y/n)
|
|
y) Yes
|
|
n) No
|
|
y/n> n
|
|
Remote config
|
|
Use auto config?
|
|
* Say Y if not sure
|
|
* Say N if you are working on a remote or headless machine
|
|
y) Yes
|
|
n) No
|
|
y/n> y
|
|
If your browser doesn\[aq]t open automatically go to the following link: http://127.0.0.1:53682/auth
|
|
Log in and authorize rclone for access
|
|
Waiting for code...
|
|
Got code
|
|
Choose a number from below, or type in an existing value
|
|
1 / OneDrive Personal or Business
|
|
\[rs] \[dq]onedrive\[dq]
|
|
2 / Sharepoint site
|
|
\[rs] \[dq]sharepoint\[dq]
|
|
3 / Type in driveID
|
|
\[rs] \[dq]driveid\[dq]
|
|
4 / Type in SiteID
|
|
\[rs] \[dq]siteid\[dq]
|
|
5 / Search a Sharepoint site
|
|
\[rs] \[dq]search\[dq]
|
|
Your choice> 1
|
|
Found 1 drives, please select the one you want to use:
|
|
0: OneDrive (business) id=b!Eqwertyuiopasdfghjklzxcvbnm\-7mnbvcxzlkjhgfdsapoiuytrewqk
|
|
Chose drive to use:> 0
|
|
Found drive \[aq]root\[aq] of type \[aq]business\[aq], URL: https://org\-my.sharepoint.com/personal/you/Documents
|
|
Is that okay?
|
|
y) Yes
|
|
n) No
|
|
y/n> y
|
|
\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-
|
|
[remote]
|
|
type = onedrive
|
|
token = {\[dq]access_token\[dq]:\[dq]youraccesstoken\[dq],\[dq]token_type\[dq]:\[dq]Bearer\[dq],\[dq]refresh_token\[dq]:\[dq]yourrefreshtoken\[dq],\[dq]expiry\[dq]:\[dq]2018\-08\-26T22:39:52.486512262+08:00\[dq]}
|
|
drive_id = b!Eqwertyuiopasdfghjklzxcvbnm\-7mnbvcxzlkjhgfdsapoiuytrewqk
|
|
drive_type = business
|
|
\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-
|
|
y) Yes this is OK
|
|
e) Edit this remote
|
|
d) Delete this remote
|
|
y/e/d> y
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
See the remote setup docs (https://rclone.org/remote_setup/) for how to
|
|
set it up on a machine with no Internet browser available.
|
|
.PP
|
|
Note that rclone runs a webserver on your local machine to collect the
|
|
token as returned from Microsoft.
|
|
This only runs from the moment it opens your browser to the moment you
|
|
get back the verification code.
|
|
This is on \f[C]http://127.0.0.1:53682/\f[R] and this it may require you
|
|
to unblock it temporarily if you are running a host firewall.
|
|
.PP
|
|
Once configured you can then use \f[C]rclone\f[R] like this,
|
|
.PP
|
|
List directories in top level of your OneDrive
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone lsd remote:
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
List all the files in your OneDrive
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone ls remote:
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
To copy a local directory to an OneDrive directory called backup
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone copy /home/source remote:backup
|
|
\f[R]
|
|
.fi
|
|
.SS Getting your own Client ID and Key
|
|
.PP
|
|
You can use your own Client ID if the default (\f[C]client_id\f[R] left
|
|
blank) one doesn\[aq]t work for you or you see lots of throttling.
|
|
The default Client ID and Key is shared by all rclone users when
|
|
performing requests.
|
|
.PP
|
|
If you are having problems with them (E.g., seeing a lot of throttling),
|
|
you can get your own Client ID and Key by following the steps below:
|
|
.IP "1." 3
|
|
Open
|
|
https://portal.azure.com/#blade/Microsoft_AAD_RegisteredApps/ApplicationsListBlade
|
|
and then click \f[C]New registration\f[R].
|
|
.IP "2." 3
|
|
Enter a name for your app, choose account type
|
|
\f[C]Accounts in any organizational directory (Any Azure AD directory \- Multitenant) and personal Microsoft accounts (e.g. Skype, Xbox)\f[R],
|
|
select \f[C]Web\f[R] in \f[C]Redirect URI\f[R], then type (do not copy
|
|
and paste) \f[C]http://localhost:53682/\f[R] and click Register.
|
|
Copy and keep the \f[C]Application (client) ID\f[R] under the app name
|
|
for later use.
|
|
.IP "3." 3
|
|
Under \f[C]manage\f[R] select \f[C]Certificates & secrets\f[R], click
|
|
\f[C]New client secret\f[R].
|
|
Copy and keep that secret for later use.
|
|
.IP "4." 3
|
|
Under \f[C]manage\f[R] select \f[C]API permissions\f[R], click
|
|
\f[C]Add a permission\f[R] and select \f[C]Microsoft Graph\f[R] then
|
|
select \f[C]delegated permissions\f[R].
|
|
.IP "5." 3
|
|
Search and select the following permissions: \f[C]Files.Read\f[R],
|
|
\f[C]Files.ReadWrite\f[R], \f[C]Files.Read.All\f[R],
|
|
\f[C]Files.ReadWrite.All\f[R], \f[C]offline_access\f[R],
|
|
\f[C]User.Read\f[R].
|
|
Once selected click \f[C]Add permissions\f[R] at the bottom.
|
|
.PP
|
|
Now the application is complete.
|
|
Run \f[C]rclone config\f[R] to create or edit a OneDrive remote.
|
|
Supply the app ID and password as Client ID and Secret, respectively.
|
|
rclone will walk you through the remaining steps.
|
|
.SS Modification time and hashes
|
|
.PP
|
|
OneDrive allows modification times to be set on objects accurate to 1
|
|
second.
|
|
These will be used to detect whether objects need syncing or not.
|
|
.PP
|
|
OneDrive personal supports SHA1 type hashes.
|
|
OneDrive for business and Sharepoint Server support
|
|
QuickXorHash (https://docs.microsoft.com/en-us/onedrive/developer/code-snippets/quickxorhash).
|
|
.PP
|
|
For all types of OneDrive you can use the \f[C]\-\-checksum\f[R] flag.
|
|
.SS Restricted filename characters
|
|
.PP
|
|
In addition to the default restricted characters
|
|
set (https://rclone.org/overview/#restricted-characters) the following
|
|
characters are also replaced:
|
|
.PP
|
|
.TS
|
|
tab(@);
|
|
l c c.
|
|
T{
|
|
Character
|
|
T}@T{
|
|
Value
|
|
T}@T{
|
|
Replacement
|
|
T}
|
|
_
|
|
T{
|
|
\[dq]
|
|
T}@T{
|
|
0x22
|
|
T}@T{
|
|
\[uFF02]
|
|
T}
|
|
T{
|
|
*
|
|
T}@T{
|
|
0x2A
|
|
T}@T{
|
|
\[uFF0A]
|
|
T}
|
|
T{
|
|
:
|
|
T}@T{
|
|
0x3A
|
|
T}@T{
|
|
\[uFF1A]
|
|
T}
|
|
T{
|
|
<
|
|
T}@T{
|
|
0x3C
|
|
T}@T{
|
|
\[uFF1C]
|
|
T}
|
|
T{
|
|
>
|
|
T}@T{
|
|
0x3E
|
|
T}@T{
|
|
\[uFF1E]
|
|
T}
|
|
T{
|
|
?
|
|
T}@T{
|
|
0x3F
|
|
T}@T{
|
|
\[uFF1F]
|
|
T}
|
|
T{
|
|
\[rs]
|
|
T}@T{
|
|
0x5C
|
|
T}@T{
|
|
\[uFF3C]
|
|
T}
|
|
T{
|
|
|
|
|
T}@T{
|
|
0x7C
|
|
T}@T{
|
|
\[uFF5C]
|
|
T}
|
|
.TE
|
|
.PP
|
|
File names can also not end with the following characters.
|
|
These only get replaced if they are the last character in the name:
|
|
.PP
|
|
.TS
|
|
tab(@);
|
|
l c c.
|
|
T{
|
|
Character
|
|
T}@T{
|
|
Value
|
|
T}@T{
|
|
Replacement
|
|
T}
|
|
_
|
|
T{
|
|
SP
|
|
T}@T{
|
|
0x20
|
|
T}@T{
|
|
\[u2420]
|
|
T}
|
|
T{
|
|
\&.
|
|
T}@T{
|
|
0x2E
|
|
T}@T{
|
|
\[uFF0E]
|
|
T}
|
|
.TE
|
|
.PP
|
|
File names can also not begin with the following characters.
|
|
These only get replaced if they are the first character in the name:
|
|
.PP
|
|
.TS
|
|
tab(@);
|
|
l c c.
|
|
T{
|
|
Character
|
|
T}@T{
|
|
Value
|
|
T}@T{
|
|
Replacement
|
|
T}
|
|
_
|
|
T{
|
|
SP
|
|
T}@T{
|
|
0x20
|
|
T}@T{
|
|
\[u2420]
|
|
T}
|
|
T{
|
|
\[ti]
|
|
T}@T{
|
|
0x7E
|
|
T}@T{
|
|
\[uFF5E]
|
|
T}
|
|
.TE
|
|
.PP
|
|
Invalid UTF\-8 bytes will also be
|
|
replaced (https://rclone.org/overview/#invalid-utf8), as they can\[aq]t
|
|
be used in JSON strings.
|
|
.SS Deleting files
|
|
.PP
|
|
Any files you delete with rclone will end up in the trash.
|
|
Microsoft doesn\[aq]t provide an API to permanently delete files, nor to
|
|
empty the trash, so you will have to do that with one of Microsoft\[aq]s
|
|
apps or via the OneDrive website.
|
|
.SS Standard Options
|
|
.PP
|
|
Here are the standard options specific to onedrive (Microsoft OneDrive).
|
|
.SS \-\-onedrive\-client\-id
|
|
.PP
|
|
OAuth Client Id Leave blank normally.
|
|
.IP \[bu] 2
|
|
Config: client_id
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_ONEDRIVE_CLIENT_ID
|
|
.IP \[bu] 2
|
|
Type: string
|
|
.IP \[bu] 2
|
|
Default: \[dq]\[dq]
|
|
.SS \-\-onedrive\-client\-secret
|
|
.PP
|
|
OAuth Client Secret Leave blank normally.
|
|
.IP \[bu] 2
|
|
Config: client_secret
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_ONEDRIVE_CLIENT_SECRET
|
|
.IP \[bu] 2
|
|
Type: string
|
|
.IP \[bu] 2
|
|
Default: \[dq]\[dq]
|
|
.SS \-\-onedrive\-region
|
|
.PP
|
|
Choose national cloud region for OneDrive.
|
|
.IP \[bu] 2
|
|
Config: region
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_ONEDRIVE_REGION
|
|
.IP \[bu] 2
|
|
Type: string
|
|
.IP \[bu] 2
|
|
Default: \[dq]global\[dq]
|
|
.IP \[bu] 2
|
|
Examples:
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
\[dq]global\[dq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Microsoft Cloud Global
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[dq]us\[dq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Microsoft Cloud for US Government
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[dq]de\[dq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Microsoft Cloud Germany
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[dq]cn\[dq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Azure and Office 365 operated by 21Vianet in China
|
|
.RE
|
|
.RE
|
|
.SS Advanced Options
|
|
.PP
|
|
Here are the advanced options specific to onedrive (Microsoft OneDrive).
|
|
.SS \-\-onedrive\-token
|
|
.PP
|
|
OAuth Access Token as a JSON blob.
|
|
.IP \[bu] 2
|
|
Config: token
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_ONEDRIVE_TOKEN
|
|
.IP \[bu] 2
|
|
Type: string
|
|
.IP \[bu] 2
|
|
Default: \[dq]\[dq]
|
|
.SS \-\-onedrive\-auth\-url
|
|
.PP
|
|
Auth server URL.
|
|
Leave blank to use the provider defaults.
|
|
.IP \[bu] 2
|
|
Config: auth_url
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_ONEDRIVE_AUTH_URL
|
|
.IP \[bu] 2
|
|
Type: string
|
|
.IP \[bu] 2
|
|
Default: \[dq]\[dq]
|
|
.SS \-\-onedrive\-token\-url
|
|
.PP
|
|
Token server url.
|
|
Leave blank to use the provider defaults.
|
|
.IP \[bu] 2
|
|
Config: token_url
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_ONEDRIVE_TOKEN_URL
|
|
.IP \[bu] 2
|
|
Type: string
|
|
.IP \[bu] 2
|
|
Default: \[dq]\[dq]
|
|
.SS \-\-onedrive\-chunk\-size
|
|
.PP
|
|
Chunk size to upload files with \- must be multiple of 320k (327,680
|
|
bytes).
|
|
.PP
|
|
Above this size files will be chunked \- must be multiple of 320k
|
|
(327,680 bytes) and should not exceed 250M (262,144,000 bytes) else you
|
|
may encounter
|
|
\[dq]Microsoft.SharePoint.Client.InvalidClientQueryException: The
|
|
request message is too big.\[dq] Note that the chunks will be buffered
|
|
into memory.
|
|
.IP \[bu] 2
|
|
Config: chunk_size
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_ONEDRIVE_CHUNK_SIZE
|
|
.IP \[bu] 2
|
|
Type: SizeSuffix
|
|
.IP \[bu] 2
|
|
Default: 10M
|
|
.SS \-\-onedrive\-drive\-id
|
|
.PP
|
|
The ID of the drive to use
|
|
.IP \[bu] 2
|
|
Config: drive_id
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_ONEDRIVE_DRIVE_ID
|
|
.IP \[bu] 2
|
|
Type: string
|
|
.IP \[bu] 2
|
|
Default: \[dq]\[dq]
|
|
.SS \-\-onedrive\-drive\-type
|
|
.PP
|
|
The type of the drive ( personal | business | documentLibrary )
|
|
.IP \[bu] 2
|
|
Config: drive_type
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_ONEDRIVE_DRIVE_TYPE
|
|
.IP \[bu] 2
|
|
Type: string
|
|
.IP \[bu] 2
|
|
Default: \[dq]\[dq]
|
|
.SS \-\-onedrive\-expose\-onenote\-files
|
|
.PP
|
|
Set to make OneNote files show up in directory listings.
|
|
.PP
|
|
By default rclone will hide OneNote files in directory listings because
|
|
operations like \[dq]Open\[dq] and \[dq]Update\[dq] won\[aq]t work on
|
|
them.
|
|
But this behaviour may also prevent you from deleting them.
|
|
If you want to delete OneNote files or otherwise want them to show up in
|
|
directory listing, set this option.
|
|
.IP \[bu] 2
|
|
Config: expose_onenote_files
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_ONEDRIVE_EXPOSE_ONENOTE_FILES
|
|
.IP \[bu] 2
|
|
Type: bool
|
|
.IP \[bu] 2
|
|
Default: false
|
|
.SS \-\-onedrive\-server\-side\-across\-configs
|
|
.PP
|
|
Allow server\-side operations (e.g.
|
|
copy) to work across different onedrive configs.
|
|
.PP
|
|
This will only work if you are copying between two OneDrive
|
|
\f[I]Personal\f[R] drives AND the files to copy are already shared
|
|
between them.
|
|
In other cases, rclone will fall back to normal copy (which will be
|
|
slightly slower).
|
|
.IP \[bu] 2
|
|
Config: server_side_across_configs
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_ONEDRIVE_SERVER_SIDE_ACROSS_CONFIGS
|
|
.IP \[bu] 2
|
|
Type: bool
|
|
.IP \[bu] 2
|
|
Default: false
|
|
.SS \-\-onedrive\-no\-versions
|
|
.PP
|
|
Remove all versions on modifying operations
|
|
.PP
|
|
Onedrive for business creates versions when rclone uploads new files
|
|
overwriting an existing one and when it sets the modification time.
|
|
.PP
|
|
These versions take up space out of the quota.
|
|
.PP
|
|
This flag checks for versions after file upload and setting modification
|
|
time and removes all but the last version.
|
|
.PP
|
|
\f[B]NB\f[R] Onedrive personal can\[aq]t currently delete versions so
|
|
don\[aq]t use this flag there.
|
|
.IP \[bu] 2
|
|
Config: no_versions
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_ONEDRIVE_NO_VERSIONS
|
|
.IP \[bu] 2
|
|
Type: bool
|
|
.IP \[bu] 2
|
|
Default: false
|
|
.SS \-\-onedrive\-link\-scope
|
|
.PP
|
|
Set the scope of the links created by the link command.
|
|
.IP \[bu] 2
|
|
Config: link_scope
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_ONEDRIVE_LINK_SCOPE
|
|
.IP \[bu] 2
|
|
Type: string
|
|
.IP \[bu] 2
|
|
Default: \[dq]anonymous\[dq]
|
|
.IP \[bu] 2
|
|
Examples:
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
\[dq]anonymous\[dq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Anyone with the link has access, without needing to sign in.
|
|
This may include people outside of your organization.
|
|
Anonymous link support may be disabled by an administrator.
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[dq]organization\[dq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Anyone signed into your organization (tenant) can use the link to get
|
|
access.
|
|
Only available in OneDrive for Business and SharePoint.
|
|
.RE
|
|
.RE
|
|
.SS \-\-onedrive\-link\-type
|
|
.PP
|
|
Set the type of the links created by the link command.
|
|
.IP \[bu] 2
|
|
Config: link_type
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_ONEDRIVE_LINK_TYPE
|
|
.IP \[bu] 2
|
|
Type: string
|
|
.IP \[bu] 2
|
|
Default: \[dq]view\[dq]
|
|
.IP \[bu] 2
|
|
Examples:
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
\[dq]view\[dq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Creates a read\-only link to the item.
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[dq]edit\[dq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Creates a read\-write link to the item.
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[dq]embed\[dq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Creates an embeddable link to the item.
|
|
.RE
|
|
.RE
|
|
.SS \-\-onedrive\-link\-password
|
|
.PP
|
|
Set the password for links created by the link command.
|
|
.PP
|
|
At the time of writing this only works with OneDrive personal paid
|
|
accounts.
|
|
.IP \[bu] 2
|
|
Config: link_password
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_ONEDRIVE_LINK_PASSWORD
|
|
.IP \[bu] 2
|
|
Type: string
|
|
.IP \[bu] 2
|
|
Default: \[dq]\[dq]
|
|
.SS \-\-onedrive\-encoding
|
|
.PP
|
|
This sets the encoding for the backend.
|
|
.PP
|
|
See: the encoding section in the
|
|
overview (https://rclone.org/overview/#encoding) for more info.
|
|
.IP \[bu] 2
|
|
Config: encoding
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_ONEDRIVE_ENCODING
|
|
.IP \[bu] 2
|
|
Type: MultiEncoder
|
|
.IP \[bu] 2
|
|
Default:
|
|
Slash,LtGt,DoubleQuote,Colon,Question,Asterisk,Pipe,BackSlash,Del,Ctl,LeftSpace,LeftTilde,RightSpace,RightPeriod,InvalidUtf8,Dot
|
|
.SS Limitations
|
|
.PP
|
|
If you don\[aq]t use rclone for 90 days the refresh token will expire.
|
|
This will result in authorization problems.
|
|
This is easy to fix by running the
|
|
\f[C]rclone config reconnect remote:\f[R] command to get a new token and
|
|
refresh token.
|
|
.SS Naming
|
|
.PP
|
|
Note that OneDrive is case insensitive so you can\[aq]t have a file
|
|
called \[dq]Hello.doc\[dq] and one called \[dq]hello.doc\[dq].
|
|
.PP
|
|
There are quite a few characters that can\[aq]t be in OneDrive file
|
|
names.
|
|
These can\[aq]t occur on Windows platforms, but on non\-Windows
|
|
platforms they are common.
|
|
Rclone will map these names to and from an identical looking unicode
|
|
equivalent.
|
|
For example if a file has a \f[C]?\f[R] in it will be mapped to
|
|
\f[C]\[uFF1F]\f[R] instead.
|
|
.SS File sizes
|
|
.PP
|
|
The largest allowed file size is 250GB for both OneDrive Personal and
|
|
OneDrive for Business (Updated 13 Jan
|
|
2021) (https://support.microsoft.com/en-us/office/invalid-file-names-and-file-types-in-onedrive-and-sharepoint-64883a5d-228e-48f5-b3d2-eb39e07630fa?ui=en-us&rs=en-us&ad=us#individualfilesize).
|
|
.SS Path length
|
|
.PP
|
|
The entire path, including the file name, must contain fewer than 400
|
|
characters for OneDrive, OneDrive for Business and SharePoint Online.
|
|
If you are encrypting file and folder names with rclone, you may want to
|
|
pay attention to this limitation because the encrypted names are
|
|
typically longer than the original ones.
|
|
.SS Number of files
|
|
.PP
|
|
OneDrive seems to be OK with at least 50,000 files in a folder, but at
|
|
100,000 rclone will get errors listing the directory like
|
|
\f[C]couldn\[cq]t list files: UnknownError:\f[R].
|
|
See #2707 (https://github.com/rclone/rclone/issues/2707) for more info.
|
|
.PP
|
|
An official document about the limitations for different types of
|
|
OneDrive can be found
|
|
here (https://support.office.com/en-us/article/invalid-file-names-and-file-types-in-onedrive-onedrive-for-business-and-sharepoint-64883a5d-228e-48f5-b3d2-eb39e07630fa).
|
|
.SS Versions
|
|
.PP
|
|
Every change in a file OneDrive causes the service to create a new
|
|
version of the the file.
|
|
This counts against a users quota.
|
|
For example changing the modification time of a file creates a second
|
|
version, so the file apparently uses twice the space.
|
|
.PP
|
|
For example the \f[C]copy\f[R] command is affected by this as rclone
|
|
copies the file and then afterwards sets the modification time to match
|
|
the source file which uses another version.
|
|
.PP
|
|
You can use the \f[C]rclone cleanup\f[R] command (see below) to remove
|
|
all old versions.
|
|
.PP
|
|
Or you can set the \f[C]no_versions\f[R] parameter to \f[C]true\f[R] and
|
|
rclone will remove versions after operations which create new versions.
|
|
This takes extra transactions so only enable it if you need it.
|
|
.PP
|
|
\f[B]Note\f[R] At the time of writing Onedrive Personal creates versions
|
|
(but not for setting the modification time) but the API for removing
|
|
them returns \[dq]API not found\[dq] so cleanup and
|
|
\f[C]no_versions\f[R] should not be used on Onedrive Personal.
|
|
.SS Disabling versioning
|
|
.PP
|
|
Starting October 2018, users will no longer be able to disable
|
|
versioning by default.
|
|
This is because Microsoft has brought an
|
|
update (https://techcommunity.microsoft.com/t5/Microsoft-OneDrive-Blog/New-Updates-to-OneDrive-and-SharePoint-Team-Site-Versioning/ba-p/204390)
|
|
to the mechanism.
|
|
To change this new default setting, a PowerShell command is required to
|
|
be run by a SharePoint admin.
|
|
If you are an admin, you can run these commands in PowerShell to change
|
|
that setting:
|
|
.IP "1." 3
|
|
\f[C]Install\-Module \-Name Microsoft.Online.SharePoint.PowerShell\f[R]
|
|
(in case you haven\[aq]t installed this already)
|
|
.IP "2." 3
|
|
\f[C]Import\-Module Microsoft.Online.SharePoint.PowerShell \-DisableNameChecking\f[R]
|
|
.IP "3." 3
|
|
\f[C]Connect\-SPOService \-Url https://YOURSITE\-admin.sharepoint.com \-Credential YOU\[at]YOURSITE.COM\f[R]
|
|
(replacing \f[C]YOURSITE\f[R], \f[C]YOU\f[R], \f[C]YOURSITE.COM\f[R]
|
|
with the actual values; this will prompt for your credentials)
|
|
.IP "4." 3
|
|
\f[C]Set\-SPOTenant \-EnableMinimumVersionRequirement $False\f[R]
|
|
.IP "5." 3
|
|
\f[C]Disconnect\-SPOService\f[R] (to disconnect from the server)
|
|
.PP
|
|
\f[I]Below are the steps for normal users to disable versioning. If you
|
|
don\[aq]t see the \[dq]No Versioning\[dq] option, make sure the above
|
|
requirements are met.\f[R]
|
|
.PP
|
|
User Weropol (https://github.com/Weropol) has found a method to disable
|
|
versioning on OneDrive
|
|
.IP "1." 3
|
|
Open the settings menu by clicking on the gear symbol at the top of the
|
|
OneDrive Business page.
|
|
.IP "2." 3
|
|
Click Site settings.
|
|
.IP "3." 3
|
|
Once on the Site settings page, navigate to Site Administration > Site
|
|
libraries and lists.
|
|
.IP "4." 3
|
|
Click Customize \[dq]Documents\[dq].
|
|
.IP "5." 3
|
|
Click General Settings > Versioning Settings.
|
|
.IP "6." 3
|
|
Under Document Version History select the option No versioning.
|
|
Note: This will disable the creation of new file versions, but will not
|
|
remove any previous versions.
|
|
Your documents are safe.
|
|
.IP "7." 3
|
|
Apply the changes by clicking OK.
|
|
.IP "8." 3
|
|
Use rclone to upload or modify files.
|
|
(I also use the \-\-no\-update\-modtime flag)
|
|
.IP "9." 3
|
|
Restore the versioning settings after using rclone.
|
|
(Optional)
|
|
.SS Cleanup
|
|
.PP
|
|
OneDrive supports \f[C]rclone cleanup\f[R] which causes rclone to look
|
|
through every file under the path supplied and delete all version but
|
|
the current version.
|
|
Because this involves traversing all the files, then querying each file
|
|
for versions it can be quite slow.
|
|
Rclone does \f[C]\-\-checkers\f[R] tests in parallel.
|
|
The command also supports \f[C]\-i\f[R] which is a great way to see what
|
|
it would do.
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone cleanup \-i remote:path/subdir # interactively remove all old version for path/subdir
|
|
rclone cleanup remote:path/subdir # unconditionally remove all old version for path/subdir
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
\f[B]NB\f[R] Onedrive personal can\[aq]t currently delete versions
|
|
.SS Troubleshooting
|
|
.SS Unexpected file size/hash differences on Sharepoint
|
|
.PP
|
|
It is a
|
|
known (https://github.com/OneDrive/onedrive-api-docs/issues/935#issuecomment-441741631)
|
|
issue that Sharepoint (not OneDrive or OneDrive for Business) silently
|
|
modifies uploaded files, mainly Office files (.docx, .xlsx, etc.),
|
|
causing file size and hash checks to fail.
|
|
There are also other situations that will cause OneDrive to report
|
|
inconsistent file sizes.
|
|
To use rclone with such affected files on Sharepoint, you may disable
|
|
these checks with the following command line arguments:
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
\-\-ignore\-checksum \-\-ignore\-size
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
Alternatively, if you have write access to the OneDrive files, it may be
|
|
possible to fix this problem for certain files, by attempting the steps
|
|
below.
|
|
Open the web interface for OneDrive (https://onedrive.live.com) and find
|
|
the affected files (which will be in the error messages/log for rclone).
|
|
Simply click on each of these files, causing OneDrive to open them on
|
|
the web.
|
|
This will cause each file to be converted in place to a format that is
|
|
functionally equivalent but which will no longer trigger the size
|
|
discrepancy.
|
|
Once all problematic files are converted you will no longer need the
|
|
ignore options above.
|
|
.SS Replacing/deleting existing files on Sharepoint gets \[dq]item not found\[dq]
|
|
.PP
|
|
It is a
|
|
known (https://github.com/OneDrive/onedrive-api-docs/issues/1068) issue
|
|
that Sharepoint (not OneDrive or OneDrive for Business) may return
|
|
\[dq]item not found\[dq] errors when users try to replace or delete
|
|
uploaded files; this seems to mainly affect Office files (.docx, .xlsx,
|
|
etc.).
|
|
As a workaround, you may use the \f[C]\-\-backup\-dir <BACKUP_DIR>\f[R]
|
|
command line argument so rclone moves the files to be replaced/deleted
|
|
into a given backup directory (instead of directly replacing/deleting
|
|
them).
|
|
For example, to instruct rclone to move the files into the directory
|
|
\f[C]rclone\-backup\-dir\f[R] on backend \f[C]mysharepoint\f[R], you may
|
|
use:
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
\-\-backup\-dir mysharepoint:rclone\-backup\-dir
|
|
\f[R]
|
|
.fi
|
|
.SS access_denied (AADSTS65005)
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
Error: access_denied
|
|
Code: AADSTS65005
|
|
Description: Using application \[aq]rclone\[aq] is currently not supported for your organization [YOUR_ORGANIZATION] because it is in an unmanaged state. An administrator needs to claim ownership of the company by DNS validation of [YOUR_ORGANIZATION] before the application rclone can be provisioned.
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
This means that rclone can\[aq]t use the OneDrive for Business API with
|
|
your account.
|
|
You can\[aq]t do much about it, maybe write an email to your admins.
|
|
.PP
|
|
However, there are other ways to interact with your OneDrive account.
|
|
Have a look at the webdav backend: https://rclone.org/webdav/#sharepoint
|
|
.SS invalid_grant (AADSTS50076)
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
Error: invalid_grant
|
|
Code: AADSTS50076
|
|
Description: Due to a configuration change made by your administrator, or because you moved to a new location, you must use multi\-factor authentication to access \[aq]...\[aq].
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
If you see the error above after enabling multi\-factor authentication
|
|
for your account, you can fix it by refreshing your OAuth refresh token.
|
|
To do that, run \f[C]rclone config\f[R], and choose to edit your
|
|
OneDrive backend.
|
|
Then, you don\[aq]t need to actually make any changes until you reach
|
|
this question: \f[C]Already have a token \- refresh?\f[R].
|
|
For this question, answer \f[C]y\f[R] and go through the process to
|
|
refresh your token, just like the first time the backend is configured.
|
|
After this, rclone should work again for this backend.
|
|
.SS OpenDrive
|
|
.PP
|
|
Paths are specified as \f[C]remote:path\f[R]
|
|
.PP
|
|
Paths may be as deep as required, e.g.
|
|
\f[C]remote:directory/subdirectory\f[R].
|
|
.PP
|
|
Here is an example of how to make a remote called \f[C]remote\f[R].
|
|
First run:
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone config
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
This will guide you through an interactive setup process:
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
n) New remote
|
|
d) Delete remote
|
|
q) Quit config
|
|
e/n/d/q> n
|
|
name> remote
|
|
Type of storage to configure.
|
|
Choose a number from below, or type in your own value
|
|
[snip]
|
|
XX / OpenDrive
|
|
\[rs] \[dq]opendrive\[dq]
|
|
[snip]
|
|
Storage> opendrive
|
|
Username
|
|
username>
|
|
Password
|
|
y) Yes type in my own password
|
|
g) Generate random password
|
|
y/g> y
|
|
Enter the password:
|
|
password:
|
|
Confirm the password:
|
|
password:
|
|
\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-
|
|
[remote]
|
|
username =
|
|
password = *** ENCRYPTED ***
|
|
\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-
|
|
y) Yes this is OK
|
|
e) Edit this remote
|
|
d) Delete this remote
|
|
y/e/d> y
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
List directories in top level of your OpenDrive
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone lsd remote:
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
List all the files in your OpenDrive
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone ls remote:
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
To copy a local directory to an OpenDrive directory called backup
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone copy /home/source remote:backup
|
|
\f[R]
|
|
.fi
|
|
.SS Modified time and MD5SUMs
|
|
.PP
|
|
OpenDrive allows modification times to be set on objects accurate to 1
|
|
second.
|
|
These will be used to detect whether objects need syncing or not.
|
|
.SS Restricted filename characters
|
|
.PP
|
|
.TS
|
|
tab(@);
|
|
l c c.
|
|
T{
|
|
Character
|
|
T}@T{
|
|
Value
|
|
T}@T{
|
|
Replacement
|
|
T}
|
|
_
|
|
T{
|
|
NUL
|
|
T}@T{
|
|
0x00
|
|
T}@T{
|
|
\[u2400]
|
|
T}
|
|
T{
|
|
/
|
|
T}@T{
|
|
0x2F
|
|
T}@T{
|
|
\[uFF0F]
|
|
T}
|
|
T{
|
|
\[dq]
|
|
T}@T{
|
|
0x22
|
|
T}@T{
|
|
\[uFF02]
|
|
T}
|
|
T{
|
|
*
|
|
T}@T{
|
|
0x2A
|
|
T}@T{
|
|
\[uFF0A]
|
|
T}
|
|
T{
|
|
:
|
|
T}@T{
|
|
0x3A
|
|
T}@T{
|
|
\[uFF1A]
|
|
T}
|
|
T{
|
|
<
|
|
T}@T{
|
|
0x3C
|
|
T}@T{
|
|
\[uFF1C]
|
|
T}
|
|
T{
|
|
>
|
|
T}@T{
|
|
0x3E
|
|
T}@T{
|
|
\[uFF1E]
|
|
T}
|
|
T{
|
|
?
|
|
T}@T{
|
|
0x3F
|
|
T}@T{
|
|
\[uFF1F]
|
|
T}
|
|
T{
|
|
\[rs]
|
|
T}@T{
|
|
0x5C
|
|
T}@T{
|
|
\[uFF3C]
|
|
T}
|
|
T{
|
|
|
|
|
T}@T{
|
|
0x7C
|
|
T}@T{
|
|
\[uFF5C]
|
|
T}
|
|
.TE
|
|
.PP
|
|
File names can also not begin or end with the following characters.
|
|
These only get replaced if they are the first or last character in the
|
|
name:
|
|
.PP
|
|
.TS
|
|
tab(@);
|
|
l c c.
|
|
T{
|
|
Character
|
|
T}@T{
|
|
Value
|
|
T}@T{
|
|
Replacement
|
|
T}
|
|
_
|
|
T{
|
|
SP
|
|
T}@T{
|
|
0x20
|
|
T}@T{
|
|
\[u2420]
|
|
T}
|
|
T{
|
|
HT
|
|
T}@T{
|
|
0x09
|
|
T}@T{
|
|
\[u2409]
|
|
T}
|
|
T{
|
|
LF
|
|
T}@T{
|
|
0x0A
|
|
T}@T{
|
|
\[u240A]
|
|
T}
|
|
T{
|
|
VT
|
|
T}@T{
|
|
0x0B
|
|
T}@T{
|
|
\[u240B]
|
|
T}
|
|
T{
|
|
CR
|
|
T}@T{
|
|
0x0D
|
|
T}@T{
|
|
\[u240D]
|
|
T}
|
|
.TE
|
|
.PP
|
|
Invalid UTF\-8 bytes will also be
|
|
replaced (https://rclone.org/overview/#invalid-utf8), as they can\[aq]t
|
|
be used in JSON strings.
|
|
.SS Standard Options
|
|
.PP
|
|
Here are the standard options specific to opendrive (OpenDrive).
|
|
.SS \-\-opendrive\-username
|
|
.PP
|
|
Username
|
|
.IP \[bu] 2
|
|
Config: username
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_OPENDRIVE_USERNAME
|
|
.IP \[bu] 2
|
|
Type: string
|
|
.IP \[bu] 2
|
|
Default: \[dq]\[dq]
|
|
.SS \-\-opendrive\-password
|
|
.PP
|
|
Password.
|
|
.PP
|
|
\f[B]NB\f[R] Input to this must be obscured \- see rclone
|
|
obscure (https://rclone.org/commands/rclone_obscure/).
|
|
.IP \[bu] 2
|
|
Config: password
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_OPENDRIVE_PASSWORD
|
|
.IP \[bu] 2
|
|
Type: string
|
|
.IP \[bu] 2
|
|
Default: \[dq]\[dq]
|
|
.SS Advanced Options
|
|
.PP
|
|
Here are the advanced options specific to opendrive (OpenDrive).
|
|
.SS \-\-opendrive\-encoding
|
|
.PP
|
|
This sets the encoding for the backend.
|
|
.PP
|
|
See: the encoding section in the
|
|
overview (https://rclone.org/overview/#encoding) for more info.
|
|
.IP \[bu] 2
|
|
Config: encoding
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_OPENDRIVE_ENCODING
|
|
.IP \[bu] 2
|
|
Type: MultiEncoder
|
|
.IP \[bu] 2
|
|
Default:
|
|
Slash,LtGt,DoubleQuote,Colon,Question,Asterisk,Pipe,BackSlash,LeftSpace,LeftCrLfHtVt,RightSpace,RightCrLfHtVt,InvalidUtf8,Dot
|
|
.SS \-\-opendrive\-chunk\-size
|
|
.PP
|
|
Files will be uploaded in chunks this size.
|
|
.PP
|
|
Note that these chunks are buffered in memory so increasing them will
|
|
increase memory use.
|
|
.IP \[bu] 2
|
|
Config: chunk_size
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_OPENDRIVE_CHUNK_SIZE
|
|
.IP \[bu] 2
|
|
Type: SizeSuffix
|
|
.IP \[bu] 2
|
|
Default: 10M
|
|
.SS Limitations
|
|
.PP
|
|
Note that OpenDrive is case insensitive so you can\[aq]t have a file
|
|
called \[dq]Hello.doc\[dq] and one called \[dq]hello.doc\[dq].
|
|
.PP
|
|
There are quite a few characters that can\[aq]t be in OpenDrive file
|
|
names.
|
|
These can\[aq]t occur on Windows platforms, but on non\-Windows
|
|
platforms they are common.
|
|
Rclone will map these names to and from an identical looking unicode
|
|
equivalent.
|
|
For example if a file has a \f[C]?\f[R] in it will be mapped to
|
|
\f[C]\[uFF1F]\f[R] instead.
|
|
.PP
|
|
\f[C]rclone about\f[R] is not supported by the OpenDrive backend.
|
|
Backends without this capability cannot determine free space for an
|
|
rclone mount or use policy \f[C]mfs\f[R] (most free space) as a member
|
|
of an rclone union remote.
|
|
.PP
|
|
See List of backends that do not support rclone
|
|
about (https://rclone.org/overview/#optional-features) See rclone
|
|
about (https://rclone.org/commands/rclone_about/)
|
|
.SS QingStor
|
|
.PP
|
|
Paths are specified as \f[C]remote:bucket\f[R] (or \f[C]remote:\f[R] for
|
|
the \f[C]lsd\f[R] command.) You may put subdirectories in too, e.g.
|
|
\f[C]remote:bucket/path/to/dir\f[R].
|
|
.PP
|
|
Here is an example of making an QingStor configuration.
|
|
First run
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone config
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
This will guide you through an interactive setup process.
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
No remotes found \- make a new one
|
|
n) New remote
|
|
r) Rename remote
|
|
c) Copy remote
|
|
s) Set configuration password
|
|
q) Quit config
|
|
n/r/c/s/q> n
|
|
name> remote
|
|
Type of storage to configure.
|
|
Choose a number from below, or type in your own value
|
|
[snip]
|
|
XX / QingStor Object Storage
|
|
\[rs] \[dq]qingstor\[dq]
|
|
[snip]
|
|
Storage> qingstor
|
|
Get QingStor credentials from runtime. Only applies if access_key_id and secret_access_key is blank.
|
|
Choose a number from below, or type in your own value
|
|
1 / Enter QingStor credentials in the next step
|
|
\[rs] \[dq]false\[dq]
|
|
2 / Get QingStor credentials from the environment (env vars or IAM)
|
|
\[rs] \[dq]true\[dq]
|
|
env_auth> 1
|
|
QingStor Access Key ID \- leave blank for anonymous access or runtime credentials.
|
|
access_key_id> access_key
|
|
QingStor Secret Access Key (password) \- leave blank for anonymous access or runtime credentials.
|
|
secret_access_key> secret_key
|
|
Enter an endpoint URL to connection QingStor API.
|
|
Leave blank will use the default value \[dq]https://qingstor.com:443\[dq]
|
|
endpoint>
|
|
Zone connect to. Default is \[dq]pek3a\[dq].
|
|
Choose a number from below, or type in your own value
|
|
/ The Beijing (China) Three Zone
|
|
1 | Needs location constraint pek3a.
|
|
\[rs] \[dq]pek3a\[dq]
|
|
/ The Shanghai (China) First Zone
|
|
2 | Needs location constraint sh1a.
|
|
\[rs] \[dq]sh1a\[dq]
|
|
zone> 1
|
|
Number of connection retry.
|
|
Leave blank will use the default value \[dq]3\[dq].
|
|
connection_retries>
|
|
Remote config
|
|
\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-
|
|
[remote]
|
|
env_auth = false
|
|
access_key_id = access_key
|
|
secret_access_key = secret_key
|
|
endpoint =
|
|
zone = pek3a
|
|
connection_retries =
|
|
\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-
|
|
y) Yes this is OK
|
|
e) Edit this remote
|
|
d) Delete this remote
|
|
y/e/d> y
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
This remote is called \f[C]remote\f[R] and can now be used like this
|
|
.PP
|
|
See all buckets
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone lsd remote:
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
Make a new bucket
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone mkdir remote:bucket
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
List the contents of a bucket
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone ls remote:bucket
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
Sync \f[C]/home/local/directory\f[R] to the remote bucket, deleting any
|
|
excess files in the bucket.
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone sync \-i /home/local/directory remote:bucket
|
|
\f[R]
|
|
.fi
|
|
.SS \-\-fast\-list
|
|
.PP
|
|
This remote supports \f[C]\-\-fast\-list\f[R] which allows you to use
|
|
fewer transactions in exchange for more memory.
|
|
See the rclone docs (https://rclone.org/docs/#fast-list) for more
|
|
details.
|
|
.SS Multipart uploads
|
|
.PP
|
|
rclone supports multipart uploads with QingStor which means that it can
|
|
upload files bigger than 5GB.
|
|
Note that files uploaded with multipart upload don\[aq]t have an MD5SUM.
|
|
.PP
|
|
Note that incomplete multipart uploads older than 24 hours can be
|
|
removed with \f[C]rclone cleanup remote:bucket\f[R] just for one bucket
|
|
\f[C]rclone cleanup remote:\f[R] for all buckets.
|
|
QingStor does not ever remove incomplete multipart uploads so it may be
|
|
necessary to run this from time to time.
|
|
.SS Buckets and Zone
|
|
.PP
|
|
With QingStor you can list buckets (\f[C]rclone lsd\f[R]) using any
|
|
zone, but you can only access the content of a bucket from the zone it
|
|
was created in.
|
|
If you attempt to access a bucket from the wrong zone, you will get an
|
|
error,
|
|
\f[C]incorrect zone, the bucket is not in \[aq]XXX\[aq] zone\f[R].
|
|
.SS Authentication
|
|
.PP
|
|
There are two ways to supply \f[C]rclone\f[R] with a set of QingStor
|
|
credentials.
|
|
In order of precedence:
|
|
.IP \[bu] 2
|
|
Directly in the rclone configuration file (as configured by
|
|
\f[C]rclone config\f[R])
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
set \f[C]access_key_id\f[R] and \f[C]secret_access_key\f[R]
|
|
.RE
|
|
.IP \[bu] 2
|
|
Runtime configuration:
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
set \f[C]env_auth\f[R] to \f[C]true\f[R] in the config file
|
|
.IP \[bu] 2
|
|
Exporting the following environment variables before running
|
|
\f[C]rclone\f[R]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Access Key ID: \f[C]QS_ACCESS_KEY_ID\f[R] or \f[C]QS_ACCESS_KEY\f[R]
|
|
.IP \[bu] 2
|
|
Secret Access Key: \f[C]QS_SECRET_ACCESS_KEY\f[R] or
|
|
\f[C]QS_SECRET_KEY\f[R]
|
|
.RE
|
|
.RE
|
|
.SS Restricted filename characters
|
|
.PP
|
|
The control characters 0x00\-0x1F and / are replaced as in the default
|
|
restricted characters
|
|
set (https://rclone.org/overview/#restricted-characters).
|
|
Note that 0x7F is not replaced.
|
|
.PP
|
|
Invalid UTF\-8 bytes will also be
|
|
replaced (https://rclone.org/overview/#invalid-utf8), as they can\[aq]t
|
|
be used in JSON strings.
|
|
.SS Standard Options
|
|
.PP
|
|
Here are the standard options specific to qingstor (QingCloud Object
|
|
Storage).
|
|
.SS \-\-qingstor\-env\-auth
|
|
.PP
|
|
Get QingStor credentials from runtime.
|
|
Only applies if access_key_id and secret_access_key is blank.
|
|
.IP \[bu] 2
|
|
Config: env_auth
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_QINGSTOR_ENV_AUTH
|
|
.IP \[bu] 2
|
|
Type: bool
|
|
.IP \[bu] 2
|
|
Default: false
|
|
.IP \[bu] 2
|
|
Examples:
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
\[dq]false\[dq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Enter QingStor credentials in the next step
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[dq]true\[dq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Get QingStor credentials from the environment (env vars or IAM)
|
|
.RE
|
|
.RE
|
|
.SS \-\-qingstor\-access\-key\-id
|
|
.PP
|
|
QingStor Access Key ID Leave blank for anonymous access or runtime
|
|
credentials.
|
|
.IP \[bu] 2
|
|
Config: access_key_id
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_QINGSTOR_ACCESS_KEY_ID
|
|
.IP \[bu] 2
|
|
Type: string
|
|
.IP \[bu] 2
|
|
Default: \[dq]\[dq]
|
|
.SS \-\-qingstor\-secret\-access\-key
|
|
.PP
|
|
QingStor Secret Access Key (password) Leave blank for anonymous access
|
|
or runtime credentials.
|
|
.IP \[bu] 2
|
|
Config: secret_access_key
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_QINGSTOR_SECRET_ACCESS_KEY
|
|
.IP \[bu] 2
|
|
Type: string
|
|
.IP \[bu] 2
|
|
Default: \[dq]\[dq]
|
|
.SS \-\-qingstor\-endpoint
|
|
.PP
|
|
Enter an endpoint URL to connection QingStor API.
|
|
Leave blank will use the default value
|
|
\[dq]https://qingstor.com:443\[dq]
|
|
.IP \[bu] 2
|
|
Config: endpoint
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_QINGSTOR_ENDPOINT
|
|
.IP \[bu] 2
|
|
Type: string
|
|
.IP \[bu] 2
|
|
Default: \[dq]\[dq]
|
|
.SS \-\-qingstor\-zone
|
|
.PP
|
|
Zone to connect to.
|
|
Default is \[dq]pek3a\[dq].
|
|
.IP \[bu] 2
|
|
Config: zone
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_QINGSTOR_ZONE
|
|
.IP \[bu] 2
|
|
Type: string
|
|
.IP \[bu] 2
|
|
Default: \[dq]\[dq]
|
|
.IP \[bu] 2
|
|
Examples:
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
\[dq]pek3a\[dq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
The Beijing (China) Three Zone
|
|
.IP \[bu] 2
|
|
Needs location constraint pek3a.
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[dq]sh1a\[dq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
The Shanghai (China) First Zone
|
|
.IP \[bu] 2
|
|
Needs location constraint sh1a.
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[dq]gd2a\[dq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
The Guangdong (China) Second Zone
|
|
.IP \[bu] 2
|
|
Needs location constraint gd2a.
|
|
.RE
|
|
.RE
|
|
.SS Advanced Options
|
|
.PP
|
|
Here are the advanced options specific to qingstor (QingCloud Object
|
|
Storage).
|
|
.SS \-\-qingstor\-connection\-retries
|
|
.PP
|
|
Number of connection retries.
|
|
.IP \[bu] 2
|
|
Config: connection_retries
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_QINGSTOR_CONNECTION_RETRIES
|
|
.IP \[bu] 2
|
|
Type: int
|
|
.IP \[bu] 2
|
|
Default: 3
|
|
.SS \-\-qingstor\-upload\-cutoff
|
|
.PP
|
|
Cutoff for switching to chunked upload
|
|
.PP
|
|
Any files larger than this will be uploaded in chunks of chunk_size.
|
|
The minimum is 0 and the maximum is 5GB.
|
|
.IP \[bu] 2
|
|
Config: upload_cutoff
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_QINGSTOR_UPLOAD_CUTOFF
|
|
.IP \[bu] 2
|
|
Type: SizeSuffix
|
|
.IP \[bu] 2
|
|
Default: 200M
|
|
.SS \-\-qingstor\-chunk\-size
|
|
.PP
|
|
Chunk size to use for uploading.
|
|
.PP
|
|
When uploading files larger than upload_cutoff they will be uploaded as
|
|
multipart uploads using this chunk size.
|
|
.PP
|
|
Note that \[dq]\-\-qingstor\-upload\-concurrency\[dq] chunks of this
|
|
size are buffered in memory per transfer.
|
|
.PP
|
|
If you are transferring large files over high\-speed links and you have
|
|
enough memory, then increasing this will speed up the transfers.
|
|
.IP \[bu] 2
|
|
Config: chunk_size
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_QINGSTOR_CHUNK_SIZE
|
|
.IP \[bu] 2
|
|
Type: SizeSuffix
|
|
.IP \[bu] 2
|
|
Default: 4M
|
|
.SS \-\-qingstor\-upload\-concurrency
|
|
.PP
|
|
Concurrency for multipart uploads.
|
|
.PP
|
|
This is the number of chunks of the same file that are uploaded
|
|
concurrently.
|
|
.PP
|
|
NB if you set this to > 1 then the checksums of multipart uploads become
|
|
corrupted (the uploads themselves are not corrupted though).
|
|
.PP
|
|
If you are uploading small numbers of large files over high\-speed links
|
|
and these uploads do not fully utilize your bandwidth, then increasing
|
|
this may help to speed up the transfers.
|
|
.IP \[bu] 2
|
|
Config: upload_concurrency
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_QINGSTOR_UPLOAD_CONCURRENCY
|
|
.IP \[bu] 2
|
|
Type: int
|
|
.IP \[bu] 2
|
|
Default: 1
|
|
.SS \-\-qingstor\-encoding
|
|
.PP
|
|
This sets the encoding for the backend.
|
|
.PP
|
|
See: the encoding section in the
|
|
overview (https://rclone.org/overview/#encoding) for more info.
|
|
.IP \[bu] 2
|
|
Config: encoding
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_QINGSTOR_ENCODING
|
|
.IP \[bu] 2
|
|
Type: MultiEncoder
|
|
.IP \[bu] 2
|
|
Default: Slash,Ctl,InvalidUtf8
|
|
.SS Limitations
|
|
.PP
|
|
\f[C]rclone about\f[R] is not supported by the qingstor backend.
|
|
Backends without this capability cannot determine free space for an
|
|
rclone mount or use policy \f[C]mfs\f[R] (most free space) as a member
|
|
of an rclone union remote.
|
|
.PP
|
|
See List of backends that do not support rclone
|
|
about (https://rclone.org/overview/#optional-features) See rclone
|
|
about (https://rclone.org/commands/rclone_about/)
|
|
.SS Swift
|
|
.PP
|
|
Swift refers to OpenStack Object
|
|
Storage (https://docs.openstack.org/swift/latest/).
|
|
Commercial implementations of that being:
|
|
.IP \[bu] 2
|
|
Rackspace Cloud Files (https://www.rackspace.com/cloud/files/)
|
|
.IP \[bu] 2
|
|
Memset Memstore (https://www.memset.com/cloud/storage/)
|
|
.IP \[bu] 2
|
|
OVH Object
|
|
Storage (https://www.ovh.co.uk/public-cloud/storage/object-storage/)
|
|
.IP \[bu] 2
|
|
Oracle Cloud Storage (https://cloud.oracle.com/storage-opc)
|
|
.IP \[bu] 2
|
|
IBM Bluemix Cloud ObjectStorage
|
|
Swift (https://console.bluemix.net/docs/infrastructure/objectstorage-swift/index.html)
|
|
.PP
|
|
Paths are specified as \f[C]remote:container\f[R] (or \f[C]remote:\f[R]
|
|
for the \f[C]lsd\f[R] command.) You may put subdirectories in too, e.g.
|
|
\f[C]remote:container/path/to/dir\f[R].
|
|
.PP
|
|
Here is an example of making a swift configuration.
|
|
First run
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone config
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
This will guide you through an interactive setup process.
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
No remotes found \- make a new one
|
|
n) New remote
|
|
s) Set configuration password
|
|
q) Quit config
|
|
n/s/q> n
|
|
name> remote
|
|
Type of storage to configure.
|
|
Choose a number from below, or type in your own value
|
|
[snip]
|
|
XX / OpenStack Swift (Rackspace Cloud Files, Memset Memstore, OVH)
|
|
\[rs] \[dq]swift\[dq]
|
|
[snip]
|
|
Storage> swift
|
|
Get swift credentials from environment variables in standard OpenStack form.
|
|
Choose a number from below, or type in your own value
|
|
1 / Enter swift credentials in the next step
|
|
\[rs] \[dq]false\[dq]
|
|
2 / Get swift credentials from environment vars. Leave other fields blank if using this.
|
|
\[rs] \[dq]true\[dq]
|
|
env_auth> true
|
|
User name to log in (OS_USERNAME).
|
|
user>
|
|
API key or password (OS_PASSWORD).
|
|
key>
|
|
Authentication URL for server (OS_AUTH_URL).
|
|
Choose a number from below, or type in your own value
|
|
1 / Rackspace US
|
|
\[rs] \[dq]https://auth.api.rackspacecloud.com/v1.0\[dq]
|
|
2 / Rackspace UK
|
|
\[rs] \[dq]https://lon.auth.api.rackspacecloud.com/v1.0\[dq]
|
|
3 / Rackspace v2
|
|
\[rs] \[dq]https://identity.api.rackspacecloud.com/v2.0\[dq]
|
|
4 / Memset Memstore UK
|
|
\[rs] \[dq]https://auth.storage.memset.com/v1.0\[dq]
|
|
5 / Memset Memstore UK v2
|
|
\[rs] \[dq]https://auth.storage.memset.com/v2.0\[dq]
|
|
6 / OVH
|
|
\[rs] \[dq]https://auth.cloud.ovh.net/v3\[dq]
|
|
auth>
|
|
User ID to log in \- optional \- most swift systems use user and leave this blank (v3 auth) (OS_USER_ID).
|
|
user_id>
|
|
User domain \- optional (v3 auth) (OS_USER_DOMAIN_NAME)
|
|
domain>
|
|
Tenant name \- optional for v1 auth, this or tenant_id required otherwise (OS_TENANT_NAME or OS_PROJECT_NAME)
|
|
tenant>
|
|
Tenant ID \- optional for v1 auth, this or tenant required otherwise (OS_TENANT_ID)
|
|
tenant_id>
|
|
Tenant domain \- optional (v3 auth) (OS_PROJECT_DOMAIN_NAME)
|
|
tenant_domain>
|
|
Region name \- optional (OS_REGION_NAME)
|
|
region>
|
|
Storage URL \- optional (OS_STORAGE_URL)
|
|
storage_url>
|
|
Auth Token from alternate authentication \- optional (OS_AUTH_TOKEN)
|
|
auth_token>
|
|
AuthVersion \- optional \- set to (1,2,3) if your auth URL has no version (ST_AUTH_VERSION)
|
|
auth_version>
|
|
Endpoint type to choose from the service catalogue (OS_ENDPOINT_TYPE)
|
|
Choose a number from below, or type in your own value
|
|
1 / Public (default, choose this if not sure)
|
|
\[rs] \[dq]public\[dq]
|
|
2 / Internal (use internal service net)
|
|
\[rs] \[dq]internal\[dq]
|
|
3 / Admin
|
|
\[rs] \[dq]admin\[dq]
|
|
endpoint_type>
|
|
Remote config
|
|
\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-
|
|
[test]
|
|
env_auth = true
|
|
user =
|
|
key =
|
|
auth =
|
|
user_id =
|
|
domain =
|
|
tenant =
|
|
tenant_id =
|
|
tenant_domain =
|
|
region =
|
|
storage_url =
|
|
auth_token =
|
|
auth_version =
|
|
endpoint_type =
|
|
\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-
|
|
y) Yes this is OK
|
|
e) Edit this remote
|
|
d) Delete this remote
|
|
y/e/d> y
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
This remote is called \f[C]remote\f[R] and can now be used like this
|
|
.PP
|
|
See all containers
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone lsd remote:
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
Make a new container
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone mkdir remote:container
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
List the contents of a container
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone ls remote:container
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
Sync \f[C]/home/local/directory\f[R] to the remote container, deleting
|
|
any excess files in the container.
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone sync \-i /home/local/directory remote:container
|
|
\f[R]
|
|
.fi
|
|
.SS Configuration from an OpenStack credentials file
|
|
.PP
|
|
An OpenStack credentials file typically looks something something like
|
|
this (without the comments)
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
export OS_AUTH_URL=https://a.provider.net/v2.0
|
|
export OS_TENANT_ID=ffffffffffffffffffffffffffffffff
|
|
export OS_TENANT_NAME=\[dq]1234567890123456\[dq]
|
|
export OS_USERNAME=\[dq]123abc567xy\[dq]
|
|
echo \[dq]Please enter your OpenStack Password: \[dq]
|
|
read \-sr OS_PASSWORD_INPUT
|
|
export OS_PASSWORD=$OS_PASSWORD_INPUT
|
|
export OS_REGION_NAME=\[dq]SBG1\[dq]
|
|
if [ \-z \[dq]$OS_REGION_NAME\[dq] ]; then unset OS_REGION_NAME; fi
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
The config file needs to look something like this where
|
|
\f[C]$OS_USERNAME\f[R] represents the value of the \f[C]OS_USERNAME\f[R]
|
|
variable \- \f[C]123abc567xy\f[R] in the example above.
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
[remote]
|
|
type = swift
|
|
user = $OS_USERNAME
|
|
key = $OS_PASSWORD
|
|
auth = $OS_AUTH_URL
|
|
tenant = $OS_TENANT_NAME
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
Note that you may (or may not) need to set \f[C]region\f[R] too \- try
|
|
without first.
|
|
.SS Configuration from the environment
|
|
.PP
|
|
If you prefer you can configure rclone to use swift using a standard set
|
|
of OpenStack environment variables.
|
|
.PP
|
|
When you run through the config, make sure you choose \f[C]true\f[R] for
|
|
\f[C]env_auth\f[R] and leave everything else blank.
|
|
.PP
|
|
rclone will then set any empty config parameters from the environment
|
|
using standard OpenStack environment variables.
|
|
There is a list of the
|
|
variables (https://godoc.org/github.com/ncw/swift#Connection.ApplyEnvironment)
|
|
in the docs for the swift library.
|
|
.SS Using an alternate authentication method
|
|
.PP
|
|
If your OpenStack installation uses a non\-standard authentication
|
|
method that might not be yet supported by rclone or the underlying swift
|
|
library, you can authenticate externally (e.g.
|
|
calling manually the \f[C]openstack\f[R] commands to get a token).
|
|
Then, you just need to pass the two configuration variables
|
|
\f[C]auth_token\f[R] and \f[C]storage_url\f[R].
|
|
If they are both provided, the other variables are ignored.
|
|
rclone will not try to authenticate but instead assume it is already
|
|
authenticated and use these two variables to access the OpenStack
|
|
installation.
|
|
.SS Using rclone without a config file
|
|
.PP
|
|
You can use rclone with swift without a config file, if desired, like
|
|
this:
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
source openstack\-credentials\-file
|
|
export RCLONE_CONFIG_MYREMOTE_TYPE=swift
|
|
export RCLONE_CONFIG_MYREMOTE_ENV_AUTH=true
|
|
rclone lsd myremote:
|
|
\f[R]
|
|
.fi
|
|
.SS \-\-fast\-list
|
|
.PP
|
|
This remote supports \f[C]\-\-fast\-list\f[R] which allows you to use
|
|
fewer transactions in exchange for more memory.
|
|
See the rclone docs (https://rclone.org/docs/#fast-list) for more
|
|
details.
|
|
.SS \-\-update and \-\-use\-server\-modtime
|
|
.PP
|
|
As noted below, the modified time is stored on metadata on the object.
|
|
It is used by default for all operations that require checking the time
|
|
a file was last updated.
|
|
It allows rclone to treat the remote more like a true filesystem, but it
|
|
is inefficient because it requires an extra API call to retrieve the
|
|
metadata.
|
|
.PP
|
|
For many operations, the time the object was last uploaded to the remote
|
|
is sufficient to determine if it is \[dq]dirty\[dq].
|
|
By using \f[C]\-\-update\f[R] along with
|
|
\f[C]\-\-use\-server\-modtime\f[R], you can avoid the extra API call and
|
|
simply upload files whose local modtime is newer than the time it was
|
|
last uploaded.
|
|
.SS Standard Options
|
|
.PP
|
|
Here are the standard options specific to swift (OpenStack Swift
|
|
(Rackspace Cloud Files, Memset Memstore, OVH)).
|
|
.SS \-\-swift\-env\-auth
|
|
.PP
|
|
Get swift credentials from environment variables in standard OpenStack
|
|
form.
|
|
.IP \[bu] 2
|
|
Config: env_auth
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_SWIFT_ENV_AUTH
|
|
.IP \[bu] 2
|
|
Type: bool
|
|
.IP \[bu] 2
|
|
Default: false
|
|
.IP \[bu] 2
|
|
Examples:
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
\[dq]false\[dq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Enter swift credentials in the next step
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[dq]true\[dq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Get swift credentials from environment vars.
|
|
Leave other fields blank if using this.
|
|
.RE
|
|
.RE
|
|
.SS \-\-swift\-user
|
|
.PP
|
|
User name to log in (OS_USERNAME).
|
|
.IP \[bu] 2
|
|
Config: user
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_SWIFT_USER
|
|
.IP \[bu] 2
|
|
Type: string
|
|
.IP \[bu] 2
|
|
Default: \[dq]\[dq]
|
|
.SS \-\-swift\-key
|
|
.PP
|
|
API key or password (OS_PASSWORD).
|
|
.IP \[bu] 2
|
|
Config: key
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_SWIFT_KEY
|
|
.IP \[bu] 2
|
|
Type: string
|
|
.IP \[bu] 2
|
|
Default: \[dq]\[dq]
|
|
.SS \-\-swift\-auth
|
|
.PP
|
|
Authentication URL for server (OS_AUTH_URL).
|
|
.IP \[bu] 2
|
|
Config: auth
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_SWIFT_AUTH
|
|
.IP \[bu] 2
|
|
Type: string
|
|
.IP \[bu] 2
|
|
Default: \[dq]\[dq]
|
|
.IP \[bu] 2
|
|
Examples:
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
\[dq]https://auth.api.rackspacecloud.com/v1.0\[dq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Rackspace US
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[dq]https://lon.auth.api.rackspacecloud.com/v1.0\[dq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Rackspace UK
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[dq]https://identity.api.rackspacecloud.com/v2.0\[dq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Rackspace v2
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[dq]https://auth.storage.memset.com/v1.0\[dq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Memset Memstore UK
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[dq]https://auth.storage.memset.com/v2.0\[dq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Memset Memstore UK v2
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[dq]https://auth.cloud.ovh.net/v3\[dq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
OVH
|
|
.RE
|
|
.RE
|
|
.SS \-\-swift\-user\-id
|
|
.PP
|
|
User ID to log in \- optional \- most swift systems use user and leave
|
|
this blank (v3 auth) (OS_USER_ID).
|
|
.IP \[bu] 2
|
|
Config: user_id
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_SWIFT_USER_ID
|
|
.IP \[bu] 2
|
|
Type: string
|
|
.IP \[bu] 2
|
|
Default: \[dq]\[dq]
|
|
.SS \-\-swift\-domain
|
|
.PP
|
|
User domain \- optional (v3 auth) (OS_USER_DOMAIN_NAME)
|
|
.IP \[bu] 2
|
|
Config: domain
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_SWIFT_DOMAIN
|
|
.IP \[bu] 2
|
|
Type: string
|
|
.IP \[bu] 2
|
|
Default: \[dq]\[dq]
|
|
.SS \-\-swift\-tenant
|
|
.PP
|
|
Tenant name \- optional for v1 auth, this or tenant_id required
|
|
otherwise (OS_TENANT_NAME or OS_PROJECT_NAME)
|
|
.IP \[bu] 2
|
|
Config: tenant
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_SWIFT_TENANT
|
|
.IP \[bu] 2
|
|
Type: string
|
|
.IP \[bu] 2
|
|
Default: \[dq]\[dq]
|
|
.SS \-\-swift\-tenant\-id
|
|
.PP
|
|
Tenant ID \- optional for v1 auth, this or tenant required otherwise
|
|
(OS_TENANT_ID)
|
|
.IP \[bu] 2
|
|
Config: tenant_id
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_SWIFT_TENANT_ID
|
|
.IP \[bu] 2
|
|
Type: string
|
|
.IP \[bu] 2
|
|
Default: \[dq]\[dq]
|
|
.SS \-\-swift\-tenant\-domain
|
|
.PP
|
|
Tenant domain \- optional (v3 auth) (OS_PROJECT_DOMAIN_NAME)
|
|
.IP \[bu] 2
|
|
Config: tenant_domain
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_SWIFT_TENANT_DOMAIN
|
|
.IP \[bu] 2
|
|
Type: string
|
|
.IP \[bu] 2
|
|
Default: \[dq]\[dq]
|
|
.SS \-\-swift\-region
|
|
.PP
|
|
Region name \- optional (OS_REGION_NAME)
|
|
.IP \[bu] 2
|
|
Config: region
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_SWIFT_REGION
|
|
.IP \[bu] 2
|
|
Type: string
|
|
.IP \[bu] 2
|
|
Default: \[dq]\[dq]
|
|
.SS \-\-swift\-storage\-url
|
|
.PP
|
|
Storage URL \- optional (OS_STORAGE_URL)
|
|
.IP \[bu] 2
|
|
Config: storage_url
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_SWIFT_STORAGE_URL
|
|
.IP \[bu] 2
|
|
Type: string
|
|
.IP \[bu] 2
|
|
Default: \[dq]\[dq]
|
|
.SS \-\-swift\-auth\-token
|
|
.PP
|
|
Auth Token from alternate authentication \- optional (OS_AUTH_TOKEN)
|
|
.IP \[bu] 2
|
|
Config: auth_token
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_SWIFT_AUTH_TOKEN
|
|
.IP \[bu] 2
|
|
Type: string
|
|
.IP \[bu] 2
|
|
Default: \[dq]\[dq]
|
|
.SS \-\-swift\-application\-credential\-id
|
|
.PP
|
|
Application Credential ID (OS_APPLICATION_CREDENTIAL_ID)
|
|
.IP \[bu] 2
|
|
Config: application_credential_id
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_SWIFT_APPLICATION_CREDENTIAL_ID
|
|
.IP \[bu] 2
|
|
Type: string
|
|
.IP \[bu] 2
|
|
Default: \[dq]\[dq]
|
|
.SS \-\-swift\-application\-credential\-name
|
|
.PP
|
|
Application Credential Name (OS_APPLICATION_CREDENTIAL_NAME)
|
|
.IP \[bu] 2
|
|
Config: application_credential_name
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_SWIFT_APPLICATION_CREDENTIAL_NAME
|
|
.IP \[bu] 2
|
|
Type: string
|
|
.IP \[bu] 2
|
|
Default: \[dq]\[dq]
|
|
.SS \-\-swift\-application\-credential\-secret
|
|
.PP
|
|
Application Credential Secret (OS_APPLICATION_CREDENTIAL_SECRET)
|
|
.IP \[bu] 2
|
|
Config: application_credential_secret
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_SWIFT_APPLICATION_CREDENTIAL_SECRET
|
|
.IP \[bu] 2
|
|
Type: string
|
|
.IP \[bu] 2
|
|
Default: \[dq]\[dq]
|
|
.SS \-\-swift\-auth\-version
|
|
.PP
|
|
AuthVersion \- optional \- set to (1,2,3) if your auth URL has no
|
|
version (ST_AUTH_VERSION)
|
|
.IP \[bu] 2
|
|
Config: auth_version
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_SWIFT_AUTH_VERSION
|
|
.IP \[bu] 2
|
|
Type: int
|
|
.IP \[bu] 2
|
|
Default: 0
|
|
.SS \-\-swift\-endpoint\-type
|
|
.PP
|
|
Endpoint type to choose from the service catalogue (OS_ENDPOINT_TYPE)
|
|
.IP \[bu] 2
|
|
Config: endpoint_type
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_SWIFT_ENDPOINT_TYPE
|
|
.IP \[bu] 2
|
|
Type: string
|
|
.IP \[bu] 2
|
|
Default: \[dq]public\[dq]
|
|
.IP \[bu] 2
|
|
Examples:
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
\[dq]public\[dq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Public (default, choose this if not sure)
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[dq]internal\[dq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Internal (use internal service net)
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[dq]admin\[dq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Admin
|
|
.RE
|
|
.RE
|
|
.SS \-\-swift\-storage\-policy
|
|
.PP
|
|
The storage policy to use when creating a new container
|
|
.PP
|
|
This applies the specified storage policy when creating a new container.
|
|
The policy cannot be changed afterwards.
|
|
The allowed configuration values and their meaning depend on your Swift
|
|
storage provider.
|
|
.IP \[bu] 2
|
|
Config: storage_policy
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_SWIFT_STORAGE_POLICY
|
|
.IP \[bu] 2
|
|
Type: string
|
|
.IP \[bu] 2
|
|
Default: \[dq]\[dq]
|
|
.IP \[bu] 2
|
|
Examples:
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
\[dq]\[dq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Default
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[dq]pcs\[dq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
OVH Public Cloud Storage
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[dq]pca\[dq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
OVH Public Cloud Archive
|
|
.RE
|
|
.RE
|
|
.SS Advanced Options
|
|
.PP
|
|
Here are the advanced options specific to swift (OpenStack Swift
|
|
(Rackspace Cloud Files, Memset Memstore, OVH)).
|
|
.SS \-\-swift\-leave\-parts\-on\-error
|
|
.PP
|
|
If true avoid calling abort upload on a failure.
|
|
It should be set to true for resuming uploads across different sessions.
|
|
.IP \[bu] 2
|
|
Config: leave_parts_on_error
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_SWIFT_LEAVE_PARTS_ON_ERROR
|
|
.IP \[bu] 2
|
|
Type: bool
|
|
.IP \[bu] 2
|
|
Default: false
|
|
.SS \-\-swift\-chunk\-size
|
|
.PP
|
|
Above this size files will be chunked into a _segments container.
|
|
.PP
|
|
Above this size files will be chunked into a _segments container.
|
|
The default for this is 5GB which is its maximum value.
|
|
.IP \[bu] 2
|
|
Config: chunk_size
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_SWIFT_CHUNK_SIZE
|
|
.IP \[bu] 2
|
|
Type: SizeSuffix
|
|
.IP \[bu] 2
|
|
Default: 5G
|
|
.SS \-\-swift\-no\-chunk
|
|
.PP
|
|
Don\[aq]t chunk files during streaming upload.
|
|
.PP
|
|
When doing streaming uploads (e.g.
|
|
using rcat or mount) setting this flag will cause the swift backend to
|
|
not upload chunked files.
|
|
.PP
|
|
This will limit the maximum upload size to 5GB.
|
|
However non chunked files are easier to deal with and have an MD5SUM.
|
|
.PP
|
|
Rclone will still chunk files bigger than chunk_size when doing normal
|
|
copy operations.
|
|
.IP \[bu] 2
|
|
Config: no_chunk
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_SWIFT_NO_CHUNK
|
|
.IP \[bu] 2
|
|
Type: bool
|
|
.IP \[bu] 2
|
|
Default: false
|
|
.SS \-\-swift\-encoding
|
|
.PP
|
|
This sets the encoding for the backend.
|
|
.PP
|
|
See: the encoding section in the
|
|
overview (https://rclone.org/overview/#encoding) for more info.
|
|
.IP \[bu] 2
|
|
Config: encoding
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_SWIFT_ENCODING
|
|
.IP \[bu] 2
|
|
Type: MultiEncoder
|
|
.IP \[bu] 2
|
|
Default: Slash,InvalidUtf8
|
|
.SS Modified time
|
|
.PP
|
|
The modified time is stored as metadata on the object as
|
|
\f[C]X\-Object\-Meta\-Mtime\f[R] as floating point since the epoch
|
|
accurate to 1 ns.
|
|
.PP
|
|
This is a de facto standard (used in the official python\-swiftclient
|
|
amongst others) for storing the modification time for an object.
|
|
.SS Restricted filename characters
|
|
.PP
|
|
.TS
|
|
tab(@);
|
|
l c c.
|
|
T{
|
|
Character
|
|
T}@T{
|
|
Value
|
|
T}@T{
|
|
Replacement
|
|
T}
|
|
_
|
|
T{
|
|
NUL
|
|
T}@T{
|
|
0x00
|
|
T}@T{
|
|
\[u2400]
|
|
T}
|
|
T{
|
|
/
|
|
T}@T{
|
|
0x2F
|
|
T}@T{
|
|
\[uFF0F]
|
|
T}
|
|
.TE
|
|
.PP
|
|
Invalid UTF\-8 bytes will also be
|
|
replaced (https://rclone.org/overview/#invalid-utf8), as they can\[aq]t
|
|
be used in JSON strings.
|
|
.SS Limitations
|
|
.PP
|
|
The Swift API doesn\[aq]t return a correct MD5SUM for segmented files
|
|
(Dynamic or Static Large Objects) so rclone won\[aq]t check or use the
|
|
MD5SUM for these.
|
|
.SS Troubleshooting
|
|
.SS Rclone gives Failed to create file system for \[dq]remote:\[dq]: Bad Request
|
|
.PP
|
|
Due to an oddity of the underlying swift library, it gives a \[dq]Bad
|
|
Request\[dq] error rather than a more sensible error when the
|
|
authentication fails for Swift.
|
|
.PP
|
|
So this most likely means your username / password is wrong.
|
|
You can investigate further with the \f[C]\-\-dump\-bodies\f[R] flag.
|
|
.PP
|
|
This may also be caused by specifying the region when you shouldn\[aq]t
|
|
have (e.g.
|
|
OVH).
|
|
.SS Rclone gives Failed to create file system: Response didn\[aq]t have storage url and auth token
|
|
.PP
|
|
This is most likely caused by forgetting to specify your tenant when
|
|
setting up a swift remote.
|
|
.SS pCloud
|
|
.PP
|
|
Paths are specified as \f[C]remote:path\f[R]
|
|
.PP
|
|
Paths may be as deep as required, e.g.
|
|
\f[C]remote:directory/subdirectory\f[R].
|
|
.PP
|
|
The initial setup for pCloud involves getting a token from pCloud which
|
|
you need to do in your browser.
|
|
\f[C]rclone config\f[R] walks you through it.
|
|
.PP
|
|
Here is an example of how to make a remote called \f[C]remote\f[R].
|
|
First run:
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone config
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
This will guide you through an interactive setup process:
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
No remotes found \- make a new one
|
|
n) New remote
|
|
s) Set configuration password
|
|
q) Quit config
|
|
n/s/q> n
|
|
name> remote
|
|
Type of storage to configure.
|
|
Choose a number from below, or type in your own value
|
|
[snip]
|
|
XX / Pcloud
|
|
\[rs] \[dq]pcloud\[dq]
|
|
[snip]
|
|
Storage> pcloud
|
|
Pcloud App Client Id \- leave blank normally.
|
|
client_id>
|
|
Pcloud App Client Secret \- leave blank normally.
|
|
client_secret>
|
|
Remote config
|
|
Use auto config?
|
|
* Say Y if not sure
|
|
* Say N if you are working on a remote or headless machine
|
|
y) Yes
|
|
n) No
|
|
y/n> y
|
|
If your browser doesn\[aq]t open automatically go to the following link: http://127.0.0.1:53682/auth
|
|
Log in and authorize rclone for access
|
|
Waiting for code...
|
|
Got code
|
|
\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-
|
|
[remote]
|
|
client_id =
|
|
client_secret =
|
|
token = {\[dq]access_token\[dq]:\[dq]XXX\[dq],\[dq]token_type\[dq]:\[dq]bearer\[dq],\[dq]expiry\[dq]:\[dq]0001\-01\-01T00:00:00Z\[dq]}
|
|
\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-
|
|
y) Yes this is OK
|
|
e) Edit this remote
|
|
d) Delete this remote
|
|
y/e/d> y
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
See the remote setup docs (https://rclone.org/remote_setup/) for how to
|
|
set it up on a machine with no Internet browser available.
|
|
.PP
|
|
Note that rclone runs a webserver on your local machine to collect the
|
|
token as returned from pCloud.
|
|
This only runs from the moment it opens your browser to the moment you
|
|
get back the verification code.
|
|
This is on \f[C]http://127.0.0.1:53682/\f[R] and this it may require you
|
|
to unblock it temporarily if you are running a host firewall.
|
|
.PP
|
|
Once configured you can then use \f[C]rclone\f[R] like this,
|
|
.PP
|
|
List directories in top level of your pCloud
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone lsd remote:
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
List all the files in your pCloud
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone ls remote:
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
To copy a local directory to an pCloud directory called backup
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone copy /home/source remote:backup
|
|
\f[R]
|
|
.fi
|
|
.SS Modified time and hashes
|
|
.PP
|
|
pCloud allows modification times to be set on objects accurate to 1
|
|
second.
|
|
These will be used to detect whether objects need syncing or not.
|
|
In order to set a Modification time pCloud requires the object be
|
|
re\-uploaded.
|
|
.PP
|
|
pCloud supports MD5 and SHA1 type hashes in the US region but and SHA1
|
|
only in the EU region, so you can use the \f[C]\-\-checksum\f[R] flag.
|
|
.PP
|
|
(Note that pCloud also support SHA256 in the EU region, but rclone does
|
|
not have support for that yet.)
|
|
.SS Restricted filename characters
|
|
.PP
|
|
In addition to the default restricted characters
|
|
set (https://rclone.org/overview/#restricted-characters) the following
|
|
characters are also replaced:
|
|
.PP
|
|
.TS
|
|
tab(@);
|
|
l c c.
|
|
T{
|
|
Character
|
|
T}@T{
|
|
Value
|
|
T}@T{
|
|
Replacement
|
|
T}
|
|
_
|
|
T{
|
|
\[rs]
|
|
T}@T{
|
|
0x5C
|
|
T}@T{
|
|
\[uFF3C]
|
|
T}
|
|
.TE
|
|
.PP
|
|
Invalid UTF\-8 bytes will also be
|
|
replaced (https://rclone.org/overview/#invalid-utf8), as they can\[aq]t
|
|
be used in JSON strings.
|
|
.SS Deleting files
|
|
.PP
|
|
Deleted files will be moved to the trash.
|
|
Your subscription level will determine how long items stay in the trash.
|
|
\f[C]rclone cleanup\f[R] can be used to empty the trash.
|
|
.SS Root folder ID
|
|
.PP
|
|
You can set the \f[C]root_folder_id\f[R] for rclone.
|
|
This is the directory (identified by its \f[C]Folder ID\f[R]) that
|
|
rclone considers to be the root of your pCloud drive.
|
|
.PP
|
|
Normally you will leave this blank and rclone will determine the correct
|
|
root to use itself.
|
|
.PP
|
|
However you can set this to restrict rclone to a specific folder
|
|
hierarchy.
|
|
.PP
|
|
In order to do this you will have to find the \f[C]Folder ID\f[R] of the
|
|
directory you wish rclone to display.
|
|
This will be the \f[C]folder\f[R] field of the URL when you open the
|
|
relevant folder in the pCloud web interface.
|
|
.PP
|
|
So if the folder you want rclone to use has a URL which looks like
|
|
\f[C]https://my.pcloud.com/#page=filemanager&folder=5xxxxxxxx8&tpl=foldergrid\f[R]
|
|
in the browser, then you use \f[C]5xxxxxxxx8\f[R] as the
|
|
\f[C]root_folder_id\f[R] in the config.
|
|
.SS Standard Options
|
|
.PP
|
|
Here are the standard options specific to pcloud (Pcloud).
|
|
.SS \-\-pcloud\-client\-id
|
|
.PP
|
|
OAuth Client Id Leave blank normally.
|
|
.IP \[bu] 2
|
|
Config: client_id
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_PCLOUD_CLIENT_ID
|
|
.IP \[bu] 2
|
|
Type: string
|
|
.IP \[bu] 2
|
|
Default: \[dq]\[dq]
|
|
.SS \-\-pcloud\-client\-secret
|
|
.PP
|
|
OAuth Client Secret Leave blank normally.
|
|
.IP \[bu] 2
|
|
Config: client_secret
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_PCLOUD_CLIENT_SECRET
|
|
.IP \[bu] 2
|
|
Type: string
|
|
.IP \[bu] 2
|
|
Default: \[dq]\[dq]
|
|
.SS Advanced Options
|
|
.PP
|
|
Here are the advanced options specific to pcloud (Pcloud).
|
|
.SS \-\-pcloud\-token
|
|
.PP
|
|
OAuth Access Token as a JSON blob.
|
|
.IP \[bu] 2
|
|
Config: token
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_PCLOUD_TOKEN
|
|
.IP \[bu] 2
|
|
Type: string
|
|
.IP \[bu] 2
|
|
Default: \[dq]\[dq]
|
|
.SS \-\-pcloud\-auth\-url
|
|
.PP
|
|
Auth server URL.
|
|
Leave blank to use the provider defaults.
|
|
.IP \[bu] 2
|
|
Config: auth_url
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_PCLOUD_AUTH_URL
|
|
.IP \[bu] 2
|
|
Type: string
|
|
.IP \[bu] 2
|
|
Default: \[dq]\[dq]
|
|
.SS \-\-pcloud\-token\-url
|
|
.PP
|
|
Token server url.
|
|
Leave blank to use the provider defaults.
|
|
.IP \[bu] 2
|
|
Config: token_url
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_PCLOUD_TOKEN_URL
|
|
.IP \[bu] 2
|
|
Type: string
|
|
.IP \[bu] 2
|
|
Default: \[dq]\[dq]
|
|
.SS \-\-pcloud\-encoding
|
|
.PP
|
|
This sets the encoding for the backend.
|
|
.PP
|
|
See: the encoding section in the
|
|
overview (https://rclone.org/overview/#encoding) for more info.
|
|
.IP \[bu] 2
|
|
Config: encoding
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_PCLOUD_ENCODING
|
|
.IP \[bu] 2
|
|
Type: MultiEncoder
|
|
.IP \[bu] 2
|
|
Default: Slash,BackSlash,Del,Ctl,InvalidUtf8,Dot
|
|
.SS \-\-pcloud\-root\-folder\-id
|
|
.PP
|
|
Fill in for rclone to use a non root folder as its starting point.
|
|
.IP \[bu] 2
|
|
Config: root_folder_id
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_PCLOUD_ROOT_FOLDER_ID
|
|
.IP \[bu] 2
|
|
Type: string
|
|
.IP \[bu] 2
|
|
Default: \[dq]d0\[dq]
|
|
.SS \-\-pcloud\-hostname
|
|
.PP
|
|
Hostname to connect to.
|
|
.PP
|
|
This is normally set when rclone initially does the oauth connection,
|
|
however you will need to set it by hand if you are using remote config
|
|
with rclone authorize.
|
|
.IP \[bu] 2
|
|
Config: hostname
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_PCLOUD_HOSTNAME
|
|
.IP \[bu] 2
|
|
Type: string
|
|
.IP \[bu] 2
|
|
Default: \[dq]api.pcloud.com\[dq]
|
|
.IP \[bu] 2
|
|
Examples:
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
\[dq]api.pcloud.com\[dq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Original/US region
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[dq]eapi.pcloud.com\[dq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
EU region
|
|
.RE
|
|
.RE
|
|
.SS premiumize.me
|
|
.PP
|
|
Paths are specified as \f[C]remote:path\f[R]
|
|
.PP
|
|
Paths may be as deep as required, e.g.
|
|
\f[C]remote:directory/subdirectory\f[R].
|
|
.PP
|
|
The initial setup for premiumize.me (https://premiumize.me/) involves
|
|
getting a token from premiumize.me which you need to do in your browser.
|
|
\f[C]rclone config\f[R] walks you through it.
|
|
.PP
|
|
Here is an example of how to make a remote called \f[C]remote\f[R].
|
|
First run:
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone config
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
This will guide you through an interactive setup process:
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
No remotes found \- make a new one
|
|
n) New remote
|
|
s) Set configuration password
|
|
q) Quit config
|
|
n/s/q> n
|
|
name> remote
|
|
Type of storage to configure.
|
|
Enter a string value. Press Enter for the default (\[dq]\[dq]).
|
|
Choose a number from below, or type in your own value
|
|
[snip]
|
|
XX / premiumize.me
|
|
\[rs] \[dq]premiumizeme\[dq]
|
|
[snip]
|
|
Storage> premiumizeme
|
|
** See help for premiumizeme backend at: https://rclone.org/premiumizeme/ **
|
|
|
|
Remote config
|
|
Use auto config?
|
|
* Say Y if not sure
|
|
* Say N if you are working on a remote or headless machine
|
|
y) Yes
|
|
n) No
|
|
y/n> y
|
|
If your browser doesn\[aq]t open automatically go to the following link: http://127.0.0.1:53682/auth
|
|
Log in and authorize rclone for access
|
|
Waiting for code...
|
|
Got code
|
|
\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-
|
|
[remote]
|
|
type = premiumizeme
|
|
token = {\[dq]access_token\[dq]:\[dq]XXX\[dq],\[dq]token_type\[dq]:\[dq]Bearer\[dq],\[dq]refresh_token\[dq]:\[dq]XXX\[dq],\[dq]expiry\[dq]:\[dq]2029\-08\-07T18:44:15.548915378+01:00\[dq]}
|
|
\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-
|
|
y) Yes this is OK
|
|
e) Edit this remote
|
|
d) Delete this remote
|
|
y/e/d>
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
See the remote setup docs (https://rclone.org/remote_setup/) for how to
|
|
set it up on a machine with no Internet browser available.
|
|
.PP
|
|
Note that rclone runs a webserver on your local machine to collect the
|
|
token as returned from premiumize.me.
|
|
This only runs from the moment it opens your browser to the moment you
|
|
get back the verification code.
|
|
This is on \f[C]http://127.0.0.1:53682/\f[R] and this it may require you
|
|
to unblock it temporarily if you are running a host firewall.
|
|
.PP
|
|
Once configured you can then use \f[C]rclone\f[R] like this,
|
|
.PP
|
|
List directories in top level of your premiumize.me
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone lsd remote:
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
List all the files in your premiumize.me
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone ls remote:
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
To copy a local directory to an premiumize.me directory called backup
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone copy /home/source remote:backup
|
|
\f[R]
|
|
.fi
|
|
.SS Modified time and hashes
|
|
.PP
|
|
premiumize.me does not support modification times or hashes, therefore
|
|
syncing will default to \f[C]\-\-size\-only\f[R] checking.
|
|
Note that using \f[C]\-\-update\f[R] will work.
|
|
.SS Restricted filename characters
|
|
.PP
|
|
In addition to the default restricted characters
|
|
set (https://rclone.org/overview/#restricted-characters) the following
|
|
characters are also replaced:
|
|
.PP
|
|
.TS
|
|
tab(@);
|
|
l c c.
|
|
T{
|
|
Character
|
|
T}@T{
|
|
Value
|
|
T}@T{
|
|
Replacement
|
|
T}
|
|
_
|
|
T{
|
|
\[rs]
|
|
T}@T{
|
|
0x5C
|
|
T}@T{
|
|
\[uFF3C]
|
|
T}
|
|
T{
|
|
\[dq]
|
|
T}@T{
|
|
0x22
|
|
T}@T{
|
|
\[uFF02]
|
|
T}
|
|
.TE
|
|
.PP
|
|
Invalid UTF\-8 bytes will also be
|
|
replaced (https://rclone.org/overview/#invalid-utf8), as they can\[aq]t
|
|
be used in JSON strings.
|
|
.SS Standard Options
|
|
.PP
|
|
Here are the standard options specific to premiumizeme (premiumize.me).
|
|
.SS \-\-premiumizeme\-api\-key
|
|
.PP
|
|
API Key.
|
|
.PP
|
|
This is not normally used \- use oauth instead.
|
|
.IP \[bu] 2
|
|
Config: api_key
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_PREMIUMIZEME_API_KEY
|
|
.IP \[bu] 2
|
|
Type: string
|
|
.IP \[bu] 2
|
|
Default: \[dq]\[dq]
|
|
.SS Advanced Options
|
|
.PP
|
|
Here are the advanced options specific to premiumizeme (premiumize.me).
|
|
.SS \-\-premiumizeme\-encoding
|
|
.PP
|
|
This sets the encoding for the backend.
|
|
.PP
|
|
See: the encoding section in the
|
|
overview (https://rclone.org/overview/#encoding) for more info.
|
|
.IP \[bu] 2
|
|
Config: encoding
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_PREMIUMIZEME_ENCODING
|
|
.IP \[bu] 2
|
|
Type: MultiEncoder
|
|
.IP \[bu] 2
|
|
Default: Slash,DoubleQuote,BackSlash,Del,Ctl,InvalidUtf8,Dot
|
|
.SS Limitations
|
|
.PP
|
|
Note that premiumize.me is case insensitive so you can\[aq]t have a file
|
|
called \[dq]Hello.doc\[dq] and one called \[dq]hello.doc\[dq].
|
|
.PP
|
|
premiumize.me file names can\[aq]t have the \f[C]\[rs]\f[R] or
|
|
\f[C]\[dq]\f[R] characters in.
|
|
rclone maps these to and from an identical looking unicode equivalents
|
|
\f[C]\[uFF3C]\f[R] and \f[C]\[uFF02]\f[R]
|
|
.PP
|
|
premiumize.me only supports filenames up to 255 characters in length.
|
|
.SS put.io
|
|
.PP
|
|
Paths are specified as \f[C]remote:path\f[R]
|
|
.PP
|
|
put.io paths may be as deep as required, e.g.
|
|
\f[C]remote:directory/subdirectory\f[R].
|
|
.PP
|
|
The initial setup for put.io involves getting a token from put.io which
|
|
you need to do in your browser.
|
|
\f[C]rclone config\f[R] walks you through it.
|
|
.PP
|
|
Here is an example of how to make a remote called \f[C]remote\f[R].
|
|
First run:
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone config
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
This will guide you through an interactive setup process:
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
No remotes found \- make a new one
|
|
n) New remote
|
|
s) Set configuration password
|
|
q) Quit config
|
|
n/s/q> n
|
|
name> putio
|
|
Type of storage to configure.
|
|
Enter a string value. Press Enter for the default (\[dq]\[dq]).
|
|
Choose a number from below, or type in your own value
|
|
[snip]
|
|
XX / Put.io
|
|
\[rs] \[dq]putio\[dq]
|
|
[snip]
|
|
Storage> putio
|
|
** See help for putio backend at: https://rclone.org/putio/ **
|
|
|
|
Remote config
|
|
Use auto config?
|
|
* Say Y if not sure
|
|
* Say N if you are working on a remote or headless machine
|
|
y) Yes
|
|
n) No
|
|
y/n> y
|
|
If your browser doesn\[aq]t open automatically go to the following link: http://127.0.0.1:53682/auth
|
|
Log in and authorize rclone for access
|
|
Waiting for code...
|
|
Got code
|
|
\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-
|
|
[putio]
|
|
type = putio
|
|
token = {\[dq]access_token\[dq]:\[dq]XXXXXXXX\[dq],\[dq]expiry\[dq]:\[dq]0001\-01\-01T00:00:00Z\[dq]}
|
|
\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-
|
|
y) Yes this is OK
|
|
e) Edit this remote
|
|
d) Delete this remote
|
|
y/e/d> y
|
|
Current remotes:
|
|
|
|
Name Type
|
|
==== ====
|
|
putio putio
|
|
|
|
e) Edit existing remote
|
|
n) New remote
|
|
d) Delete remote
|
|
r) Rename remote
|
|
c) Copy remote
|
|
s) Set configuration password
|
|
q) Quit config
|
|
e/n/d/r/c/s/q> q
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
Note that rclone runs a webserver on your local machine to collect the
|
|
token as returned from Google if you use auto config mode.
|
|
This only runs from the moment it opens your browser to the moment you
|
|
get back the verification code.
|
|
This is on \f[C]http://127.0.0.1:53682/\f[R] and this it may require you
|
|
to unblock it temporarily if you are running a host firewall, or use
|
|
manual mode.
|
|
.PP
|
|
You can then use it like this,
|
|
.PP
|
|
List directories in top level of your put.io
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone lsd remote:
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
List all the files in your put.io
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone ls remote:
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
To copy a local directory to a put.io directory called backup
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone copy /home/source remote:backup
|
|
\f[R]
|
|
.fi
|
|
.SS Restricted filename characters
|
|
.PP
|
|
In addition to the default restricted characters
|
|
set (https://rclone.org/overview/#restricted-characters) the following
|
|
characters are also replaced:
|
|
.PP
|
|
.TS
|
|
tab(@);
|
|
l c c.
|
|
T{
|
|
Character
|
|
T}@T{
|
|
Value
|
|
T}@T{
|
|
Replacement
|
|
T}
|
|
_
|
|
T{
|
|
\[rs]
|
|
T}@T{
|
|
0x5C
|
|
T}@T{
|
|
\[uFF3C]
|
|
T}
|
|
.TE
|
|
.PP
|
|
Invalid UTF\-8 bytes will also be
|
|
replaced (https://rclone.org/overview/#invalid-utf8), as they can\[aq]t
|
|
be used in JSON strings.
|
|
.SS Advanced Options
|
|
.PP
|
|
Here are the advanced options specific to putio (Put.io).
|
|
.SS \-\-putio\-encoding
|
|
.PP
|
|
This sets the encoding for the backend.
|
|
.PP
|
|
See: the encoding section in the
|
|
overview (https://rclone.org/overview/#encoding) for more info.
|
|
.IP \[bu] 2
|
|
Config: encoding
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_PUTIO_ENCODING
|
|
.IP \[bu] 2
|
|
Type: MultiEncoder
|
|
.IP \[bu] 2
|
|
Default: Slash,BackSlash,Del,Ctl,InvalidUtf8,Dot
|
|
.SS Seafile
|
|
.PP
|
|
This is a backend for the Seafile (https://www.seafile.com/) storage
|
|
service: \- It works with both the free community edition or the
|
|
professional edition.
|
|
\- Seafile versions 6.x and 7.x are all supported.
|
|
\- Encrypted libraries are also supported.
|
|
\- It supports 2FA enabled users
|
|
.SS Root mode vs Library mode
|
|
.PP
|
|
There are two distinct modes you can setup your remote: \- you point
|
|
your remote to the \f[B]root of the server\f[R], meaning you don\[aq]t
|
|
specify a library during the configuration: Paths are specified as
|
|
\f[C]remote:library\f[R].
|
|
You may put subdirectories in too, e.g.
|
|
\f[C]remote:library/path/to/dir\f[R].
|
|
\- you point your remote to a specific library during the configuration:
|
|
Paths are specified as \f[C]remote:path/to/dir\f[R].
|
|
\f[B]This is the recommended mode when using encrypted libraries\f[R].
|
|
(\f[I]This mode is possibly slightly faster than the root mode\f[R])
|
|
.SS Configuration in root mode
|
|
.PP
|
|
Here is an example of making a seafile configuration for a user with
|
|
\f[B]no\f[R] two\-factor authentication.
|
|
First run
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone config
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
This will guide you through an interactive setup process.
|
|
To authenticate you will need the URL of your server, your email (or
|
|
username) and your password.
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
No remotes found \- make a new one
|
|
n) New remote
|
|
s) Set configuration password
|
|
q) Quit config
|
|
n/s/q> n
|
|
name> seafile
|
|
Type of storage to configure.
|
|
Enter a string value. Press Enter for the default (\[dq]\[dq]).
|
|
Choose a number from below, or type in your own value
|
|
[snip]
|
|
XX / Seafile
|
|
\[rs] \[dq]seafile\[dq]
|
|
[snip]
|
|
Storage> seafile
|
|
** See help for seafile backend at: https://rclone.org/seafile/ **
|
|
|
|
URL of seafile host to connect to
|
|
Enter a string value. Press Enter for the default (\[dq]\[dq]).
|
|
Choose a number from below, or type in your own value
|
|
1 / Connect to cloud.seafile.com
|
|
\[rs] \[dq]https://cloud.seafile.com/\[dq]
|
|
url> http://my.seafile.server/
|
|
User name (usually email address)
|
|
Enter a string value. Press Enter for the default (\[dq]\[dq]).
|
|
user> me\[at]example.com
|
|
Password
|
|
y) Yes type in my own password
|
|
g) Generate random password
|
|
n) No leave this optional password blank (default)
|
|
y/g> y
|
|
Enter the password:
|
|
password:
|
|
Confirm the password:
|
|
password:
|
|
Two\-factor authentication (\[aq]true\[aq] if the account has 2FA enabled)
|
|
Enter a boolean value (true or false). Press Enter for the default (\[dq]false\[dq]).
|
|
2fa> false
|
|
Name of the library. Leave blank to access all non\-encrypted libraries.
|
|
Enter a string value. Press Enter for the default (\[dq]\[dq]).
|
|
library>
|
|
Library password (for encrypted libraries only). Leave blank if you pass it through the command line.
|
|
y) Yes type in my own password
|
|
g) Generate random password
|
|
n) No leave this optional password blank (default)
|
|
y/g/n> n
|
|
Edit advanced config? (y/n)
|
|
y) Yes
|
|
n) No (default)
|
|
y/n> n
|
|
Remote config
|
|
Two\-factor authentication is not enabled on this account.
|
|
\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-
|
|
[seafile]
|
|
type = seafile
|
|
url = http://my.seafile.server/
|
|
user = me\[at]example.com
|
|
pass = *** ENCRYPTED ***
|
|
2fa = false
|
|
\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-
|
|
y) Yes this is OK (default)
|
|
e) Edit this remote
|
|
d) Delete this remote
|
|
y/e/d> y
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
This remote is called \f[C]seafile\f[R].
|
|
It\[aq]s pointing to the root of your seafile server and can now be used
|
|
like this:
|
|
.PP
|
|
See all libraries
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone lsd seafile:
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
Create a new library
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone mkdir seafile:library
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
List the contents of a library
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone ls seafile:library
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
Sync \f[C]/home/local/directory\f[R] to the remote library, deleting any
|
|
excess files in the library.
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone sync \-i /home/local/directory seafile:library
|
|
\f[R]
|
|
.fi
|
|
.SS Configuration in library mode
|
|
.PP
|
|
Here\[aq]s an example of a configuration in library mode with a user
|
|
that has the two\-factor authentication enabled.
|
|
Your 2FA code will be asked at the end of the configuration, and will
|
|
attempt to authenticate you:
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
No remotes found \- make a new one
|
|
n) New remote
|
|
s) Set configuration password
|
|
q) Quit config
|
|
n/s/q> n
|
|
name> seafile
|
|
Type of storage to configure.
|
|
Enter a string value. Press Enter for the default (\[dq]\[dq]).
|
|
Choose a number from below, or type in your own value
|
|
[snip]
|
|
XX / Seafile
|
|
\[rs] \[dq]seafile\[dq]
|
|
[snip]
|
|
Storage> seafile
|
|
** See help for seafile backend at: https://rclone.org/seafile/ **
|
|
|
|
URL of seafile host to connect to
|
|
Enter a string value. Press Enter for the default (\[dq]\[dq]).
|
|
Choose a number from below, or type in your own value
|
|
1 / Connect to cloud.seafile.com
|
|
\[rs] \[dq]https://cloud.seafile.com/\[dq]
|
|
url> http://my.seafile.server/
|
|
User name (usually email address)
|
|
Enter a string value. Press Enter for the default (\[dq]\[dq]).
|
|
user> me\[at]example.com
|
|
Password
|
|
y) Yes type in my own password
|
|
g) Generate random password
|
|
n) No leave this optional password blank (default)
|
|
y/g> y
|
|
Enter the password:
|
|
password:
|
|
Confirm the password:
|
|
password:
|
|
Two\-factor authentication (\[aq]true\[aq] if the account has 2FA enabled)
|
|
Enter a boolean value (true or false). Press Enter for the default (\[dq]false\[dq]).
|
|
2fa> true
|
|
Name of the library. Leave blank to access all non\-encrypted libraries.
|
|
Enter a string value. Press Enter for the default (\[dq]\[dq]).
|
|
library> My Library
|
|
Library password (for encrypted libraries only). Leave blank if you pass it through the command line.
|
|
y) Yes type in my own password
|
|
g) Generate random password
|
|
n) No leave this optional password blank (default)
|
|
y/g/n> n
|
|
Edit advanced config? (y/n)
|
|
y) Yes
|
|
n) No (default)
|
|
y/n> n
|
|
Remote config
|
|
Two\-factor authentication: please enter your 2FA code
|
|
2fa code> 123456
|
|
Authenticating...
|
|
Success!
|
|
\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-
|
|
[seafile]
|
|
type = seafile
|
|
url = http://my.seafile.server/
|
|
user = me\[at]example.com
|
|
pass =
|
|
2fa = true
|
|
library = My Library
|
|
\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-
|
|
y) Yes this is OK (default)
|
|
e) Edit this remote
|
|
d) Delete this remote
|
|
y/e/d> y
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
You\[aq]ll notice your password is blank in the configuration.
|
|
It\[aq]s because we only need the password to authenticate you once.
|
|
.PP
|
|
You specified \f[C]My Library\f[R] during the configuration.
|
|
The root of the remote is pointing at the root of the library
|
|
\f[C]My Library\f[R]:
|
|
.PP
|
|
See all files in the library:
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone lsd seafile:
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
Create a new directory inside the library
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone mkdir seafile:directory
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
List the contents of a directory
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone ls seafile:directory
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
Sync \f[C]/home/local/directory\f[R] to the remote library, deleting any
|
|
excess files in the library.
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone sync \-i /home/local/directory seafile:
|
|
\f[R]
|
|
.fi
|
|
.SS \-\-fast\-list
|
|
.PP
|
|
Seafile version 7+ supports \f[C]\-\-fast\-list\f[R] which allows you to
|
|
use fewer transactions in exchange for more memory.
|
|
See the rclone docs (https://rclone.org/docs/#fast-list) for more
|
|
details.
|
|
Please note this is not supported on seafile server version 6.x
|
|
.SS Restricted filename characters
|
|
.PP
|
|
In addition to the default restricted characters
|
|
set (https://rclone.org/overview/#restricted-characters) the following
|
|
characters are also replaced:
|
|
.PP
|
|
.TS
|
|
tab(@);
|
|
l c c.
|
|
T{
|
|
Character
|
|
T}@T{
|
|
Value
|
|
T}@T{
|
|
Replacement
|
|
T}
|
|
_
|
|
T{
|
|
/
|
|
T}@T{
|
|
0x2F
|
|
T}@T{
|
|
\[uFF0F]
|
|
T}
|
|
T{
|
|
\[dq]
|
|
T}@T{
|
|
0x22
|
|
T}@T{
|
|
\[uFF02]
|
|
T}
|
|
T{
|
|
\[rs]
|
|
T}@T{
|
|
0x5C
|
|
T}@T{
|
|
\[uFF3C]
|
|
T}
|
|
.TE
|
|
.PP
|
|
Invalid UTF\-8 bytes will also be
|
|
replaced (https://rclone.org/overview/#invalid-utf8), as they can\[aq]t
|
|
be used in JSON strings.
|
|
.SS Seafile and rclone link
|
|
.PP
|
|
Rclone supports generating share links for non\-encrypted libraries
|
|
only.
|
|
They can either be for a file or a directory:
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone link seafile:seafile\-tutorial.doc
|
|
http://my.seafile.server/f/fdcd8a2f93f84b8b90f4/
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
or if run on a directory you will get:
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone link seafile:dir
|
|
http://my.seafile.server/d/9ea2455f6f55478bbb0d/
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
Please note a share link is unique for each file or directory.
|
|
If you run a link command on a file/dir that has already been shared,
|
|
you will get the exact same link.
|
|
.SS Compatibility
|
|
.PP
|
|
It has been actively tested using the seafile docker
|
|
image (https://github.com/haiwen/seafile-docker) of these versions: \-
|
|
6.3.4 community edition \- 7.0.5 community edition \- 7.1.3 community
|
|
edition
|
|
.PP
|
|
Versions below 6.0 are not supported.
|
|
Versions between 6.0 and 6.3 haven\[aq]t been tested and might not work
|
|
properly.
|
|
.SS Standard Options
|
|
.PP
|
|
Here are the standard options specific to seafile (seafile).
|
|
.SS \-\-seafile\-url
|
|
.PP
|
|
URL of seafile host to connect to
|
|
.IP \[bu] 2
|
|
Config: url
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_SEAFILE_URL
|
|
.IP \[bu] 2
|
|
Type: string
|
|
.IP \[bu] 2
|
|
Default: \[dq]\[dq]
|
|
.IP \[bu] 2
|
|
Examples:
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
\[dq]https://cloud.seafile.com/\[dq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Connect to cloud.seafile.com
|
|
.RE
|
|
.RE
|
|
.SS \-\-seafile\-user
|
|
.PP
|
|
User name (usually email address)
|
|
.IP \[bu] 2
|
|
Config: user
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_SEAFILE_USER
|
|
.IP \[bu] 2
|
|
Type: string
|
|
.IP \[bu] 2
|
|
Default: \[dq]\[dq]
|
|
.SS \-\-seafile\-pass
|
|
.PP
|
|
Password
|
|
.PP
|
|
\f[B]NB\f[R] Input to this must be obscured \- see rclone
|
|
obscure (https://rclone.org/commands/rclone_obscure/).
|
|
.IP \[bu] 2
|
|
Config: pass
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_SEAFILE_PASS
|
|
.IP \[bu] 2
|
|
Type: string
|
|
.IP \[bu] 2
|
|
Default: \[dq]\[dq]
|
|
.SS \-\-seafile\-2fa
|
|
.PP
|
|
Two\-factor authentication (\[aq]true\[aq] if the account has 2FA
|
|
enabled)
|
|
.IP \[bu] 2
|
|
Config: 2fa
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_SEAFILE_2FA
|
|
.IP \[bu] 2
|
|
Type: bool
|
|
.IP \[bu] 2
|
|
Default: false
|
|
.SS \-\-seafile\-library
|
|
.PP
|
|
Name of the library.
|
|
Leave blank to access all non\-encrypted libraries.
|
|
.IP \[bu] 2
|
|
Config: library
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_SEAFILE_LIBRARY
|
|
.IP \[bu] 2
|
|
Type: string
|
|
.IP \[bu] 2
|
|
Default: \[dq]\[dq]
|
|
.SS \-\-seafile\-library\-key
|
|
.PP
|
|
Library password (for encrypted libraries only).
|
|
Leave blank if you pass it through the command line.
|
|
.PP
|
|
\f[B]NB\f[R] Input to this must be obscured \- see rclone
|
|
obscure (https://rclone.org/commands/rclone_obscure/).
|
|
.IP \[bu] 2
|
|
Config: library_key
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_SEAFILE_LIBRARY_KEY
|
|
.IP \[bu] 2
|
|
Type: string
|
|
.IP \[bu] 2
|
|
Default: \[dq]\[dq]
|
|
.SS \-\-seafile\-auth\-token
|
|
.PP
|
|
Authentication token
|
|
.IP \[bu] 2
|
|
Config: auth_token
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_SEAFILE_AUTH_TOKEN
|
|
.IP \[bu] 2
|
|
Type: string
|
|
.IP \[bu] 2
|
|
Default: \[dq]\[dq]
|
|
.SS Advanced Options
|
|
.PP
|
|
Here are the advanced options specific to seafile (seafile).
|
|
.SS \-\-seafile\-create\-library
|
|
.PP
|
|
Should rclone create a library if it doesn\[aq]t exist
|
|
.IP \[bu] 2
|
|
Config: create_library
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_SEAFILE_CREATE_LIBRARY
|
|
.IP \[bu] 2
|
|
Type: bool
|
|
.IP \[bu] 2
|
|
Default: false
|
|
.SS \-\-seafile\-encoding
|
|
.PP
|
|
This sets the encoding for the backend.
|
|
.PP
|
|
See: the encoding section in the
|
|
overview (https://rclone.org/overview/#encoding) for more info.
|
|
.IP \[bu] 2
|
|
Config: encoding
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_SEAFILE_ENCODING
|
|
.IP \[bu] 2
|
|
Type: MultiEncoder
|
|
.IP \[bu] 2
|
|
Default: Slash,DoubleQuote,BackSlash,Ctl,InvalidUtf8
|
|
.SS SFTP
|
|
.PP
|
|
SFTP is the Secure (or SSH) File Transfer
|
|
Protocol (https://en.wikipedia.org/wiki/SSH_File_Transfer_Protocol).
|
|
.PP
|
|
The SFTP backend can be used with a number of different providers:
|
|
.IP \[bu] 2
|
|
C14
|
|
.IP \[bu] 2
|
|
rsync.net
|
|
.PP
|
|
SFTP runs over SSH v2 and is installed as standard with most modern SSH
|
|
installations.
|
|
.PP
|
|
Paths are specified as \f[C]remote:path\f[R].
|
|
If the path does not begin with a \f[C]/\f[R] it is relative to the home
|
|
directory of the user.
|
|
An empty path \f[C]remote:\f[R] refers to the user\[aq]s home directory.
|
|
.PP
|
|
\[dq]Note that some SFTP servers will need the leading / \- Synology is
|
|
a good example of this.
|
|
rsync.net, on the other hand, requires users to OMIT the leading /.
|
|
.PP
|
|
Here is an example of making an SFTP configuration.
|
|
First run
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone config
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
This will guide you through an interactive setup process.
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
No remotes found \- make a new one
|
|
n) New remote
|
|
s) Set configuration password
|
|
q) Quit config
|
|
n/s/q> n
|
|
name> remote
|
|
Type of storage to configure.
|
|
Choose a number from below, or type in your own value
|
|
[snip]
|
|
XX / SSH/SFTP Connection
|
|
\[rs] \[dq]sftp\[dq]
|
|
[snip]
|
|
Storage> sftp
|
|
SSH host to connect to
|
|
Choose a number from below, or type in your own value
|
|
1 / Connect to example.com
|
|
\[rs] \[dq]example.com\[dq]
|
|
host> example.com
|
|
SSH username, leave blank for current username, $USER
|
|
user> sftpuser
|
|
SSH port, leave blank to use default (22)
|
|
port>
|
|
SSH password, leave blank to use ssh\-agent.
|
|
y) Yes type in my own password
|
|
g) Generate random password
|
|
n) No leave this optional password blank
|
|
y/g/n> n
|
|
Path to unencrypted PEM\-encoded private key file, leave blank to use ssh\-agent.
|
|
key_file>
|
|
Remote config
|
|
\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-
|
|
[remote]
|
|
host = example.com
|
|
user = sftpuser
|
|
port =
|
|
pass =
|
|
key_file =
|
|
\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-
|
|
y) Yes this is OK
|
|
e) Edit this remote
|
|
d) Delete this remote
|
|
y/e/d> y
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
This remote is called \f[C]remote\f[R] and can now be used like this:
|
|
.PP
|
|
See all directories in the home directory
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone lsd remote:
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
Make a new directory
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone mkdir remote:path/to/directory
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
List the contents of a directory
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone ls remote:path/to/directory
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
Sync \f[C]/home/local/directory\f[R] to the remote directory, deleting
|
|
any excess files in the directory.
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone sync \-i /home/local/directory remote:directory
|
|
\f[R]
|
|
.fi
|
|
.SS SSH Authentication
|
|
.PP
|
|
The SFTP remote supports three authentication methods:
|
|
.IP \[bu] 2
|
|
Password
|
|
.IP \[bu] 2
|
|
Key file, including certificate signed keys
|
|
.IP \[bu] 2
|
|
ssh\-agent
|
|
.PP
|
|
Key files should be PEM\-encoded private key files.
|
|
For instance \f[C]/home/$USER/.ssh/id_rsa\f[R].
|
|
Only unencrypted OpenSSH or PEM encrypted files are supported.
|
|
.PP
|
|
The key file can be specified in either an external file (key_file) or
|
|
contained within the rclone config file (key_pem).
|
|
If using key_pem in the config file, the entry should be on a single
|
|
line with new line (\[aq]\[aq] or \[aq]\[aq]) separating lines.
|
|
i.e.
|
|
.PP
|
|
key_pem = \-\-\-\-\-BEGIN RSA PRIVATE
|
|
KEY\-\-\-\-\-0gAMbMbaSsd\-\-\-\-\-END RSA PRIVATE KEY\-\-\-\-\-
|
|
.PP
|
|
This will generate it correctly for key_pem for use in the config:
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
awk \[aq]{printf \[dq]%s\[rs]\[rs]n\[dq], $0}\[aq] < \[ti]/.ssh/id_rsa
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
If you don\[aq]t specify \f[C]pass\f[R], \f[C]key_file\f[R], or
|
|
\f[C]key_pem\f[R] then rclone will attempt to contact an ssh\-agent.
|
|
.PP
|
|
You can also specify \f[C]key_use_agent\f[R] to force the usage of an
|
|
ssh\-agent.
|
|
In this case \f[C]key_file\f[R] or \f[C]key_pem\f[R] can also be
|
|
specified to force the usage of a specific key in the ssh\-agent.
|
|
.PP
|
|
Using an ssh\-agent is the only way to load encrypted OpenSSH keys at
|
|
the moment.
|
|
.PP
|
|
If you set the \f[C]\-\-sftp\-ask\-password\f[R] option, rclone will
|
|
prompt for a password when needed and no password has been configured.
|
|
.PP
|
|
If you have a certificate then you can provide the path to the public
|
|
key that contains the certificate.
|
|
For example:
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
[remote]
|
|
type = sftp
|
|
host = example.com
|
|
user = sftpuser
|
|
key_file = \[ti]/id_rsa
|
|
pubkey_file = \[ti]/id_rsa\-cert.pub
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
If you concatenate a cert with a private key then you can specify the
|
|
merged file in both places.
|
|
.PP
|
|
Note: the cert must come first in the file.
|
|
e.g.
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
cat id_rsa\-cert.pub id_rsa > merged_key
|
|
\f[R]
|
|
.fi
|
|
.SS Host key validation
|
|
.PP
|
|
By default rclone will not check the server\[aq]s host key for
|
|
validation.
|
|
This can allow an attacker to replace a server with their own and if you
|
|
use password authentication then this can lead to that password being
|
|
exposed.
|
|
.PP
|
|
Host key matching, using standard \f[C]known_hosts\f[R] files can be
|
|
turned on by enabling the \f[C]known_hosts_file\f[R] option.
|
|
This can point to the file maintained by \f[C]OpenSSH\f[R] or can point
|
|
to a unique file.
|
|
.PP
|
|
e.g.
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
[remote]
|
|
type = sftp
|
|
host = example.com
|
|
user = sftpuser
|
|
pass =
|
|
known_hosts_file = \[ti]/.ssh/known_hosts
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
There are some limitations:
|
|
.IP \[bu] 2
|
|
\f[C]rclone\f[R] will not \f[I]manage\f[R] this file for you.
|
|
If the key is missing or wrong then the connection will be refused.
|
|
.IP \[bu] 2
|
|
If the server is set up for a certificate host key then the entry in the
|
|
\f[C]known_hosts\f[R] file \f[I]must\f[R] be the
|
|
\f[C]\[at]cert\-authority\f[R] entry for the CA
|
|
.IP \[bu] 2
|
|
Unlike \f[C]OpenSSH\f[R], the libraries used by \f[C]rclone\f[R] do not
|
|
permit (at time of writing) multiple host keys to be listed for a
|
|
server.
|
|
Only the first entry is used.
|
|
.PP
|
|
If the host key provided by the server does not match the one in the
|
|
file (or is missing) then the connection will be aborted and an error
|
|
returned such as
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
NewFs: couldn\[aq]t connect SSH: ssh: handshake failed: knownhosts: key mismatch
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
or
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
NewFs: couldn\[aq]t connect SSH: ssh: handshake failed: knownhosts: key is unknown
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
If you see an error such as
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
NewFs: couldn\[aq]t connect SSH: ssh: handshake failed: ssh: no authorities for hostname: example.com:22
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
then it is likely the server has presented a CA signed host certificate
|
|
and you will need to add the appropriate \f[C]\[at]cert\-authority\f[R]
|
|
entry.
|
|
.PP
|
|
The \f[C]known_hosts_file\f[R] setting can be set during
|
|
\f[C]rclone config\f[R] as an advanced option.
|
|
.SS ssh\-agent on macOS
|
|
.PP
|
|
Note that there seem to be various problems with using an ssh\-agent on
|
|
macOS due to recent changes in the OS.
|
|
The most effective work\-around seems to be to start an ssh\-agent in
|
|
each session, e.g.
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
eval \[ga]ssh\-agent \-s\[ga] && ssh\-add \-A
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
And then at the end of the session
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
eval \[ga]ssh\-agent \-k\[ga]
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
These commands can be used in scripts of course.
|
|
.SS Modified time
|
|
.PP
|
|
Modified times are stored on the server to 1 second precision.
|
|
.PP
|
|
Modified times are used in syncing and are fully supported.
|
|
.PP
|
|
Some SFTP servers disable setting/modifying the file modification time
|
|
after upload (for example, certain configurations of ProFTPd with
|
|
mod_sftp).
|
|
If you are using one of these servers, you can set the option
|
|
\f[C]set_modtime = false\f[R] in your RClone backend configuration to
|
|
disable this behaviour.
|
|
.SS Standard Options
|
|
.PP
|
|
Here are the standard options specific to sftp (SSH/SFTP Connection).
|
|
.SS \-\-sftp\-host
|
|
.PP
|
|
SSH host to connect to
|
|
.IP \[bu] 2
|
|
Config: host
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_SFTP_HOST
|
|
.IP \[bu] 2
|
|
Type: string
|
|
.IP \[bu] 2
|
|
Default: \[dq]\[dq]
|
|
.IP \[bu] 2
|
|
Examples:
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
\[dq]example.com\[dq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Connect to example.com
|
|
.RE
|
|
.RE
|
|
.SS \-\-sftp\-user
|
|
.PP
|
|
SSH username, leave blank for current username, $USER
|
|
.IP \[bu] 2
|
|
Config: user
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_SFTP_USER
|
|
.IP \[bu] 2
|
|
Type: string
|
|
.IP \[bu] 2
|
|
Default: \[dq]\[dq]
|
|
.SS \-\-sftp\-port
|
|
.PP
|
|
SSH port, leave blank to use default (22)
|
|
.IP \[bu] 2
|
|
Config: port
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_SFTP_PORT
|
|
.IP \[bu] 2
|
|
Type: string
|
|
.IP \[bu] 2
|
|
Default: \[dq]\[dq]
|
|
.SS \-\-sftp\-pass
|
|
.PP
|
|
SSH password, leave blank to use ssh\-agent.
|
|
.PP
|
|
\f[B]NB\f[R] Input to this must be obscured \- see rclone
|
|
obscure (https://rclone.org/commands/rclone_obscure/).
|
|
.IP \[bu] 2
|
|
Config: pass
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_SFTP_PASS
|
|
.IP \[bu] 2
|
|
Type: string
|
|
.IP \[bu] 2
|
|
Default: \[dq]\[dq]
|
|
.SS \-\-sftp\-key\-pem
|
|
.PP
|
|
Raw PEM\-encoded private key, If specified, will override key_file
|
|
parameter.
|
|
.IP \[bu] 2
|
|
Config: key_pem
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_SFTP_KEY_PEM
|
|
.IP \[bu] 2
|
|
Type: string
|
|
.IP \[bu] 2
|
|
Default: \[dq]\[dq]
|
|
.SS \-\-sftp\-key\-file
|
|
.PP
|
|
Path to PEM\-encoded private key file, leave blank or set
|
|
key\-use\-agent to use ssh\-agent.
|
|
.PP
|
|
Leading \f[C]\[ti]\f[R] will be expanded in the file name as will
|
|
environment variables such as \f[C]${RCLONE_CONFIG_DIR}\f[R].
|
|
.IP \[bu] 2
|
|
Config: key_file
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_SFTP_KEY_FILE
|
|
.IP \[bu] 2
|
|
Type: string
|
|
.IP \[bu] 2
|
|
Default: \[dq]\[dq]
|
|
.SS \-\-sftp\-key\-file\-pass
|
|
.PP
|
|
The passphrase to decrypt the PEM\-encoded private key file.
|
|
.PP
|
|
Only PEM encrypted key files (old OpenSSH format) are supported.
|
|
Encrypted keys in the new OpenSSH format can\[aq]t be used.
|
|
.PP
|
|
\f[B]NB\f[R] Input to this must be obscured \- see rclone
|
|
obscure (https://rclone.org/commands/rclone_obscure/).
|
|
.IP \[bu] 2
|
|
Config: key_file_pass
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_SFTP_KEY_FILE_PASS
|
|
.IP \[bu] 2
|
|
Type: string
|
|
.IP \[bu] 2
|
|
Default: \[dq]\[dq]
|
|
.SS \-\-sftp\-pubkey\-file
|
|
.PP
|
|
Optional path to public key file.
|
|
.PP
|
|
Set this if you have a signed certificate you want to use for
|
|
authentication.
|
|
.PP
|
|
Leading \f[C]\[ti]\f[R] will be expanded in the file name as will
|
|
environment variables such as \f[C]${RCLONE_CONFIG_DIR}\f[R].
|
|
.IP \[bu] 2
|
|
Config: pubkey_file
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_SFTP_PUBKEY_FILE
|
|
.IP \[bu] 2
|
|
Type: string
|
|
.IP \[bu] 2
|
|
Default: \[dq]\[dq]
|
|
.SS \-\-sftp\-key\-use\-agent
|
|
.PP
|
|
When set forces the usage of the ssh\-agent.
|
|
.PP
|
|
When key\-file is also set, the \[dq].pub\[dq] file of the specified
|
|
key\-file is read and only the associated key is requested from the
|
|
ssh\-agent.
|
|
This allows to avoid
|
|
\f[C]Too many authentication failures for *username*\f[R] errors when
|
|
the ssh\-agent contains many keys.
|
|
.IP \[bu] 2
|
|
Config: key_use_agent
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_SFTP_KEY_USE_AGENT
|
|
.IP \[bu] 2
|
|
Type: bool
|
|
.IP \[bu] 2
|
|
Default: false
|
|
.SS \-\-sftp\-use\-insecure\-cipher
|
|
.PP
|
|
Enable the use of insecure ciphers and key exchange methods.
|
|
.PP
|
|
This enables the use of the following insecure ciphers and key exchange
|
|
methods:
|
|
.IP \[bu] 2
|
|
aes128\-cbc
|
|
.IP \[bu] 2
|
|
aes192\-cbc
|
|
.IP \[bu] 2
|
|
aes256\-cbc
|
|
.IP \[bu] 2
|
|
3des\-cbc
|
|
.IP \[bu] 2
|
|
diffie\-hellman\-group\-exchange\-sha256
|
|
.IP \[bu] 2
|
|
diffie\-hellman\-group\-exchange\-sha1
|
|
.PP
|
|
Those algorithms are insecure and may allow plaintext data to be
|
|
recovered by an attacker.
|
|
.IP \[bu] 2
|
|
Config: use_insecure_cipher
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_SFTP_USE_INSECURE_CIPHER
|
|
.IP \[bu] 2
|
|
Type: bool
|
|
.IP \[bu] 2
|
|
Default: false
|
|
.IP \[bu] 2
|
|
Examples:
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
\[dq]false\[dq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Use default Cipher list.
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[dq]true\[dq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Enables the use of the aes128\-cbc cipher and
|
|
diffie\-hellman\-group\-exchange\-sha256,
|
|
diffie\-hellman\-group\-exchange\-sha1 key exchange.
|
|
.RE
|
|
.RE
|
|
.SS \-\-sftp\-disable\-hashcheck
|
|
.PP
|
|
Disable the execution of SSH commands to determine if remote file
|
|
hashing is available.
|
|
Leave blank or set to false to enable hashing (recommended), set to true
|
|
to disable hashing.
|
|
.IP \[bu] 2
|
|
Config: disable_hashcheck
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_SFTP_DISABLE_HASHCHECK
|
|
.IP \[bu] 2
|
|
Type: bool
|
|
.IP \[bu] 2
|
|
Default: false
|
|
.SS Advanced Options
|
|
.PP
|
|
Here are the advanced options specific to sftp (SSH/SFTP Connection).
|
|
.SS \-\-sftp\-known\-hosts\-file
|
|
.PP
|
|
Optional path to known_hosts file.
|
|
.PP
|
|
Set this value to enable server host key validation.
|
|
.PP
|
|
Leading \f[C]\[ti]\f[R] will be expanded in the file name as will
|
|
environment variables such as \f[C]${RCLONE_CONFIG_DIR}\f[R].
|
|
.IP \[bu] 2
|
|
Config: known_hosts_file
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_SFTP_KNOWN_HOSTS_FILE
|
|
.IP \[bu] 2
|
|
Type: string
|
|
.IP \[bu] 2
|
|
Default: \[dq]\[dq]
|
|
.IP \[bu] 2
|
|
Examples:
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
\[dq]\[ti]/.ssh/known_hosts\[dq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Use OpenSSH\[aq]s known_hosts file
|
|
.RE
|
|
.RE
|
|
.SS \-\-sftp\-ask\-password
|
|
.PP
|
|
Allow asking for SFTP password when needed.
|
|
.PP
|
|
If this is set and no password is supplied then rclone will: \- ask for
|
|
a password \- not contact the ssh agent
|
|
.IP \[bu] 2
|
|
Config: ask_password
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_SFTP_ASK_PASSWORD
|
|
.IP \[bu] 2
|
|
Type: bool
|
|
.IP \[bu] 2
|
|
Default: false
|
|
.SS \-\-sftp\-path\-override
|
|
.PP
|
|
Override path used by SSH connection.
|
|
.PP
|
|
This allows checksum calculation when SFTP and SSH paths are different.
|
|
This issue affects among others Synology NAS boxes.
|
|
.PP
|
|
Shared folders can be found in directories representing volumes
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone sync /home/local/directory remote:/directory \-\-ssh\-path\-override /volume2/directory
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
Home directory can be found in a shared folder called \[dq]home\[dq]
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone sync /home/local/directory remote:/home/directory \-\-ssh\-path\-override /volume1/homes/USER/directory
|
|
\f[R]
|
|
.fi
|
|
.IP \[bu] 2
|
|
Config: path_override
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_SFTP_PATH_OVERRIDE
|
|
.IP \[bu] 2
|
|
Type: string
|
|
.IP \[bu] 2
|
|
Default: \[dq]\[dq]
|
|
.SS \-\-sftp\-set\-modtime
|
|
.PP
|
|
Set the modified time on the remote if set.
|
|
.IP \[bu] 2
|
|
Config: set_modtime
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_SFTP_SET_MODTIME
|
|
.IP \[bu] 2
|
|
Type: bool
|
|
.IP \[bu] 2
|
|
Default: true
|
|
.SS \-\-sftp\-md5sum\-command
|
|
.PP
|
|
The command used to read md5 hashes.
|
|
Leave blank for autodetect.
|
|
.IP \[bu] 2
|
|
Config: md5sum_command
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_SFTP_MD5SUM_COMMAND
|
|
.IP \[bu] 2
|
|
Type: string
|
|
.IP \[bu] 2
|
|
Default: \[dq]\[dq]
|
|
.SS \-\-sftp\-sha1sum\-command
|
|
.PP
|
|
The command used to read sha1 hashes.
|
|
Leave blank for autodetect.
|
|
.IP \[bu] 2
|
|
Config: sha1sum_command
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_SFTP_SHA1SUM_COMMAND
|
|
.IP \[bu] 2
|
|
Type: string
|
|
.IP \[bu] 2
|
|
Default: \[dq]\[dq]
|
|
.SS \-\-sftp\-skip\-links
|
|
.PP
|
|
Set to skip any symlinks and any other non regular files.
|
|
.IP \[bu] 2
|
|
Config: skip_links
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_SFTP_SKIP_LINKS
|
|
.IP \[bu] 2
|
|
Type: bool
|
|
.IP \[bu] 2
|
|
Default: false
|
|
.SS \-\-sftp\-subsystem
|
|
.PP
|
|
Specifies the SSH2 subsystem on the remote host.
|
|
.IP \[bu] 2
|
|
Config: subsystem
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_SFTP_SUBSYSTEM
|
|
.IP \[bu] 2
|
|
Type: string
|
|
.IP \[bu] 2
|
|
Default: \[dq]sftp\[dq]
|
|
.SS \-\-sftp\-server\-command
|
|
.PP
|
|
Specifies the path or command to run a sftp server on the remote host.
|
|
.PP
|
|
The subsystem option is ignored when server_command is defined.
|
|
.IP \[bu] 2
|
|
Config: server_command
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_SFTP_SERVER_COMMAND
|
|
.IP \[bu] 2
|
|
Type: string
|
|
.IP \[bu] 2
|
|
Default: \[dq]\[dq]
|
|
.SS \-\-sftp\-use\-fstat
|
|
.PP
|
|
If set use fstat instead of stat
|
|
.PP
|
|
Some servers limit the amount of open files and calling Stat after
|
|
opening the file will throw an error from the server.
|
|
Setting this flag will call Fstat instead of Stat which is called on an
|
|
already open file handle.
|
|
.PP
|
|
It has been found that this helps with IBM Sterling SFTP servers which
|
|
have \[dq]extractability\[dq] level set to 1 which means only 1 file can
|
|
be opened at any given time.
|
|
.IP \[bu] 2
|
|
Config: use_fstat
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_SFTP_USE_FSTAT
|
|
.IP \[bu] 2
|
|
Type: bool
|
|
.IP \[bu] 2
|
|
Default: false
|
|
.SS \-\-sftp\-disable\-concurrent\-reads
|
|
.PP
|
|
If set don\[aq]t use concurrent reads
|
|
.PP
|
|
Normally concurrent reads are safe to use and not using them will
|
|
degrade performance, so this option is disabled by default.
|
|
.PP
|
|
Some servers limit the amount number of times a file can be downloaded.
|
|
Using concurrent reads can trigger this limit, so if you have a server
|
|
which returns
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
Failed to copy: file does not exist
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
Then you may need to enable this flag.
|
|
.PP
|
|
If concurrent reads are disabled, the use_fstat option is ignored.
|
|
.IP \[bu] 2
|
|
Config: disable_concurrent_reads
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_SFTP_DISABLE_CONCURRENT_READS
|
|
.IP \[bu] 2
|
|
Type: bool
|
|
.IP \[bu] 2
|
|
Default: false
|
|
.SS \-\-sftp\-idle\-timeout
|
|
.PP
|
|
Max time before closing idle connections
|
|
.PP
|
|
If no connections have been returned to the connection pool in the time
|
|
given, rclone will empty the connection pool.
|
|
.PP
|
|
Set to 0 to keep connections indefinitely.
|
|
.IP \[bu] 2
|
|
Config: idle_timeout
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_SFTP_IDLE_TIMEOUT
|
|
.IP \[bu] 2
|
|
Type: Duration
|
|
.IP \[bu] 2
|
|
Default: 1m0s
|
|
.SS Limitations
|
|
.PP
|
|
SFTP supports checksums if the same login has shell access and
|
|
\f[C]md5sum\f[R] or \f[C]sha1sum\f[R] as well as \f[C]echo\f[R] are in
|
|
the remote\[aq]s PATH.
|
|
This remote checksumming (file hashing) is recommended and enabled by
|
|
default.
|
|
Disabling the checksumming may be required if you are connecting to SFTP
|
|
servers which are not under your control, and to which the execution of
|
|
remote commands is prohibited.
|
|
Set the configuration option \f[C]disable_hashcheck\f[R] to
|
|
\f[C]true\f[R] to disable checksumming.
|
|
.PP
|
|
SFTP also supports \f[C]about\f[R] if the same login has shell access
|
|
and \f[C]df\f[R] are in the remote\[aq]s PATH.
|
|
\f[C]about\f[R] will return the total space, free space, and used space
|
|
on the remote for the disk of the specified path on the remote or, if
|
|
not set, the disk of the root on the remote.
|
|
\f[C]about\f[R] will fail if it does not have shell access or if
|
|
\f[C]df\f[R] is not in the remote\[aq]s PATH.
|
|
.PP
|
|
Note that some SFTP servers (e.g.
|
|
Synology) the paths are different for SSH and SFTP so the hashes
|
|
can\[aq]t be calculated properly.
|
|
For them using \f[C]disable_hashcheck\f[R] is a good idea.
|
|
.PP
|
|
The only ssh agent supported under Windows is Putty\[aq]s pageant.
|
|
.PP
|
|
The Go SSH library disables the use of the aes128\-cbc cipher by
|
|
default, due to security concerns.
|
|
This can be re\-enabled on a per\-connection basis by setting the
|
|
\f[C]use_insecure_cipher\f[R] setting in the configuration file to
|
|
\f[C]true\f[R].
|
|
Further details on the insecurity of this cipher can be found in this
|
|
paper (http://www.isg.rhul.ac.uk/~kp/SandPfinal.pdf).
|
|
.PP
|
|
SFTP isn\[aq]t supported under plan9 until this
|
|
issue (https://github.com/pkg/sftp/issues/156) is fixed.
|
|
.PP
|
|
Note that since SFTP isn\[aq]t HTTP based the following flags don\[aq]t
|
|
work with it: \f[C]\-\-dump\-headers\f[R], \f[C]\-\-dump\-bodies\f[R],
|
|
\f[C]\-\-dump\-auth\f[R]
|
|
.PP
|
|
Note that \f[C]\-\-timeout\f[R] isn\[aq]t supported (but
|
|
\f[C]\-\-contimeout\f[R] is).
|
|
.SS C14
|
|
.PP
|
|
C14 is supported through the SFTP backend.
|
|
.PP
|
|
See C14\[aq]s
|
|
documentation (https://www.online.net/en/storage/c14-cold-storage)
|
|
.SS rsync.net
|
|
.PP
|
|
rsync.net is supported through the SFTP backend.
|
|
.PP
|
|
See rsync.net\[aq]s documentation of rclone
|
|
examples (https://www.rsync.net/products/rclone.html).
|
|
.SS SugarSync
|
|
.PP
|
|
SugarSync (https://sugarsync.com) is a cloud service that enables active
|
|
synchronization of files across computers and other devices for file
|
|
backup, access, syncing, and sharing.
|
|
.PP
|
|
The initial setup for SugarSync involves getting a token from SugarSync
|
|
which you can do with rclone.
|
|
\f[C]rclone config\f[R] walks you through it.
|
|
.PP
|
|
Here is an example of how to make a remote called \f[C]remote\f[R].
|
|
First run:
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone config
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
This will guide you through an interactive setup process:
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
No remotes found \- make a new one
|
|
n) New remote
|
|
s) Set configuration password
|
|
q) Quit config
|
|
n/s/q> n
|
|
name> remote
|
|
Type of storage to configure.
|
|
Enter a string value. Press Enter for the default (\[dq]\[dq]).
|
|
Choose a number from below, or type in your own value
|
|
[snip]
|
|
XX / Sugarsync
|
|
\[rs] \[dq]sugarsync\[dq]
|
|
[snip]
|
|
Storage> sugarsync
|
|
** See help for sugarsync backend at: https://rclone.org/sugarsync/ **
|
|
|
|
Sugarsync App ID.
|
|
Leave blank to use rclone\[aq]s.
|
|
Enter a string value. Press Enter for the default (\[dq]\[dq]).
|
|
app_id>
|
|
Sugarsync Access Key ID.
|
|
Leave blank to use rclone\[aq]s.
|
|
Enter a string value. Press Enter for the default (\[dq]\[dq]).
|
|
access_key_id>
|
|
Sugarsync Private Access Key
|
|
Leave blank to use rclone\[aq]s.
|
|
Enter a string value. Press Enter for the default (\[dq]\[dq]).
|
|
private_access_key>
|
|
Permanently delete files if true
|
|
otherwise put them in the deleted files.
|
|
Enter a boolean value (true or false). Press Enter for the default (\[dq]false\[dq]).
|
|
hard_delete>
|
|
Edit advanced config? (y/n)
|
|
y) Yes
|
|
n) No (default)
|
|
y/n> n
|
|
Remote config
|
|
Username (email address)> nick\[at]craig\-wood.com
|
|
Your Sugarsync password is only required during setup and will not be stored.
|
|
password:
|
|
\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-
|
|
[remote]
|
|
type = sugarsync
|
|
refresh_token = https://api.sugarsync.com/app\-authorization/XXXXXXXXXXXXXXXXXX
|
|
\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-
|
|
y) Yes this is OK (default)
|
|
e) Edit this remote
|
|
d) Delete this remote
|
|
y/e/d> y
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
Note that the config asks for your email and password but doesn\[aq]t
|
|
store them, it only uses them to get the initial token.
|
|
.PP
|
|
Once configured you can then use \f[C]rclone\f[R] like this,
|
|
.PP
|
|
List directories (sync folders) in top level of your SugarSync
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone lsd remote:
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
List all the files in your SugarSync folder \[dq]Test\[dq]
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone ls remote:Test
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
To copy a local directory to an SugarSync folder called backup
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone copy /home/source remote:backup
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
Paths are specified as \f[C]remote:path\f[R]
|
|
.PP
|
|
Paths may be as deep as required, e.g.
|
|
\f[C]remote:directory/subdirectory\f[R].
|
|
.PP
|
|
\f[B]NB\f[R] you can\[aq]t create files in the top level folder you have
|
|
to create a folder, which rclone will create as a \[dq]Sync Folder\[dq]
|
|
with SugarSync.
|
|
.SS Modified time and hashes
|
|
.PP
|
|
SugarSync does not support modification times or hashes, therefore
|
|
syncing will default to \f[C]\-\-size\-only\f[R] checking.
|
|
Note that using \f[C]\-\-update\f[R] will work as rclone can read the
|
|
time files were uploaded.
|
|
.SS Restricted filename characters
|
|
.PP
|
|
SugarSync replaces the default restricted characters
|
|
set (https://rclone.org/overview/#restricted-characters) except for DEL.
|
|
.PP
|
|
Invalid UTF\-8 bytes will also be
|
|
replaced (https://rclone.org/overview/#invalid-utf8), as they can\[aq]t
|
|
be used in XML strings.
|
|
.SS Deleting files
|
|
.PP
|
|
Deleted files will be moved to the \[dq]Deleted items\[dq] folder by
|
|
default.
|
|
.PP
|
|
However you can supply the flag \f[C]\-\-sugarsync\-hard\-delete\f[R] or
|
|
set the config parameter \f[C]hard_delete = true\f[R] if you would like
|
|
files to be deleted straight away.
|
|
.SS Standard Options
|
|
.PP
|
|
Here are the standard options specific to sugarsync (Sugarsync).
|
|
.SS \-\-sugarsync\-app\-id
|
|
.PP
|
|
Sugarsync App ID.
|
|
.PP
|
|
Leave blank to use rclone\[aq]s.
|
|
.IP \[bu] 2
|
|
Config: app_id
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_SUGARSYNC_APP_ID
|
|
.IP \[bu] 2
|
|
Type: string
|
|
.IP \[bu] 2
|
|
Default: \[dq]\[dq]
|
|
.SS \-\-sugarsync\-access\-key\-id
|
|
.PP
|
|
Sugarsync Access Key ID.
|
|
.PP
|
|
Leave blank to use rclone\[aq]s.
|
|
.IP \[bu] 2
|
|
Config: access_key_id
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_SUGARSYNC_ACCESS_KEY_ID
|
|
.IP \[bu] 2
|
|
Type: string
|
|
.IP \[bu] 2
|
|
Default: \[dq]\[dq]
|
|
.SS \-\-sugarsync\-private\-access\-key
|
|
.PP
|
|
Sugarsync Private Access Key
|
|
.PP
|
|
Leave blank to use rclone\[aq]s.
|
|
.IP \[bu] 2
|
|
Config: private_access_key
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_SUGARSYNC_PRIVATE_ACCESS_KEY
|
|
.IP \[bu] 2
|
|
Type: string
|
|
.IP \[bu] 2
|
|
Default: \[dq]\[dq]
|
|
.SS \-\-sugarsync\-hard\-delete
|
|
.PP
|
|
Permanently delete files if true otherwise put them in the deleted
|
|
files.
|
|
.IP \[bu] 2
|
|
Config: hard_delete
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_SUGARSYNC_HARD_DELETE
|
|
.IP \[bu] 2
|
|
Type: bool
|
|
.IP \[bu] 2
|
|
Default: false
|
|
.SS Advanced Options
|
|
.PP
|
|
Here are the advanced options specific to sugarsync (Sugarsync).
|
|
.SS \-\-sugarsync\-refresh\-token
|
|
.PP
|
|
Sugarsync refresh token
|
|
.PP
|
|
Leave blank normally, will be auto configured by rclone.
|
|
.IP \[bu] 2
|
|
Config: refresh_token
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_SUGARSYNC_REFRESH_TOKEN
|
|
.IP \[bu] 2
|
|
Type: string
|
|
.IP \[bu] 2
|
|
Default: \[dq]\[dq]
|
|
.SS \-\-sugarsync\-authorization
|
|
.PP
|
|
Sugarsync authorization
|
|
.PP
|
|
Leave blank normally, will be auto configured by rclone.
|
|
.IP \[bu] 2
|
|
Config: authorization
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_SUGARSYNC_AUTHORIZATION
|
|
.IP \[bu] 2
|
|
Type: string
|
|
.IP \[bu] 2
|
|
Default: \[dq]\[dq]
|
|
.SS \-\-sugarsync\-authorization\-expiry
|
|
.PP
|
|
Sugarsync authorization expiry
|
|
.PP
|
|
Leave blank normally, will be auto configured by rclone.
|
|
.IP \[bu] 2
|
|
Config: authorization_expiry
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_SUGARSYNC_AUTHORIZATION_EXPIRY
|
|
.IP \[bu] 2
|
|
Type: string
|
|
.IP \[bu] 2
|
|
Default: \[dq]\[dq]
|
|
.SS \-\-sugarsync\-user
|
|
.PP
|
|
Sugarsync user
|
|
.PP
|
|
Leave blank normally, will be auto configured by rclone.
|
|
.IP \[bu] 2
|
|
Config: user
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_SUGARSYNC_USER
|
|
.IP \[bu] 2
|
|
Type: string
|
|
.IP \[bu] 2
|
|
Default: \[dq]\[dq]
|
|
.SS \-\-sugarsync\-root\-id
|
|
.PP
|
|
Sugarsync root id
|
|
.PP
|
|
Leave blank normally, will be auto configured by rclone.
|
|
.IP \[bu] 2
|
|
Config: root_id
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_SUGARSYNC_ROOT_ID
|
|
.IP \[bu] 2
|
|
Type: string
|
|
.IP \[bu] 2
|
|
Default: \[dq]\[dq]
|
|
.SS \-\-sugarsync\-deleted\-id
|
|
.PP
|
|
Sugarsync deleted folder id
|
|
.PP
|
|
Leave blank normally, will be auto configured by rclone.
|
|
.IP \[bu] 2
|
|
Config: deleted_id
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_SUGARSYNC_DELETED_ID
|
|
.IP \[bu] 2
|
|
Type: string
|
|
.IP \[bu] 2
|
|
Default: \[dq]\[dq]
|
|
.SS \-\-sugarsync\-encoding
|
|
.PP
|
|
This sets the encoding for the backend.
|
|
.PP
|
|
See: the encoding section in the
|
|
overview (https://rclone.org/overview/#encoding) for more info.
|
|
.IP \[bu] 2
|
|
Config: encoding
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_SUGARSYNC_ENCODING
|
|
.IP \[bu] 2
|
|
Type: MultiEncoder
|
|
.IP \[bu] 2
|
|
Default: Slash,Ctl,InvalidUtf8,Dot
|
|
.SS Limitations
|
|
.PP
|
|
\f[C]rclone about\f[R] is not supported by the SugarSync backend.
|
|
Backends without this capability cannot determine free space for an
|
|
rclone mount or use policy \f[C]mfs\f[R] (most free space) as a member
|
|
of an rclone union remote.
|
|
.PP
|
|
See List of backends that do not support rclone
|
|
about (https://rclone.org/overview/#optional-features) See rclone
|
|
about (https://rclone.org/commands/rclone_about/)
|
|
.SS Tardigrade
|
|
.PP
|
|
Tardigrade (https://tardigrade.io) is an encrypted, secure, and
|
|
cost\-effective object storage service that enables you to store, back
|
|
up, and archive large amounts of data in a decentralized manner.
|
|
.SS Setup
|
|
.PP
|
|
To make a new Tardigrade configuration you need one of the following: *
|
|
Access Grant that someone else shared with you.
|
|
* API
|
|
Key (https://documentation.tardigrade.io/getting-started/uploading-your-first-object/create-an-api-key)
|
|
of a Tardigrade project you are a member of.
|
|
.PP
|
|
Here is an example of how to make a remote called \f[C]remote\f[R].
|
|
First run:
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone config
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
This will guide you through an interactive setup process:
|
|
.SS Setup with access grant
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
No remotes found \- make a new one
|
|
n) New remote
|
|
s) Set configuration password
|
|
q) Quit config
|
|
n/s/q> n
|
|
name> remote
|
|
Type of storage to configure.
|
|
Enter a string value. Press Enter for the default (\[dq]\[dq]).
|
|
Choose a number from below, or type in your own value
|
|
[snip]
|
|
XX / Tardigrade Decentralized Cloud Storage
|
|
\[rs] \[dq]tardigrade\[dq]
|
|
[snip]
|
|
Storage> tardigrade
|
|
** See help for tardigrade backend at: https://rclone.org/tardigrade/ **
|
|
|
|
Choose an authentication method.
|
|
Enter a string value. Press Enter for the default (\[dq]existing\[dq]).
|
|
Choose a number from below, or type in your own value
|
|
1 / Use an existing access grant.
|
|
\[rs] \[dq]existing\[dq]
|
|
2 / Create a new access grant from satellite address, API key, and passphrase.
|
|
\[rs] \[dq]new\[dq]
|
|
provider> existing
|
|
Access Grant.
|
|
Enter a string value. Press Enter for the default (\[dq]\[dq]).
|
|
access_grant> your\-access\-grant\-received\-by\-someone\-else
|
|
Remote config
|
|
\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-
|
|
[remote]
|
|
type = tardigrade
|
|
access_grant = your\-access\-grant\-received\-by\-someone\-else
|
|
\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-
|
|
y) Yes this is OK (default)
|
|
e) Edit this remote
|
|
d) Delete this remote
|
|
y/e/d> y
|
|
\f[R]
|
|
.fi
|
|
.SS Setup with API key and passphrase
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
No remotes found \- make a new one
|
|
n) New remote
|
|
s) Set configuration password
|
|
q) Quit config
|
|
n/s/q> n
|
|
name> remote
|
|
Type of storage to configure.
|
|
Enter a string value. Press Enter for the default (\[dq]\[dq]).
|
|
Choose a number from below, or type in your own value
|
|
[snip]
|
|
XX / Tardigrade Decentralized Cloud Storage
|
|
\[rs] \[dq]tardigrade\[dq]
|
|
[snip]
|
|
Storage> tardigrade
|
|
** See help for tardigrade backend at: https://rclone.org/tardigrade/ **
|
|
|
|
Choose an authentication method.
|
|
Enter a string value. Press Enter for the default (\[dq]existing\[dq]).
|
|
Choose a number from below, or type in your own value
|
|
1 / Use an existing access grant.
|
|
\[rs] \[dq]existing\[dq]
|
|
2 / Create a new access grant from satellite address, API key, and passphrase.
|
|
\[rs] \[dq]new\[dq]
|
|
provider> new
|
|
Satellite Address. Custom satellite address should match the format: \[ga]<nodeid>\[at]<address>:<port>\[ga].
|
|
Enter a string value. Press Enter for the default (\[dq]us\-central\-1.tardigrade.io\[dq]).
|
|
Choose a number from below, or type in your own value
|
|
1 / US Central 1
|
|
\[rs] \[dq]us\-central\-1.tardigrade.io\[dq]
|
|
2 / Europe West 1
|
|
\[rs] \[dq]europe\-west\-1.tardigrade.io\[dq]
|
|
3 / Asia East 1
|
|
\[rs] \[dq]asia\-east\-1.tardigrade.io\[dq]
|
|
satellite_address> 1
|
|
API Key.
|
|
Enter a string value. Press Enter for the default (\[dq]\[dq]).
|
|
api_key> your\-api\-key\-for\-your\-tardigrade\-project
|
|
Encryption Passphrase. To access existing objects enter passphrase used for uploading.
|
|
Enter a string value. Press Enter for the default (\[dq]\[dq]).
|
|
passphrase> your\-human\-readable\-encryption\-passphrase
|
|
Remote config
|
|
\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-
|
|
[remote]
|
|
type = tardigrade
|
|
satellite_address = 12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S\[at]us\-central\-1.tardigrade.io:7777
|
|
api_key = your\-api\-key\-for\-your\-tardigrade\-project
|
|
passphrase = your\-human\-readable\-encryption\-passphrase
|
|
access_grant = the\-access\-grant\-generated\-from\-the\-api\-key\-and\-passphrase
|
|
\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-
|
|
y) Yes this is OK (default)
|
|
e) Edit this remote
|
|
d) Delete this remote
|
|
y/e/d> y
|
|
\f[R]
|
|
.fi
|
|
.SS Usage
|
|
.PP
|
|
Paths are specified as \f[C]remote:bucket\f[R] (or \f[C]remote:\f[R] for
|
|
the \f[C]lsf\f[R] command.) You may put subdirectories in too, e.g.
|
|
\f[C]remote:bucket/path/to/dir\f[R].
|
|
.PP
|
|
Once configured you can then use \f[C]rclone\f[R] like this.
|
|
.SS Create a new bucket
|
|
.PP
|
|
Use the \f[C]mkdir\f[R] command to create new bucket, e.g.
|
|
\f[C]bucket\f[R].
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone mkdir remote:bucket
|
|
\f[R]
|
|
.fi
|
|
.SS List all buckets
|
|
.PP
|
|
Use the \f[C]lsf\f[R] command to list all buckets.
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone lsf remote:
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
Note the colon (\f[C]:\f[R]) character at the end of the command line.
|
|
.SS Delete a bucket
|
|
.PP
|
|
Use the \f[C]rmdir\f[R] command to delete an empty bucket.
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone rmdir remote:bucket
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
Use the \f[C]purge\f[R] command to delete a non\-empty bucket with all
|
|
its content.
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone purge remote:bucket
|
|
\f[R]
|
|
.fi
|
|
.SS Upload objects
|
|
.PP
|
|
Use the \f[C]copy\f[R] command to upload an object.
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone copy \-\-progress /home/local/directory/file.ext remote:bucket/path/to/dir/
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
The \f[C]\-\-progress\f[R] flag is for displaying progress information.
|
|
Remove it if you don\[aq]t need this information.
|
|
.PP
|
|
Use a folder in the local path to upload all its objects.
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone copy \-\-progress /home/local/directory/ remote:bucket/path/to/dir/
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
Only modified files will be copied.
|
|
.SS List objects
|
|
.PP
|
|
Use the \f[C]ls\f[R] command to list recursively all objects in a
|
|
bucket.
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone ls remote:bucket
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
Add the folder to the remote path to list recursively all objects in
|
|
this folder.
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone ls remote:bucket/path/to/dir/
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
Use the \f[C]lsf\f[R] command to list non\-recursively all objects in a
|
|
bucket or a folder.
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone lsf remote:bucket/path/to/dir/
|
|
\f[R]
|
|
.fi
|
|
.SS Download objects
|
|
.PP
|
|
Use the \f[C]copy\f[R] command to download an object.
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone copy \-\-progress remote:bucket/path/to/dir/file.ext /home/local/directory/
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
The \f[C]\-\-progress\f[R] flag is for displaying progress information.
|
|
Remove it if you don\[aq]t need this information.
|
|
.PP
|
|
Use a folder in the remote path to download all its objects.
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone copy \-\-progress remote:bucket/path/to/dir/ /home/local/directory/
|
|
\f[R]
|
|
.fi
|
|
.SS Delete objects
|
|
.PP
|
|
Use the \f[C]deletefile\f[R] command to delete a single object.
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone deletefile remote:bucket/path/to/dir/file.ext
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
Use the \f[C]delete\f[R] command to delete all object in a folder.
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone delete remote:bucket/path/to/dir/
|
|
\f[R]
|
|
.fi
|
|
.SS Print the total size of objects
|
|
.PP
|
|
Use the \f[C]size\f[R] command to print the total size of objects in a
|
|
bucket or a folder.
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone size remote:bucket/path/to/dir/
|
|
\f[R]
|
|
.fi
|
|
.SS Sync two Locations
|
|
.PP
|
|
Use the \f[C]sync\f[R] command to sync the source to the destination,
|
|
changing the destination only, deleting any excess files.
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone sync \-i \-\-progress /home/local/directory/ remote:bucket/path/to/dir/
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
The \f[C]\-\-progress\f[R] flag is for displaying progress information.
|
|
Remove it if you don\[aq]t need this information.
|
|
.PP
|
|
Since this can cause data loss, test first with the
|
|
\f[C]\-\-dry\-run\f[R] flag to see exactly what would be copied and
|
|
deleted.
|
|
.PP
|
|
The sync can be done also from Tardigrade to the local file system.
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone sync \-i \-\-progress remote:bucket/path/to/dir/ /home/local/directory/
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
Or between two Tardigrade buckets.
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone sync \-i \-\-progress remote\-us:bucket/path/to/dir/ remote\-europe:bucket/path/to/dir/
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
Or even between another cloud storage and Tardigrade.
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone sync \-i \-\-progress s3:bucket/path/to/dir/ tardigrade:bucket/path/to/dir/
|
|
\f[R]
|
|
.fi
|
|
.SS Standard Options
|
|
.PP
|
|
Here are the standard options specific to tardigrade (Tardigrade
|
|
Decentralized Cloud Storage).
|
|
.SS \-\-tardigrade\-provider
|
|
.PP
|
|
Choose an authentication method.
|
|
.IP \[bu] 2
|
|
Config: provider
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_TARDIGRADE_PROVIDER
|
|
.IP \[bu] 2
|
|
Type: string
|
|
.IP \[bu] 2
|
|
Default: \[dq]existing\[dq]
|
|
.IP \[bu] 2
|
|
Examples:
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
\[dq]existing\[dq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Use an existing access grant.
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[dq]new\[dq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Create a new access grant from satellite address, API key, and
|
|
passphrase.
|
|
.RE
|
|
.RE
|
|
.SS \-\-tardigrade\-access\-grant
|
|
.PP
|
|
Access Grant.
|
|
.IP \[bu] 2
|
|
Config: access_grant
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_TARDIGRADE_ACCESS_GRANT
|
|
.IP \[bu] 2
|
|
Type: string
|
|
.IP \[bu] 2
|
|
Default: \[dq]\[dq]
|
|
.SS \-\-tardigrade\-satellite\-address
|
|
.PP
|
|
Satellite Address.
|
|
Custom satellite address should match the format:
|
|
\f[C]<nodeid>\[at]<address>:<port>\f[R].
|
|
.IP \[bu] 2
|
|
Config: satellite_address
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_TARDIGRADE_SATELLITE_ADDRESS
|
|
.IP \[bu] 2
|
|
Type: string
|
|
.IP \[bu] 2
|
|
Default: \[dq]us\-central\-1.tardigrade.io\[dq]
|
|
.IP \[bu] 2
|
|
Examples:
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
\[dq]us\-central\-1.tardigrade.io\[dq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
US Central 1
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[dq]europe\-west\-1.tardigrade.io\[dq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Europe West 1
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[dq]asia\-east\-1.tardigrade.io\[dq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Asia East 1
|
|
.RE
|
|
.RE
|
|
.SS \-\-tardigrade\-api\-key
|
|
.PP
|
|
API Key.
|
|
.IP \[bu] 2
|
|
Config: api_key
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_TARDIGRADE_API_KEY
|
|
.IP \[bu] 2
|
|
Type: string
|
|
.IP \[bu] 2
|
|
Default: \[dq]\[dq]
|
|
.SS \-\-tardigrade\-passphrase
|
|
.PP
|
|
Encryption Passphrase.
|
|
To access existing objects enter passphrase used for uploading.
|
|
.IP \[bu] 2
|
|
Config: passphrase
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_TARDIGRADE_PASSPHRASE
|
|
.IP \[bu] 2
|
|
Type: string
|
|
.IP \[bu] 2
|
|
Default: \[dq]\[dq]
|
|
.SS Limitations
|
|
.PP
|
|
\f[C]rclone about\f[R] is not supported by the rclone Tardigrade
|
|
backend.
|
|
Backends without this capability cannot determine free space for an
|
|
rclone mount or use policy \f[C]mfs\f[R] (most free space) as a member
|
|
of an rclone union remote.
|
|
.PP
|
|
See List of backends that do not support rclone
|
|
about (https://rclone.org/overview/#optional-features) See rclone
|
|
about (https://rclone.org/commands/rclone_about/)
|
|
.SS Union
|
|
.PP
|
|
The \f[C]union\f[R] remote provides a unification similar to UnionFS
|
|
using other remotes.
|
|
.PP
|
|
Paths may be as deep as required or a local path, e.g.
|
|
\f[C]remote:directory/subdirectory\f[R] or
|
|
\f[C]/directory/subdirectory\f[R].
|
|
.PP
|
|
During the initial setup with \f[C]rclone config\f[R] you will specify
|
|
the upstream remotes as a space separated list.
|
|
The upstream remotes can either be a local paths or other remotes.
|
|
.PP
|
|
Attribute \f[C]:ro\f[R] and \f[C]:nc\f[R] can be attach to the end of
|
|
path to tag the remote as \f[B]read only\f[R] or \f[B]no create\f[R],
|
|
e.g.
|
|
\f[C]remote:directory/subdirectory:ro\f[R] or
|
|
\f[C]remote:directory/subdirectory:nc\f[R].
|
|
.PP
|
|
Subfolders can be used in upstream remotes.
|
|
Assume a union remote named \f[C]backup\f[R] with the remotes
|
|
\f[C]mydrive:private/backup\f[R].
|
|
Invoking \f[C]rclone mkdir backup:desktop\f[R] is exactly the same as
|
|
invoking \f[C]rclone mkdir mydrive:private/backup/desktop\f[R].
|
|
.PP
|
|
There will be no special handling of paths containing \f[C]..\f[R]
|
|
segments.
|
|
Invoking \f[C]rclone mkdir backup:../desktop\f[R] is exactly the same as
|
|
invoking \f[C]rclone mkdir mydrive:private/backup/../desktop\f[R].
|
|
.SS Behavior / Policies
|
|
.PP
|
|
The behavior of union backend is inspired by
|
|
trapexit/mergerfs (https://github.com/trapexit/mergerfs).
|
|
All functions are grouped into 3 categories: \f[B]action\f[R],
|
|
\f[B]create\f[R] and \f[B]search\f[R].
|
|
These functions and categories can be assigned a policy which dictates
|
|
what file or directory is chosen when performing that behavior.
|
|
Any policy can be assigned to a function or category though some may not
|
|
be very useful in practice.
|
|
For instance: \f[B]rand\f[R] (random) may be useful for file creation
|
|
(create) but could lead to very odd behavior if used for
|
|
\f[C]delete\f[R] if there were more than one copy of the file.
|
|
.SS Function / Category classifications
|
|
.PP
|
|
.TS
|
|
tab(@);
|
|
lw(5.8n) lw(15.0n) lw(49.2n).
|
|
T{
|
|
Category
|
|
T}@T{
|
|
Description
|
|
T}@T{
|
|
Functions
|
|
T}
|
|
_
|
|
T{
|
|
action
|
|
T}@T{
|
|
Writing Existing file
|
|
T}@T{
|
|
move, rmdir, rmdirs, delete, purge and copy, sync (as destination when
|
|
file exist)
|
|
T}
|
|
T{
|
|
create
|
|
T}@T{
|
|
Create non\-existing file
|
|
T}@T{
|
|
copy, sync (as destination when file not exist)
|
|
T}
|
|
T{
|
|
search
|
|
T}@T{
|
|
Reading and listing file
|
|
T}@T{
|
|
ls, lsd, lsl, cat, md5sum, sha1sum and copy, sync (as source)
|
|
T}
|
|
T{
|
|
N/A
|
|
T}@T{
|
|
T}@T{
|
|
size, about
|
|
T}
|
|
.TE
|
|
.SS Path Preservation
|
|
.PP
|
|
Policies, as described below, are of two basic types.
|
|
\f[C]path preserving\f[R] and \f[C]non\-path preserving\f[R].
|
|
.PP
|
|
All policies which start with \f[C]ep\f[R] (\f[B]epff\f[R],
|
|
\f[B]eplfs\f[R], \f[B]eplus\f[R], \f[B]epmfs\f[R], \f[B]eprand\f[R]) are
|
|
\f[C]path preserving\f[R].
|
|
\f[C]ep\f[R] stands for \f[C]existing path\f[R].
|
|
.PP
|
|
A path preserving policy will only consider upstreams where the relative
|
|
path being accessed already exists.
|
|
.PP
|
|
When using non\-path preserving policies paths will be created in target
|
|
upstreams as necessary.
|
|
.SS Quota Relevant Policies
|
|
.PP
|
|
Some policies rely on quota information.
|
|
These policies should be used only if your upstreams support the
|
|
respective quota fields.
|
|
.PP
|
|
.TS
|
|
tab(@);
|
|
l l.
|
|
T{
|
|
Policy
|
|
T}@T{
|
|
Required Field
|
|
T}
|
|
_
|
|
T{
|
|
lfs, eplfs
|
|
T}@T{
|
|
Free
|
|
T}
|
|
T{
|
|
mfs, epmfs
|
|
T}@T{
|
|
Free
|
|
T}
|
|
T{
|
|
lus, eplus
|
|
T}@T{
|
|
Used
|
|
T}
|
|
T{
|
|
lno, eplno
|
|
T}@T{
|
|
Objects
|
|
T}
|
|
.TE
|
|
.PP
|
|
To check if your upstream supports the field, run
|
|
\f[C]rclone about remote: [flags]\f[R] and see if the required field
|
|
exists.
|
|
.SS Filters
|
|
.PP
|
|
Policies basically search upstream remotes and create a list of files /
|
|
paths for functions to work on.
|
|
The policy is responsible for filtering and sorting.
|
|
The policy type defines the sorting but filtering is mostly uniform as
|
|
described below.
|
|
.IP \[bu] 2
|
|
No \f[B]search\f[R] policies filter.
|
|
.IP \[bu] 2
|
|
All \f[B]action\f[R] policies will filter out remotes which are tagged
|
|
as \f[B]read\-only\f[R].
|
|
.IP \[bu] 2
|
|
All \f[B]create\f[R] policies will filter out remotes which are tagged
|
|
\f[B]read\-only\f[R] or \f[B]no\-create\f[R].
|
|
.PP
|
|
If all remotes are filtered an error will be returned.
|
|
.SS Policy descriptions
|
|
.PP
|
|
The policies definition are inspired by
|
|
trapexit/mergerfs (https://github.com/trapexit/mergerfs) but not exactly
|
|
the same.
|
|
Some policy definition could be different due to the much larger latency
|
|
of remote file systems.
|
|
.PP
|
|
.TS
|
|
tab(@);
|
|
lw(16.2n) lw(53.8n).
|
|
T{
|
|
Policy
|
|
T}@T{
|
|
Description
|
|
T}
|
|
_
|
|
T{
|
|
all
|
|
T}@T{
|
|
Search category: same as \f[B]epall\f[R].
|
|
Action category: same as \f[B]epall\f[R].
|
|
Create category: act on all upstreams.
|
|
T}
|
|
T{
|
|
epall (existing path, all)
|
|
T}@T{
|
|
Search category: Given this order configured, act on the first one found
|
|
where the relative path exists.
|
|
Action category: apply to all found.
|
|
Create category: act on all upstreams where the relative path exists.
|
|
T}
|
|
T{
|
|
epff (existing path, first found)
|
|
T}@T{
|
|
Act on the first one found, by the time upstreams reply, where the
|
|
relative path exists.
|
|
T}
|
|
T{
|
|
eplfs (existing path, least free space)
|
|
T}@T{
|
|
Of all the upstreams on which the relative path exists choose the one
|
|
with the least free space.
|
|
T}
|
|
T{
|
|
eplus (existing path, least used space)
|
|
T}@T{
|
|
Of all the upstreams on which the relative path exists choose the one
|
|
with the least used space.
|
|
T}
|
|
T{
|
|
eplno (existing path, least number of objects)
|
|
T}@T{
|
|
Of all the upstreams on which the relative path exists choose the one
|
|
with the least number of objects.
|
|
T}
|
|
T{
|
|
epmfs (existing path, most free space)
|
|
T}@T{
|
|
Of all the upstreams on which the relative path exists choose the one
|
|
with the most free space.
|
|
T}
|
|
T{
|
|
eprand (existing path, random)
|
|
T}@T{
|
|
Calls \f[B]epall\f[R] and then randomizes.
|
|
Returns only one upstream.
|
|
T}
|
|
T{
|
|
ff (first found)
|
|
T}@T{
|
|
Search category: same as \f[B]epff\f[R].
|
|
Action category: same as \f[B]epff\f[R].
|
|
Create category: Act on the first one found by the time upstreams reply.
|
|
T}
|
|
T{
|
|
lfs (least free space)
|
|
T}@T{
|
|
Search category: same as \f[B]eplfs\f[R].
|
|
Action category: same as \f[B]eplfs\f[R].
|
|
Create category: Pick the upstream with the least available free space.
|
|
T}
|
|
T{
|
|
lus (least used space)
|
|
T}@T{
|
|
Search category: same as \f[B]eplus\f[R].
|
|
Action category: same as \f[B]eplus\f[R].
|
|
Create category: Pick the upstream with the least used space.
|
|
T}
|
|
T{
|
|
lno (least number of objects)
|
|
T}@T{
|
|
Search category: same as \f[B]eplno\f[R].
|
|
Action category: same as \f[B]eplno\f[R].
|
|
Create category: Pick the upstream with the least number of objects.
|
|
T}
|
|
T{
|
|
mfs (most free space)
|
|
T}@T{
|
|
Search category: same as \f[B]epmfs\f[R].
|
|
Action category: same as \f[B]epmfs\f[R].
|
|
Create category: Pick the upstream with the most available free space.
|
|
T}
|
|
T{
|
|
newest
|
|
T}@T{
|
|
Pick the file / directory with the largest mtime.
|
|
T}
|
|
T{
|
|
rand (random)
|
|
T}@T{
|
|
Calls \f[B]all\f[R] and then randomizes.
|
|
Returns only one upstream.
|
|
T}
|
|
.TE
|
|
.SS Setup
|
|
.PP
|
|
Here is an example of how to make a union called \f[C]remote\f[R] for
|
|
local folders.
|
|
First run:
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone config
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
This will guide you through an interactive setup process:
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
No remotes found \- make a new one
|
|
n) New remote
|
|
s) Set configuration password
|
|
q) Quit config
|
|
n/s/q> n
|
|
name> remote
|
|
Type of storage to configure.
|
|
Choose a number from below, or type in your own value
|
|
[snip]
|
|
XX / Union merges the contents of several remotes
|
|
\[rs] \[dq]union\[dq]
|
|
[snip]
|
|
Storage> union
|
|
List of space separated upstreams.
|
|
Can be \[aq]upstreama:test/dir upstreamb:\[aq], \[aq]\[rs]\[dq]upstreama:test/space:ro dir\[rs]\[dq] upstreamb:\[aq], etc.
|
|
Enter a string value. Press Enter for the default (\[dq]\[dq]).
|
|
upstreams> remote1:dir1 remote2:dir2 remote3:dir3
|
|
Policy to choose upstream on ACTION class.
|
|
Enter a string value. Press Enter for the default (\[dq]epall\[dq]).
|
|
action_policy>
|
|
Policy to choose upstream on CREATE class.
|
|
Enter a string value. Press Enter for the default (\[dq]epmfs\[dq]).
|
|
create_policy>
|
|
Policy to choose upstream on SEARCH class.
|
|
Enter a string value. Press Enter for the default (\[dq]ff\[dq]).
|
|
search_policy>
|
|
Cache time of usage and free space (in seconds). This option is only useful when a path preserving policy is used.
|
|
Enter a signed integer. Press Enter for the default (\[dq]120\[dq]).
|
|
cache_time>
|
|
Remote config
|
|
\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-
|
|
[remote]
|
|
type = union
|
|
upstreams = remote1:dir1 remote2:dir2 remote3:dir3
|
|
\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-
|
|
y) Yes this is OK
|
|
e) Edit this remote
|
|
d) Delete this remote
|
|
y/e/d> y
|
|
Current remotes:
|
|
|
|
Name Type
|
|
==== ====
|
|
remote union
|
|
|
|
e) Edit existing remote
|
|
n) New remote
|
|
d) Delete remote
|
|
r) Rename remote
|
|
c) Copy remote
|
|
s) Set configuration password
|
|
q) Quit config
|
|
e/n/d/r/c/s/q> q
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
Once configured you can then use \f[C]rclone\f[R] like this,
|
|
.PP
|
|
List directories in top level in \f[C]remote1:dir1\f[R],
|
|
\f[C]remote2:dir2\f[R] and \f[C]remote3:dir3\f[R]
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone lsd remote:
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
List all the files in \f[C]remote1:dir1\f[R], \f[C]remote2:dir2\f[R] and
|
|
\f[C]remote3:dir3\f[R]
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone ls remote:
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
Copy another local directory to the union directory called source, which
|
|
will be placed into \f[C]remote3:dir3\f[R]
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone copy C:\[rs]source remote:source
|
|
\f[R]
|
|
.fi
|
|
.SS Standard Options
|
|
.PP
|
|
Here are the standard options specific to union (Union merges the
|
|
contents of several upstream fs).
|
|
.SS \-\-union\-upstreams
|
|
.PP
|
|
List of space separated upstreams.
|
|
Can be \[aq]upstreama:test/dir upstreamb:\[aq],
|
|
\[aq]\[dq]upstreama:test/space:ro dir\[dq] upstreamb:\[aq], etc.
|
|
.IP \[bu] 2
|
|
Config: upstreams
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_UNION_UPSTREAMS
|
|
.IP \[bu] 2
|
|
Type: string
|
|
.IP \[bu] 2
|
|
Default: \[dq]\[dq]
|
|
.SS \-\-union\-action\-policy
|
|
.PP
|
|
Policy to choose upstream on ACTION category.
|
|
.IP \[bu] 2
|
|
Config: action_policy
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_UNION_ACTION_POLICY
|
|
.IP \[bu] 2
|
|
Type: string
|
|
.IP \[bu] 2
|
|
Default: \[dq]epall\[dq]
|
|
.SS \-\-union\-create\-policy
|
|
.PP
|
|
Policy to choose upstream on CREATE category.
|
|
.IP \[bu] 2
|
|
Config: create_policy
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_UNION_CREATE_POLICY
|
|
.IP \[bu] 2
|
|
Type: string
|
|
.IP \[bu] 2
|
|
Default: \[dq]epmfs\[dq]
|
|
.SS \-\-union\-search\-policy
|
|
.PP
|
|
Policy to choose upstream on SEARCH category.
|
|
.IP \[bu] 2
|
|
Config: search_policy
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_UNION_SEARCH_POLICY
|
|
.IP \[bu] 2
|
|
Type: string
|
|
.IP \[bu] 2
|
|
Default: \[dq]ff\[dq]
|
|
.SS \-\-union\-cache\-time
|
|
.PP
|
|
Cache time of usage and free space (in seconds).
|
|
This option is only useful when a path preserving policy is used.
|
|
.IP \[bu] 2
|
|
Config: cache_time
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_UNION_CACHE_TIME
|
|
.IP \[bu] 2
|
|
Type: int
|
|
.IP \[bu] 2
|
|
Default: 120
|
|
.SS WebDAV
|
|
.PP
|
|
Paths are specified as \f[C]remote:path\f[R]
|
|
.PP
|
|
Paths may be as deep as required, e.g.
|
|
\f[C]remote:directory/subdirectory\f[R].
|
|
.PP
|
|
To configure the WebDAV remote you will need to have a URL for it, and a
|
|
username and password.
|
|
If you know what kind of system you are connecting to then rclone can
|
|
enable extra features.
|
|
.PP
|
|
Here is an example of how to make a remote called \f[C]remote\f[R].
|
|
First run:
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone config
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
This will guide you through an interactive setup process:
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
No remotes found \- make a new one
|
|
n) New remote
|
|
s) Set configuration password
|
|
q) Quit config
|
|
n/s/q> n
|
|
name> remote
|
|
Type of storage to configure.
|
|
Choose a number from below, or type in your own value
|
|
[snip]
|
|
XX / Webdav
|
|
\[rs] \[dq]webdav\[dq]
|
|
[snip]
|
|
Storage> webdav
|
|
URL of http host to connect to
|
|
Choose a number from below, or type in your own value
|
|
1 / Connect to example.com
|
|
\[rs] \[dq]https://example.com\[dq]
|
|
url> https://example.com/remote.php/webdav/
|
|
Name of the Webdav site/service/software you are using
|
|
Choose a number from below, or type in your own value
|
|
1 / Nextcloud
|
|
\[rs] \[dq]nextcloud\[dq]
|
|
2 / Owncloud
|
|
\[rs] \[dq]owncloud\[dq]
|
|
3 / Sharepoint Online, authenticated by Microsoft account.
|
|
\[rs] \[dq]sharepoint\[dq]
|
|
4 / Sharepoint with NTLM authentication. Usually self\-hosted or on\-premises.
|
|
\[rs] \[dq]sharepoint\-ntlm\[dq]
|
|
5 / Other site/service or software
|
|
\[rs] \[dq]other\[dq]
|
|
vendor> 1
|
|
User name
|
|
user> user
|
|
Password.
|
|
y) Yes type in my own password
|
|
g) Generate random password
|
|
n) No leave this optional password blank
|
|
y/g/n> y
|
|
Enter the password:
|
|
password:
|
|
Confirm the password:
|
|
password:
|
|
Bearer token instead of user/pass (e.g. a Macaroon)
|
|
bearer_token>
|
|
Remote config
|
|
\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-
|
|
[remote]
|
|
type = webdav
|
|
url = https://example.com/remote.php/webdav/
|
|
vendor = nextcloud
|
|
user = user
|
|
pass = *** ENCRYPTED ***
|
|
bearer_token =
|
|
\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-
|
|
y) Yes this is OK
|
|
e) Edit this remote
|
|
d) Delete this remote
|
|
y/e/d> y
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
Once configured you can then use \f[C]rclone\f[R] like this,
|
|
.PP
|
|
List directories in top level of your WebDAV
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone lsd remote:
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
List all the files in your WebDAV
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone ls remote:
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
To copy a local directory to an WebDAV directory called backup
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone copy /home/source remote:backup
|
|
\f[R]
|
|
.fi
|
|
.SS Modified time and hashes
|
|
.PP
|
|
Plain WebDAV does not support modified times.
|
|
However when used with Owncloud or Nextcloud rclone will support
|
|
modified times.
|
|
.PP
|
|
Likewise plain WebDAV does not support hashes, however when used with
|
|
Owncloud or Nextcloud rclone will support SHA1 and MD5 hashes.
|
|
Depending on the exact version of Owncloud or Nextcloud hashes may
|
|
appear on all objects, or only on objects which had a hash uploaded with
|
|
them.
|
|
.SS Standard Options
|
|
.PP
|
|
Here are the standard options specific to webdav (Webdav).
|
|
.SS \-\-webdav\-url
|
|
.PP
|
|
URL of http host to connect to
|
|
.IP \[bu] 2
|
|
Config: url
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_WEBDAV_URL
|
|
.IP \[bu] 2
|
|
Type: string
|
|
.IP \[bu] 2
|
|
Default: \[dq]\[dq]
|
|
.IP \[bu] 2
|
|
Examples:
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
\[dq]https://example.com\[dq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Connect to example.com
|
|
.RE
|
|
.RE
|
|
.SS \-\-webdav\-vendor
|
|
.PP
|
|
Name of the Webdav site/service/software you are using
|
|
.IP \[bu] 2
|
|
Config: vendor
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_WEBDAV_VENDOR
|
|
.IP \[bu] 2
|
|
Type: string
|
|
.IP \[bu] 2
|
|
Default: \[dq]\[dq]
|
|
.IP \[bu] 2
|
|
Examples:
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
\[dq]nextcloud\[dq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Nextcloud
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[dq]owncloud\[dq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Owncloud
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[dq]sharepoint\[dq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Sharepoint Online, authenticated by Microsoft account.
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[dq]sharepoint\-ntlm\[dq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Sharepoint with NTLM authentication.
|
|
Usually self\-hosted or on\-premises.
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[dq]other\[dq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Other site/service or software
|
|
.RE
|
|
.RE
|
|
.SS \-\-webdav\-user
|
|
.PP
|
|
User name.
|
|
In case NTLM authentication is used, the username should be in the
|
|
format \[aq]Domain\[aq].
|
|
.IP \[bu] 2
|
|
Config: user
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_WEBDAV_USER
|
|
.IP \[bu] 2
|
|
Type: string
|
|
.IP \[bu] 2
|
|
Default: \[dq]\[dq]
|
|
.SS \-\-webdav\-pass
|
|
.PP
|
|
Password.
|
|
.PP
|
|
\f[B]NB\f[R] Input to this must be obscured \- see rclone
|
|
obscure (https://rclone.org/commands/rclone_obscure/).
|
|
.IP \[bu] 2
|
|
Config: pass
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_WEBDAV_PASS
|
|
.IP \[bu] 2
|
|
Type: string
|
|
.IP \[bu] 2
|
|
Default: \[dq]\[dq]
|
|
.SS \-\-webdav\-bearer\-token
|
|
.PP
|
|
Bearer token instead of user/pass (e.g.
|
|
a Macaroon)
|
|
.IP \[bu] 2
|
|
Config: bearer_token
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_WEBDAV_BEARER_TOKEN
|
|
.IP \[bu] 2
|
|
Type: string
|
|
.IP \[bu] 2
|
|
Default: \[dq]\[dq]
|
|
.SS Advanced Options
|
|
.PP
|
|
Here are the advanced options specific to webdav (Webdav).
|
|
.SS \-\-webdav\-bearer\-token\-command
|
|
.PP
|
|
Command to run to get a bearer token
|
|
.IP \[bu] 2
|
|
Config: bearer_token_command
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_WEBDAV_BEARER_TOKEN_COMMAND
|
|
.IP \[bu] 2
|
|
Type: string
|
|
.IP \[bu] 2
|
|
Default: \[dq]\[dq]
|
|
.SS \-\-webdav\-encoding
|
|
.PP
|
|
This sets the encoding for the backend.
|
|
.PP
|
|
See: the encoding section in the
|
|
overview (https://rclone.org/overview/#encoding) for more info.
|
|
.PP
|
|
Default encoding is
|
|
Slash,LtGt,DoubleQuote,Colon,Question,Asterisk,Pipe,Hash,Percent,BackSlash,Del,Ctl,LeftSpace,LeftTilde,RightSpace,RightPeriod,InvalidUtf8
|
|
for sharepoint\-ntlm or identity otherwise.
|
|
.IP \[bu] 2
|
|
Config: encoding
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_WEBDAV_ENCODING
|
|
.IP \[bu] 2
|
|
Type: string
|
|
.IP \[bu] 2
|
|
Default: \[dq]\[dq]
|
|
.SS Provider notes
|
|
.PP
|
|
See below for notes on specific providers.
|
|
.SS Owncloud
|
|
.PP
|
|
Click on the settings cog in the bottom right of the page and this will
|
|
show the WebDAV URL that rclone needs in the config step.
|
|
It will look something like
|
|
\f[C]https://example.com/remote.php/webdav/\f[R].
|
|
.PP
|
|
Owncloud supports modified times using the \f[C]X\-OC\-Mtime\f[R]
|
|
header.
|
|
.SS Nextcloud
|
|
.PP
|
|
This is configured in an identical way to Owncloud.
|
|
Note that Nextcloud initially did not support streaming of files
|
|
(\f[C]rcat\f[R]) whereas Owncloud did, but
|
|
this (https://github.com/nextcloud/nextcloud-snap/issues/365) seems to
|
|
be fixed as of 2020\-11\-27 (tested with rclone v1.53.1 and Nextcloud
|
|
Server v19).
|
|
.SS Sharepoint Online
|
|
.PP
|
|
Rclone can be used with Sharepoint provided by OneDrive for Business or
|
|
Office365 Education Accounts.
|
|
This feature is only needed for a few of these Accounts, mostly
|
|
Office365 Education ones.
|
|
These accounts are sometimes not verified by the domain owner
|
|
github#1975 (https://github.com/rclone/rclone/issues/1975)
|
|
.PP
|
|
This means that these accounts can\[aq]t be added using the official API
|
|
(other Accounts should work with the \[dq]onedrive\[dq] option).
|
|
However, it is possible to access them using webdav.
|
|
.PP
|
|
To use a sharepoint remote with rclone, add it like this: First, you
|
|
need to get your remote\[aq]s URL:
|
|
.IP \[bu] 2
|
|
Go here (https://onedrive.live.com/about/en-us/signin/) to open your
|
|
OneDrive or to sign in
|
|
.IP \[bu] 2
|
|
Now take a look at your address bar, the URL should look like this:
|
|
\f[C]https://[YOUR\-DOMAIN]\-my.sharepoint.com/personal/[YOUR\-EMAIL]/_layouts/15/onedrive.aspx\f[R]
|
|
.PP
|
|
You\[aq]ll only need this URL up to the email address.
|
|
After that, you\[aq]ll most likely want to add \[dq]/Documents\[dq].
|
|
That subdirectory contains the actual data stored on your OneDrive.
|
|
.PP
|
|
Add the remote to rclone like this: Configure the \f[C]url\f[R] as
|
|
\f[C]https://[YOUR\-DOMAIN]\-my.sharepoint.com/personal/[YOUR\-EMAIL]/Documents\f[R]
|
|
and use your normal account email and password for \f[C]user\f[R] and
|
|
\f[C]pass\f[R].
|
|
If you have 2FA enabled, you have to generate an app password.
|
|
Set the \f[C]vendor\f[R] to \f[C]sharepoint\f[R].
|
|
.PP
|
|
Your config file should look like this:
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
[sharepoint]
|
|
type = webdav
|
|
url = https://[YOUR\-DOMAIN]\-my.sharepoint.com/personal/[YOUR\-EMAIL]/Documents
|
|
vendor = sharepoint
|
|
user = YourEmailAddress
|
|
pass = encryptedpassword
|
|
\f[R]
|
|
.fi
|
|
.SS Sharepoint with NTLM Authentication
|
|
.PP
|
|
Use this option in case your (hosted) Sharepoint is not tied to OneDrive
|
|
accounts and uses NTLM authentication.
|
|
.PP
|
|
To get the \f[C]url\f[R] configuration, similarly to the above, first
|
|
navigate to the desired directory in your browser to get the URL, then
|
|
strip everything after the name of the opened directory.
|
|
.PP
|
|
Example: If the URL is:
|
|
https://example.sharepoint.com/sites/12345/Documents/Forms/AllItems.aspx
|
|
.PP
|
|
The configuration to use would be:
|
|
https://example.sharepoint.com/sites/12345/Documents
|
|
.PP
|
|
Set the \f[C]vendor\f[R] to \f[C]sharepoint\-ntlm\f[R].
|
|
.PP
|
|
NTLM uses domain and user name combination for authentication, set
|
|
\f[C]user\f[R] to \f[C]DOMAIN\[rs]username\f[R].
|
|
.PP
|
|
Your config file should look like this:
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
[sharepoint]
|
|
type = webdav
|
|
url = https://[YOUR\-DOMAIN]/some\-path\-to/Documents
|
|
vendor = sharepoint\-ntlm
|
|
user = DOMAIN\[rs]user
|
|
pass = encryptedpassword
|
|
\f[R]
|
|
.fi
|
|
.SS Required Flags for SharePoint
|
|
.PP
|
|
As SharePoint does some special things with uploaded documents, you
|
|
won\[aq]t be able to use the documents size or the documents hash to
|
|
compare if a file has been changed since the upload / which file is
|
|
newer.
|
|
.PP
|
|
For Rclone calls copying files (especially Office files such as .docx,
|
|
\&.xlsx, etc.) from/to SharePoint (like copy, sync, etc.), you should
|
|
append these flags to ensure Rclone uses the \[dq]Last Modified\[dq]
|
|
datetime property to compare your documents:
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
\-\-ignore\-size \-\-ignore\-checksum \-\-update
|
|
\f[R]
|
|
.fi
|
|
.SS dCache
|
|
.PP
|
|
dCache is a storage system that supports many protocols and
|
|
authentication/authorisation schemes.
|
|
For WebDAV clients, it allows users to authenticate with username and
|
|
password (BASIC), X.509, Kerberos, and various bearer tokens, including
|
|
Macaroons (https://www.dcache.org/manuals/workshop-2017-05-29-Umea/000-Final/anupam_macaroons_v02.pdf)
|
|
and OpenID\-Connect (https://en.wikipedia.org/wiki/OpenID_Connect)
|
|
access tokens.
|
|
.PP
|
|
Configure as normal using the \f[C]other\f[R] type.
|
|
Don\[aq]t enter a username or password, instead enter your Macaroon as
|
|
the \f[C]bearer_token\f[R].
|
|
.PP
|
|
The config will end up looking something like this.
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
[dcache]
|
|
type = webdav
|
|
url = https://dcache...
|
|
vendor = other
|
|
user =
|
|
pass =
|
|
bearer_token = your\-macaroon
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
There is a
|
|
script (https://github.com/sara-nl/GridScripts/blob/master/get-macaroon)
|
|
that obtains a Macaroon from a dCache WebDAV endpoint, and creates an
|
|
rclone config file.
|
|
.PP
|
|
Macaroons may also be obtained from the dCacheView
|
|
web\-browser/JavaScript client that comes with dCache.
|
|
.SS OpenID\-Connect
|
|
.PP
|
|
dCache also supports authenticating with OpenID\-Connect access tokens.
|
|
OpenID\-Connect is a protocol (based on OAuth 2.0) that allows services
|
|
to identify users who have authenticated with some central service.
|
|
.PP
|
|
Support for OpenID\-Connect in rclone is currently achieved using
|
|
another software package called
|
|
oidc\-agent (https://github.com/indigo-dc/oidc-agent).
|
|
This is a command\-line tool that facilitates obtaining an access token.
|
|
Once installed and configured, an access token is obtained by running
|
|
the \f[C]oidc\-token\f[R] command.
|
|
The following example shows a (shortened) access token obtained from the
|
|
\f[I]XDC\f[R] OIDC Provider.
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
paul\[at]celebrimbor:\[ti]$ oidc\-token XDC
|
|
eyJraWQ[...]QFXDt0
|
|
paul\[at]celebrimbor:\[ti]$
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
\f[B]Note\f[R] Before the \f[C]oidc\-token\f[R] command will work, the
|
|
refresh token must be loaded into the oidc agent.
|
|
This is done with the \f[C]oidc\-add\f[R] command (e.g.,
|
|
\f[C]oidc\-add XDC\f[R]).
|
|
This is typically done once per login session.
|
|
Full details on this and how to register oidc\-agent with your OIDC
|
|
Provider are provided in the oidc\-agent
|
|
documentation (https://indigo-dc.gitbooks.io/oidc-agent/).
|
|
.PP
|
|
The rclone \f[C]bearer_token_command\f[R] configuration option is used
|
|
to fetch the access token from oidc\-agent.
|
|
.PP
|
|
Configure as a normal WebDAV endpoint, using the \[aq]other\[aq] vendor,
|
|
leaving the username and password empty.
|
|
When prompted, choose to edit the advanced config and enter the command
|
|
to get a bearer token (e.g., \f[C]oidc\-agent XDC\f[R]).
|
|
.PP
|
|
The following example config shows a WebDAV endpoint that uses
|
|
oidc\-agent to supply an access token from the \f[I]XDC\f[R] OIDC
|
|
Provider.
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
[dcache]
|
|
type = webdav
|
|
url = https://dcache.example.org/
|
|
vendor = other
|
|
bearer_token_command = oidc\-token XDC
|
|
\f[R]
|
|
.fi
|
|
.SS Yandex Disk
|
|
.PP
|
|
Yandex Disk (https://disk.yandex.com) is a cloud storage solution
|
|
created by Yandex (https://yandex.com).
|
|
.PP
|
|
Here is an example of making a yandex configuration.
|
|
First run
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone config
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
This will guide you through an interactive setup process:
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
No remotes found \- make a new one
|
|
n) New remote
|
|
s) Set configuration password
|
|
n/s> n
|
|
name> remote
|
|
Type of storage to configure.
|
|
Choose a number from below, or type in your own value
|
|
[snip]
|
|
XX / Yandex Disk
|
|
\[rs] \[dq]yandex\[dq]
|
|
[snip]
|
|
Storage> yandex
|
|
Yandex Client Id \- leave blank normally.
|
|
client_id>
|
|
Yandex Client Secret \- leave blank normally.
|
|
client_secret>
|
|
Remote config
|
|
Use auto config?
|
|
* Say Y if not sure
|
|
* Say N if you are working on a remote or headless machine
|
|
y) Yes
|
|
n) No
|
|
y/n> y
|
|
If your browser doesn\[aq]t open automatically go to the following link: http://127.0.0.1:53682/auth
|
|
Log in and authorize rclone for access
|
|
Waiting for code...
|
|
Got code
|
|
\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-
|
|
[remote]
|
|
client_id =
|
|
client_secret =
|
|
token = {\[dq]access_token\[dq]:\[dq]xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx\[dq],\[dq]token_type\[dq]:\[dq]bearer\[dq],\[dq]expiry\[dq]:\[dq]2016\-12\-29T12:27:11.362788025Z\[dq]}
|
|
\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-
|
|
y) Yes this is OK
|
|
e) Edit this remote
|
|
d) Delete this remote
|
|
y/e/d> y
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
See the remote setup docs (https://rclone.org/remote_setup/) for how to
|
|
set it up on a machine with no Internet browser available.
|
|
.PP
|
|
Note that rclone runs a webserver on your local machine to collect the
|
|
token as returned from Yandex Disk.
|
|
This only runs from the moment it opens your browser to the moment you
|
|
get back the verification code.
|
|
This is on \f[C]http://127.0.0.1:53682/\f[R] and this it may require you
|
|
to unblock it temporarily if you are running a host firewall.
|
|
.PP
|
|
Once configured you can then use \f[C]rclone\f[R] like this,
|
|
.PP
|
|
See top level directories
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone lsd remote:
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
Make a new directory
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone mkdir remote:directory
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
List the contents of a directory
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone ls remote:directory
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
Sync \f[C]/home/local/directory\f[R] to the remote path, deleting any
|
|
excess files in the path.
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone sync \-i /home/local/directory remote:directory
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
Yandex paths may be as deep as required, e.g.
|
|
\f[C]remote:directory/subdirectory\f[R].
|
|
.SS Modified time
|
|
.PP
|
|
Modified times are supported and are stored accurate to 1 ns in custom
|
|
metadata called \f[C]rclone_modified\f[R] in RFC3339 with nanoseconds
|
|
format.
|
|
.SS MD5 checksums
|
|
.PP
|
|
MD5 checksums are natively supported by Yandex Disk.
|
|
.SS Emptying Trash
|
|
.PP
|
|
If you wish to empty your trash you can use the
|
|
\f[C]rclone cleanup remote:\f[R] command which will permanently delete
|
|
all your trashed files.
|
|
This command does not take any path arguments.
|
|
.SS Quota information
|
|
.PP
|
|
To view your current quota you can use the
|
|
\f[C]rclone about remote:\f[R] command which will display your usage
|
|
limit (quota) and the current usage.
|
|
.SS Restricted filename characters
|
|
.PP
|
|
The default restricted characters
|
|
set (https://rclone.org/overview/#restricted-characters) are replaced.
|
|
.PP
|
|
Invalid UTF\-8 bytes will also be
|
|
replaced (https://rclone.org/overview/#invalid-utf8), as they can\[aq]t
|
|
be used in JSON strings.
|
|
.SS Limitations
|
|
.PP
|
|
When uploading very large files (bigger than about 5GB) you will need to
|
|
increase the \f[C]\-\-timeout\f[R] parameter.
|
|
This is because Yandex pauses (perhaps to calculate the MD5SUM for the
|
|
entire file) before returning confirmation that the file has been
|
|
uploaded.
|
|
The default handling of timeouts in rclone is to assume a 5 minute pause
|
|
is an error and close the connection \- you\[aq]ll see
|
|
\f[C]net/http: timeout awaiting response headers\f[R] errors in the logs
|
|
if this is happening.
|
|
Setting the timeout to twice the max size of file in GB should be
|
|
enough, so if you want to upload a 30GB file set a timeout of
|
|
\f[C]2 * 30 = 60m\f[R], that is \f[C]\-\-timeout 60m\f[R].
|
|
.SS Standard Options
|
|
.PP
|
|
Here are the standard options specific to yandex (Yandex Disk).
|
|
.SS \-\-yandex\-client\-id
|
|
.PP
|
|
OAuth Client Id Leave blank normally.
|
|
.IP \[bu] 2
|
|
Config: client_id
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_YANDEX_CLIENT_ID
|
|
.IP \[bu] 2
|
|
Type: string
|
|
.IP \[bu] 2
|
|
Default: \[dq]\[dq]
|
|
.SS \-\-yandex\-client\-secret
|
|
.PP
|
|
OAuth Client Secret Leave blank normally.
|
|
.IP \[bu] 2
|
|
Config: client_secret
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_YANDEX_CLIENT_SECRET
|
|
.IP \[bu] 2
|
|
Type: string
|
|
.IP \[bu] 2
|
|
Default: \[dq]\[dq]
|
|
.SS Advanced Options
|
|
.PP
|
|
Here are the advanced options specific to yandex (Yandex Disk).
|
|
.SS \-\-yandex\-token
|
|
.PP
|
|
OAuth Access Token as a JSON blob.
|
|
.IP \[bu] 2
|
|
Config: token
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_YANDEX_TOKEN
|
|
.IP \[bu] 2
|
|
Type: string
|
|
.IP \[bu] 2
|
|
Default: \[dq]\[dq]
|
|
.SS \-\-yandex\-auth\-url
|
|
.PP
|
|
Auth server URL.
|
|
Leave blank to use the provider defaults.
|
|
.IP \[bu] 2
|
|
Config: auth_url
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_YANDEX_AUTH_URL
|
|
.IP \[bu] 2
|
|
Type: string
|
|
.IP \[bu] 2
|
|
Default: \[dq]\[dq]
|
|
.SS \-\-yandex\-token\-url
|
|
.PP
|
|
Token server url.
|
|
Leave blank to use the provider defaults.
|
|
.IP \[bu] 2
|
|
Config: token_url
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_YANDEX_TOKEN_URL
|
|
.IP \[bu] 2
|
|
Type: string
|
|
.IP \[bu] 2
|
|
Default: \[dq]\[dq]
|
|
.SS \-\-yandex\-encoding
|
|
.PP
|
|
This sets the encoding for the backend.
|
|
.PP
|
|
See: the encoding section in the
|
|
overview (https://rclone.org/overview/#encoding) for more info.
|
|
.IP \[bu] 2
|
|
Config: encoding
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_YANDEX_ENCODING
|
|
.IP \[bu] 2
|
|
Type: MultiEncoder
|
|
.IP \[bu] 2
|
|
Default: Slash,Del,Ctl,InvalidUtf8,Dot
|
|
.SS Zoho Workdrive
|
|
.PP
|
|
Zoho WorkDrive (https://www.zoho.com/workdrive/) is a cloud storage
|
|
solution created by Zoho (https://zoho.com).
|
|
.PP
|
|
Here is an example of making a zoho configuration.
|
|
First run
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone config
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
This will guide you through an interactive setup process:
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
No remotes found \- make a new one
|
|
n) New remote
|
|
s) Set configuration password
|
|
n/s> n
|
|
name> remote
|
|
Type of storage to configure.
|
|
Enter a string value. Press Enter for the default (\[dq]\[dq]).
|
|
Choose a number from below, or type in your own value
|
|
[snip]
|
|
XX / Zoho
|
|
\[rs] \[dq]zoho\[dq]
|
|
[snip]
|
|
Storage> zoho
|
|
** See help for zoho backend at: https://rclone.org/zoho/ **
|
|
|
|
OAuth Client Id
|
|
Leave blank normally.
|
|
Enter a string value. Press Enter for the default (\[dq]\[dq]).
|
|
client_id>
|
|
OAuth Client Secret
|
|
Leave blank normally.
|
|
Enter a string value. Press Enter for the default (\[dq]\[dq]).
|
|
client_secret>
|
|
Edit advanced config? (y/n)
|
|
y) Yes
|
|
n) No (default)
|
|
y/n> n
|
|
Remote config
|
|
Use auto config?
|
|
* Say Y if not sure
|
|
* Say N if you are working on a remote or headless machine
|
|
y) Yes (default)
|
|
n) No
|
|
y/n>
|
|
If your browser doesn\[aq]t open automatically go to the following link: http://127.0.0.1:53682/auth?state=LVn0IHzxej1ZkmQw31d0wQ
|
|
Log in and authorize rclone for access
|
|
Waiting for code...
|
|
Got code
|
|
Choose a number from below, or type in your own value
|
|
1 / MyTeam
|
|
\[rs] \[dq]4u28602177065ff22426787a6745dba8954eb\[dq]
|
|
Enter a Team ID> 1
|
|
Choose a number from below, or type in your own value
|
|
1 / General
|
|
\[rs] \[dq]4u2869d2aa6fca04f4f2f896b6539243b85b1\[dq]
|
|
Enter a Workspace ID> 1
|
|
\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-
|
|
[remote]
|
|
type = zoho
|
|
token = {\[dq]access_token\[dq]:\[dq]xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx\[dq],\[dq]token_type\[dq]:\[dq]Zoho\-oauthtoken\[dq],\[dq]refresh_token\[dq]:\[dq]xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx\[dq],\[dq]expiry\[dq]:\[dq]2020\-10\-12T00:54:52.370275223+02:00\[dq]}
|
|
root_folder_id = xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
|
|
\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-
|
|
y) Yes this is OK (default)
|
|
e) Edit this remote
|
|
d) Delete this remote
|
|
y/e/d>
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
See the remote setup docs (https://rclone.org/remote_setup/) for how to
|
|
set it up on a machine with no Internet browser available.
|
|
.PP
|
|
Rclone runs a webserver on your local computer to collect the
|
|
authorization token from Zoho Workdrive.
|
|
This is only from the moment your browser is opened until the token is
|
|
returned.
|
|
The webserver runs on \f[C]http://127.0.0.1:53682/\f[R].
|
|
If local port \f[C]53682\f[R] is protected by a firewall you may need to
|
|
temporarily unblock the firewall to complete authorization.
|
|
.PP
|
|
Once configured you can then use \f[C]rclone\f[R] like this,
|
|
.PP
|
|
See top level directories
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone lsd remote:
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
Make a new directory
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone mkdir remote:directory
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
List the contents of a directory
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone ls remote:directory
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
Sync \f[C]/home/local/directory\f[R] to the remote path, deleting any
|
|
excess files in the path.
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone sync \-i /home/local/directory remote:directory
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
Zoho paths may be as deep as required, eg
|
|
\f[C]remote:directory/subdirectory\f[R].
|
|
.SS Modified time
|
|
.PP
|
|
Modified times are currently not supported for Zoho Workdrive
|
|
.SS Checksums
|
|
.PP
|
|
No checksums are supported.
|
|
.SS Usage information
|
|
.PP
|
|
To view your current quota you can use the
|
|
\f[C]rclone about remote:\f[R] command which will display your current
|
|
usage.
|
|
.SS Restricted filename characters
|
|
.PP
|
|
Only control characters and invalid UTF\-8 are replaced.
|
|
In addition most Unicode full\-width characters are not supported at all
|
|
and will be removed from filenames during upload.
|
|
.SS Standard Options
|
|
.PP
|
|
Here are the standard options specific to zoho (Zoho).
|
|
.SS \-\-zoho\-client\-id
|
|
.PP
|
|
OAuth Client Id Leave blank normally.
|
|
.IP \[bu] 2
|
|
Config: client_id
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_ZOHO_CLIENT_ID
|
|
.IP \[bu] 2
|
|
Type: string
|
|
.IP \[bu] 2
|
|
Default: \[dq]\[dq]
|
|
.SS \-\-zoho\-client\-secret
|
|
.PP
|
|
OAuth Client Secret Leave blank normally.
|
|
.IP \[bu] 2
|
|
Config: client_secret
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_ZOHO_CLIENT_SECRET
|
|
.IP \[bu] 2
|
|
Type: string
|
|
.IP \[bu] 2
|
|
Default: \[dq]\[dq]
|
|
.SS \-\-zoho\-region
|
|
.PP
|
|
Zoho region to connect to.
|
|
You\[aq]ll have to use the region you organization is registered in.
|
|
.IP \[bu] 2
|
|
Config: region
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_ZOHO_REGION
|
|
.IP \[bu] 2
|
|
Type: string
|
|
.IP \[bu] 2
|
|
Default: \[dq]\[dq]
|
|
.IP \[bu] 2
|
|
Examples:
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
\[dq]com\[dq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
United states / Global
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[dq]eu\[dq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Europe
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[dq]in\[dq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
India
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[dq]com.au\[dq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Australia
|
|
.RE
|
|
.RE
|
|
.SS Advanced Options
|
|
.PP
|
|
Here are the advanced options specific to zoho (Zoho).
|
|
.SS \-\-zoho\-token
|
|
.PP
|
|
OAuth Access Token as a JSON blob.
|
|
.IP \[bu] 2
|
|
Config: token
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_ZOHO_TOKEN
|
|
.IP \[bu] 2
|
|
Type: string
|
|
.IP \[bu] 2
|
|
Default: \[dq]\[dq]
|
|
.SS \-\-zoho\-auth\-url
|
|
.PP
|
|
Auth server URL.
|
|
Leave blank to use the provider defaults.
|
|
.IP \[bu] 2
|
|
Config: auth_url
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_ZOHO_AUTH_URL
|
|
.IP \[bu] 2
|
|
Type: string
|
|
.IP \[bu] 2
|
|
Default: \[dq]\[dq]
|
|
.SS \-\-zoho\-token\-url
|
|
.PP
|
|
Token server url.
|
|
Leave blank to use the provider defaults.
|
|
.IP \[bu] 2
|
|
Config: token_url
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_ZOHO_TOKEN_URL
|
|
.IP \[bu] 2
|
|
Type: string
|
|
.IP \[bu] 2
|
|
Default: \[dq]\[dq]
|
|
.SS \-\-zoho\-encoding
|
|
.PP
|
|
This sets the encoding for the backend.
|
|
.PP
|
|
See: the encoding section in the
|
|
overview (https://rclone.org/overview/#encoding) for more info.
|
|
.IP \[bu] 2
|
|
Config: encoding
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_ZOHO_ENCODING
|
|
.IP \[bu] 2
|
|
Type: MultiEncoder
|
|
.IP \[bu] 2
|
|
Default: Del,Ctl,InvalidUtf8
|
|
.SS Local Filesystem
|
|
.PP
|
|
Local paths are specified as normal filesystem paths, e.g.
|
|
\f[C]/path/to/wherever\f[R], so
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone sync \-i /home/source /tmp/destination
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
Will sync \f[C]/home/source\f[R] to \f[C]/tmp/destination\f[R].
|
|
.PP
|
|
For consistencies sake one can also configure a remote of type
|
|
\f[C]local\f[R] in the config file, and access the local filesystem
|
|
using rclone remote paths, e.g.
|
|
\f[C]remote:path/to/wherever\f[R], but it is probably easier not to.
|
|
.SS Modified time
|
|
.PP
|
|
Rclone reads and writes the modified time using an accuracy determined
|
|
by the OS.
|
|
Typically this is 1ns on Linux, 10 ns on Windows and 1 Second on OS X.
|
|
.SS Filenames
|
|
.PP
|
|
Filenames should be encoded in UTF\-8 on disk.
|
|
This is the normal case for Windows and OS X.
|
|
.PP
|
|
There is a bit more uncertainty in the Linux world, but new
|
|
distributions will have UTF\-8 encoded files names.
|
|
If you are using an old Linux filesystem with non UTF\-8 file names
|
|
(e.g.
|
|
latin1) then you can use the \f[C]convmv\f[R] tool to convert the
|
|
filesystem to UTF\-8.
|
|
This tool is available in most distributions\[aq] package managers.
|
|
.PP
|
|
If an invalid (non\-UTF8) filename is read, the invalid characters will
|
|
be replaced with a quoted representation of the invalid bytes.
|
|
The name \f[C]gro\[rs]xdf\f[R] will be transferred as
|
|
\f[C]gro\[u201B]DF\f[R].
|
|
\f[C]rclone\f[R] will emit a debug message in this case (use
|
|
\f[C]\-v\f[R] to see), e.g.
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
Local file system at .: Replacing invalid UTF\-8 characters in \[dq]gro\[rs]xdf\[dq]
|
|
\f[R]
|
|
.fi
|
|
.SS Restricted characters
|
|
.PP
|
|
On non Windows platforms the following characters are replaced when
|
|
handling file names.
|
|
.PP
|
|
.TS
|
|
tab(@);
|
|
l c c.
|
|
T{
|
|
Character
|
|
T}@T{
|
|
Value
|
|
T}@T{
|
|
Replacement
|
|
T}
|
|
_
|
|
T{
|
|
NUL
|
|
T}@T{
|
|
0x00
|
|
T}@T{
|
|
\[u2400]
|
|
T}
|
|
T{
|
|
/
|
|
T}@T{
|
|
0x2F
|
|
T}@T{
|
|
\[uFF0F]
|
|
T}
|
|
.TE
|
|
.PP
|
|
When running on Windows the following characters are replaced.
|
|
This list is based on the Windows file naming
|
|
conventions (https://docs.microsoft.com/de-de/windows/desktop/FileIO/naming-a-file#naming-conventions).
|
|
.PP
|
|
.TS
|
|
tab(@);
|
|
l c c.
|
|
T{
|
|
Character
|
|
T}@T{
|
|
Value
|
|
T}@T{
|
|
Replacement
|
|
T}
|
|
_
|
|
T{
|
|
NUL
|
|
T}@T{
|
|
0x00
|
|
T}@T{
|
|
\[u2400]
|
|
T}
|
|
T{
|
|
SOH
|
|
T}@T{
|
|
0x01
|
|
T}@T{
|
|
\[u2401]
|
|
T}
|
|
T{
|
|
STX
|
|
T}@T{
|
|
0x02
|
|
T}@T{
|
|
\[u2402]
|
|
T}
|
|
T{
|
|
ETX
|
|
T}@T{
|
|
0x03
|
|
T}@T{
|
|
\[u2403]
|
|
T}
|
|
T{
|
|
EOT
|
|
T}@T{
|
|
0x04
|
|
T}@T{
|
|
\[u2404]
|
|
T}
|
|
T{
|
|
ENQ
|
|
T}@T{
|
|
0x05
|
|
T}@T{
|
|
\[u2405]
|
|
T}
|
|
T{
|
|
ACK
|
|
T}@T{
|
|
0x06
|
|
T}@T{
|
|
\[u2406]
|
|
T}
|
|
T{
|
|
BEL
|
|
T}@T{
|
|
0x07
|
|
T}@T{
|
|
\[u2407]
|
|
T}
|
|
T{
|
|
BS
|
|
T}@T{
|
|
0x08
|
|
T}@T{
|
|
\[u2408]
|
|
T}
|
|
T{
|
|
HT
|
|
T}@T{
|
|
0x09
|
|
T}@T{
|
|
\[u2409]
|
|
T}
|
|
T{
|
|
LF
|
|
T}@T{
|
|
0x0A
|
|
T}@T{
|
|
\[u240A]
|
|
T}
|
|
T{
|
|
VT
|
|
T}@T{
|
|
0x0B
|
|
T}@T{
|
|
\[u240B]
|
|
T}
|
|
T{
|
|
FF
|
|
T}@T{
|
|
0x0C
|
|
T}@T{
|
|
\[u240C]
|
|
T}
|
|
T{
|
|
CR
|
|
T}@T{
|
|
0x0D
|
|
T}@T{
|
|
\[u240D]
|
|
T}
|
|
T{
|
|
SO
|
|
T}@T{
|
|
0x0E
|
|
T}@T{
|
|
\[u240E]
|
|
T}
|
|
T{
|
|
SI
|
|
T}@T{
|
|
0x0F
|
|
T}@T{
|
|
\[u240F]
|
|
T}
|
|
T{
|
|
DLE
|
|
T}@T{
|
|
0x10
|
|
T}@T{
|
|
\[u2410]
|
|
T}
|
|
T{
|
|
DC1
|
|
T}@T{
|
|
0x11
|
|
T}@T{
|
|
\[u2411]
|
|
T}
|
|
T{
|
|
DC2
|
|
T}@T{
|
|
0x12
|
|
T}@T{
|
|
\[u2412]
|
|
T}
|
|
T{
|
|
DC3
|
|
T}@T{
|
|
0x13
|
|
T}@T{
|
|
\[u2413]
|
|
T}
|
|
T{
|
|
DC4
|
|
T}@T{
|
|
0x14
|
|
T}@T{
|
|
\[u2414]
|
|
T}
|
|
T{
|
|
NAK
|
|
T}@T{
|
|
0x15
|
|
T}@T{
|
|
\[u2415]
|
|
T}
|
|
T{
|
|
SYN
|
|
T}@T{
|
|
0x16
|
|
T}@T{
|
|
\[u2416]
|
|
T}
|
|
T{
|
|
ETB
|
|
T}@T{
|
|
0x17
|
|
T}@T{
|
|
\[u2417]
|
|
T}
|
|
T{
|
|
CAN
|
|
T}@T{
|
|
0x18
|
|
T}@T{
|
|
\[u2418]
|
|
T}
|
|
T{
|
|
EM
|
|
T}@T{
|
|
0x19
|
|
T}@T{
|
|
\[u2419]
|
|
T}
|
|
T{
|
|
SUB
|
|
T}@T{
|
|
0x1A
|
|
T}@T{
|
|
\[u241A]
|
|
T}
|
|
T{
|
|
ESC
|
|
T}@T{
|
|
0x1B
|
|
T}@T{
|
|
\[u241B]
|
|
T}
|
|
T{
|
|
FS
|
|
T}@T{
|
|
0x1C
|
|
T}@T{
|
|
\[u241C]
|
|
T}
|
|
T{
|
|
GS
|
|
T}@T{
|
|
0x1D
|
|
T}@T{
|
|
\[u241D]
|
|
T}
|
|
T{
|
|
RS
|
|
T}@T{
|
|
0x1E
|
|
T}@T{
|
|
\[u241E]
|
|
T}
|
|
T{
|
|
US
|
|
T}@T{
|
|
0x1F
|
|
T}@T{
|
|
\[u241F]
|
|
T}
|
|
T{
|
|
/
|
|
T}@T{
|
|
0x2F
|
|
T}@T{
|
|
\[uFF0F]
|
|
T}
|
|
T{
|
|
\[dq]
|
|
T}@T{
|
|
0x22
|
|
T}@T{
|
|
\[uFF02]
|
|
T}
|
|
T{
|
|
*
|
|
T}@T{
|
|
0x2A
|
|
T}@T{
|
|
\[uFF0A]
|
|
T}
|
|
T{
|
|
:
|
|
T}@T{
|
|
0x3A
|
|
T}@T{
|
|
\[uFF1A]
|
|
T}
|
|
T{
|
|
<
|
|
T}@T{
|
|
0x3C
|
|
T}@T{
|
|
\[uFF1C]
|
|
T}
|
|
T{
|
|
>
|
|
T}@T{
|
|
0x3E
|
|
T}@T{
|
|
\[uFF1E]
|
|
T}
|
|
T{
|
|
?
|
|
T}@T{
|
|
0x3F
|
|
T}@T{
|
|
\[uFF1F]
|
|
T}
|
|
T{
|
|
\[rs]
|
|
T}@T{
|
|
0x5C
|
|
T}@T{
|
|
\[uFF3C]
|
|
T}
|
|
T{
|
|
|
|
|
T}@T{
|
|
0x7C
|
|
T}@T{
|
|
\[uFF5C]
|
|
T}
|
|
.TE
|
|
.PP
|
|
File names on Windows can also not end with the following characters.
|
|
These only get replaced if they are the last character in the name:
|
|
.PP
|
|
.TS
|
|
tab(@);
|
|
l c c.
|
|
T{
|
|
Character
|
|
T}@T{
|
|
Value
|
|
T}@T{
|
|
Replacement
|
|
T}
|
|
_
|
|
T{
|
|
SP
|
|
T}@T{
|
|
0x20
|
|
T}@T{
|
|
\[u2420]
|
|
T}
|
|
T{
|
|
\&.
|
|
T}@T{
|
|
0x2E
|
|
T}@T{
|
|
\[uFF0E]
|
|
T}
|
|
.TE
|
|
.PP
|
|
Invalid UTF\-8 bytes will also be
|
|
replaced (https://rclone.org/overview/#invalid-utf8), as they can\[aq]t
|
|
be converted to UTF\-16.
|
|
.SS Paths on Windows
|
|
.PP
|
|
On Windows there are many ways of specifying a path to a file system
|
|
resource.
|
|
Both absolute paths like \f[C]C:\[rs]path\[rs]to\[rs]wherever\f[R], and
|
|
relative paths like \f[C]..\[rs]wherever\f[R] can be used, and path
|
|
separator can be either \f[C]\[rs]\f[R] (as in
|
|
\f[C]C:\[rs]path\[rs]to\[rs]wherever\f[R]) or \f[C]/\f[R] (as in
|
|
\f[C]C:/path/to/wherever\f[R]).
|
|
Length of these paths are limited to 259 characters for files and 247
|
|
characters for directories, but there is an alternative extended\-length
|
|
path format increasing the limit to (approximately) 32,767 characters.
|
|
This format requires absolute paths and the use of prefix
|
|
\f[C]\[rs]\[rs]?\[rs]\f[R], e.g.
|
|
\f[C]\[rs]\[rs]?\[rs]D:\[rs]some\[rs]very\[rs]long\[rs]path\f[R].
|
|
For convenience rclone will automatically convert regular paths into the
|
|
corresponding extended\-length paths, so in most cases you do not have
|
|
to worry about this (read more below).
|
|
.PP
|
|
Note that Windows supports using the same prefix
|
|
\f[C]\[rs]\[rs]?\[rs]\f[R] to specify path to volumes identified by
|
|
their GUID, e.g.
|
|
\f[C]\[rs]\[rs]?\[rs]Volume{b75e2c83\-0000\-0000\-0000\-602f00000000}\[rs]some\[rs]path\f[R].
|
|
This is \f[I]not\f[R] supported in rclone, due to an
|
|
issue (https://github.com/golang/go/issues/39785) in go.
|
|
.SS Long paths
|
|
.PP
|
|
Rclone handles long paths automatically, by converting all paths to
|
|
extended\-length path
|
|
format (https://docs.microsoft.com/en-us/windows/win32/fileio/maximum-file-path-limitation),
|
|
which allows paths up to 32,767 characters.
|
|
.PP
|
|
This conversion will ensure paths are absolute and prefix them with the
|
|
\f[C]\[rs]\[rs]?\[rs]\f[R].
|
|
This is why you will see that your paths, for instance
|
|
\f[C].\[rs]files\f[R] is shown as path
|
|
\f[C]\[rs]\[rs]?\[rs]C:\[rs]files\f[R] in the output, and
|
|
\f[C]\[rs]\[rs]server\[rs]share\f[R] as
|
|
\f[C]\[rs]\[rs]?\[rs]UNC\[rs]server\[rs]share\f[R].
|
|
.PP
|
|
However, in rare cases this may cause problems with buggy file system
|
|
drivers like EncFS (https://github.com/rclone/rclone/issues/261).
|
|
To disable UNC conversion globally, add this to your
|
|
\f[C].rclone.conf\f[R] file:
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
[local]
|
|
nounc = true
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
If you want to selectively disable UNC, you can add it to a separate
|
|
entry like this:
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
[nounc]
|
|
type = local
|
|
nounc = true
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
And use rclone like this:
|
|
.PP
|
|
\f[C]rclone copy c:\[rs]src nounc:z:\[rs]dst\f[R]
|
|
.PP
|
|
This will use UNC paths on \f[C]c:\[rs]src\f[R] but not on
|
|
\f[C]z:\[rs]dst\f[R].
|
|
Of course this will cause problems if the absolute path length of a file
|
|
exceeds 259 characters on z, so only use this option if you have to.
|
|
.SS Symlinks / Junction points
|
|
.PP
|
|
Normally rclone will ignore symlinks or junction points (which behave
|
|
like symlinks under Windows).
|
|
.PP
|
|
If you supply \f[C]\-\-copy\-links\f[R] or \f[C]\-L\f[R] then rclone
|
|
will follow the symlink and copy the pointed to file or directory.
|
|
Note that this flag is incompatible with \f[C]\-links\f[R] /
|
|
\f[C]\-l\f[R].
|
|
.PP
|
|
This flag applies to all commands.
|
|
.PP
|
|
For example, supposing you have a directory structure like this
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
$ tree /tmp/a
|
|
/tmp/a
|
|
\[u251C]\[u2500]\[u2500] b \-> ../b
|
|
\[u251C]\[u2500]\[u2500] expected \-> ../expected
|
|
\[u251C]\[u2500]\[u2500] one
|
|
\[u2514]\[u2500]\[u2500] two
|
|
\[u2514]\[u2500]\[u2500] three
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
Then you can see the difference with and without the flag like this
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
$ rclone ls /tmp/a
|
|
6 one
|
|
6 two/three
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
and
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
$ rclone \-L ls /tmp/a
|
|
4174 expected
|
|
6 one
|
|
6 two/three
|
|
6 b/two
|
|
6 b/one
|
|
\f[R]
|
|
.fi
|
|
.SS \-\-links, \-l
|
|
.PP
|
|
Normally rclone will ignore symlinks or junction points (which behave
|
|
like symlinks under Windows).
|
|
.PP
|
|
If you supply this flag then rclone will copy symbolic links from the
|
|
local storage, and store them as text files, with a
|
|
\[aq].rclonelink\[aq] suffix in the remote storage.
|
|
.PP
|
|
The text file will contain the target of the symbolic link (see
|
|
example).
|
|
.PP
|
|
This flag applies to all commands.
|
|
.PP
|
|
For example, supposing you have a directory structure like this
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
$ tree /tmp/a
|
|
/tmp/a
|
|
\[u251C]\[u2500]\[u2500] file1 \-> ./file4
|
|
\[u2514]\[u2500]\[u2500] file2 \-> /home/user/file3
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
Copying the entire directory with \[aq]\-l\[aq]
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
$ rclone copyto \-l /tmp/a/file1 remote:/tmp/a/
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
The remote files are created with a \[aq].rclonelink\[aq] suffix
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
$ rclone ls remote:/tmp/a
|
|
5 file1.rclonelink
|
|
14 file2.rclonelink
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
The remote files will contain the target of the symbolic links
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
$ rclone cat remote:/tmp/a/file1.rclonelink
|
|
\&./file4
|
|
|
|
$ rclone cat remote:/tmp/a/file2.rclonelink
|
|
/home/user/file3
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
Copying them back with \[aq]\-l\[aq]
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
$ rclone copyto \-l remote:/tmp/a/ /tmp/b/
|
|
|
|
$ tree /tmp/b
|
|
/tmp/b
|
|
\[u251C]\[u2500]\[u2500] file1 \-> ./file4
|
|
\[u2514]\[u2500]\[u2500] file2 \-> /home/user/file3
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
However, if copied back without \[aq]\-l\[aq]
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
$ rclone copyto remote:/tmp/a/ /tmp/b/
|
|
|
|
$ tree /tmp/b
|
|
/tmp/b
|
|
\[u251C]\[u2500]\[u2500] file1.rclonelink
|
|
\[u2514]\[u2500]\[u2500] file2.rclonelink
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
Note that this flag is incompatible with \f[C]\-copy\-links\f[R] /
|
|
\f[C]\-L\f[R].
|
|
.SS Restricting filesystems with \-\-one\-file\-system
|
|
.PP
|
|
Normally rclone will recurse through filesystems as mounted.
|
|
.PP
|
|
However if you set \f[C]\-\-one\-file\-system\f[R] or \f[C]\-x\f[R] this
|
|
tells rclone to stay in the filesystem specified by the root and not to
|
|
recurse into different file systems.
|
|
.PP
|
|
For example if you have a directory hierarchy like this
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
root
|
|
\[u251C]\[u2500]\[u2500] disk1 \- disk1 mounted on the root
|
|
\[br]\ \ \[u2514]\[u2500]\[u2500] file3 \- stored on disk1
|
|
\[u251C]\[u2500]\[u2500] disk2 \- disk2 mounted on the root
|
|
\[br]\ \ \[u2514]\[u2500]\[u2500] file4 \- stored on disk12
|
|
\[u251C]\[u2500]\[u2500] file1 \- stored on the root disk
|
|
\[u2514]\[u2500]\[u2500] file2 \- stored on the root disk
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
Using \f[C]rclone \-\-one\-file\-system copy root remote:\f[R] will only
|
|
copy \f[C]file1\f[R] and \f[C]file2\f[R].
|
|
Eg
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
$ rclone \-q \-\-one\-file\-system ls root
|
|
0 file1
|
|
0 file2
|
|
\f[R]
|
|
.fi
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
$ rclone \-q ls root
|
|
0 disk1/file3
|
|
0 disk2/file4
|
|
0 file1
|
|
0 file2
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
\f[B]NB\f[R] Rclone (like most unix tools such as \f[C]du\f[R],
|
|
\f[C]rsync\f[R] and \f[C]tar\f[R]) treats a bind mount to the same
|
|
device as being on the same filesystem.
|
|
.PP
|
|
\f[B]NB\f[R] This flag is only available on Unix based systems.
|
|
On systems where it isn\[aq]t supported (e.g.
|
|
Windows) it will be ignored.
|
|
.SS Advanced Options
|
|
.PP
|
|
Here are the advanced options specific to local (Local Disk).
|
|
.SS \-\-local\-nounc
|
|
.PP
|
|
Disable UNC (long path names) conversion on Windows
|
|
.IP \[bu] 2
|
|
Config: nounc
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_LOCAL_NOUNC
|
|
.IP \[bu] 2
|
|
Type: string
|
|
.IP \[bu] 2
|
|
Default: \[dq]\[dq]
|
|
.IP \[bu] 2
|
|
Examples:
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
\[dq]true\[dq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Disables long file names
|
|
.RE
|
|
.RE
|
|
.SS \-\-copy\-links / \-L
|
|
.PP
|
|
Follow symlinks and copy the pointed to item.
|
|
.IP \[bu] 2
|
|
Config: copy_links
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_LOCAL_COPY_LINKS
|
|
.IP \[bu] 2
|
|
Type: bool
|
|
.IP \[bu] 2
|
|
Default: false
|
|
.SS \-\-links / \-l
|
|
.PP
|
|
Translate symlinks to/from regular files with a \[aq].rclonelink\[aq]
|
|
extension
|
|
.IP \[bu] 2
|
|
Config: links
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_LOCAL_LINKS
|
|
.IP \[bu] 2
|
|
Type: bool
|
|
.IP \[bu] 2
|
|
Default: false
|
|
.SS \-\-skip\-links
|
|
.PP
|
|
Don\[aq]t warn about skipped symlinks.
|
|
This flag disables warning messages on skipped symlinks or junction
|
|
points, as you explicitly acknowledge that they should be skipped.
|
|
.IP \[bu] 2
|
|
Config: skip_links
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_LOCAL_SKIP_LINKS
|
|
.IP \[bu] 2
|
|
Type: bool
|
|
.IP \[bu] 2
|
|
Default: false
|
|
.SS \-\-local\-zero\-size\-links
|
|
.PP
|
|
Assume the Stat size of links is zero (and read them instead)
|
|
.PP
|
|
On some virtual filesystems (such ash LucidLink), reading a link size
|
|
via a Stat call always returns 0.
|
|
However, on unix it reads as the length of the text in the link.
|
|
This may cause errors like this when syncing:
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
Failed to copy: corrupted on transfer: sizes differ 0 vs 13
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
Setting this flag causes rclone to read the link and use that as the
|
|
size of the link instead of 0 which in most cases fixes the problem.
|
|
.IP \[bu] 2
|
|
Config: zero_size_links
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_LOCAL_ZERO_SIZE_LINKS
|
|
.IP \[bu] 2
|
|
Type: bool
|
|
.IP \[bu] 2
|
|
Default: false
|
|
.SS \-\-local\-no\-unicode\-normalization
|
|
.PP
|
|
Don\[aq]t apply unicode normalization to paths and filenames
|
|
(Deprecated)
|
|
.PP
|
|
This flag is deprecated now.
|
|
Rclone no longer normalizes unicode file names, but it compares them
|
|
with unicode normalization in the sync routine instead.
|
|
.IP \[bu] 2
|
|
Config: no_unicode_normalization
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_LOCAL_NO_UNICODE_NORMALIZATION
|
|
.IP \[bu] 2
|
|
Type: bool
|
|
.IP \[bu] 2
|
|
Default: false
|
|
.SS \-\-local\-no\-check\-updated
|
|
.PP
|
|
Don\[aq]t check to see if the files change during upload
|
|
.PP
|
|
Normally rclone checks the size and modification time of files as they
|
|
are being uploaded and aborts with a message which starts \[dq]can\[aq]t
|
|
copy \- source file is being updated\[dq] if the file changes during
|
|
upload.
|
|
.PP
|
|
However on some file systems this modification time check may fail (e.g.
|
|
Glusterfs #2206 (https://github.com/rclone/rclone/issues/2206)) so this
|
|
check can be disabled with this flag.
|
|
.PP
|
|
If this flag is set, rclone will use its best efforts to transfer a file
|
|
which is being updated.
|
|
If the file is only having things appended to it (e.g.
|
|
a log) then rclone will transfer the log file with the size it had the
|
|
first time rclone saw it.
|
|
.PP
|
|
If the file is being modified throughout (not just appended to) then the
|
|
transfer may fail with a hash check failure.
|
|
.PP
|
|
In detail, once the file has had stat() called on it for the first time
|
|
we:
|
|
.IP \[bu] 2
|
|
Only transfer the size that stat gave
|
|
.IP \[bu] 2
|
|
Only checksum the size that stat gave
|
|
.IP \[bu] 2
|
|
Don\[aq]t update the stat info for the file
|
|
.IP \[bu] 2
|
|
Config: no_check_updated
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_LOCAL_NO_CHECK_UPDATED
|
|
.IP \[bu] 2
|
|
Type: bool
|
|
.IP \[bu] 2
|
|
Default: false
|
|
.SS \-\-one\-file\-system / \-x
|
|
.PP
|
|
Don\[aq]t cross filesystem boundaries (unix/macOS only).
|
|
.IP \[bu] 2
|
|
Config: one_file_system
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_LOCAL_ONE_FILE_SYSTEM
|
|
.IP \[bu] 2
|
|
Type: bool
|
|
.IP \[bu] 2
|
|
Default: false
|
|
.SS \-\-local\-case\-sensitive
|
|
.PP
|
|
Force the filesystem to report itself as case sensitive.
|
|
.PP
|
|
Normally the local backend declares itself as case insensitive on
|
|
Windows/macOS and case sensitive for everything else.
|
|
Use this flag to override the default choice.
|
|
.IP \[bu] 2
|
|
Config: case_sensitive
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_LOCAL_CASE_SENSITIVE
|
|
.IP \[bu] 2
|
|
Type: bool
|
|
.IP \[bu] 2
|
|
Default: false
|
|
.SS \-\-local\-case\-insensitive
|
|
.PP
|
|
Force the filesystem to report itself as case insensitive
|
|
.PP
|
|
Normally the local backend declares itself as case insensitive on
|
|
Windows/macOS and case sensitive for everything else.
|
|
Use this flag to override the default choice.
|
|
.IP \[bu] 2
|
|
Config: case_insensitive
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_LOCAL_CASE_INSENSITIVE
|
|
.IP \[bu] 2
|
|
Type: bool
|
|
.IP \[bu] 2
|
|
Default: false
|
|
.SS \-\-local\-no\-preallocate
|
|
.PP
|
|
Disable preallocation of disk space for transferred files
|
|
.PP
|
|
Preallocation of disk space helps prevent filesystem fragmentation.
|
|
However, some virtual filesystem layers (such as Google Drive File
|
|
Stream) may incorrectly set the actual file size equal to the
|
|
preallocated space, causing checksum and file size checks to fail.
|
|
Use this flag to disable preallocation.
|
|
.IP \[bu] 2
|
|
Config: no_preallocate
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_LOCAL_NO_PREALLOCATE
|
|
.IP \[bu] 2
|
|
Type: bool
|
|
.IP \[bu] 2
|
|
Default: false
|
|
.SS \-\-local\-no\-sparse
|
|
.PP
|
|
Disable sparse files for multi\-thread downloads
|
|
.PP
|
|
On Windows platforms rclone will make sparse files when doing
|
|
multi\-thread downloads.
|
|
This avoids long pauses on large files where the OS zeros the file.
|
|
However sparse files may be undesirable as they cause disk fragmentation
|
|
and can be slow to work with.
|
|
.IP \[bu] 2
|
|
Config: no_sparse
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_LOCAL_NO_SPARSE
|
|
.IP \[bu] 2
|
|
Type: bool
|
|
.IP \[bu] 2
|
|
Default: false
|
|
.SS \-\-local\-no\-set\-modtime
|
|
.PP
|
|
Disable setting modtime
|
|
.PP
|
|
Normally rclone updates modification time of files after they are done
|
|
uploading.
|
|
This can cause permissions issues on Linux platforms when the user
|
|
rclone is running as does not own the file uploaded, such as when
|
|
copying to a CIFS mount owned by another user.
|
|
If this option is enabled, rclone will no longer update the modtime
|
|
after copying a file.
|
|
.IP \[bu] 2
|
|
Config: no_set_modtime
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_LOCAL_NO_SET_MODTIME
|
|
.IP \[bu] 2
|
|
Type: bool
|
|
.IP \[bu] 2
|
|
Default: false
|
|
.SS \-\-local\-encoding
|
|
.PP
|
|
This sets the encoding for the backend.
|
|
.PP
|
|
See: the encoding section in the
|
|
overview (https://rclone.org/overview/#encoding) for more info.
|
|
.IP \[bu] 2
|
|
Config: encoding
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_LOCAL_ENCODING
|
|
.IP \[bu] 2
|
|
Type: MultiEncoder
|
|
.IP \[bu] 2
|
|
Default: Slash,Dot
|
|
.SS Backend commands
|
|
.PP
|
|
Here are the commands specific to the local backend.
|
|
.PP
|
|
Run them with
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone backend COMMAND remote:
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
The help below will explain what arguments each command takes.
|
|
.PP
|
|
See the \[dq]rclone backend\[dq]
|
|
command (https://rclone.org/commands/rclone_backend/) for more info on
|
|
how to pass options and arguments.
|
|
.PP
|
|
These can be run on a running backend using the rc command
|
|
backend/command (https://rclone.org/rc/#backend/command).
|
|
.SS noop
|
|
.PP
|
|
A null operation for testing backend commands
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone backend noop remote: [options] [<arguments>+]
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
This is a test command which has some options you can try to change the
|
|
output.
|
|
.PP
|
|
Options:
|
|
.IP \[bu] 2
|
|
\[dq]echo\[dq]: echo the input arguments
|
|
.IP \[bu] 2
|
|
\[dq]error\[dq]: return an error based on option value
|
|
.SH Changelog
|
|
.SS v1.55.0 \- 2021\-03\-31
|
|
.PP
|
|
See commits (https://github.com/rclone/rclone/compare/v1.54.0...v1.55.0)
|
|
.IP \[bu] 2
|
|
New commands
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
selfupdate (https://rclone.org/commands/rclone_selfupdate/) (Ivan
|
|
Andreev)
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Allows rclone to update itself in\-place or via a package (using
|
|
\f[C]\-\-package\f[R] flag)
|
|
.IP \[bu] 2
|
|
Reads cryptographically signed signatures for non beta releases
|
|
.IP \[bu] 2
|
|
Works on all OSes.
|
|
.RE
|
|
.IP \[bu] 2
|
|
test (https://rclone.org/commands/rclone_test/) \- these are test
|
|
commands \- use with care!
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
\f[C]histogram\f[R] \- Makes a histogram of file name characters.
|
|
.IP \[bu] 2
|
|
\f[C]info\f[R] \- Discovers file name or other limitations for paths.
|
|
.IP \[bu] 2
|
|
\f[C]makefiles\f[R] \- Make a random file hierarchy for testing.
|
|
.IP \[bu] 2
|
|
\f[C]memory\f[R] \- Load all the objects at remote:path into memory and
|
|
report memory stats.
|
|
.RE
|
|
.RE
|
|
.IP \[bu] 2
|
|
New Features
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Connection strings (https://rclone.org/docs/#connection-strings)
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Config parameters can now be passed as part of the remote name as a
|
|
connection string.
|
|
.IP \[bu] 2
|
|
For example to do the equivalent of
|
|
\f[C]\-\-drive\-shared\-with\-me\f[R] use
|
|
\f[C]drive,shared_with_me:\f[R]
|
|
.IP \[bu] 2
|
|
Make sure we don\[aq]t save on the fly remote config to the config file
|
|
(Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Make sure backends with additional config have a different name for
|
|
caching (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
This work was sponsored by CERN, through the CS3MESH4EOSC
|
|
Project (https://cs3mesh4eosc.eu/).
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
CS3MESH4EOSC has received funding from the European Union\[cq]s Horizon
|
|
2020
|
|
.IP \[bu] 2
|
|
research and innovation programme under Grant Agreement no.
|
|
863353.
|
|
.RE
|
|
.RE
|
|
.IP \[bu] 2
|
|
build
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Update go build version to go1.16 and raise minimum go version to go1.13
|
|
(Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Make a macOS ARM64 build to support Apple Silicon (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Install macfuse 4.x instead of osxfuse 3.x (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Use \f[C]GO386=softfloat\f[R] instead of deprecated \f[C]GO386=387\f[R]
|
|
for 386 builds (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Disable IOS builds for the time being (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Androids builds made with up to date NDK (x0b)
|
|
.IP \[bu] 2
|
|
Add an rclone user to the Docker image but don\[aq]t use it by default
|
|
(cynthia kwok)
|
|
.RE
|
|
.IP \[bu] 2
|
|
dedupe: Make largest directory primary to minimize data moved (Saksham
|
|
Khanna)
|
|
.IP \[bu] 2
|
|
config
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Wrap config library in an interface (Fionera)
|
|
.IP \[bu] 2
|
|
Make config file system pluggable (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
\f[C]\-\-config \[dq]\[dq]\f[R] or \f[C]\[dq]/notfound\[dq]\f[R] for in
|
|
memory config only (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Clear fs cache of stale entries when altering config (Nick Craig\-Wood)
|
|
.RE
|
|
.IP \[bu] 2
|
|
copyurl: Add option to print resulting auto\-filename (albertony)
|
|
.IP \[bu] 2
|
|
delete: Make \f[C]\-\-rmdirs\f[R] obey the filters (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
docs \- many fixes and reworks from edwardxml, albertony, pvalls, Ivan
|
|
Andreev, Evan Harris, buengese, Alexey Tabakman
|
|
.IP \[bu] 2
|
|
encoder/filename \- add SCSU as tables (Klaus Post)
|
|
.IP \[bu] 2
|
|
Add multiple paths support to \f[C]\-\-compare\-dest\f[R] and
|
|
\f[C]\-\-copy\-dest\f[R] flag (K265)
|
|
.IP \[bu] 2
|
|
filter: Make \f[C]\-\-exclude \[dq]dir/\[dq]\f[R] equivalent to
|
|
\f[C]\-\-exclude \[dq]dir/**\[dq]\f[R] (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
fshttp: Add DSCP support with \f[C]\-\-dscp\f[R] for QoS with
|
|
differentiated services (Max Sum)
|
|
.IP \[bu] 2
|
|
lib/cache: Add Delete and DeletePrefix methods (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
lib/file
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Make pre\-allocate detect disk full errors and return them (Nick
|
|
Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Don\[aq]t run preallocate concurrently (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Retry preallocate on EINTR (Nick Craig\-Wood)
|
|
.RE
|
|
.IP \[bu] 2
|
|
operations: Made copy and sync operations obey a RetryAfterError (Ankur
|
|
Gupta)
|
|
.IP \[bu] 2
|
|
rc
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Add string alternatives for setting options over the rc (Nick
|
|
Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Add \f[C]options/local\f[R] to see the options configured in the context
|
|
(Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Add \f[C]_config\f[R] parameter to set global config for just this rc
|
|
call (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Implement passing filter config with \f[C]_filter\f[R] parameter (Nick
|
|
Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Add \f[C]fscache/clear\f[R] and \f[C]fscache/entries\f[R] to control the
|
|
fs cache (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Avoid +Inf value for speed in \f[C]core/stats\f[R] (albertony)
|
|
.IP \[bu] 2
|
|
Add a full set of stats to \f[C]core/stats\f[R] (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Allow \f[C]fs=\f[R] params to be a JSON blob (Nick Craig\-Wood)
|
|
.RE
|
|
.IP \[bu] 2
|
|
rcd: Added systemd notification during the \f[C]rclone rcd\f[R] command.
|
|
(Naveen Honest Raj)
|
|
.IP \[bu] 2
|
|
rmdirs: Make \f[C]\-\-rmdirs\f[R] obey the filters (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
version: Show build tags and type of executable (Ivan Andreev)
|
|
.RE
|
|
.IP \[bu] 2
|
|
Bug Fixes
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
install.sh: make it fail on download errors (Ivan Andreev)
|
|
.IP \[bu] 2
|
|
Fix excessive retries missing \f[C]\-\-max\-duration\f[R] timeout (Nick
|
|
Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Fix crash when \f[C]\-\-low\-level\-retries=0\f[R] (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Fix failed token refresh on mounts created via the rc (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
fshttp: Fix bandwidth limiting after bad merge (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
lib/atexit
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Unregister interrupt handler once it has fired so users can interrupt
|
|
again (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Fix occasional failure to unmount with CTRL\-C (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Fix deadlock calling Finalise while Run is running (Nick Craig\-Wood)
|
|
.RE
|
|
.IP \[bu] 2
|
|
lib/rest: Fix multipart uploads not stopping on context cancel (Nick
|
|
Craig\-Wood)
|
|
.RE
|
|
.IP \[bu] 2
|
|
Mount
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Allow mounting to root directory on windows (albertony)
|
|
.IP \[bu] 2
|
|
Improved handling of relative paths on windows (albertony)
|
|
.IP \[bu] 2
|
|
Fix unicode issues with accented characters on macOS (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Docs: document the new FileSecurity option in WinFsp 2021 (albertony)
|
|
.IP \[bu] 2
|
|
Docs: add note about volume path syntax on windows (albertony)
|
|
.IP \[bu] 2
|
|
Fix caching of old directories after renaming them (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Update cgofuse to the latest version to bring in macfuse 4 fix (Nick
|
|
Craig\-Wood)
|
|
.RE
|
|
.IP \[bu] 2
|
|
VFS
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
\f[C]\-\-vfs\-used\-is\-size\f[R] to report used space using recursive
|
|
scan (tYYGH)
|
|
.IP \[bu] 2
|
|
Don\[aq]t set modification time if it was already correct (Nick
|
|
Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Fix Create causing windows explorer to truncate files on CTRL\-C CTRL\-V
|
|
(Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Fix modtimes not updating when writing via cache (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Fix modtimes changing by fractional seconds after upload (Nick
|
|
Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Fix modtime set if \f[C]\-\-vfs\-cache\-mode writes\f[R]/\f[C]full\f[R]
|
|
and no write (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Rename files in cache and cancel uploads on directory rename (Nick
|
|
Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Fix directory renaming by renaming dirs cached in memory (Nick
|
|
Craig\-Wood)
|
|
.RE
|
|
.IP \[bu] 2
|
|
Local
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Add flag \f[C]\-\-local\-no\-preallocate\f[R] (David Sze)
|
|
.IP \[bu] 2
|
|
Make \f[C]nounc\f[R] an advanced option except on Windows (albertony)
|
|
.IP \[bu] 2
|
|
Don\[aq]t ignore preallocate disk full errors (Nick Craig\-Wood)
|
|
.RE
|
|
.IP \[bu] 2
|
|
Cache
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Add \f[C]\-\-fs\-cache\-expire\-duration\f[R] to control the fs cache
|
|
(Nick Craig\-Wood)
|
|
.RE
|
|
.IP \[bu] 2
|
|
Crypt
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Add option to not encrypt data (Vesnyx)
|
|
.IP \[bu] 2
|
|
Log hash ok on upload (albertony)
|
|
.RE
|
|
.IP \[bu] 2
|
|
Azure Blob
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Add container public access level support.
|
|
(Manish Kumar)
|
|
.RE
|
|
.IP \[bu] 2
|
|
B2
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Fix HTML files downloaded via cloudflare (Nick Craig\-Wood)
|
|
.RE
|
|
.IP \[bu] 2
|
|
Box
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Fix transfers getting stuck on token expiry after API change (Nick
|
|
Craig\-Wood)
|
|
.RE
|
|
.IP \[bu] 2
|
|
Chunker
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Partially implement no\-rename transactions (Maxwell Calman)
|
|
.RE
|
|
.IP \[bu] 2
|
|
Drive
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Don\[aq]t stop server side copy if couldn\[aq]t read description (Nick
|
|
Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Pass context on to drive SDK \- to help with cancellation (Nick
|
|
Craig\-Wood)
|
|
.RE
|
|
.IP \[bu] 2
|
|
Dropbox
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Add polling for changes support (Robert Thomas)
|
|
.IP \[bu] 2
|
|
Make \f[C]\-\-timeout 0\f[R] work properly (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Raise priority of rate limited message to INFO to make it more
|
|
noticeable (Nick Craig\-Wood)
|
|
.RE
|
|
.IP \[bu] 2
|
|
Fichier
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Implement copy & move (buengese)
|
|
.IP \[bu] 2
|
|
Implement public link (buengese)
|
|
.RE
|
|
.IP \[bu] 2
|
|
FTP
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Implement Shutdown method (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Close idle connections after \f[C]\-\-ftp\-idle\-timeout\f[R] (1m by
|
|
default) (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Make \f[C]\-\-timeout 0\f[R] work properly (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Add \f[C]\-\-ftp\-close\-timeout\f[R] flag for use with awkward ftp
|
|
servers (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Retry connections and logins on 421 errors (Nick Craig\-Wood)
|
|
.RE
|
|
.IP \[bu] 2
|
|
Hdfs
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Fix permissions for when directory is created (Lucas Messenger)
|
|
.RE
|
|
.IP \[bu] 2
|
|
Onedrive
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Make \f[C]\-\-timeout 0\f[R] work properly (Nick Craig\-Wood)
|
|
.RE
|
|
.IP \[bu] 2
|
|
S3
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Fix \f[C]\-\-s3\-profile\f[R] which wasn\[aq]t working (Nick
|
|
Craig\-Wood)
|
|
.RE
|
|
.IP \[bu] 2
|
|
SFTP
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Close idle connections after \f[C]\-\-sftp\-idle\-timeout\f[R] (1m by
|
|
default) (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Fix \[dq]file not found\[dq] errors for read once servers (Nick
|
|
Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Fix SetModTime stat failed: object not found with
|
|
\f[C]\-\-sftp\-set\-modtime=false\f[R] (Nick Craig\-Wood)
|
|
.RE
|
|
.IP \[bu] 2
|
|
Swift
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Update github.com/ncw/swift to v2.0.0 (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Implement copying large objects (nguyenhuuluan434)
|
|
.RE
|
|
.IP \[bu] 2
|
|
Union
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Fix crash when using epff policy (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Fix union attempting to update files on a read only file system (Nick
|
|
Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Refactor to use fspath.SplitFs instead of fs.ParseRemote (Nick
|
|
Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Fix initialisation broken in refactor (Nick Craig\-Wood)
|
|
.RE
|
|
.IP \[bu] 2
|
|
WebDAV
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Add support for sharepoint with NTLM authentication (Rauno Ots)
|
|
.IP \[bu] 2
|
|
Make sharepoint\-ntlm docs more consistent (Alex Chen)
|
|
.IP \[bu] 2
|
|
Improve terminology in sharepoint\-ntlm docs (Ivan Andreev)
|
|
.IP \[bu] 2
|
|
Disable HTTP/2 for NTLM authentication (georne)
|
|
.IP \[bu] 2
|
|
Fix sharepoint\-ntlm error 401 for parallel actions (Ivan Andreev)
|
|
.IP \[bu] 2
|
|
Check that purged directory really exists (Ivan Andreev)
|
|
.RE
|
|
.IP \[bu] 2
|
|
Yandex
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Make \f[C]\-\-timeout 0\f[R] work properly (Nick Craig\-Wood)
|
|
.RE
|
|
.IP \[bu] 2
|
|
Zoho
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Replace client id \- you will need to \f[C]rclone config reconnect\f[R]
|
|
after this (buengese)
|
|
.IP \[bu] 2
|
|
Add forgotten setupRegion() to NewFs \- this finally fixes regions other
|
|
than EU (buengese)
|
|
.RE
|
|
.SS v1.54.1 \- 2021\-03\-08
|
|
.PP
|
|
See commits (https://github.com/rclone/rclone/compare/v1.54.0...v1.54.1)
|
|
.IP \[bu] 2
|
|
Bug Fixes
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
accounting: Fix \-\-bwlimit when up or down is off (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
docs
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Fix nesting of brackets and backticks in ftp docs (edwardxml)
|
|
.IP \[bu] 2
|
|
Fix broken link in sftp page (edwardxml)
|
|
.IP \[bu] 2
|
|
Fix typo in crypt.md (Romeo Kienzler)
|
|
.IP \[bu] 2
|
|
Changelog: Correct link to digitalis.io (Alex JOST)
|
|
.IP \[bu] 2
|
|
Replace #file\-caching with #vfs\-file\-caching (Miron Veryanskiy)
|
|
.IP \[bu] 2
|
|
Convert bogus example link to code (edwardxml)
|
|
.IP \[bu] 2
|
|
Remove dead link from rc.md (edwardxml)
|
|
.RE
|
|
.IP \[bu] 2
|
|
rc: Sync,copy,move: document createEmptySrcDirs parameter (Nick
|
|
Craig\-Wood)
|
|
.IP \[bu] 2
|
|
lsjson: Fix unterminated JSON in the presence of errors (Nick
|
|
Craig\-Wood)
|
|
.RE
|
|
.IP \[bu] 2
|
|
Mount
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Fix mount dropping on macOS by setting \-\-daemon\-timeout 10m (Nick
|
|
Craig\-Wood)
|
|
.RE
|
|
.IP \[bu] 2
|
|
VFS
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Document simultaneous usage with the same cache shouldn\[aq]t be used
|
|
(Nick Craig\-Wood)
|
|
.RE
|
|
.IP \[bu] 2
|
|
B2
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Automatically raise upload cutoff to avoid spurious error (Nick
|
|
Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Fix failed to create file system with application key limited to a
|
|
prefix (Nick Craig\-Wood)
|
|
.RE
|
|
.IP \[bu] 2
|
|
Drive
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Refer to Shared Drives instead of Team Drives (Nick Craig\-Wood)
|
|
.RE
|
|
.IP \[bu] 2
|
|
Dropbox
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Add scopes to oauth request and optionally \[dq]members.read\[dq] (Nick
|
|
Craig\-Wood)
|
|
.RE
|
|
.IP \[bu] 2
|
|
S3
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Fix failed to create file system with folder level permissions policy
|
|
(Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Fix Wasabi HEAD requests returning stale data by using only 1 transport
|
|
(Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Fix shared_credentials_file auth (Dmitry Chepurovskiy)
|
|
.IP \[bu] 2
|
|
Add \-\-s3\-no\-head to reducing costs docs (Nick Craig\-Wood)
|
|
.RE
|
|
.IP \[bu] 2
|
|
Union
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Fix mkdir at root with remote:/ (Nick Craig\-Wood)
|
|
.RE
|
|
.IP \[bu] 2
|
|
Zoho
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Fix custom client id\[aq]s (buengese)
|
|
.RE
|
|
.SS v1.54.0 \- 2021\-02\-02
|
|
.PP
|
|
See commits (https://github.com/rclone/rclone/compare/v1.53.0...v1.54.0)
|
|
.IP \[bu] 2
|
|
New backends
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Compression remote (experimental) (buengese)
|
|
.IP \[bu] 2
|
|
Enterprise File Fabric (Nick Craig\-Wood)
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
This work was sponsored by Storage Made
|
|
Easy (https://storagemadeeasy.com/)
|
|
.RE
|
|
.IP \[bu] 2
|
|
HDFS (Hadoop Distributed File System) (Yury Stankevich)
|
|
.IP \[bu] 2
|
|
Zoho workdrive (buengese)
|
|
.RE
|
|
.IP \[bu] 2
|
|
New Features
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Deglobalise the config (Nick Craig\-Wood)
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Global config now read from the context
|
|
.IP \[bu] 2
|
|
This will enable passing of global config via the rc
|
|
.IP \[bu] 2
|
|
This work was sponsored by Digitalis (https://digitalis.io/)
|
|
.RE
|
|
.IP \[bu] 2
|
|
Add \f[C]\-\-bwlimit\f[R] for upload and download (Nick Craig\-Wood)
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Obey bwlimit in http Transport for better limiting
|
|
.RE
|
|
.IP \[bu] 2
|
|
Enhance systemd integration (Hekmon)
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
log level identification, manual activation with flag, automatic systemd
|
|
launch detection
|
|
.IP \[bu] 2
|
|
Don\[aq]t compile systemd log integration for non unix systems (Benjamin
|
|
Gustin)
|
|
.RE
|
|
.IP \[bu] 2
|
|
Add a \f[C]\-\-download\f[R] flag to md5sum/sha1sum/hashsum to force
|
|
rclone to download and hash files locally (lostheli)
|
|
.IP \[bu] 2
|
|
Add \f[C]\-\-progress\-terminal\-title\f[R] to print ETA to terminal
|
|
title (LaSombra)
|
|
.IP \[bu] 2
|
|
Make backend env vars show in help as the defaults for backend flags
|
|
(Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
build
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Raise minimum go version to go1.12 (Nick Craig\-Wood)
|
|
.RE
|
|
.IP \[bu] 2
|
|
dedupe
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Add \f[C]\-\-by\-hash\f[R] to dedupe on content hash not file name (Nick
|
|
Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Add \f[C]\-\-dedupe\-mode list\f[R] to just list dupes, changing nothing
|
|
(Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Add warning if used on a remote which can\[aq]t have duplicate names
|
|
(Nick Craig\-Wood)
|
|
.RE
|
|
.IP \[bu] 2
|
|
fs
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Add Shutdown optional method for backends (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
When using \f[C]\-\-files\-from\f[R] check files concurrently (zhucan)
|
|
.IP \[bu] 2
|
|
Accumulate stats when using \f[C]\-\-dry\-run\f[R] (Ingo Weiss)
|
|
.IP \[bu] 2
|
|
Always show stats when using \f[C]\-\-dry\-run\f[R] or
|
|
\f[C]\-\-interactive\f[R] (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Add support for flag \f[C]\-\-no\-console\f[R] on windows to hide the
|
|
console window (albertony)
|
|
.RE
|
|
.IP \[bu] 2
|
|
genautocomplete: Add support to output to stdout (Ingo)
|
|
.IP \[bu] 2
|
|
ncdu
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Highlight read errors instead of aborting (Claudio Bantaloukas)
|
|
.IP \[bu] 2
|
|
Add sort by average size in directory (Adam Pl\['a]nsk\['y])
|
|
.IP \[bu] 2
|
|
Add toggle option for average s3ize in directory \- key \[aq]a\[aq]
|
|
(Adam Pl\['a]nsk\['y])
|
|
.IP \[bu] 2
|
|
Add empty folder flag into ncdu browser (Adam Pl\['a]nsk\['y])
|
|
.IP \[bu] 2
|
|
Add \f[C]!\f[R] (errror) and \f[C].\f[R] (unreadable) file flags to go
|
|
with \f[C]e\f[R] (empty) (Nick Craig\-Wood)
|
|
.RE
|
|
.IP \[bu] 2
|
|
obscure: Make \f[C]rclone osbcure \-\f[R] ignore newline at end of line
|
|
(Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
operations
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Add logs when need to upload files to set mod times (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Move and copy log name of the destination object in verbose (Adam
|
|
Pl\['a]nsk\['y])
|
|
.IP \[bu] 2
|
|
Add size if known to skipped items and JSON log (Nick Craig\-Wood)
|
|
.RE
|
|
.IP \[bu] 2
|
|
rc
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Prefer actual listener address if using \[dq]:port\[dq] or
|
|
\[dq]addr:0\[dq] only (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Add listener for finished jobs (Aleksandar Jankovic)
|
|
.RE
|
|
.IP \[bu] 2
|
|
serve ftp: Add options to enable TLS (Deepak Sah)
|
|
.IP \[bu] 2
|
|
serve http/webdav: Redirect requests to the base url without the / (Nick
|
|
Craig\-Wood)
|
|
.IP \[bu] 2
|
|
serve restic: Implement object cache (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
stats: Add counter for deleted directories (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
sync: Only print \[dq]There was nothing to transfer\[dq] if no errors
|
|
(Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
webui
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Prompt user for updating webui if an update is available (Chaitanya
|
|
Bankanhal)
|
|
.IP \[bu] 2
|
|
Fix plugins initialization (negative0)
|
|
.RE
|
|
.RE
|
|
.IP \[bu] 2
|
|
Bug Fixes
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
fs
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Fix nil pointer on copy & move operations directly to remote (Anagh
|
|
Kumar Baranwal)
|
|
.IP \[bu] 2
|
|
Fix parsing of ..
|
|
when joining remotes (Nick Craig\-Wood)
|
|
.RE
|
|
.IP \[bu] 2
|
|
log: Fix enabling systemd logging when using \f[C]\-\-log\-file\f[R]
|
|
(Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
check
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Make the error count match up in the log message (Nick Craig\-Wood)
|
|
.RE
|
|
.IP \[bu] 2
|
|
move: Fix data loss when source and destination are the same object
|
|
(Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
operations
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Fix \f[C]\-\-cutof\-mode\f[R] hard not cutting off immediately (Nick
|
|
Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Fix \f[C]\-\-immutable\f[R] error message (Nick Craig\-Wood)
|
|
.RE
|
|
.IP \[bu] 2
|
|
sync
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Fix \f[C]\-\-cutoff\-mode\f[R] soft & cautious so it doesn\[aq]t end the
|
|
transfer early (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Fix \f[C]\-\-immutable\f[R] errors retrying many times (Nick
|
|
Craig\-Wood)
|
|
.RE
|
|
.RE
|
|
.IP \[bu] 2
|
|
Docs
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Many fixes and a rewrite of the filtering docs (edwardxml)
|
|
.IP \[bu] 2
|
|
Many spelling and grammar fixes (Josh Soref)
|
|
.IP \[bu] 2
|
|
Doc fixes for commands delete, purge, rmdir, rmdirs and mount
|
|
(albertony)
|
|
.IP \[bu] 2
|
|
And thanks to these people for many doc fixes too numerous to list
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Ameer Dawood, Antoine GIRARD, Bob Bagwill, Christopher Stewart
|
|
.IP \[bu] 2
|
|
CokeMine, David, Dov Murik, Durval Menezes, Evan Harris, gtorelly
|
|
.IP \[bu] 2
|
|
Ilyess Bachiri, Janne Johansson, Kerry Su, Marcin Zelent,
|
|
.IP \[bu] 2
|
|
Martin Michlmayr, Milly, S\[u01A1]n Tr\[u1EA7]n\-Nguy\[u1EC5]n
|
|
.RE
|
|
.RE
|
|
.IP \[bu] 2
|
|
Mount
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Update systemd status with cache stats (Hekmon)
|
|
.IP \[bu] 2
|
|
Disable bazil/fuse based mount on macOS (Nick Craig\-Wood)
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Make \f[C]rclone mount\f[R] actually run \f[C]rclone cmount\f[R] under
|
|
macOS (Nick Craig\-Wood)
|
|
.RE
|
|
.IP \[bu] 2
|
|
Implement mknod to make NFS file creation work (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Make sure we don\[aq]t call umount more than once (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
More user friendly mounting as network drive on windows (albertony)
|
|
.IP \[bu] 2
|
|
Detect if uid or gid are set in same option string: \-o uid=123,gid=456
|
|
(albertony)
|
|
.IP \[bu] 2
|
|
Don\[aq]t attempt to unmount if fs has been destroyed already (Nick
|
|
Craig\-Wood)
|
|
.RE
|
|
.IP \[bu] 2
|
|
VFS
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Fix virtual entries causing deleted files to still appear (Nick
|
|
Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Fix \[dq]file already exists\[dq] error for stale cache files (Nick
|
|
Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Fix file leaks with \f[C]\-\-vfs\-cache\-mode\f[R] full and
|
|
\f[C]\-\-buffer\-size 0\f[R] (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Fix invalid cache path on windows when using :backend: as remote
|
|
(albertony)
|
|
.RE
|
|
.IP \[bu] 2
|
|
Local
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Continue listing files/folders when a circular symlink is detected
|
|
(Manish Gupta)
|
|
.IP \[bu] 2
|
|
New flag \f[C]\-\-local\-zero\-size\-links\f[R] to fix sync on some
|
|
virtual filesystems (Riccardo Iaconelli)
|
|
.RE
|
|
.IP \[bu] 2
|
|
Azure Blob
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Add support for service principals (James Lim)
|
|
.IP \[bu] 2
|
|
Add support for managed identities (Brad Ackerman)
|
|
.IP \[bu] 2
|
|
Add examples for access tier (Bob Pusateri)
|
|
.IP \[bu] 2
|
|
Utilize the streaming capabilities from the SDK for multipart uploads
|
|
(Denis Neuling)
|
|
.IP \[bu] 2
|
|
Fix setting of mime types (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Fix crash when listing outside a SAS URL\[aq]s root (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Delete archive tier blobs before update if
|
|
\f[C]\-\-azureblob\-archive\-tier\-delete\f[R] (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Fix crash on startup (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Fix memory usage by upgrading the SDK to v0.13.0 and implementing a
|
|
TransferManager (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Require go1.14+ to compile due to SDK changes (Nick Craig\-Wood)
|
|
.RE
|
|
.IP \[bu] 2
|
|
B2
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Make NewObject use less expensive API calls (Nick Craig\-Wood)
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
This will improve \f[C]\-\-files\-from\f[R] and \f[C]restic serve\f[R]
|
|
in particular
|
|
.RE
|
|
.IP \[bu] 2
|
|
Fixed crash on an empty file name (lluuaapp)
|
|
.RE
|
|
.IP \[bu] 2
|
|
Box
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Fix NewObject for files that differ in case (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Fix finding directories in a case insentive way (Nick Craig\-Wood)
|
|
.RE
|
|
.IP \[bu] 2
|
|
Chunker
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Skip long local hashing, hash in\-transit (fixes) (Ivan Andreev)
|
|
.IP \[bu] 2
|
|
Set Features ReadMimeType to false as Object.MimeType not supported
|
|
(Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Fix case\-insensitive NewObject, test metadata detection (Ivan Andreev)
|
|
.RE
|
|
.IP \[bu] 2
|
|
Drive
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Implement \f[C]rclone backend copyid\f[R] command for copying files by
|
|
ID (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Added flag \f[C]\-\-drive\-stop\-on\-download\-limit\f[R] to stop
|
|
transfers when the download limit is exceeded (Anagh Kumar Baranwal)
|
|
.IP \[bu] 2
|
|
Implement CleanUp workaround for team drives (buengese)
|
|
.IP \[bu] 2
|
|
Allow shortcut resolution and creation to be retried (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Log that emptying the trash can take some time (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Add xdg office icons to xdg desktop files (Pau Rodriguez\-Estivill)
|
|
.RE
|
|
.IP \[bu] 2
|
|
Dropbox
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Add support for viewing shared files and folders (buengese)
|
|
.IP \[bu] 2
|
|
Enable short lived access tokens (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Implement IDer on Objects so \f[C]rclone lsf\f[R] etc can read the IDs
|
|
(buengese)
|
|
.IP \[bu] 2
|
|
Set Features ReadMimeType to false as Object.MimeType not supported
|
|
(Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Make malformed_path errors from too long files not retriable (Nick
|
|
Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Test file name length before upload to fix upload loop (Nick
|
|
Craig\-Wood)
|
|
.RE
|
|
.IP \[bu] 2
|
|
Fichier
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Set Features ReadMimeType to true as Object.MimeType is supported (Nick
|
|
Craig\-Wood)
|
|
.RE
|
|
.IP \[bu] 2
|
|
FTP
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Add \f[C]\-\-ftp\-disable\-msld\f[R] option to ignore MLSD for really
|
|
old servers (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Make \f[C]\-\-tpslimit apply\f[R] (Nick Craig\-Wood)
|
|
.RE
|
|
.IP \[bu] 2
|
|
Google Cloud Storage
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Storage class object header support (Laurens Janssen)
|
|
.IP \[bu] 2
|
|
Fix anonymous client to use rclone\[aq]s HTTP client (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Fix
|
|
\f[C]Entry doesn\[aq]t belong in directory \[dq]\[dq] (same as directory) \- ignoring\f[R]
|
|
(Nick Craig\-Wood)
|
|
.RE
|
|
.IP \[bu] 2
|
|
Googlephotos
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
New flag \f[C]\-\-gphotos\-include\-archived\f[R] to show archived
|
|
photos as well (Nicolas Rueff)
|
|
.RE
|
|
.IP \[bu] 2
|
|
Jottacloud
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Don\[aq]t erroneously report support for writing mime types (buengese)
|
|
.IP \[bu] 2
|
|
Add support for Telia Cloud (Patrik Nordl\['e]n)
|
|
.RE
|
|
.IP \[bu] 2
|
|
Mailru
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Accept special folders eg camera\-upload (Ivan Andreev)
|
|
.IP \[bu] 2
|
|
Avoid prehashing of large local files (Ivan Andreev)
|
|
.IP \[bu] 2
|
|
Fix uploads after recent changes on server (Ivan Andreev)
|
|
.IP \[bu] 2
|
|
Fix range requests after June 2020 changes on server (Ivan Andreev)
|
|
.IP \[bu] 2
|
|
Fix invalid timestamp on corrupted files (fixes) (Ivan Andreev)
|
|
.IP \[bu] 2
|
|
Remove deprecated protocol quirks (Ivan Andreev)
|
|
.RE
|
|
.IP \[bu] 2
|
|
Memory
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Fix setting of mime types (Nick Craig\-Wood)
|
|
.RE
|
|
.IP \[bu] 2
|
|
Onedrive
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Add support for China region operated by 21vianet and other regional
|
|
suppliers (NyaMisty)
|
|
.IP \[bu] 2
|
|
Warn on gateway timeout errors (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Fall back to normal copy if server\-side copy unavailable (Alex Chen)
|
|
.IP \[bu] 2
|
|
Fix server\-side copy completely disabled on OneDrive for Business
|
|
(Cnly)
|
|
.IP \[bu] 2
|
|
(business only) workaround to replace existing file on server\-side copy
|
|
(Alex Chen)
|
|
.IP \[bu] 2
|
|
Enhance link creation with expiry, scope, type and password (Nick
|
|
Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Remove % and # from the set of encoded characters (Alex Chen)
|
|
.IP \[bu] 2
|
|
Support addressing site by server\-relative URL (kice)
|
|
.RE
|
|
.IP \[bu] 2
|
|
Opendrive
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Fix finding directories in a case insensitive way (Nick Craig\-Wood)
|
|
.RE
|
|
.IP \[bu] 2
|
|
Pcloud
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Fix setting of mime types (Nick Craig\-Wood)
|
|
.RE
|
|
.IP \[bu] 2
|
|
Premiumizeme
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Fix finding directories in a case insensitive way (Nick Craig\-Wood)
|
|
.RE
|
|
.IP \[bu] 2
|
|
Qingstor
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Fix error propagation in CleanUp (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Fix rclone cleanup (Nick Craig\-Wood)
|
|
.RE
|
|
.IP \[bu] 2
|
|
S3
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Added \f[C]\-\-s3\-disable\-http2\f[R] to disable http/2 (Anagh Kumar
|
|
Baranwal)
|
|
.IP \[bu] 2
|
|
Complete SSE\-C implementation (Nick Craig\-Wood)
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Fix hashes on small files with AWS:KMS and SSE\-C (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Add MD5 metadata to objects uploaded with SSE\-AWS/SSE\-C (Nick
|
|
Craig\-Wood)
|
|
.RE
|
|
.IP \[bu] 2
|
|
Add \f[C]\-\-s3\-no\-head parameter\f[R] to minimise transactions on
|
|
upload (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Update docs with a Reducing Costs section (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Added error handling for error code 429 indicating too many requests
|
|
(Anagh Kumar Baranwal)
|
|
.IP \[bu] 2
|
|
Add requester pays option (kelv)
|
|
.IP \[bu] 2
|
|
Fix copy multipart with v2 auth failing with
|
|
\[aq]SignatureDoesNotMatch\[aq] (Louis Koo)
|
|
.RE
|
|
.IP \[bu] 2
|
|
SFTP
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Allow cert based auth via optional pubkey (Stephen Harris)
|
|
.IP \[bu] 2
|
|
Allow user to optionally check server hosts key to add security (Stephen
|
|
Harris)
|
|
.IP \[bu] 2
|
|
Defer asking for user passwords until the SSH connection succeeds
|
|
(Stephen Harris)
|
|
.IP \[bu] 2
|
|
Remember entered password in AskPass mode (Stephen Harris)
|
|
.IP \[bu] 2
|
|
Implement Shutdown method (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Implement keyboard interactive authentication (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Make \f[C]\-\-tpslimit\f[R] apply (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Implement \f[C]\-\-sftp\-use\-fstat\f[R] for unusual SFTP servers (Nick
|
|
Craig\-Wood)
|
|
.RE
|
|
.IP \[bu] 2
|
|
Sugarsync
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Fix NewObject for files that differ in case (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Fix finding directories in a case insentive way (Nick Craig\-Wood)
|
|
.RE
|
|
.IP \[bu] 2
|
|
Swift
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Fix deletion of parts of Static Large Object (SLO) (Nguy\[u1EC5]n
|
|
H\[u1EEF]u Lu\[^a]n)
|
|
.IP \[bu] 2
|
|
Ensure partially uploaded large files are uploaded unless
|
|
\f[C]\-\-swift\-leave\-parts\-on\-error\f[R] (Nguy\[u1EC5]n H\[u1EEF]u
|
|
Lu\[^a]n)
|
|
.RE
|
|
.IP \[bu] 2
|
|
Tardigrade
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Upgrade to uplink v1.4.1 (Caleb Case)
|
|
.RE
|
|
.IP \[bu] 2
|
|
WebDAV
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Updated docs to show streaming to nextcloud is working (Durval Menezes)
|
|
.RE
|
|
.IP \[bu] 2
|
|
Yandex
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Set Features WriteMimeType to false as Yandex ignores mime types (Nick
|
|
Craig\-Wood)
|
|
.RE
|
|
.SS v1.53.4 \- 2021\-01\-20
|
|
.PP
|
|
See commits (https://github.com/rclone/rclone/compare/v1.53.3...v1.53.4)
|
|
.IP \[bu] 2
|
|
Bug Fixes
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
accounting: Fix data race in Transferred() (Maciej Zimnoch)
|
|
.IP \[bu] 2
|
|
build
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Stop tagged releases making a current beta (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Upgrade docker buildx action (Matteo Pietro Dazzi)
|
|
.IP \[bu] 2
|
|
Add \-buildmode to cross\-compile.go (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Fix docker build by upgrading ilteoood/docker_buildx (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Revert GitHub actions brew fix since this is now fixed (Nick
|
|
Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Fix brew install \-\-cask syntax for macOS build (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Update nfpm syntax to fix build of .deb/.rpm packages (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Fix for Windows build errors (Ivan Andreev)
|
|
.RE
|
|
.IP \[bu] 2
|
|
fs: Parseduration: fixed tests to use UTC time (Ankur Gupta)
|
|
.IP \[bu] 2
|
|
fshttp: Prevent overlap of HTTP headers in logs (Nathan Collins)
|
|
.IP \[bu] 2
|
|
rc
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Fix core/command giving 500 internal error (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Add Copy method to rc.Params (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Fix 500 error when marshalling errors from core/command (Nick
|
|
Craig\-Wood)
|
|
.IP \[bu] 2
|
|
plugins: Create plugins files only if webui is enabled.
|
|
(negative0)
|
|
.RE
|
|
.IP \[bu] 2
|
|
serve http: Fix serving files of unknown length (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
serve sftp: Fix authentication on one connection blocking others (Nick
|
|
Craig\-Wood)
|
|
.RE
|
|
.IP \[bu] 2
|
|
Mount
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Add optional \f[C]brew\f[R] tag to throw an error when using mount in
|
|
the binaries installed via Homebrew (Anagh Kumar Baranwal)
|
|
.IP \[bu] 2
|
|
Add \[dq].\[dq] and \[dq]..\[dq] to directories to match cmount and
|
|
expectations (Nick Craig\-Wood)
|
|
.RE
|
|
.IP \[bu] 2
|
|
VFS
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Make cache dir absolute before using it to fix path too long errors
|
|
(Nick Craig\-Wood)
|
|
.RE
|
|
.IP \[bu] 2
|
|
Chunker
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Improve detection of incompatible metadata (Ivan Andreev)
|
|
.RE
|
|
.IP \[bu] 2
|
|
Google Cloud Storage
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Fix server side copy of large objects (Nick Craig\-Wood)
|
|
.RE
|
|
.IP \[bu] 2
|
|
Jottacloud
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Fix token renewer to fix long uploads (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Fix token refresh failed: is not a regular file error (Nick Craig\-Wood)
|
|
.RE
|
|
.IP \[bu] 2
|
|
Pcloud
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Only use SHA1 hashes in EU region (Nick Craig\-Wood)
|
|
.RE
|
|
.IP \[bu] 2
|
|
Sharefile
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Undo Fix backend due to API swapping integers for strings (Nick
|
|
Craig\-Wood)
|
|
.RE
|
|
.IP \[bu] 2
|
|
WebDAV
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Fix Open Range requests to fix 4shared mount (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Add \[dq]Depth: 0\[dq] to GET requests to fix bitrix (Nick Craig\-Wood)
|
|
.RE
|
|
.SS v1.53.3 \- 2020\-11\-19
|
|
.PP
|
|
See commits (https://github.com/rclone/rclone/compare/v1.53.2...v1.53.3)
|
|
.IP \[bu] 2
|
|
Bug Fixes
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
random: Fix incorrect use of math/rand instead of crypto/rand
|
|
CVE\-2020\-28924 (Nick Craig\-Wood)
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Passwords you have generated with \f[C]rclone config\f[R] may be
|
|
insecure
|
|
.IP \[bu] 2
|
|
See issue #4783 (https://github.com/rclone/rclone/issues/4783) for more
|
|
details and a checking tool
|
|
.RE
|
|
.IP \[bu] 2
|
|
random: Seed math/rand in one place with crypto strong seed (Nick
|
|
Craig\-Wood)
|
|
.RE
|
|
.IP \[bu] 2
|
|
VFS
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Fix vfs/refresh calls with fs= parameter (Nick Craig\-Wood)
|
|
.RE
|
|
.IP \[bu] 2
|
|
Sharefile
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Fix backend due to API swapping integers for strings (Nick Craig\-Wood)
|
|
.RE
|
|
.SS v1.53.2 \- 2020\-10\-26
|
|
.PP
|
|
See commits (https://github.com/rclone/rclone/compare/v1.53.1...v1.53.2)
|
|
.IP \[bu] 2
|
|
Bug Fixes
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
acounting
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Fix incorrect speed and transferTime in core/stats (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Stabilize display order of transfers on Windows (Nick Craig\-Wood)
|
|
.RE
|
|
.IP \[bu] 2
|
|
operations
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Fix use of \-\-suffix without \-\-backup\-dir (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Fix spurious \[dq]\-\-checksum is in use but the source and destination
|
|
have no hashes in common\[dq] (Nick Craig\-Wood)
|
|
.RE
|
|
.IP \[bu] 2
|
|
build
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Work around GitHub actions brew problem (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Stop using set\-env and set\-path in the GitHub actions (Nick
|
|
Craig\-Wood)
|
|
.RE
|
|
.RE
|
|
.IP \[bu] 2
|
|
Mount
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
mount2: Fix the swapped UID / GID values (Russell Cattelan)
|
|
.RE
|
|
.IP \[bu] 2
|
|
VFS
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Detect and recover from a file being removed externally from the cache
|
|
(Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Fix a deadlock vulnerability in downloaders.Close (Leo Luan)
|
|
.IP \[bu] 2
|
|
Fix a race condition in retryFailedResets (Leo Luan)
|
|
.IP \[bu] 2
|
|
Fix missed concurrency control between some item operations and reset
|
|
(Leo Luan)
|
|
.IP \[bu] 2
|
|
Add exponential backoff during ENOSPC retries (Leo Luan)
|
|
.IP \[bu] 2
|
|
Add a missed update of used cache space (Leo Luan)
|
|
.IP \[bu] 2
|
|
Fix \-\-no\-modtime to not attempt to set modtimes (as documented) (Nick
|
|
Craig\-Wood)
|
|
.RE
|
|
.IP \[bu] 2
|
|
Local
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Fix sizes and syncing with \-\-links option on Windows (Nick
|
|
Craig\-Wood)
|
|
.RE
|
|
.IP \[bu] 2
|
|
Chunker
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Disable ListR to fix missing files on GDrive (workaround) (Ivan Andreev)
|
|
.IP \[bu] 2
|
|
Fix upload over crypt (Ivan Andreev)
|
|
.RE
|
|
.IP \[bu] 2
|
|
Fichier
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Increase maximum file size from 100GB to 300GB (gyutw)
|
|
.RE
|
|
.IP \[bu] 2
|
|
Jottacloud
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Remove clientSecret from config when upgrading to token based
|
|
authentication (buengese)
|
|
.IP \[bu] 2
|
|
Avoid double url escaping of device/mountpoint (albertony)
|
|
.IP \[bu] 2
|
|
Remove DirMove workaround as it\[aq]s not required anymore \- also
|
|
(buengese)
|
|
.RE
|
|
.IP \[bu] 2
|
|
Mailru
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Fix uploads after recent changes on server (Ivan Andreev)
|
|
.IP \[bu] 2
|
|
Fix range requests after june changes on server (Ivan Andreev)
|
|
.IP \[bu] 2
|
|
Fix invalid timestamp on corrupted files (fixes) (Ivan Andreev)
|
|
.RE
|
|
.IP \[bu] 2
|
|
Onedrive
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Fix disk usage for sharepoint (Nick Craig\-Wood)
|
|
.RE
|
|
.IP \[bu] 2
|
|
S3
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Add missing regions for AWS (Anagh Kumar Baranwal)
|
|
.RE
|
|
.IP \[bu] 2
|
|
Seafile
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Fix accessing libraries > 2GB on 32 bit systems (Muffin King)
|
|
.RE
|
|
.IP \[bu] 2
|
|
SFTP
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Always convert the checksum to lower case (buengese)
|
|
.RE
|
|
.IP \[bu] 2
|
|
Union
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Create root directories if none exist (Nick Craig\-Wood)
|
|
.RE
|
|
.SS v1.53.1 \- 2020\-09\-13
|
|
.PP
|
|
See commits (https://github.com/rclone/rclone/compare/v1.53.0...v1.53.1)
|
|
.IP \[bu] 2
|
|
Bug Fixes
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
accounting: Remove new line from end of \-\-stats\-one\-line display
|
|
(Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
check
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Add back missing \-\-download flag (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Fix docs (Nick Craig\-Wood)
|
|
.RE
|
|
.IP \[bu] 2
|
|
docs
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Note \-\-log\-file does append (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Add full stops for consistency in rclone \-\-help (edwardxml)
|
|
.IP \[bu] 2
|
|
Add Tencent COS to s3 provider list (wjielai)
|
|
.IP \[bu] 2
|
|
Updated mount command to reflect that it requires Go 1.13 or newer (Evan
|
|
Harris)
|
|
.IP \[bu] 2
|
|
jottacloud: Mention that uploads from local disk will not need to cache
|
|
files to disk for md5 calculation (albertony)
|
|
.IP \[bu] 2
|
|
Fix formatting of rc docs page (Nick Craig\-Wood)
|
|
.RE
|
|
.IP \[bu] 2
|
|
build
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Include vendor tar ball in release and fix startdev (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Fix \[dq]Illegal instruction\[dq] error for ARMv6 builds (Nick
|
|
Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Fix architecture name in ARMv7 build (Nick Craig\-Wood)
|
|
.RE
|
|
.RE
|
|
.IP \[bu] 2
|
|
VFS
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Fix spurious error \[dq]vfs cache: failed to _ensure cache EOF\[dq]
|
|
(Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Log an ERROR if we fail to set the file to be sparse (Nick Craig\-Wood)
|
|
.RE
|
|
.IP \[bu] 2
|
|
Local
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Log an ERROR if we fail to set the file to be sparse (Nick Craig\-Wood)
|
|
.RE
|
|
.IP \[bu] 2
|
|
Drive
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Re\-adds special oauth help text (Tim Gallant)
|
|
.RE
|
|
.IP \[bu] 2
|
|
Opendrive
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Do not retry 400 errors (Evan Harris)
|
|
.RE
|
|
.SS v1.53.0 \- 2020\-09\-02
|
|
.PP
|
|
See commits (https://github.com/rclone/rclone/compare/v1.52.0...v1.53.0)
|
|
.IP \[bu] 2
|
|
New Features
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
The VFS
|
|
layer (https://rclone.org/commands/rclone_mount/#vfs-virtual-file-system)
|
|
was heavily reworked for this release \- see below for more details
|
|
.IP \[bu] 2
|
|
Interactive mode
|
|
\-i/\-\-interactive (https://rclone.org/docs/#interactive) for
|
|
destructive operations (fishbullet)
|
|
.IP \[bu] 2
|
|
Add
|
|
\-\-bwlimit\-file (https://rclone.org/docs/#bwlimit-file-bandwidth-spec)
|
|
flag to limit speeds of individual file transfers (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Transfers are sorted by start time in the stats and progress output (Max
|
|
Sum)
|
|
.IP \[bu] 2
|
|
Make sure backends expand \f[C]\[ti]\f[R] and environment vars in file
|
|
names they use (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Add \-\-refresh\-times (https://rclone.org/docs/#refresh-times) flag to
|
|
set modtimes on hashless backends (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
build
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Remove vendor directory in favour of Go modules (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Build with go1.15.x by default (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Drop macOS 386 build as it is no longer supported by go1.15 (Nick
|
|
Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Add ARMv7 to the supported builds (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Enable \f[C]rclone cmount\f[R] on macOS (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Make rclone build with gccgo (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Make rclone build with wasm (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Change beta numbering to be semver compatible (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Add file properties and icon to Windows executable (albertony)
|
|
.IP \[bu] 2
|
|
Add experimental interface for integrating rclone into browsers (Nick
|
|
Craig\-Wood)
|
|
.RE
|
|
.IP \[bu] 2
|
|
lib: Add file name compression (Klaus Post)
|
|
.IP \[bu] 2
|
|
rc
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Allow installation and use of plugins and test plugins with
|
|
rclone\-webui (Chaitanya Bankanhal)
|
|
.IP \[bu] 2
|
|
Add reverse proxy pluginsHandler for serving plugins (Chaitanya
|
|
Bankanhal)
|
|
.IP \[bu] 2
|
|
Add \f[C]mount/listmounts\f[R] option for listing current mounts
|
|
(Chaitanya Bankanhal)
|
|
.IP \[bu] 2
|
|
Add \f[C]operations/uploadfile\f[R] to upload a file through rc using
|
|
encoding multipart/form\-data (Chaitanya Bankanhal)
|
|
.IP \[bu] 2
|
|
Add \f[C]core/command\f[R] to execute rclone terminal commands.
|
|
(Chaitanya Bankanhal)
|
|
.RE
|
|
.IP \[bu] 2
|
|
\f[C]rclone check\f[R]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Add reporting of filenames for same/missing/changed (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Make check command obey
|
|
\f[C]\-\-dry\-run\f[R]/\f[C]\-i\f[R]/\f[C]\-\-interactive\f[R] (Nick
|
|
Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Make check do \f[C]\-\-checkers\f[R] files concurrently (Nick
|
|
Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Retry downloads if they fail when using the \f[C]\-\-download\f[R] flag
|
|
(Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Make it show stats by default (Nick Craig\-Wood)
|
|
.RE
|
|
.IP \[bu] 2
|
|
\f[C]rclone obscure\f[R]: Allow obscure command to accept password on
|
|
STDIN (David Ibarra)
|
|
.IP \[bu] 2
|
|
\f[C]rclone config\f[R]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Set RCLONE_CONFIG_DIR for use in config files and subprocesses (Nick
|
|
Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Reject remote names starting with a dash.
|
|
(jtagcat)
|
|
.RE
|
|
.IP \[bu] 2
|
|
\f[C]rclone cryptcheck\f[R]: Add reporting of filenames for
|
|
same/missing/changed (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
\f[C]rclone dedupe\f[R]: Make it obey the \f[C]\-\-size\-only\f[R] flag
|
|
for duplicate detection (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
\f[C]rclone link\f[R]: Add \f[C]\-\-expire\f[R] and \f[C]\-\-unlink\f[R]
|
|
flags (Roman Kredentser)
|
|
.IP \[bu] 2
|
|
\f[C]rclone mkdir\f[R]: Warn when using mkdir on remotes which can\[aq]t
|
|
have empty directories (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
\f[C]rclone rc\f[R]: Allow JSON parameters to simplify command line
|
|
usage (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
\f[C]rclone serve ftp\f[R]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Don\[aq]t compile on < go1.13 after dependency update (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Add error message if auth proxy fails (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Use refactored goftp.io/server library for binary shrink (Nick
|
|
Craig\-Wood)
|
|
.RE
|
|
.IP \[bu] 2
|
|
\f[C]rclone serve restic\f[R]: Expose interfaces so that rclone can be
|
|
used as a library from within restic (Jack)
|
|
.IP \[bu] 2
|
|
\f[C]rclone sync\f[R]: Add \f[C]\-\-track\-renames\-strategy leaf\f[R]
|
|
(Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
\f[C]rclone touch\f[R]: Add ability to set nanosecond resolution times
|
|
(Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
\f[C]rclone tree\f[R]: Remove \f[C]\-i\f[R] shorthand for
|
|
\f[C]\-\-noindent\f[R] as it conflicts with
|
|
\f[C]\-i\f[R]/\f[C]\-\-interactive\f[R] (Nick Craig\-Wood)
|
|
.RE
|
|
.IP \[bu] 2
|
|
Bug Fixes
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
accounting
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Fix documentation for \f[C]speed\f[R]/\f[C]speedAvg\f[R] (Nick
|
|
Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Fix elapsed time not show actual time since beginning (Chaitanya
|
|
Bankanhal)
|
|
.IP \[bu] 2
|
|
Fix deadlock in stats printing (Nick Craig\-Wood)
|
|
.RE
|
|
.IP \[bu] 2
|
|
build
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Fix file handle leak in GitHub release tool (Garrett Squire)
|
|
.RE
|
|
.IP \[bu] 2
|
|
\f[C]rclone check\f[R]: Fix successful retries with
|
|
\f[C]\-\-download\f[R] counting errors (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
\f[C]rclone dedupe\f[R]: Fix logging to be easier to understand (Nick
|
|
Craig\-Wood)
|
|
.RE
|
|
.IP \[bu] 2
|
|
Mount
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Warn macOS users that mount implementation is changing (Nick
|
|
Craig\-Wood)
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
to test the new implementation use \f[C]rclone cmount\f[R] instead of
|
|
\f[C]rclone mount\f[R]
|
|
.IP \[bu] 2
|
|
this is because the library rclone uses has dropped macOS support
|
|
.RE
|
|
.IP \[bu] 2
|
|
rc interface
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Add call for unmount all (Chaitanya Bankanhal)
|
|
.IP \[bu] 2
|
|
Make \f[C]mount/mount\f[R] remote control take vfsOpt option (Nick
|
|
Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Add mountOpt to \f[C]mount/mount\f[R] (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Add VFS and Mount options to \f[C]mount/listmounts\f[R] (Nick
|
|
Craig\-Wood)
|
|
.RE
|
|
.IP \[bu] 2
|
|
Catch panics in cgofuse initialization and turn into error messages
|
|
(Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Always supply stat information in Readdir (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Add support for reading unknown length files using direct IO (Windows)
|
|
(Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Fix On Windows don\[aq]t add \f[C]\-o uid/gid=\-1\f[R] if user supplies
|
|
\f[C]\-o uid/gid\f[R].
|
|
(Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Fix macOS losing directory contents in cmount (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Fix volume name broken in recent refactor (Nick Craig\-Wood)
|
|
.RE
|
|
.IP \[bu] 2
|
|
VFS
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Implement partial reads for \f[C]\-\-vfs\-cache\-mode full\f[R] (Nick
|
|
Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Add \f[C]\-\-vfs\-writeback\f[R] option to delay writes back to cloud
|
|
storage (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Add \f[C]\-\-vfs\-read\-ahead\f[R] parameter for use with
|
|
\f[C]\-\-vfs\-cache\-mode full\f[R] (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Restart pending uploads on restart of the cache (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Support synchronous cache space recovery upon ENOSPC (Leo Luan)
|
|
.IP \[bu] 2
|
|
Allow ReadAt and WriteAt to run concurrently with themselves (Nick
|
|
Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Change modtime of file before upload to current (Rob Calistri)
|
|
.IP \[bu] 2
|
|
Recommend \f[C]\-\-vfs\-cache\-modes writes\f[R] on backends which
|
|
can\[aq]t stream (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Add an optional \f[C]fs\f[R] parameter to vfs rc methods (Nick
|
|
Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Fix errors when using > 260 char files in the cache in Windows (Nick
|
|
Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Fix renaming of items while they are being uploaded (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Fix very high load caused by slow directory listings (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Fix renamed files not being uploaded with
|
|
\f[C]\-\-vfs\-cache\-mode minimal\f[R] (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Fix directory locking caused by slow directory listings (Nick
|
|
Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Fix saving from chrome without \f[C]\-\-vfs\-cache\-mode writes\f[R]
|
|
(Nick Craig\-Wood)
|
|
.RE
|
|
.IP \[bu] 2
|
|
Local
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Add \f[C]\-\-local\-no\-updated\f[R] to provide a consistent view of
|
|
changing objects (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Add \f[C]\-\-local\-no\-set\-modtime\f[R] option to prevent modtime
|
|
changes (tyhuber1)
|
|
.IP \[bu] 2
|
|
Fix race conditions updating and reading Object metadata (Nick
|
|
Craig\-Wood)
|
|
.RE
|
|
.IP \[bu] 2
|
|
Cache
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Make any created backends be cached to fix rc problems (Nick
|
|
Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Fix dedupe on caches wrapping drives (Nick Craig\-Wood)
|
|
.RE
|
|
.IP \[bu] 2
|
|
Crypt
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Add \f[C]\-\-crypt\-server\-side\-across\-configs\f[R] flag (Nick
|
|
Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Make any created backends be cached to fix rc problems (Nick
|
|
Craig\-Wood)
|
|
.RE
|
|
.IP \[bu] 2
|
|
Alias
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Make any created backends be cached to fix rc problems (Nick
|
|
Craig\-Wood)
|
|
.RE
|
|
.IP \[bu] 2
|
|
Azure Blob
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Don\[aq]t compile on < go1.13 after dependency update (Nick Craig\-Wood)
|
|
.RE
|
|
.IP \[bu] 2
|
|
B2
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Implement server\-side copy for files > 5GB (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Cancel in progress multipart uploads and copies on rclone exit (Nick
|
|
Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Note that b2\[aq]s encoding now allows \ but rclone\[aq]s hasn\[aq]t
|
|
changed (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Fix transfers when using download_url (Nick Craig\-Wood)
|
|
.RE
|
|
.IP \[bu] 2
|
|
Box
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Implement rclone cleanup (buengese)
|
|
.IP \[bu] 2
|
|
Cancel in progress multipart uploads and copies on rclone exit (Nick
|
|
Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Allow authentication with access token (David)
|
|
.RE
|
|
.IP \[bu] 2
|
|
Chunker
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Make any created backends be cached to fix rc problems (Nick
|
|
Craig\-Wood)
|
|
.RE
|
|
.IP \[bu] 2
|
|
Drive
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Add \f[C]rclone backend drives\f[R] to list shared drives (teamdrives)
|
|
(Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Implement \f[C]rclone backend untrash\f[R] (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Work around drive bug which didn\[aq]t set modtime of copied docs (Nick
|
|
Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Added \f[C]\-\-drive\-starred\-only\f[R] to only show starred files (Jay
|
|
McEntire)
|
|
.IP \[bu] 2
|
|
Deprecate \f[C]\-\-drive\-alternate\-export\f[R] as it is no longer
|
|
needed (themylogin)
|
|
.IP \[bu] 2
|
|
Fix duplication of Google docs on server\-side copy (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Fix \[dq]panic: send on closed channel\[dq] when recycling dir entries
|
|
(Nick Craig\-Wood)
|
|
.RE
|
|
.IP \[bu] 2
|
|
Dropbox
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Add copyright detector info in limitations section in the docs (Alex
|
|
Guerrero)
|
|
.IP \[bu] 2
|
|
Fix \f[C]rclone link\f[R] by removing expires parameter (Nick
|
|
Craig\-Wood)
|
|
.RE
|
|
.IP \[bu] 2
|
|
Fichier
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Detect Flood detected: IP Locked error and sleep for 30s (Nick
|
|
Craig\-Wood)
|
|
.RE
|
|
.IP \[bu] 2
|
|
FTP
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Add explicit TLS support (Heiko Bornholdt)
|
|
.IP \[bu] 2
|
|
Add support for \f[C]\-\-dump bodies\f[R] and \f[C]\-\-dump auth\f[R]
|
|
for debugging (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Fix interoperation with pure\-ftpd (Nick Craig\-Wood)
|
|
.RE
|
|
.IP \[bu] 2
|
|
Google Cloud Storage
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Add support for anonymous access (Kai L\[:u]ke)
|
|
.RE
|
|
.IP \[bu] 2
|
|
Jottacloud
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Bring back legacy authentication for use with whitelabel versions
|
|
(buengese)
|
|
.IP \[bu] 2
|
|
Switch to new api root \- also implement a very ugly workaround for the
|
|
DirMove failures (buengese)
|
|
.RE
|
|
.IP \[bu] 2
|
|
Onedrive
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Rework cancel of multipart uploads on rclone exit (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Implement rclone cleanup (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Add \f[C]\-\-onedrive\-no\-versions\f[R] flag to remove old versions
|
|
(Nick Craig\-Wood)
|
|
.RE
|
|
.IP \[bu] 2
|
|
Pcloud
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Implement \f[C]rclone link\f[R] for public link creation (buengese)
|
|
.RE
|
|
.IP \[bu] 2
|
|
Qingstor
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Cancel in progress multipart uploads on rclone exit (Nick Craig\-Wood)
|
|
.RE
|
|
.IP \[bu] 2
|
|
S3
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Preserve metadata when doing multipart copy (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Cancel in progress multipart uploads and copies on rclone exit (Nick
|
|
Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Add \f[C]rclone link\f[R] for public link sharing (Roman Kredentser)
|
|
.IP \[bu] 2
|
|
Add \f[C]rclone backend restore\f[R] command to restore objects from
|
|
GLACIER (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Add \f[C]rclone cleanup\f[R] and \f[C]rclone backend cleanup\f[R] to
|
|
clean unfinished multipart uploads (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Add \f[C]rclone backend list\-multipart\-uploads\f[R] to list unfinished
|
|
multipart uploads (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Add \f[C]\-\-s3\-max\-upload\-parts\f[R] support (Kamil
|
|
Trzci\[u0144]ski)
|
|
.IP \[bu] 2
|
|
Add \f[C]\-\-s3\-no\-check\-bucket\f[R] for minimising rclone
|
|
transactions and perms (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Add \f[C]\-\-s3\-profile\f[R] and
|
|
\f[C]\-\-s3\-shared\-credentials\-file\f[R] options (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Use regional s3 us\-east\-1 endpoint (David)
|
|
.IP \[bu] 2
|
|
Add Scaleway provider (Vincent Feltz)
|
|
.IP \[bu] 2
|
|
Update IBM COS endpoints (Egor Margineanu)
|
|
.IP \[bu] 2
|
|
Reduce the default \f[C]\-\-s3\-copy\-cutoff\f[R] to < 5GB for Backblaze
|
|
S3 compatibility (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Fix detection of bucket existing (Nick Craig\-Wood)
|
|
.RE
|
|
.IP \[bu] 2
|
|
SFTP
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Use the absolute path instead of the relative path for listing for
|
|
improved compatibility (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Add \f[C]\-\-sftp\-subsystem\f[R] and
|
|
\f[C]\-\-sftp\-server\-command\f[R] options (aus)
|
|
.RE
|
|
.IP \[bu] 2
|
|
Swift
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Fix dangling large objects breaking the listing (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Fix purge not deleting directory markers (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Fix update multipart object removing all of its own parts (Nick
|
|
Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Fix missing hash from object returned from upload (Nick Craig\-Wood)
|
|
.RE
|
|
.IP \[bu] 2
|
|
Tardigrade
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Upgrade to uplink v1.2.0 (Kaloyan Raev)
|
|
.RE
|
|
.IP \[bu] 2
|
|
Union
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Fix writing with the all policy (Nick Craig\-Wood)
|
|
.RE
|
|
.IP \[bu] 2
|
|
WebDAV
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Fix directory creation with 4shared (Nick Craig\-Wood)
|
|
.RE
|
|
.SS v1.52.3 \- 2020\-08\-07
|
|
.PP
|
|
See commits (https://github.com/rclone/rclone/compare/v1.52.2...v1.52.3)
|
|
.IP \[bu] 2
|
|
Bug Fixes
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
docs
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Disable smart typography (e.g.
|
|
en\-dash) in MANUAL.* and man page (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Update install.md to reflect minimum Go version (Evan Harris)
|
|
.IP \[bu] 2
|
|
Update install from source instructions (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
make_manual: Support SOURCE_DATE_EPOCH (Morten Linderud)
|
|
.RE
|
|
.IP \[bu] 2
|
|
log: Fix \-\-use\-json\-log going to stderr not \-\-log\-file on Windows
|
|
(Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
serve dlna: Fix file list on Samsung Series 6+ TVs (Matteo Pietro Dazzi)
|
|
.IP \[bu] 2
|
|
sync: Fix deadlock with \-\-track\-renames\-strategy modtime (Nick
|
|
Craig\-Wood)
|
|
.RE
|
|
.IP \[bu] 2
|
|
Cache
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Fix moveto/copyto remote:file remote:file2 (Nick Craig\-Wood)
|
|
.RE
|
|
.IP \[bu] 2
|
|
Drive
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Stop using root_folder_id as a cache (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Make dangling shortcuts appear in listings (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Drop \[dq]Disabling ListR\[dq] messages down to debug (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Workaround and policy for Google Drive API (Dmitry Ustalov)
|
|
.RE
|
|
.IP \[bu] 2
|
|
FTP
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Add note to docs about home vs root directory selection (Nick
|
|
Craig\-Wood)
|
|
.RE
|
|
.IP \[bu] 2
|
|
Onedrive
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Fix reverting to Copy when Move would have worked (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Avoid comma rendered in URL in onedrive.md (Kevin)
|
|
.RE
|
|
.IP \[bu] 2
|
|
Pcloud
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Fix oauth on European region \[dq]eapi.pcloud.com\[dq] (Nick
|
|
Craig\-Wood)
|
|
.RE
|
|
.IP \[bu] 2
|
|
S3
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Fix bucket Region auto detection when Region unset in config (Nick
|
|
Craig\-Wood)
|
|
.RE
|
|
.SS v1.52.2 \- 2020\-06\-24
|
|
.PP
|
|
See commits (https://github.com/rclone/rclone/compare/v1.52.1...v1.52.2)
|
|
.IP \[bu] 2
|
|
Bug Fixes
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
build
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Fix docker release build action (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Fix custom timezone in Docker image (NoLooseEnds)
|
|
.RE
|
|
.IP \[bu] 2
|
|
check: Fix misleading message which printed errors instead of
|
|
differences (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
errors: Add WSAECONNREFUSED and more to the list of retriable Windows
|
|
errors (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
rcd: Fix incorrect prometheus metrics (Gary Kim)
|
|
.IP \[bu] 2
|
|
serve restic: Fix flags so they use environment variables (Nick
|
|
Craig\-Wood)
|
|
.IP \[bu] 2
|
|
serve webdav: Fix flags so they use environment variables (Nick
|
|
Craig\-Wood)
|
|
.IP \[bu] 2
|
|
sync: Fix \-\-track\-renames\-strategy modtime (Nick Craig\-Wood)
|
|
.RE
|
|
.IP \[bu] 2
|
|
Drive
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Fix not being able to delete a directory with a trashed shortcut (Nick
|
|
Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Fix creating a directory inside a shortcut (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Fix \-\-drive\-impersonate with cached root_folder_id (Nick Craig\-Wood)
|
|
.RE
|
|
.IP \[bu] 2
|
|
SFTP
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Fix SSH key PEM loading (Zac Rubin)
|
|
.RE
|
|
.IP \[bu] 2
|
|
Swift
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Speed up deletes by not retrying segment container deletes (Nick
|
|
Craig\-Wood)
|
|
.RE
|
|
.IP \[bu] 2
|
|
Tardigrade
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Upgrade to uplink v1.1.1 (Caleb Case)
|
|
.RE
|
|
.IP \[bu] 2
|
|
WebDAV
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Fix free/used display for rclone about/df for certain backends (Nick
|
|
Craig\-Wood)
|
|
.RE
|
|
.SS v1.52.1 \- 2020\-06\-10
|
|
.PP
|
|
See commits (https://github.com/rclone/rclone/compare/v1.52.0...v1.52.1)
|
|
.IP \[bu] 2
|
|
Bug Fixes
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
lib/file: Fix SetSparse on Windows 7 which fixes downloads of files >
|
|
250MB (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
build
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Update go.mod to go1.14 to enable \-mod=vendor build (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Remove quicktest from Dockerfile (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Build Docker images with GitHub actions (Matteo Pietro Dazzi)
|
|
.IP \[bu] 2
|
|
Update Docker build workflows (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Set user_allow_other in /etc/fuse.conf in the Docker image (Nick
|
|
Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Fix xgo build after go1.14 go.mod update (Nick Craig\-Wood)
|
|
.RE
|
|
.IP \[bu] 2
|
|
docs
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Add link to source and modified time to footer of every page (Nick
|
|
Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Remove manually set dates and use git dates instead (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Minor tense, punctuation, brevity and positivity changes for the home
|
|
page (edwardxml)
|
|
.IP \[bu] 2
|
|
Remove leading slash in page reference in footer when present (Nick
|
|
Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Note commands which need obscured input in the docs (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
obscure: Write more help as we are referencing it elsewhere (Nick
|
|
Craig\-Wood)
|
|
.RE
|
|
.RE
|
|
.IP \[bu] 2
|
|
VFS
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Fix OS vs Unix path confusion \- fixes ChangeNotify on Windows (Nick
|
|
Craig\-Wood)
|
|
.RE
|
|
.IP \[bu] 2
|
|
Drive
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Fix missing items when listing using \-\-fast\-list / ListR (Nick
|
|
Craig\-Wood)
|
|
.RE
|
|
.IP \[bu] 2
|
|
Putio
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Fix panic on Object.Open (Cenk Alti)
|
|
.RE
|
|
.IP \[bu] 2
|
|
S3
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Fix upload of single files into buckets without create permission (Nick
|
|
Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Fix \-\-header\-upload (Nick Craig\-Wood)
|
|
.RE
|
|
.IP \[bu] 2
|
|
Tardigrade
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Fix listing bug by upgrading to v1.0.7
|
|
.IP \[bu] 2
|
|
Set UserAgent to rclone (Caleb Case)
|
|
.RE
|
|
.SS v1.52.0 \- 2020\-05\-27
|
|
.PP
|
|
Special thanks to Martin Michlmayr for proof reading and correcting all
|
|
the docs and Edward Barker for helping re\-write the front page.
|
|
.PP
|
|
See commits (https://github.com/rclone/rclone/compare/v1.51.0...v1.52.0)
|
|
.IP \[bu] 2
|
|
New backends
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Tardigrade (https://rclone.org/tardigrade/) backend for use with
|
|
storj.io (Caleb Case)
|
|
.IP \[bu] 2
|
|
Union (https://rclone.org/union/) re\-write to have multiple writable
|
|
remotes (Max Sum)
|
|
.IP \[bu] 2
|
|
Seafile (/seafile) for Seafile server (Fred \[at]creativeprojects)
|
|
.RE
|
|
.IP \[bu] 2
|
|
New commands
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
backend: command for backend specific commands (see backends) (Nick
|
|
Craig\-Wood)
|
|
.IP \[bu] 2
|
|
cachestats: Deprecate in favour of \f[C]rclone backend stats cache:\f[R]
|
|
(Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
dbhashsum: Deprecate in favour of \f[C]rclone hashsum DropboxHash\f[R]
|
|
(Nick Craig\-Wood)
|
|
.RE
|
|
.IP \[bu] 2
|
|
New Features
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Add \f[C]\-\-header\-download\f[R] and \f[C]\-\-header\-upload\f[R]
|
|
flags for setting HTTP headers when uploading/downloading (Tim Gallant)
|
|
.IP \[bu] 2
|
|
Add \f[C]\-\-header\f[R] flag to add HTTP headers to every HTTP
|
|
transaction (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Add \f[C]\-\-check\-first\f[R] to do all checking before starting
|
|
transfers (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Add \f[C]\-\-track\-renames\-strategy\f[R] for configurable matching
|
|
criteria for \f[C]\-\-track\-renames\f[R] (Bernd Schoolmann)
|
|
.IP \[bu] 2
|
|
Add \f[C]\-\-cutoff\-mode\f[R] hard,soft,cautious (Shing Kit Chan &
|
|
Franklyn Tackitt)
|
|
.IP \[bu] 2
|
|
Filter flags (e.g.
|
|
\f[C]\-\-files\-from \-\f[R]) can read from stdin (fishbullet)
|
|
.IP \[bu] 2
|
|
Add \f[C]\-\-error\-on\-no\-transfer\f[R] option (Jon Fautley)
|
|
.IP \[bu] 2
|
|
Implement \f[C]\-\-order\-by xxx,mixed\f[R] for copying some small and
|
|
some big files (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Allow \f[C]\-\-max\-backlog\f[R] to be negative meaning as large as
|
|
possible (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Added \f[C]\-\-no\-unicode\-normalization\f[R] flag to allow Unicode
|
|
filenames to remain unique (Ben Zenker)
|
|
.IP \[bu] 2
|
|
Allow \f[C]\-\-min\-age\f[R]/\f[C]\-\-max\-age\f[R] to take a date as
|
|
well as a duration (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Add rename statistics for file and directory renames (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Add statistics output to JSON log (reddi)
|
|
.IP \[bu] 2
|
|
Make stats be printed on non\-zero exit code (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
When running \f[C]\-\-password\-command\f[R] allow use of stdin
|
|
(S\['e]bastien Gross)
|
|
.IP \[bu] 2
|
|
Stop empty strings being a valid remote path (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
accounting: support WriterTo for less memory copying (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
build
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Update to use go1.14 for the build (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Add \f[C]\-trimpath\f[R] to release build for reproduceable builds (Nick
|
|
Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Remove GOOS and GOARCH from Dockerfile (Brandon Philips)
|
|
.RE
|
|
.IP \[bu] 2
|
|
config
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Fsync the config file after writing to save more reliably (Nick
|
|
Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Add \f[C]\-\-obscure\f[R] and \f[C]\-\-no\-obscure\f[R] flags to
|
|
\f[C]config create\f[R]/\f[C]update\f[R] (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Make \f[C]config show\f[R] take \f[C]remote:\f[R] as well as
|
|
\f[C]remote\f[R] (Nick Craig\-Wood)
|
|
.RE
|
|
.IP \[bu] 2
|
|
copyurl: Add \f[C]\-\-no\-clobber\f[R] flag (Denis)
|
|
.IP \[bu] 2
|
|
delete: Added \f[C]\-\-rmdirs\f[R] flag to delete directories as well
|
|
(Kush)
|
|
.IP \[bu] 2
|
|
filter: Added \f[C]\-\-files\-from\-raw\f[R] flag (Ankur Gupta)
|
|
.IP \[bu] 2
|
|
genautocomplete: Add support for fish shell (Matan Rosenberg)
|
|
.IP \[bu] 2
|
|
log: Add support for syslog LOCAL facilities (Patryk Jakuszew)
|
|
.IP \[bu] 2
|
|
lsjson: Add \f[C]\-\-hash\-type\f[R] parameter and use it in lsf to
|
|
speed up hashing (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
rc
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Add \f[C]\-o\f[R]/\f[C]\-\-opt\f[R] and \f[C]\-a\f[R]/\f[C]\-\-arg\f[R]
|
|
for more structured input (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Implement \f[C]backend/command\f[R] for running backend specific
|
|
commands remotely (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Add \f[C]mount/mount\f[R] command for starting \f[C]rclone mount\f[R]
|
|
via the API (Chaitanya)
|
|
.RE
|
|
.IP \[bu] 2
|
|
rcd: Add Prometheus metrics support (Gary Kim)
|
|
.IP \[bu] 2
|
|
serve http
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Added a \f[C]\-\-template\f[R] flag for user defined markup (calistri)
|
|
.IP \[bu] 2
|
|
Add Last\-Modified headers to files and directories (Nick Craig\-Wood)
|
|
.RE
|
|
.IP \[bu] 2
|
|
serve sftp: Add support for multiple host keys by repeating
|
|
\f[C]\-\-key\f[R] flag (Maxime Suret)
|
|
.IP \[bu] 2
|
|
touch: Add \f[C]\-\-localtime\f[R] flag to make \f[C]\-\-timestamp\f[R]
|
|
localtime not UTC (Nick Craig\-Wood)
|
|
.RE
|
|
.IP \[bu] 2
|
|
Bug Fixes
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
accounting
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Restore \[dq]Max number of stats groups reached\[dq] log line
|
|
(Micha\[/l] Matczuk)
|
|
.IP \[bu] 2
|
|
Correct exitcode on Transfer Limit Exceeded flag.
|
|
(Anuar Serdaliyev)
|
|
.IP \[bu] 2
|
|
Reset bytes read during copy retry (Ankur Gupta)
|
|
.IP \[bu] 2
|
|
Fix race clearing stats (Nick Craig\-Wood)
|
|
.RE
|
|
.IP \[bu] 2
|
|
copy: Only create empty directories when they don\[aq]t exist on the
|
|
remote (Ishuah Kariuki)
|
|
.IP \[bu] 2
|
|
dedupe: Stop dedupe deleting files with identical IDs (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
oauth
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Use custom http client so that \f[C]\-\-no\-check\-certificate\f[R] is
|
|
honored by oauth token fetch (Mark Spieth)
|
|
.IP \[bu] 2
|
|
Replace deprecated oauth2.NoContext (Lars Lehtonen)
|
|
.RE
|
|
.IP \[bu] 2
|
|
operations
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Fix setting the timestamp on Windows for multithread copy (Nick
|
|
Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Make rcat obey \f[C]\-\-ignore\-checksum\f[R] (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Make \f[C]\-\-max\-transfer\f[R] more accurate (Nick Craig\-Wood)
|
|
.RE
|
|
.IP \[bu] 2
|
|
rc
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Fix dropped error (Lars Lehtonen)
|
|
.IP \[bu] 2
|
|
Fix misplaced http server config (Xiaoxing Ye)
|
|
.IP \[bu] 2
|
|
Disable duplicate log (ElonH)
|
|
.RE
|
|
.IP \[bu] 2
|
|
serve dlna
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Cds: don\[aq]t specify childCount at all when unknown (Dan Walters)
|
|
.IP \[bu] 2
|
|
Cds: use modification time as date in dlna metadata (Dan Walters)
|
|
.RE
|
|
.IP \[bu] 2
|
|
serve restic: Fix tests after restic project removed vendoring (Nick
|
|
Craig\-Wood)
|
|
.IP \[bu] 2
|
|
sync
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Fix incorrect \[dq]nothing to transfer\[dq] message using
|
|
\f[C]\-\-delete\-before\f[R] (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Only create empty directories when they don\[aq]t exist on the remote
|
|
(Ishuah Kariuki)
|
|
.RE
|
|
.RE
|
|
.IP \[bu] 2
|
|
Mount
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Add \f[C]\-\-async\-read\f[R] flag to disable asynchronous reads (Nick
|
|
Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Ignore \f[C]\-\-allow\-root\f[R] flag with a warning as it has been
|
|
removed upstream (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Warn if \f[C]\-\-allow\-non\-empty\f[R] used on Windows and clarify docs
|
|
(Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Constrain to go1.13 or above otherwise bazil.org/fuse fails to compile
|
|
(Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Fix fail because of too long volume name (evileye)
|
|
.IP \[bu] 2
|
|
Report 1PB free for unknown disk sizes (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Map more rclone errors into file systems errors (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Fix disappearing cwd problem (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Use ReaddirPlus on Windows to improve directory listing performance
|
|
(Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Send a hint as to whether the filesystem is case insensitive or not
|
|
(Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Add rc command \f[C]mount/types\f[R] (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Change maximum leaf name length to 1024 bytes (Nick Craig\-Wood)
|
|
.RE
|
|
.IP \[bu] 2
|
|
VFS
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Add \f[C]\-\-vfs\-read\-wait\f[R] and \f[C]\-\-vfs\-write\-wait\f[R]
|
|
flags to control time waiting for a sequential read/write (Nick
|
|
Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Change default \f[C]\-\-vfs\-read\-wait\f[R] to 20ms (it was 5ms and not
|
|
configurable) (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Make \f[C]df\f[R] output more consistent on a rclone mount.
|
|
(Yves G)
|
|
.IP \[bu] 2
|
|
Report 1PB free for unknown disk sizes (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Fix race condition caused by unlocked reading of Dir.path (Nick
|
|
Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Make File lock and Dir lock not overlap to avoid deadlock (Nick
|
|
Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Implement lock ordering between File and Dir to eliminate deadlocks
|
|
(Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Factor the vfs cache into its own package (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Pin the Fs in use in the Fs cache (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Add SetSys() methods to Node to allow caching stuff on a node (Nick
|
|
Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Ignore file not found errors from Hash in Read.Release (Nick
|
|
Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Fix hang in read wait code (Nick Craig\-Wood)
|
|
.RE
|
|
.IP \[bu] 2
|
|
Local
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Speed up multi thread downloads by using sparse files on Windows (Nick
|
|
Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Implement \f[C]\-\-local\-no\-sparse\f[R] flag for disabling sparse
|
|
files (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Implement \f[C]rclone backend noop\f[R] for testing purposes (Nick
|
|
Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Fix \[dq]file not found\[dq] errors on post transfer Hash calculation
|
|
(Nick Craig\-Wood)
|
|
.RE
|
|
.IP \[bu] 2
|
|
Cache
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Implement \f[C]rclone backend stats\f[R] command (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Fix Server Side Copy with Temp Upload (Brandon McNama)
|
|
.IP \[bu] 2
|
|
Remove Unused Functions (Lars Lehtonen)
|
|
.IP \[bu] 2
|
|
Disable race tests until bbolt is fixed (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Move methods used for testing into test file (greatroar)
|
|
.IP \[bu] 2
|
|
Add Pin and Unpin and canonicalised lookup (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Use proper import path go.etcd.io/bbolt (Robert\-Andr\['e] Mauchin)
|
|
.RE
|
|
.IP \[bu] 2
|
|
Crypt
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Calculate hashes for uploads from local disk (Nick Craig\-Wood)
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
This allows crypted Jottacloud uploads without using local disk
|
|
.IP \[bu] 2
|
|
This means crypted s3/b2 uploads will now have hashes
|
|
.RE
|
|
.IP \[bu] 2
|
|
Added \f[C]rclone backend decode\f[R]/\f[C]encode\f[R] commands to
|
|
replicate functionality of \f[C]cryptdecode\f[R] (Anagh Kumar Baranwal)
|
|
.IP \[bu] 2
|
|
Get rid of the unused Cipher interface as it obfuscated the code (Nick
|
|
Craig\-Wood)
|
|
.RE
|
|
.IP \[bu] 2
|
|
Azure Blob
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Implement streaming of unknown sized files so \f[C]rcat\f[R] is now
|
|
supported (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Implement memory pooling to control memory use (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Add \f[C]\-\-azureblob\-disable\-checksum\f[R] flag (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Retry \f[C]InvalidBlobOrBlock\f[R] error as it may indicate block
|
|
concurrency problems (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Remove unused \f[C]Object.parseTimeString()\f[R] (Lars Lehtonen)
|
|
.IP \[bu] 2
|
|
Fix permission error on SAS URL limited to container (Nick Craig\-Wood)
|
|
.RE
|
|
.IP \[bu] 2
|
|
B2
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Add support for \f[C]\-\-header\-upload\f[R] and
|
|
\f[C]\-\-header\-download\f[R] (Tim Gallant)
|
|
.IP \[bu] 2
|
|
Ignore directory markers at the root also (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Force the case of the SHA1 to lowercase (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Remove unused \f[C]largeUpload.clearUploadURL()\f[R] (Lars Lehtonen)
|
|
.RE
|
|
.IP \[bu] 2
|
|
Box
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Add support for \f[C]\-\-header\-upload\f[R] and
|
|
\f[C]\-\-header\-download\f[R] (Tim Gallant)
|
|
.IP \[bu] 2
|
|
Implement About to read size used (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Add token renew function for jwt auth (David Bramwell)
|
|
.IP \[bu] 2
|
|
Added support for interchangeable root folder for Box backend (Sunil
|
|
Patra)
|
|
.IP \[bu] 2
|
|
Remove unnecessary iat from jws claims (David)
|
|
.RE
|
|
.IP \[bu] 2
|
|
Drive
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Follow shortcuts by default, skip with
|
|
\f[C]\-\-drive\-skip\-shortcuts\f[R] (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Implement \f[C]rclone backend shortcut\f[R] command for creating
|
|
shortcuts (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Added \f[C]rclone backend\f[R] command to change
|
|
\f[C]service_account_file\f[R] and \f[C]chunk_size\f[R] (Anagh Kumar
|
|
Baranwal)
|
|
.IP \[bu] 2
|
|
Fix missing files when using \f[C]\-\-fast\-list\f[R] and
|
|
\f[C]\-\-drive\-shared\-with\-me\f[R] (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Fix duplicate items when using \f[C]\-\-drive\-shared\-with\-me\f[R]
|
|
(Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Extend \f[C]\-\-drive\-stop\-on\-upload\-limit\f[R] to respond to
|
|
\f[C]teamDriveFileLimitExceeded\f[R].
|
|
(harry)
|
|
.IP \[bu] 2
|
|
Don\[aq]t delete files with multiple parents to avoid data loss (Nick
|
|
Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Server side copy docs use default description if empty (Nick
|
|
Craig\-Wood)
|
|
.RE
|
|
.IP \[bu] 2
|
|
Dropbox
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Make error insufficient space to be fatal (harry)
|
|
.IP \[bu] 2
|
|
Add info about required redirect url (Elan Ruusam\[:a]e)
|
|
.RE
|
|
.IP \[bu] 2
|
|
Fichier
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Add support for \f[C]\-\-header\-upload\f[R] and
|
|
\f[C]\-\-header\-download\f[R] (Tim Gallant)
|
|
.IP \[bu] 2
|
|
Implement custom pacer to deal with the new rate limiting (buengese)
|
|
.RE
|
|
.IP \[bu] 2
|
|
FTP
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Fix lockup when using concurrency limit on failed connections (Nick
|
|
Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Fix lockup on failed upload when using concurrency limit (Nick
|
|
Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Fix lockup on Close failures when using concurrency limit (Nick
|
|
Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Work around pureftp sending spurious 150 messages (Nick Craig\-Wood)
|
|
.RE
|
|
.IP \[bu] 2
|
|
Google Cloud Storage
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Add support for \f[C]\-\-header\-upload\f[R] and
|
|
\f[C]\-\-header\-download\f[R] (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Add \f[C]ARCHIVE\f[R] storage class to help (Adam Stroud)
|
|
.IP \[bu] 2
|
|
Ignore directory markers at the root (Nick Craig\-Wood)
|
|
.RE
|
|
.IP \[bu] 2
|
|
Googlephotos
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Make the start year configurable (Daven)
|
|
.IP \[bu] 2
|
|
Add support for \f[C]\-\-header\-upload\f[R] and
|
|
\f[C]\-\-header\-download\f[R] (Tim Gallant)
|
|
.IP \[bu] 2
|
|
Create feature/favorites directory (Brandon Philips)
|
|
.IP \[bu] 2
|
|
Fix \[dq]concurrent map write\[dq] error (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Don\[aq]t put an image in error message (Nick Craig\-Wood)
|
|
.RE
|
|
.IP \[bu] 2
|
|
HTTP
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Improved directory listing with new template from Caddy project
|
|
(calisro)
|
|
.RE
|
|
.IP \[bu] 2
|
|
Jottacloud
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Implement \f[C]\-\-jottacloud\-trashed\-only\f[R] (buengese)
|
|
.IP \[bu] 2
|
|
Add support for \f[C]\-\-header\-upload\f[R] and
|
|
\f[C]\-\-header\-download\f[R] (Tim Gallant)
|
|
.IP \[bu] 2
|
|
Use \f[C]RawURLEncoding\f[R] when decoding base64 encoded login token
|
|
(buengese)
|
|
.IP \[bu] 2
|
|
Implement cleanup (buengese)
|
|
.IP \[bu] 2
|
|
Update docs regarding cleanup, removed remains from old auth, and added
|
|
warning about special mountpoints.
|
|
(albertony)
|
|
.RE
|
|
.IP \[bu] 2
|
|
Mailru
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Describe 2FA requirements (valery1707)
|
|
.RE
|
|
.IP \[bu] 2
|
|
Onedrive
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Implement \f[C]\-\-onedrive\-server\-side\-across\-configs\f[R] (Nick
|
|
Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Add support for \f[C]\-\-header\-upload\f[R] and
|
|
\f[C]\-\-header\-download\f[R] (Tim Gallant)
|
|
.IP \[bu] 2
|
|
Fix occasional 416 errors on multipart uploads (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Added maximum chunk size limit warning in the docs (Harry)
|
|
.IP \[bu] 2
|
|
Fix missing drive on config (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Make error \f[C]quotaLimitReached\f[R] to be fatal (harry)
|
|
.RE
|
|
.IP \[bu] 2
|
|
Opendrive
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Add support for \f[C]\-\-header\-upload\f[R] and
|
|
\f[C]\-\-header\-download\f[R] (Tim Gallant)
|
|
.RE
|
|
.IP \[bu] 2
|
|
Pcloud
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Added support for interchangeable root folder for pCloud backend (Sunil
|
|
Patra)
|
|
.IP \[bu] 2
|
|
Add support for \f[C]\-\-header\-upload\f[R] and
|
|
\f[C]\-\-header\-download\f[R] (Tim Gallant)
|
|
.IP \[bu] 2
|
|
Fix initial config \[dq]Auth state doesn\[aq]t match\[dq] message (Nick
|
|
Craig\-Wood)
|
|
.RE
|
|
.IP \[bu] 2
|
|
Premiumizeme
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Add support for \f[C]\-\-header\-upload\f[R] and
|
|
\f[C]\-\-header\-download\f[R] (Tim Gallant)
|
|
.IP \[bu] 2
|
|
Prune unused functions (Lars Lehtonen)
|
|
.RE
|
|
.IP \[bu] 2
|
|
Putio
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Add support for \f[C]\-\-header\-upload\f[R] and
|
|
\f[C]\-\-header\-download\f[R] (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Make downloading files use the rclone http Client (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Fix parsing of remotes with leading and trailing / (Nick Craig\-Wood)
|
|
.RE
|
|
.IP \[bu] 2
|
|
Qingstor
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Make \f[C]rclone cleanup\f[R] remove pending multipart uploads older
|
|
than 24h (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Try harder to cancel failed multipart uploads (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Prune \f[C]multiUploader.list()\f[R] (Lars Lehtonen)
|
|
.IP \[bu] 2
|
|
Lint fix (Lars Lehtonen)
|
|
.RE
|
|
.IP \[bu] 2
|
|
S3
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Add support for \f[C]\-\-header\-upload\f[R] and
|
|
\f[C]\-\-header\-download\f[R] (Tim Gallant)
|
|
.IP \[bu] 2
|
|
Use memory pool for buffer allocations (Maciej Zimnoch)
|
|
.IP \[bu] 2
|
|
Add SSE\-C support for AWS, Ceph, and MinIO (Jack Anderson)
|
|
.IP \[bu] 2
|
|
Fail fast multipart upload (Micha\[/l] Matczuk)
|
|
.IP \[bu] 2
|
|
Report errors on bucket creation (mkdir) correctly (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Specify that Minio supports URL encoding in listings (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Added 500 as retryErrorCode (Micha\[/l] Matczuk)
|
|
.IP \[bu] 2
|
|
Use \f[C]\-\-low\-level\-retries\f[R] as the number of SDK retries
|
|
(Aleksandar Jankovi\['c])
|
|
.IP \[bu] 2
|
|
Fix multipart abort context (Aleksandar Jankovic)
|
|
.IP \[bu] 2
|
|
Replace deprecated \f[C]session.New()\f[R] with
|
|
\f[C]session.NewSession()\f[R] (Lars Lehtonen)
|
|
.IP \[bu] 2
|
|
Use the provided size parameter when allocating a new memory pool
|
|
(Joachim Brandon LeBlanc)
|
|
.IP \[bu] 2
|
|
Use rclone\[aq]s low level retries instead of AWS SDK to fix listing
|
|
retries (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Ignore directory markers at the root also (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Use single memory pool (Micha\[/l] Matczuk)
|
|
.IP \[bu] 2
|
|
Do not resize buf on put to memBuf (Micha\[/l] Matczuk)
|
|
.IP \[bu] 2
|
|
Improve docs for \f[C]\-\-s3\-disable\-checksum\f[R] (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Don\[aq]t leak memory or tokens in edge cases for multipart upload (Nick
|
|
Craig\-Wood)
|
|
.RE
|
|
.IP \[bu] 2
|
|
Seafile
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Implement 2FA (Fred)
|
|
.RE
|
|
.IP \[bu] 2
|
|
SFTP
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Added \f[C]\-\-sftp\-pem\-key\f[R] to support inline key files (calisro)
|
|
.IP \[bu] 2
|
|
Fix post transfer copies failing with 0 size when using
|
|
\f[C]set_modtime=false\f[R] (Nick Craig\-Wood)
|
|
.RE
|
|
.IP \[bu] 2
|
|
Sharefile
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Add support for \f[C]\-\-header\-upload\f[R] and
|
|
\f[C]\-\-header\-download\f[R] (Tim Gallant)
|
|
.RE
|
|
.IP \[bu] 2
|
|
Sugarsync
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Add support for \f[C]\-\-header\-upload\f[R] and
|
|
\f[C]\-\-header\-download\f[R] (Tim Gallant)
|
|
.RE
|
|
.IP \[bu] 2
|
|
Swift
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Add support for \f[C]\-\-header\-upload\f[R] and
|
|
\f[C]\-\-header\-download\f[R] (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Fix cosmetic issue in error message (Martin Michlmayr)
|
|
.RE
|
|
.IP \[bu] 2
|
|
Union
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Implement multiple writable remotes (Max Sum)
|
|
.IP \[bu] 2
|
|
Fix server\-side copy (Max Sum)
|
|
.IP \[bu] 2
|
|
Implement ListR (Max Sum)
|
|
.IP \[bu] 2
|
|
Enable ListR when upstreams contain local (Max Sum)
|
|
.RE
|
|
.IP \[bu] 2
|
|
WebDAV
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Add support for \f[C]\-\-header\-upload\f[R] and
|
|
\f[C]\-\-header\-download\f[R] (Tim Gallant)
|
|
.IP \[bu] 2
|
|
Fix \f[C]X\-OC\-Mtime\f[R] header for Transip compatibility (Nick
|
|
Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Report full and consistent usage with \f[C]about\f[R] (Yves G)
|
|
.RE
|
|
.IP \[bu] 2
|
|
Yandex
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Add support for \f[C]\-\-header\-upload\f[R] and
|
|
\f[C]\-\-header\-download\f[R] (Tim Gallant)
|
|
.RE
|
|
.SS v1.51.0 \- 2020\-02\-01
|
|
.IP \[bu] 2
|
|
New backends
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Memory (https://rclone.org/memory/) (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Sugarsync (https://rclone.org/sugarsync/) (Nick Craig\-Wood)
|
|
.RE
|
|
.IP \[bu] 2
|
|
New Features
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Adjust all backends to have \f[C]\-\-backend\-encoding\f[R] parameter
|
|
(Nick Craig\-Wood)
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
this enables the encoding for special characters to be adjusted or
|
|
disabled
|
|
.RE
|
|
.IP \[bu] 2
|
|
Add \f[C]\-\-max\-duration\f[R] flag to control the maximum duration of
|
|
a transfer session (boosh)
|
|
.IP \[bu] 2
|
|
Add \f[C]\-\-expect\-continue\-timeout\f[R] flag, default 1s (Nick
|
|
Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Add \f[C]\-\-no\-check\-dest\f[R] flag for copying without testing the
|
|
destination (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Implement \f[C]\-\-order\-by\f[R] flag to order transfers (Nick
|
|
Craig\-Wood)
|
|
.IP \[bu] 2
|
|
accounting
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Don\[aq]t show entries in both transferring and checking (Nick
|
|
Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Add option to delete stats (Aleksandar Jankovic)
|
|
.RE
|
|
.IP \[bu] 2
|
|
build
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Compress the test builds with gzip (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Implement a framework for starting test servers during tests (Nick
|
|
Craig\-Wood)
|
|
.RE
|
|
.IP \[bu] 2
|
|
cmd: Always print elapsed time to tenth place seconds in progress (Gary
|
|
Kim)
|
|
.IP \[bu] 2
|
|
config
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Add \f[C]\-\-password\-command\f[R] to allow dynamic config password
|
|
(Damon Permezel)
|
|
.IP \[bu] 2
|
|
Give config questions default values (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Check a remote exists when creating a new one (Nick Craig\-Wood)
|
|
.RE
|
|
.IP \[bu] 2
|
|
copyurl: Add \f[C]\-\-stdout\f[R] flag to write to stdout (Nick
|
|
Craig\-Wood)
|
|
.IP \[bu] 2
|
|
dedupe: Implement keep smallest too (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
hashsum: Add flag \f[C]\-\-base64\f[R] flag (landall)
|
|
.IP \[bu] 2
|
|
lsf: Speed up on s3/swift/etc by not reading mimetype by default (Nick
|
|
Craig\-Wood)
|
|
.IP \[bu] 2
|
|
lsjson: Add \f[C]\-\-no\-mimetype\f[R] flag (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
rc: Add methods to turn on blocking and mutex profiling (Nick
|
|
Craig\-Wood)
|
|
.IP \[bu] 2
|
|
rcd
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Adding group parameter to stats (Chaitanya)
|
|
.IP \[bu] 2
|
|
Move webgui apart; option to disable browser (Xiaoxing Ye)
|
|
.RE
|
|
.IP \[bu] 2
|
|
serve sftp: Add support for public key with auth proxy (Paul Tinsley)
|
|
.IP \[bu] 2
|
|
stats: Show deletes in stats and hide zero stats (anuar45)
|
|
.RE
|
|
.IP \[bu] 2
|
|
Bug Fixes
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
accounting
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Fix error counter counting multiple times (Ankur Gupta)
|
|
.IP \[bu] 2
|
|
Fix error count shown as checks (Cnly)
|
|
.IP \[bu] 2
|
|
Clear finished transfer in stats\-reset (Maciej Zimnoch)
|
|
.IP \[bu] 2
|
|
Added StatsInfo locking in statsGroups sum function (Micha\[/l] Matczuk)
|
|
.RE
|
|
.IP \[bu] 2
|
|
asyncreader: Fix EOF error (buengese)
|
|
.IP \[bu] 2
|
|
check: Fix \f[C]\-\-one\-way\f[R] recursing more directories than it
|
|
needs to (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
chunkedreader: Disable hash calculation for first segment (Nick
|
|
Craig\-Wood)
|
|
.IP \[bu] 2
|
|
config
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Do not open browser on headless on drive/gcs/google photos (Xiaoxing Ye)
|
|
.IP \[bu] 2
|
|
SetValueAndSave ignore error if config section does not exist yet
|
|
(buengese)
|
|
.RE
|
|
.IP \[bu] 2
|
|
cmd: Fix completion with an encrypted config (Danil Semelenov)
|
|
.IP \[bu] 2
|
|
dbhashsum: Stop it returning UNSUPPORTED on dropbox (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
dedupe: Add missing modes to help string (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
operations
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Fix dedupe continuing on errors like insufficientFilePersimmon
|
|
(SezalAgrawal)
|
|
.IP \[bu] 2
|
|
Clear accounting before low level retry (Maciej Zimnoch)
|
|
.IP \[bu] 2
|
|
Write debug message when hashes could not be checked (Ole Sch\[:u]tt)
|
|
.IP \[bu] 2
|
|
Move interface assertion to tests to remove pflag dependency (Nick
|
|
Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Make NewOverrideObjectInfo public and factor uses (Nick Craig\-Wood)
|
|
.RE
|
|
.IP \[bu] 2
|
|
proxy: Replace use of bcrypt with sha256 (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
vendor
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Update bazil.org/fuse to fix FreeBSD 12.1 (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Update github.com/t3rm1n4l/go\-mega to fix mega \[dq]illegal base64 data
|
|
at input byte 22\[dq] (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Update termbox\-go to fix ncdu command on FreeBSD (Kuang\-che Wu)
|
|
.IP \[bu] 2
|
|
Update t3rm1n4l/go\-mega \- fixes mega: couldn\[aq]t login: crypto/aes:
|
|
invalid key size 0 (Nick Craig\-Wood)
|
|
.RE
|
|
.RE
|
|
.IP \[bu] 2
|
|
Mount
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Enable async reads for a 20% speedup (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Replace use of WriteAt with Write for cache mode >= writes and O_APPEND
|
|
(Brett Dutro)
|
|
.IP \[bu] 2
|
|
Make sure we call unmount when exiting (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Don\[aq]t build on go1.10 as bazil/fuse no longer supports it (Nick
|
|
Craig\-Wood)
|
|
.IP \[bu] 2
|
|
When setting dates discard out of range dates (Nick Craig\-Wood)
|
|
.RE
|
|
.IP \[bu] 2
|
|
VFS
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Add a newly created file straight into the directory (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Only calculate one hash for reads for a speedup (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Make ReadAt for non cached files work better with non\-sequential reads
|
|
(Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Fix edge cases when reading ModTime from file (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Make sure existing files opened for write show correct size (Nick
|
|
Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Don\[aq]t cache the path in RW file objects to fix renaming (Nick
|
|
Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Fix rename of open files when using the VFS cache (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
When renaming files in the cache, rename the cache item in memory too
|
|
(Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Fix open file renaming on drive when using
|
|
\f[C]\-\-vfs\-cache\-mode writes\f[R] (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Fix incorrect modtime for mv into mount with
|
|
\f[C]\-\-vfs\-cache\-modes writes\f[R] (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
On rename, rename in cache too if the file exists (Anagh Kumar Baranwal)
|
|
.RE
|
|
.IP \[bu] 2
|
|
Local
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Make source file being updated errors be NoLowLevelRetry errors (Nick
|
|
Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Fix update of hidden files on Windows (Nick Craig\-Wood)
|
|
.RE
|
|
.IP \[bu] 2
|
|
Cache
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Follow move of upstream library github.com/coreos/bbolt
|
|
github.com/etcd\-io/bbolt (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Fix \f[C]fatal error: concurrent map writes\f[R] (Nick Craig\-Wood)
|
|
.RE
|
|
.IP \[bu] 2
|
|
Crypt
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Reorder the filename encryption options (Thomas Eales)
|
|
.IP \[bu] 2
|
|
Correctly handle trailing dot (buengese)
|
|
.RE
|
|
.IP \[bu] 2
|
|
Chunker
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Reduce length of temporary suffix (Ivan Andreev)
|
|
.RE
|
|
.IP \[bu] 2
|
|
Drive
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Add \f[C]\-\-drive\-stop\-on\-upload\-limit\f[R] flag to stop syncs when
|
|
upload limit reached (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Add \f[C]\-\-drive\-use\-shared\-date\f[R] to use date file was shared
|
|
instead of modified date (Garry McNulty)
|
|
.IP \[bu] 2
|
|
Make sure invalid auth for teamdrives always reports an error (Nick
|
|
Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Fix \f[C]\-\-fast\-list\f[R] when using appDataFolder (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Use multipart resumable uploads for streaming and uploads in mount (Nick
|
|
Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Log an ERROR if an incomplete search is returned (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Hide dangerous config from the configurator (Nick Craig\-Wood)
|
|
.RE
|
|
.IP \[bu] 2
|
|
Dropbox
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Treat \f[C]insufficient_space\f[R] errors as non retriable errors (Nick
|
|
Craig\-Wood)
|
|
.RE
|
|
.IP \[bu] 2
|
|
Jottacloud
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Use new auth method used by official client (buengese)
|
|
.IP \[bu] 2
|
|
Add URL to generate Login Token to config wizard (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Add support whitelabel versions (buengese)
|
|
.RE
|
|
.IP \[bu] 2
|
|
Koofr
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Use rclone HTTP client.
|
|
(jaKa)
|
|
.RE
|
|
.IP \[bu] 2
|
|
Onedrive
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Add Sites.Read.All permission (Benjamin Richter)
|
|
.IP \[bu] 2
|
|
Add support \[dq]Retry\-After\[dq] header (Motonori IWAMURO)
|
|
.RE
|
|
.IP \[bu] 2
|
|
Opendrive
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Implement \f[C]\-\-opendrive\-chunk\-size\f[R] (Nick Craig\-Wood)
|
|
.RE
|
|
.IP \[bu] 2
|
|
S3
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Re\-implement multipart upload to fix memory issues (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Add \f[C]\-\-s3\-copy\-cutoff\f[R] for size to switch to multipart copy
|
|
(Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Add new region Asia Pacific (Hong Kong) (Outvi V)
|
|
.IP \[bu] 2
|
|
Reduce memory usage streaming files by reducing max stream upload size
|
|
(Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Add \f[C]\-\-s3\-list\-chunk\f[R] option for bucket listing (Thomas
|
|
Kriechbaumer)
|
|
.IP \[bu] 2
|
|
Force path style bucket access to off for AWS deprecation (Nick
|
|
Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Use AWS web identity role provider if available (Tennix)
|
|
.IP \[bu] 2
|
|
Add StackPath Object Storage Support (Dave Koston)
|
|
.IP \[bu] 2
|
|
Fix ExpiryWindow value (Aleksandar Jankovic)
|
|
.IP \[bu] 2
|
|
Fix DisableChecksum condition (Aleksandar Jankovi\['c])
|
|
.IP \[bu] 2
|
|
Fix URL decoding of NextMarker (Nick Craig\-Wood)
|
|
.RE
|
|
.IP \[bu] 2
|
|
SFTP
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Add \f[C]\-\-sftp\-skip\-links\f[R] to skip symlinks and non regular
|
|
files (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Retry Creation of Connection (Sebastian Brandt)
|
|
.IP \[bu] 2
|
|
Fix \[dq]failed to parse private key file: ssh: not an encrypted
|
|
key\[dq] error (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Open files for update write only to fix AWS SFTP interop (Nick
|
|
Craig\-Wood)
|
|
.RE
|
|
.IP \[bu] 2
|
|
Swift
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Reserve segments of dynamic large object when delete objects in
|
|
container what was enabled versioning.
|
|
(Nguy\[u1EC5]n H\[u1EEF]u Lu\[^a]n)
|
|
.IP \[bu] 2
|
|
Fix parsing of X\-Object\-Manifest (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Update OVH API endpoint (unbelauscht)
|
|
.RE
|
|
.IP \[bu] 2
|
|
WebDAV
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Make nextcloud only upload SHA1 checksums (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Fix case of \[dq]Bearer\[dq] in Authorization: header to agree with RFC
|
|
(Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Add Referer header to fix problems with WAFs (Nick Craig\-Wood)
|
|
.RE
|
|
.SS v1.50.2 \- 2019\-11\-19
|
|
.IP \[bu] 2
|
|
Bug Fixes
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
accounting: Fix memory leak on retries operations (Nick Craig\-Wood)
|
|
.RE
|
|
.IP \[bu] 2
|
|
Drive
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Fix listing of the root directory with drive.files scope (Nick
|
|
Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Fix \-\-drive\-root\-folder\-id with team/shared drives (Nick
|
|
Craig\-Wood)
|
|
.RE
|
|
.SS v1.50.1 \- 2019\-11\-02
|
|
.IP \[bu] 2
|
|
Bug Fixes
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
hash: Fix accidentally changed hash names for \f[C]DropboxHash\f[R] and
|
|
\f[C]CRC\-32\f[R] (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
fshttp: Fix error reporting on tpslimit token bucket errors (Nick
|
|
Craig\-Wood)
|
|
.IP \[bu] 2
|
|
fshttp: Don\[aq]t print token bucket errors on context cancelled (Nick
|
|
Craig\-Wood)
|
|
.RE
|
|
.IP \[bu] 2
|
|
Local
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Fix listings of .
|
|
on Windows (Nick Craig\-Wood)
|
|
.RE
|
|
.IP \[bu] 2
|
|
Onedrive
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Fix DirMove/Move after Onedrive change (Xiaoxing Ye)
|
|
.RE
|
|
.SS v1.50.0 \- 2019\-10\-26
|
|
.IP \[bu] 2
|
|
New backends
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Citrix Sharefile (https://rclone.org/sharefile/) (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Chunker (https://rclone.org/chunker/) \- an overlay backend to split
|
|
files into smaller parts (Ivan Andreev)
|
|
.IP \[bu] 2
|
|
Mail.ru Cloud (https://rclone.org/mailru/) (Ivan Andreev)
|
|
.RE
|
|
.IP \[bu] 2
|
|
New Features
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
encodings (Fabian M\[:o]ller & Nick Craig\-Wood)
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
All backends now use file name encoding to ensure any file name can be
|
|
written to any backend.
|
|
.IP \[bu] 2
|
|
See the restricted file name
|
|
docs (https://rclone.org/overview/#restricted-filenames) for more info
|
|
and the local backend docs (/local/#filenames).
|
|
.IP \[bu] 2
|
|
Some file names may look different in rclone if you are using any
|
|
control characters in names or unicode FULLWIDTH
|
|
symbols (https://en.wikipedia.org/wiki/Halfwidth_and_Fullwidth_Forms_(Unicode_block)).
|
|
.RE
|
|
.IP \[bu] 2
|
|
build
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Update to use go1.13 for the build (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Drop support for go1.9 (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Build rclone with GitHub actions (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Convert python scripts to python3 (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Swap Azure/go\-ansiterm for mattn/go\-colorable (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Dockerfile fixes (Matei David)
|
|
.IP \[bu] 2
|
|
Add plugin
|
|
support (https://github.com/rclone/rclone/blob/master/CONTRIBUTING.md#writing-a-plugin)
|
|
for backends and commands (Richard Patel)
|
|
.RE
|
|
.IP \[bu] 2
|
|
config
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Use alternating Red/Green in config to make more obvious (Nick
|
|
Craig\-Wood)
|
|
.RE
|
|
.IP \[bu] 2
|
|
contrib
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Add sample DLNA server Docker Compose manifest.
|
|
(pataquets)
|
|
.IP \[bu] 2
|
|
Add sample WebDAV server Docker Compose manifest.
|
|
(pataquets)
|
|
.RE
|
|
.IP \[bu] 2
|
|
copyurl
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Add \f[C]\-\-auto\-filename\f[R] flag for using file name from URL in
|
|
destination path (Denis)
|
|
.RE
|
|
.IP \[bu] 2
|
|
serve dlna:
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Many compatibility improvements (Dan Walters)
|
|
.IP \[bu] 2
|
|
Support for external srt subtitles (Dan Walters)
|
|
.RE
|
|
.IP \[bu] 2
|
|
rc
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Added command core/quit (Saksham Khanna)
|
|
.RE
|
|
.RE
|
|
.IP \[bu] 2
|
|
Bug Fixes
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
sync
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Make \f[C]\-\-update\f[R]/\f[C]\-u\f[R] not transfer files that
|
|
haven\[aq]t changed (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Free objects after they come out of the transfer pipe to save memory
|
|
(Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Fix \f[C]\-\-files\-from without \-\-no\-traverse\f[R] doing a recursive
|
|
scan (Nick Craig\-Wood)
|
|
.RE
|
|
.IP \[bu] 2
|
|
operations
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Fix accounting for server\-side copies (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Display \[aq]All duplicates removed\[aq] only if dedupe successful
|
|
(Sezal Agrawal)
|
|
.IP \[bu] 2
|
|
Display \[aq]Deleted X extra copies\[aq] only if dedupe successful
|
|
(Sezal Agrawal)
|
|
.RE
|
|
.IP \[bu] 2
|
|
accounting
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Only allow up to 100 completed transfers in the accounting list to save
|
|
memory (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Cull the old time ranges when possible to save memory (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Fix panic due to server\-side copy fallback (Ivan Andreev)
|
|
.IP \[bu] 2
|
|
Fix memory leak noticeable for transfers of large numbers of objects
|
|
(Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Fix total duration calculation (Nick Craig\-Wood)
|
|
.RE
|
|
.IP \[bu] 2
|
|
cmd
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Fix environment variables not setting command line flags (Nick
|
|
Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Make autocomplete compatible with bash\[aq]s posix mode for macOS (Danil
|
|
Semelenov)
|
|
.IP \[bu] 2
|
|
Make \f[C]\-\-progress\f[R] work in git bash on Windows (Nick
|
|
Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Fix \[aq]compopt: command not found\[aq] on autocomplete on macOS (Danil
|
|
Semelenov)
|
|
.RE
|
|
.IP \[bu] 2
|
|
config
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Fix setting of non top level flags from environment variables (Nick
|
|
Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Check config names more carefully and report errors (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Remove error: can\[aq]t use \f[C]\-\-size\-only\f[R] and
|
|
\f[C]\-\-ignore\-size\f[R] together.
|
|
(Nick Craig\-Wood)
|
|
.RE
|
|
.IP \[bu] 2
|
|
filter: Prevent mixing options when \f[C]\-\-files\-from\f[R] is in use
|
|
(Michele Caci)
|
|
.IP \[bu] 2
|
|
serve sftp: Fix crash on unsupported operations (e.g.
|
|
Readlink) (Nick Craig\-Wood)
|
|
.RE
|
|
.IP \[bu] 2
|
|
Mount
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Allow files of unknown size to be read properly (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Skip tests on <= 2 CPUs to avoid lockup (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Fix panic on File.Open (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Fix \[dq]mount_fusefs: \-o timeout=: option not supported\[dq] on
|
|
FreeBSD (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Don\[aq]t pass huge filenames (>4k) to FUSE as it can\[aq]t cope (Nick
|
|
Craig\-Wood)
|
|
.RE
|
|
.IP \[bu] 2
|
|
VFS
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Add flag \f[C]\-\-vfs\-case\-insensitive\f[R] for windows/macOS mounts
|
|
(Ivan Andreev)
|
|
.IP \[bu] 2
|
|
Make objects of unknown size readable through the VFS (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Move writeback of dirty data out of close() method into its own method
|
|
(FlushWrites) and remove close() call from Flush() (Brett Dutro)
|
|
.IP \[bu] 2
|
|
Stop empty dirs disappearing when renamed on bucket based remotes (Nick
|
|
Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Stop change notify polling clearing so much of the directory cache (Nick
|
|
Craig\-Wood)
|
|
.RE
|
|
.IP \[bu] 2
|
|
Azure Blob
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Disable logging to the Windows event log (Nick Craig\-Wood)
|
|
.RE
|
|
.IP \[bu] 2
|
|
B2
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Remove \f[C]unverified:\f[R] prefix on sha1 to improve interop (e.g.
|
|
with CyberDuck) (Nick Craig\-Wood)
|
|
.RE
|
|
.IP \[bu] 2
|
|
Box
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Add options to get access token via JWT auth (David)
|
|
.RE
|
|
.IP \[bu] 2
|
|
Drive
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Disable HTTP/2 by default to work around INTERNAL_ERROR problems (Nick
|
|
Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Make sure that drive root ID is always canonical (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Fix \f[C]\-\-drive\-shared\-with\-me\f[R] from the root with lsand
|
|
\f[C]\-\-fast\-list\f[R] (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Fix ChangeNotify polling for shared drives (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Fix change notify polling when using appDataFolder (Nick Craig\-Wood)
|
|
.RE
|
|
.IP \[bu] 2
|
|
Dropbox
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Make disallowed filenames errors not retry (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Fix nil pointer exception on restricted files (Nick Craig\-Wood)
|
|
.RE
|
|
.IP \[bu] 2
|
|
Fichier
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Fix accessing files > 2GB on 32 bit systems (Nick Craig\-Wood)
|
|
.RE
|
|
.IP \[bu] 2
|
|
FTP
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Allow disabling EPSV mode (Jon Fautley)
|
|
.RE
|
|
.IP \[bu] 2
|
|
HTTP
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
HEAD directory entries in parallel to speedup (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Add \f[C]\-\-http\-no\-head\f[R] to stop rclone doing HEAD in listings
|
|
(Nick Craig\-Wood)
|
|
.RE
|
|
.IP \[bu] 2
|
|
Putio
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Add ability to resume uploads (Cenk Alti)
|
|
.RE
|
|
.IP \[bu] 2
|
|
S3
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Fix signature v2_auth headers (Anthony Rusdi)
|
|
.IP \[bu] 2
|
|
Fix encoding for control characters (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Only ask for URL encoded directory listings if we need them on Ceph
|
|
(Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Add option for multipart failure behaviour (Aleksandar Jankovic)
|
|
.IP \[bu] 2
|
|
Support for multipart copy (\[u5E84]\[u5929]\[u7FFC])
|
|
.IP \[bu] 2
|
|
Fix nil pointer reference if no metadata returned for object (Nick
|
|
Craig\-Wood)
|
|
.RE
|
|
.IP \[bu] 2
|
|
SFTP
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Fix \f[C]\-\-sftp\-ask\-password\f[R] trying to contact the ssh agent
|
|
(Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Fix hashes of files with backslashes (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Include more ciphers with \f[C]\-\-sftp\-use\-insecure\-cipher\f[R]
|
|
(Carlos Ferreyra)
|
|
.RE
|
|
.IP \[bu] 2
|
|
WebDAV
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Parse and return Sharepoint error response (Henning Surmeier)
|
|
.RE
|
|
.SS v1.49.5 \- 2019\-10\-05
|
|
.IP \[bu] 2
|
|
Bug Fixes
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Revert back to go1.12.x for the v1.49.x builds as go1.13.x was causing
|
|
issues (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Fix rpm packages by using master builds of nfpm (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Fix macOS build after brew changes (Nick Craig\-Wood)
|
|
.RE
|
|
.SS v1.49.4 \- 2019\-09\-29
|
|
.IP \[bu] 2
|
|
Bug Fixes
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
cmd/rcd: Address ZipSlip vulnerability (Richard Patel)
|
|
.IP \[bu] 2
|
|
accounting: Fix file handle leak on errors (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
oauthutil: Fix security problem when running with two users on the same
|
|
machine (Nick Craig\-Wood)
|
|
.RE
|
|
.IP \[bu] 2
|
|
FTP
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Fix listing of an empty root returning: error dir not found (Nick
|
|
Craig\-Wood)
|
|
.RE
|
|
.IP \[bu] 2
|
|
S3
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Fix SetModTime on GLACIER/ARCHIVE objects and implement set/get tier
|
|
(Nick Craig\-Wood)
|
|
.RE
|
|
.SS v1.49.3 \- 2019\-09\-15
|
|
.IP \[bu] 2
|
|
Bug Fixes
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
accounting
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Fix total duration calculation (Aleksandar Jankovic)
|
|
.IP \[bu] 2
|
|
Fix \[dq]file already closed\[dq] on transfer retries (Nick Craig\-Wood)
|
|
.RE
|
|
.RE
|
|
.SS v1.49.2 \- 2019\-09\-08
|
|
.IP \[bu] 2
|
|
New Features
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
build: Add Docker workflow support (Alfonso Montero)
|
|
.RE
|
|
.IP \[bu] 2
|
|
Bug Fixes
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
accounting: Fix locking in Transfer to avoid deadlock with
|
|
\f[C]\-\-progress\f[R] (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
docs: Fix template argument for mktemp in install.sh (Cnly)
|
|
.IP \[bu] 2
|
|
operations: Fix \f[C]\-u\f[R]/\f[C]\-\-update\f[R] with google photos /
|
|
files of unknown size (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
rc: Fix docs for config/create /update /password (Nick Craig\-Wood)
|
|
.RE
|
|
.IP \[bu] 2
|
|
Google Cloud Storage
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Fix need for elevated permissions on SetModTime (Nick Craig\-Wood)
|
|
.RE
|
|
.SS v1.49.1 \- 2019\-08\-28
|
|
.IP \[bu] 2
|
|
Bug Fixes
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
config: Fix generated passwords being stored as empty password (Nick
|
|
Craig\-Wood)
|
|
.IP \[bu] 2
|
|
rcd: Added missing parameter for web\-gui info logs.
|
|
(Chaitanya)
|
|
.RE
|
|
.IP \[bu] 2
|
|
Googlephotos
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Fix crash on error response (Nick Craig\-Wood)
|
|
.RE
|
|
.IP \[bu] 2
|
|
Onedrive
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Fix crash on error response (Nick Craig\-Wood)
|
|
.RE
|
|
.SS v1.49.0 \- 2019\-08\-26
|
|
.IP \[bu] 2
|
|
New backends
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
1fichier (https://rclone.org/fichier/) (Laura Hausmann)
|
|
.IP \[bu] 2
|
|
Google Photos (https://rclone.org/googlephotos/) (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Putio (https://rclone.org/putio/) (Cenk Alti)
|
|
.IP \[bu] 2
|
|
premiumize.me (https://rclone.org/premiumizeme/) (Nick Craig\-Wood)
|
|
.RE
|
|
.IP \[bu] 2
|
|
New Features
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Experimental web GUI (https://rclone.org/gui/) (Chaitanya Bankanhal)
|
|
.IP \[bu] 2
|
|
Implement \f[C]\-\-compare\-dest\f[R] & \f[C]\-\-copy\-dest\f[R]
|
|
(yparitcher)
|
|
.IP \[bu] 2
|
|
Implement \f[C]\-\-suffix\f[R] without \f[C]\-\-backup\-dir\f[R] for
|
|
backup to current dir (yparitcher)
|
|
.IP \[bu] 2
|
|
\f[C]config reconnect\f[R] to re\-login (re\-run the oauth login) for
|
|
the backend.
|
|
(Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
\f[C]config userinfo\f[R] to discover which user you are logged in as.
|
|
(Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
\f[C]config disconnect\f[R] to disconnect you (log out) from the
|
|
backend.
|
|
(Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Add \f[C]\-\-use\-json\-log\f[R] for JSON logging (justinalin)
|
|
.IP \[bu] 2
|
|
Add context propagation to rclone (Aleksandar Jankovic)
|
|
.IP \[bu] 2
|
|
Reworking internal statistics interfaces so they work with rc jobs
|
|
(Aleksandar Jankovic)
|
|
.IP \[bu] 2
|
|
Add Higher units for ETA (AbelThar)
|
|
.IP \[bu] 2
|
|
Update rclone logos to new design (Andreas Chlupka)
|
|
.IP \[bu] 2
|
|
hash: Add CRC\-32 support (Cenk Alti)
|
|
.IP \[bu] 2
|
|
help showbackend: Fixed advanced option category when there are no
|
|
standard options (buengese)
|
|
.IP \[bu] 2
|
|
ncdu: Display/Copy to Clipboard Current Path (Gary Kim)
|
|
.IP \[bu] 2
|
|
operations:
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Run hashing operations in parallel (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Don\[aq]t calculate checksums when using \f[C]\-\-ignore\-checksum\f[R]
|
|
(Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Check transfer hashes when using \f[C]\-\-size\-only\f[R] mode (Nick
|
|
Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Disable multi thread copy for local to local copies (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Debug successful hashes as well as failures (Nick Craig\-Wood)
|
|
.RE
|
|
.IP \[bu] 2
|
|
rc
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Add ability to stop async jobs (Aleksandar Jankovic)
|
|
.IP \[bu] 2
|
|
Return current settings if core/bwlimit called without parameters (Nick
|
|
Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Rclone\-WebUI integration with rclone (Chaitanya Bankanhal)
|
|
.IP \[bu] 2
|
|
Added command line parameter to control the cross origin resource
|
|
sharing (CORS) in the rcd.
|
|
(Security Improvement) (Chaitanya Bankanhal)
|
|
.IP \[bu] 2
|
|
Add anchor tags to the docs so links are consistent (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Remove _async key from input parameters after parsing so later
|
|
operations won\[aq]t get confused (buengese)
|
|
.IP \[bu] 2
|
|
Add call to clear stats (Aleksandar Jankovic)
|
|
.RE
|
|
.IP \[bu] 2
|
|
rcd
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Auto\-login for web\-gui (Chaitanya Bankanhal)
|
|
.IP \[bu] 2
|
|
Implement \f[C]\-\-baseurl\f[R] for rcd and web\-gui (Chaitanya
|
|
Bankanhal)
|
|
.RE
|
|
.IP \[bu] 2
|
|
serve dlna
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Only select interfaces which can multicast for SSDP (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Add more builtin mime types to cover standard audio/video (Nick
|
|
Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Fix missing mime types on Android causing missing videos (Nick
|
|
Craig\-Wood)
|
|
.RE
|
|
.IP \[bu] 2
|
|
serve ftp
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Refactor to bring into line with other serve commands (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Implement \f[C]\-\-auth\-proxy\f[R] (Nick Craig\-Wood)
|
|
.RE
|
|
.IP \[bu] 2
|
|
serve http: Implement \f[C]\-\-baseurl\f[R] (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
serve restic: Implement \f[C]\-\-baseurl\f[R] (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
serve sftp
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Implement auth proxy (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Fix detection of whether server is authorized (Nick Craig\-Wood)
|
|
.RE
|
|
.IP \[bu] 2
|
|
serve webdav
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Implement \f[C]\-\-baseurl\f[R] (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Support \f[C]\-\-auth\-proxy\f[R] (Nick Craig\-Wood)
|
|
.RE
|
|
.RE
|
|
.IP \[bu] 2
|
|
Bug Fixes
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Make \[dq]bad record MAC\[dq] a retriable error (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
copyurl: Fix copying files that return HTTP errors (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
march: Fix checking sub\-directories when using
|
|
\f[C]\-\-no\-traverse\f[R] (buengese)
|
|
.IP \[bu] 2
|
|
rc
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Fix unmarshalable http.AuthFn in options and put in test for
|
|
marshalability (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Move job expire flags to rc to fix initialization problem (Nick
|
|
Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Fix \f[C]\-\-loopback\f[R] with rc/list and others (Nick Craig\-Wood)
|
|
.RE
|
|
.IP \[bu] 2
|
|
rcat: Fix slowdown on systems with multiple hashes (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
rcd: Fix permissions problems on cache directory with web gui download
|
|
(Nick Craig\-Wood)
|
|
.RE
|
|
.IP \[bu] 2
|
|
Mount
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Default \f[C]\-\-daemon\-timout\f[R] to 15 minutes on macOS and FreeBSD
|
|
(Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Update docs to show mounting from root OK for bucket based (Nick
|
|
Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Remove nonseekable flag from write files (Nick Craig\-Wood)
|
|
.RE
|
|
.IP \[bu] 2
|
|
VFS
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Make write without cache more efficient (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Fix \f[C]\-\-vfs\-cache\-mode minimal\f[R] and \f[C]writes\f[R] ignoring
|
|
cached files (Nick Craig\-Wood)
|
|
.RE
|
|
.IP \[bu] 2
|
|
Local
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Add \f[C]\-\-local\-case\-sensitive\f[R] and
|
|
\f[C]\-\-local\-case\-insensitive\f[R] (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Avoid polluting page cache when uploading local files to remote backends
|
|
(Micha\[/l] Matczuk)
|
|
.IP \[bu] 2
|
|
Don\[aq]t calculate any hashes by default (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Fadvise run syscall on a dedicated go routine (Micha\[/l] Matczuk)
|
|
.RE
|
|
.IP \[bu] 2
|
|
Azure Blob
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Azure Storage Emulator support (Sandeep)
|
|
.IP \[bu] 2
|
|
Updated config help details to remove connection string references
|
|
(Sandeep)
|
|
.IP \[bu] 2
|
|
Make all operations work from the root (Nick Craig\-Wood)
|
|
.RE
|
|
.IP \[bu] 2
|
|
B2
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Implement link sharing (yparitcher)
|
|
.IP \[bu] 2
|
|
Enable server\-side copy to copy between buckets (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Make all operations work from the root (Nick Craig\-Wood)
|
|
.RE
|
|
.IP \[bu] 2
|
|
Drive
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Fix server\-side copy of big files (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Update API for teamdrive use (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Add error for purge with \f[C]\-\-drive\-trashed\-only\f[R] (ginvine)
|
|
.RE
|
|
.IP \[bu] 2
|
|
Fichier
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Make FolderID int and adjust related code (buengese)
|
|
.RE
|
|
.IP \[bu] 2
|
|
Google Cloud Storage
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Reduce oauth scope requested as suggested by Google (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Make all operations work from the root (Nick Craig\-Wood)
|
|
.RE
|
|
.IP \[bu] 2
|
|
HTTP
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Add \f[C]\-\-http\-headers\f[R] flag for setting arbitrary headers (Nick
|
|
Craig\-Wood)
|
|
.RE
|
|
.IP \[bu] 2
|
|
Jottacloud
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Use new api for retrieving internal username (buengese)
|
|
.IP \[bu] 2
|
|
Refactor configuration and minor cleanup (buengese)
|
|
.RE
|
|
.IP \[bu] 2
|
|
Koofr
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Support setting modification times on Koofr backend.
|
|
(jaKa)
|
|
.RE
|
|
.IP \[bu] 2
|
|
Opendrive
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Refactor to use existing lib/rest facilities for uploads (Nick
|
|
Craig\-Wood)
|
|
.RE
|
|
.IP \[bu] 2
|
|
Qingstor
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Upgrade to v3 SDK and fix listing loop (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Make all operations work from the root (Nick Craig\-Wood)
|
|
.RE
|
|
.IP \[bu] 2
|
|
S3
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Add INTELLIGENT_TIERING storage class (Matti Niemenmaa)
|
|
.IP \[bu] 2
|
|
Make all operations work from the root (Nick Craig\-Wood)
|
|
.RE
|
|
.IP \[bu] 2
|
|
SFTP
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Add missing interface check and fix About (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Completely ignore all modtime checks if SetModTime=false (Jon Fautley)
|
|
.IP \[bu] 2
|
|
Support md5/sha1 with rsync.net (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Save the md5/sha1 command in use to the config file for efficiency (Nick
|
|
Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Opt\-in support for diffie\-hellman\-group\-exchange\-sha256
|
|
diffie\-hellman\-group\-exchange\-sha1 (Yi FU)
|
|
.RE
|
|
.IP \[bu] 2
|
|
Swift
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Use FixRangeOption to fix 0 length files via the VFS (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Fix upload when using no_chunk to return the correct size (Nick
|
|
Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Make all operations work from the root (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Fix segments leak during failed large file uploads.
|
|
(nguyenhuuluan434)
|
|
.RE
|
|
.IP \[bu] 2
|
|
WebDAV
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Add \f[C]\-\-webdav\-bearer\-token\-command\f[R] (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Refresh token when it expires with
|
|
\f[C]\-\-webdav\-bearer\-token\-command\f[R] (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Add docs for using bearer_token_command with oidc\-agent (Paul Millar)
|
|
.RE
|
|
.SS v1.48.0 \- 2019\-06\-15
|
|
.IP \[bu] 2
|
|
New commands
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
serve sftp: Serve an rclone remote over SFTP (Nick Craig\-Wood)
|
|
.RE
|
|
.IP \[bu] 2
|
|
New Features
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Multi threaded downloads to local storage (Nick Craig\-Wood)
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
controlled with \f[C]\-\-multi\-thread\-cutoff\f[R] and
|
|
\f[C]\-\-multi\-thread\-streams\f[R]
|
|
.RE
|
|
.IP \[bu] 2
|
|
Use rclone.conf from rclone executable directory to enable portable use
|
|
(albertony)
|
|
.IP \[bu] 2
|
|
Allow sync of a file and a directory with the same name (forgems)
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
this is common on bucket based remotes, e.g.
|
|
s3, gcs
|
|
.RE
|
|
.IP \[bu] 2
|
|
Add \f[C]\-\-ignore\-case\-sync\f[R] for forced case insensitivity
|
|
(garry415)
|
|
.IP \[bu] 2
|
|
Implement \f[C]\-\-stats\-one\-line\-date\f[R] and
|
|
\f[C]\-\-stats\-one\-line\-date\-format\f[R] (Peter Berbec)
|
|
.IP \[bu] 2
|
|
Log an ERROR for all commands which exit with non\-zero status (Nick
|
|
Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Use go\-homedir to read the home directory more reliably (Nick
|
|
Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Enable creating encrypted config through external script invocation
|
|
(Wojciech Smigielski)
|
|
.IP \[bu] 2
|
|
build: Drop support for go1.8 (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
config: Make config create/update encrypt passwords where necessary
|
|
(Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
copyurl: Honor \f[C]\-\-no\-check\-certificate\f[R] (Stefan Breunig)
|
|
.IP \[bu] 2
|
|
install: Linux skip man pages if no mandb (didil)
|
|
.IP \[bu] 2
|
|
lsf: Support showing the Tier of the object (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
lsjson
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Added EncryptedPath to output (calisro)
|
|
.IP \[bu] 2
|
|
Support showing the Tier of the object (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Add IsBucket field for bucket based remote listing of the root (Nick
|
|
Craig\-Wood)
|
|
.RE
|
|
.IP \[bu] 2
|
|
rc
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Add \f[C]\-\-loopback\f[R] flag to run commands directly without a
|
|
server (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Add operations/fsinfo: Return information about the remote (Nick
|
|
Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Skip auth for OPTIONS request (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
cmd/providers: Add DefaultStr, ValueStr and Type fields (Nick
|
|
Craig\-Wood)
|
|
.IP \[bu] 2
|
|
jobs: Make job expiry timeouts configurable (Aleksandar Jankovic)
|
|
.RE
|
|
.IP \[bu] 2
|
|
serve dlna reworked and improved (Dan Walters)
|
|
.IP \[bu] 2
|
|
serve ftp: add \f[C]\-\-ftp\-public\-ip\f[R] flag to specify public IP
|
|
(calistri)
|
|
.IP \[bu] 2
|
|
serve restic: Add support for \f[C]\-\-private\-repos\f[R] in
|
|
\f[C]serve restic\f[R] (Florian Apolloner)
|
|
.IP \[bu] 2
|
|
serve webdav: Combine serve webdav and serve http (Gary Kim)
|
|
.IP \[bu] 2
|
|
size: Ignore negative sizes when calculating total (Garry McNulty)
|
|
.RE
|
|
.IP \[bu] 2
|
|
Bug Fixes
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Make move and copy individual files obey \f[C]\-\-backup\-dir\f[R] (Nick
|
|
Craig\-Wood)
|
|
.IP \[bu] 2
|
|
If \f[C]\-\-ignore\-checksum\f[R] is in effect, don\[aq]t calculate
|
|
checksum (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
moveto: Fix case\-insensitive same remote move (Gary Kim)
|
|
.IP \[bu] 2
|
|
rc: Fix serving bucket based objects with \f[C]\-\-rc\-serve\f[R] (Nick
|
|
Craig\-Wood)
|
|
.IP \[bu] 2
|
|
serve webdav: Fix serveDir not being updated with changes from webdav
|
|
(Gary Kim)
|
|
.RE
|
|
.IP \[bu] 2
|
|
Mount
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Fix poll interval documentation (Animosity022)
|
|
.RE
|
|
.IP \[bu] 2
|
|
VFS
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Make WriteAt for non cached files work with non\-sequential writes (Nick
|
|
Craig\-Wood)
|
|
.RE
|
|
.IP \[bu] 2
|
|
Local
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Only calculate the required hashes for big speedup (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Log errors when listing instead of returning an error (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Fix preallocate warning on Linux with ZFS (Nick Craig\-Wood)
|
|
.RE
|
|
.IP \[bu] 2
|
|
Crypt
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Make rclone dedupe work through crypt (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Fix wrapping of ChangeNotify to decrypt directories properly (Nick
|
|
Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Support PublicLink (rclone link) of underlying backend (Nick
|
|
Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Implement Optional methods SetTier, GetTier (Nick Craig\-Wood)
|
|
.RE
|
|
.IP \[bu] 2
|
|
B2
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Implement server\-side copy (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Implement SetModTime (Nick Craig\-Wood)
|
|
.RE
|
|
.IP \[bu] 2
|
|
Drive
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Fix move and copy from TeamDrive to GDrive (Fionera)
|
|
.IP \[bu] 2
|
|
Add notes that cleanup works in the background on drive (Nick
|
|
Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Add \f[C]\-\-drive\-server\-side\-across\-configs\f[R] to default back
|
|
to old server\-side copy semantics by default (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Add \f[C]\-\-drive\-size\-as\-quota\f[R] to show storage quota usage for
|
|
file size (Garry McNulty)
|
|
.RE
|
|
.IP \[bu] 2
|
|
FTP
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Add FTP List timeout (Jeff Quinn)
|
|
.IP \[bu] 2
|
|
Add FTP over TLS support (Gary Kim)
|
|
.IP \[bu] 2
|
|
Add \f[C]\-\-ftp\-no\-check\-certificate\f[R] option for FTPS (Gary Kim)
|
|
.RE
|
|
.IP \[bu] 2
|
|
Google Cloud Storage
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Fix upload errors when uploading pre 1970 files (Nick Craig\-Wood)
|
|
.RE
|
|
.IP \[bu] 2
|
|
Jottacloud
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Add support for selecting device and mountpoint.
|
|
(buengese)
|
|
.RE
|
|
.IP \[bu] 2
|
|
Mega
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Add cleanup support (Gary Kim)
|
|
.RE
|
|
.IP \[bu] 2
|
|
Onedrive
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
More accurately check if root is found (Cnly)
|
|
.RE
|
|
.IP \[bu] 2
|
|
S3
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Support S3 Accelerated endpoints with
|
|
\f[C]\-\-s3\-use\-accelerate\-endpoint\f[R] (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Add config info for Wasabi\[aq]s EU Central endpoint (Robert Marko)
|
|
.IP \[bu] 2
|
|
Make SetModTime work for GLACIER while syncing (Philip Harvey)
|
|
.RE
|
|
.IP \[bu] 2
|
|
SFTP
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Add About support (Gary Kim)
|
|
.IP \[bu] 2
|
|
Fix about parsing of \f[C]df\f[R] results so it can cope with \-ve
|
|
results (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Send custom client version and debug server version (Nick Craig\-Wood)
|
|
.RE
|
|
.IP \[bu] 2
|
|
WebDAV
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Retry on 423 Locked errors (Nick Craig\-Wood)
|
|
.RE
|
|
.SS v1.47.0 \- 2019\-04\-13
|
|
.IP \[bu] 2
|
|
New backends
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Backend for Koofr cloud storage service.
|
|
(jaKa)
|
|
.RE
|
|
.IP \[bu] 2
|
|
New Features
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Resume downloads if the reader fails in copy (Nick Craig\-Wood)
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
this means rclone will restart transfers if the source has an error
|
|
.IP \[bu] 2
|
|
this is most useful for downloads or cloud to cloud copies
|
|
.RE
|
|
.IP \[bu] 2
|
|
Use \f[C]\-\-fast\-list\f[R] for listing operations where it won\[aq]t
|
|
use more memory (Nick Craig\-Wood)
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
this should speed up the following operations on remotes which support
|
|
\f[C]ListR\f[R]
|
|
.IP \[bu] 2
|
|
\f[C]dedupe\f[R], \f[C]serve restic\f[R] \f[C]lsf\f[R], \f[C]ls\f[R],
|
|
\f[C]lsl\f[R], \f[C]lsjson\f[R], \f[C]lsd\f[R], \f[C]md5sum\f[R],
|
|
\f[C]sha1sum\f[R], \f[C]hashsum\f[R], \f[C]size\f[R], \f[C]delete\f[R],
|
|
\f[C]cat\f[R], \f[C]settier\f[R]
|
|
.IP \[bu] 2
|
|
use \f[C]\-\-disable ListR\f[R] to get old behaviour if required
|
|
.RE
|
|
.IP \[bu] 2
|
|
Make \f[C]\-\-files\-from\f[R] traverse the destination unless
|
|
\f[C]\-\-no\-traverse\f[R] is set (Nick Craig\-Wood)
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
this fixes \f[C]\-\-files\-from\f[R] with Google drive and excessive API
|
|
use in general.
|
|
.RE
|
|
.IP \[bu] 2
|
|
Make server\-side copy account bytes and obey
|
|
\f[C]\-\-max\-transfer\f[R] (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Add \f[C]\-\-create\-empty\-src\-dirs\f[R] flag and default to not
|
|
creating empty dirs (ishuah)
|
|
.IP \[bu] 2
|
|
Add client side TLS/SSL flags
|
|
\f[C]\-\-ca\-cert\f[R]/\f[C]\-\-client\-cert\f[R]/\f[C]\-\-client\-key\f[R]
|
|
(Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Implement \f[C]\-\-suffix\-keep\-extension\f[R] for use with
|
|
\f[C]\-\-suffix\f[R] (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
build:
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Switch to semver compliant version tags to be go modules compliant (Nick
|
|
Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Update to use go1.12.x for the build (Nick Craig\-Wood)
|
|
.RE
|
|
.IP \[bu] 2
|
|
serve dlna: Add connection manager service description to improve
|
|
compatibility (Dan Walters)
|
|
.IP \[bu] 2
|
|
lsf: Add \[aq]e\[aq] format to show encrypted names and \[aq]o\[aq] for
|
|
original IDs (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
lsjson: Added \f[C]\-\-files\-only\f[R] and \f[C]\-\-dirs\-only\f[R]
|
|
flags (calistri)
|
|
.IP \[bu] 2
|
|
rc: Implement operations/publiclink the equivalent of
|
|
\f[C]rclone link\f[R] (Nick Craig\-Wood)
|
|
.RE
|
|
.IP \[bu] 2
|
|
Bug Fixes
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
accounting: Fix total ETA when \f[C]\-\-stats\-unit bits\f[R] is in
|
|
effect (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Bash TAB completion
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Use private custom func to fix clash between rclone and kubectl (Nick
|
|
Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Fix for remotes with underscores in their names (Six)
|
|
.IP \[bu] 2
|
|
Fix completion of remotes (Florian Gamb\[:o]ck)
|
|
.IP \[bu] 2
|
|
Fix autocompletion of remote paths with spaces (Danil Semelenov)
|
|
.RE
|
|
.IP \[bu] 2
|
|
serve dlna: Fix root XML service descriptor (Dan Walters)
|
|
.IP \[bu] 2
|
|
ncdu: Fix display corruption with Chinese characters (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Add SIGTERM to signals which run the exit handlers on unix (Nick
|
|
Craig\-Wood)
|
|
.IP \[bu] 2
|
|
rc: Reload filter when the options are set via the rc (Nick Craig\-Wood)
|
|
.RE
|
|
.IP \[bu] 2
|
|
VFS / Mount
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Fix FreeBSD: Ignore Truncate if called with no readers and already the
|
|
correct size (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Read directory and check for a file before mkdir (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Shorten the locking window for vfs/refresh (Nick Craig\-Wood)
|
|
.RE
|
|
.IP \[bu] 2
|
|
Azure Blob
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Enable MD5 checksums when uploading files bigger than the
|
|
\[dq]Cutoff\[dq] (Dr.Rx)
|
|
.IP \[bu] 2
|
|
Fix SAS URL support (Nick Craig\-Wood)
|
|
.RE
|
|
.IP \[bu] 2
|
|
B2
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Allow manual configuration of backblaze downloadUrl (Vince)
|
|
.IP \[bu] 2
|
|
Ignore already_hidden error on remove (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Ignore malformed \f[C]src_last_modified_millis\f[R] (Nick Craig\-Wood)
|
|
.RE
|
|
.IP \[bu] 2
|
|
Drive
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Add \f[C]\-\-skip\-checksum\-gphotos\f[R] to ignore incorrect checksums
|
|
on Google Photos (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Allow server\-side move/copy between different remotes.
|
|
(Fionera)
|
|
.IP \[bu] 2
|
|
Add docs on team drives and \f[C]\-\-fast\-list\f[R] eventual
|
|
consistency (Nestar47)
|
|
.IP \[bu] 2
|
|
Fix imports of text files (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Fix range requests on 0 length files (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Fix creation of duplicates with server\-side copy (Nick Craig\-Wood)
|
|
.RE
|
|
.IP \[bu] 2
|
|
Dropbox
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Retry blank errors to fix long listings (Nick Craig\-Wood)
|
|
.RE
|
|
.IP \[bu] 2
|
|
FTP
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Add \f[C]\-\-ftp\-concurrency\f[R] to limit maximum number of
|
|
connections (Nick Craig\-Wood)
|
|
.RE
|
|
.IP \[bu] 2
|
|
Google Cloud Storage
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Fall back to default application credentials (marcintustin)
|
|
.IP \[bu] 2
|
|
Allow bucket policy only buckets (Nick Craig\-Wood)
|
|
.RE
|
|
.IP \[bu] 2
|
|
HTTP
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Add \f[C]\-\-http\-no\-slash\f[R] for websites with directories with no
|
|
slashes (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Remove duplicates from listings (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Fix socket leak on 404 errors (Nick Craig\-Wood)
|
|
.RE
|
|
.IP \[bu] 2
|
|
Jottacloud
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Fix token refresh (Sebastian B\[:u]nger)
|
|
.IP \[bu] 2
|
|
Add device registration (Oliver Heyme)
|
|
.RE
|
|
.IP \[bu] 2
|
|
Onedrive
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Implement graceful cancel of multipart uploads if rclone is interrupted
|
|
(Cnly)
|
|
.IP \[bu] 2
|
|
Always add trailing colon to path when addressing items, (Cnly)
|
|
.IP \[bu] 2
|
|
Return errors instead of panic for invalid uploads (Fabian M\[:o]ller)
|
|
.RE
|
|
.IP \[bu] 2
|
|
S3
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Add support for \[dq]Glacier Deep Archive\[dq] storage class (Manu)
|
|
.IP \[bu] 2
|
|
Update Dreamhost endpoint (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Note incompatibility with CEPH Jewel (Nick Craig\-Wood)
|
|
.RE
|
|
.IP \[bu] 2
|
|
SFTP
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Allow custom ssh client config (Alexandru Bumbacea)
|
|
.RE
|
|
.IP \[bu] 2
|
|
Swift
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Obey Retry\-After to enable OVH restore from cold storage (Nick
|
|
Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Work around token expiry on CEPH (Nick Craig\-Wood)
|
|
.RE
|
|
.IP \[bu] 2
|
|
WebDAV
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Allow IsCollection property to be integer or boolean (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Fix race when creating directories (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Fix About/df when reading the available/total returns 0 (Nick
|
|
Craig\-Wood)
|
|
.RE
|
|
.SS v1.46 \- 2019\-02\-09
|
|
.IP \[bu] 2
|
|
New backends
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Support Alibaba Cloud (Aliyun) OSS via the s3 backend (Nick Craig\-Wood)
|
|
.RE
|
|
.IP \[bu] 2
|
|
New commands
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
serve dlna: serves a remove via DLNA for the local network (nicolov)
|
|
.RE
|
|
.IP \[bu] 2
|
|
New Features
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
copy, move: Restore deprecated \f[C]\-\-no\-traverse\f[R] flag (Nick
|
|
Craig\-Wood)
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
This is useful for when transferring a small number of files into a
|
|
large destination
|
|
.RE
|
|
.IP \[bu] 2
|
|
genautocomplete: Add remote path completion for bash completion
|
|
(Christopher Peterson & Danil Semelenov)
|
|
.IP \[bu] 2
|
|
Buffer memory handling reworked to return memory to the OS better (Nick
|
|
Craig\-Wood)
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Buffer recycling library to replace sync.Pool
|
|
.IP \[bu] 2
|
|
Optionally use memory mapped memory for better memory shrinking
|
|
.IP \[bu] 2
|
|
Enable with \f[C]\-\-use\-mmap\f[R] if having memory problems \- not
|
|
default yet
|
|
.RE
|
|
.IP \[bu] 2
|
|
Parallelise reading of files specified by \f[C]\-\-files\-from\f[R]
|
|
(Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
check: Add stats showing total files matched.
|
|
(Dario Guzik)
|
|
.IP \[bu] 2
|
|
Allow rename/delete open files under Windows (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
lsjson: Use exactly the correct number of decimal places in the seconds
|
|
(Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Add cookie support with cmdline switch \f[C]\-\-use\-cookies\f[R] for
|
|
all HTTP based remotes (qip)
|
|
.IP \[bu] 2
|
|
Warn if \f[C]\-\-checksum\f[R] is set but there are no hashes available
|
|
(Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Rework rate limiting (pacer) to be more accurate and allow bursting
|
|
(Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Improve error reporting for too many/few arguments in commands (Nick
|
|
Craig\-Wood)
|
|
.IP \[bu] 2
|
|
listremotes: Remove \f[C]\-l\f[R] short flag as it conflicts with the
|
|
new global flag (weetmuts)
|
|
.IP \[bu] 2
|
|
Make http serving with auth generate INFO messages on auth fail (Nick
|
|
Craig\-Wood)
|
|
.RE
|
|
.IP \[bu] 2
|
|
Bug Fixes
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Fix layout of stats (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Fix \f[C]\-\-progress\f[R] crash under Windows Jenkins (Nick
|
|
Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Fix transfer of google/onedrive docs by calling Rcat in Copy when size
|
|
is \-1 (Cnly)
|
|
.IP \[bu] 2
|
|
copyurl: Fix checking of \f[C]\-\-dry\-run\f[R] (Denis Skovpen)
|
|
.RE
|
|
.IP \[bu] 2
|
|
Mount
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Check that mountpoint and local directory to mount don\[aq]t overlap
|
|
(Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Fix mount size under 32 bit Windows (Nick Craig\-Wood)
|
|
.RE
|
|
.IP \[bu] 2
|
|
VFS
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Implement renaming of directories for backends without DirMove (Nick
|
|
Craig\-Wood)
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
now all backends except b2 support renaming directories
|
|
.RE
|
|
.IP \[bu] 2
|
|
Implement \f[C]\-\-vfs\-cache\-max\-size\f[R] to limit the total size of
|
|
the cache (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Add \f[C]\-\-dir\-perms\f[R] and \f[C]\-\-file\-perms\f[R] flags to set
|
|
default permissions (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Fix deadlock on concurrent operations on a directory (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Fix deadlock between RWFileHandle.close and File.Remove (Nick
|
|
Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Fix renaming/deleting open files with cache mode \[dq]writes\[dq] under
|
|
Windows (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Fix panic on rename with \f[C]\-\-dry\-run\f[R] set (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Fix vfs/refresh with recurse=true needing the \f[C]\-\-fast\-list\f[R]
|
|
flag
|
|
.RE
|
|
.IP \[bu] 2
|
|
Local
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Add support for \f[C]\-l\f[R]/\f[C]\-\-links\f[R] (symbolic link
|
|
translation) (yair\[at]unicorn)
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
this works by showing links as \f[C]link.rclonelink\f[R] \- see local
|
|
backend docs for more info
|
|
.IP \[bu] 2
|
|
this errors if used with \f[C]\-L\f[R]/\f[C]\-\-copy\-links\f[R]
|
|
.RE
|
|
.IP \[bu] 2
|
|
Fix renaming/deleting open files on Windows (Nick Craig\-Wood)
|
|
.RE
|
|
.IP \[bu] 2
|
|
Crypt
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Check for maximum length before decrypting filename to fix panic (Garry
|
|
McNulty)
|
|
.RE
|
|
.IP \[bu] 2
|
|
Azure Blob
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Allow building azureblob backend on *BSD (themylogin)
|
|
.IP \[bu] 2
|
|
Use the rclone HTTP client to support \f[C]\-\-dump headers\f[R],
|
|
\f[C]\-\-tpslimit\f[R], etc.
|
|
(Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Use the s3 pacer for 0 delay in non error conditions (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Ignore directory markers (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Stop Mkdir attempting to create existing containers (Nick Craig\-Wood)
|
|
.RE
|
|
.IP \[bu] 2
|
|
B2
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
cleanup: will remove unfinished large files >24hrs old (Garry McNulty)
|
|
.IP \[bu] 2
|
|
For a bucket limited application key check the bucket name (Nick
|
|
Craig\-Wood)
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
before this, rclone would use the authorised bucket regardless of what
|
|
you put on the command line
|
|
.RE
|
|
.IP \[bu] 2
|
|
Added \f[C]\-\-b2\-disable\-checksum\f[R] flag (Wojciech Smigielski)
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
this enables large files to be uploaded without a SHA\-1 hash for speed
|
|
reasons
|
|
.RE
|
|
.RE
|
|
.IP \[bu] 2
|
|
Drive
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Set default pacer to 100ms for 10 tps (Nick Craig\-Wood)
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
This fits the Google defaults much better and reduces the 403 errors
|
|
massively
|
|
.IP \[bu] 2
|
|
Add \f[C]\-\-drive\-pacer\-min\-sleep\f[R] and
|
|
\f[C]\-\-drive\-pacer\-burst\f[R] to control the pacer
|
|
.RE
|
|
.IP \[bu] 2
|
|
Improve ChangeNotify support for items with multiple parents (Fabian
|
|
M\[:o]ller)
|
|
.IP \[bu] 2
|
|
Fix ListR for items with multiple parents \- this fixes oddities with
|
|
\f[C]vfs/refresh\f[R] (Fabian M\[:o]ller)
|
|
.IP \[bu] 2
|
|
Fix using \f[C]\-\-drive\-impersonate\f[R] and appfolders (Nick
|
|
Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Fix google docs in rclone mount for some (not all) applications (Nick
|
|
Craig\-Wood)
|
|
.RE
|
|
.IP \[bu] 2
|
|
Dropbox
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Retry\-After support for Dropbox backend (Mathieu Carbou)
|
|
.RE
|
|
.IP \[bu] 2
|
|
FTP
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Wait for 60 seconds for a connection to Close then declare it dead (Nick
|
|
Craig\-Wood)
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
helps with indefinite hangs on some FTP servers
|
|
.RE
|
|
.RE
|
|
.IP \[bu] 2
|
|
Google Cloud Storage
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Update google cloud storage endpoints (weetmuts)
|
|
.RE
|
|
.IP \[bu] 2
|
|
HTTP
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Add an example with username and password which is supported but
|
|
wasn\[aq]t documented (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Fix backend with \f[C]\-\-files\-from\f[R] and non\-existent files (Nick
|
|
Craig\-Wood)
|
|
.RE
|
|
.IP \[bu] 2
|
|
Hubic
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Make error message more informative if authentication fails (Nick
|
|
Craig\-Wood)
|
|
.RE
|
|
.IP \[bu] 2
|
|
Jottacloud
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Resume and deduplication support (Oliver Heyme)
|
|
.IP \[bu] 2
|
|
Use token auth for all API requests Don\[aq]t store password anymore
|
|
(Sebastian B\[:u]nger)
|
|
.IP \[bu] 2
|
|
Add support for 2\-factor authentication (Sebastian B\[:u]nger)
|
|
.RE
|
|
.IP \[bu] 2
|
|
Mega
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Implement v2 account login which fixes logins for newer Mega accounts
|
|
(Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Return error if an unknown length file is attempted to be uploaded (Nick
|
|
Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Add new error codes for better error reporting (Nick Craig\-Wood)
|
|
.RE
|
|
.IP \[bu] 2
|
|
Onedrive
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Fix broken support for \[dq]shared with me\[dq] folders (Alex Chen)
|
|
.IP \[bu] 2
|
|
Fix root ID not normalised (Cnly)
|
|
.IP \[bu] 2
|
|
Return err instead of panic on unknown\-sized uploads (Cnly)
|
|
.RE
|
|
.IP \[bu] 2
|
|
Qingstor
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Fix go routine leak on multipart upload errors (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Add upload chunk size/concurrency/cutoff control (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Default \f[C]\-\-qingstor\-upload\-concurrency\f[R] to 1 to work around
|
|
bug (Nick Craig\-Wood)
|
|
.RE
|
|
.IP \[bu] 2
|
|
S3
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Implement \f[C]\-\-s3\-upload\-cutoff\f[R] for single part uploads below
|
|
this (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Change \f[C]\-\-s3\-upload\-concurrency\f[R] default to 4 to increase
|
|
performance (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Add \f[C]\-\-s3\-bucket\-acl\f[R] to control bucket ACL (Nick
|
|
Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Auto detect region for buckets on operation failure (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Add GLACIER storage class (William Cocker)
|
|
.IP \[bu] 2
|
|
Add Scaleway to s3 documentation (R\['e]my L\['e]one)
|
|
.IP \[bu] 2
|
|
Add AWS endpoint eu\-north\-1 (weetmuts)
|
|
.RE
|
|
.IP \[bu] 2
|
|
SFTP
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Add support for PEM encrypted private keys (Fabian M\[:o]ller)
|
|
.IP \[bu] 2
|
|
Add option to force the usage of an ssh\-agent (Fabian M\[:o]ller)
|
|
.IP \[bu] 2
|
|
Perform environment variable expansion on key\-file (Fabian M\[:o]ller)
|
|
.IP \[bu] 2
|
|
Fix rmdir on Windows based servers (e.g.
|
|
CrushFTP) (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Fix rmdir deleting directory contents on some SFTP servers (Nick
|
|
Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Fix error on dangling symlinks (Nick Craig\-Wood)
|
|
.RE
|
|
.IP \[bu] 2
|
|
Swift
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Add \f[C]\-\-swift\-no\-chunk\f[R] to disable segmented uploads in
|
|
rcat/mount (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Introduce application credential auth support (kayrus)
|
|
.IP \[bu] 2
|
|
Fix memory usage by slimming Object (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Fix extra requests on upload (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Fix reauth on big files (Nick Craig\-Wood)
|
|
.RE
|
|
.IP \[bu] 2
|
|
Union
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Fix poll\-interval not working (Nick Craig\-Wood)
|
|
.RE
|
|
.IP \[bu] 2
|
|
WebDAV
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Support About which means rclone mount will show the correct disk size
|
|
(Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Support MD5 and SHA1 hashes with Owncloud and Nextcloud (Nick
|
|
Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Fail soft on time parsing errors (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Fix infinite loop on failed directory creation (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Fix identification of directories for Bitrix Site Manager (Nick
|
|
Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Fix upload of 0 length files on some servers (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Fix if MKCOL fails with 423 Locked assume the directory exists (Nick
|
|
Craig\-Wood)
|
|
.RE
|
|
.SS v1.45 \- 2018\-11\-24
|
|
.IP \[bu] 2
|
|
New backends
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
The Yandex backend was re\-written \- see below for details (Sebastian
|
|
B\[:u]nger)
|
|
.RE
|
|
.IP \[bu] 2
|
|
New commands
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
rcd: New command just to serve the remote control API (Nick Craig\-Wood)
|
|
.RE
|
|
.IP \[bu] 2
|
|
New Features
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
The remote control API (rc) was greatly expanded to allow full control
|
|
over rclone (Nick Craig\-Wood)
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
sensitive operations require authorization or the
|
|
\f[C]\-\-rc\-no\-auth\f[R] flag
|
|
.IP \[bu] 2
|
|
config/* operations to configure rclone
|
|
.IP \[bu] 2
|
|
options/* for reading/setting command line flags
|
|
.IP \[bu] 2
|
|
operations/* for all low level operations, e.g.
|
|
copy file, list directory
|
|
.IP \[bu] 2
|
|
sync/* for sync, copy and move
|
|
.IP \[bu] 2
|
|
\f[C]\-\-rc\-files\f[R] flag to serve files on the rc http server
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
this is for building web native GUIs for rclone
|
|
.RE
|
|
.IP \[bu] 2
|
|
Optionally serving objects on the rc http server
|
|
.IP \[bu] 2
|
|
Ensure rclone fails to start up if the \f[C]\-\-rc\f[R] port is in use
|
|
already
|
|
.IP \[bu] 2
|
|
See the rc docs (https://rclone.org/rc/) for more info
|
|
.RE
|
|
.IP \[bu] 2
|
|
sync/copy/move
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Make \f[C]\-\-files\-from\f[R] only read the objects specified and
|
|
don\[aq]t scan directories (Nick Craig\-Wood)
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
This is a huge speed improvement for destinations with lots of files
|
|
.RE
|
|
.RE
|
|
.IP \[bu] 2
|
|
filter: Add \f[C]\-\-ignore\-case\f[R] flag (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
ncdu: Add remove function (\[aq]d\[aq] key) (Henning Surmeier)
|
|
.IP \[bu] 2
|
|
rc command
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Add \f[C]\-\-json\f[R] flag for structured JSON input (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Add \f[C]\-\-user\f[R] and \f[C]\-\-pass\f[R] flags and interpret
|
|
\f[C]\-\-rc\-user\f[R], \f[C]\-\-rc\-pass\f[R], \f[C]\-\-rc\-addr\f[R]
|
|
(Nick Craig\-Wood)
|
|
.RE
|
|
.IP \[bu] 2
|
|
build
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Require go1.8 or later for compilation (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Enable softfloat on MIPS arch (Scott Edlund)
|
|
.IP \[bu] 2
|
|
Integration test framework revamped with a better report and better
|
|
retries (Nick Craig\-Wood)
|
|
.RE
|
|
.RE
|
|
.IP \[bu] 2
|
|
Bug Fixes
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
cmd: Make \f[C]\-\-progress\f[R] update the stats correctly at the end
|
|
(Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
config: Create config directory on save if it is missing (Nick
|
|
Craig\-Wood)
|
|
.IP \[bu] 2
|
|
dedupe: Check for existing filename before renaming a dupe file (ssaqua)
|
|
.IP \[bu] 2
|
|
move: Don\[aq]t create directories with \f[C]\-\-dry\-run\f[R] (Nick
|
|
Craig\-Wood)
|
|
.IP \[bu] 2
|
|
operations: Fix Purge and Rmdirs when dir is not the root (Nick
|
|
Craig\-Wood)
|
|
.IP \[bu] 2
|
|
serve http/webdav/restic: Ensure rclone exits if the port is in use
|
|
(Nick Craig\-Wood)
|
|
.RE
|
|
.IP \[bu] 2
|
|
Mount
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Make \f[C]\-\-volname\f[R] work for Windows and macOS (Nick Craig\-Wood)
|
|
.RE
|
|
.IP \[bu] 2
|
|
Azure Blob
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Avoid context deadline exceeded error by setting a large TryTimeout
|
|
value (brused27)
|
|
.IP \[bu] 2
|
|
Fix erroneous Rmdir error \[dq]directory not empty\[dq] (Nick
|
|
Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Wait for up to 60s to create a just deleted container (Nick Craig\-Wood)
|
|
.RE
|
|
.IP \[bu] 2
|
|
Dropbox
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Add dropbox impersonate support (Jake Coggiano)
|
|
.RE
|
|
.IP \[bu] 2
|
|
Jottacloud
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Fix bug in \f[C]\-\-fast\-list\f[R] handing of empty folders (albertony)
|
|
.RE
|
|
.IP \[bu] 2
|
|
Opendrive
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Fix transfer of files with \f[C]+\f[R] and \f[C]&\f[R] in (Nick
|
|
Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Fix retries of upload chunks (Nick Craig\-Wood)
|
|
.RE
|
|
.IP \[bu] 2
|
|
S3
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Set ACL for server\-side copies to that provided by the user (Nick
|
|
Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Fix role_arn, credential_source, ...
|
|
(Erik Swanson)
|
|
.IP \[bu] 2
|
|
Add config info for Wasabi\[aq]s US\-West endpoint (Henry Ptasinski)
|
|
.RE
|
|
.IP \[bu] 2
|
|
SFTP
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Ensure file hash checking is really disabled (Jon Fautley)
|
|
.RE
|
|
.IP \[bu] 2
|
|
Swift
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Add pacer for retries to make swift more reliable (Nick Craig\-Wood)
|
|
.RE
|
|
.IP \[bu] 2
|
|
WebDAV
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Add Content\-Type to PUT requests (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Fix config parsing so \f[C]\-\-webdav\-user\f[R] and
|
|
\f[C]\-\-webdav\-pass\f[R] flags work (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Add RFC3339 date format (Ralf Hemberger)
|
|
.RE
|
|
.IP \[bu] 2
|
|
Yandex
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
The yandex backend was re\-written (Sebastian B\[:u]nger)
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
This implements low level retries (Sebastian B\[:u]nger)
|
|
.IP \[bu] 2
|
|
Copy, Move, DirMove, PublicLink and About optional interfaces (Sebastian
|
|
B\[:u]nger)
|
|
.IP \[bu] 2
|
|
Improved general error handling (Sebastian B\[:u]nger)
|
|
.IP \[bu] 2
|
|
Removed ListR for now due to inconsistent behaviour (Sebastian
|
|
B\[:u]nger)
|
|
.RE
|
|
.RE
|
|
.SS v1.44 \- 2018\-10\-15
|
|
.IP \[bu] 2
|
|
New commands
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
serve ftp: Add ftp server (Antoine GIRARD)
|
|
.IP \[bu] 2
|
|
settier: perform storage tier changes on supported remotes (sandeepkru)
|
|
.RE
|
|
.IP \[bu] 2
|
|
New Features
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Reworked command line help
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Make default help less verbose (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Split flags up into global and backend flags (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Implement specialised help for flags and backends (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Show URL of backend help page when starting config (Nick Craig\-Wood)
|
|
.RE
|
|
.IP \[bu] 2
|
|
stats: Long names now split in center (Joanna Marek)
|
|
.IP \[bu] 2
|
|
Add \f[C]\-\-log\-format\f[R] flag for more control over log output
|
|
(dcpu)
|
|
.IP \[bu] 2
|
|
rc: Add support for OPTIONS and basic CORS (frenos)
|
|
.IP \[bu] 2
|
|
stats: show FatalErrors and NoRetryErrors in stats (C\['e]dric Connes)
|
|
.RE
|
|
.IP \[bu] 2
|
|
Bug Fixes
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Fix \-P not ending with a new line (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
config: don\[aq]t create default config dir when user supplies
|
|
\f[C]\-\-config\f[R] (albertony)
|
|
.IP \[bu] 2
|
|
Don\[aq]t print non\-ASCII characters with \f[C]\-\-progress\f[R] on
|
|
windows (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Correct logs for excluded items (ssaqua)
|
|
.RE
|
|
.IP \[bu] 2
|
|
Mount
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Remove EXPERIMENTAL tags (Nick Craig\-Wood)
|
|
.RE
|
|
.IP \[bu] 2
|
|
VFS
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Fix race condition detected by serve ftp tests (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Add vfs/poll\-interval rc command (Fabian M\[:o]ller)
|
|
.IP \[bu] 2
|
|
Enable rename for nearly all remotes using server\-side Move or Copy
|
|
(Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Reduce directory cache cleared by poll\-interval (Fabian M\[:o]ller)
|
|
.IP \[bu] 2
|
|
Remove EXPERIMENTAL tags (Nick Craig\-Wood)
|
|
.RE
|
|
.IP \[bu] 2
|
|
Local
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Skip bad symlinks in dir listing with \-L enabled (C\['e]dric Connes)
|
|
.IP \[bu] 2
|
|
Preallocate files on Windows to reduce fragmentation (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Preallocate files on linux with fallocate(2) (Nick Craig\-Wood)
|
|
.RE
|
|
.IP \[bu] 2
|
|
Cache
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Add cache/fetch rc function (Fabian M\[:o]ller)
|
|
.IP \[bu] 2
|
|
Fix worker scale down (Fabian M\[:o]ller)
|
|
.IP \[bu] 2
|
|
Improve performance by not sending info requests for cached chunks
|
|
(dcpu)
|
|
.IP \[bu] 2
|
|
Fix error return value of cache/fetch rc method (Fabian M\[:o]ller)
|
|
.IP \[bu] 2
|
|
Documentation fix for cache\-chunk\-total\-size (Anagh Kumar Baranwal)
|
|
.IP \[bu] 2
|
|
Preserve leading / in wrapped remote path (Fabian M\[:o]ller)
|
|
.IP \[bu] 2
|
|
Add plex_insecure option to skip certificate validation (Fabian
|
|
M\[:o]ller)
|
|
.IP \[bu] 2
|
|
Remove entries that no longer exist in the source (dcpu)
|
|
.RE
|
|
.IP \[bu] 2
|
|
Crypt
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Preserve leading / in wrapped remote path (Fabian M\[:o]ller)
|
|
.RE
|
|
.IP \[bu] 2
|
|
Alias
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Fix handling of Windows network paths (Nick Craig\-Wood)
|
|
.RE
|
|
.IP \[bu] 2
|
|
Azure Blob
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Add \f[C]\-\-azureblob\-list\-chunk\f[R] parameter (Santiago
|
|
Rodr\['i]guez)
|
|
.IP \[bu] 2
|
|
Implemented settier command support on azureblob remote.
|
|
(sandeepkru)
|
|
.IP \[bu] 2
|
|
Work around SDK bug which causes errors for chunk\-sized files (Nick
|
|
Craig\-Wood)
|
|
.RE
|
|
.IP \[bu] 2
|
|
Box
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Implement link sharing.
|
|
(Sebastian B\[:u]nger)
|
|
.RE
|
|
.IP \[bu] 2
|
|
Drive
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Add \f[C]\-\-drive\-import\-formats\f[R] \- google docs can now be
|
|
imported (Fabian M\[:o]ller)
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Rewrite mime type and extension handling (Fabian M\[:o]ller)
|
|
.IP \[bu] 2
|
|
Add document links (Fabian M\[:o]ller)
|
|
.IP \[bu] 2
|
|
Add support for multipart document extensions (Fabian M\[:o]ller)
|
|
.IP \[bu] 2
|
|
Add support for apps\-script to json export (Fabian M\[:o]ller)
|
|
.IP \[bu] 2
|
|
Fix escaped chars in documents during list (Fabian M\[:o]ller)
|
|
.RE
|
|
.IP \[bu] 2
|
|
Add \f[C]\-\-drive\-v2\-download\-min\-size\f[R] a workaround for slow
|
|
downloads (Fabian M\[:o]ller)
|
|
.IP \[bu] 2
|
|
Improve directory notifications in ChangeNotify (Fabian M\[:o]ller)
|
|
.IP \[bu] 2
|
|
When listing team drives in config, continue on failure (Nick
|
|
Craig\-Wood)
|
|
.RE
|
|
.IP \[bu] 2
|
|
FTP
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Add a small pause after failed upload before deleting file (Nick
|
|
Craig\-Wood)
|
|
.RE
|
|
.IP \[bu] 2
|
|
Google Cloud Storage
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Fix service_account_file being ignored (Fabian M\[:o]ller)
|
|
.RE
|
|
.IP \[bu] 2
|
|
Jottacloud
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Minor improvement in quota info (omit if unlimited) (albertony)
|
|
.IP \[bu] 2
|
|
Add \f[C]\-\-fast\-list\f[R] support (albertony)
|
|
.IP \[bu] 2
|
|
Add permanent delete support: \f[C]\-\-jottacloud\-hard\-delete\f[R]
|
|
(albertony)
|
|
.IP \[bu] 2
|
|
Add link sharing support (albertony)
|
|
.IP \[bu] 2
|
|
Fix handling of reserved characters.
|
|
(Sebastian B\[:u]nger)
|
|
.IP \[bu] 2
|
|
Fix socket leak on Object.Remove (Nick Craig\-Wood)
|
|
.RE
|
|
.IP \[bu] 2
|
|
Onedrive
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Rework to support Microsoft Graph (Cnly)
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
\f[B]NB\f[R] this will require re\-authenticating the remote
|
|
.RE
|
|
.IP \[bu] 2
|
|
Removed upload cutoff and always do session uploads (Oliver Heyme)
|
|
.IP \[bu] 2
|
|
Use single\-part upload for empty files (Cnly)
|
|
.IP \[bu] 2
|
|
Fix new fields not saved when editing old config (Alex Chen)
|
|
.IP \[bu] 2
|
|
Fix sometimes special chars in filenames not replaced (Alex Chen)
|
|
.IP \[bu] 2
|
|
Ignore OneNote files by default (Alex Chen)
|
|
.IP \[bu] 2
|
|
Add link sharing support (jackyzy823)
|
|
.RE
|
|
.IP \[bu] 2
|
|
S3
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Use custom pacer, to retry operations when reasonable (Craig Miskell)
|
|
.IP \[bu] 2
|
|
Use configured server\-side\-encryption and storage class options when
|
|
calling CopyObject() (Paul Kohout)
|
|
.IP \[bu] 2
|
|
Make \f[C]\-\-s3\-v2\-auth\f[R] flag (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Fix v2 auth on files with spaces (Nick Craig\-Wood)
|
|
.RE
|
|
.IP \[bu] 2
|
|
Union
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Implement union backend which reads from multiple backends (Felix
|
|
Brucker)
|
|
.IP \[bu] 2
|
|
Implement optional interfaces (Move, DirMove, Copy, etc.) (Nick
|
|
Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Fix ChangeNotify to support multiple remotes (Fabian M\[:o]ller)
|
|
.IP \[bu] 2
|
|
Fix \f[C]\-\-backup\-dir\f[R] on union backend (Nick Craig\-Wood)
|
|
.RE
|
|
.IP \[bu] 2
|
|
WebDAV
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Add another time format (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Add a small pause after failed upload before deleting file (Nick
|
|
Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Add workaround for missing mtime (buergi)
|
|
.IP \[bu] 2
|
|
Sharepoint: Renew cookies after 12hrs (Henning Surmeier)
|
|
.RE
|
|
.IP \[bu] 2
|
|
Yandex
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Remove redundant nil checks (teresy)
|
|
.RE
|
|
.SS v1.43.1 \- 2018\-09\-07
|
|
.PP
|
|
Point release to fix hubic and azureblob backends.
|
|
.IP \[bu] 2
|
|
Bug Fixes
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
ncdu: Return error instead of log.Fatal in Show (Fabian M\[:o]ller)
|
|
.IP \[bu] 2
|
|
cmd: Fix crash with \f[C]\-\-progress\f[R] and \f[C]\-\-stats 0\f[R]
|
|
(Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
docs: Tidy website display (Anagh Kumar Baranwal)
|
|
.RE
|
|
.IP \[bu] 2
|
|
Azure Blob:
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Fix multi\-part uploads.
|
|
(sandeepkru)
|
|
.RE
|
|
.IP \[bu] 2
|
|
Hubic
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Fix uploads (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Retry auth fetching if it fails to make hubic more reliable (Nick
|
|
Craig\-Wood)
|
|
.RE
|
|
.SS v1.43 \- 2018\-09\-01
|
|
.IP \[bu] 2
|
|
New backends
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Jottacloud (Sebastian B\[:u]nger)
|
|
.RE
|
|
.IP \[bu] 2
|
|
New commands
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
copyurl: copies a URL to a remote (Denis)
|
|
.RE
|
|
.IP \[bu] 2
|
|
New Features
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Reworked config for backends (Nick Craig\-Wood)
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
All backend config can now be supplied by command line, env var or
|
|
config file
|
|
.IP \[bu] 2
|
|
Advanced section in the config wizard for the optional items
|
|
.IP \[bu] 2
|
|
A large step towards rclone backends being usable in other go software
|
|
.IP \[bu] 2
|
|
Allow on the fly remotes with :backend: syntax
|
|
.RE
|
|
.IP \[bu] 2
|
|
Stats revamp
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Add \f[C]\-\-progress\f[R]/\f[C]\-P\f[R] flag to show interactive
|
|
progress (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Show the total progress of the sync in the stats (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Add \f[C]\-\-stats\-one\-line\f[R] flag for single line stats (Nick
|
|
Craig\-Wood)
|
|
.RE
|
|
.IP \[bu] 2
|
|
Added weekday schedule into \f[C]\-\-bwlimit\f[R] (Mateusz)
|
|
.IP \[bu] 2
|
|
lsjson: Add option to show the original object IDs (Fabian M\[:o]ller)
|
|
.IP \[bu] 2
|
|
serve webdav: Make Content\-Type without reading the file and add
|
|
\f[C]\-\-etag\-hash\f[R] (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
build
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Build macOS with native compiler (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Update to use go1.11 for the build (Nick Craig\-Wood)
|
|
.RE
|
|
.IP \[bu] 2
|
|
rc
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Added core/stats to return the stats (reddi1)
|
|
.RE
|
|
.IP \[bu] 2
|
|
\f[C]version \-\-check\f[R]: Prints the current release and beta
|
|
versions (Nick Craig\-Wood)
|
|
.RE
|
|
.IP \[bu] 2
|
|
Bug Fixes
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
accounting
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Fix time to completion estimates (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Fix moving average speed for file stats (Nick Craig\-Wood)
|
|
.RE
|
|
.IP \[bu] 2
|
|
config: Fix error reading password from piped input (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
move: Fix \f[C]\-\-delete\-empty\-src\-dirs\f[R] flag to delete all
|
|
empty dirs on move (ishuah)
|
|
.RE
|
|
.IP \[bu] 2
|
|
Mount
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Implement \f[C]\-\-daemon\-timeout\f[R] flag for OSXFUSE (Nick
|
|
Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Fix mount \f[C]\-\-daemon\f[R] not working with encrypted config (Alex
|
|
Chen)
|
|
.IP \[bu] 2
|
|
Clip the number of blocks to 2\[ha]32\-1 on macOS \- fixes borg backup
|
|
(Nick Craig\-Wood)
|
|
.RE
|
|
.IP \[bu] 2
|
|
VFS
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Enable vfs\-read\-chunk\-size by default (Fabian M\[:o]ller)
|
|
.IP \[bu] 2
|
|
Add the vfs/refresh rc command (Fabian M\[:o]ller)
|
|
.IP \[bu] 2
|
|
Add non recursive mode to vfs/refresh rc command (Fabian M\[:o]ller)
|
|
.IP \[bu] 2
|
|
Try to seek buffer on read only files (Fabian M\[:o]ller)
|
|
.RE
|
|
.IP \[bu] 2
|
|
Local
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Fix crash when deprecated
|
|
\f[C]\-\-local\-no\-unicode\-normalization\f[R] is supplied (Nick
|
|
Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Fix mkdir error when trying to copy files to the root of a drive on
|
|
windows (Nick Craig\-Wood)
|
|
.RE
|
|
.IP \[bu] 2
|
|
Cache
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Fix nil pointer deref when using lsjson on cached directory (Nick
|
|
Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Fix nil pointer deref for occasional crash on playback (Nick
|
|
Craig\-Wood)
|
|
.RE
|
|
.IP \[bu] 2
|
|
Crypt
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Fix accounting when checking hashes on upload (Nick Craig\-Wood)
|
|
.RE
|
|
.IP \[bu] 2
|
|
Amazon Cloud Drive
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Make very clear in the docs that rclone has no ACD keys (Nick
|
|
Craig\-Wood)
|
|
.RE
|
|
.IP \[bu] 2
|
|
Azure Blob
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Add connection string and SAS URL auth (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
List the container to see if it exists (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Port new Azure Blob Storage SDK (sandeepkru)
|
|
.IP \[bu] 2
|
|
Added blob tier, tier between Hot, Cool and Archive.
|
|
(sandeepkru)
|
|
.IP \[bu] 2
|
|
Remove leading / from paths (Nick Craig\-Wood)
|
|
.RE
|
|
.IP \[bu] 2
|
|
B2
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Support Application Keys (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Remove leading / from paths (Nick Craig\-Wood)
|
|
.RE
|
|
.IP \[bu] 2
|
|
Box
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Fix upload of > 2GB files on 32 bit platforms (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Make \f[C]\-\-box\-commit\-retries\f[R] flag defaulting to 100 to fix
|
|
large uploads (Nick Craig\-Wood)
|
|
.RE
|
|
.IP \[bu] 2
|
|
Drive
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Add \f[C]\-\-drive\-keep\-revision\-forever\f[R] flag (lewapm)
|
|
.IP \[bu] 2
|
|
Handle gdocs when filtering file names in list (Fabian M\[:o]ller)
|
|
.IP \[bu] 2
|
|
Support using \f[C]\-\-fast\-list\f[R] for large speedups (Fabian
|
|
M\[:o]ller)
|
|
.RE
|
|
.IP \[bu] 2
|
|
FTP
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Fix Put mkParentDir failed: 521 for BunnyCDN (Nick Craig\-Wood)
|
|
.RE
|
|
.IP \[bu] 2
|
|
Google Cloud Storage
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Fix index out of range error with \f[C]\-\-fast\-list\f[R] (Nick
|
|
Craig\-Wood)
|
|
.RE
|
|
.IP \[bu] 2
|
|
Jottacloud
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Fix MD5 error check (Oliver Heyme)
|
|
.IP \[bu] 2
|
|
Handle empty time values (Martin Polden)
|
|
.IP \[bu] 2
|
|
Calculate missing MD5s (Oliver Heyme)
|
|
.IP \[bu] 2
|
|
Docs, fixes and tests for MD5 calculation (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Add optional MimeTyper interface.
|
|
(Sebastian B\[:u]nger)
|
|
.IP \[bu] 2
|
|
Implement optional About interface (for \f[C]df\f[R] support).
|
|
(Sebastian B\[:u]nger)
|
|
.RE
|
|
.IP \[bu] 2
|
|
Mega
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Wait for events instead of arbitrary sleeping (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Add \f[C]\-\-mega\-hard\-delete\f[R] flag (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Fix failed logins with upper case chars in email (Nick Craig\-Wood)
|
|
.RE
|
|
.IP \[bu] 2
|
|
Onedrive
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Shared folder support (Yoni Jah)
|
|
.IP \[bu] 2
|
|
Implement DirMove (Cnly)
|
|
.IP \[bu] 2
|
|
Fix rmdir sometimes deleting directories with contents (Nick
|
|
Craig\-Wood)
|
|
.RE
|
|
.IP \[bu] 2
|
|
Pcloud
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Delete half uploaded files on upload error (Nick Craig\-Wood)
|
|
.RE
|
|
.IP \[bu] 2
|
|
Qingstor
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Remove leading / from paths (Nick Craig\-Wood)
|
|
.RE
|
|
.IP \[bu] 2
|
|
S3
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Fix index out of range error with \f[C]\-\-fast\-list\f[R] (Nick
|
|
Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Add \f[C]\-\-s3\-force\-path\-style\f[R] (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Add support for KMS Key ID (bsteiss)
|
|
.IP \[bu] 2
|
|
Remove leading / from paths (Nick Craig\-Wood)
|
|
.RE
|
|
.IP \[bu] 2
|
|
Swift
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Add \f[C]storage_policy\f[R] (Ruben Vandamme)
|
|
.IP \[bu] 2
|
|
Make it so just \f[C]storage_url\f[R] or \f[C]auth_token\f[R] can be
|
|
overridden (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Fix server\-side copy bug for unusual file names (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Remove leading / from paths (Nick Craig\-Wood)
|
|
.RE
|
|
.IP \[bu] 2
|
|
WebDAV
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Ensure we call MKCOL with a URL with a trailing / for QNAP interop (Nick
|
|
Craig\-Wood)
|
|
.IP \[bu] 2
|
|
If root ends with / then don\[aq]t check if it is a file (Nick
|
|
Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Don\[aq]t accept redirects when reading metadata (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Add bearer token (Macaroon) support for dCache (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Document dCache and Macaroons (Onno Zweers)
|
|
.IP \[bu] 2
|
|
Sharepoint recursion with different depth (Henning)
|
|
.IP \[bu] 2
|
|
Attempt to remove failed uploads (Nick Craig\-Wood)
|
|
.RE
|
|
.IP \[bu] 2
|
|
Yandex
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Fix listing/deleting files in the root (Nick Craig\-Wood)
|
|
.RE
|
|
.SS v1.42 \- 2018\-06\-16
|
|
.IP \[bu] 2
|
|
New backends
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
OpenDrive (Oliver Heyme, Jakub Karlicek, ncw)
|
|
.RE
|
|
.IP \[bu] 2
|
|
New commands
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
deletefile command (Filip Bartodziej)
|
|
.RE
|
|
.IP \[bu] 2
|
|
New Features
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
copy, move: Copy single files directly, don\[aq]t use
|
|
\f[C]\-\-files\-from\f[R] work\-around
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
this makes them much more efficient
|
|
.RE
|
|
.IP \[bu] 2
|
|
Implement \f[C]\-\-max\-transfer\f[R] flag to quit transferring at a
|
|
limit
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
make exit code 8 for \f[C]\-\-max\-transfer\f[R] exceeded
|
|
.RE
|
|
.IP \[bu] 2
|
|
copy: copy empty source directories to destination (Ishuah Kariuki)
|
|
.IP \[bu] 2
|
|
check: Add \f[C]\-\-one\-way\f[R] flag (Kasper Byrdal Nielsen)
|
|
.IP \[bu] 2
|
|
Add siginfo handler for macOS for ctrl\-T stats (kubatasiemski)
|
|
.IP \[bu] 2
|
|
rc
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
add core/gc to run a garbage collection on demand
|
|
.IP \[bu] 2
|
|
enable go profiling by default on the \f[C]\-\-rc\f[R] port
|
|
.IP \[bu] 2
|
|
return error from remote on failure
|
|
.RE
|
|
.IP \[bu] 2
|
|
lsf
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Add \f[C]\-\-absolute\f[R] flag to add a leading / onto path names
|
|
.IP \[bu] 2
|
|
Add \f[C]\-\-csv\f[R] flag for compliant CSV output
|
|
.IP \[bu] 2
|
|
Add \[aq]m\[aq] format specifier to show the MimeType
|
|
.IP \[bu] 2
|
|
Implement \[aq]i\[aq] format for showing object ID
|
|
.RE
|
|
.IP \[bu] 2
|
|
lsjson
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Add MimeType to the output
|
|
.IP \[bu] 2
|
|
Add ID field to output to show Object ID
|
|
.RE
|
|
.IP \[bu] 2
|
|
Add \f[C]\-\-retries\-sleep\f[R] flag (Benjamin Joseph Dag)
|
|
.IP \[bu] 2
|
|
Oauth tidy up web page and error handling (Henning Surmeier)
|
|
.RE
|
|
.IP \[bu] 2
|
|
Bug Fixes
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Password prompt output with \f[C]\-\-log\-file\f[R] fixed for unix
|
|
(Filip Bartodziej)
|
|
.IP \[bu] 2
|
|
Calculate ModifyWindow each time on the fly to fix various problems
|
|
(Stefan Breunig)
|
|
.RE
|
|
.IP \[bu] 2
|
|
Mount
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Only print \[dq]File.rename error\[dq] if there actually is an error
|
|
(Stefan Breunig)
|
|
.IP \[bu] 2
|
|
Delay rename if file has open writers instead of failing outright
|
|
(Stefan Breunig)
|
|
.IP \[bu] 2
|
|
Ensure atexit gets run on interrupt
|
|
.IP \[bu] 2
|
|
macOS enhancements
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Make \f[C]\-\-noappledouble\f[R] \f[C]\-\-noapplexattr\f[R]
|
|
.IP \[bu] 2
|
|
Add \f[C]\-\-volname\f[R] flag and remove special chars from it
|
|
.IP \[bu] 2
|
|
Make Get/List/Set/Remove xattr return ENOSYS for efficiency
|
|
.IP \[bu] 2
|
|
Make \f[C]\-\-daemon\f[R] work for macOS without CGO
|
|
.RE
|
|
.RE
|
|
.IP \[bu] 2
|
|
VFS
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Add \f[C]\-\-vfs\-read\-chunk\-size\f[R] and
|
|
\f[C]\-\-vfs\-read\-chunk\-size\-limit\f[R] (Fabian M\[:o]ller)
|
|
.IP \[bu] 2
|
|
Fix ChangeNotify for new or changed folders (Fabian M\[:o]ller)
|
|
.RE
|
|
.IP \[bu] 2
|
|
Local
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Fix symlink/junction point directory handling under Windows
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
\f[B]NB\f[R] you will need to add \f[C]\-L\f[R] to your command line to
|
|
copy files with reparse points
|
|
.RE
|
|
.RE
|
|
.IP \[bu] 2
|
|
Cache
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Add non cached dirs on notifications (Remus Bunduc)
|
|
.IP \[bu] 2
|
|
Allow root to be expired from rc (Remus Bunduc)
|
|
.IP \[bu] 2
|
|
Clean remaining empty folders from temp upload path (Remus Bunduc)
|
|
.IP \[bu] 2
|
|
Cache lists using batch writes (Remus Bunduc)
|
|
.IP \[bu] 2
|
|
Use secure websockets for HTTPS Plex addresses (John Clayton)
|
|
.IP \[bu] 2
|
|
Reconnect plex websocket on failures (Remus Bunduc)
|
|
.IP \[bu] 2
|
|
Fix panic when running without plex configs (Remus Bunduc)
|
|
.IP \[bu] 2
|
|
Fix root folder caching (Remus Bunduc)
|
|
.RE
|
|
.IP \[bu] 2
|
|
Crypt
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Check the crypted hash of files when uploading for extra data security
|
|
.RE
|
|
.IP \[bu] 2
|
|
Dropbox
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Make Dropbox for business folders accessible using an initial
|
|
\f[C]/\f[R] in the path
|
|
.RE
|
|
.IP \[bu] 2
|
|
Google Cloud Storage
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Low level retry all operations if necessary
|
|
.RE
|
|
.IP \[bu] 2
|
|
Google Drive
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Add \f[C]\-\-drive\-acknowledge\-abuse\f[R] to download flagged files
|
|
.IP \[bu] 2
|
|
Add \f[C]\-\-drive\-alternate\-export\f[R] to fix large doc export
|
|
.IP \[bu] 2
|
|
Don\[aq]t attempt to choose Team Drives when using rclone config create
|
|
.IP \[bu] 2
|
|
Fix change list polling with team drives
|
|
.IP \[bu] 2
|
|
Fix ChangeNotify for folders (Fabian M\[:o]ller)
|
|
.IP \[bu] 2
|
|
Fix about (and df on a mount) for team drives
|
|
.RE
|
|
.IP \[bu] 2
|
|
Onedrive
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Errorhandler for onedrive for business requests (Henning Surmeier)
|
|
.RE
|
|
.IP \[bu] 2
|
|
S3
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Adjust upload concurrency with \f[C]\-\-s3\-upload\-concurrency\f[R]
|
|
(themylogin)
|
|
.IP \[bu] 2
|
|
Fix \f[C]\-\-s3\-chunk\-size\f[R] which was always using the minimum
|
|
.RE
|
|
.IP \[bu] 2
|
|
SFTP
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Add \f[C]\-\-ssh\-path\-override\f[R] flag (Piotr Oleszczyk)
|
|
.IP \[bu] 2
|
|
Fix slow downloads for long latency connections
|
|
.RE
|
|
.IP \[bu] 2
|
|
Webdav
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Add workarounds for biz.mail.ru
|
|
.IP \[bu] 2
|
|
Ignore Reason\-Phrase in status line to fix 4shared (Rodrigo)
|
|
.IP \[bu] 2
|
|
Better error message generation
|
|
.RE
|
|
.SS v1.41 \- 2018\-04\-28
|
|
.IP \[bu] 2
|
|
New backends
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Mega support added
|
|
.IP \[bu] 2
|
|
Webdav now supports SharePoint cookie authentication (hensur)
|
|
.RE
|
|
.IP \[bu] 2
|
|
New commands
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
link: create public link to files and folders (Stefan Breunig)
|
|
.IP \[bu] 2
|
|
about: gets quota info from a remote (a\-roussos, ncw)
|
|
.IP \[bu] 2
|
|
hashsum: a generic tool for any hash to produce md5sum like output
|
|
.RE
|
|
.IP \[bu] 2
|
|
New Features
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
lsd: Add \-R flag and fix and update docs for all ls commands
|
|
.IP \[bu] 2
|
|
ncdu: added a \[dq]refresh\[dq] key \- CTRL\-L (Keith Goldfarb)
|
|
.IP \[bu] 2
|
|
serve restic: Add append\-only mode (Steve Kriss)
|
|
.IP \[bu] 2
|
|
serve restic: Disallow overwriting files in append\-only mode (Alexander
|
|
Neumann)
|
|
.IP \[bu] 2
|
|
serve restic: Print actual listener address (Matt Holt)
|
|
.IP \[bu] 2
|
|
size: Add \-\-json flag (Matthew Holt)
|
|
.IP \[bu] 2
|
|
sync: implement \-\-ignore\-errors (Mateusz Pabian)
|
|
.IP \[bu] 2
|
|
dedupe: Add dedupe largest functionality (Richard Yang)
|
|
.IP \[bu] 2
|
|
fs: Extend SizeSuffix to include TB and PB for rclone about
|
|
.IP \[bu] 2
|
|
fs: add \-\-dump goroutines and \-\-dump openfiles for debugging
|
|
.IP \[bu] 2
|
|
rc: implement core/memstats to print internal memory usage info
|
|
.IP \[bu] 2
|
|
rc: new call rc/pid (Michael P.
|
|
Dubner)
|
|
.RE
|
|
.IP \[bu] 2
|
|
Compile
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Drop support for go1.6
|
|
.RE
|
|
.IP \[bu] 2
|
|
Release
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Fix \f[C]make tarball\f[R] (Chih\-Hsuan Yen)
|
|
.RE
|
|
.IP \[bu] 2
|
|
Bug Fixes
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
filter: fix \-\-min\-age and \-\-max\-age together check
|
|
.IP \[bu] 2
|
|
fs: limit MaxIdleConns and MaxIdleConnsPerHost in transport
|
|
.IP \[bu] 2
|
|
lsd,lsf: make sure all times we output are in local time
|
|
.IP \[bu] 2
|
|
rc: fix setting bwlimit to unlimited
|
|
.IP \[bu] 2
|
|
rc: take note of the \-\-rc\-addr flag too as per the docs
|
|
.RE
|
|
.IP \[bu] 2
|
|
Mount
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Use About to return the correct disk total/used/free (e.g.
|
|
in \f[C]df\f[R])
|
|
.IP \[bu] 2
|
|
Set \f[C]\-\-attr\-timeout default\f[R] to \f[C]1s\f[R] \- fixes:
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
rclone using too much memory
|
|
.IP \[bu] 2
|
|
rclone not serving files to samba
|
|
.IP \[bu] 2
|
|
excessive time listing directories
|
|
.RE
|
|
.IP \[bu] 2
|
|
Fix \f[C]df \-i\f[R] (upstream fix)
|
|
.RE
|
|
.IP \[bu] 2
|
|
VFS
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Filter files \f[C].\f[R] and \f[C]..\f[R] from directory listing
|
|
.IP \[bu] 2
|
|
Only make the VFS cache if \-\-vfs\-cache\-mode > Off
|
|
.RE
|
|
.IP \[bu] 2
|
|
Local
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Add \-\-local\-no\-check\-updated to disable updated file checks
|
|
.IP \[bu] 2
|
|
Retry remove on Windows sharing violation error
|
|
.RE
|
|
.IP \[bu] 2
|
|
Cache
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Flush the memory cache after close
|
|
.IP \[bu] 2
|
|
Purge file data on notification
|
|
.IP \[bu] 2
|
|
Always forget parent dir for notifications
|
|
.IP \[bu] 2
|
|
Integrate with Plex websocket
|
|
.IP \[bu] 2
|
|
Add rc cache/stats (seuffert)
|
|
.IP \[bu] 2
|
|
Add info log on notification
|
|
.RE
|
|
.IP \[bu] 2
|
|
Box
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Fix failure reading large directories \- parse file/directory size as
|
|
float
|
|
.RE
|
|
.IP \[bu] 2
|
|
Dropbox
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Fix crypt+obfuscate on dropbox
|
|
.IP \[bu] 2
|
|
Fix repeatedly uploading the same files
|
|
.RE
|
|
.IP \[bu] 2
|
|
FTP
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Work around strange response from box FTP server
|
|
.IP \[bu] 2
|
|
More workarounds for FTP servers to fix mkParentDir error
|
|
.IP \[bu] 2
|
|
Fix no error on listing non\-existent directory
|
|
.RE
|
|
.IP \[bu] 2
|
|
Google Cloud Storage
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Add service_account_credentials (Matt Holt)
|
|
.IP \[bu] 2
|
|
Detect bucket presence by listing it \- minimises permissions needed
|
|
.IP \[bu] 2
|
|
Ignore zero length directory markers
|
|
.RE
|
|
.IP \[bu] 2
|
|
Google Drive
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Add service_account_credentials (Matt Holt)
|
|
.IP \[bu] 2
|
|
Fix directory move leaving a hardlinked directory behind
|
|
.IP \[bu] 2
|
|
Return proper google errors when Opening files
|
|
.IP \[bu] 2
|
|
When initialized with a filepath, optional features used incorrect root
|
|
path (Stefan Breunig)
|
|
.RE
|
|
.IP \[bu] 2
|
|
HTTP
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Fix sync for servers which don\[aq]t return Content\-Length in HEAD
|
|
.RE
|
|
.IP \[bu] 2
|
|
Onedrive
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Add QuickXorHash support for OneDrive for business
|
|
.IP \[bu] 2
|
|
Fix socket leak in multipart session upload
|
|
.RE
|
|
.IP \[bu] 2
|
|
S3
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Look in S3 named profile files for credentials
|
|
.IP \[bu] 2
|
|
Add \f[C]\-\-s3\-disable\-checksum\f[R] to disable checksum uploading
|
|
(Chris Redekop)
|
|
.IP \[bu] 2
|
|
Hierarchical configuration support (Giri Badanahatti)
|
|
.IP \[bu] 2
|
|
Add in config for all the supported S3 providers
|
|
.IP \[bu] 2
|
|
Add One Zone Infrequent Access storage class (Craig Rachel)
|
|
.IP \[bu] 2
|
|
Add \-\-use\-server\-modtime support (Peter Baumgartner)
|
|
.IP \[bu] 2
|
|
Add \-\-s3\-chunk\-size option to control multipart uploads
|
|
.IP \[bu] 2
|
|
Ignore zero length directory markers
|
|
.RE
|
|
.IP \[bu] 2
|
|
SFTP
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Update docs to match code, fix typos and clarify disable_hashcheck
|
|
prompt (Michael G.
|
|
Noll)
|
|
.IP \[bu] 2
|
|
Update docs with Synology quirks
|
|
.IP \[bu] 2
|
|
Fail soft with a debug on hash failure
|
|
.RE
|
|
.IP \[bu] 2
|
|
Swift
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Add \-\-use\-server\-modtime support (Peter Baumgartner)
|
|
.RE
|
|
.IP \[bu] 2
|
|
Webdav
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Support SharePoint cookie authentication (hensur)
|
|
.IP \[bu] 2
|
|
Strip leading and trailing / off root
|
|
.RE
|
|
.SS v1.40 \- 2018\-03\-19
|
|
.IP \[bu] 2
|
|
New backends
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Alias backend to create aliases for existing remote names (Fabian
|
|
M\[:o]ller)
|
|
.RE
|
|
.IP \[bu] 2
|
|
New commands
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
\f[C]lsf\f[R]: list for parsing purposes (Jakub Tasiemski)
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
by default this is a simple non recursive list of files and directories
|
|
.IP \[bu] 2
|
|
it can be configured to add more info in an easy to parse way
|
|
.RE
|
|
.IP \[bu] 2
|
|
\f[C]serve restic\f[R]: for serving a remote as a Restic REST endpoint
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
This enables restic to use any backends that rclone can access
|
|
.IP \[bu] 2
|
|
Thanks Alexander Neumann for help, patches and review
|
|
.RE
|
|
.IP \[bu] 2
|
|
\f[C]rc\f[R]: enable the remote control of a running rclone
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
The running rclone must be started with \-\-rc and related flags.
|
|
.IP \[bu] 2
|
|
Currently there is support for bwlimit, and flushing for mount and
|
|
cache.
|
|
.RE
|
|
.RE
|
|
.IP \[bu] 2
|
|
New Features
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
\f[C]\-\-max\-delete\f[R] flag to add a delete threshold (Bj\[/o]rn Erik
|
|
Pedersen)
|
|
.IP \[bu] 2
|
|
All backends now support RangeOption for ranged Open
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
\f[C]cat\f[R]: Use RangeOption for limited fetches to make more
|
|
efficient
|
|
.IP \[bu] 2
|
|
\f[C]cryptcheck\f[R]: make reading of nonce more efficient with
|
|
RangeOption
|
|
.RE
|
|
.IP \[bu] 2
|
|
serve http/webdav/restic
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
support SSL/TLS
|
|
.IP \[bu] 2
|
|
add \f[C]\-\-user\f[R] \f[C]\-\-pass\f[R] and \f[C]\-\-htpasswd\f[R] for
|
|
authentication
|
|
.RE
|
|
.IP \[bu] 2
|
|
\f[C]copy\f[R]/\f[C]move\f[R]: detect file size change during copy/move
|
|
and abort transfer (ishuah)
|
|
.IP \[bu] 2
|
|
\f[C]cryptdecode\f[R]: added option to return encrypted file names.
|
|
(ishuah)
|
|
.IP \[bu] 2
|
|
\f[C]lsjson\f[R]: add \f[C]\-\-encrypted\f[R] to show encrypted name
|
|
(Jakub Tasiemski)
|
|
.IP \[bu] 2
|
|
Add \f[C]\-\-stats\-file\-name\-length\f[R] to specify the printed file
|
|
name length for stats (Will Gunn)
|
|
.RE
|
|
.IP \[bu] 2
|
|
Compile
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Code base was shuffled and factored
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
backends moved into a backend directory
|
|
.IP \[bu] 2
|
|
large packages split up
|
|
.IP \[bu] 2
|
|
See the CONTRIBUTING.md doc for info as to what lives where now
|
|
.RE
|
|
.IP \[bu] 2
|
|
Update to using go1.10 as the default go version
|
|
.IP \[bu] 2
|
|
Implement daily full integration
|
|
tests (https://pub.rclone.org/integration-tests/)
|
|
.RE
|
|
.IP \[bu] 2
|
|
Release
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Include a source tarball and sign it and the binaries
|
|
.IP \[bu] 2
|
|
Sign the git tags as part of the release process
|
|
.IP \[bu] 2
|
|
Add .deb and .rpm packages as part of the build
|
|
.IP \[bu] 2
|
|
Make a beta release for all branches on the main repo (but not pull
|
|
requests)
|
|
.RE
|
|
.IP \[bu] 2
|
|
Bug Fixes
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
config: fixes errors on non existing config by loading config file only
|
|
on first access
|
|
.IP \[bu] 2
|
|
config: retry saving the config after failure (Mateusz)
|
|
.IP \[bu] 2
|
|
sync: when using \f[C]\-\-backup\-dir\f[R] don\[aq]t delete files if we
|
|
can\[aq]t set their modtime
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
this fixes odd behaviour with Dropbox and \f[C]\-\-backup\-dir\f[R]
|
|
.RE
|
|
.IP \[bu] 2
|
|
fshttp: fix idle timeouts for HTTP connections
|
|
.IP \[bu] 2
|
|
\f[C]serve http\f[R]: fix serving files with : in \- fixes
|
|
.IP \[bu] 2
|
|
Fix \f[C]\-\-exclude\-if\-present\f[R] to ignore directories which it
|
|
doesn\[aq]t have permission for (Iakov Davydov)
|
|
.IP \[bu] 2
|
|
Make accounting work properly with crypt and b2
|
|
.IP \[bu] 2
|
|
remove \f[C]\-\-no\-traverse\f[R] flag because it is obsolete
|
|
.RE
|
|
.IP \[bu] 2
|
|
Mount
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Add \f[C]\-\-attr\-timeout\f[R] flag to control attribute caching in
|
|
kernel
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
this now defaults to 0 which is correct but less efficient
|
|
.IP \[bu] 2
|
|
see the mount
|
|
docs (https://rclone.org/commands/rclone_mount/#attribute-caching) for
|
|
more info
|
|
.RE
|
|
.IP \[bu] 2
|
|
Add \f[C]\-\-daemon\f[R] flag to allow mount to run in the background
|
|
(ishuah)
|
|
.IP \[bu] 2
|
|
Fix: Return ENOSYS rather than EIO on attempted link
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
This fixes FileZilla accessing an rclone mount served over sftp.
|
|
.RE
|
|
.IP \[bu] 2
|
|
Fix setting modtime twice
|
|
.IP \[bu] 2
|
|
Mount tests now run on CI for Linux (mount & cmount)/Mac/Windows
|
|
.IP \[bu] 2
|
|
Many bugs fixed in the VFS layer \- see below
|
|
.RE
|
|
.IP \[bu] 2
|
|
VFS
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Many fixes for \f[C]\-\-vfs\-cache\-mode\f[R] writes and above
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Update cached copy if we know it has changed (fixes stale data)
|
|
.IP \[bu] 2
|
|
Clean path names before using them in the cache
|
|
.IP \[bu] 2
|
|
Disable cache cleaner if \f[C]\-\-vfs\-cache\-poll\-interval=0\f[R]
|
|
.IP \[bu] 2
|
|
Fill and clean the cache immediately on startup
|
|
.RE
|
|
.IP \[bu] 2
|
|
Fix Windows opening every file when it stats the file
|
|
.IP \[bu] 2
|
|
Fix applying modtime for an open Write Handle
|
|
.IP \[bu] 2
|
|
Fix creation of files when truncating
|
|
.IP \[bu] 2
|
|
Write 0 bytes when flushing unwritten handles to avoid race conditions
|
|
in FUSE
|
|
.IP \[bu] 2
|
|
Downgrade \[dq]poll\-interval is not supported\[dq] message to Info
|
|
.IP \[bu] 2
|
|
Make OpenFile and friends return EINVAL if O_RDONLY and O_TRUNC
|
|
.RE
|
|
.IP \[bu] 2
|
|
Local
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Downgrade \[dq]invalid cross\-device link: trying copy\[dq] to debug
|
|
.IP \[bu] 2
|
|
Make DirMove return fs.ErrorCantDirMove to allow fallback to Copy for
|
|
cross device
|
|
.IP \[bu] 2
|
|
Fix race conditions updating the hashes
|
|
.RE
|
|
.IP \[bu] 2
|
|
Cache
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Add support for polling \- cache will update when remote changes on
|
|
supported backends
|
|
.IP \[bu] 2
|
|
Reduce log level for Plex api
|
|
.IP \[bu] 2
|
|
Fix dir cache issue
|
|
.IP \[bu] 2
|
|
Implement \f[C]\-\-cache\-db\-wait\-time\f[R] flag
|
|
.IP \[bu] 2
|
|
Improve efficiency with RangeOption and RangeSeek
|
|
.IP \[bu] 2
|
|
Fix dirmove with temp fs enabled
|
|
.IP \[bu] 2
|
|
Notify vfs when using temp fs
|
|
.IP \[bu] 2
|
|
Offline uploading
|
|
.IP \[bu] 2
|
|
Remote control support for path flushing
|
|
.RE
|
|
.IP \[bu] 2
|
|
Amazon cloud drive
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Rclone no longer has any working keys \- disable integration tests
|
|
.IP \[bu] 2
|
|
Implement DirChangeNotify to notify cache/vfs/mount of changes
|
|
.RE
|
|
.IP \[bu] 2
|
|
Azureblob
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Don\[aq]t check for bucket/container presence if listing was OK
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
this makes rclone do one less request per invocation
|
|
.RE
|
|
.IP \[bu] 2
|
|
Improve accounting for chunked uploads
|
|
.RE
|
|
.IP \[bu] 2
|
|
Backblaze B2
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Don\[aq]t check for bucket/container presence if listing was OK
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
this makes rclone do one less request per invocation
|
|
.RE
|
|
.RE
|
|
.IP \[bu] 2
|
|
Box
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Improve accounting for chunked uploads
|
|
.RE
|
|
.IP \[bu] 2
|
|
Dropbox
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Fix custom oauth client parameters
|
|
.RE
|
|
.IP \[bu] 2
|
|
Google Cloud Storage
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Don\[aq]t check for bucket/container presence if listing was OK
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
this makes rclone do one less request per invocation
|
|
.RE
|
|
.RE
|
|
.IP \[bu] 2
|
|
Google Drive
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Migrate to api v3 (Fabian M\[:o]ller)
|
|
.IP \[bu] 2
|
|
Add scope configuration and root folder selection
|
|
.IP \[bu] 2
|
|
Add \f[C]\-\-drive\-impersonate\f[R] for service accounts
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
thanks to everyone who tested, explored and contributed docs
|
|
.RE
|
|
.IP \[bu] 2
|
|
Add \f[C]\-\-drive\-use\-created\-date\f[R] to use created date as
|
|
modified date (nbuchanan)
|
|
.IP \[bu] 2
|
|
Request the export formats only when required
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
This makes rclone quicker when there are no google docs
|
|
.RE
|
|
.IP \[bu] 2
|
|
Fix finding paths with latin1 chars (a workaround for a drive bug)
|
|
.IP \[bu] 2
|
|
Fix copying of a single Google doc file
|
|
.IP \[bu] 2
|
|
Fix \f[C]\-\-drive\-auth\-owner\-only\f[R] to look in all directories
|
|
.RE
|
|
.IP \[bu] 2
|
|
HTTP
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Fix handling of directories with & in
|
|
.RE
|
|
.IP \[bu] 2
|
|
Onedrive
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Removed upload cutoff and always do session uploads
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
this stops the creation of multiple versions on business onedrive
|
|
.RE
|
|
.IP \[bu] 2
|
|
Overwrite object size value with real size when reading file.
|
|
(Victor)
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
this fixes oddities when onedrive misreports the size of images
|
|
.RE
|
|
.RE
|
|
.IP \[bu] 2
|
|
Pcloud
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Remove unused chunked upload flag and code
|
|
.RE
|
|
.IP \[bu] 2
|
|
Qingstor
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Don\[aq]t check for bucket/container presence if listing was OK
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
this makes rclone do one less request per invocation
|
|
.RE
|
|
.RE
|
|
.IP \[bu] 2
|
|
S3
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Support hashes for multipart files (Chris Redekop)
|
|
.IP \[bu] 2
|
|
Initial support for IBM COS (S3) (Giri Badanahatti)
|
|
.IP \[bu] 2
|
|
Update docs to discourage use of v2 auth with CEPH and others
|
|
.IP \[bu] 2
|
|
Don\[aq]t check for bucket/container presence if listing was OK
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
this makes rclone do one less request per invocation
|
|
.RE
|
|
.IP \[bu] 2
|
|
Fix server\-side copy and set modtime on files with + in
|
|
.RE
|
|
.IP \[bu] 2
|
|
SFTP
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Add option to disable remote hash check command execution (Jon Fautley)
|
|
.IP \[bu] 2
|
|
Add \f[C]\-\-sftp\-ask\-password\f[R] flag to prompt for password when
|
|
needed (Leo R.
|
|
Lundgren)
|
|
.IP \[bu] 2
|
|
Add \f[C]set_modtime\f[R] configuration option
|
|
.IP \[bu] 2
|
|
Fix following of symlinks
|
|
.IP \[bu] 2
|
|
Fix reading config file outside of Fs setup
|
|
.IP \[bu] 2
|
|
Fix reading $USER in username fallback not $HOME
|
|
.IP \[bu] 2
|
|
Fix running under crontab \- Use correct OS way of reading username
|
|
.RE
|
|
.IP \[bu] 2
|
|
Swift
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Fix refresh of authentication token
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
in v1.39 a bug was introduced which ignored new tokens \- this fixes it
|
|
.RE
|
|
.IP \[bu] 2
|
|
Fix extra HEAD transaction when uploading a new file
|
|
.IP \[bu] 2
|
|
Don\[aq]t check for bucket/container presence if listing was OK
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
this makes rclone do one less request per invocation
|
|
.RE
|
|
.RE
|
|
.IP \[bu] 2
|
|
Webdav
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Add new time formats to support mydrive.ch and others
|
|
.RE
|
|
.SS v1.39 \- 2017\-12\-23
|
|
.IP \[bu] 2
|
|
New backends
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
WebDAV
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
tested with nextcloud, owncloud, put.io and others!
|
|
.RE
|
|
.IP \[bu] 2
|
|
Pcloud
|
|
.IP \[bu] 2
|
|
cache \- wraps a cache around other backends (Remus Bunduc)
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
useful in combination with mount
|
|
.IP \[bu] 2
|
|
NB this feature is in beta so use with care
|
|
.RE
|
|
.RE
|
|
.IP \[bu] 2
|
|
New commands
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
serve command with subcommands:
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
serve webdav: this implements a webdav server for any rclone remote.
|
|
.IP \[bu] 2
|
|
serve http: command to serve a remote over HTTP
|
|
.RE
|
|
.IP \[bu] 2
|
|
config: add sub commands for full config file management
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
create/delete/dump/edit/file/password/providers/show/update
|
|
.RE
|
|
.IP \[bu] 2
|
|
touch: to create or update the timestamp of a file (Jakub Tasiemski)
|
|
.RE
|
|
.IP \[bu] 2
|
|
New Features
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
curl install for rclone (Filip Bartodziej)
|
|
.IP \[bu] 2
|
|
\-\-stats now shows percentage, size, rate and ETA in condensed form
|
|
(Ishuah Kariuki)
|
|
.IP \[bu] 2
|
|
\-\-exclude\-if\-present to exclude a directory if a file is present
|
|
(Iakov Davydov)
|
|
.IP \[bu] 2
|
|
rmdirs: add \-\-leave\-root flag (lewapm)
|
|
.IP \[bu] 2
|
|
move: add \-\-delete\-empty\-src\-dirs flag to remove dirs after move
|
|
(Ishuah Kariuki)
|
|
.IP \[bu] 2
|
|
Add \-\-dump flag, introduce \-\-dump requests, responses and remove
|
|
\-\-dump\-auth, \-\-dump\-filters
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Obscure X\-Auth\-Token: from headers when dumping too
|
|
.RE
|
|
.IP \[bu] 2
|
|
Document and implement exit codes for different failure modes (Ishuah
|
|
Kariuki)
|
|
.RE
|
|
.IP \[bu] 2
|
|
Compile
|
|
.IP \[bu] 2
|
|
Bug Fixes
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Retry lots more different types of errors to make multipart transfers
|
|
more reliable
|
|
.IP \[bu] 2
|
|
Save the config before asking for a token, fixes disappearing oauth
|
|
config
|
|
.IP \[bu] 2
|
|
Warn the user if \-\-include and \-\-exclude are used together (Ernest
|
|
Borowski)
|
|
.IP \[bu] 2
|
|
Fix duplicate files (e.g.
|
|
on Google drive) causing spurious copies
|
|
.IP \[bu] 2
|
|
Allow trailing and leading whitespace for passwords (Jason Rose)
|
|
.IP \[bu] 2
|
|
ncdu: fix crashes on empty directories
|
|
.IP \[bu] 2
|
|
rcat: fix goroutine leak
|
|
.IP \[bu] 2
|
|
moveto/copyto: Fix to allow copying to the same name
|
|
.RE
|
|
.IP \[bu] 2
|
|
Mount
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
\-\-vfs\-cache mode to make writes into mounts more reliable.
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
this requires caching files on the disk (see \-\-cache\-dir)
|
|
.IP \[bu] 2
|
|
As this is a new feature, use with care
|
|
.RE
|
|
.IP \[bu] 2
|
|
Use sdnotify to signal systemd the mount is ready (Fabian M\[:o]ller)
|
|
.IP \[bu] 2
|
|
Check if directory is not empty before mounting (Ernest Borowski)
|
|
.RE
|
|
.IP \[bu] 2
|
|
Local
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Add error message for cross file system moves
|
|
.IP \[bu] 2
|
|
Fix equality check for times
|
|
.RE
|
|
.IP \[bu] 2
|
|
Dropbox
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Rework multipart upload
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
buffer the chunks when uploading large files so they can be retried
|
|
.IP \[bu] 2
|
|
change default chunk size to 48MB now we are buffering them in memory
|
|
.IP \[bu] 2
|
|
retry every error after the first chunk is done successfully
|
|
.RE
|
|
.IP \[bu] 2
|
|
Fix error when renaming directories
|
|
.RE
|
|
.IP \[bu] 2
|
|
Swift
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Fix crash on bad authentication
|
|
.RE
|
|
.IP \[bu] 2
|
|
Google Drive
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Add service account support (Tim Cooijmans)
|
|
.RE
|
|
.IP \[bu] 2
|
|
S3
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Make it work properly with Digital Ocean Spaces (Andrew
|
|
Starr\-Bochicchio)
|
|
.IP \[bu] 2
|
|
Fix crash if a bad listing is received
|
|
.IP \[bu] 2
|
|
Add support for ECS task IAM roles (David Minor)
|
|
.RE
|
|
.IP \[bu] 2
|
|
Backblaze B2
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Fix multipart upload retries
|
|
.IP \[bu] 2
|
|
Fix \-\-hard\-delete to make it work 100% of the time
|
|
.RE
|
|
.IP \[bu] 2
|
|
Swift
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Allow authentication with storage URL and auth key (Giovanni Pizzi)
|
|
.IP \[bu] 2
|
|
Add new fields for swift configuration to support IBM Bluemix Swift
|
|
(Pierre Carlson)
|
|
.IP \[bu] 2
|
|
Add OS_TENANT_ID and OS_USER_ID to config
|
|
.IP \[bu] 2
|
|
Allow configs with user id instead of user name
|
|
.IP \[bu] 2
|
|
Check if swift segments container exists before creating (John Leach)
|
|
.IP \[bu] 2
|
|
Fix memory leak in swift transfers (upstream fix)
|
|
.RE
|
|
.IP \[bu] 2
|
|
SFTP
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Add option to enable the use of aes128\-cbc cipher (Jon Fautley)
|
|
.RE
|
|
.IP \[bu] 2
|
|
Amazon cloud drive
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Fix download of large files failing with \[dq]Only one auth mechanism
|
|
allowed\[dq]
|
|
.RE
|
|
.IP \[bu] 2
|
|
crypt
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Option to encrypt directory names or leave them intact
|
|
.IP \[bu] 2
|
|
Implement DirChangeNotify (Fabian M\[:o]ller)
|
|
.RE
|
|
.IP \[bu] 2
|
|
onedrive
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Add option to choose resourceURL during setup of OneDrive Business
|
|
account if more than one is available for user
|
|
.RE
|
|
.SS v1.38 \- 2017\-09\-30
|
|
.IP \[bu] 2
|
|
New backends
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Azure Blob Storage (thanks Andrei Dragomir)
|
|
.IP \[bu] 2
|
|
Box
|
|
.IP \[bu] 2
|
|
Onedrive for Business (thanks Oliver Heyme)
|
|
.IP \[bu] 2
|
|
QingStor from QingCloud (thanks wuyu)
|
|
.RE
|
|
.IP \[bu] 2
|
|
New commands
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
\f[C]rcat\f[R] \- read from standard input and stream upload
|
|
.IP \[bu] 2
|
|
\f[C]tree\f[R] \- shows a nicely formatted recursive listing
|
|
.IP \[bu] 2
|
|
\f[C]cryptdecode\f[R] \- decode crypted file names (thanks ishuah)
|
|
.IP \[bu] 2
|
|
\f[C]config show\f[R] \- print the config file
|
|
.IP \[bu] 2
|
|
\f[C]config file\f[R] \- print the config file location
|
|
.RE
|
|
.IP \[bu] 2
|
|
New Features
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Empty directories are deleted on \f[C]sync\f[R]
|
|
.IP \[bu] 2
|
|
\f[C]dedupe\f[R] \- implement merging of duplicate directories
|
|
.IP \[bu] 2
|
|
\f[C]check\f[R] and \f[C]cryptcheck\f[R] made more consistent and use
|
|
less memory
|
|
.IP \[bu] 2
|
|
\f[C]cleanup\f[R] for remaining remotes (thanks ishuah)
|
|
.IP \[bu] 2
|
|
\f[C]\-\-immutable\f[R] for ensuring that files don\[aq]t change (thanks
|
|
Jacob McNamee)
|
|
.IP \[bu] 2
|
|
\f[C]\-\-user\-agent\f[R] option (thanks Alex McGrath Kraak)
|
|
.IP \[bu] 2
|
|
\f[C]\-\-disable\f[R] flag to disable optional features
|
|
.IP \[bu] 2
|
|
\f[C]\-\-bind\f[R] flag for choosing the local addr on outgoing
|
|
connections
|
|
.IP \[bu] 2
|
|
Support for zsh auto\-completion (thanks bpicode)
|
|
.IP \[bu] 2
|
|
Stop normalizing file names but do a normalized compare in
|
|
\f[C]sync\f[R]
|
|
.RE
|
|
.IP \[bu] 2
|
|
Compile
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Update to using go1.9 as the default go version
|
|
.IP \[bu] 2
|
|
Remove snapd build due to maintenance problems
|
|
.RE
|
|
.IP \[bu] 2
|
|
Bug Fixes
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Improve retriable error detection which makes multipart uploads better
|
|
.IP \[bu] 2
|
|
Make \f[C]check\f[R] obey \f[C]\-\-ignore\-size\f[R]
|
|
.IP \[bu] 2
|
|
Fix bwlimit toggle in conjunction with schedules (thanks cbruegg)
|
|
.IP \[bu] 2
|
|
\f[C]config\f[R] ensures newly written config is on the same mount
|
|
.RE
|
|
.IP \[bu] 2
|
|
Local
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Revert to copy when moving file across file system boundaries
|
|
.IP \[bu] 2
|
|
\f[C]\-\-skip\-links\f[R] to suppress symlink warnings (thanks Zhiming
|
|
Wang)
|
|
.RE
|
|
.IP \[bu] 2
|
|
Mount
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Re\-use \f[C]rcat\f[R] internals to support uploads from all remotes
|
|
.RE
|
|
.IP \[bu] 2
|
|
Dropbox
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Fix \[dq]entry doesn\[aq]t belong in directory\[dq] error
|
|
.IP \[bu] 2
|
|
Stop using deprecated API methods
|
|
.RE
|
|
.IP \[bu] 2
|
|
Swift
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Fix server\-side copy to empty container with \f[C]\-\-fast\-list\f[R]
|
|
.RE
|
|
.IP \[bu] 2
|
|
Google Drive
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Change the default for \f[C]\-\-drive\-use\-trash\f[R] to \f[C]true\f[R]
|
|
.RE
|
|
.IP \[bu] 2
|
|
S3
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Set session token when using STS (thanks Girish Ramakrishnan)
|
|
.IP \[bu] 2
|
|
Glacier docs and error messages (thanks Jan Varho)
|
|
.IP \[bu] 2
|
|
Read 1000 (not 1024) items in dir listings to fix Wasabi
|
|
.RE
|
|
.IP \[bu] 2
|
|
Backblaze B2
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Fix SHA1 mismatch when downloading files with no SHA1
|
|
.IP \[bu] 2
|
|
Calculate missing hashes on the fly instead of spooling
|
|
.IP \[bu] 2
|
|
\f[C]\-\-b2\-hard\-delete\f[R] to permanently delete (not hide) files
|
|
(thanks John Papandriopoulos)
|
|
.RE
|
|
.IP \[bu] 2
|
|
Hubic
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Fix creating containers \- no longer have to use the \f[C]default\f[R]
|
|
container
|
|
.RE
|
|
.IP \[bu] 2
|
|
Swift
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Optionally configure from a standard set of OpenStack environment vars
|
|
.IP \[bu] 2
|
|
Add \f[C]endpoint_type\f[R] config
|
|
.RE
|
|
.IP \[bu] 2
|
|
Google Cloud Storage
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Fix bucket creation to work with limited permission users
|
|
.RE
|
|
.IP \[bu] 2
|
|
SFTP
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Implement connection pooling for multiple ssh connections
|
|
.IP \[bu] 2
|
|
Limit new connections per second
|
|
.IP \[bu] 2
|
|
Add support for MD5 and SHA1 hashes where available (thanks Christian
|
|
Br\[:u]ggemann)
|
|
.RE
|
|
.IP \[bu] 2
|
|
HTTP
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Fix URL encoding issues
|
|
.IP \[bu] 2
|
|
Fix directories with \f[C]:\f[R] in
|
|
.IP \[bu] 2
|
|
Fix panic with URL encoded content
|
|
.RE
|
|
.SS v1.37 \- 2017\-07\-22
|
|
.IP \[bu] 2
|
|
New backends
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
FTP \- thanks to Antonio Messina
|
|
.IP \[bu] 2
|
|
HTTP \- thanks to Vasiliy Tolstov
|
|
.RE
|
|
.IP \[bu] 2
|
|
New commands
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
rclone ncdu \- for exploring a remote with a text based user interface.
|
|
.IP \[bu] 2
|
|
rclone lsjson \- for listing with a machine readable output
|
|
.IP \[bu] 2
|
|
rclone dbhashsum \- to show Dropbox style hashes of files (local or
|
|
Dropbox)
|
|
.RE
|
|
.IP \[bu] 2
|
|
New Features
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Implement \-\-fast\-list flag
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
This allows remotes to list recursively if they can
|
|
.IP \[bu] 2
|
|
This uses less transactions (important if you pay for them)
|
|
.IP \[bu] 2
|
|
This may or may not be quicker
|
|
.IP \[bu] 2
|
|
This will use more memory as it has to hold the listing in memory
|
|
.IP \[bu] 2
|
|
\-\-old\-sync\-method deprecated \- the remaining uses are covered by
|
|
\-\-fast\-list
|
|
.IP \[bu] 2
|
|
This involved a major re\-write of all the listing code
|
|
.RE
|
|
.IP \[bu] 2
|
|
Add \-\-tpslimit and \-\-tpslimit\-burst to limit transactions per
|
|
second
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
this is useful in conjunction with \f[C]rclone mount\f[R] to limit
|
|
external apps
|
|
.RE
|
|
.IP \[bu] 2
|
|
Add \-\-stats\-log\-level so can see \-\-stats without \-v
|
|
.IP \[bu] 2
|
|
Print password prompts to stderr \- Hraban Luyat
|
|
.IP \[bu] 2
|
|
Warn about duplicate files when syncing
|
|
.IP \[bu] 2
|
|
Oauth improvements
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
allow auth_url and token_url to be set in the config file
|
|
.IP \[bu] 2
|
|
Print redirection URI if using own credentials.
|
|
.RE
|
|
.IP \[bu] 2
|
|
Don\[aq]t Mkdir at the start of sync to save transactions
|
|
.RE
|
|
.IP \[bu] 2
|
|
Compile
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Update build to go1.8.3
|
|
.IP \[bu] 2
|
|
Require go1.6 for building rclone
|
|
.IP \[bu] 2
|
|
Compile 386 builds with \[dq]GO386=387\[dq] for maximum compatibility
|
|
.RE
|
|
.IP \[bu] 2
|
|
Bug Fixes
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Fix menu selection when no remotes
|
|
.IP \[bu] 2
|
|
Config saving reworked to not kill the file if disk gets full
|
|
.IP \[bu] 2
|
|
Don\[aq]t delete remote if name does not change while renaming
|
|
.IP \[bu] 2
|
|
moveto, copyto: report transfers and checks as per move and copy
|
|
.RE
|
|
.IP \[bu] 2
|
|
Local
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Add \-\-local\-no\-unicode\-normalization flag \- Bob Potter
|
|
.RE
|
|
.IP \[bu] 2
|
|
Mount
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Now supported on Windows using cgofuse and WinFsp \- thanks to Bill
|
|
Zissimopoulos for much help
|
|
.IP \[bu] 2
|
|
Compare checksums on upload/download via FUSE
|
|
.IP \[bu] 2
|
|
Unmount when program ends with SIGINT (Ctrl+C) or SIGTERM \-
|
|
J\['e]r\[^o]me Vizcaino
|
|
.IP \[bu] 2
|
|
On read only open of file, make open pending until first read
|
|
.IP \[bu] 2
|
|
Make \-\-read\-only reject modify operations
|
|
.IP \[bu] 2
|
|
Implement ModTime via FUSE for remotes that support it
|
|
.IP \[bu] 2
|
|
Allow modTime to be changed even before all writers are closed
|
|
.IP \[bu] 2
|
|
Fix panic on renames
|
|
.IP \[bu] 2
|
|
Fix hang on errored upload
|
|
.RE
|
|
.IP \[bu] 2
|
|
Crypt
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Report the name:root as specified by the user
|
|
.IP \[bu] 2
|
|
Add an \[dq]obfuscate\[dq] option for filename encryption \- Stephen
|
|
Harris
|
|
.RE
|
|
.IP \[bu] 2
|
|
Amazon Drive
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Fix initialization order for token renewer
|
|
.IP \[bu] 2
|
|
Remove revoked credentials, allow oauth proxy config and update docs
|
|
.RE
|
|
.IP \[bu] 2
|
|
B2
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Reduce minimum chunk size to 5MB
|
|
.RE
|
|
.IP \[bu] 2
|
|
Drive
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Add team drive support
|
|
.IP \[bu] 2
|
|
Reduce bandwidth by adding fields for partial responses \- Martin
|
|
Kristensen
|
|
.IP \[bu] 2
|
|
Implement \-\-drive\-shared\-with\-me flag to view shared with me files
|
|
\- Danny Tsai
|
|
.IP \[bu] 2
|
|
Add \-\-drive\-trashed\-only to read only the files in the trash
|
|
.IP \[bu] 2
|
|
Remove obsolete \-\-drive\-full\-list
|
|
.IP \[bu] 2
|
|
Add missing seek to start on retries of chunked uploads
|
|
.IP \[bu] 2
|
|
Fix stats accounting for upload
|
|
.IP \[bu] 2
|
|
Convert / in names to a unicode equivalent (\[uFF0F])
|
|
.IP \[bu] 2
|
|
Poll for Google Drive changes when mounted
|
|
.RE
|
|
.IP \[bu] 2
|
|
OneDrive
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Fix the uploading of files with spaces
|
|
.IP \[bu] 2
|
|
Fix initialization order for token renewer
|
|
.IP \[bu] 2
|
|
Display speeds accurately when uploading \- Yoni Jah
|
|
.IP \[bu] 2
|
|
Swap to using http://localhost:53682/ as redirect URL \- Michael Ledin
|
|
.IP \[bu] 2
|
|
Retry on token expired error, reset upload body on retry \- Yoni Jah
|
|
.RE
|
|
.IP \[bu] 2
|
|
Google Cloud Storage
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Add ability to specify location and storage class via config and command
|
|
line \- thanks gdm85
|
|
.IP \[bu] 2
|
|
Create container if necessary on server\-side copy
|
|
.IP \[bu] 2
|
|
Increase directory listing chunk to 1000 to increase performance
|
|
.IP \[bu] 2
|
|
Obtain a refresh token for GCS \- Steven Lu
|
|
.RE
|
|
.IP \[bu] 2
|
|
Yandex
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Fix the name reported in log messages (was empty)
|
|
.IP \[bu] 2
|
|
Correct error return for listing empty directory
|
|
.RE
|
|
.IP \[bu] 2
|
|
Dropbox
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Rewritten to use the v2 API
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Now supports ModTime
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Can only set by uploading the file again
|
|
.IP \[bu] 2
|
|
If you uploaded with an old rclone, rclone may upload everything again
|
|
.IP \[bu] 2
|
|
Use \f[C]\-\-size\-only\f[R] or \f[C]\-\-checksum\f[R] to avoid this
|
|
.RE
|
|
.IP \[bu] 2
|
|
Now supports the Dropbox content hashing scheme
|
|
.IP \[bu] 2
|
|
Now supports low level retries
|
|
.RE
|
|
.RE
|
|
.IP \[bu] 2
|
|
S3
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Work around eventual consistency in bucket creation
|
|
.IP \[bu] 2
|
|
Create container if necessary on server\-side copy
|
|
.IP \[bu] 2
|
|
Add us\-east\-2 (Ohio) and eu\-west\-2 (London) S3 regions \- Zahiar
|
|
Ahmed
|
|
.RE
|
|
.IP \[bu] 2
|
|
Swift, Hubic
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Fix zero length directory markers showing in the subdirectory listing
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
this caused lots of duplicate transfers
|
|
.RE
|
|
.IP \[bu] 2
|
|
Fix paged directory listings
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
this caused duplicate directory errors
|
|
.RE
|
|
.IP \[bu] 2
|
|
Create container if necessary on server\-side copy
|
|
.IP \[bu] 2
|
|
Increase directory listing chunk to 1000 to increase performance
|
|
.IP \[bu] 2
|
|
Make sensible error if the user forgets the container
|
|
.RE
|
|
.IP \[bu] 2
|
|
SFTP
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Add support for using ssh key files
|
|
.IP \[bu] 2
|
|
Fix under Windows
|
|
.IP \[bu] 2
|
|
Fix ssh agent on Windows
|
|
.IP \[bu] 2
|
|
Adapt to latest version of library \- Igor Kharin
|
|
.RE
|
|
.SS v1.36 \- 2017\-03\-18
|
|
.IP \[bu] 2
|
|
New Features
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
SFTP remote (Jack Schmidt)
|
|
.IP \[bu] 2
|
|
Re\-implement sync routine to work a directory at a time reducing memory
|
|
usage
|
|
.IP \[bu] 2
|
|
Logging revamped to be more inline with rsync \- now much quieter * \-v
|
|
only shows transfers * \-vv is for full debug * \-\-syslog to log to
|
|
syslog on capable platforms
|
|
.IP \[bu] 2
|
|
Implement \-\-backup\-dir and \-\-suffix
|
|
.IP \[bu] 2
|
|
Implement \-\-track\-renames (initial implementation by Bj\[/o]rn Erik
|
|
Pedersen)
|
|
.IP \[bu] 2
|
|
Add time\-based bandwidth limits (Lukas Loesche)
|
|
.IP \[bu] 2
|
|
rclone cryptcheck: checks integrity of crypt remotes
|
|
.IP \[bu] 2
|
|
Allow all config file variables and options to be set from environment
|
|
variables
|
|
.IP \[bu] 2
|
|
Add \-\-buffer\-size parameter to control buffer size for copy
|
|
.IP \[bu] 2
|
|
Make \-\-delete\-after the default
|
|
.IP \[bu] 2
|
|
Add \-\-ignore\-checksum flag (fixed by Hisham Zarka)
|
|
.IP \[bu] 2
|
|
rclone check: Add \-\-download flag to check all the data, not just
|
|
hashes
|
|
.IP \[bu] 2
|
|
rclone cat: add \-\-head, \-\-tail, \-\-offset, \-\-count and
|
|
\-\-discard
|
|
.IP \[bu] 2
|
|
rclone config: when choosing from a list, allow the value to be entered
|
|
too
|
|
.IP \[bu] 2
|
|
rclone config: allow rename and copy of remotes
|
|
.IP \[bu] 2
|
|
rclone obscure: for generating encrypted passwords for rclone\[aq]s
|
|
config (T.C.
|
|
Ferguson)
|
|
.IP \[bu] 2
|
|
Comply with XDG Base Directory specification (Dario Giovannetti)
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
this moves the default location of the config file in a backwards
|
|
compatible way
|
|
.RE
|
|
.IP \[bu] 2
|
|
Release changes
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Ubuntu snap support (Dedsec1)
|
|
.IP \[bu] 2
|
|
Compile with go 1.8
|
|
.IP \[bu] 2
|
|
MIPS/Linux big and little endian support
|
|
.RE
|
|
.RE
|
|
.IP \[bu] 2
|
|
Bug Fixes
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Fix copyto copying things to the wrong place if the destination dir
|
|
didn\[aq]t exist
|
|
.IP \[bu] 2
|
|
Fix parsing of remotes in moveto and copyto
|
|
.IP \[bu] 2
|
|
Fix \-\-delete\-before deleting files on copy
|
|
.IP \[bu] 2
|
|
Fix \-\-files\-from with an empty file copying everything
|
|
.IP \[bu] 2
|
|
Fix sync: don\[aq]t update mod times if \-\-dry\-run set
|
|
.IP \[bu] 2
|
|
Fix MimeType propagation
|
|
.IP \[bu] 2
|
|
Fix filters to add ** rules to directory rules
|
|
.RE
|
|
.IP \[bu] 2
|
|
Local
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Implement \-L, \-\-copy\-links flag to allow rclone to follow symlinks
|
|
.IP \[bu] 2
|
|
Open files in write only mode so rclone can write to an rclone mount
|
|
.IP \[bu] 2
|
|
Fix unnormalised unicode causing problems reading directories
|
|
.IP \[bu] 2
|
|
Fix interaction between \-x flag and \-\-max\-depth
|
|
.RE
|
|
.IP \[bu] 2
|
|
Mount
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Implement proper directory handling (mkdir, rmdir, renaming)
|
|
.IP \[bu] 2
|
|
Make include and exclude filters apply to mount
|
|
.IP \[bu] 2
|
|
Implement read and write async buffers \- control with \-\-buffer\-size
|
|
.IP \[bu] 2
|
|
Fix fsync on for directories
|
|
.IP \[bu] 2
|
|
Fix retry on network failure when reading off crypt
|
|
.RE
|
|
.IP \[bu] 2
|
|
Crypt
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Add \-\-crypt\-show\-mapping to show encrypted file mapping
|
|
.IP \[bu] 2
|
|
Fix crypt writer getting stuck in a loop
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
\f[B]IMPORTANT\f[R] this bug had the potential to cause data corruption
|
|
when
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
reading data from a network based remote and
|
|
.IP \[bu] 2
|
|
writing to a crypt on Google Drive
|
|
.RE
|
|
.IP \[bu] 2
|
|
Use the cryptcheck command to validate your data if you are concerned
|
|
.IP \[bu] 2
|
|
If syncing two crypt remotes, sync the unencrypted remote
|
|
.RE
|
|
.RE
|
|
.IP \[bu] 2
|
|
Amazon Drive
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Fix panics on Move (rename)
|
|
.IP \[bu] 2
|
|
Fix panic on token expiry
|
|
.RE
|
|
.IP \[bu] 2
|
|
B2
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Fix inconsistent listings and rclone check
|
|
.IP \[bu] 2
|
|
Fix uploading empty files with go1.8
|
|
.IP \[bu] 2
|
|
Constrain memory usage when doing multipart uploads
|
|
.IP \[bu] 2
|
|
Fix upload url not being refreshed properly
|
|
.RE
|
|
.IP \[bu] 2
|
|
Drive
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Fix Rmdir on directories with trashed files
|
|
.IP \[bu] 2
|
|
Fix \[dq]Ignoring unknown object\[dq] when downloading
|
|
.IP \[bu] 2
|
|
Add \-\-drive\-list\-chunk
|
|
.IP \[bu] 2
|
|
Add \-\-drive\-skip\-gdocs (K\['a]roly Ol\['a]h)
|
|
.RE
|
|
.IP \[bu] 2
|
|
OneDrive
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Implement Move
|
|
.IP \[bu] 2
|
|
Fix Copy
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Fix overwrite detection in Copy
|
|
.IP \[bu] 2
|
|
Fix waitForJob to parse errors correctly
|
|
.RE
|
|
.IP \[bu] 2
|
|
Use token renewer to stop auth errors on long uploads
|
|
.IP \[bu] 2
|
|
Fix uploading empty files with go1.8
|
|
.RE
|
|
.IP \[bu] 2
|
|
Google Cloud Storage
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Fix depth 1 directory listings
|
|
.RE
|
|
.IP \[bu] 2
|
|
Yandex
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Fix single level directory listing
|
|
.RE
|
|
.IP \[bu] 2
|
|
Dropbox
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Normalise the case for single level directory listings
|
|
.IP \[bu] 2
|
|
Fix depth 1 listing
|
|
.RE
|
|
.IP \[bu] 2
|
|
S3
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Added ca\-central\-1 region (Jon Yergatian)
|
|
.RE
|
|
.SS v1.35 \- 2017\-01\-02
|
|
.IP \[bu] 2
|
|
New Features
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
moveto and copyto commands for choosing a destination name on copy/move
|
|
.IP \[bu] 2
|
|
rmdirs command to recursively delete empty directories
|
|
.IP \[bu] 2
|
|
Allow repeated \-\-include/\-\-exclude/\-\-filter options
|
|
.IP \[bu] 2
|
|
Only show transfer stats on commands which transfer stuff
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
show stats on any command using the \f[C]\-\-stats\f[R] flag
|
|
.RE
|
|
.IP \[bu] 2
|
|
Allow overlapping directories in move when server\-side dir move is
|
|
supported
|
|
.IP \[bu] 2
|
|
Add \-\-stats\-unit option \- thanks Scott McGillivray
|
|
.RE
|
|
.IP \[bu] 2
|
|
Bug Fixes
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Fix the config file being overwritten when two rclone instances are
|
|
running
|
|
.IP \[bu] 2
|
|
Make rclone lsd obey the filters properly
|
|
.IP \[bu] 2
|
|
Fix compilation on mips
|
|
.IP \[bu] 2
|
|
Fix not transferring files that don\[aq]t differ in size
|
|
.IP \[bu] 2
|
|
Fix panic on nil retry/fatal error
|
|
.RE
|
|
.IP \[bu] 2
|
|
Mount
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Retry reads on error \- should help with reliability a lot
|
|
.IP \[bu] 2
|
|
Report the modification times for directories from the remote
|
|
.IP \[bu] 2
|
|
Add bandwidth accounting and limiting (fixes \-\-bwlimit)
|
|
.IP \[bu] 2
|
|
If \-\-stats provided will show stats and which files are transferring
|
|
.IP \[bu] 2
|
|
Support R/W files if truncate is set.
|
|
.IP \[bu] 2
|
|
Implement statfs interface so df works
|
|
.IP \[bu] 2
|
|
Note that write is now supported on Amazon Drive
|
|
.IP \[bu] 2
|
|
Report number of blocks in a file \- thanks Stefan Breunig
|
|
.RE
|
|
.IP \[bu] 2
|
|
Crypt
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Prevent the user pointing crypt at itself
|
|
.IP \[bu] 2
|
|
Fix failed to authenticate decrypted block errors
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
these will now return the underlying unexpected EOF instead
|
|
.RE
|
|
.RE
|
|
.IP \[bu] 2
|
|
Amazon Drive
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Add support for server\-side move and directory move \- thanks Stefan
|
|
Breunig
|
|
.IP \[bu] 2
|
|
Fix nil pointer deref on size attribute
|
|
.RE
|
|
.IP \[bu] 2
|
|
B2
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Use new prefix and delimiter parameters in directory listings
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
This makes \-\-max\-depth 1 dir listings as used in mount much faster
|
|
.RE
|
|
.IP \[bu] 2
|
|
Reauth the account while doing uploads too \- should help with token
|
|
expiry
|
|
.RE
|
|
.IP \[bu] 2
|
|
Drive
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Make DirMove more efficient and complain about moving the root
|
|
.IP \[bu] 2
|
|
Create destination directory on Move()
|
|
.RE
|
|
.SS v1.34 \- 2016\-11\-06
|
|
.IP \[bu] 2
|
|
New Features
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Stop single file and \f[C]\-\-files\-from\f[R] operations iterating
|
|
through the source bucket.
|
|
.IP \[bu] 2
|
|
Stop removing failed upload to cloud storage remotes
|
|
.IP \[bu] 2
|
|
Make ContentType be preserved for cloud to cloud copies
|
|
.IP \[bu] 2
|
|
Add support to toggle bandwidth limits via SIGUSR2 \- thanks Marco
|
|
Paganini
|
|
.IP \[bu] 2
|
|
\f[C]rclone check\f[R] shows count of hashes that couldn\[aq]t be
|
|
checked
|
|
.IP \[bu] 2
|
|
\f[C]rclone listremotes\f[R] command
|
|
.IP \[bu] 2
|
|
Support linux/arm64 build \- thanks Fredrik Fornwall
|
|
.IP \[bu] 2
|
|
Remove \f[C]Authorization:\f[R] lines from \f[C]\-\-dump\-headers\f[R]
|
|
output
|
|
.RE
|
|
.IP \[bu] 2
|
|
Bug Fixes
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Ignore files with control characters in the names
|
|
.IP \[bu] 2
|
|
Fix \f[C]rclone move\f[R] command
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Delete src files which already existed in dst
|
|
.IP \[bu] 2
|
|
Fix deletion of src file when dst file older
|
|
.RE
|
|
.IP \[bu] 2
|
|
Fix \f[C]rclone check\f[R] on crypted file systems
|
|
.IP \[bu] 2
|
|
Make failed uploads not count as \[dq]Transferred\[dq]
|
|
.IP \[bu] 2
|
|
Make sure high level retries show with \f[C]\-q\f[R]
|
|
.IP \[bu] 2
|
|
Use a vendor directory with godep for repeatable builds
|
|
.RE
|
|
.IP \[bu] 2
|
|
\f[C]rclone mount\f[R] \- FUSE
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Implement FUSE mount options
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
\f[C]\-\-no\-modtime\f[R], \f[C]\-\-debug\-fuse\f[R],
|
|
\f[C]\-\-read\-only\f[R], \f[C]\-\-allow\-non\-empty\f[R],
|
|
\f[C]\-\-allow\-root\f[R], \f[C]\-\-allow\-other\f[R]
|
|
.IP \[bu] 2
|
|
\f[C]\-\-default\-permissions\f[R], \f[C]\-\-write\-back\-cache\f[R],
|
|
\f[C]\-\-max\-read\-ahead\f[R], \f[C]\-\-umask\f[R], \f[C]\-\-uid\f[R],
|
|
\f[C]\-\-gid\f[R]
|
|
.RE
|
|
.IP \[bu] 2
|
|
Add \f[C]\-\-dir\-cache\-time\f[R] to control caching of directory
|
|
entries
|
|
.IP \[bu] 2
|
|
Implement seek for files opened for read (useful for video players)
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
with \f[C]\-no\-seek\f[R] flag to disable
|
|
.RE
|
|
.IP \[bu] 2
|
|
Fix crash on 32 bit ARM (alignment of 64 bit counter)
|
|
.IP \[bu] 2
|
|
\&...and many more internal fixes and improvements!
|
|
.RE
|
|
.IP \[bu] 2
|
|
Crypt
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Don\[aq]t show encrypted password in configurator to stop confusion
|
|
.RE
|
|
.IP \[bu] 2
|
|
Amazon Drive
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
New wait for upload option \f[C]\-\-acd\-upload\-wait\-per\-gb\f[R]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
upload timeouts scale by file size and can be disabled
|
|
.RE
|
|
.IP \[bu] 2
|
|
Add 502 Bad Gateway to list of errors we retry
|
|
.IP \[bu] 2
|
|
Fix overwriting a file with a zero length file
|
|
.IP \[bu] 2
|
|
Fix ACD file size warning limit \- thanks Felix B\[:u]nemann
|
|
.RE
|
|
.IP \[bu] 2
|
|
Local
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Unix: implement \f[C]\-x\f[R]/\f[C]\-\-one\-file\-system\f[R] to stay on
|
|
a single file system
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
thanks Durval Menezes and Luiz Carlos Rumbelsperger Viana
|
|
.RE
|
|
.IP \[bu] 2
|
|
Windows: ignore the symlink bit on files
|
|
.IP \[bu] 2
|
|
Windows: Ignore directory based junction points
|
|
.RE
|
|
.IP \[bu] 2
|
|
B2
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Make sure each upload has at least one upload slot \- fixes strange
|
|
upload stats
|
|
.IP \[bu] 2
|
|
Fix uploads when using crypt
|
|
.IP \[bu] 2
|
|
Fix download of large files (sha1 mismatch)
|
|
.IP \[bu] 2
|
|
Return error when we try to create a bucket which someone else owns
|
|
.IP \[bu] 2
|
|
Update B2 docs with Data usage, and Crypt section \- thanks Tomasz Mazur
|
|
.RE
|
|
.IP \[bu] 2
|
|
S3
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Command line and config file support for
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Setting/overriding ACL \- thanks Radek Senfeld
|
|
.IP \[bu] 2
|
|
Setting storage class \- thanks Asko Tamm
|
|
.RE
|
|
.RE
|
|
.IP \[bu] 2
|
|
Drive
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Make exponential backoff work exactly as per Google specification
|
|
.IP \[bu] 2
|
|
add \f[C].epub\f[R], \f[C].odp\f[R] and \f[C].tsv\f[R] as export
|
|
formats.
|
|
.RE
|
|
.IP \[bu] 2
|
|
Swift
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Don\[aq]t read metadata for directory marker objects
|
|
.RE
|
|
.SS v1.33 \- 2016\-08\-24
|
|
.IP \[bu] 2
|
|
New Features
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Implement encryption
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
data encrypted in NACL secretbox format
|
|
.IP \[bu] 2
|
|
with optional file name encryption
|
|
.RE
|
|
.IP \[bu] 2
|
|
New commands
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
rclone mount \- implements FUSE mounting of remotes (EXPERIMENTAL)
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
works on Linux, FreeBSD and OS X (need testers for the last 2!)
|
|
.RE
|
|
.IP \[bu] 2
|
|
rclone cat \- outputs remote file or files to the terminal
|
|
.IP \[bu] 2
|
|
rclone genautocomplete \- command to make a bash completion script for
|
|
rclone
|
|
.RE
|
|
.IP \[bu] 2
|
|
Editing a remote using \f[C]rclone config\f[R] now goes through the
|
|
wizard
|
|
.IP \[bu] 2
|
|
Compile with go 1.7 \- this fixes rclone on macOS Sierra and on 386
|
|
processors
|
|
.IP \[bu] 2
|
|
Use cobra for sub commands and docs generation
|
|
.RE
|
|
.IP \[bu] 2
|
|
drive
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Document how to make your own client_id
|
|
.RE
|
|
.IP \[bu] 2
|
|
s3
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
User\-configurable Amazon S3 ACL (thanks Radek \[vS]enfeld)
|
|
.RE
|
|
.IP \[bu] 2
|
|
b2
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Fix stats accounting for upload \- no more jumping to 100% done
|
|
.IP \[bu] 2
|
|
On cleanup delete hide marker if it is the current file
|
|
.IP \[bu] 2
|
|
New B2 API endpoint (thanks Per Cederberg)
|
|
.IP \[bu] 2
|
|
Set maximum backoff to 5 Minutes
|
|
.RE
|
|
.IP \[bu] 2
|
|
onedrive
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Fix URL escaping in file names \- e.g.
|
|
uploading files with \f[C]+\f[R] in them.
|
|
.RE
|
|
.IP \[bu] 2
|
|
amazon cloud drive
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Fix token expiry during large uploads
|
|
.IP \[bu] 2
|
|
Work around 408 REQUEST_TIMEOUT and 504 GATEWAY_TIMEOUT errors
|
|
.RE
|
|
.IP \[bu] 2
|
|
local
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Fix filenames with invalid UTF\-8 not being uploaded
|
|
.IP \[bu] 2
|
|
Fix problem with some UTF\-8 characters on OS X
|
|
.RE
|
|
.SS v1.32 \- 2016\-07\-13
|
|
.IP \[bu] 2
|
|
Backblaze B2
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Fix upload of files large files not in root
|
|
.RE
|
|
.SS v1.31 \- 2016\-07\-13
|
|
.IP \[bu] 2
|
|
New Features
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Reduce memory on sync by about 50%
|
|
.IP \[bu] 2
|
|
Implement \-\-no\-traverse flag to stop copy traversing the destination
|
|
remote.
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
This can be used to reduce memory usage down to the smallest possible.
|
|
.IP \[bu] 2
|
|
Useful to copy a small number of files into a large destination folder.
|
|
.RE
|
|
.IP \[bu] 2
|
|
Implement cleanup command for emptying trash / removing old versions of
|
|
files
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Currently B2 only
|
|
.RE
|
|
.IP \[bu] 2
|
|
Single file handling improved
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Now copied with \-\-files\-from
|
|
.IP \[bu] 2
|
|
Automatically sets \-\-no\-traverse when copying a single file
|
|
.RE
|
|
.IP \[bu] 2
|
|
Info on using installing with ansible \- thanks Stefan Weichinger
|
|
.IP \[bu] 2
|
|
Implement \-\-no\-update\-modtime flag to stop rclone fixing the remote
|
|
modified times.
|
|
.RE
|
|
.IP \[bu] 2
|
|
Bug Fixes
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Fix move command \- stop it running for overlapping Fses \- this was
|
|
causing data loss.
|
|
.RE
|
|
.IP \[bu] 2
|
|
Local
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Fix incomplete hashes \- this was causing problems for B2.
|
|
.RE
|
|
.IP \[bu] 2
|
|
Amazon Drive
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Rename Amazon Cloud Drive to Amazon Drive \- no changes to config file
|
|
needed.
|
|
.RE
|
|
.IP \[bu] 2
|
|
Swift
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Add support for non\-default project domain \- thanks Antonio Messina.
|
|
.RE
|
|
.IP \[bu] 2
|
|
S3
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Add instructions on how to use rclone with minio.
|
|
.IP \[bu] 2
|
|
Add ap\-northeast\-2 (Seoul) and ap\-south\-1 (Mumbai) regions.
|
|
.IP \[bu] 2
|
|
Skip setting the modified time for objects > 5GB as it isn\[aq]t
|
|
possible.
|
|
.RE
|
|
.IP \[bu] 2
|
|
Backblaze B2
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Add \-\-b2\-versions flag so old versions can be listed and retrieved.
|
|
.IP \[bu] 2
|
|
Treat 403 errors (e.g.
|
|
cap exceeded) as fatal.
|
|
.IP \[bu] 2
|
|
Implement cleanup command for deleting old file versions.
|
|
.IP \[bu] 2
|
|
Make error handling compliant with B2 integrations notes.
|
|
.IP \[bu] 2
|
|
Fix handling of token expiry.
|
|
.IP \[bu] 2
|
|
Implement \-\-b2\-test\-mode to set \f[C]X\-Bz\-Test\-Mode\f[R] header.
|
|
.IP \[bu] 2
|
|
Set cutoff for chunked upload to 200MB as per B2 guidelines.
|
|
.IP \[bu] 2
|
|
Make upload multi\-threaded.
|
|
.RE
|
|
.IP \[bu] 2
|
|
Dropbox
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Don\[aq]t retry 461 errors.
|
|
.RE
|
|
.SS v1.30 \- 2016\-06\-18
|
|
.IP \[bu] 2
|
|
New Features
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Directory listing code reworked for more features and better error
|
|
reporting (thanks to Klaus Post for help).
|
|
This enables
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Directory include filtering for efficiency
|
|
.IP \[bu] 2
|
|
\-\-max\-depth parameter
|
|
.IP \[bu] 2
|
|
Better error reporting
|
|
.IP \[bu] 2
|
|
More to come
|
|
.RE
|
|
.IP \[bu] 2
|
|
Retry more errors
|
|
.IP \[bu] 2
|
|
Add \-\-ignore\-size flag \- for uploading images to onedrive
|
|
.IP \[bu] 2
|
|
Log \-v output to stdout by default
|
|
.IP \[bu] 2
|
|
Display the transfer stats in more human readable form
|
|
.IP \[bu] 2
|
|
Make 0 size files specifiable with \f[C]\-\-max\-size 0b\f[R]
|
|
.IP \[bu] 2
|
|
Add \f[C]b\f[R] suffix so we can specify bytes in \-\-bwlimit,
|
|
\-\-min\-size, etc.
|
|
.IP \[bu] 2
|
|
Use \[dq]password:\[dq] instead of \[dq]password>\[dq] prompt \- thanks
|
|
Klaus Post and Leigh Klotz
|
|
.RE
|
|
.IP \[bu] 2
|
|
Bug Fixes
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Fix retry doing one too many retries
|
|
.RE
|
|
.IP \[bu] 2
|
|
Local
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Fix problems with OS X and UTF\-8 characters
|
|
.RE
|
|
.IP \[bu] 2
|
|
Amazon Drive
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Check a file exists before uploading to help with 408 Conflict errors
|
|
.IP \[bu] 2
|
|
Reauth on 401 errors \- this has been causing a lot of problems
|
|
.IP \[bu] 2
|
|
Work around spurious 403 errors
|
|
.IP \[bu] 2
|
|
Restart directory listings on error
|
|
.RE
|
|
.IP \[bu] 2
|
|
Google Drive
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Check a file exists before uploading to help with duplicates
|
|
.IP \[bu] 2
|
|
Fix retry of multipart uploads
|
|
.RE
|
|
.IP \[bu] 2
|
|
Backblaze B2
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Implement large file uploading
|
|
.RE
|
|
.IP \[bu] 2
|
|
S3
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Add AES256 server\-side encryption for \- thanks Justin R.
|
|
Wilson
|
|
.RE
|
|
.IP \[bu] 2
|
|
Google Cloud Storage
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Make sure we don\[aq]t use conflicting content types on upload
|
|
.IP \[bu] 2
|
|
Add service account support \- thanks Michal Witkowski
|
|
.RE
|
|
.IP \[bu] 2
|
|
Swift
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Add auth version parameter
|
|
.IP \[bu] 2
|
|
Add domain option for openstack (v3 auth) \- thanks Fabian Ruff
|
|
.RE
|
|
.SS v1.29 \- 2016\-04\-18
|
|
.IP \[bu] 2
|
|
New Features
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Implement \f[C]\-I, \-\-ignore\-times\f[R] for unconditional upload
|
|
.IP \[bu] 2
|
|
Improve \f[C]dedupe\f[R]command
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Now removes identical copies without asking
|
|
.IP \[bu] 2
|
|
Now obeys \f[C]\-\-dry\-run\f[R]
|
|
.IP \[bu] 2
|
|
Implement \f[C]\-\-dedupe\-mode\f[R] for non interactive running
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
\f[C]\-\-dedupe\-mode interactive\f[R] \- interactive the default.
|
|
.IP \[bu] 2
|
|
\f[C]\-\-dedupe\-mode skip\f[R] \- removes identical files then skips
|
|
anything left.
|
|
.IP \[bu] 2
|
|
\f[C]\-\-dedupe\-mode first\f[R] \- removes identical files then keeps
|
|
the first one.
|
|
.IP \[bu] 2
|
|
\f[C]\-\-dedupe\-mode newest\f[R] \- removes identical files then keeps
|
|
the newest one.
|
|
.IP \[bu] 2
|
|
\f[C]\-\-dedupe\-mode oldest\f[R] \- removes identical files then keeps
|
|
the oldest one.
|
|
.IP \[bu] 2
|
|
\f[C]\-\-dedupe\-mode rename\f[R] \- removes identical files then
|
|
renames the rest to be different.
|
|
.RE
|
|
.RE
|
|
.RE
|
|
.IP \[bu] 2
|
|
Bug fixes
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Make rclone check obey the \f[C]\-\-size\-only\f[R] flag.
|
|
.IP \[bu] 2
|
|
Use \[dq]application/octet\-stream\[dq] if discovered mime type is
|
|
invalid.
|
|
.IP \[bu] 2
|
|
Fix missing \[dq]quit\[dq] option when there are no remotes.
|
|
.RE
|
|
.IP \[bu] 2
|
|
Google Drive
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Increase default chunk size to 8 MB \- increases upload speed of big
|
|
files
|
|
.IP \[bu] 2
|
|
Speed up directory listings and make more reliable
|
|
.IP \[bu] 2
|
|
Add missing retries for Move and DirMove \- increases reliability
|
|
.IP \[bu] 2
|
|
Preserve mime type on file update
|
|
.RE
|
|
.IP \[bu] 2
|
|
Backblaze B2
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Enable mod time syncing
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
This means that B2 will now check modification times
|
|
.IP \[bu] 2
|
|
It will upload new files to update the modification times
|
|
.IP \[bu] 2
|
|
(there isn\[aq]t an API to just set the mod time.)
|
|
.IP \[bu] 2
|
|
If you want the old behaviour use \f[C]\-\-size\-only\f[R].
|
|
.RE
|
|
.IP \[bu] 2
|
|
Update API to new version
|
|
.IP \[bu] 2
|
|
Fix parsing of mod time when not in metadata
|
|
.RE
|
|
.IP \[bu] 2
|
|
Swift/Hubic
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Don\[aq]t return an MD5SUM for static large objects
|
|
.RE
|
|
.IP \[bu] 2
|
|
S3
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Fix uploading files bigger than 50GB
|
|
.RE
|
|
.SS v1.28 \- 2016\-03\-01
|
|
.IP \[bu] 2
|
|
New Features
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Configuration file encryption \- thanks Klaus Post
|
|
.IP \[bu] 2
|
|
Improve \f[C]rclone config\f[R] adding more help and making it easier to
|
|
understand
|
|
.IP \[bu] 2
|
|
Implement \f[C]\-u\f[R]/\f[C]\-\-update\f[R] so creation times can be
|
|
used on all remotes
|
|
.IP \[bu] 2
|
|
Implement \f[C]\-\-low\-level\-retries\f[R] flag
|
|
.IP \[bu] 2
|
|
Optionally disable gzip compression on downloads with
|
|
\f[C]\-\-no\-gzip\-encoding\f[R]
|
|
.RE
|
|
.IP \[bu] 2
|
|
Bug fixes
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Don\[aq]t make directories if \f[C]\-\-dry\-run\f[R] set
|
|
.IP \[bu] 2
|
|
Fix and document the \f[C]move\f[R] command
|
|
.IP \[bu] 2
|
|
Fix redirecting stderr on unix\-like OSes when using
|
|
\f[C]\-\-log\-file\f[R]
|
|
.IP \[bu] 2
|
|
Fix \f[C]delete\f[R] command to wait until all finished \- fixes missing
|
|
deletes.
|
|
.RE
|
|
.IP \[bu] 2
|
|
Backblaze B2
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Use one upload URL per go routine fixes
|
|
\f[C]more than one upload using auth token\f[R]
|
|
.IP \[bu] 2
|
|
Add pacing, retries and reauthentication \- fixes token expiry problems
|
|
.IP \[bu] 2
|
|
Upload without using a temporary file from local (and remotes which
|
|
support SHA1)
|
|
.IP \[bu] 2
|
|
Fix reading metadata for all files when it shouldn\[aq]t have been
|
|
.RE
|
|
.IP \[bu] 2
|
|
Drive
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Fix listing drive documents at root
|
|
.IP \[bu] 2
|
|
Disable copy and move for Google docs
|
|
.RE
|
|
.IP \[bu] 2
|
|
Swift
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Fix uploading of chunked files with non ASCII characters
|
|
.IP \[bu] 2
|
|
Allow setting of \f[C]storage_url\f[R] in the config \- thanks Xavier
|
|
Lucas
|
|
.RE
|
|
.IP \[bu] 2
|
|
S3
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Allow IAM role and credentials from environment variables \- thanks
|
|
Brian Stengaard
|
|
.IP \[bu] 2
|
|
Allow low privilege users to use S3 (check if directory exists during
|
|
Mkdir) \- thanks Jakub Gedeon
|
|
.RE
|
|
.IP \[bu] 2
|
|
Amazon Drive
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Retry on more things to make directory listings more reliable
|
|
.RE
|
|
.SS v1.27 \- 2016\-01\-31
|
|
.IP \[bu] 2
|
|
New Features
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Easier headless configuration with \f[C]rclone authorize\f[R]
|
|
.IP \[bu] 2
|
|
Add support for multiple hash types \- we now check SHA1 as well as MD5
|
|
hashes.
|
|
.IP \[bu] 2
|
|
\f[C]delete\f[R] command which does obey the filters (unlike
|
|
\f[C]purge\f[R])
|
|
.IP \[bu] 2
|
|
\f[C]dedupe\f[R] command to deduplicate a remote.
|
|
Useful with Google Drive.
|
|
.IP \[bu] 2
|
|
Add \f[C]\-\-ignore\-existing\f[R] flag to skip all files that exist on
|
|
destination.
|
|
.IP \[bu] 2
|
|
Add \f[C]\-\-delete\-before\f[R], \f[C]\-\-delete\-during\f[R],
|
|
\f[C]\-\-delete\-after\f[R] flags.
|
|
.IP \[bu] 2
|
|
Add \f[C]\-\-memprofile\f[R] flag to debug memory use.
|
|
.IP \[bu] 2
|
|
Warn the user about files with same name but different case
|
|
.IP \[bu] 2
|
|
Make \f[C]\-\-include\f[R] rules add their implicit exclude * at the end
|
|
of the filter list
|
|
.IP \[bu] 2
|
|
Deprecate compiling with go1.3
|
|
.RE
|
|
.IP \[bu] 2
|
|
Amazon Drive
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Fix download of files > 10 GB
|
|
.IP \[bu] 2
|
|
Fix directory traversal (\[dq]Next token is expired\[dq]) for large
|
|
directory listings
|
|
.IP \[bu] 2
|
|
Remove 409 conflict from error codes we will retry \- stops very long
|
|
pauses
|
|
.RE
|
|
.IP \[bu] 2
|
|
Backblaze B2
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
SHA1 hashes now checked by rclone core
|
|
.RE
|
|
.IP \[bu] 2
|
|
Drive
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Add \f[C]\-\-drive\-auth\-owner\-only\f[R] to only consider files owned
|
|
by the user \- thanks Bj\[:o]rn Harrtell
|
|
.IP \[bu] 2
|
|
Export Google documents
|
|
.RE
|
|
.IP \[bu] 2
|
|
Dropbox
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Make file exclusion error controllable with \-q
|
|
.RE
|
|
.IP \[bu] 2
|
|
Swift
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Fix upload from unprivileged user.
|
|
.RE
|
|
.IP \[bu] 2
|
|
S3
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Fix updating of mod times of files with \f[C]+\f[R] in.
|
|
.RE
|
|
.IP \[bu] 2
|
|
Local
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Add local file system option to disable UNC on Windows.
|
|
.RE
|
|
.SS v1.26 \- 2016\-01\-02
|
|
.IP \[bu] 2
|
|
New Features
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Yandex storage backend \- thank you Dmitry Burdeev (\[dq]dibu\[dq])
|
|
.IP \[bu] 2
|
|
Implement Backblaze B2 storage backend
|
|
.IP \[bu] 2
|
|
Add \-\-min\-age and \-\-max\-age flags \- thank you Adriano Aur\['e]lio
|
|
Meirelles
|
|
.IP \[bu] 2
|
|
Make ls/lsl/md5sum/size/check obey includes and excludes
|
|
.RE
|
|
.IP \[bu] 2
|
|
Fixes
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Fix crash in http logging
|
|
.IP \[bu] 2
|
|
Upload releases to github too
|
|
.RE
|
|
.IP \[bu] 2
|
|
Swift
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Fix sync for chunked files
|
|
.RE
|
|
.IP \[bu] 2
|
|
OneDrive
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Re\-enable server\-side copy
|
|
.IP \[bu] 2
|
|
Don\[aq]t mask HTTP error codes with JSON decode error
|
|
.RE
|
|
.IP \[bu] 2
|
|
S3
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Fix corrupting Content\-Type on mod time update (thanks Joseph Spurrier)
|
|
.RE
|
|
.SS v1.25 \- 2015\-11\-14
|
|
.IP \[bu] 2
|
|
New features
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Implement Hubic storage system
|
|
.RE
|
|
.IP \[bu] 2
|
|
Fixes
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Fix deletion of some excluded files without \-\-delete\-excluded
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
This could have deleted files unexpectedly on sync
|
|
.IP \[bu] 2
|
|
Always check first with \f[C]\-\-dry\-run\f[R]!
|
|
.RE
|
|
.RE
|
|
.IP \[bu] 2
|
|
Swift
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Stop SetModTime losing metadata (e.g.
|
|
X\-Object\-Manifest)
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
This could have caused data loss for files > 5GB in size
|
|
.RE
|
|
.IP \[bu] 2
|
|
Use ContentType from Object to avoid lookups in listings
|
|
.RE
|
|
.IP \[bu] 2
|
|
OneDrive
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
disable server\-side copy as it seems to be broken at Microsoft
|
|
.RE
|
|
.SS v1.24 \- 2015\-11\-07
|
|
.IP \[bu] 2
|
|
New features
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Add support for Microsoft OneDrive
|
|
.IP \[bu] 2
|
|
Add \f[C]\-\-no\-check\-certificate\f[R] option to disable server
|
|
certificate verification
|
|
.IP \[bu] 2
|
|
Add async readahead buffer for faster transfer of big files
|
|
.RE
|
|
.IP \[bu] 2
|
|
Fixes
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Allow spaces in remotes and check remote names for validity at creation
|
|
time
|
|
.IP \[bu] 2
|
|
Allow \[aq]&\[aq] and disallow \[aq]:\[aq] in Windows filenames.
|
|
.RE
|
|
.IP \[bu] 2
|
|
Swift
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Ignore directory marker objects where appropriate \- allows working with
|
|
Hubic
|
|
.IP \[bu] 2
|
|
Don\[aq]t delete the container if fs wasn\[aq]t at root
|
|
.RE
|
|
.IP \[bu] 2
|
|
S3
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Don\[aq]t delete the bucket if fs wasn\[aq]t at root
|
|
.RE
|
|
.IP \[bu] 2
|
|
Google Cloud Storage
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Don\[aq]t delete the bucket if fs wasn\[aq]t at root
|
|
.RE
|
|
.SS v1.23 \- 2015\-10\-03
|
|
.IP \[bu] 2
|
|
New features
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Implement \f[C]rclone size\f[R] for measuring remotes
|
|
.RE
|
|
.IP \[bu] 2
|
|
Fixes
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Fix headless config for drive and gcs
|
|
.IP \[bu] 2
|
|
Tell the user they should try again if the webserver method failed
|
|
.IP \[bu] 2
|
|
Improve output of \f[C]\-\-dump\-headers\f[R]
|
|
.RE
|
|
.IP \[bu] 2
|
|
S3
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Allow anonymous access to public buckets
|
|
.RE
|
|
.IP \[bu] 2
|
|
Swift
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Stop chunked operations logging \[dq]Failed to read info: Object Not
|
|
Found\[dq]
|
|
.IP \[bu] 2
|
|
Use Content\-Length on uploads for extra reliability
|
|
.RE
|
|
.SS v1.22 \- 2015\-09\-28
|
|
.IP \[bu] 2
|
|
Implement rsync like include and exclude flags
|
|
.IP \[bu] 2
|
|
swift
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Support files > 5GB \- thanks Sergey Tolmachev
|
|
.RE
|
|
.SS v1.21 \- 2015\-09\-22
|
|
.IP \[bu] 2
|
|
New features
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Display individual transfer progress
|
|
.IP \[bu] 2
|
|
Make lsl output times in localtime
|
|
.RE
|
|
.IP \[bu] 2
|
|
Fixes
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Fix allowing user to override credentials again in Drive, GCS and ACD
|
|
.RE
|
|
.IP \[bu] 2
|
|
Amazon Drive
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Implement compliant pacing scheme
|
|
.RE
|
|
.IP \[bu] 2
|
|
Google Drive
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Make directory reads concurrent for increased speed.
|
|
.RE
|
|
.SS v1.20 \- 2015\-09\-15
|
|
.IP \[bu] 2
|
|
New features
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Amazon Drive support
|
|
.IP \[bu] 2
|
|
Oauth support redone \- fix many bugs and improve usability
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Use \[dq]golang.org/x/oauth2\[dq] as oauth library of choice
|
|
.IP \[bu] 2
|
|
Improve oauth usability for smoother initial signup
|
|
.IP \[bu] 2
|
|
drive, googlecloudstorage: optionally use auto config for the oauth
|
|
token
|
|
.RE
|
|
.IP \[bu] 2
|
|
Implement \-\-dump\-headers and \-\-dump\-bodies debug flags
|
|
.IP \[bu] 2
|
|
Show multiple matched commands if abbreviation too short
|
|
.IP \[bu] 2
|
|
Implement server\-side move where possible
|
|
.RE
|
|
.IP \[bu] 2
|
|
local
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Always use UNC paths internally on Windows \- fixes a lot of bugs
|
|
.RE
|
|
.IP \[bu] 2
|
|
dropbox
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
force use of our custom transport which makes timeouts work
|
|
.RE
|
|
.IP \[bu] 2
|
|
Thanks to Klaus Post for lots of help with this release
|
|
.SS v1.19 \- 2015\-08\-28
|
|
.IP \[bu] 2
|
|
New features
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Server side copies for s3/swift/drive/dropbox/gcs
|
|
.IP \[bu] 2
|
|
Move command \- uses server\-side copies if it can
|
|
.IP \[bu] 2
|
|
Implement \-\-retries flag \- tries 3 times by default
|
|
.IP \[bu] 2
|
|
Build for plan9/amd64 and solaris/amd64 too
|
|
.RE
|
|
.IP \[bu] 2
|
|
Fixes
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Make a current version download with a fixed URL for scripting
|
|
.IP \[bu] 2
|
|
Ignore rmdir in limited fs rather than throwing error
|
|
.RE
|
|
.IP \[bu] 2
|
|
dropbox
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Increase chunk size to improve upload speeds massively
|
|
.IP \[bu] 2
|
|
Issue an error message when trying to upload bad file name
|
|
.RE
|
|
.SS v1.18 \- 2015\-08\-17
|
|
.IP \[bu] 2
|
|
drive
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Add \f[C]\-\-drive\-use\-trash\f[R] flag so rclone trashes instead of
|
|
deletes
|
|
.IP \[bu] 2
|
|
Add \[dq]Forbidden to download\[dq] message for files with no
|
|
downloadURL
|
|
.RE
|
|
.IP \[bu] 2
|
|
dropbox
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Remove datastore
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
This was deprecated and it caused a lot of problems
|
|
.IP \[bu] 2
|
|
Modification times and MD5SUMs no longer stored
|
|
.RE
|
|
.IP \[bu] 2
|
|
Fix uploading files > 2GB
|
|
.RE
|
|
.IP \[bu] 2
|
|
s3
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
use official AWS SDK from github.com/aws/aws\-sdk\-go
|
|
.IP \[bu] 2
|
|
\f[B]NB\f[R] will most likely require you to delete and recreate remote
|
|
.IP \[bu] 2
|
|
enable multipart upload which enables files > 5GB
|
|
.IP \[bu] 2
|
|
tested with Ceph / RadosGW / S3 emulation
|
|
.IP \[bu] 2
|
|
many thanks to Sam Liston and Brian Haymore at the Utah Center for High
|
|
Performance Computing (https://www.chpc.utah.edu/) for a Ceph test
|
|
account
|
|
.RE
|
|
.IP \[bu] 2
|
|
misc
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Show errors when reading the config file
|
|
.IP \[bu] 2
|
|
Do not print stats in quiet mode \- thanks Leonid Shalupov
|
|
.IP \[bu] 2
|
|
Add FAQ
|
|
.IP \[bu] 2
|
|
Fix created directories not obeying umask
|
|
.IP \[bu] 2
|
|
Linux installation instructions \- thanks Shimon Doodkin
|
|
.RE
|
|
.SS v1.17 \- 2015\-06\-14
|
|
.IP \[bu] 2
|
|
dropbox: fix case insensitivity issues \- thanks Leonid Shalupov
|
|
.SS v1.16 \- 2015\-06\-09
|
|
.IP \[bu] 2
|
|
Fix uploading big files which was causing timeouts or panics
|
|
.IP \[bu] 2
|
|
Don\[aq]t check md5sum after download with \-\-size\-only
|
|
.SS v1.15 \- 2015\-06\-06
|
|
.IP \[bu] 2
|
|
Add \-\-checksum flag to only discard transfers by MD5SUM \- thanks Alex
|
|
Couper
|
|
.IP \[bu] 2
|
|
Implement \-\-size\-only flag to sync on size not checksum & modtime
|
|
.IP \[bu] 2
|
|
Expand docs and remove duplicated information
|
|
.IP \[bu] 2
|
|
Document rclone\[aq]s limitations with directories
|
|
.IP \[bu] 2
|
|
dropbox: update docs about case insensitivity
|
|
.SS v1.14 \- 2015\-05\-21
|
|
.IP \[bu] 2
|
|
local: fix encoding of non utf\-8 file names \- fixes a duplicate file
|
|
problem
|
|
.IP \[bu] 2
|
|
drive: docs about rate limiting
|
|
.IP \[bu] 2
|
|
google cloud storage: Fix compile after API change in
|
|
\[dq]google.golang.org/api/storage/v1\[dq]
|
|
.SS v1.13 \- 2015\-05\-10
|
|
.IP \[bu] 2
|
|
Revise documentation (especially sync)
|
|
.IP \[bu] 2
|
|
Implement \-\-timeout and \-\-conntimeout
|
|
.IP \[bu] 2
|
|
s3: ignore etags from multipart uploads which aren\[aq]t md5sums
|
|
.SS v1.12 \- 2015\-03\-15
|
|
.IP \[bu] 2
|
|
drive: Use chunked upload for files above a certain size
|
|
.IP \[bu] 2
|
|
drive: add \-\-drive\-chunk\-size and \-\-drive\-upload\-cutoff
|
|
parameters
|
|
.IP \[bu] 2
|
|
drive: switch to insert from update when a failed copy deletes the
|
|
upload
|
|
.IP \[bu] 2
|
|
core: Log duplicate files if they are detected
|
|
.SS v1.11 \- 2015\-03\-04
|
|
.IP \[bu] 2
|
|
swift: add region parameter
|
|
.IP \[bu] 2
|
|
drive: fix crash on failed to update remote mtime
|
|
.IP \[bu] 2
|
|
In remote paths, change native directory separators to /
|
|
.IP \[bu] 2
|
|
Add synchronization to ls/lsl/lsd output to stop corruptions
|
|
.IP \[bu] 2
|
|
Ensure all stats/log messages to go stderr
|
|
.IP \[bu] 2
|
|
Add \-\-log\-file flag to log everything (including panics) to file
|
|
.IP \[bu] 2
|
|
Make it possible to disable stats printing with \-\-stats=0
|
|
.IP \[bu] 2
|
|
Implement \-\-bwlimit to limit data transfer bandwidth
|
|
.SS v1.10 \- 2015\-02\-12
|
|
.IP \[bu] 2
|
|
s3: list an unlimited number of items
|
|
.IP \[bu] 2
|
|
Fix getting stuck in the configurator
|
|
.SS v1.09 \- 2015\-02\-07
|
|
.IP \[bu] 2
|
|
windows: Stop drive letters (e.g.
|
|
C:) getting mixed up with remotes (e.g.
|
|
drive:)
|
|
.IP \[bu] 2
|
|
local: Fix directory separators on Windows
|
|
.IP \[bu] 2
|
|
drive: fix rate limit exceeded errors
|
|
.SS v1.08 \- 2015\-02\-04
|
|
.IP \[bu] 2
|
|
drive: fix subdirectory listing to not list entire drive
|
|
.IP \[bu] 2
|
|
drive: Fix SetModTime
|
|
.IP \[bu] 2
|
|
dropbox: adapt code to recent library changes
|
|
.SS v1.07 \- 2014\-12\-23
|
|
.IP \[bu] 2
|
|
google cloud storage: fix memory leak
|
|
.SS v1.06 \- 2014\-12\-12
|
|
.IP \[bu] 2
|
|
Fix \[dq]Couldn\[aq]t find home directory\[dq] on OSX
|
|
.IP \[bu] 2
|
|
swift: Add tenant parameter
|
|
.IP \[bu] 2
|
|
Use new location of Google API packages
|
|
.SS v1.05 \- 2014\-08\-09
|
|
.IP \[bu] 2
|
|
Improved tests and consequently lots of minor fixes
|
|
.IP \[bu] 2
|
|
core: Fix race detected by go race detector
|
|
.IP \[bu] 2
|
|
core: Fixes after running errcheck
|
|
.IP \[bu] 2
|
|
drive: reset root directory on Rmdir and Purge
|
|
.IP \[bu] 2
|
|
fs: Document that Purger returns error on empty directory, test and fix
|
|
.IP \[bu] 2
|
|
google cloud storage: fix ListDir on subdirectory
|
|
.IP \[bu] 2
|
|
google cloud storage: re\-read metadata in SetModTime
|
|
.IP \[bu] 2
|
|
s3: make reading metadata more reliable to work around eventual
|
|
consistency problems
|
|
.IP \[bu] 2
|
|
s3: strip trailing / from ListDir()
|
|
.IP \[bu] 2
|
|
swift: return directories without / in ListDir
|
|
.SS v1.04 \- 2014\-07\-21
|
|
.IP \[bu] 2
|
|
google cloud storage: Fix crash on Update
|
|
.SS v1.03 \- 2014\-07\-20
|
|
.IP \[bu] 2
|
|
swift, s3, dropbox: fix updated files being marked as corrupted
|
|
.IP \[bu] 2
|
|
Make compile with go 1.1 again
|
|
.SS v1.02 \- 2014\-07\-19
|
|
.IP \[bu] 2
|
|
Implement Dropbox remote
|
|
.IP \[bu] 2
|
|
Implement Google Cloud Storage remote
|
|
.IP \[bu] 2
|
|
Verify Md5sums and Sizes after copies
|
|
.IP \[bu] 2
|
|
Remove times from \[dq]ls\[dq] command \- lists sizes only
|
|
.IP \[bu] 2
|
|
Add add \[dq]lsl\[dq] \- lists times and sizes
|
|
.IP \[bu] 2
|
|
Add \[dq]md5sum\[dq] command
|
|
.SS v1.01 \- 2014\-07\-04
|
|
.IP \[bu] 2
|
|
drive: fix transfer of big files using up lots of memory
|
|
.SS v1.00 \- 2014\-07\-03
|
|
.IP \[bu] 2
|
|
drive: fix whole second dates
|
|
.SS v0.99 \- 2014\-06\-26
|
|
.IP \[bu] 2
|
|
Fix \-\-dry\-run not working
|
|
.IP \[bu] 2
|
|
Make compatible with go 1.1
|
|
.SS v0.98 \- 2014\-05\-30
|
|
.IP \[bu] 2
|
|
s3: Treat missing Content\-Length as 0 for some ceph installations
|
|
.IP \[bu] 2
|
|
rclonetest: add file with a space in
|
|
.SS v0.97 \- 2014\-05\-05
|
|
.IP \[bu] 2
|
|
Implement copying of single files
|
|
.IP \[bu] 2
|
|
s3 & swift: support paths inside containers/buckets
|
|
.SS v0.96 \- 2014\-04\-24
|
|
.IP \[bu] 2
|
|
drive: Fix multiple files of same name being created
|
|
.IP \[bu] 2
|
|
drive: Use o.Update and fs.Put to optimise transfers
|
|
.IP \[bu] 2
|
|
Add version number, \-V and \-\-version
|
|
.SS v0.95 \- 2014\-03\-28
|
|
.IP \[bu] 2
|
|
rclone.org: website, docs and graphics
|
|
.IP \[bu] 2
|
|
drive: fix path parsing
|
|
.SS v0.94 \- 2014\-03\-27
|
|
.IP \[bu] 2
|
|
Change remote format one last time
|
|
.IP \[bu] 2
|
|
GNU style flags
|
|
.SS v0.93 \- 2014\-03\-16
|
|
.IP \[bu] 2
|
|
drive: store token in config file
|
|
.IP \[bu] 2
|
|
cross compile other versions
|
|
.IP \[bu] 2
|
|
set strict permissions on config file
|
|
.SS v0.92 \- 2014\-03\-15
|
|
.IP \[bu] 2
|
|
Config fixes and \-\-config option
|
|
.SS v0.91 \- 2014\-03\-15
|
|
.IP \[bu] 2
|
|
Make config file
|
|
.SS v0.90 \- 2013\-06\-27
|
|
.IP \[bu] 2
|
|
Project named rclone
|
|
.SS v0.00 \- 2012\-11\-18
|
|
.IP \[bu] 2
|
|
Project started
|
|
.SH Bugs and Limitations
|
|
.SS Limitations
|
|
.SS Directory timestamps aren\[aq]t preserved
|
|
.PP
|
|
Rclone doesn\[aq]t currently preserve the timestamps of directories.
|
|
This is because rclone only really considers objects when syncing.
|
|
.SS Rclone struggles with millions of files in a directory/bucket
|
|
.PP
|
|
Currently rclone loads each directory/bucket entirely into memory before
|
|
using it.
|
|
Since each rclone object takes 0.5k\-1k of memory this can take a very
|
|
long time and use a large amount of memory.
|
|
.PP
|
|
Millions of files in a directory tends to occur on bucket\-based remotes
|
|
(e.g.
|
|
S3 buckets) since those remotes do not segregate subdirectories within
|
|
the bucket.
|
|
.SS Bucket based remotes and folders
|
|
.PP
|
|
Bucket based remotes (e.g.
|
|
S3/GCS/Swift/B2) do not have a concept of directories.
|
|
Rclone therefore cannot create directories in them which means that
|
|
empty directories on a bucket based remote will tend to disappear.
|
|
.PP
|
|
Some software creates empty keys ending in \f[C]/\f[R] as directory
|
|
markers.
|
|
Rclone doesn\[aq]t do this as it potentially creates more objects and
|
|
costs more.
|
|
This ability may be added in the future (probably via a flag/option).
|
|
.SS Bugs
|
|
.PP
|
|
Bugs are stored in rclone\[aq]s GitHub project:
|
|
.IP \[bu] 2
|
|
Reported
|
|
bugs (https://github.com/rclone/rclone/issues?q=is%3Aopen+is%3Aissue+label%3Abug)
|
|
.IP \[bu] 2
|
|
Known
|
|
issues (https://github.com/rclone/rclone/issues?q=is%3Aopen+is%3Aissue+milestone%3A%22Known+Problem%22)
|
|
.SS Frequently Asked Questions
|
|
.SS Do all cloud storage systems support all rclone commands
|
|
.PP
|
|
Yes they do.
|
|
All the rclone commands (e.g.
|
|
\f[C]sync\f[R], \f[C]copy\f[R], etc.) will work on all the remote
|
|
storage systems.
|
|
.SS Can I copy the config from one machine to another
|
|
.PP
|
|
Sure! Rclone stores all of its config in a single file.
|
|
If you want to find this file, run \f[C]rclone config file\f[R] which
|
|
will tell you where it is.
|
|
.PP
|
|
See the remote setup docs (https://rclone.org/remote_setup/) for more
|
|
info.
|
|
.SS How do I configure rclone on a remote / headless box with no browser?
|
|
.PP
|
|
This has now been documented in its own remote setup
|
|
page (https://rclone.org/remote_setup/).
|
|
.SS Can rclone sync directly from drive to s3
|
|
.PP
|
|
Rclone can sync between two remote cloud storage systems just fine.
|
|
.PP
|
|
Note that it effectively downloads the file and uploads it again, so the
|
|
node running rclone would need to have lots of bandwidth.
|
|
.PP
|
|
The syncs would be incremental (on a file by file basis).
|
|
.PP
|
|
Eg
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone sync \-i drive:Folder s3:bucket
|
|
\f[R]
|
|
.fi
|
|
.SS Using rclone from multiple locations at the same time
|
|
.PP
|
|
You can use rclone from multiple places at the same time if you choose
|
|
different subdirectory for the output, e.g.
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
Server A> rclone sync \-i /tmp/whatever remote:ServerA
|
|
Server B> rclone sync \-i /tmp/whatever remote:ServerB
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
If you sync to the same directory then you should use rclone copy
|
|
otherwise the two instances of rclone may delete each other\[aq]s files,
|
|
e.g.
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
Server A> rclone copy /tmp/whatever remote:Backup
|
|
Server B> rclone copy /tmp/whatever remote:Backup
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
The file names you upload from Server A and Server B should be different
|
|
in this case, otherwise some file systems (e.g.
|
|
Drive) may make duplicates.
|
|
.SS Why doesn\[aq]t rclone support partial transfers / binary diffs like rsync?
|
|
.PP
|
|
Rclone stores each file you transfer as a native object on the remote
|
|
cloud storage system.
|
|
This means that you can see the files you upload as expected using
|
|
alternative access methods (e.g.
|
|
using the Google Drive web interface).
|
|
There is a 1:1 mapping between files on your hard disk and objects
|
|
created in the cloud storage system.
|
|
.PP
|
|
Cloud storage systems (at least none I\[aq]ve come across yet) don\[aq]t
|
|
support partially uploading an object.
|
|
You can\[aq]t take an existing object, and change some bytes in the
|
|
middle of it.
|
|
.PP
|
|
It would be possible to make a sync system which stored binary diffs
|
|
instead of whole objects like rclone does, but that would break the 1:1
|
|
mapping of files on your hard disk to objects in the remote cloud
|
|
storage system.
|
|
.PP
|
|
All the cloud storage systems support partial downloads of content, so
|
|
it would be possible to make partial downloads work.
|
|
However to make this work efficiently this would require storing a
|
|
significant amount of metadata, which breaks the desired 1:1 mapping of
|
|
files to objects.
|
|
.SS Can rclone do bi\-directional sync?
|
|
.PP
|
|
No, not at present.
|
|
rclone only does uni\-directional sync from A \-> B.
|
|
It may do in the future though since it has all the primitives \- it
|
|
just requires writing the algorithm to do it.
|
|
.SS Can I use rclone with an HTTP proxy?
|
|
.PP
|
|
Yes.
|
|
rclone will follow the standard environment variables for proxies,
|
|
similar to cURL and other programs.
|
|
.PP
|
|
In general the variables are called \f[C]http_proxy\f[R] (for services
|
|
reached over \f[C]http\f[R]) and \f[C]https_proxy\f[R] (for services
|
|
reached over \f[C]https\f[R]).
|
|
Most public services will be using \f[C]https\f[R], but you may wish to
|
|
set both.
|
|
.PP
|
|
The content of the variable is \f[C]protocol://server:port\f[R].
|
|
The protocol value is the one used to talk to the proxy server, itself,
|
|
and is commonly either \f[C]http\f[R] or \f[C]socks5\f[R].
|
|
.PP
|
|
Slightly annoyingly, there is no \f[I]standard\f[R] for the name; some
|
|
applications may use \f[C]http_proxy\f[R] but another one
|
|
\f[C]HTTP_PROXY\f[R].
|
|
The \f[C]Go\f[R] libraries used by \f[C]rclone\f[R] will try both
|
|
variations, but you may wish to set all possibilities.
|
|
So, on Linux, you may end up with code similar to
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
export http_proxy=http://proxyserver:12345
|
|
export https_proxy=$http_proxy
|
|
export HTTP_PROXY=$http_proxy
|
|
export HTTPS_PROXY=$http_proxy
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
The \f[C]NO_PROXY\f[R] allows you to disable the proxy for specific
|
|
hosts.
|
|
Hosts must be comma separated, and can contain domains or parts.
|
|
For instance \[dq]foo.com\[dq] also matches \[dq]bar.foo.com\[dq].
|
|
.PP
|
|
e.g.
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
export no_proxy=localhost,127.0.0.0/8,my.host.name
|
|
export NO_PROXY=$no_proxy
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
Note that the ftp backend does not support \f[C]ftp_proxy\f[R] yet.
|
|
.SS Rclone gives x509: failed to load system roots and no roots provided error
|
|
.PP
|
|
This means that \f[C]rclone\f[R] can\[aq]t find the SSL root
|
|
certificates.
|
|
Likely you are running \f[C]rclone\f[R] on a NAS with a cut\-down Linux
|
|
OS, or possibly on Solaris.
|
|
.PP
|
|
Rclone (via the Go runtime) tries to load the root certificates from
|
|
these places on Linux.
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
\[dq]/etc/ssl/certs/ca\-certificates.crt\[dq], // Debian/Ubuntu/Gentoo etc.
|
|
\[dq]/etc/pki/tls/certs/ca\-bundle.crt\[dq], // Fedora/RHEL
|
|
\[dq]/etc/ssl/ca\-bundle.pem\[dq], // OpenSUSE
|
|
\[dq]/etc/pki/tls/cacert.pem\[dq], // OpenELEC
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
So doing something like this should fix the problem.
|
|
It also sets the time which is important for SSL to work properly.
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
mkdir \-p /etc/ssl/certs/
|
|
curl \-o /etc/ssl/certs/ca\-certificates.crt https://raw.githubusercontent.com/bagder/ca\-bundle/master/ca\-bundle.crt
|
|
ntpclient \-s \-h pool.ntp.org
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
The two environment variables \f[C]SSL_CERT_FILE\f[R] and
|
|
\f[C]SSL_CERT_DIR\f[R], mentioned in the x509
|
|
package (https://godoc.org/crypto/x509), provide an additional way to
|
|
provide the SSL root certificates.
|
|
.PP
|
|
Note that you may need to add the \f[C]\-\-insecure\f[R] option to the
|
|
\f[C]curl\f[R] command line if it doesn\[aq]t work without.
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
curl \-\-insecure \-o /etc/ssl/certs/ca\-certificates.crt https://raw.githubusercontent.com/bagder/ca\-bundle/master/ca\-bundle.crt
|
|
\f[R]
|
|
.fi
|
|
.SS Rclone gives Failed to load config file: function not implemented error
|
|
.PP
|
|
Likely this means that you are running rclone on Linux version not
|
|
supported by the go runtime, ie earlier than version 2.6.23.
|
|
.PP
|
|
See the system requirements section in the go install
|
|
docs (https://golang.org/doc/install) for full details.
|
|
.SS All my uploaded docx/xlsx/pptx files appear as archive/zip
|
|
.PP
|
|
This is caused by uploading these files from a Windows computer which
|
|
hasn\[aq]t got the Microsoft Office suite installed.
|
|
The easiest way to fix is to install the Word viewer and the Microsoft
|
|
Office Compatibility Pack for Word, Excel, and PowerPoint 2007 and later
|
|
versions\[aq] file formats
|
|
.SS tcp lookup some.domain.com no such host
|
|
.PP
|
|
This happens when rclone cannot resolve a domain.
|
|
Please check that your DNS setup is generally working, e.g.
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
# both should print a long list of possible IP addresses
|
|
dig www.googleapis.com # resolve using your default DNS
|
|
dig www.googleapis.com \[at]8.8.8.8 # resolve with Google\[aq]s DNS server
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
If you are using \f[C]systemd\-resolved\f[R] (default on Arch Linux),
|
|
ensure it is at version 233 or higher.
|
|
Previous releases contain a bug which causes not all domains to be
|
|
resolved properly.
|
|
.PP
|
|
Additionally with the \f[C]GODEBUG=netdns=\f[R] environment variable the
|
|
Go resolver decision can be influenced.
|
|
This also allows to resolve certain issues with DNS resolution.
|
|
See the name resolution section in the go
|
|
docs (https://golang.org/pkg/net/#hdr-Name_Resolution).
|
|
.SS The total size reported in the stats for a sync is wrong and keeps changing
|
|
.PP
|
|
It is likely you have more than 10,000 files that need to be synced.
|
|
By default rclone only gets 10,000 files ahead in a sync so as not to
|
|
use up too much memory.
|
|
You can change this default with the
|
|
\-\-max\-backlog (https://rclone.org/docs/#max-backlog-n) flag.
|
|
.SS Rclone is using too much memory or appears to have a memory leak
|
|
.PP
|
|
Rclone is written in Go which uses a garbage collector.
|
|
The default settings for the garbage collector mean that it runs when
|
|
the heap size has doubled.
|
|
.PP
|
|
However it is possible to tune the garbage collector to use less memory
|
|
by setting GOGC (https://dave.cheney.net/tag/gogc) to a lower value, say
|
|
\f[C]export GOGC=20\f[R].
|
|
This will make the garbage collector work harder, reducing memory size
|
|
at the expense of CPU usage.
|
|
.PP
|
|
The most common cause of rclone using lots of memory is a single
|
|
directory with thousands or millions of files in.
|
|
Rclone has to load this entirely into memory as rclone objects.
|
|
Each rclone object takes 0.5k\-1k of memory.
|
|
.SS License
|
|
.PP
|
|
This is free software under the terms of MIT the license (check the
|
|
COPYING file included with the source code).
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
Copyright (C) 2019 by Nick Craig\-Wood https://www.craig\-wood.com/nick/
|
|
|
|
Permission is hereby granted, free of charge, to any person obtaining a copy
|
|
of this software and associated documentation files (the \[dq]Software\[dq]), to deal
|
|
in the Software without restriction, including without limitation the rights
|
|
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
|
|
copies of the Software, and to permit persons to whom the Software is
|
|
furnished to do so, subject to the following conditions:
|
|
|
|
The above copyright notice and this permission notice shall be included in
|
|
all copies or substantial portions of the Software.
|
|
|
|
THE SOFTWARE IS PROVIDED \[dq]AS IS\[dq], WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
|
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
|
|
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
|
|
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
|
|
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
|
|
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
|
|
THE SOFTWARE.
|
|
\f[R]
|
|
.fi
|
|
.SS Authors
|
|
.IP \[bu] 2
|
|
Nick Craig\-Wood <nick@craig-wood.com>
|
|
.SS Contributors
|
|
.PP
|
|
{{< rem
|
|
\f[C]email addresses removed from here need to be addeed to bin/.ignore\-emails to make sure update\-authors.py doesn\[aq]t immediately put them back in again.\f[R]
|
|
>}}
|
|
.IP \[bu] 2
|
|
Alex Couper <amcouper@gmail.com>
|
|
.IP \[bu] 2
|
|
Leonid Shalupov <leonid@shalupov.com> <shalupov@diverse.org.ru>
|
|
.IP \[bu] 2
|
|
Shimon Doodkin <helpmepro1@gmail.com>
|
|
.IP \[bu] 2
|
|
Colin Nicholson <colin@colinn.com>
|
|
.IP \[bu] 2
|
|
Klaus Post <klauspost@gmail.com>
|
|
.IP \[bu] 2
|
|
Sergey Tolmachev <tolsi.ru@gmail.com>
|
|
.IP \[bu] 2
|
|
Adriano Aur\['e]lio Meirelles <adriano@atinge.com>
|
|
.IP \[bu] 2
|
|
C.
|
|
Bess <cbess@users.noreply.github.com>
|
|
.IP \[bu] 2
|
|
Dmitry Burdeev <dibu28@gmail.com>
|
|
.IP \[bu] 2
|
|
Joseph Spurrier <github@josephspurrier.com>
|
|
.IP \[bu] 2
|
|
Bj\[:o]rn Harrtell <bjorn@wololo.org>
|
|
.IP \[bu] 2
|
|
Xavier Lucas <xavier.lucas@corp.ovh.com>
|
|
.IP \[bu] 2
|
|
Werner Beroux <werner@beroux.com>
|
|
.IP \[bu] 2
|
|
Brian Stengaard <brian@stengaard.eu>
|
|
.IP \[bu] 2
|
|
Jakub Gedeon <jgedeon@sofi.com>
|
|
.IP \[bu] 2
|
|
Jim Tittsler <jwt@onjapan.net>
|
|
.IP \[bu] 2
|
|
Michal Witkowski <michal@improbable.io>
|
|
.IP \[bu] 2
|
|
Fabian Ruff <fabian.ruff@sap.com>
|
|
.IP \[bu] 2
|
|
Leigh Klotz <klotz@quixey.com>
|
|
.IP \[bu] 2
|
|
Romain Lapray <lapray.romain@gmail.com>
|
|
.IP \[bu] 2
|
|
Justin R.
|
|
Wilson <jrw972@gmail.com>
|
|
.IP \[bu] 2
|
|
Antonio Messina <antonio.s.messina@gmail.com>
|
|
.IP \[bu] 2
|
|
Stefan G.
|
|
Weichinger <office@oops.co.at>
|
|
.IP \[bu] 2
|
|
Per Cederberg <cederberg@gmail.com>
|
|
.IP \[bu] 2
|
|
Radek \[vS]enfeld <rush@logic.cz>
|
|
.IP \[bu] 2
|
|
Fredrik Fornwall <fredrik@fornwall.net>
|
|
.IP \[bu] 2
|
|
Asko Tamm <asko@deekit.net>
|
|
.IP \[bu] 2
|
|
xor\-zz <xor@gstocco.com>
|
|
.IP \[bu] 2
|
|
Tomasz Mazur <tmazur90@gmail.com>
|
|
.IP \[bu] 2
|
|
Marco Paganini <paganini@paganini.net>
|
|
.IP \[bu] 2
|
|
Felix B\[:u]nemann <buenemann@louis.info>
|
|
.IP \[bu] 2
|
|
Durval Menezes <jmrclone@durval.com>
|
|
.IP \[bu] 2
|
|
Luiz Carlos Rumbelsperger Viana <maxd13_luiz_carlos@hotmail.com>
|
|
.IP \[bu] 2
|
|
Stefan Breunig <stefan-github@yrden.de>
|
|
.IP \[bu] 2
|
|
Alishan Ladhani <ali-l@users.noreply.github.com>
|
|
.IP \[bu] 2
|
|
0xJAKE <0xJAKE@users.noreply.github.com>
|
|
.IP \[bu] 2
|
|
Thibault Molleman <thibaultmol@users.noreply.github.com>
|
|
.IP \[bu] 2
|
|
Scott McGillivray <scott.mcgillivray@gmail.com>
|
|
.IP \[bu] 2
|
|
Bj\[/o]rn Erik Pedersen <bjorn.erik.pedersen@gmail.com>
|
|
.IP \[bu] 2
|
|
Lukas Loesche <lukas@mesosphere.io>
|
|
.IP \[bu] 2
|
|
emyarod <allllaboutyou@gmail.com>
|
|
.IP \[bu] 2
|
|
T.C.
|
|
Ferguson <tcf909@gmail.com>
|
|
.IP \[bu] 2
|
|
Brandur <brandur@mutelight.org>
|
|
.IP \[bu] 2
|
|
Dario Giovannetti <dev@dariogiovannetti.net>
|
|
.IP \[bu] 2
|
|
K\['a]roly Ol\['a]h <okaresz@aol.com>
|
|
.IP \[bu] 2
|
|
Jon Yergatian <jon@macfanatic.ca>
|
|
.IP \[bu] 2
|
|
Jack Schmidt <github@mowsey.org>
|
|
.IP \[bu] 2
|
|
Dedsec1 <Dedsec1@users.noreply.github.com>
|
|
.IP \[bu] 2
|
|
Hisham Zarka <hzarka@gmail.com>
|
|
.IP \[bu] 2
|
|
J\['e]r\[^o]me Vizcaino <jerome.vizcaino@gmail.com>
|
|
.IP \[bu] 2
|
|
Mike Tesch <mjt6129@rit.edu>
|
|
.IP \[bu] 2
|
|
Marvin Watson <marvwatson@users.noreply.github.com>
|
|
.IP \[bu] 2
|
|
Danny Tsai <danny8376@gmail.com>
|
|
.IP \[bu] 2
|
|
Yoni Jah <yonjah+git@gmail.com> <yonjah+github@gmail.com>
|
|
.IP \[bu] 2
|
|
Stephen Harris <github@spuddy.org> <sweharris@users.noreply.github.com>
|
|
.IP \[bu] 2
|
|
Ihor Dvoretskyi <ihor.dvoretskyi@gmail.com>
|
|
.IP \[bu] 2
|
|
Jon Craton <jncraton@gmail.com>
|
|
.IP \[bu] 2
|
|
Hraban Luyat <hraban@0brg.net>
|
|
.IP \[bu] 2
|
|
Michael Ledin <mledin89@gmail.com>
|
|
.IP \[bu] 2
|
|
Martin Kristensen <me@azgul.com>
|
|
.IP \[bu] 2
|
|
Too Much IO <toomuchio@users.noreply.github.com>
|
|
.IP \[bu] 2
|
|
Anisse Astier <anisse@astier.eu>
|
|
.IP \[bu] 2
|
|
Zahiar Ahmed <zahiar@live.com>
|
|
.IP \[bu] 2
|
|
Igor Kharin <igorkharin@gmail.com>
|
|
.IP \[bu] 2
|
|
Bill Zissimopoulos <billziss@navimatics.com>
|
|
.IP \[bu] 2
|
|
Bob Potter <bobby.potter@gmail.com>
|
|
.IP \[bu] 2
|
|
Steven Lu <tacticalazn@gmail.com>
|
|
.IP \[bu] 2
|
|
Sjur Fredriksen <sjurtf@ifi.uio.no>
|
|
.IP \[bu] 2
|
|
Ruwbin <hubus12345@gmail.com>
|
|
.IP \[bu] 2
|
|
Fabian M\[:o]ller <fabianm88@gmail.com> <f.moeller@nynex.de>
|
|
.IP \[bu] 2
|
|
Edward Q.
|
|
Bridges <github@eqbridges.com>
|
|
.IP \[bu] 2
|
|
Vasiliy Tolstov <v.tolstov@selfip.ru>
|
|
.IP \[bu] 2
|
|
Harshavardhana <harsha@minio.io>
|
|
.IP \[bu] 2
|
|
sainaen <sainaen@gmail.com>
|
|
.IP \[bu] 2
|
|
gdm85 <gdm85@users.noreply.github.com>
|
|
.IP \[bu] 2
|
|
Yaroslav Halchenko <debian@onerussian.com>
|
|
.IP \[bu] 2
|
|
John Papandriopoulos <jpap@users.noreply.github.com>
|
|
.IP \[bu] 2
|
|
Zhiming Wang <zmwangx@gmail.com>
|
|
.IP \[bu] 2
|
|
Andy Pilate <cubox@cubox.me>
|
|
.IP \[bu] 2
|
|
Oliver Heyme <olihey@googlemail.com> <olihey@users.noreply.github.com>
|
|
<de8olihe@lego.com>
|
|
.IP \[bu] 2
|
|
wuyu <wuyu@yunify.com>
|
|
.IP \[bu] 2
|
|
Andrei Dragomir <adragomi@adobe.com>
|
|
.IP \[bu] 2
|
|
Christian Br\[:u]ggemann <mail@cbruegg.com>
|
|
.IP \[bu] 2
|
|
Alex McGrath Kraak <amkdude@gmail.com>
|
|
.IP \[bu] 2
|
|
bpicode <bjoern.pirnay@googlemail.com>
|
|
.IP \[bu] 2
|
|
Daniel Jagszent <daniel@jagszent.de>
|
|
.IP \[bu] 2
|
|
Josiah White <thegenius2009@gmail.com>
|
|
.IP \[bu] 2
|
|
Ishuah Kariuki <kariuki@ishuah.com> <ishuah91@gmail.com>
|
|
.IP \[bu] 2
|
|
Jan Varho <jan@varho.org>
|
|
.IP \[bu] 2
|
|
Girish Ramakrishnan <girish@cloudron.io>
|
|
.IP \[bu] 2
|
|
LingMan <LingMan@users.noreply.github.com>
|
|
.IP \[bu] 2
|
|
Jacob McNamee <jacobmcnamee@gmail.com>
|
|
.IP \[bu] 2
|
|
jersou <jertux@gmail.com>
|
|
.IP \[bu] 2
|
|
thierry <thierry@substantiel.fr>
|
|
.IP \[bu] 2
|
|
Simon Leinen <simon.leinen@gmail.com> <ubuntu@s3-test.novalocal>
|
|
.IP \[bu] 2
|
|
Dan Dascalescu <ddascalescu+github@gmail.com>
|
|
.IP \[bu] 2
|
|
Jason Rose <jason@jro.io>
|
|
.IP \[bu] 2
|
|
Andrew Starr\-Bochicchio <a.starr.b@gmail.com>
|
|
.IP \[bu] 2
|
|
John Leach <john@johnleach.co.uk>
|
|
.IP \[bu] 2
|
|
Corban Raun <craun@instructure.com>
|
|
.IP \[bu] 2
|
|
Pierre Carlson <mpcarl@us.ibm.com>
|
|
.IP \[bu] 2
|
|
Ernest Borowski <er.borowski@gmail.com>
|
|
.IP \[bu] 2
|
|
Remus Bunduc <remus.bunduc@gmail.com>
|
|
.IP \[bu] 2
|
|
Iakov Davydov <iakov.davydov@unil.ch> <dav05.gith@myths.ru>
|
|
.IP \[bu] 2
|
|
Jakub Tasiemski <tasiemski@gmail.com>
|
|
.IP \[bu] 2
|
|
David Minor <dminor@saymedia.com>
|
|
.IP \[bu] 2
|
|
Tim Cooijmans <cooijmans.tim@gmail.com>
|
|
.IP \[bu] 2
|
|
Laurence <liuxy6@gmail.com>
|
|
.IP \[bu] 2
|
|
Giovanni Pizzi <gio.piz@gmail.com>
|
|
.IP \[bu] 2
|
|
Filip Bartodziej <filipbartodziej@gmail.com>
|
|
.IP \[bu] 2
|
|
Jon Fautley <jon@dead.li>
|
|
.IP \[bu] 2
|
|
lewapm <32110057+lewapm@users.noreply.github.com>
|
|
.IP \[bu] 2
|
|
Yassine Imounachen <yassine256@gmail.com>
|
|
.IP \[bu] 2
|
|
Chris Redekop <chris-redekop@users.noreply.github.com>
|
|
<chris.redekop@gmail.com>
|
|
.IP \[bu] 2
|
|
Jon Fautley <jon@adenoid.appstal.co.uk>
|
|
.IP \[bu] 2
|
|
Will Gunn <WillGunn@users.noreply.github.com>
|
|
.IP \[bu] 2
|
|
Lucas Bremgartner <lucas@bremis.ch>
|
|
.IP \[bu] 2
|
|
Jody Frankowski <jody.frankowski@gmail.com>
|
|
.IP \[bu] 2
|
|
Andreas Roussos <arouss1980@gmail.com>
|
|
.IP \[bu] 2
|
|
nbuchanan <nbuchanan@utah.gov>
|
|
.IP \[bu] 2
|
|
Durval Menezes <rclone@durval.com>
|
|
.IP \[bu] 2
|
|
Victor <vb-github@viblo.se>
|
|
.IP \[bu] 2
|
|
Mateusz <pabian.mateusz@gmail.com>
|
|
.IP \[bu] 2
|
|
Daniel Loader <spicypixel@gmail.com>
|
|
.IP \[bu] 2
|
|
David0rk <davidork@gmail.com>
|
|
.IP \[bu] 2
|
|
Alexander Neumann <alexander@bumpern.de>
|
|
.IP \[bu] 2
|
|
Giri Badanahatti <gbadanahatti@us.ibm.com@Giris-MacBook-Pro.local>
|
|
.IP \[bu] 2
|
|
Leo R.
|
|
Lundgren <leo@finalresort.org>
|
|
.IP \[bu] 2
|
|
wolfv <wolfv6@users.noreply.github.com>
|
|
.IP \[bu] 2
|
|
Dave Pedu <dave@davepedu.com>
|
|
.IP \[bu] 2
|
|
Stefan Lindblom <lindblom@spotify.com>
|
|
.IP \[bu] 2
|
|
seuffert <oliver@seuffert.biz>
|
|
.IP \[bu] 2
|
|
gbadanahatti <37121690+gbadanahatti@users.noreply.github.com>
|
|
.IP \[bu] 2
|
|
Keith Goldfarb <barkofdelight@gmail.com>
|
|
.IP \[bu] 2
|
|
Steve Kriss <steve@heptio.com>
|
|
.IP \[bu] 2
|
|
Chih\-Hsuan Yen <yan12125@gmail.com>
|
|
.IP \[bu] 2
|
|
Alexander Neumann <fd0@users.noreply.github.com>
|
|
.IP \[bu] 2
|
|
Matt Holt <mholt@users.noreply.github.com>
|
|
.IP \[bu] 2
|
|
Eri Bastos <bastos.eri@gmail.com>
|
|
.IP \[bu] 2
|
|
Michael P.
|
|
Dubner <pywebmail@list.ru>
|
|
.IP \[bu] 2
|
|
Antoine GIRARD <sapk@users.noreply.github.com>
|
|
.IP \[bu] 2
|
|
Mateusz Piotrowski <mpp302@gmail.com>
|
|
.IP \[bu] 2
|
|
Animosity022 <animosity22@users.noreply.github.com>
|
|
<earl.texter@gmail.com>
|
|
.IP \[bu] 2
|
|
Peter Baumgartner <pete@lincolnloop.com>
|
|
.IP \[bu] 2
|
|
Craig Rachel <craig@craigrachel.com>
|
|
.IP \[bu] 2
|
|
Michael G.
|
|
Noll <miguno@users.noreply.github.com>
|
|
.IP \[bu] 2
|
|
hensur <me@hensur.de>
|
|
.IP \[bu] 2
|
|
Oliver Heyme <de8olihe@lego.com>
|
|
.IP \[bu] 2
|
|
Richard Yang <richard@yenforyang.com>
|
|
.IP \[bu] 2
|
|
Piotr Oleszczyk <piotr.oleszczyk@gmail.com>
|
|
.IP \[bu] 2
|
|
Rodrigo <rodarima@gmail.com>
|
|
.IP \[bu] 2
|
|
NoLooseEnds <NoLooseEnds@users.noreply.github.com>
|
|
.IP \[bu] 2
|
|
Jakub Karlicek <jakub@karlicek.me>
|
|
.IP \[bu] 2
|
|
John Clayton <john@codemonkeylabs.com>
|
|
.IP \[bu] 2
|
|
Kasper Byrdal Nielsen <byrdal76@gmail.com>
|
|
.IP \[bu] 2
|
|
Benjamin Joseph Dag <bjdag1234@users.noreply.github.com>
|
|
.IP \[bu] 2
|
|
themylogin <themylogin@gmail.com>
|
|
.IP \[bu] 2
|
|
Onno Zweers <onno.zweers@surfsara.nl>
|
|
.IP \[bu] 2
|
|
Jasper Lievisse Adriaanse <jasper@humppa.nl>
|
|
.IP \[bu] 2
|
|
sandeepkru <sandeep.ummadi@gmail.com>
|
|
<sandeepkru@users.noreply.github.com>
|
|
.IP \[bu] 2
|
|
HerrH <atomtigerzoo@users.noreply.github.com>
|
|
.IP \[bu] 2
|
|
Andrew <4030760+sparkyman215@users.noreply.github.com>
|
|
.IP \[bu] 2
|
|
dan smith <XX1011@gmail.com>
|
|
.IP \[bu] 2
|
|
Oleg Kovalov <iamolegkovalov@gmail.com>
|
|
.IP \[bu] 2
|
|
Ruben Vandamme <github-com-00ff86@vandamme.email>
|
|
.IP \[bu] 2
|
|
Cnly <minecnly@gmail.com>
|
|
.IP \[bu] 2
|
|
Andres Alvarez <1671935+kir4h@users.noreply.github.com>
|
|
.IP \[bu] 2
|
|
reddi1 <xreddi@gmail.com>
|
|
.IP \[bu] 2
|
|
Matt Tucker <matthewtckr@gmail.com>
|
|
.IP \[bu] 2
|
|
Sebastian B\[:u]nger <buengese@gmail.com> <buengese@protonmail.com>
|
|
.IP \[bu] 2
|
|
Martin Polden <mpolden@mpolden.no>
|
|
.IP \[bu] 2
|
|
Alex Chen <Cnly@users.noreply.github.com>
|
|
.IP \[bu] 2
|
|
Denis <deniskovpen@gmail.com>
|
|
.IP \[bu] 2
|
|
bsteiss <35940619+bsteiss@users.noreply.github.com>
|
|
.IP \[bu] 2
|
|
C\['e]dric Connes <cedric.connes@gmail.com>
|
|
.IP \[bu] 2
|
|
Dr.
|
|
Tobias Quathamer <toddy15@users.noreply.github.com>
|
|
.IP \[bu] 2
|
|
dcpu <42736967+dcpu@users.noreply.github.com>
|
|
.IP \[bu] 2
|
|
Sheldon Rupp <me@shel.io>
|
|
.IP \[bu] 2
|
|
albertony <12441419+albertony@users.noreply.github.com>
|
|
.IP \[bu] 2
|
|
cron410 <cron410@gmail.com>
|
|
.IP \[bu] 2
|
|
Anagh Kumar Baranwal <6824881+darthShadow@users.noreply.github.com>
|
|
.IP \[bu] 2
|
|
Felix Brucker <felix@felixbrucker.com>
|
|
.IP \[bu] 2
|
|
Santiago Rodr\['i]guez <scollazo@users.noreply.github.com>
|
|
.IP \[bu] 2
|
|
Craig Miskell <craig.miskell@fluxfederation.com>
|
|
.IP \[bu] 2
|
|
Antoine GIRARD <sapk@sapk.fr>
|
|
.IP \[bu] 2
|
|
Joanna Marek <joanna.marek@u2i.com>
|
|
.IP \[bu] 2
|
|
frenos <frenos@users.noreply.github.com>
|
|
.IP \[bu] 2
|
|
ssaqua <ssaqua@users.noreply.github.com>
|
|
.IP \[bu] 2
|
|
xnaas <me@xnaas.info>
|
|
.IP \[bu] 2
|
|
Frantisek Fuka <fuka@fuxoft.cz>
|
|
.IP \[bu] 2
|
|
Paul Kohout <pauljkohout@yahoo.com>
|
|
.IP \[bu] 2
|
|
dcpu <43330287+dcpu@users.noreply.github.com>
|
|
.IP \[bu] 2
|
|
jackyzy823 <jackyzy823@gmail.com>
|
|
.IP \[bu] 2
|
|
David Haguenauer <ml@kurokatta.org>
|
|
.IP \[bu] 2
|
|
teresy <hi.teresy@gmail.com>
|
|
.IP \[bu] 2
|
|
buergi <patbuergi@gmx.de>
|
|
.IP \[bu] 2
|
|
Florian Gamboeck <mail@floga.de>
|
|
.IP \[bu] 2
|
|
Ralf Hemberger <10364191+rhemberger@users.noreply.github.com>
|
|
.IP \[bu] 2
|
|
Scott Edlund <sedlund@users.noreply.github.com>
|
|
.IP \[bu] 2
|
|
Erik Swanson <erik@retailnext.net>
|
|
.IP \[bu] 2
|
|
Jake Coggiano <jake@stripe.com>
|
|
.IP \[bu] 2
|
|
brused27 <brused27@noemailaddress>
|
|
.IP \[bu] 2
|
|
Peter Kaminski <kaminski@istori.com>
|
|
.IP \[bu] 2
|
|
Henry Ptasinski <henry@logout.com>
|
|
.IP \[bu] 2
|
|
Alexander <kharkovalexander@gmail.com>
|
|
.IP \[bu] 2
|
|
Garry McNulty <garrmcnu@gmail.com>
|
|
.IP \[bu] 2
|
|
Mathieu Carbou <mathieu.carbou@gmail.com>
|
|
.IP \[bu] 2
|
|
Mark Otway <mark@otway.com>
|
|
.IP \[bu] 2
|
|
William Cocker <37018962+WilliamCocker@users.noreply.github.com>
|
|
.IP \[bu] 2
|
|
Fran\[,c]ois Leurent <131.js@cloudyks.org>
|
|
.IP \[bu] 2
|
|
Arkadius Stefanski <arkste@gmail.com>
|
|
.IP \[bu] 2
|
|
Jay <dev@jaygoel.com>
|
|
.IP \[bu] 2
|
|
andrea rota <a@xelera.eu>
|
|
.IP \[bu] 2
|
|
nicolov <nicolov@users.noreply.github.com>
|
|
.IP \[bu] 2
|
|
Dario Guzik <dario@guzik.com.ar>
|
|
.IP \[bu] 2
|
|
qip <qip@users.noreply.github.com>
|
|
.IP \[bu] 2
|
|
yair\[at]unicorn <yair@unicorn>
|
|
.IP \[bu] 2
|
|
Matt Robinson <brimstone@the.narro.ws>
|
|
.IP \[bu] 2
|
|
kayrus <kay.diam@gmail.com>
|
|
.IP \[bu] 2
|
|
R\['e]my L\['e]one <remy.leone@gmail.com>
|
|
.IP \[bu] 2
|
|
Wojciech Smigielski <wojciech.hieronim.smigielski@gmail.com>
|
|
.IP \[bu] 2
|
|
weetmuts <oehrstroem@gmail.com>
|
|
.IP \[bu] 2
|
|
Jonathan <vanillajonathan@users.noreply.github.com>
|
|
.IP \[bu] 2
|
|
James Carpenter <orbsmiv@users.noreply.github.com>
|
|
.IP \[bu] 2
|
|
Vince <vince0villamora@gmail.com>
|
|
.IP \[bu] 2
|
|
Nestar47 <47841759+Nestar47@users.noreply.github.com>
|
|
.IP \[bu] 2
|
|
Six <brbsix@gmail.com>
|
|
.IP \[bu] 2
|
|
Alexandru Bumbacea <alexandru.bumbacea@booking.com>
|
|
.IP \[bu] 2
|
|
calisro <robert.calistri@gmail.com>
|
|
.IP \[bu] 2
|
|
Dr.Rx <david.rey@nventive.com>
|
|
.IP \[bu] 2
|
|
marcintustin <marcintustin@users.noreply.github.com>
|
|
.IP \[bu] 2
|
|
jaKa Mo\[u010D]nik <jaka@koofr.net>
|
|
.IP \[bu] 2
|
|
Fionera <fionera@fionera.de>
|
|
.IP \[bu] 2
|
|
Dan Walters <dan@walters.io>
|
|
.IP \[bu] 2
|
|
Danil Semelenov <sgtpep@users.noreply.github.com>
|
|
.IP \[bu] 2
|
|
xopez <28950736+xopez@users.noreply.github.com>
|
|
.IP \[bu] 2
|
|
Ben Boeckel <mathstuf@gmail.com>
|
|
.IP \[bu] 2
|
|
Manu <manu@snapdragon.cc>
|
|
.IP \[bu] 2
|
|
Kyle E.
|
|
Mitchell <kyle@kemitchell.com>
|
|
.IP \[bu] 2
|
|
Gary Kim <gary@garykim.dev>
|
|
.IP \[bu] 2
|
|
Jon <jonathn@github.com>
|
|
.IP \[bu] 2
|
|
Jeff Quinn <jeffrey.quinn@bluevoyant.com>
|
|
.IP \[bu] 2
|
|
Peter Berbec <peter@berbec.com>
|
|
.IP \[bu] 2
|
|
didil <1284255+didil@users.noreply.github.com>
|
|
.IP \[bu] 2
|
|
id01 <gaviniboom@gmail.com>
|
|
.IP \[bu] 2
|
|
Robert Marko <robimarko@gmail.com>
|
|
.IP \[bu] 2
|
|
Philip Harvey <32467456+pharveybattelle@users.noreply.github.com>
|
|
.IP \[bu] 2
|
|
JorisE <JorisE@users.noreply.github.com>
|
|
.IP \[bu] 2
|
|
garry415 <garry.415@gmail.com>
|
|
.IP \[bu] 2
|
|
forgems <forgems@gmail.com>
|
|
.IP \[bu] 2
|
|
Florian Apolloner <florian@apolloner.eu>
|
|
.IP \[bu] 2
|
|
Aleksandar Jankovi\['c] <office@ajankovic.com>
|
|
<ajankovic@users.noreply.github.com>
|
|
.IP \[bu] 2
|
|
Maran <maran@protonmail.com>
|
|
.IP \[bu] 2
|
|
nguyenhuuluan434 <nguyenhuuluan434@gmail.com>
|
|
.IP \[bu] 2
|
|
Laura Hausmann <zotan@zotan.pw> <laura@hausmann.dev>
|
|
.IP \[bu] 2
|
|
yparitcher <y@paritcher.com>
|
|
.IP \[bu] 2
|
|
AbelThar <abela.tharen@gmail.com>
|
|
.IP \[bu] 2
|
|
Matti Niemenmaa <matti.niemenmaa+git@iki.fi>
|
|
.IP \[bu] 2
|
|
Russell Davis <russelldavis@users.noreply.github.com>
|
|
.IP \[bu] 2
|
|
Yi FU <yi.fu@tink.se>
|
|
.IP \[bu] 2
|
|
Paul Millar <paul.millar@desy.de>
|
|
.IP \[bu] 2
|
|
justinalin <justinalin@qnap.com>
|
|
.IP \[bu] 2
|
|
EliEron <subanimehd@gmail.com>
|
|
.IP \[bu] 2
|
|
justina777 <chiahuei.lin@gmail.com>
|
|
.IP \[bu] 2
|
|
Chaitanya Bankanhal <bchaitanya15@gmail.com>
|
|
.IP \[bu] 2
|
|
Micha\[/l] Matczuk <michal@scylladb.com>
|
|
.IP \[bu] 2
|
|
Macavirus <macavirus@zoho.com>
|
|
.IP \[bu] 2
|
|
Abhinav Sharma <abhi18av@users.noreply.github.com>
|
|
.IP \[bu] 2
|
|
ginvine <34869051+ginvine@users.noreply.github.com>
|
|
.IP \[bu] 2
|
|
Patrick Wang <mail6543210@yahoo.com.tw>
|
|
.IP \[bu] 2
|
|
Cenk Alti <cenkalti@gmail.com>
|
|
.IP \[bu] 2
|
|
Andreas Chlupka <andy@chlupka.com>
|
|
.IP \[bu] 2
|
|
Alfonso Montero <amontero@tinet.org>
|
|
.IP \[bu] 2
|
|
Ivan Andreev <ivandeex@gmail.com>
|
|
.IP \[bu] 2
|
|
David Baumgold <david@davidbaumgold.com>
|
|
.IP \[bu] 2
|
|
Lars Lehtonen <lars.lehtonen@gmail.com>
|
|
.IP \[bu] 2
|
|
Matei David <matei.david@gmail.com>
|
|
.IP \[bu] 2
|
|
David <david.bramwell@endemolshine.com>
|
|
.IP \[bu] 2
|
|
Anthony Rusdi <33247310+antrusd@users.noreply.github.com>
|
|
.IP \[bu] 2
|
|
Richard Patel <me@terorie.dev>
|
|
.IP \[bu] 2
|
|
\[u5E84]\[u5929]\[u7FFC] <zty0826@gmail.com>
|
|
.IP \[bu] 2
|
|
SwitchJS <dev@switchjs.com>
|
|
.IP \[bu] 2
|
|
Raphael <PowershellNinja@users.noreply.github.com>
|
|
.IP \[bu] 2
|
|
Sezal Agrawal <sezalagrawal@gmail.com>
|
|
.IP \[bu] 2
|
|
Tyler <TylerNakamura@users.noreply.github.com>
|
|
.IP \[bu] 2
|
|
Brett Dutro <brett.dutro@gmail.com>
|
|
.IP \[bu] 2
|
|
Vighnesh SK <booterror99@gmail.com>
|
|
.IP \[bu] 2
|
|
Arijit Biswas <dibbyo456@gmail.com>
|
|
.IP \[bu] 2
|
|
Michele Caci <michele.caci@gmail.com>
|
|
.IP \[bu] 2
|
|
AlexandrBoltris <ua2fgb@gmail.com>
|
|
.IP \[bu] 2
|
|
Bryce Larson <blarson@saltstack.com>
|
|
.IP \[bu] 2
|
|
Carlos Ferreyra <crypticmind@gmail.com>
|
|
.IP \[bu] 2
|
|
Saksham Khanna <sakshamkhanna@outlook.com>
|
|
.IP \[bu] 2
|
|
dausruddin <5763466+dausruddin@users.noreply.github.com>
|
|
.IP \[bu] 2
|
|
zero\-24 <zero-24@users.noreply.github.com>
|
|
.IP \[bu] 2
|
|
Xiaoxing Ye <ye@xiaoxing.us>
|
|
.IP \[bu] 2
|
|
Barry Muldrey <barry@muldrey.net>
|
|
.IP \[bu] 2
|
|
Sebastian Brandt <sebastian.brandt@friday.de>
|
|
.IP \[bu] 2
|
|
Marco Molteni <marco.molteni@mailbox.org>
|
|
.IP \[bu] 2
|
|
Ankur Gupta <ankur0493@gmail.com>
|
|
<7876747+ankur0493@users.noreply.github.com>
|
|
.IP \[bu] 2
|
|
Maciej Zimnoch <maciej@scylladb.com>
|
|
.IP \[bu] 2
|
|
anuar45 <serdaliyev.anuar@gmail.com>
|
|
.IP \[bu] 2
|
|
Fernando <ferferga@users.noreply.github.com>
|
|
.IP \[bu] 2
|
|
David Cole <david.cole@sohonet.com>
|
|
.IP \[bu] 2
|
|
Wei He <git@weispot.com>
|
|
.IP \[bu] 2
|
|
Outvi V <19144373+outloudvi@users.noreply.github.com>
|
|
.IP \[bu] 2
|
|
Thomas Kriechbaumer <thomas@kriechbaumer.name>
|
|
.IP \[bu] 2
|
|
Tennix <tennix@users.noreply.github.com>
|
|
.IP \[bu] 2
|
|
Ole Sch\[:u]tt <ole@schuett.name>
|
|
.IP \[bu] 2
|
|
Kuang\-che Wu <kcwu@csie.org>
|
|
.IP \[bu] 2
|
|
Thomas Eales <wingsuit@users.noreply.github.com>
|
|
.IP \[bu] 2
|
|
Paul Tinsley <paul.tinsley@vitalsource.com>
|
|
.IP \[bu] 2
|
|
Felix Hungenberg <git@shiftgeist.com>
|
|
.IP \[bu] 2
|
|
Benjamin Richter <github@dev.telepath.de>
|
|
.IP \[bu] 2
|
|
landall <cst_zf@qq.com>
|
|
.IP \[bu] 2
|
|
thestigma <thestigma@gmail.com>
|
|
.IP \[bu] 2
|
|
jtagcat <38327267+jtagcat@users.noreply.github.com>
|
|
.IP \[bu] 2
|
|
Damon Permezel <permezel@me.com>
|
|
.IP \[bu] 2
|
|
boosh <boosh@users.noreply.github.com>
|
|
.IP \[bu] 2
|
|
unbelauscht <58393353+unbelauscht@users.noreply.github.com>
|
|
.IP \[bu] 2
|
|
Motonori IWAMURO <vmi@nifty.com>
|
|
.IP \[bu] 2
|
|
Benjapol Worakan <benwrk@live.com>
|
|
.IP \[bu] 2
|
|
Dave Koston <dave.koston@stackpath.com>
|
|
.IP \[bu] 2
|
|
Durval Menezes <DurvalMenezes@users.noreply.github.com>
|
|
.IP \[bu] 2
|
|
Tim Gallant <me@timgallant.us>
|
|
.IP \[bu] 2
|
|
Frederick Zhang <frederick888@tsundere.moe>
|
|
.IP \[bu] 2
|
|
valery1707 <valery1707@gmail.com>
|
|
.IP \[bu] 2
|
|
Yves G <theYinYeti@yalis.fr>
|
|
.IP \[bu] 2
|
|
Shing Kit Chan <chanshingkit@gmail.com>
|
|
.IP \[bu] 2
|
|
Franklyn Tackitt <franklyn@tackitt.net>
|
|
.IP \[bu] 2
|
|
Robert\-Andr\['e] Mauchin <zebob.m@gmail.com>
|
|
.IP \[bu] 2
|
|
evileye <48332831+ibiruai@users.noreply.github.com>
|
|
.IP \[bu] 2
|
|
Joachim Brandon LeBlanc <brandon@leblanc.codes>
|
|
.IP \[bu] 2
|
|
Patryk Jakuszew <patryk.jakuszew@gmail.com>
|
|
.IP \[bu] 2
|
|
fishbullet <shindu666@gmail.com>
|
|
.IP \[bu] 2
|
|
greatroar <\[at]>
|
|
.IP \[bu] 2
|
|
Bernd Schoolmann <mail@quexten.com>
|
|
.IP \[bu] 2
|
|
Elan Ruusam\[:a]e <glen@pld-linux.org>
|
|
.IP \[bu] 2
|
|
Max Sum <max@lolyculture.com>
|
|
.IP \[bu] 2
|
|
Mark Spieth <mspieth@users.noreply.github.com>
|
|
.IP \[bu] 2
|
|
harry <me@harry.plus>
|
|
.IP \[bu] 2
|
|
Samantha McVey <samantham@posteo.net>
|
|
.IP \[bu] 2
|
|
Jack Anderson <jack.anderson@metaswitch.com>
|
|
.IP \[bu] 2
|
|
Michael G <draget@speciesm.net>
|
|
.IP \[bu] 2
|
|
Brandon Philips <brandon@ifup.org>
|
|
.IP \[bu] 2
|
|
Daven <dooven@users.noreply.github.com>
|
|
.IP \[bu] 2
|
|
Martin Stone <martin@d7415.co.uk>
|
|
.IP \[bu] 2
|
|
David Bramwell <13053834+dbramwell@users.noreply.github.com>
|
|
.IP \[bu] 2
|
|
Sunil Patra <snl_su@live.com>
|
|
.IP \[bu] 2
|
|
Adam Stroud <adam.stroud@gmail.com>
|
|
.IP \[bu] 2
|
|
Kush <kushsharma@users.noreply.github.com>
|
|
.IP \[bu] 2
|
|
Matan Rosenberg <matan129@gmail.com>
|
|
.IP \[bu] 2
|
|
gitch1 <63495046+gitch1@users.noreply.github.com>
|
|
.IP \[bu] 2
|
|
ElonH <elonhhuang@gmail.com>
|
|
.IP \[bu] 2
|
|
Fred <fred@creativeprojects.tech>
|
|
.IP \[bu] 2
|
|
S\['e]bastien Gross <renard@users.noreply.github.com>
|
|
.IP \[bu] 2
|
|
Maxime Suret <11944422+msuret@users.noreply.github.com>
|
|
.IP \[bu] 2
|
|
Caleb Case <caleb@storj.io>
|
|
.IP \[bu] 2
|
|
Ben Zenker <imbenzenker@gmail.com>
|
|
.IP \[bu] 2
|
|
Martin Michlmayr <tbm@cyrius.com>
|
|
.IP \[bu] 2
|
|
Brandon McNama <bmcnama@pagerduty.com>
|
|
.IP \[bu] 2
|
|
Daniel Slyman <github@skylayer.eu>
|
|
.IP \[bu] 2
|
|
Alex Guerrero <guerrero@users.noreply.github.com>
|
|
.IP \[bu] 2
|
|
Matteo Pietro Dazzi <matteopietro.dazzi@gft.com>
|
|
.IP \[bu] 2
|
|
edwardxml <56691903+edwardxml@users.noreply.github.com>
|
|
.IP \[bu] 2
|
|
Roman Kredentser <shareed2k@gmail.com>
|
|
.IP \[bu] 2
|
|
Kamil Trzci\[u0144]ski <ayufan@ayufan.eu>
|
|
.IP \[bu] 2
|
|
Zac Rubin <z-0@users.noreply.github.com>
|
|
.IP \[bu] 2
|
|
Vincent Feltz <psycho@feltzv.fr>
|
|
.IP \[bu] 2
|
|
Heiko Bornholdt <bornholdt@informatik.uni-hamburg.de>
|
|
.IP \[bu] 2
|
|
Matteo Pietro Dazzi <matteopietro.dazzi@gmail.com>
|
|
.IP \[bu] 2
|
|
jtagcat <gitlab@c7.ee>
|
|
.IP \[bu] 2
|
|
Petri Salminen <petri@salminen.dev>
|
|
.IP \[bu] 2
|
|
Tim Burke <tim.burke@gmail.com>
|
|
.IP \[bu] 2
|
|
Kai L\[:u]ke <kai@kinvolk.io>
|
|
.IP \[bu] 2
|
|
Garrett Squire <github@garrettsquire.com>
|
|
.IP \[bu] 2
|
|
Evan Harris <eharris@puremagic.com>
|
|
.IP \[bu] 2
|
|
Kevin <keyam@microsoft.com>
|
|
.IP \[bu] 2
|
|
Morten Linderud <morten@linderud.pw>
|
|
.IP \[bu] 2
|
|
Dmitry Ustalov <dmitry.ustalov@gmail.com>
|
|
.IP \[bu] 2
|
|
Jack <196648+jdeng@users.noreply.github.com>
|
|
.IP \[bu] 2
|
|
kcris <cristian.tarsoaga@gmail.com>
|
|
.IP \[bu] 2
|
|
tyhuber1 <68970760+tyhuber1@users.noreply.github.com>
|
|
.IP \[bu] 2
|
|
David Ibarra <david.ibarra@realty.com>
|
|
.IP \[bu] 2
|
|
Tim Gallant <tim@lilt.com>
|
|
.IP \[bu] 2
|
|
Kaloyan Raev <kaloyan@storj.io>
|
|
.IP \[bu] 2
|
|
Jay McEntire <jay.mcentire@gmail.com>
|
|
.IP \[bu] 2
|
|
Leo Luan <leoluan@us.ibm.com>
|
|
.IP \[bu] 2
|
|
aus <549081+aus@users.noreply.github.com>
|
|
.IP \[bu] 2
|
|
Aaron Gokaslan <agokaslan@fb.com>
|
|
.IP \[bu] 2
|
|
Egor Margineanu <egmar@users.noreply.github.com>
|
|
.IP \[bu] 2
|
|
Lucas Kanashiro <lucas.kanashiro@canonical.com>
|
|
.IP \[bu] 2
|
|
WarpedPixel <WarpedPixel@users.noreply.github.com>
|
|
.IP \[bu] 2
|
|
Sam Edwards <sam@samedwards.ca>
|
|
.IP \[bu] 2
|
|
wjielai <gouki0123@gmail.com>
|
|
.IP \[bu] 2
|
|
Muffin King <jinxz_k@live.com>
|
|
.IP \[bu] 2
|
|
Christopher Stewart <6573710+1f47a@users.noreply.github.com>
|
|
.IP \[bu] 2
|
|
Russell Cattelan <cattelan@digitalelves.com>
|
|
.IP \[bu] 2
|
|
gyutw <30371241+gyutw@users.noreply.github.com>
|
|
.IP \[bu] 2
|
|
Hekmon <edouardhur@gmail.com>
|
|
.IP \[bu] 2
|
|
LaSombra <lasombra@users.noreply.github.com>
|
|
.IP \[bu] 2
|
|
Dov Murik <dov.murik@gmail.com>
|
|
.IP \[bu] 2
|
|
Ameer Dawood <ameer1234567890@gmail.com>
|
|
.IP \[bu] 2
|
|
Dan Hipschman <dan.hipschman@opendoor.com>
|
|
.IP \[bu] 2
|
|
Josh Soref <jsoref@users.noreply.github.com>
|
|
.IP \[bu] 2
|
|
David <david@staron.nl>
|
|
.IP \[bu] 2
|
|
Ingo <ingo@hoffmann.cx>
|
|
.IP \[bu] 2
|
|
Adam Pl\['a]nsk\['y] <adamplansky@users.noreply.github.com>
|
|
<adamplansky@gmail.com>
|
|
.IP \[bu] 2
|
|
Manish Gupta <manishgupta.ait@gmail.com>
|
|
.IP \[bu] 2
|
|
Deepak Sah <sah.sslpu@gmail.com>
|
|
.IP \[bu] 2
|
|
Marcin Zelent <marcin@zelent.net>
|
|
.IP \[bu] 2
|
|
zhucan <zhucan.k8s@gmail.com>
|
|
.IP \[bu] 2
|
|
James Lim <james.lim@samsara.com>
|
|
.IP \[bu] 2
|
|
Laurens Janssen <BD69BM@insim.biz>
|
|
.IP \[bu] 2
|
|
Bob Bagwill <bobbagwill@gmail.com>
|
|
.IP \[bu] 2
|
|
Nathan Collins <colli372@msu.edu>
|
|
.IP \[bu] 2
|
|
lostheli
|
|
.IP \[bu] 2
|
|
kelv <kelvin@acks.org>
|
|
.IP \[bu] 2
|
|
Milly <milly.ca@gmail.com>
|
|
.IP \[bu] 2
|
|
gtorelly <gtorelly@gmail.com>
|
|
.IP \[bu] 2
|
|
Brad Ackerman <brad@facefault.org>
|
|
.IP \[bu] 2
|
|
Mitsuo Heijo <mitsuo.heijo@gmail.com>
|
|
.IP \[bu] 2
|
|
Claudio Bantaloukas <rockdreamer@gmail.com>
|
|
.IP \[bu] 2
|
|
Benjamin Gustin <gustin.ben@gmail.com>
|
|
.IP \[bu] 2
|
|
Ingo Weiss <ingo@redhat.com>
|
|
.IP \[bu] 2
|
|
Kerry Su <me@sshockwave.net>
|
|
.IP \[bu] 2
|
|
Ilyess Bachiri <ilyess.bachiri@sonder.com>
|
|
.IP \[bu] 2
|
|
Yury Stankevich <urykhy@gmail.com>
|
|
.IP \[bu] 2
|
|
kice <wslikerqs@gmail.com>
|
|
.IP \[bu] 2
|
|
Denis Neuling <denisneuling@gmail.com>
|
|
.IP \[bu] 2
|
|
Janne Johansson <icepic.dz@gmail.com>
|
|
.IP \[bu] 2
|
|
Patrik Nordl\['e]n <patriki@gmail.com>
|
|
.IP \[bu] 2
|
|
CokeMine <aptx4561@gmail.com>
|
|
.IP \[bu] 2
|
|
S\[u01A1]n Tr\[u1EA7]n\-Nguy\[u1EC5]n <github@sntran.com>
|
|
.IP \[bu] 2
|
|
lluuaapp <266615+lluuaapp@users.noreply.github.com>
|
|
.IP \[bu] 2
|
|
Zach Kipp <kipp.zach@gmail.com>
|
|
.IP \[bu] 2
|
|
Riccardo Iaconelli <riccardo@kde.org>
|
|
.IP \[bu] 2
|
|
Sakuragawa Misty <gyc990326@gmail.com>
|
|
.IP \[bu] 2
|
|
Nicolas Rueff <nicolas@rueff.fr>
|
|
.IP \[bu] 2
|
|
Pau Rodriguez\-Estivill <prodrigestivill@gmail.com>
|
|
.IP \[bu] 2
|
|
Bob Pusateri <BobPusateri@users.noreply.github.com>
|
|
.IP \[bu] 2
|
|
Alex JOST <25005220+dimejo@users.noreply.github.com>
|
|
.IP \[bu] 2
|
|
Alexey Tabakman <samosad.ru@gmail.com>
|
|
.IP \[bu] 2
|
|
David Sze <sze.david@gmail.com>
|
|
.IP \[bu] 2
|
|
cynthia kwok <cynthia.m.kwok@gmail.com>
|
|
.IP \[bu] 2
|
|
Ankur Gupta <agupta@egnyte.com>
|
|
.IP \[bu] 2
|
|
Miron Veryanskiy <MironVeryanskiy@gmail.com>
|
|
.IP \[bu] 2
|
|
K265 <k.265@qq.com>
|
|
.IP \[bu] 2
|
|
Vesnyx <Vesnyx@users.noreply.github.com>
|
|
.IP \[bu] 2
|
|
Dmitry Chepurovskiy <me@dm3ch.net>
|
|
.IP \[bu] 2
|
|
Rauno Ots <rauno.ots@cgi.com>
|
|
.IP \[bu] 2
|
|
Georg Neugschwandtner <georg.neugschwandtner@gmx.net>
|
|
.IP \[bu] 2
|
|
pvalls <polvallsrue@gmail.com>
|
|
.IP \[bu] 2
|
|
Robert Thomas <31854736+wolveix@users.noreply.github.com>
|
|
.IP \[bu] 2
|
|
Romeo Kienzler <romeo.kienzler@gmail.com>
|
|
.IP \[bu] 2
|
|
tYYGH <tYYGH@users.noreply.github.com>
|
|
.IP \[bu] 2
|
|
georne <77802995+georne@users.noreply.github.com>
|
|
.IP \[bu] 2
|
|
Maxwell Calman <mcalman@MacBook-Pro.local>
|
|
.IP \[bu] 2
|
|
Naveen Honest Raj <naveendurai19@gmail.com>
|
|
.IP \[bu] 2
|
|
Lucas Messenger <lmesseng@cisco.com>
|
|
.IP \[bu] 2
|
|
Manish Kumar <krmanish260@gmail.com>
|
|
.IP \[bu] 2
|
|
x0b <x0bdev@gmail.com>
|
|
.IP \[bu] 2
|
|
CERN through the CS3MESH4EOSC Project
|
|
.SH Contact the rclone project
|
|
.SS Forum
|
|
.PP
|
|
Forum for questions and general discussion:
|
|
.IP \[bu] 2
|
|
https://forum.rclone.org
|
|
.SS GitHub repository
|
|
.PP
|
|
The project\[aq]s repository is located at:
|
|
.IP \[bu] 2
|
|
https://github.com/rclone/rclone
|
|
.PP
|
|
There you can file bug reports or contribute with pull requests.
|
|
.SS Twitter
|
|
.PP
|
|
You can also follow me on twitter for rclone announcements:
|
|
.IP \[bu] 2
|
|
[\[at]njcw](https://twitter.com/njcw)
|
|
.SS Email
|
|
.PP
|
|
Or if all else fails or you want to ask something private or
|
|
confidential email Nick Craig\-Wood (mailto:nick@craig-wood.com).
|
|
Please don\[aq]t email me requests for help \- those are better directed
|
|
to the forum.
|
|
Thanks!
|
|
.SH AUTHORS
|
|
Nick Craig\-Wood.
|