mirror of
https://github.com/rclone/rclone
synced 2024-12-23 14:23:44 +01:00
39748 lines
1.0 MiB
Generated
39748 lines
1.0 MiB
Generated
.\"t
|
|
.\" Automatically generated by Pandoc 2.5
|
|
.\"
|
|
.TH "rclone" "1" "May 27, 2020" "User Manual" ""
|
|
.hy
|
|
.SH Rclone syncs your files to cloud storage
|
|
.PP
|
|
.IP \[bu] 2
|
|
About rclone
|
|
.IP \[bu] 2
|
|
What can rclone do for you
|
|
.IP \[bu] 2
|
|
What features does rclone have
|
|
.IP \[bu] 2
|
|
What providers does rclone support
|
|
.IP \[bu] 2
|
|
Download (https://rclone.org/downloads/)
|
|
.IP \[bu] 2
|
|
Install (https://rclone.org/install/)
|
|
.IP \[bu] 2
|
|
Donate. (https://rclone.org/donate/)
|
|
.SS About rclone
|
|
.PP
|
|
Rclone is a command line program to manage files on cloud storage.
|
|
It is a feature rich alternative to cloud vendors\[cq] web storage
|
|
interfaces.
|
|
Over 40 cloud storage products support rclone including S3 object
|
|
stores, business & consumer file storage services, as well as standard
|
|
transfer protocols.
|
|
.PP
|
|
Rclone has powerful cloud equivalents to the unix commands rsync, cp,
|
|
mv, mount, ls, ncdu, tree, rm, and cat.
|
|
Rclone\[cq]s familiar syntax includes shell pipeline support, and
|
|
\f[C]\-\-dry\-run\f[R] protection.
|
|
It can be used at the command line, in scripts or via its API (/rc).
|
|
.PP
|
|
Users have called rclone \f[I]\[lq]The Swiss army knife of cloud
|
|
storage\[rq]\f[R] and \f[I]\[lq]Technology indistinguishable from
|
|
magic\[rq]\f[R].
|
|
.PP
|
|
Rclone really looks after your data.
|
|
It preserves timestamps and verifies your data at all times.
|
|
Transfers over limited bandwidth; intermittent connections, or subject
|
|
to quota can be restarted, from the last good file transferred.
|
|
You can check (https://rclone.org/commands/rclone_check/) the integrity
|
|
of your files.
|
|
Where possible, rclone employs server side transfers to minimise local
|
|
bandwidth use and transfers from one provider to another without using
|
|
your local disk.
|
|
.PP
|
|
Virtual backends wrap local and cloud file systems to apply
|
|
encryption (https://rclone.org/crypt/),
|
|
caching (https://rclone.org/cache/),
|
|
chunking (https://rclone.org/chunker/) and
|
|
joining (https://rclone.org/union/).
|
|
.PP
|
|
Rclone can mount (https://rclone.org/commands/rclone_mount/) any local,
|
|
cloud or virtual filesystem as a disk on Windows, macOS, linux and
|
|
FreeBSD, and also serve these over
|
|
SFTP (https://rclone.org/commands/rclone_serve_sftp/),
|
|
HTTP (https://rclone.org/commands/rclone_serve_http/),
|
|
WebDAV (https://rclone.org/commands/rclone_serve_webdav/),
|
|
FTP (https://rclone.org/commands/rclone_serve_ftp/) and
|
|
DLNA (https://rclone.org/commands/rclone_serve_dlna/).
|
|
.PP
|
|
Rclone is mature, open source software originally inspired by rsync and
|
|
written in Go (https://golang.org).
|
|
The friendly support community are familiar with varied use cases.
|
|
Official Ubuntu, Debian, Fedora, Brew and Chocolatey repos.
|
|
include rclone.
|
|
For the latest version downloading from
|
|
rclone.org (https://rclone.org/downloads/) is recommended.
|
|
.PP
|
|
Rclone is widely used on Linux, Windows and Mac.
|
|
Third party developers have built innovative backup, restore, GUI and
|
|
business process solutions using the rclone command line or API.
|
|
.PP
|
|
Let rclone do the heavy lifting of communicating with cloud storage.
|
|
.SS What can rclone do for you
|
|
.PP
|
|
Rclone can help you:
|
|
.IP \[bu] 2
|
|
Backup (and encrypt) files to cloud storage
|
|
.IP \[bu] 2
|
|
Restore (and decrypt) files from cloud storage
|
|
.IP \[bu] 2
|
|
Mirror cloud data to other cloud services or locally
|
|
.IP \[bu] 2
|
|
Migrate data to cloud, or between cloud storage vendors
|
|
.IP \[bu] 2
|
|
Mount multiple, encrypted, cached or diverse cloud storage as a disk
|
|
.IP \[bu] 2
|
|
Analyse and account for data held on cloud storage using
|
|
lsf (https://rclone.org/commands/rclone_lsf/),
|
|
ljson (https://rclone.org/commands/rclone_lsjson/),
|
|
size (https://rclone.org/commands/rclone_size/),
|
|
ncdu (https://rclone.org/commands/rclone_ncdu/)
|
|
.IP \[bu] 2
|
|
Union (https://rclone.org/union/) file systems together to present
|
|
multiple local and/or cloud file systems as one
|
|
.SS Features
|
|
.IP \[bu] 2
|
|
Transfers
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
MD5, SHA1 hashes are checked at all times for file integrity
|
|
.IP \[bu] 2
|
|
Timestamps are preserved on files
|
|
.IP \[bu] 2
|
|
Operations can be restarted at any time
|
|
.IP \[bu] 2
|
|
Can be to and from network, eg two different cloud providers
|
|
.IP \[bu] 2
|
|
Can use multi\-threaded downloads to local disk
|
|
.RE
|
|
.IP \[bu] 2
|
|
Copy (https://rclone.org/commands/rclone_copy/) new or changed files to
|
|
cloud storage
|
|
.IP \[bu] 2
|
|
Sync (https://rclone.org/commands/rclone_sync/) (one way) to make a
|
|
directory identical
|
|
.IP \[bu] 2
|
|
Move (https://rclone.org/commands/rclone_move/) files to cloud storage
|
|
deleting the local after verification
|
|
.IP \[bu] 2
|
|
Check (https://rclone.org/commands/rclone_check/) hashes and for
|
|
missing/extra files
|
|
.IP \[bu] 2
|
|
Mount (https://rclone.org/commands/rclone_mount/) your cloud storage as
|
|
a network disk
|
|
.IP \[bu] 2
|
|
Serve (https://rclone.org/commands/rclone_serve/) local or remote files
|
|
over
|
|
HTTP (https://rclone.org/commands/rclone_serve_http/)/WebDav (https://rclone.org/commands/rclone_serve_webdav/)/FTP (https://rclone.org/commands/rclone_serve_ftp/)/SFTP (https://rclone.org/commands/rclone_serve_sftp/)/dlna (https://rclone.org/commands/rclone_serve_dlna/)
|
|
.IP \[bu] 2
|
|
Experimental Web based GUI (https://rclone.org/gui/)
|
|
.SS Supported providers
|
|
.PP
|
|
(There are many other providers, built on standard protocols such as
|
|
WebDAV or S3, that work out of the box.)
|
|
.IP \[bu] 2
|
|
1Fichier
|
|
.IP \[bu] 2
|
|
Alibaba Cloud (Aliyun) Object Storage System (OSS)
|
|
.IP \[bu] 2
|
|
Amazon Drive
|
|
.IP \[bu] 2
|
|
Amazon S3
|
|
.IP \[bu] 2
|
|
Backblaze B2
|
|
.IP \[bu] 2
|
|
Box
|
|
.IP \[bu] 2
|
|
Ceph
|
|
.IP \[bu] 2
|
|
Citrix ShareFile
|
|
.IP \[bu] 2
|
|
C14
|
|
.IP \[bu] 2
|
|
DigitalOcean Spaces
|
|
.IP \[bu] 2
|
|
Dreamhost
|
|
.IP \[bu] 2
|
|
Dropbox
|
|
.IP \[bu] 2
|
|
FTP
|
|
.IP \[bu] 2
|
|
Google Cloud Storage
|
|
.IP \[bu] 2
|
|
Google Drive
|
|
.IP \[bu] 2
|
|
Google Photos
|
|
.IP \[bu] 2
|
|
HTTP
|
|
.IP \[bu] 2
|
|
Hubic
|
|
.IP \[bu] 2
|
|
Jottacloud
|
|
.IP \[bu] 2
|
|
IBM COS S3
|
|
.IP \[bu] 2
|
|
Koofr
|
|
.IP \[bu] 2
|
|
Mail.ru Cloud
|
|
.IP \[bu] 2
|
|
Memset Memstore
|
|
.IP \[bu] 2
|
|
Mega
|
|
.IP \[bu] 2
|
|
Memory
|
|
.IP \[bu] 2
|
|
Microsoft Azure Blob Storage
|
|
.IP \[bu] 2
|
|
Microsoft OneDrive
|
|
.IP \[bu] 2
|
|
Minio
|
|
.IP \[bu] 2
|
|
Nextcloud
|
|
.IP \[bu] 2
|
|
OVH
|
|
.IP \[bu] 2
|
|
OpenDrive
|
|
.IP \[bu] 2
|
|
OpenStack Swift
|
|
.IP \[bu] 2
|
|
Oracle Cloud Storage
|
|
.IP \[bu] 2
|
|
ownCloud
|
|
.IP \[bu] 2
|
|
pCloud
|
|
.IP \[bu] 2
|
|
premiumize.me
|
|
.IP \[bu] 2
|
|
put.io
|
|
.IP \[bu] 2
|
|
QingStor
|
|
.IP \[bu] 2
|
|
Rackspace Cloud Files
|
|
.IP \[bu] 2
|
|
rsync.net
|
|
.IP \[bu] 2
|
|
Scaleway
|
|
.IP \[bu] 2
|
|
Seafile
|
|
.IP \[bu] 2
|
|
SFTP
|
|
.IP \[bu] 2
|
|
StackPath
|
|
.IP \[bu] 2
|
|
SugarSync
|
|
.IP \[bu] 2
|
|
Tardigrade
|
|
.IP \[bu] 2
|
|
Wasabi
|
|
.IP \[bu] 2
|
|
WebDAV
|
|
.IP \[bu] 2
|
|
Yandex Disk
|
|
.IP \[bu] 2
|
|
The local filesystem
|
|
.PP
|
|
Links
|
|
.IP \[bu] 2
|
|
Home page (https://rclone.org/)
|
|
.IP \[bu] 2
|
|
GitHub project page for source and bug
|
|
tracker (https://github.com/rclone/rclone)
|
|
.IP \[bu] 2
|
|
Rclone Forum (https://forum.rclone.org)
|
|
.IP \[bu] 2
|
|
Downloads (https://rclone.org/downloads/)
|
|
.SH Install
|
|
.PP
|
|
Rclone is a Go program and comes as a single binary file.
|
|
.SS Quickstart
|
|
.IP \[bu] 2
|
|
Download (https://rclone.org/downloads/) the relevant binary.
|
|
.IP \[bu] 2
|
|
Extract the \f[C]rclone\f[R] or \f[C]rclone.exe\f[R] binary from the
|
|
archive
|
|
.IP \[bu] 2
|
|
Run \f[C]rclone config\f[R] to setup.
|
|
See rclone config docs (https://rclone.org/docs/) for more details.
|
|
.PP
|
|
See below for some expanded Linux / macOS instructions.
|
|
.PP
|
|
See the Usage section (https://rclone.org/docs/#usage) of the docs for
|
|
how to use rclone, or run \f[C]rclone \-h\f[R].
|
|
.SS Script installation
|
|
.PP
|
|
To install rclone on Linux/macOS/BSD systems, run:
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
curl https://rclone.org/install.sh | sudo bash
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
For beta installation, run:
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
curl https://rclone.org/install.sh | sudo bash \-s beta
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
Note that this script checks the version of rclone installed first and
|
|
won\[cq]t re\-download if not needed.
|
|
.SS Linux installation from precompiled binary
|
|
.PP
|
|
Fetch and unpack
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
curl \-O https://downloads.rclone.org/rclone\-current\-linux\-amd64.zip
|
|
unzip rclone\-current\-linux\-amd64.zip
|
|
cd rclone\-*\-linux\-amd64
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
Copy binary file
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
sudo cp rclone /usr/bin/
|
|
sudo chown root:root /usr/bin/rclone
|
|
sudo chmod 755 /usr/bin/rclone
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
Install manpage
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
sudo mkdir \-p /usr/local/share/man/man1
|
|
sudo cp rclone.1 /usr/local/share/man/man1/
|
|
sudo mandb
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
Run \f[C]rclone config\f[R] to setup.
|
|
See rclone config docs (https://rclone.org/docs/) for more details.
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone config
|
|
\f[R]
|
|
.fi
|
|
.SS macOS installation with brew
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
brew install rclone
|
|
\f[R]
|
|
.fi
|
|
.SS macOS installation from precompiled binary, using curl
|
|
.PP
|
|
To avoid problems with macOS gatekeeper enforcing the binary to be
|
|
signed and notarized it is enough to download with \f[C]curl\f[R].
|
|
.PP
|
|
Download the latest version of rclone.
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
cd && curl \-O https://downloads.rclone.org/rclone\-current\-osx\-amd64.zip
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
Unzip the download and cd to the extracted folder.
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
unzip \-a rclone\-current\-osx\-amd64.zip && cd rclone\-*\-osx\-amd64
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
Move rclone to your $PATH.
|
|
You will be prompted for your password.
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
sudo mkdir \-p /usr/local/bin
|
|
sudo mv rclone /usr/local/bin/
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
(the \f[C]mkdir\f[R] command is safe to run, even if the directory
|
|
already exists).
|
|
.PP
|
|
Remove the leftover files.
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
cd .. && rm \-rf rclone\-*\-osx\-amd64 rclone\-current\-osx\-amd64.zip
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
Run \f[C]rclone config\f[R] to setup.
|
|
See rclone config docs (https://rclone.org/docs/) for more details.
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone config
|
|
\f[R]
|
|
.fi
|
|
.SS macOS installation from precompiled binary, using a web browser
|
|
.PP
|
|
When downloading a binary with a web browser, the browser will set the
|
|
macOS gatekeeper quarantine attribute.
|
|
Starting from Catalina, when attempting to run \f[C]rclone\f[R], a
|
|
pop\-up will appear saying:
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
\[lq]rclone\[rq] cannot be opened because the developer cannot be verified.
|
|
macOS cannot verify that this app is free from malware.
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
The simplest fix is to run
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
xattr \-d com.apple.quarantine rclone
|
|
\f[R]
|
|
.fi
|
|
.SS Install with docker
|
|
.PP
|
|
The rclone maintains a docker image for
|
|
rclone (https://hub.docker.com/r/rclone/rclone).
|
|
These images are autobuilt by docker hub from the rclone source based on
|
|
a minimal Alpine linux image.
|
|
.PP
|
|
The \f[C]:latest\f[R] tag will always point to the latest stable
|
|
release.
|
|
You can use the \f[C]:beta\f[R] tag to get the latest build from master.
|
|
You can also use version tags, eg \f[C]:1.49.1\f[R], \f[C]:1.49\f[R] or
|
|
\f[C]:1\f[R].
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
$ docker pull rclone/rclone:latest
|
|
latest: Pulling from rclone/rclone
|
|
Digest: sha256:0e0ced72671989bb837fea8e88578b3fc48371aa45d209663683e24cfdaa0e11
|
|
\&...
|
|
$ docker run \-\-rm rclone/rclone:latest version
|
|
rclone v1.49.1
|
|
\- os/arch: linux/amd64
|
|
\- go version: go1.12.9
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
There are a few command line options to consider when starting an rclone
|
|
Docker container from the rclone image.
|
|
.IP \[bu] 2
|
|
You need to mount the host rclone config dir at \f[C]/config/rclone\f[R]
|
|
into the Docker container.
|
|
Due to the fact that rclone updates tokens inside its config file, and
|
|
that the update process involves a file rename, you need to mount the
|
|
whole host rclone config dir, not just the single host rclone config
|
|
file.
|
|
.IP \[bu] 2
|
|
You need to mount a host data dir at \f[C]/data\f[R] into the Docker
|
|
container.
|
|
.IP \[bu] 2
|
|
By default, the rclone binary inside a Docker container runs with UID=0
|
|
(root).
|
|
As a result, all files created in a run will have UID=0.
|
|
If your config and data files reside on the host with a non\-root
|
|
UID:GID, you need to pass these on the container start command line.
|
|
.IP \[bu] 2
|
|
It is possible to use \f[C]rclone mount\f[R] inside a userspace Docker
|
|
container, and expose the resulting fuse mount to the host.
|
|
The exact \f[C]docker run\f[R] options to do that might vary slightly
|
|
between hosts.
|
|
See, e.g.\ the discussion in this
|
|
thread (https://github.com/moby/moby/issues/9448).
|
|
.RS 2
|
|
.PP
|
|
You also need to mount the host \f[C]/etc/passwd\f[R] and
|
|
\f[C]/etc/group\f[R] for fuse to work inside the container.
|
|
.RE
|
|
.PP
|
|
Here are some commands tested on an Ubuntu 18.04.3 host:
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
# config on host at \[ti]/.config/rclone/rclone.conf
|
|
# data on host at \[ti]/data
|
|
|
|
# make sure the config is ok by listing the remotes
|
|
docker run \-\-rm \[rs]
|
|
\-\-volume \[ti]/.config/rclone:/config/rclone \[rs]
|
|
\-\-volume \[ti]/data:/data:shared \[rs]
|
|
\-\-user $(id \-u):$(id \-g) \[rs]
|
|
rclone/rclone \[rs]
|
|
listremotes
|
|
|
|
# perform mount inside Docker container, expose result to host
|
|
mkdir \-p \[ti]/data/mount
|
|
docker run \-\-rm \[rs]
|
|
\-\-volume \[ti]/.config/rclone:/config/rclone \[rs]
|
|
\-\-volume \[ti]/data:/data:shared \[rs]
|
|
\-\-user $(id \-u):$(id \-g) \[rs]
|
|
\-\-volume /etc/passwd:/etc/passwd:ro \-\-volume /etc/group:/etc/group:ro \[rs]
|
|
\-\-device /dev/fuse \-\-cap\-add SYS_ADMIN \-\-security\-opt apparmor:unconfined \[rs]
|
|
rclone/rclone \[rs]
|
|
mount dropbox:Photos /data/mount &
|
|
ls \[ti]/data/mount
|
|
kill %1
|
|
\f[R]
|
|
.fi
|
|
.SS Install from source
|
|
.PP
|
|
Make sure you have at least Go (https://golang.org/) 1.7 installed.
|
|
Download go (https://golang.org/dl/) if necessary.
|
|
The latest release is recommended.
|
|
Then
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
git clone https://github.com/rclone/rclone.git
|
|
cd rclone
|
|
go build
|
|
\&./rclone version
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
You can also build and install rclone in the
|
|
GOPATH (https://github.com/golang/go/wiki/GOPATH) (which defaults to
|
|
\f[C]\[ti]/go\f[R]) with:
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
go get \-u \-v github.com/rclone/rclone
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
and this will build the binary in \f[C]$GOPATH/bin\f[R]
|
|
(\f[C]\[ti]/go/bin/rclone\f[R] by default) after downloading the source
|
|
to \f[C]$GOPATH/src/github.com/rclone/rclone\f[R]
|
|
(\f[C]\[ti]/go/src/github.com/rclone/rclone\f[R] by default).
|
|
.SS Installation with Ansible
|
|
.PP
|
|
This can be done with Stefan Weichinger\[cq]s ansible
|
|
role (https://github.com/stefangweichinger/ansible-rclone).
|
|
.PP
|
|
Instructions
|
|
.IP "1." 3
|
|
\f[C]git clone https://github.com/stefangweichinger/ansible\-rclone.git\f[R]
|
|
into your local roles\-directory
|
|
.IP "2." 3
|
|
add the role to the hosts you want rclone installed to:
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
\- hosts: rclone\-hosts
|
|
roles:
|
|
\- rclone
|
|
\f[R]
|
|
.fi
|
|
.SS Configure
|
|
.PP
|
|
First, you\[cq]ll need to configure rclone.
|
|
As the object storage systems have quite complicated authentication
|
|
these are kept in a config file.
|
|
(See the \f[C]\-\-config\f[R] entry for how to find the config file and
|
|
choose its location.)
|
|
.PP
|
|
The easiest way to make the config is to run rclone with the config
|
|
option:
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone config
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
See the following for detailed instructions for
|
|
.IP \[bu] 2
|
|
1Fichier (https://rclone.org/fichier/)
|
|
.IP \[bu] 2
|
|
Alias (https://rclone.org/alias/)
|
|
.IP \[bu] 2
|
|
Amazon Drive (https://rclone.org/amazonclouddrive/)
|
|
.IP \[bu] 2
|
|
Amazon S3 (https://rclone.org/s3/)
|
|
.IP \[bu] 2
|
|
Backblaze B2 (https://rclone.org/b2/)
|
|
.IP \[bu] 2
|
|
Box (https://rclone.org/box/)
|
|
.IP \[bu] 2
|
|
Cache (https://rclone.org/cache/)
|
|
.IP \[bu] 2
|
|
Chunker (https://rclone.org/chunker/) \- transparently splits large
|
|
files for other remotes
|
|
.IP \[bu] 2
|
|
Citrix ShareFile (https://rclone.org/sharefile/)
|
|
.IP \[bu] 2
|
|
Crypt (https://rclone.org/crypt/) \- to encrypt other remotes
|
|
.IP \[bu] 2
|
|
DigitalOcean Spaces (https://rclone.org/s3/#digitalocean-spaces)
|
|
.IP \[bu] 2
|
|
Dropbox (https://rclone.org/dropbox/)
|
|
.IP \[bu] 2
|
|
FTP (https://rclone.org/ftp/)
|
|
.IP \[bu] 2
|
|
Google Cloud Storage (https://rclone.org/googlecloudstorage/)
|
|
.IP \[bu] 2
|
|
Google Drive (https://rclone.org/drive/)
|
|
.IP \[bu] 2
|
|
Google Photos (https://rclone.org/googlephotos/)
|
|
.IP \[bu] 2
|
|
HTTP (https://rclone.org/http/)
|
|
.IP \[bu] 2
|
|
Hubic (https://rclone.org/hubic/)
|
|
.IP \[bu] 2
|
|
Jottacloud / GetSky.no (https://rclone.org/jottacloud/)
|
|
.IP \[bu] 2
|
|
Koofr (https://rclone.org/koofr/)
|
|
.IP \[bu] 2
|
|
Mail.ru Cloud (https://rclone.org/mailru/)
|
|
.IP \[bu] 2
|
|
Mega (https://rclone.org/mega/)
|
|
.IP \[bu] 2
|
|
Memory (https://rclone.org/memory/)
|
|
.IP \[bu] 2
|
|
Microsoft Azure Blob Storage (https://rclone.org/azureblob/)
|
|
.IP \[bu] 2
|
|
Microsoft OneDrive (https://rclone.org/onedrive/)
|
|
.IP \[bu] 2
|
|
OpenStack Swift / Rackspace Cloudfiles / Memset
|
|
Memstore (https://rclone.org/swift/)
|
|
.IP \[bu] 2
|
|
OpenDrive (https://rclone.org/opendrive/)
|
|
.IP \[bu] 2
|
|
Pcloud (https://rclone.org/pcloud/)
|
|
.IP \[bu] 2
|
|
premiumize.me (https://rclone.org/premiumizeme/)
|
|
.IP \[bu] 2
|
|
put.io (https://rclone.org/putio/)
|
|
.IP \[bu] 2
|
|
QingStor (https://rclone.org/qingstor/)
|
|
.IP \[bu] 2
|
|
Seafile (https://rclone.org/seafile/)
|
|
.IP \[bu] 2
|
|
SFTP (https://rclone.org/sftp/)
|
|
.IP \[bu] 2
|
|
SugarSync (https://rclone.org/sugarsync/)
|
|
.IP \[bu] 2
|
|
Tardigrade (https://rclone.org/tardigrade/)
|
|
.IP \[bu] 2
|
|
Union (https://rclone.org/union/)
|
|
.IP \[bu] 2
|
|
WebDAV (https://rclone.org/webdav/)
|
|
.IP \[bu] 2
|
|
Yandex Disk (https://rclone.org/yandex/)
|
|
.IP \[bu] 2
|
|
The local filesystem (https://rclone.org/local/)
|
|
.SS Usage
|
|
.PP
|
|
Rclone syncs a directory tree from one storage system to another.
|
|
.PP
|
|
Its syntax is like this
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
Syntax: [options] subcommand <parameters> <parameters...>
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
Source and destination paths are specified by the name you gave the
|
|
storage system in the config file then the sub path, eg
|
|
\[lq]drive:myfolder\[rq] to look at \[lq]myfolder\[rq] in Google drive.
|
|
.PP
|
|
You can define as many storage paths as you like in the config file.
|
|
.SS Subcommands
|
|
.PP
|
|
rclone uses a system of subcommands.
|
|
For example
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone ls remote:path # lists a remote
|
|
rclone copy /local/path remote:path # copies /local/path to the remote
|
|
rclone sync /local/path remote:path # syncs /local/path to the remote
|
|
\f[R]
|
|
.fi
|
|
.SH rclone config
|
|
.PP
|
|
Enter an interactive configuration session.
|
|
.SS Synopsis
|
|
.PP
|
|
Enter an interactive configuration session where you can setup new
|
|
remotes and manage existing ones.
|
|
You may also set or remove a password to protect your configuration.
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone config [flags]
|
|
\f[R]
|
|
.fi
|
|
.SS Options
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
\-h, \-\-help help for config
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
See the global flags page (https://rclone.org/flags/) for global options
|
|
not listed here.
|
|
.SS SEE ALSO
|
|
.IP \[bu] 2
|
|
rclone (https://rclone.org/commands/rclone/) \- Show help for rclone
|
|
commands, flags and backends.
|
|
.IP \[bu] 2
|
|
rclone config create (https://rclone.org/commands/rclone_config_create/)
|
|
\- Create a new remote with name, type and options.
|
|
.IP \[bu] 2
|
|
rclone config delete (https://rclone.org/commands/rclone_config_delete/)
|
|
\- Delete an existing remote \f[C]name\f[R].
|
|
.IP \[bu] 2
|
|
rclone config
|
|
disconnect (https://rclone.org/commands/rclone_config_disconnect/) \-
|
|
Disconnects user from remote
|
|
.IP \[bu] 2
|
|
rclone config dump (https://rclone.org/commands/rclone_config_dump/) \-
|
|
Dump the config file as JSON.
|
|
.IP \[bu] 2
|
|
rclone config edit (https://rclone.org/commands/rclone_config_edit/) \-
|
|
Enter an interactive configuration session.
|
|
.IP \[bu] 2
|
|
rclone config file (https://rclone.org/commands/rclone_config_file/) \-
|
|
Show path of configuration file in use.
|
|
.IP \[bu] 2
|
|
rclone config
|
|
password (https://rclone.org/commands/rclone_config_password/) \- Update
|
|
password in an existing remote.
|
|
.IP \[bu] 2
|
|
rclone config
|
|
providers (https://rclone.org/commands/rclone_config_providers/) \- List
|
|
in JSON format all the providers and options.
|
|
.IP \[bu] 2
|
|
rclone config
|
|
reconnect (https://rclone.org/commands/rclone_config_reconnect/) \-
|
|
Re\-authenticates user with remote.
|
|
.IP \[bu] 2
|
|
rclone config show (https://rclone.org/commands/rclone_config_show/) \-
|
|
Print (decrypted) config file, or the config for a single remote.
|
|
.IP \[bu] 2
|
|
rclone config update (https://rclone.org/commands/rclone_config_update/)
|
|
\- Update options in an existing remote.
|
|
.IP \[bu] 2
|
|
rclone config
|
|
userinfo (https://rclone.org/commands/rclone_config_userinfo/) \- Prints
|
|
info about logged in user of remote.
|
|
.SH rclone copy
|
|
.PP
|
|
Copy files from source to dest, skipping already copied
|
|
.SS Synopsis
|
|
.PP
|
|
Copy the source to the destination.
|
|
Doesn\[cq]t transfer unchanged files, testing by size and modification
|
|
time or MD5SUM.
|
|
Doesn\[cq]t delete files from the destination.
|
|
.PP
|
|
Note that it is always the contents of the directory that is synced, not
|
|
the directory so when source:path is a directory, it\[cq]s the contents
|
|
of source:path that are copied, not the directory name and contents.
|
|
.PP
|
|
If dest:path doesn\[cq]t exist, it is created and the source:path
|
|
contents go there.
|
|
.PP
|
|
For example
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone copy source:sourcepath dest:destpath
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
Let\[cq]s say there are two files in sourcepath
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
sourcepath/one.txt
|
|
sourcepath/two.txt
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
This copies them to
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
destpath/one.txt
|
|
destpath/two.txt
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
Not to
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
destpath/sourcepath/one.txt
|
|
destpath/sourcepath/two.txt
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
If you are familiar with \f[C]rsync\f[R], rclone always works as if you
|
|
had written a trailing / \- meaning \[lq]copy the contents of this
|
|
directory\[rq].
|
|
This applies to all commands and whether you are talking about the
|
|
source or destination.
|
|
.PP
|
|
See the \[en]no\-traverse (https://rclone.org/docs/#no-traverse) option
|
|
for controlling whether rclone lists the destination directory or not.
|
|
Supplying this option when copying a small number of files into a large
|
|
destination can speed transfers up greatly.
|
|
.PP
|
|
For example, if you have many files in /path/to/src but only a few of
|
|
them change every day, you can copy all the files which have changed
|
|
recently very efficiently like this:
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone copy \-\-max\-age 24h \-\-no\-traverse /path/to/src remote:
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
\f[B]Note\f[R]: Use the \f[C]\-P\f[R]/\f[C]\-\-progress\f[R] flag to
|
|
view real\-time transfer statistics
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone copy source:path dest:path [flags]
|
|
\f[R]
|
|
.fi
|
|
.SS Options
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
\-\-create\-empty\-src\-dirs Create empty source dirs on destination after copy
|
|
\-h, \-\-help help for copy
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
See the global flags page (https://rclone.org/flags/) for global options
|
|
not listed here.
|
|
.SS SEE ALSO
|
|
.IP \[bu] 2
|
|
rclone (https://rclone.org/commands/rclone/) \- Show help for rclone
|
|
commands, flags and backends.
|
|
.SH rclone sync
|
|
.PP
|
|
Make source and dest identical, modifying destination only.
|
|
.SS Synopsis
|
|
.PP
|
|
Sync the source to the destination, changing the destination only.
|
|
Doesn\[cq]t transfer unchanged files, testing by size and modification
|
|
time or MD5SUM.
|
|
Destination is updated to match source, including deleting files if
|
|
necessary.
|
|
.PP
|
|
\f[B]Important\f[R]: Since this can cause data loss, test first with the
|
|
\f[C]\-\-dry\-run\f[R] flag to see exactly what would be copied and
|
|
deleted.
|
|
.PP
|
|
Note that files in the destination won\[cq]t be deleted if there were
|
|
any errors at any point.
|
|
.PP
|
|
It is always the contents of the directory that is synced, not the
|
|
directory so when source:path is a directory, it\[cq]s the contents of
|
|
source:path that are copied, not the directory name and contents.
|
|
See extended explanation in the \f[C]copy\f[R] command above if unsure.
|
|
.PP
|
|
If dest:path doesn\[cq]t exist, it is created and the source:path
|
|
contents go there.
|
|
.PP
|
|
\f[B]Note\f[R]: Use the \f[C]\-P\f[R]/\f[C]\-\-progress\f[R] flag to
|
|
view real\-time transfer statistics
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone sync source:path dest:path [flags]
|
|
\f[R]
|
|
.fi
|
|
.SS Options
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
\-\-create\-empty\-src\-dirs Create empty source dirs on destination after sync
|
|
\-h, \-\-help help for sync
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
See the global flags page (https://rclone.org/flags/) for global options
|
|
not listed here.
|
|
.SS SEE ALSO
|
|
.IP \[bu] 2
|
|
rclone (https://rclone.org/commands/rclone/) \- Show help for rclone
|
|
commands, flags and backends.
|
|
.SH rclone move
|
|
.PP
|
|
Move files from source to dest.
|
|
.SS Synopsis
|
|
.PP
|
|
Moves the contents of the source directory to the destination directory.
|
|
Rclone will error if the source and destination overlap and the remote
|
|
does not support a server side directory move operation.
|
|
.PP
|
|
If no filters are in use and if possible this will server side move
|
|
\f[C]source:path\f[R] into \f[C]dest:path\f[R].
|
|
After this \f[C]source:path\f[R] will no longer exist.
|
|
.PP
|
|
Otherwise for each file in \f[C]source:path\f[R] selected by the filters
|
|
(if any) this will move it into \f[C]dest:path\f[R].
|
|
If possible a server side move will be used, otherwise it will copy it
|
|
(server side if possible) into \f[C]dest:path\f[R] then delete the
|
|
original (if no errors on copy) in \f[C]source:path\f[R].
|
|
.PP
|
|
If you want to delete empty source directories after move, use the
|
|
\[en]delete\-empty\-src\-dirs flag.
|
|
.PP
|
|
See the \[en]no\-traverse (https://rclone.org/docs/#no-traverse) option
|
|
for controlling whether rclone lists the destination directory or not.
|
|
Supplying this option when moving a small number of files into a large
|
|
destination can speed transfers up greatly.
|
|
.PP
|
|
\f[B]Important\f[R]: Since this can cause data loss, test first with the
|
|
\[en]dry\-run flag.
|
|
.PP
|
|
\f[B]Note\f[R]: Use the \f[C]\-P\f[R]/\f[C]\-\-progress\f[R] flag to
|
|
view real\-time transfer statistics.
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone move source:path dest:path [flags]
|
|
\f[R]
|
|
.fi
|
|
.SS Options
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
\-\-create\-empty\-src\-dirs Create empty source dirs on destination after move
|
|
\-\-delete\-empty\-src\-dirs Delete empty source dirs after move
|
|
\-h, \-\-help help for move
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
See the global flags page (https://rclone.org/flags/) for global options
|
|
not listed here.
|
|
.SS SEE ALSO
|
|
.IP \[bu] 2
|
|
rclone (https://rclone.org/commands/rclone/) \- Show help for rclone
|
|
commands, flags and backends.
|
|
.SH rclone delete
|
|
.PP
|
|
Remove the contents of path.
|
|
.SS Synopsis
|
|
.PP
|
|
Remove the files in path.
|
|
Unlike \f[C]purge\f[R] it obeys include/exclude filters so can be used
|
|
to selectively delete files.
|
|
.PP
|
|
\f[C]rclone delete\f[R] only deletes objects but leaves the directory
|
|
structure alone.
|
|
If you want to delete a directory and all of its contents use
|
|
\f[C]rclone purge\f[R]
|
|
.PP
|
|
If you supply the \[en]rmdirs flag, it will remove all empty directories
|
|
along with it.
|
|
.PP
|
|
Eg delete all files bigger than 100MBytes
|
|
.PP
|
|
Check what would be deleted first (use either)
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone \-\-min\-size 100M lsl remote:path
|
|
rclone \-\-dry\-run \-\-min\-size 100M delete remote:path
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
Then delete
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone \-\-min\-size 100M delete remote:path
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
That reads \[lq]delete everything with a minimum size of 100 MB\[rq],
|
|
hence delete all files bigger than 100MBytes.
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone delete remote:path [flags]
|
|
\f[R]
|
|
.fi
|
|
.SS Options
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
\-h, \-\-help help for delete
|
|
\-\-rmdirs rmdirs removes empty directories but leaves root intact
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
See the global flags page (https://rclone.org/flags/) for global options
|
|
not listed here.
|
|
.SS SEE ALSO
|
|
.IP \[bu] 2
|
|
rclone (https://rclone.org/commands/rclone/) \- Show help for rclone
|
|
commands, flags and backends.
|
|
.SH rclone purge
|
|
.PP
|
|
Remove the path and all of its contents.
|
|
.SS Synopsis
|
|
.PP
|
|
Remove the path and all of its contents.
|
|
Note that this does not obey include/exclude filters \- everything will
|
|
be removed.
|
|
Use \f[C]delete\f[R] if you want to selectively delete files.
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone purge remote:path [flags]
|
|
\f[R]
|
|
.fi
|
|
.SS Options
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
\-h, \-\-help help for purge
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
See the global flags page (https://rclone.org/flags/) for global options
|
|
not listed here.
|
|
.SS SEE ALSO
|
|
.IP \[bu] 2
|
|
rclone (https://rclone.org/commands/rclone/) \- Show help for rclone
|
|
commands, flags and backends.
|
|
.SH rclone mkdir
|
|
.PP
|
|
Make the path if it doesn\[cq]t already exist.
|
|
.SS Synopsis
|
|
.PP
|
|
Make the path if it doesn\[cq]t already exist.
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone mkdir remote:path [flags]
|
|
\f[R]
|
|
.fi
|
|
.SS Options
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
\-h, \-\-help help for mkdir
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
See the global flags page (https://rclone.org/flags/) for global options
|
|
not listed here.
|
|
.SS SEE ALSO
|
|
.IP \[bu] 2
|
|
rclone (https://rclone.org/commands/rclone/) \- Show help for rclone
|
|
commands, flags and backends.
|
|
.SH rclone rmdir
|
|
.PP
|
|
Remove the path if empty.
|
|
.SS Synopsis
|
|
.PP
|
|
Remove the path.
|
|
Note that you can\[cq]t remove a path with objects in it, use purge for
|
|
that.
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone rmdir remote:path [flags]
|
|
\f[R]
|
|
.fi
|
|
.SS Options
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
\-h, \-\-help help for rmdir
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
See the global flags page (https://rclone.org/flags/) for global options
|
|
not listed here.
|
|
.SS SEE ALSO
|
|
.IP \[bu] 2
|
|
rclone (https://rclone.org/commands/rclone/) \- Show help for rclone
|
|
commands, flags and backends.
|
|
.SH rclone check
|
|
.PP
|
|
Checks the files in the source and destination match.
|
|
.SS Synopsis
|
|
.PP
|
|
Checks the files in the source and destination match.
|
|
It compares sizes and hashes (MD5 or SHA1) and logs a report of files
|
|
which don\[cq]t match.
|
|
It doesn\[cq]t alter the source or destination.
|
|
.PP
|
|
If you supply the \[en]size\-only flag, it will only compare the sizes
|
|
not the hashes as well.
|
|
Use this for a quick check.
|
|
.PP
|
|
If you supply the \[en]download flag, it will download the data from
|
|
both remotes and check them against each other on the fly.
|
|
This can be useful for remotes that don\[cq]t support hashes or if you
|
|
really want to check all the data.
|
|
.PP
|
|
If you supply the \[en]one\-way flag, it will only check that files in
|
|
source match the files in destination, not the other way around.
|
|
Meaning extra files in destination that are not in the source will not
|
|
trigger an error.
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone check source:path dest:path [flags]
|
|
\f[R]
|
|
.fi
|
|
.SS Options
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
\-\-download Check by downloading rather than with hash.
|
|
\-h, \-\-help help for check
|
|
\-\-one\-way Check one way only, source files must exist on remote
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
See the global flags page (https://rclone.org/flags/) for global options
|
|
not listed here.
|
|
.SS SEE ALSO
|
|
.IP \[bu] 2
|
|
rclone (https://rclone.org/commands/rclone/) \- Show help for rclone
|
|
commands, flags and backends.
|
|
.SH rclone ls
|
|
.PP
|
|
List the objects in the path with size and path.
|
|
.SS Synopsis
|
|
.PP
|
|
Lists the objects in the source path to standard output in a human
|
|
readable format with size and path.
|
|
Recurses by default.
|
|
.PP
|
|
Eg
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
$ rclone ls swift:bucket
|
|
60295 bevajer5jef
|
|
90613 canole
|
|
94467 diwogej7
|
|
37600 fubuwic
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
Any of the filtering options can be applied to this command.
|
|
.PP
|
|
There are several related list commands
|
|
.IP \[bu] 2
|
|
\f[C]ls\f[R] to list size and path of objects only
|
|
.IP \[bu] 2
|
|
\f[C]lsl\f[R] to list modification time, size and path of objects only
|
|
.IP \[bu] 2
|
|
\f[C]lsd\f[R] to list directories only
|
|
.IP \[bu] 2
|
|
\f[C]lsf\f[R] to list objects and directories in easy to parse format
|
|
.IP \[bu] 2
|
|
\f[C]lsjson\f[R] to list objects and directories in JSON format
|
|
.PP
|
|
\f[C]ls\f[R],\f[C]lsl\f[R],\f[C]lsd\f[R] are designed to be human
|
|
readable.
|
|
\f[C]lsf\f[R] is designed to be human and machine readable.
|
|
\f[C]lsjson\f[R] is designed to be machine readable.
|
|
.PP
|
|
Note that \f[C]ls\f[R] and \f[C]lsl\f[R] recurse by default \- use
|
|
\[lq]\[en]max\-depth 1\[rq] to stop the recursion.
|
|
.PP
|
|
The other list commands \f[C]lsd\f[R],\f[C]lsf\f[R],\f[C]lsjson\f[R] do
|
|
not recurse by default \- use \[lq]\-R\[rq] to make them recurse.
|
|
.PP
|
|
Listing a non existent directory will produce an error except for
|
|
remotes which can\[cq]t have empty directories (eg s3, swift, gcs, etc
|
|
\- the bucket based remotes).
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone ls remote:path [flags]
|
|
\f[R]
|
|
.fi
|
|
.SS Options
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
\-h, \-\-help help for ls
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
See the global flags page (https://rclone.org/flags/) for global options
|
|
not listed here.
|
|
.SS SEE ALSO
|
|
.IP \[bu] 2
|
|
rclone (https://rclone.org/commands/rclone/) \- Show help for rclone
|
|
commands, flags and backends.
|
|
.SH rclone lsd
|
|
.PP
|
|
List all directories/containers/buckets in the path.
|
|
.SS Synopsis
|
|
.PP
|
|
Lists the directories in the source path to standard output.
|
|
Does not recurse by default.
|
|
Use the \-R flag to recurse.
|
|
.PP
|
|
This command lists the total size of the directory (if known, \-1 if
|
|
not), the modification time (if known, the current time if not), the
|
|
number of objects in the directory (if known, \-1 if not) and the name
|
|
of the directory, Eg
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
$ rclone lsd swift:
|
|
494000 2018\-04\-26 08:43:20 10000 10000files
|
|
65 2018\-04\-26 08:43:20 1 1File
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
Or
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
$ rclone lsd drive:test
|
|
\-1 2016\-10\-17 17:41:53 \-1 1000files
|
|
\-1 2017\-01\-03 14:40:54 \-1 2500files
|
|
\-1 2017\-07\-08 14:39:28 \-1 4000files
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
If you just want the directory names use \[lq]rclone lsf
|
|
\[en]dirs\-only\[rq].
|
|
.PP
|
|
Any of the filtering options can be applied to this command.
|
|
.PP
|
|
There are several related list commands
|
|
.IP \[bu] 2
|
|
\f[C]ls\f[R] to list size and path of objects only
|
|
.IP \[bu] 2
|
|
\f[C]lsl\f[R] to list modification time, size and path of objects only
|
|
.IP \[bu] 2
|
|
\f[C]lsd\f[R] to list directories only
|
|
.IP \[bu] 2
|
|
\f[C]lsf\f[R] to list objects and directories in easy to parse format
|
|
.IP \[bu] 2
|
|
\f[C]lsjson\f[R] to list objects and directories in JSON format
|
|
.PP
|
|
\f[C]ls\f[R],\f[C]lsl\f[R],\f[C]lsd\f[R] are designed to be human
|
|
readable.
|
|
\f[C]lsf\f[R] is designed to be human and machine readable.
|
|
\f[C]lsjson\f[R] is designed to be machine readable.
|
|
.PP
|
|
Note that \f[C]ls\f[R] and \f[C]lsl\f[R] recurse by default \- use
|
|
\[lq]\[en]max\-depth 1\[rq] to stop the recursion.
|
|
.PP
|
|
The other list commands \f[C]lsd\f[R],\f[C]lsf\f[R],\f[C]lsjson\f[R] do
|
|
not recurse by default \- use \[lq]\-R\[rq] to make them recurse.
|
|
.PP
|
|
Listing a non existent directory will produce an error except for
|
|
remotes which can\[cq]t have empty directories (eg s3, swift, gcs, etc
|
|
\- the bucket based remotes).
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone lsd remote:path [flags]
|
|
\f[R]
|
|
.fi
|
|
.SS Options
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
\-h, \-\-help help for lsd
|
|
\-R, \-\-recursive Recurse into the listing.
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
See the global flags page (https://rclone.org/flags/) for global options
|
|
not listed here.
|
|
.SS SEE ALSO
|
|
.IP \[bu] 2
|
|
rclone (https://rclone.org/commands/rclone/) \- Show help for rclone
|
|
commands, flags and backends.
|
|
.SH rclone lsl
|
|
.PP
|
|
List the objects in path with modification time, size and path.
|
|
.SS Synopsis
|
|
.PP
|
|
Lists the objects in the source path to standard output in a human
|
|
readable format with modification time, size and path.
|
|
Recurses by default.
|
|
.PP
|
|
Eg
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
$ rclone lsl swift:bucket
|
|
60295 2016\-06\-25 18:55:41.062626927 bevajer5jef
|
|
90613 2016\-06\-25 18:55:43.302607074 canole
|
|
94467 2016\-06\-25 18:55:43.046609333 diwogej7
|
|
37600 2016\-06\-25 18:55:40.814629136 fubuwic
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
Any of the filtering options can be applied to this command.
|
|
.PP
|
|
There are several related list commands
|
|
.IP \[bu] 2
|
|
\f[C]ls\f[R] to list size and path of objects only
|
|
.IP \[bu] 2
|
|
\f[C]lsl\f[R] to list modification time, size and path of objects only
|
|
.IP \[bu] 2
|
|
\f[C]lsd\f[R] to list directories only
|
|
.IP \[bu] 2
|
|
\f[C]lsf\f[R] to list objects and directories in easy to parse format
|
|
.IP \[bu] 2
|
|
\f[C]lsjson\f[R] to list objects and directories in JSON format
|
|
.PP
|
|
\f[C]ls\f[R],\f[C]lsl\f[R],\f[C]lsd\f[R] are designed to be human
|
|
readable.
|
|
\f[C]lsf\f[R] is designed to be human and machine readable.
|
|
\f[C]lsjson\f[R] is designed to be machine readable.
|
|
.PP
|
|
Note that \f[C]ls\f[R] and \f[C]lsl\f[R] recurse by default \- use
|
|
\[lq]\[en]max\-depth 1\[rq] to stop the recursion.
|
|
.PP
|
|
The other list commands \f[C]lsd\f[R],\f[C]lsf\f[R],\f[C]lsjson\f[R] do
|
|
not recurse by default \- use \[lq]\-R\[rq] to make them recurse.
|
|
.PP
|
|
Listing a non existent directory will produce an error except for
|
|
remotes which can\[cq]t have empty directories (eg s3, swift, gcs, etc
|
|
\- the bucket based remotes).
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone lsl remote:path [flags]
|
|
\f[R]
|
|
.fi
|
|
.SS Options
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
\-h, \-\-help help for lsl
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
See the global flags page (https://rclone.org/flags/) for global options
|
|
not listed here.
|
|
.SS SEE ALSO
|
|
.IP \[bu] 2
|
|
rclone (https://rclone.org/commands/rclone/) \- Show help for rclone
|
|
commands, flags and backends.
|
|
.SH rclone md5sum
|
|
.PP
|
|
Produces an md5sum file for all the objects in the path.
|
|
.SS Synopsis
|
|
.PP
|
|
Produces an md5sum file for all the objects in the path.
|
|
This is in the same format as the standard md5sum tool produces.
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone md5sum remote:path [flags]
|
|
\f[R]
|
|
.fi
|
|
.SS Options
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
\-h, \-\-help help for md5sum
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
See the global flags page (https://rclone.org/flags/) for global options
|
|
not listed here.
|
|
.SS SEE ALSO
|
|
.IP \[bu] 2
|
|
rclone (https://rclone.org/commands/rclone/) \- Show help for rclone
|
|
commands, flags and backends.
|
|
.SH rclone sha1sum
|
|
.PP
|
|
Produces an sha1sum file for all the objects in the path.
|
|
.SS Synopsis
|
|
.PP
|
|
Produces an sha1sum file for all the objects in the path.
|
|
This is in the same format as the standard sha1sum tool produces.
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone sha1sum remote:path [flags]
|
|
\f[R]
|
|
.fi
|
|
.SS Options
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
\-h, \-\-help help for sha1sum
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
See the global flags page (https://rclone.org/flags/) for global options
|
|
not listed here.
|
|
.SS SEE ALSO
|
|
.IP \[bu] 2
|
|
rclone (https://rclone.org/commands/rclone/) \- Show help for rclone
|
|
commands, flags and backends.
|
|
.SH rclone size
|
|
.PP
|
|
Prints the total size and number of objects in remote:path.
|
|
.SS Synopsis
|
|
.PP
|
|
Prints the total size and number of objects in remote:path.
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone size remote:path [flags]
|
|
\f[R]
|
|
.fi
|
|
.SS Options
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
\-h, \-\-help help for size
|
|
\-\-json format output as JSON
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
See the global flags page (https://rclone.org/flags/) for global options
|
|
not listed here.
|
|
.SS SEE ALSO
|
|
.IP \[bu] 2
|
|
rclone (https://rclone.org/commands/rclone/) \- Show help for rclone
|
|
commands, flags and backends.
|
|
.SH rclone version
|
|
.PP
|
|
Show the version number.
|
|
.SS Synopsis
|
|
.PP
|
|
Show the version number, the go version and the architecture.
|
|
.PP
|
|
Eg
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
$ rclone version
|
|
rclone v1.41
|
|
\- os/arch: linux/amd64
|
|
\- go version: go1.10
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
If you supply the \[en]check flag, then it will do an online check to
|
|
compare your version with the latest release and the latest beta.
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
$ rclone version \-\-check
|
|
yours: 1.42.0.6
|
|
latest: 1.42 (released 2018\-06\-16)
|
|
beta: 1.42.0.5 (released 2018\-06\-17)
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
Or
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
$ rclone version \-\-check
|
|
yours: 1.41
|
|
latest: 1.42 (released 2018\-06\-16)
|
|
upgrade: https://downloads.rclone.org/v1.42
|
|
beta: 1.42.0.5 (released 2018\-06\-17)
|
|
upgrade: https://beta.rclone.org/v1.42\-005\-g56e1e820
|
|
\f[R]
|
|
.fi
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone version [flags]
|
|
\f[R]
|
|
.fi
|
|
.SS Options
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
\-\-check Check for new version.
|
|
\-h, \-\-help help for version
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
See the global flags page (https://rclone.org/flags/) for global options
|
|
not listed here.
|
|
.SS SEE ALSO
|
|
.IP \[bu] 2
|
|
rclone (https://rclone.org/commands/rclone/) \- Show help for rclone
|
|
commands, flags and backends.
|
|
.SH rclone cleanup
|
|
.PP
|
|
Clean up the remote if possible
|
|
.SS Synopsis
|
|
.PP
|
|
Clean up the remote if possible.
|
|
Empty the trash or delete old file versions.
|
|
Not supported by all remotes.
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone cleanup remote:path [flags]
|
|
\f[R]
|
|
.fi
|
|
.SS Options
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
\-h, \-\-help help for cleanup
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
See the global flags page (https://rclone.org/flags/) for global options
|
|
not listed here.
|
|
.SS SEE ALSO
|
|
.IP \[bu] 2
|
|
rclone (https://rclone.org/commands/rclone/) \- Show help for rclone
|
|
commands, flags and backends.
|
|
.SH rclone dedupe
|
|
.PP
|
|
Interactively find duplicate files and delete/rename them.
|
|
.SS Synopsis
|
|
.PP
|
|
By default \f[C]dedupe\f[R] interactively finds duplicate files and
|
|
offers to delete all but one or rename them to be different.
|
|
Only useful with Google Drive which can have duplicate file names.
|
|
.PP
|
|
In the first pass it will merge directories with the same name.
|
|
It will do this iteratively until all the identical directories have
|
|
been merged.
|
|
.PP
|
|
The \f[C]dedupe\f[R] command will delete all but one of any identical
|
|
(same md5sum) files it finds without confirmation.
|
|
This means that for most duplicated files the \f[C]dedupe\f[R] command
|
|
will not be interactive.
|
|
You can use \f[C]\-\-dry\-run\f[R] to see what would happen without
|
|
doing anything.
|
|
.PP
|
|
Here is an example run.
|
|
.PP
|
|
Before \- with duplicates
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
$ rclone lsl drive:dupes
|
|
6048320 2016\-03\-05 16:23:16.798000000 one.txt
|
|
6048320 2016\-03\-05 16:23:11.775000000 one.txt
|
|
564374 2016\-03\-05 16:23:06.731000000 one.txt
|
|
6048320 2016\-03\-05 16:18:26.092000000 one.txt
|
|
6048320 2016\-03\-05 16:22:46.185000000 two.txt
|
|
1744073 2016\-03\-05 16:22:38.104000000 two.txt
|
|
564374 2016\-03\-05 16:22:52.118000000 two.txt
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
Now the \f[C]dedupe\f[R] session
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
$ rclone dedupe drive:dupes
|
|
2016/03/05 16:24:37 Google drive root \[aq]dupes\[aq]: Looking for duplicates using interactive mode.
|
|
one.txt: Found 4 duplicates \- deleting identical copies
|
|
one.txt: Deleting 2/3 identical duplicates (md5sum \[dq]1eedaa9fe86fd4b8632e2ac549403b36\[dq])
|
|
one.txt: 2 duplicates remain
|
|
1: 6048320 bytes, 2016\-03\-05 16:23:16.798000000, md5sum 1eedaa9fe86fd4b8632e2ac549403b36
|
|
2: 564374 bytes, 2016\-03\-05 16:23:06.731000000, md5sum 7594e7dc9fc28f727c42ee3e0749de81
|
|
s) Skip and do nothing
|
|
k) Keep just one (choose which in next step)
|
|
r) Rename all to be different (by changing file.jpg to file\-1.jpg)
|
|
s/k/r> k
|
|
Enter the number of the file to keep> 1
|
|
one.txt: Deleted 1 extra copies
|
|
two.txt: Found 3 duplicates \- deleting identical copies
|
|
two.txt: 3 duplicates remain
|
|
1: 564374 bytes, 2016\-03\-05 16:22:52.118000000, md5sum 7594e7dc9fc28f727c42ee3e0749de81
|
|
2: 6048320 bytes, 2016\-03\-05 16:22:46.185000000, md5sum 1eedaa9fe86fd4b8632e2ac549403b36
|
|
3: 1744073 bytes, 2016\-03\-05 16:22:38.104000000, md5sum 851957f7fb6f0bc4ce76be966d336802
|
|
s) Skip and do nothing
|
|
k) Keep just one (choose which in next step)
|
|
r) Rename all to be different (by changing file.jpg to file\-1.jpg)
|
|
s/k/r> r
|
|
two\-1.txt: renamed from: two.txt
|
|
two\-2.txt: renamed from: two.txt
|
|
two\-3.txt: renamed from: two.txt
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
The result being
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
$ rclone lsl drive:dupes
|
|
6048320 2016\-03\-05 16:23:16.798000000 one.txt
|
|
564374 2016\-03\-05 16:22:52.118000000 two\-1.txt
|
|
6048320 2016\-03\-05 16:22:46.185000000 two\-2.txt
|
|
1744073 2016\-03\-05 16:22:38.104000000 two\-3.txt
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
Dedupe can be run non interactively using the \f[C]\-\-dedupe\-mode\f[R]
|
|
flag or by using an extra parameter with the same value
|
|
.IP \[bu] 2
|
|
\f[C]\-\-dedupe\-mode interactive\f[R] \- interactive as above.
|
|
.IP \[bu] 2
|
|
\f[C]\-\-dedupe\-mode skip\f[R] \- removes identical files then skips
|
|
anything left.
|
|
.IP \[bu] 2
|
|
\f[C]\-\-dedupe\-mode first\f[R] \- removes identical files then keeps
|
|
the first one.
|
|
.IP \[bu] 2
|
|
\f[C]\-\-dedupe\-mode newest\f[R] \- removes identical files then keeps
|
|
the newest one.
|
|
.IP \[bu] 2
|
|
\f[C]\-\-dedupe\-mode oldest\f[R] \- removes identical files then keeps
|
|
the oldest one.
|
|
.IP \[bu] 2
|
|
\f[C]\-\-dedupe\-mode largest\f[R] \- removes identical files then keeps
|
|
the largest one.
|
|
.IP \[bu] 2
|
|
\f[C]\-\-dedupe\-mode smallest\f[R] \- removes identical files then
|
|
keeps the smallest one.
|
|
.IP \[bu] 2
|
|
\f[C]\-\-dedupe\-mode rename\f[R] \- removes identical files then
|
|
renames the rest to be different.
|
|
.PP
|
|
For example to rename all the identically named photos in your Google
|
|
Photos directory, do
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone dedupe \-\-dedupe\-mode rename \[dq]drive:Google Photos\[dq]
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
Or
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone dedupe rename \[dq]drive:Google Photos\[dq]
|
|
\f[R]
|
|
.fi
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone dedupe [mode] remote:path [flags]
|
|
\f[R]
|
|
.fi
|
|
.SS Options
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
\-\-dedupe\-mode string Dedupe mode interactive|skip|first|newest|oldest|largest|smallest|rename. (default \[dq]interactive\[dq])
|
|
\-h, \-\-help help for dedupe
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
See the global flags page (https://rclone.org/flags/) for global options
|
|
not listed here.
|
|
.SS SEE ALSO
|
|
.IP \[bu] 2
|
|
rclone (https://rclone.org/commands/rclone/) \- Show help for rclone
|
|
commands, flags and backends.
|
|
.SH rclone about
|
|
.PP
|
|
Get quota information from the remote.
|
|
.SS Synopsis
|
|
.PP
|
|
Get quota information from the remote, like bytes used/free/quota and
|
|
bytes used in the trash.
|
|
Not supported by all remotes.
|
|
.PP
|
|
This will print to stdout something like this:
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
Total: 17G
|
|
Used: 7.444G
|
|
Free: 1.315G
|
|
Trashed: 100.000M
|
|
Other: 8.241G
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
Where the fields are:
|
|
.IP \[bu] 2
|
|
Total: total size available.
|
|
.IP \[bu] 2
|
|
Used: total size used
|
|
.IP \[bu] 2
|
|
Free: total amount this user could upload.
|
|
.IP \[bu] 2
|
|
Trashed: total amount in the trash
|
|
.IP \[bu] 2
|
|
Other: total amount in other storage (eg Gmail, Google Photos)
|
|
.IP \[bu] 2
|
|
Objects: total number of objects in the storage
|
|
.PP
|
|
Note that not all the backends provide all the fields \- they will be
|
|
missing if they are not known for that backend.
|
|
Where it is known that the value is unlimited the value will also be
|
|
omitted.
|
|
.PP
|
|
Use the \[en]full flag to see the numbers written out in full, eg
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
Total: 18253611008
|
|
Used: 7993453766
|
|
Free: 1411001220
|
|
Trashed: 104857602
|
|
Other: 8849156022
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
Use the \[en]json flag for a computer readable output, eg
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
{
|
|
\[dq]total\[dq]: 18253611008,
|
|
\[dq]used\[dq]: 7993453766,
|
|
\[dq]trashed\[dq]: 104857602,
|
|
\[dq]other\[dq]: 8849156022,
|
|
\[dq]free\[dq]: 1411001220
|
|
}
|
|
\f[R]
|
|
.fi
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone about remote: [flags]
|
|
\f[R]
|
|
.fi
|
|
.SS Options
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
\-\-full Full numbers instead of SI units
|
|
\-h, \-\-help help for about
|
|
\-\-json Format output as JSON
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
See the global flags page (https://rclone.org/flags/) for global options
|
|
not listed here.
|
|
.SS SEE ALSO
|
|
.IP \[bu] 2
|
|
rclone (https://rclone.org/commands/rclone/) \- Show help for rclone
|
|
commands, flags and backends.
|
|
.SH rclone authorize
|
|
.PP
|
|
Remote authorization.
|
|
.SS Synopsis
|
|
.PP
|
|
Remote authorization.
|
|
Used to authorize a remote or headless rclone from a machine with a
|
|
browser \- use as instructed by rclone config.
|
|
.PP
|
|
Use the \[en]auth\-no\-open\-browser to prevent rclone to open auth link
|
|
in default browser automatically.
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone authorize [flags]
|
|
\f[R]
|
|
.fi
|
|
.SS Options
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
\-\-auth\-no\-open\-browser Do not automatically open auth link in default browser
|
|
\-h, \-\-help help for authorize
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
See the global flags page (https://rclone.org/flags/) for global options
|
|
not listed here.
|
|
.SS SEE ALSO
|
|
.IP \[bu] 2
|
|
rclone (https://rclone.org/commands/rclone/) \- Show help for rclone
|
|
commands, flags and backends.
|
|
.SH rclone backend
|
|
.PP
|
|
Run a backend specific command.
|
|
.SS Synopsis
|
|
.PP
|
|
This runs a backend specific command.
|
|
The commands themselves (except for \[lq]help\[rq] and
|
|
\[lq]features\[rq]) are defined by the backends and you should see the
|
|
backend docs for definitions.
|
|
.PP
|
|
You can discover what commands a backend implements by using
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone backend help remote:
|
|
rclone backend help <backendname>
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
You can also discover information about the backend using (see
|
|
operations/fsinfo (https://rclone.org/rc/#operations/fsinfo) in the
|
|
remote control docs for more info).
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone backend features remote:
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
Pass options to the backend command with \-o.
|
|
This should be key=value or key, eg:
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone backend stats remote:path stats \-o format=json \-o long
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
Pass arguments to the backend by placing them on the end of the line
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone backend cleanup remote:path file1 file2 file3
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
Note to run these commands on a running backend then see
|
|
backend/command (https://rclone.org/rc/#backend/command) in the rc docs.
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone backend <command> remote:path [opts] <args> [flags]
|
|
\f[R]
|
|
.fi
|
|
.SS Options
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
\-h, \-\-help help for backend
|
|
\-\-json Always output in JSON format.
|
|
\-o, \-\-option stringArray Option in the form name=value or name.
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
See the global flags page (https://rclone.org/flags/) for global options
|
|
not listed here.
|
|
.SS SEE ALSO
|
|
.IP \[bu] 2
|
|
rclone (https://rclone.org/commands/rclone/) \- Show help for rclone
|
|
commands, flags and backends.
|
|
.SH rclone cat
|
|
.PP
|
|
Concatenates any files and sends them to stdout.
|
|
.SS Synopsis
|
|
.PP
|
|
rclone cat sends any files to standard output.
|
|
.PP
|
|
You can use it like this to output a single file
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone cat remote:path/to/file
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
Or like this to output any file in dir or its subdirectories.
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone cat remote:path/to/dir
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
Or like this to output any .txt files in dir or its subdirectories.
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone \-\-include \[dq]*.txt\[dq] cat remote:path/to/dir
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
Use the \[en]head flag to print characters only at the start, \[en]tail
|
|
for the end and \[en]offset and \[en]count to print a section in the
|
|
middle.
|
|
Note that if offset is negative it will count from the end, so
|
|
\[en]offset \-1 \[en]count 1 is equivalent to \[en]tail 1.
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone cat remote:path [flags]
|
|
\f[R]
|
|
.fi
|
|
.SS Options
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
\-\-count int Only print N characters. (default \-1)
|
|
\-\-discard Discard the output instead of printing.
|
|
\-\-head int Only print the first N characters.
|
|
\-h, \-\-help help for cat
|
|
\-\-offset int Start printing at offset N (or from end if \-ve).
|
|
\-\-tail int Only print the last N characters.
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
See the global flags page (https://rclone.org/flags/) for global options
|
|
not listed here.
|
|
.SS SEE ALSO
|
|
.IP \[bu] 2
|
|
rclone (https://rclone.org/commands/rclone/) \- Show help for rclone
|
|
commands, flags and backends.
|
|
.SH rclone config create
|
|
.PP
|
|
Create a new remote with name, type and options.
|
|
.SS Synopsis
|
|
.PP
|
|
Create a new remote of \f[C]name\f[R] with \f[C]type\f[R] and options.
|
|
The options should be passed in pairs of \f[C]key\f[R] \f[C]value\f[R].
|
|
.PP
|
|
For example to make a swift remote of name myremote using auto config
|
|
you would do:
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone config create myremote swift env_auth true
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
Note that if the config process would normally ask a question the
|
|
default is taken.
|
|
Each time that happens rclone will print a message saying how to affect
|
|
the value taken.
|
|
.PP
|
|
If any of the parameters passed is a password field, then rclone will
|
|
automatically obscure them if they aren\[cq]t already obscured before
|
|
putting them in the config file.
|
|
.PP
|
|
\f[B]NB\f[R] If the password parameter is 22 characters or longer and
|
|
consists only of base64 characters then rclone can get confused about
|
|
whether the password is already obscured or not and put unobscured
|
|
passwords into the config file.
|
|
If you want to be 100% certain that the passwords get obscured then use
|
|
the \[lq]\[en]obscure\[rq] flag, or if you are 100% certain you are
|
|
already passing obscured passwords then use \[lq]\[en]no\-obscure\[rq].
|
|
You can also set osbscured passwords using the \[lq]rclone config
|
|
password\[rq] command.
|
|
.PP
|
|
So for example if you wanted to configure a Google Drive remote but
|
|
using remote authorization you would do this:
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone config create mydrive drive config_is_local false
|
|
\f[R]
|
|
.fi
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone config create \[ga]name\[ga] \[ga]type\[ga] [\[ga]key\[ga] \[ga]value\[ga]]* [flags]
|
|
\f[R]
|
|
.fi
|
|
.SS Options
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
\-h, \-\-help help for create
|
|
\-\-no\-obscure Force any passwords not to be obscured.
|
|
\-\-obscure Force any passwords to be obscured.
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
See the global flags page (https://rclone.org/flags/) for global options
|
|
not listed here.
|
|
.SS SEE ALSO
|
|
.IP \[bu] 2
|
|
rclone config (https://rclone.org/commands/rclone_config/) \- Enter an
|
|
interactive configuration session.
|
|
.SH rclone config delete
|
|
.PP
|
|
Delete an existing remote \f[C]name\f[R].
|
|
.SS Synopsis
|
|
.PP
|
|
Delete an existing remote \f[C]name\f[R].
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone config delete \[ga]name\[ga] [flags]
|
|
\f[R]
|
|
.fi
|
|
.SS Options
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
\-h, \-\-help help for delete
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
See the global flags page (https://rclone.org/flags/) for global options
|
|
not listed here.
|
|
.SS SEE ALSO
|
|
.IP \[bu] 2
|
|
rclone config (https://rclone.org/commands/rclone_config/) \- Enter an
|
|
interactive configuration session.
|
|
.SH rclone config disconnect
|
|
.PP
|
|
Disconnects user from remote
|
|
.SS Synopsis
|
|
.PP
|
|
This disconnects the remote: passed in to the cloud storage system.
|
|
.PP
|
|
This normally means revoking the oauth token.
|
|
.PP
|
|
To reconnect use \[lq]rclone config reconnect\[rq].
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone config disconnect remote: [flags]
|
|
\f[R]
|
|
.fi
|
|
.SS Options
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
\-h, \-\-help help for disconnect
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
See the global flags page (https://rclone.org/flags/) for global options
|
|
not listed here.
|
|
.SS SEE ALSO
|
|
.IP \[bu] 2
|
|
rclone config (https://rclone.org/commands/rclone_config/) \- Enter an
|
|
interactive configuration session.
|
|
.SH rclone config dump
|
|
.PP
|
|
Dump the config file as JSON.
|
|
.SS Synopsis
|
|
.PP
|
|
Dump the config file as JSON.
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone config dump [flags]
|
|
\f[R]
|
|
.fi
|
|
.SS Options
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
\-h, \-\-help help for dump
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
See the global flags page (https://rclone.org/flags/) for global options
|
|
not listed here.
|
|
.SS SEE ALSO
|
|
.IP \[bu] 2
|
|
rclone config (https://rclone.org/commands/rclone_config/) \- Enter an
|
|
interactive configuration session.
|
|
.SH rclone config edit
|
|
.PP
|
|
Enter an interactive configuration session.
|
|
.SS Synopsis
|
|
.PP
|
|
Enter an interactive configuration session where you can setup new
|
|
remotes and manage existing ones.
|
|
You may also set or remove a password to protect your configuration.
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone config edit [flags]
|
|
\f[R]
|
|
.fi
|
|
.SS Options
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
\-h, \-\-help help for edit
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
See the global flags page (https://rclone.org/flags/) for global options
|
|
not listed here.
|
|
.SS SEE ALSO
|
|
.IP \[bu] 2
|
|
rclone config (https://rclone.org/commands/rclone_config/) \- Enter an
|
|
interactive configuration session.
|
|
.SH rclone config file
|
|
.PP
|
|
Show path of configuration file in use.
|
|
.SS Synopsis
|
|
.PP
|
|
Show path of configuration file in use.
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone config file [flags]
|
|
\f[R]
|
|
.fi
|
|
.SS Options
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
\-h, \-\-help help for file
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
See the global flags page (https://rclone.org/flags/) for global options
|
|
not listed here.
|
|
.SS SEE ALSO
|
|
.IP \[bu] 2
|
|
rclone config (https://rclone.org/commands/rclone_config/) \- Enter an
|
|
interactive configuration session.
|
|
.SH rclone config password
|
|
.PP
|
|
Update password in an existing remote.
|
|
.SS Synopsis
|
|
.PP
|
|
Update an existing remote\[cq]s password.
|
|
The password should be passed in pairs of \f[C]key\f[R] \f[C]value\f[R].
|
|
.PP
|
|
For example to set password of a remote of name myremote you would do:
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone config password myremote fieldname mypassword
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
This command is obsolete now that \[lq]config update\[rq] and
|
|
\[lq]config create\[rq] both support obscuring passwords directly.
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone config password \[ga]name\[ga] [\[ga]key\[ga] \[ga]value\[ga]]+ [flags]
|
|
\f[R]
|
|
.fi
|
|
.SS Options
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
\-h, \-\-help help for password
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
See the global flags page (https://rclone.org/flags/) for global options
|
|
not listed here.
|
|
.SS SEE ALSO
|
|
.IP \[bu] 2
|
|
rclone config (https://rclone.org/commands/rclone_config/) \- Enter an
|
|
interactive configuration session.
|
|
.SH rclone config providers
|
|
.PP
|
|
List in JSON format all the providers and options.
|
|
.SS Synopsis
|
|
.PP
|
|
List in JSON format all the providers and options.
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone config providers [flags]
|
|
\f[R]
|
|
.fi
|
|
.SS Options
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
\-h, \-\-help help for providers
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
See the global flags page (https://rclone.org/flags/) for global options
|
|
not listed here.
|
|
.SS SEE ALSO
|
|
.IP \[bu] 2
|
|
rclone config (https://rclone.org/commands/rclone_config/) \- Enter an
|
|
interactive configuration session.
|
|
.SH rclone config reconnect
|
|
.PP
|
|
Re\-authenticates user with remote.
|
|
.SS Synopsis
|
|
.PP
|
|
This reconnects remote: passed in to the cloud storage system.
|
|
.PP
|
|
To disconnect the remote use \[lq]rclone config disconnect\[rq].
|
|
.PP
|
|
This normally means going through the interactive oauth flow again.
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone config reconnect remote: [flags]
|
|
\f[R]
|
|
.fi
|
|
.SS Options
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
\-h, \-\-help help for reconnect
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
See the global flags page (https://rclone.org/flags/) for global options
|
|
not listed here.
|
|
.SS SEE ALSO
|
|
.IP \[bu] 2
|
|
rclone config (https://rclone.org/commands/rclone_config/) \- Enter an
|
|
interactive configuration session.
|
|
.SH rclone config show
|
|
.PP
|
|
Print (decrypted) config file, or the config for a single remote.
|
|
.SS Synopsis
|
|
.PP
|
|
Print (decrypted) config file, or the config for a single remote.
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone config show [<remote>] [flags]
|
|
\f[R]
|
|
.fi
|
|
.SS Options
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
\-h, \-\-help help for show
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
See the global flags page (https://rclone.org/flags/) for global options
|
|
not listed here.
|
|
.SS SEE ALSO
|
|
.IP \[bu] 2
|
|
rclone config (https://rclone.org/commands/rclone_config/) \- Enter an
|
|
interactive configuration session.
|
|
.SH rclone config update
|
|
.PP
|
|
Update options in an existing remote.
|
|
.SS Synopsis
|
|
.PP
|
|
Update an existing remote\[cq]s options.
|
|
The options should be passed in in pairs of \f[C]key\f[R]
|
|
\f[C]value\f[R].
|
|
.PP
|
|
For example to update the env_auth field of a remote of name myremote
|
|
you would do:
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone config update myremote swift env_auth true
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
If any of the parameters passed is a password field, then rclone will
|
|
automatically obscure them if they aren\[cq]t already obscured before
|
|
putting them in the config file.
|
|
.PP
|
|
\f[B]NB\f[R] If the password parameter is 22 characters or longer and
|
|
consists only of base64 characters then rclone can get confused about
|
|
whether the password is already obscured or not and put unobscured
|
|
passwords into the config file.
|
|
If you want to be 100% certain that the passwords get obscured then use
|
|
the \[lq]\[en]obscure\[rq] flag, or if you are 100% certain you are
|
|
already passing obscured passwords then use \[lq]\[en]no\-obscure\[rq].
|
|
You can also set osbscured passwords using the \[lq]rclone config
|
|
password\[rq] command.
|
|
.PP
|
|
If the remote uses OAuth the token will be updated, if you don\[cq]t
|
|
require this add an extra parameter thus:
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone config update myremote swift env_auth true config_refresh_token false
|
|
\f[R]
|
|
.fi
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone config update \[ga]name\[ga] [\[ga]key\[ga] \[ga]value\[ga]]+ [flags]
|
|
\f[R]
|
|
.fi
|
|
.SS Options
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
\-h, \-\-help help for update
|
|
\-\-no\-obscure Force any passwords not to be obscured.
|
|
\-\-obscure Force any passwords to be obscured.
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
See the global flags page (https://rclone.org/flags/) for global options
|
|
not listed here.
|
|
.SS SEE ALSO
|
|
.IP \[bu] 2
|
|
rclone config (https://rclone.org/commands/rclone_config/) \- Enter an
|
|
interactive configuration session.
|
|
.SH rclone config userinfo
|
|
.PP
|
|
Prints info about logged in user of remote.
|
|
.SS Synopsis
|
|
.PP
|
|
This prints the details of the person logged in to the cloud storage
|
|
system.
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone config userinfo remote: [flags]
|
|
\f[R]
|
|
.fi
|
|
.SS Options
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
\-h, \-\-help help for userinfo
|
|
\-\-json Format output as JSON
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
See the global flags page (https://rclone.org/flags/) for global options
|
|
not listed here.
|
|
.SS SEE ALSO
|
|
.IP \[bu] 2
|
|
rclone config (https://rclone.org/commands/rclone_config/) \- Enter an
|
|
interactive configuration session.
|
|
.SH rclone copyto
|
|
.PP
|
|
Copy files from source to dest, skipping already copied
|
|
.SS Synopsis
|
|
.PP
|
|
If source:path is a file or directory then it copies it to a file or
|
|
directory named dest:path.
|
|
.PP
|
|
This can be used to upload single files to other than their current
|
|
name.
|
|
If the source is a directory then it acts exactly like the copy command.
|
|
.PP
|
|
So
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone copyto src dst
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
where src and dst are rclone paths, either remote:path or /path/to/local
|
|
or C:.
|
|
.PP
|
|
This will:
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
if src is file
|
|
copy it to dst, overwriting an existing file if it exists
|
|
if src is directory
|
|
copy it to dst, overwriting existing files if they exist
|
|
see copy command for full details
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
This doesn\[cq]t transfer unchanged files, testing by size and
|
|
modification time or MD5SUM.
|
|
It doesn\[cq]t delete files from the destination.
|
|
.PP
|
|
\f[B]Note\f[R]: Use the \f[C]\-P\f[R]/\f[C]\-\-progress\f[R] flag to
|
|
view real\-time transfer statistics
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone copyto source:path dest:path [flags]
|
|
\f[R]
|
|
.fi
|
|
.SS Options
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
\-h, \-\-help help for copyto
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
See the global flags page (https://rclone.org/flags/) for global options
|
|
not listed here.
|
|
.SS SEE ALSO
|
|
.IP \[bu] 2
|
|
rclone (https://rclone.org/commands/rclone/) \- Show help for rclone
|
|
commands, flags and backends.
|
|
.SH rclone copyurl
|
|
.PP
|
|
Copy url content to dest.
|
|
.SS Synopsis
|
|
.PP
|
|
Download a URL\[cq]s content and copy it to the destination without
|
|
saving it in temporary storage.
|
|
.PP
|
|
Setting \[en]auto\-filename will cause the file name to be retrieved
|
|
from the from URL (after any redirections) and used in the destination
|
|
path.
|
|
.PP
|
|
Setting \[en]no\-clobber will prevent overwriting file on the
|
|
destination if there is one with the same name.
|
|
.PP
|
|
Setting \[en]stdout or making the output file name \[lq]\-\[rq] will
|
|
cause the output to be written to standard output.
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone copyurl https://example.com dest:path [flags]
|
|
\f[R]
|
|
.fi
|
|
.SS Options
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
\-a, \-\-auto\-filename Get the file name from the URL and use it for destination file path
|
|
\-h, \-\-help help for copyurl
|
|
\-\-no\-clobber Prevent overwriting file with same name
|
|
\-\-stdout Write the output to stdout rather than a file
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
See the global flags page (https://rclone.org/flags/) for global options
|
|
not listed here.
|
|
.SS SEE ALSO
|
|
.IP \[bu] 2
|
|
rclone (https://rclone.org/commands/rclone/) \- Show help for rclone
|
|
commands, flags and backends.
|
|
.SH rclone cryptcheck
|
|
.PP
|
|
Cryptcheck checks the integrity of a crypted remote.
|
|
.SS Synopsis
|
|
.PP
|
|
rclone cryptcheck checks a remote against a crypted remote.
|
|
This is the equivalent of running rclone check, but able to check the
|
|
checksums of the crypted remote.
|
|
.PP
|
|
For it to work the underlying remote of the cryptedremote must support
|
|
some kind of checksum.
|
|
.PP
|
|
It works by reading the nonce from each file on the cryptedremote: and
|
|
using that to encrypt each file on the remote:.
|
|
It then checks the checksum of the underlying file on the cryptedremote:
|
|
against the checksum of the file it has just encrypted.
|
|
.PP
|
|
Use it like this
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone cryptcheck /path/to/files encryptedremote:path
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
You can use it like this also, but that will involve downloading all the
|
|
files in remote:path.
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone cryptcheck remote:path encryptedremote:path
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
After it has run it will log the status of the encryptedremote:.
|
|
.PP
|
|
If you supply the \[en]one\-way flag, it will only check that files in
|
|
source match the files in destination, not the other way around.
|
|
Meaning extra files in destination that are not in the source will not
|
|
trigger an error.
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone cryptcheck remote:path cryptedremote:path [flags]
|
|
\f[R]
|
|
.fi
|
|
.SS Options
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
\-h, \-\-help help for cryptcheck
|
|
\-\-one\-way Check one way only, source files must exist on destination
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
See the global flags page (https://rclone.org/flags/) for global options
|
|
not listed here.
|
|
.SS SEE ALSO
|
|
.IP \[bu] 2
|
|
rclone (https://rclone.org/commands/rclone/) \- Show help for rclone
|
|
commands, flags and backends.
|
|
.SH rclone cryptdecode
|
|
.PP
|
|
Cryptdecode returns unencrypted file names.
|
|
.SS Synopsis
|
|
.PP
|
|
rclone cryptdecode returns unencrypted file names when provided with a
|
|
list of encrypted file names.
|
|
List limit is 10 items.
|
|
.PP
|
|
If you supply the \[en]reverse flag, it will return encrypted file
|
|
names.
|
|
.PP
|
|
use it like this
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone cryptdecode encryptedremote: encryptedfilename1 encryptedfilename2
|
|
|
|
rclone cryptdecode \-\-reverse encryptedremote: filename1 filename2
|
|
\f[R]
|
|
.fi
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone cryptdecode encryptedremote: encryptedfilename [flags]
|
|
\f[R]
|
|
.fi
|
|
.SS Options
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
\-h, \-\-help help for cryptdecode
|
|
\-\-reverse Reverse cryptdecode, encrypts filenames
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
See the global flags page (https://rclone.org/flags/) for global options
|
|
not listed here.
|
|
.SS SEE ALSO
|
|
.IP \[bu] 2
|
|
rclone (https://rclone.org/commands/rclone/) \- Show help for rclone
|
|
commands, flags and backends.
|
|
.SH rclone deletefile
|
|
.PP
|
|
Remove a single file from remote.
|
|
.SS Synopsis
|
|
.PP
|
|
Remove a single file from remote.
|
|
Unlike \f[C]delete\f[R] it cannot be used to remove a directory and it
|
|
doesn\[cq]t obey include/exclude filters \- if the specified file
|
|
exists, it will always be removed.
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone deletefile remote:path [flags]
|
|
\f[R]
|
|
.fi
|
|
.SS Options
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
\-h, \-\-help help for deletefile
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
See the global flags page (https://rclone.org/flags/) for global options
|
|
not listed here.
|
|
.SS SEE ALSO
|
|
.IP \[bu] 2
|
|
rclone (https://rclone.org/commands/rclone/) \- Show help for rclone
|
|
commands, flags and backends.
|
|
.SH rclone genautocomplete
|
|
.PP
|
|
Output completion script for a given shell.
|
|
.SS Synopsis
|
|
.PP
|
|
Generates a shell completion script for rclone.
|
|
Run with \[en]help to list the supported shells.
|
|
.SS Options
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
\-h, \-\-help help for genautocomplete
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
See the global flags page (https://rclone.org/flags/) for global options
|
|
not listed here.
|
|
.SS SEE ALSO
|
|
.IP \[bu] 2
|
|
rclone (https://rclone.org/commands/rclone/) \- Show help for rclone
|
|
commands, flags and backends.
|
|
.IP \[bu] 2
|
|
rclone genautocomplete
|
|
bash (https://rclone.org/commands/rclone_genautocomplete_bash/) \-
|
|
Output bash completion script for rclone.
|
|
.IP \[bu] 2
|
|
rclone genautocomplete
|
|
fish (https://rclone.org/commands/rclone_genautocomplete_fish/) \-
|
|
Output fish completion script for rclone.
|
|
.IP \[bu] 2
|
|
rclone genautocomplete
|
|
zsh (https://rclone.org/commands/rclone_genautocomplete_zsh/) \- Output
|
|
zsh completion script for rclone.
|
|
.SH rclone genautocomplete bash
|
|
.PP
|
|
Output bash completion script for rclone.
|
|
.SS Synopsis
|
|
.PP
|
|
Generates a bash shell autocompletion script for rclone.
|
|
.PP
|
|
This writes to /etc/bash_completion.d/rclone by default so will probably
|
|
need to be run with sudo or as root, eg
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
sudo rclone genautocomplete bash
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
Logout and login again to use the autocompletion scripts, or source them
|
|
directly
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
\&. /etc/bash_completion
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
If you supply a command line argument the script will be written there.
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone genautocomplete bash [output_file] [flags]
|
|
\f[R]
|
|
.fi
|
|
.SS Options
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
\-h, \-\-help help for bash
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
See the global flags page (https://rclone.org/flags/) for global options
|
|
not listed here.
|
|
.SS SEE ALSO
|
|
.IP \[bu] 2
|
|
rclone
|
|
genautocomplete (https://rclone.org/commands/rclone_genautocomplete/) \-
|
|
Output completion script for a given shell.
|
|
.SH rclone genautocomplete fish
|
|
.PP
|
|
Output fish completion script for rclone.
|
|
.SS Synopsis
|
|
.PP
|
|
Generates a fish autocompletion script for rclone.
|
|
.PP
|
|
This writes to /etc/fish/completions/rclone.fish by default so will
|
|
probably need to be run with sudo or as root, eg
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
sudo rclone genautocomplete fish
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
Logout and login again to use the autocompletion scripts, or source them
|
|
directly
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
\&. /etc/fish/completions/rclone.fish
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
If you supply a command line argument the script will be written there.
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone genautocomplete fish [output_file] [flags]
|
|
\f[R]
|
|
.fi
|
|
.SS Options
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
\-h, \-\-help help for fish
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
See the global flags page (https://rclone.org/flags/) for global options
|
|
not listed here.
|
|
.SS SEE ALSO
|
|
.IP \[bu] 2
|
|
rclone
|
|
genautocomplete (https://rclone.org/commands/rclone_genautocomplete/) \-
|
|
Output completion script for a given shell.
|
|
.SH rclone genautocomplete zsh
|
|
.PP
|
|
Output zsh completion script for rclone.
|
|
.SS Synopsis
|
|
.PP
|
|
Generates a zsh autocompletion script for rclone.
|
|
.PP
|
|
This writes to /usr/share/zsh/vendor\-completions/_rclone by default so
|
|
will probably need to be run with sudo or as root, eg
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
sudo rclone genautocomplete zsh
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
Logout and login again to use the autocompletion scripts, or source them
|
|
directly
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
autoload \-U compinit && compinit
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
If you supply a command line argument the script will be written there.
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone genautocomplete zsh [output_file] [flags]
|
|
\f[R]
|
|
.fi
|
|
.SS Options
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
\-h, \-\-help help for zsh
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
See the global flags page (https://rclone.org/flags/) for global options
|
|
not listed here.
|
|
.SS SEE ALSO
|
|
.IP \[bu] 2
|
|
rclone
|
|
genautocomplete (https://rclone.org/commands/rclone_genautocomplete/) \-
|
|
Output completion script for a given shell.
|
|
.SH rclone gendocs
|
|
.PP
|
|
Output markdown docs for rclone to the directory supplied.
|
|
.SS Synopsis
|
|
.PP
|
|
This produces markdown docs for the rclone commands to the directory
|
|
supplied.
|
|
These are in a format suitable for hugo to render into the rclone.org
|
|
website.
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone gendocs output_directory [flags]
|
|
\f[R]
|
|
.fi
|
|
.SS Options
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
\-h, \-\-help help for gendocs
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
See the global flags page (https://rclone.org/flags/) for global options
|
|
not listed here.
|
|
.SS SEE ALSO
|
|
.IP \[bu] 2
|
|
rclone (https://rclone.org/commands/rclone/) \- Show help for rclone
|
|
commands, flags and backends.
|
|
.SH rclone hashsum
|
|
.PP
|
|
Produces a hashsum file for all the objects in the path.
|
|
.SS Synopsis
|
|
.PP
|
|
Produces a hash file for all the objects in the path using the hash
|
|
named.
|
|
The output is in the same format as the standard md5sum/sha1sum tool.
|
|
.PP
|
|
Run without a hash to see the list of supported hashes, eg
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
$ rclone hashsum
|
|
Supported hashes are:
|
|
* MD5
|
|
* SHA\-1
|
|
* DropboxHash
|
|
* QuickXorHash
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
Then
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
$ rclone hashsum MD5 remote:path
|
|
\f[R]
|
|
.fi
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone hashsum <hash> remote:path [flags]
|
|
\f[R]
|
|
.fi
|
|
.SS Options
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
\-\-base64 Output base64 encoded hashsum
|
|
\-h, \-\-help help for hashsum
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
See the global flags page (https://rclone.org/flags/) for global options
|
|
not listed here.
|
|
.SS SEE ALSO
|
|
.IP \[bu] 2
|
|
rclone (https://rclone.org/commands/rclone/) \- Show help for rclone
|
|
commands, flags and backends.
|
|
.SH rclone link
|
|
.PP
|
|
Generate public link to file/folder.
|
|
.SS Synopsis
|
|
.PP
|
|
rclone link will create or retrieve a public link to the given file or
|
|
folder.
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone link remote:path/to/file
|
|
rclone link remote:path/to/folder/
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
If successful, the last line of the output will contain the link.
|
|
Exact capabilities depend on the remote, but the link will always be
|
|
created with the least constraints \[en] e.g.\ no expiry, no password
|
|
protection, accessible without account.
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone link remote:path [flags]
|
|
\f[R]
|
|
.fi
|
|
.SS Options
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
\-h, \-\-help help for link
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
See the global flags page (https://rclone.org/flags/) for global options
|
|
not listed here.
|
|
.SS SEE ALSO
|
|
.IP \[bu] 2
|
|
rclone (https://rclone.org/commands/rclone/) \- Show help for rclone
|
|
commands, flags and backends.
|
|
.SH rclone listremotes
|
|
.PP
|
|
List all the remotes in the config file.
|
|
.SS Synopsis
|
|
.PP
|
|
rclone listremotes lists all the available remotes from the config file.
|
|
.PP
|
|
When uses with the \-l flag it lists the types too.
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone listremotes [flags]
|
|
\f[R]
|
|
.fi
|
|
.SS Options
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
\-h, \-\-help help for listremotes
|
|
\-\-long Show the type as well as names.
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
See the global flags page (https://rclone.org/flags/) for global options
|
|
not listed here.
|
|
.SS SEE ALSO
|
|
.IP \[bu] 2
|
|
rclone (https://rclone.org/commands/rclone/) \- Show help for rclone
|
|
commands, flags and backends.
|
|
.SH rclone lsf
|
|
.PP
|
|
List directories and objects in remote:path formatted for parsing
|
|
.SS Synopsis
|
|
.PP
|
|
List the contents of the source path (directories and objects) to
|
|
standard output in a form which is easy to parse by scripts.
|
|
By default this will just be the names of the objects and directories,
|
|
one per line.
|
|
The directories will have a / suffix.
|
|
.PP
|
|
Eg
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
$ rclone lsf swift:bucket
|
|
bevajer5jef
|
|
canole
|
|
diwogej7
|
|
ferejej3gux/
|
|
fubuwic
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
Use the \[en]format option to control what gets listed.
|
|
By default this is just the path, but you can use these parameters to
|
|
control the output:
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
p \- path
|
|
s \- size
|
|
t \- modification time
|
|
h \- hash
|
|
i \- ID of object
|
|
o \- Original ID of underlying object
|
|
m \- MimeType of object if known
|
|
e \- encrypted name
|
|
T \- tier of storage if known, eg \[dq]Hot\[dq] or \[dq]Cool\[dq]
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
So if you wanted the path, size and modification time, you would use
|
|
\[en]format \[lq]pst\[rq], or maybe \[en]format \[lq]tsp\[rq] to put the
|
|
path last.
|
|
.PP
|
|
Eg
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
$ rclone lsf \-\-format \[dq]tsp\[dq] swift:bucket
|
|
2016\-06\-25 18:55:41;60295;bevajer5jef
|
|
2016\-06\-25 18:55:43;90613;canole
|
|
2016\-06\-25 18:55:43;94467;diwogej7
|
|
2018\-04\-26 08:50:45;0;ferejej3gux/
|
|
2016\-06\-25 18:55:40;37600;fubuwic
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
If you specify \[lq]h\[rq] in the format you will get the MD5 hash by
|
|
default, use the \[lq]\[en]hash\[rq] flag to change which hash you want.
|
|
Note that this can be returned as an empty string if it isn\[cq]t
|
|
available on the object (and for directories), \[lq]ERROR\[rq] if there
|
|
was an error reading it from the object and \[lq]UNSUPPORTED\[rq] if
|
|
that object does not support that hash type.
|
|
.PP
|
|
For example to emulate the md5sum command you can use
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone lsf \-R \-\-hash MD5 \-\-format hp \-\-separator \[dq] \[dq] \-\-files\-only .
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
Eg
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
$ rclone lsf \-R \-\-hash MD5 \-\-format hp \-\-separator \[dq] \[dq] \-\-files\-only swift:bucket
|
|
7908e352297f0f530b84a756f188baa3 bevajer5jef
|
|
cd65ac234e6fea5925974a51cdd865cc canole
|
|
03b5341b4f234b9d984d03ad076bae91 diwogej7
|
|
8fd37c3810dd660778137ac3a66cc06d fubuwic
|
|
99713e14a4c4ff553acaf1930fad985b gixacuh7ku
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
(Though \[lq]rclone md5sum .\[rq] is an easier way of typing this.)
|
|
.PP
|
|
By default the separator is \[lq];\[rq] this can be changed with the
|
|
\[en]separator flag.
|
|
Note that separators aren\[cq]t escaped in the path so putting it last
|
|
is a good strategy.
|
|
.PP
|
|
Eg
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
$ rclone lsf \-\-separator \[dq],\[dq] \-\-format \[dq]tshp\[dq] swift:bucket
|
|
2016\-06\-25 18:55:41,60295,7908e352297f0f530b84a756f188baa3,bevajer5jef
|
|
2016\-06\-25 18:55:43,90613,cd65ac234e6fea5925974a51cdd865cc,canole
|
|
2016\-06\-25 18:55:43,94467,03b5341b4f234b9d984d03ad076bae91,diwogej7
|
|
2018\-04\-26 08:52:53,0,,ferejej3gux/
|
|
2016\-06\-25 18:55:40,37600,8fd37c3810dd660778137ac3a66cc06d,fubuwic
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
You can output in CSV standard format.
|
|
This will escape things in \[dq] if they contain ,
|
|
.PP
|
|
Eg
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
$ rclone lsf \-\-csv \-\-files\-only \-\-format ps remote:path
|
|
test.log,22355
|
|
test.sh,449
|
|
\[dq]this file contains a comma, in the file name.txt\[dq],6
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
Note that the \[en]absolute parameter is useful for making lists of
|
|
files to pass to an rclone copy with the \[en]files\-from\-raw flag.
|
|
.PP
|
|
For example to find all the files modified within one day and copy those
|
|
only (without traversing the whole directory structure):
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone lsf \-\-absolute \-\-files\-only \-\-max\-age 1d /path/to/local > new_files
|
|
rclone copy \-\-files\-from\-raw new_files /path/to/local remote:path
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
Any of the filtering options can be applied to this command.
|
|
.PP
|
|
There are several related list commands
|
|
.IP \[bu] 2
|
|
\f[C]ls\f[R] to list size and path of objects only
|
|
.IP \[bu] 2
|
|
\f[C]lsl\f[R] to list modification time, size and path of objects only
|
|
.IP \[bu] 2
|
|
\f[C]lsd\f[R] to list directories only
|
|
.IP \[bu] 2
|
|
\f[C]lsf\f[R] to list objects and directories in easy to parse format
|
|
.IP \[bu] 2
|
|
\f[C]lsjson\f[R] to list objects and directories in JSON format
|
|
.PP
|
|
\f[C]ls\f[R],\f[C]lsl\f[R],\f[C]lsd\f[R] are designed to be human
|
|
readable.
|
|
\f[C]lsf\f[R] is designed to be human and machine readable.
|
|
\f[C]lsjson\f[R] is designed to be machine readable.
|
|
.PP
|
|
Note that \f[C]ls\f[R] and \f[C]lsl\f[R] recurse by default \- use
|
|
\[lq]\[en]max\-depth 1\[rq] to stop the recursion.
|
|
.PP
|
|
The other list commands \f[C]lsd\f[R],\f[C]lsf\f[R],\f[C]lsjson\f[R] do
|
|
not recurse by default \- use \[lq]\-R\[rq] to make them recurse.
|
|
.PP
|
|
Listing a non existent directory will produce an error except for
|
|
remotes which can\[cq]t have empty directories (eg s3, swift, gcs, etc
|
|
\- the bucket based remotes).
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone lsf remote:path [flags]
|
|
\f[R]
|
|
.fi
|
|
.SS Options
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
\-\-absolute Put a leading / in front of path names.
|
|
\-\-csv Output in CSV format.
|
|
\-d, \-\-dir\-slash Append a slash to directory names. (default true)
|
|
\-\-dirs\-only Only list directories.
|
|
\-\-files\-only Only list files.
|
|
\-F, \-\-format string Output format \- see help for details (default \[dq]p\[dq])
|
|
\-\-hash h Use this hash when h is used in the format MD5|SHA\-1|DropboxHash (default \[dq]MD5\[dq])
|
|
\-h, \-\-help help for lsf
|
|
\-R, \-\-recursive Recurse into the listing.
|
|
\-s, \-\-separator string Separator for the items in the format. (default \[dq];\[dq])
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
See the global flags page (https://rclone.org/flags/) for global options
|
|
not listed here.
|
|
.SS SEE ALSO
|
|
.IP \[bu] 2
|
|
rclone (https://rclone.org/commands/rclone/) \- Show help for rclone
|
|
commands, flags and backends.
|
|
.SH rclone lsjson
|
|
.PP
|
|
List directories and objects in the path in JSON format.
|
|
.SS Synopsis
|
|
.PP
|
|
List directories and objects in the path in JSON format.
|
|
.PP
|
|
The output is an array of Items, where each Item looks like this
|
|
.PP
|
|
{ \[lq]Hashes\[rq] : { \[lq]SHA\-1\[rq] :
|
|
\[lq]f572d396fae9206628714fb2ce00f72e94f2258f\[rq], \[lq]MD5\[rq] :
|
|
\[lq]b1946ac92492d2347c6235b4d2611184\[rq], \[lq]DropboxHash\[rq] :
|
|
\[lq]ecb65bb98f9d905b70458986c39fcbad7715e5f2fcc3b1f07767d7c83e2438cc\[rq]
|
|
}, \[lq]ID\[rq]: \[lq]y2djkhiujf83u33\[rq], \[lq]OrigID\[rq]:
|
|
\[lq]UYOJVTUW00Q1RzTDA\[rq], \[lq]IsBucket\[rq] : false, \[lq]IsDir\[rq]
|
|
: false, \[lq]MimeType\[rq] : \[lq]application/octet\-stream\[rq],
|
|
\[lq]ModTime\[rq] : \[lq]2017\-05\-31T16:15:57.034468261+01:00\[rq],
|
|
\[lq]Name\[rq] : \[lq]file.txt\[rq], \[lq]Encrypted\[rq] :
|
|
\[lq]v0qpsdq8anpci8n929v3uu9338\[rq], \[lq]EncryptedPath\[rq] :
|
|
\[lq]kja9098349023498/v0qpsdq8anpci8n929v3uu9338\[rq], \[lq]Path\[rq] :
|
|
\[lq]full/path/goes/here/file.txt\[rq], \[lq]Size\[rq] : 6,
|
|
\[lq]Tier\[rq] : \[lq]hot\[rq], }
|
|
.PP
|
|
If \[en]hash is not specified the Hashes property won\[cq]t be emitted.
|
|
The types of hash can be specified with the \[en]hash\-type parameter
|
|
(which may be repeated).
|
|
If \[en]hash\-type is set then it implies \[en]hash.
|
|
.PP
|
|
If \[en]no\-modtime is specified then ModTime will be blank.
|
|
This can speed things up on remotes where reading the ModTime takes an
|
|
extra request (eg s3, swift).
|
|
.PP
|
|
If \[en]no\-mimetype is specified then MimeType will be blank.
|
|
This can speed things up on remotes where reading the MimeType takes an
|
|
extra request (eg s3, swift).
|
|
.PP
|
|
If \[en]encrypted is not specified the Encrypted won\[cq]t be emitted.
|
|
.PP
|
|
If \[en]dirs\-only is not specified files in addition to directories are
|
|
returned
|
|
.PP
|
|
If \[en]files\-only is not specified directories in addition to the
|
|
files will be returned.
|
|
.PP
|
|
The Path field will only show folders below the remote path being
|
|
listed.
|
|
If \[lq]remote:path\[rq] contains the file \[lq]subfolder/file.txt\[rq],
|
|
the Path for \[lq]file.txt\[rq] will be \[lq]subfolder/file.txt\[rq],
|
|
not \[lq]remote:path/subfolder/file.txt\[rq].
|
|
When used without \[en]recursive the Path will always be the same as
|
|
Name.
|
|
.PP
|
|
If the directory is a bucket in a bucket based backend, then
|
|
\[lq]IsBucket\[rq] will be set to true.
|
|
This key won\[cq]t be present unless it is \[lq]true\[rq].
|
|
.PP
|
|
The time is in RFC3339 format with up to nanosecond precision.
|
|
The number of decimal digits in the seconds will depend on the precision
|
|
that the remote can hold the times, so if times are accurate to the
|
|
nearest millisecond (eg Google Drive) then 3 digits will always be shown
|
|
(\[lq]2017\-05\-31T16:15:57.034+01:00\[rq]) whereas if the times are
|
|
accurate to the nearest second (Dropbox, Box, WebDav etc) no digits will
|
|
be shown (\[lq]2017\-05\-31T16:15:57+01:00\[rq]).
|
|
.PP
|
|
The whole output can be processed as a JSON blob, or alternatively it
|
|
can be processed line by line as each item is written one to a line.
|
|
.PP
|
|
Any of the filtering options can be applied to this command.
|
|
.PP
|
|
There are several related list commands
|
|
.IP \[bu] 2
|
|
\f[C]ls\f[R] to list size and path of objects only
|
|
.IP \[bu] 2
|
|
\f[C]lsl\f[R] to list modification time, size and path of objects only
|
|
.IP \[bu] 2
|
|
\f[C]lsd\f[R] to list directories only
|
|
.IP \[bu] 2
|
|
\f[C]lsf\f[R] to list objects and directories in easy to parse format
|
|
.IP \[bu] 2
|
|
\f[C]lsjson\f[R] to list objects and directories in JSON format
|
|
.PP
|
|
\f[C]ls\f[R],\f[C]lsl\f[R],\f[C]lsd\f[R] are designed to be human
|
|
readable.
|
|
\f[C]lsf\f[R] is designed to be human and machine readable.
|
|
\f[C]lsjson\f[R] is designed to be machine readable.
|
|
.PP
|
|
Note that \f[C]ls\f[R] and \f[C]lsl\f[R] recurse by default \- use
|
|
\[lq]\[en]max\-depth 1\[rq] to stop the recursion.
|
|
.PP
|
|
The other list commands \f[C]lsd\f[R],\f[C]lsf\f[R],\f[C]lsjson\f[R] do
|
|
not recurse by default \- use \[lq]\-R\[rq] to make them recurse.
|
|
.PP
|
|
Listing a non existent directory will produce an error except for
|
|
remotes which can\[cq]t have empty directories (eg s3, swift, gcs, etc
|
|
\- the bucket based remotes).
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone lsjson remote:path [flags]
|
|
\f[R]
|
|
.fi
|
|
.SS Options
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
\-\-dirs\-only Show only directories in the listing.
|
|
\-M, \-\-encrypted Show the encrypted names.
|
|
\-\-files\-only Show only files in the listing.
|
|
\-\-hash Include hashes in the output (may take longer).
|
|
\-\-hash\-type stringArray Show only this hash type (may be repeated).
|
|
\-h, \-\-help help for lsjson
|
|
\-\-no\-mimetype Don\[aq]t read the mime type (can speed things up).
|
|
\-\-no\-modtime Don\[aq]t read the modification time (can speed things up).
|
|
\-\-original Show the ID of the underlying Object.
|
|
\-R, \-\-recursive Recurse into the listing.
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
See the global flags page (https://rclone.org/flags/) for global options
|
|
not listed here.
|
|
.SS SEE ALSO
|
|
.IP \[bu] 2
|
|
rclone (https://rclone.org/commands/rclone/) \- Show help for rclone
|
|
commands, flags and backends.
|
|
.SH rclone mount
|
|
.PP
|
|
Mount the remote as file system on a mountpoint.
|
|
.SS Synopsis
|
|
.PP
|
|
rclone mount allows Linux, FreeBSD, macOS and Windows to mount any of
|
|
Rclone\[cq]s cloud storage systems as a file system with FUSE.
|
|
.PP
|
|
First set up your remote using \f[C]rclone config\f[R].
|
|
Check it works with \f[C]rclone ls\f[R] etc.
|
|
.PP
|
|
You can either run mount in foreground mode or background (daemon) mode.
|
|
Mount runs in foreground mode by default, use the \[en]daemon flag to
|
|
specify background mode mode.
|
|
Background mode is only supported on Linux and OSX, you can only run
|
|
mount in foreground mode on Windows.
|
|
.PP
|
|
On Linux/macOS/FreeBSD Start the mount like this where
|
|
\f[C]/path/to/local/mount\f[R] is an \f[B]empty\f[R] \f[B]existing\f[R]
|
|
directory.
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone mount remote:path/to/files /path/to/local/mount
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
Or on Windows like this where \f[C]X:\f[R] is an unused drive letter or
|
|
use a path to \f[B]non\-existent\f[R] directory.
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone mount remote:path/to/files X:
|
|
rclone mount remote:path/to/files C:\[rs]path\[rs]to\[rs]nonexistent\[rs]directory
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
When running in background mode the user will have to stop the mount
|
|
manually (specified below).
|
|
.PP
|
|
When the program ends while in foreground mode, either via Ctrl+C or
|
|
receiving a SIGINT or SIGTERM signal, the mount is automatically
|
|
stopped.
|
|
.PP
|
|
The umount operation can fail, for example when the mountpoint is busy.
|
|
When that happens, it is the user\[cq]s responsibility to stop the mount
|
|
manually.
|
|
.PP
|
|
Stopping the mount manually:
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
# Linux
|
|
fusermount \-u /path/to/local/mount
|
|
# OS X
|
|
umount /path/to/local/mount
|
|
\f[R]
|
|
.fi
|
|
.SS Installing on Windows
|
|
.PP
|
|
To run rclone mount on Windows, you will need to download and install
|
|
WinFsp (http://www.secfs.net/winfsp/).
|
|
.PP
|
|
WinFsp (https://github.com/billziss-gh/winfsp) is an open source Windows
|
|
File System Proxy which makes it easy to write user space file systems
|
|
for Windows.
|
|
It provides a FUSE emulation layer which rclone uses combination with
|
|
cgofuse (https://github.com/billziss-gh/cgofuse).
|
|
Both of these packages are by Bill Zissimopoulos who was very helpful
|
|
during the implementation of rclone mount for Windows.
|
|
.SS Windows caveats
|
|
.PP
|
|
Note that drives created as Administrator are not visible by other
|
|
accounts (including the account that was elevated as Administrator).
|
|
So if you start a Windows drive from an Administrative Command Prompt
|
|
and then try to access the same drive from Explorer (which does not run
|
|
as Administrator), you will not be able to see the new drive.
|
|
.PP
|
|
The easiest way around this is to start the drive from a normal command
|
|
prompt.
|
|
It is also possible to start a drive from the SYSTEM account (using the
|
|
WinFsp.Launcher
|
|
infrastructure (https://github.com/billziss-gh/winfsp/wiki/WinFsp-Service-Architecture))
|
|
which creates drives accessible for everyone on the system or
|
|
alternatively using the nssm service manager (https://nssm.cc/usage).
|
|
.SS Mount as a network drive
|
|
.PP
|
|
By default, rclone will mount the remote as a normal drive.
|
|
However, you can also mount it as a \f[B]Network Drive\f[R] (or
|
|
\f[B]Network Share\f[R], as mentioned in some places)
|
|
.PP
|
|
Unlike other systems, Windows provides a different filesystem type for
|
|
network drives.
|
|
Windows and other programs treat the network drives and fixed/removable
|
|
drives differently: In network drives, many I/O operations are
|
|
optimized, as the high latency and low reliability (compared to a normal
|
|
drive) of a network is expected.
|
|
.PP
|
|
Although many people prefer network shares to be mounted as normal
|
|
system drives, this might cause some issues, such as programs not
|
|
working as expected or freezes and errors while operating with the
|
|
mounted remote in Windows Explorer.
|
|
If you experience any of those, consider mounting rclone remotes as
|
|
network shares, as Windows expects normal drives to be fast and
|
|
reliable, while cloud storage is far from that.
|
|
See also Limitations section below for more info
|
|
.PP
|
|
Add \[lq]\[en]fuse\-flag \[en]VolumePrefix=\[rq] to your \[lq]mount\[rq]
|
|
command, \f[B]replacing \[lq]share\[rq] with any other name of your
|
|
choice if you are mounting more than one remote\f[R].
|
|
Otherwise, the mountpoints will conflict and your mounted filesystems
|
|
will overlap.
|
|
.PP
|
|
Read more about drive
|
|
mapping (https://en.wikipedia.org/wiki/Drive_mapping)
|
|
.SS Limitations
|
|
.PP
|
|
Without the use of \[lq]\[en]vfs\-cache\-mode\[rq] this can only write
|
|
files sequentially, it can only seek when reading.
|
|
This means that many applications won\[cq]t work with their files on an
|
|
rclone mount without \[lq]\[en]vfs\-cache\-mode writes\[rq] or
|
|
\[lq]\[en]vfs\-cache\-mode full\[rq].
|
|
See the File Caching section for more info.
|
|
.PP
|
|
The bucket based remotes (eg Swift, S3, Google Compute Storage, B2,
|
|
Hubic) do not support the concept of empty directories, so empty
|
|
directories will have a tendency to disappear once they fall out of the
|
|
directory cache.
|
|
.PP
|
|
Only supported on Linux, FreeBSD, OS X and Windows at the moment.
|
|
.SS rclone mount vs rclone sync/copy
|
|
.PP
|
|
File systems expect things to be 100% reliable, whereas cloud storage
|
|
systems are a long way from 100% reliable.
|
|
The rclone sync/copy commands cope with this with lots of retries.
|
|
However rclone mount can\[cq]t use retries in the same way without
|
|
making local copies of the uploads.
|
|
Look at the file caching for solutions to make mount more reliable.
|
|
.SS Attribute caching
|
|
.PP
|
|
You can use the flag \[en]attr\-timeout to set the time the kernel
|
|
caches the attributes (size, modification time etc) for directory
|
|
entries.
|
|
.PP
|
|
The default is \[lq]1s\[rq] which caches files just long enough to avoid
|
|
too many callbacks to rclone from the kernel.
|
|
.PP
|
|
In theory 0s should be the correct value for filesystems which can
|
|
change outside the control of the kernel.
|
|
However this causes quite a few problems such as rclone using too much
|
|
memory (https://github.com/rclone/rclone/issues/2157), rclone not
|
|
serving files to
|
|
samba (https://forum.rclone.org/t/rclone-1-39-vs-1-40-mount-issue/5112)
|
|
and excessive time listing
|
|
directories (https://github.com/rclone/rclone/issues/2095#issuecomment-371141147).
|
|
.PP
|
|
The kernel can cache the info about a file for the time given by
|
|
\[lq]\[en]attr\-timeout\[rq].
|
|
You may see corruption if the remote file changes length during this
|
|
window.
|
|
It will show up as either a truncated file or a file with garbage on the
|
|
end.
|
|
With \[lq]\[en]attr\-timeout 1s\[rq] this is very unlikely but not
|
|
impossible.
|
|
The higher you set \[lq]\[en]attr\-timeout\[rq] the more likely it is.
|
|
The default setting of \[lq]1s\[rq] is the lowest setting which
|
|
mitigates the problems above.
|
|
.PP
|
|
If you set it higher (`10s' or `1m' say) then the kernel will call back
|
|
to rclone less often making it more efficient, however there is more
|
|
chance of the corruption issue above.
|
|
.PP
|
|
If files don\[cq]t change on the remote outside of the control of rclone
|
|
then there is no chance of corruption.
|
|
.PP
|
|
This is the same as setting the attr_timeout option in mount.fuse.
|
|
.SS Filters
|
|
.PP
|
|
Note that all the rclone filters can be used to select a subset of the
|
|
files to be visible in the mount.
|
|
.SS systemd
|
|
.PP
|
|
When running rclone mount as a systemd service, it is possible to use
|
|
Type=notify.
|
|
In this case the service will enter the started state after the
|
|
mountpoint has been successfully set up.
|
|
Units having the rclone mount service specified as a requirement will
|
|
see all files and folders immediately in this mode.
|
|
.SS chunked reading
|
|
.PP
|
|
\[en]vfs\-read\-chunk\-size will enable reading the source objects in
|
|
parts.
|
|
This can reduce the used download quota for some remotes by requesting
|
|
only chunks from the remote that are actually read at the cost of an
|
|
increased number of requests.
|
|
.PP
|
|
When \[en]vfs\-read\-chunk\-size\-limit is also specified and greater
|
|
than \[en]vfs\-read\-chunk\-size, the chunk size for each open file will
|
|
get doubled for each chunk read, until the specified value is reached.
|
|
A value of \-1 will disable the limit and the chunk size will grow
|
|
indefinitely.
|
|
.PP
|
|
With \[en]vfs\-read\-chunk\-size 100M and
|
|
\[en]vfs\-read\-chunk\-size\-limit 0 the following parts will be
|
|
downloaded: 0\-100M, 100M\-200M, 200M\-300M, 300M\-400M and so on.
|
|
When \[en]vfs\-read\-chunk\-size\-limit 500M is specified, the result
|
|
would be 0\-100M, 100M\-300M, 300M\-700M, 700M\-1200M, 1200M\-1700M and
|
|
so on.
|
|
.PP
|
|
Chunked reading will only work with \[en]vfs\-cache\-mode < full, as the
|
|
file will always be copied to the vfs cache before opening with
|
|
\[en]vfs\-cache\-mode full.
|
|
.SS Directory Cache
|
|
.PP
|
|
Using the \f[C]\-\-dir\-cache\-time\f[R] flag, you can set how long a
|
|
directory should be considered up to date and not refreshed from the
|
|
backend.
|
|
Changes made locally in the mount may appear immediately or invalidate
|
|
the cache.
|
|
However, changes done on the remote will only be picked up once the
|
|
cache expires if the backend configured does not support polling for
|
|
changes.
|
|
If the backend supports polling, changes will be picked up on within the
|
|
polling interval.
|
|
.PP
|
|
Alternatively, you can send a \f[C]SIGHUP\f[R] signal to rclone for it
|
|
to flush all directory caches, regardless of how old they are.
|
|
Assuming only one rclone instance is running, you can reset the cache
|
|
like this:
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
kill \-SIGHUP $(pidof rclone)
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
If you configure rclone with a remote control (/rc) then you can use
|
|
rclone rc to flush the whole directory cache:
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone rc vfs/forget
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
Or individual files or directories:
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone rc vfs/forget file=path/to/file dir=path/to/dir
|
|
\f[R]
|
|
.fi
|
|
.SS File Buffering
|
|
.PP
|
|
The \f[C]\-\-buffer\-size\f[R] flag determines the amount of memory,
|
|
that will be used to buffer data in advance.
|
|
.PP
|
|
Each open file descriptor will try to keep the specified amount of data
|
|
in memory at all times.
|
|
The buffered data is bound to one file descriptor and won\[cq]t be
|
|
shared between multiple open file descriptors of the same file.
|
|
.PP
|
|
This flag is a upper limit for the used memory per file descriptor.
|
|
The buffer will only use memory for data that is downloaded but not not
|
|
yet read.
|
|
If the buffer is empty, only a small amount of memory will be used.
|
|
The maximum memory used by rclone for buffering can be up to
|
|
\f[C]\-\-buffer\-size * open files\f[R].
|
|
.SS File Caching
|
|
.PP
|
|
These flags control the VFS file caching options.
|
|
The VFS layer is used by rclone mount to make a cloud storage system
|
|
work more like a normal file system.
|
|
.PP
|
|
You\[cq]ll need to enable VFS caching if you want, for example, to read
|
|
and write simultaneously to a file.
|
|
See below for more details.
|
|
.PP
|
|
Note that the VFS cache works in addition to the cache backend and you
|
|
may find that you need one or the other or both.
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
\-\-cache\-dir string Directory rclone will use for caching.
|
|
\-\-vfs\-cache\-max\-age duration Max age of objects in the cache. (default 1h0m0s)
|
|
\-\-vfs\-cache\-mode string Cache mode off|minimal|writes|full (default \[dq]off\[dq])
|
|
\-\-vfs\-cache\-poll\-interval duration Interval to poll the cache for stale objects. (default 1m0s)
|
|
\-\-vfs\-cache\-max\-size int Max total size of objects in the cache. (default off)
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
If run with \f[C]\-vv\f[R] rclone will print the location of the file
|
|
cache.
|
|
The files are stored in the user cache file area which is OS dependent
|
|
but can be controlled with \f[C]\-\-cache\-dir\f[R] or setting the
|
|
appropriate environment variable.
|
|
.PP
|
|
The cache has 4 different modes selected by
|
|
\f[C]\-\-vfs\-cache\-mode\f[R].
|
|
The higher the cache mode the more compatible rclone becomes at the cost
|
|
of using disk space.
|
|
.PP
|
|
Note that files are written back to the remote only when they are closed
|
|
so if rclone is quit or dies with open files then these won\[cq]t get
|
|
written back to the remote.
|
|
However they will still be in the on disk cache.
|
|
.PP
|
|
If using \[en]vfs\-cache\-max\-size note that the cache may exceed this
|
|
size for two reasons.
|
|
Firstly because it is only checked every
|
|
\[en]vfs\-cache\-poll\-interval.
|
|
Secondly because open files cannot be evicted from the cache.
|
|
.SS \[en]vfs\-cache\-mode off
|
|
.PP
|
|
In this mode the cache will read directly from the remote and write
|
|
directly to the remote without caching anything on disk.
|
|
.PP
|
|
This will mean some operations are not possible
|
|
.IP \[bu] 2
|
|
Files can\[cq]t be opened for both read AND write
|
|
.IP \[bu] 2
|
|
Files opened for write can\[cq]t be seeked
|
|
.IP \[bu] 2
|
|
Existing files opened for write must have O_TRUNC set
|
|
.IP \[bu] 2
|
|
Files open for read with O_TRUNC will be opened write only
|
|
.IP \[bu] 2
|
|
Files open for write only will behave as if O_TRUNC was supplied
|
|
.IP \[bu] 2
|
|
Open modes O_APPEND, O_TRUNC are ignored
|
|
.IP \[bu] 2
|
|
If an upload fails it can\[cq]t be retried
|
|
.SS \[en]vfs\-cache\-mode minimal
|
|
.PP
|
|
This is very similar to \[lq]off\[rq] except that files opened for read
|
|
AND write will be buffered to disks.
|
|
This means that files opened for write will be a lot more compatible,
|
|
but uses the minimal disk space.
|
|
.PP
|
|
These operations are not possible
|
|
.IP \[bu] 2
|
|
Files opened for write only can\[cq]t be seeked
|
|
.IP \[bu] 2
|
|
Existing files opened for write must have O_TRUNC set
|
|
.IP \[bu] 2
|
|
Files opened for write only will ignore O_APPEND, O_TRUNC
|
|
.IP \[bu] 2
|
|
If an upload fails it can\[cq]t be retried
|
|
.SS \[en]vfs\-cache\-mode writes
|
|
.PP
|
|
In this mode files opened for read only are still read directly from the
|
|
remote, write only and read/write files are buffered to disk first.
|
|
.PP
|
|
This mode should support all normal file system operations.
|
|
.PP
|
|
If an upload fails it will be retried up to \[en]low\-level\-retries
|
|
times.
|
|
.SS \[en]vfs\-cache\-mode full
|
|
.PP
|
|
In this mode all reads and writes are buffered to and from disk.
|
|
When a file is opened for read it will be downloaded in its entirety
|
|
first.
|
|
.PP
|
|
This may be appropriate for your needs, or you may prefer to look at the
|
|
cache backend which does a much more sophisticated job of caching,
|
|
including caching directory hierarchies and chunks of files.
|
|
.PP
|
|
In this mode, unlike the others, when a file is written to the disk, it
|
|
will be kept on the disk after it is written to the remote.
|
|
It will be purged on a schedule according to
|
|
\f[C]\-\-vfs\-cache\-max\-age\f[R].
|
|
.PP
|
|
This mode should support all normal file system operations.
|
|
.PP
|
|
If an upload or download fails it will be retried up to
|
|
\[en]low\-level\-retries times.
|
|
.SS Case Sensitivity
|
|
.PP
|
|
Linux file systems are case\-sensitive: two files can differ only by
|
|
case, and the exact case must be used when opening a file.
|
|
.PP
|
|
Windows is not like most other operating systems supported by rclone.
|
|
File systems in modern Windows are case\-insensitive but
|
|
case\-preserving: although existing files can be opened using any case,
|
|
the exact case used to create the file is preserved and available for
|
|
programs to query.
|
|
It is not allowed for two files in the same directory to differ only by
|
|
case.
|
|
.PP
|
|
Usually file systems on macOS are case\-insensitive.
|
|
It is possible to make macOS file systems case\-sensitive but that is
|
|
not the default
|
|
.PP
|
|
The \[lq]\[en]vfs\-case\-insensitive\[rq] mount flag controls how rclone
|
|
handles these two cases.
|
|
If its value is \[lq]false\[rq], rclone passes file names to the mounted
|
|
file system as is.
|
|
If the flag is \[lq]true\[rq] (or appears without a value on command
|
|
line), rclone may perform a \[lq]fixup\[rq] as explained below.
|
|
.PP
|
|
The user may specify a file name to open/delete/rename/etc with a case
|
|
different than what is stored on mounted file system.
|
|
If an argument refers to an existing file with exactly the same name,
|
|
then the case of the existing file on the disk will be used.
|
|
However, if a file name with exactly the same name is not found but a
|
|
name differing only by case exists, rclone will transparently fixup the
|
|
name.
|
|
This fixup happens only when an existing file is requested.
|
|
Case sensitivity of file names created anew by rclone is controlled by
|
|
an underlying mounted file system.
|
|
.PP
|
|
Note that case sensitivity of the operating system running rclone (the
|
|
target) may differ from case sensitivity of a file system mounted by
|
|
rclone (the source).
|
|
The flag controls whether \[lq]fixup\[rq] is performed to satisfy the
|
|
target.
|
|
.PP
|
|
If the flag is not provided on command line, then its default value
|
|
depends on the operating system where rclone runs: \[lq]true\[rq] on
|
|
Windows and macOS, \[lq]false\[rq] otherwise.
|
|
If the flag is provided without a value, then it is \[lq]true\[rq].
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone mount remote:path /path/to/mountpoint [flags]
|
|
\f[R]
|
|
.fi
|
|
.SS Options
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
\-\-allow\-non\-empty Allow mounting over a non\-empty directory (not Windows).
|
|
\-\-allow\-other Allow access to other users.
|
|
\-\-allow\-root Allow access to root user.
|
|
\-\-async\-read Use asynchronous reads. (default true)
|
|
\-\-attr\-timeout duration Time for which file/directory attributes are cached. (default 1s)
|
|
\-\-daemon Run mount as a daemon (background mode).
|
|
\-\-daemon\-timeout duration Time limit for rclone to respond to kernel (not supported by all OSes).
|
|
\-\-debug\-fuse Debug the FUSE internals \- needs \-v.
|
|
\-\-default\-permissions Makes kernel enforce access control based on the file mode.
|
|
\-\-dir\-cache\-time duration Time to cache directory entries for. (default 5m0s)
|
|
\-\-dir\-perms FileMode Directory permissions (default 0777)
|
|
\-\-file\-perms FileMode File permissions (default 0666)
|
|
\-\-fuse\-flag stringArray Flags or arguments to be passed direct to libfuse/WinFsp. Repeat if required.
|
|
\-\-gid uint32 Override the gid field set by the filesystem. (default 1000)
|
|
\-h, \-\-help help for mount
|
|
\-\-max\-read\-ahead SizeSuffix The number of bytes that can be prefetched for sequential reads. (default 128k)
|
|
\-\-no\-checksum Don\[aq]t compare checksums on up/download.
|
|
\-\-no\-modtime Don\[aq]t read/write the modification time (can speed things up).
|
|
\-\-no\-seek Don\[aq]t allow seeking in files.
|
|
\-o, \-\-option stringArray Option for libfuse/WinFsp. Repeat if required.
|
|
\-\-poll\-interval duration Time to wait between polling for changes. Must be smaller than dir\-cache\-time. Only on supported remotes. Set to 0 to disable. (default 1m0s)
|
|
\-\-read\-only Mount read\-only.
|
|
\-\-uid uint32 Override the uid field set by the filesystem. (default 1000)
|
|
\-\-umask int Override the permission bits set by the filesystem.
|
|
\-\-vfs\-cache\-max\-age duration Max age of objects in the cache. (default 1h0m0s)
|
|
\-\-vfs\-cache\-max\-size SizeSuffix Max total size of objects in the cache. (default off)
|
|
\-\-vfs\-cache\-mode CacheMode Cache mode off|minimal|writes|full (default off)
|
|
\-\-vfs\-cache\-poll\-interval duration Interval to poll the cache for stale objects. (default 1m0s)
|
|
\-\-vfs\-case\-insensitive If a file name not found, find a case insensitive match.
|
|
\-\-vfs\-read\-chunk\-size SizeSuffix Read the source objects in chunks. (default 128M)
|
|
\-\-vfs\-read\-chunk\-size\-limit SizeSuffix If greater than \-\-vfs\-read\-chunk\-size, double the chunk size after each chunk read, until the limit is reached. \[aq]off\[aq] is unlimited. (default off)
|
|
\-\-vfs\-read\-wait duration Time to wait for in\-sequence read before seeking. (default 20ms)
|
|
\-\-vfs\-write\-wait duration Time to wait for in\-sequence write before giving error. (default 1s)
|
|
\-\-volname string Set the volume name (not supported by all OSes).
|
|
\-\-write\-back\-cache Makes kernel buffer writes before sending them to rclone. Without this, writethrough caching is used.
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
See the global flags page (https://rclone.org/flags/) for global options
|
|
not listed here.
|
|
.SS SEE ALSO
|
|
.IP \[bu] 2
|
|
rclone (https://rclone.org/commands/rclone/) \- Show help for rclone
|
|
commands, flags and backends.
|
|
.SH rclone moveto
|
|
.PP
|
|
Move file or directory from source to dest.
|
|
.SS Synopsis
|
|
.PP
|
|
If source:path is a file or directory then it moves it to a file or
|
|
directory named dest:path.
|
|
.PP
|
|
This can be used to rename files or upload single files to other than
|
|
their existing name.
|
|
If the source is a directory then it acts exactly like the move command.
|
|
.PP
|
|
So
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone moveto src dst
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
where src and dst are rclone paths, either remote:path or /path/to/local
|
|
or C:.
|
|
.PP
|
|
This will:
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
if src is file
|
|
move it to dst, overwriting an existing file if it exists
|
|
if src is directory
|
|
move it to dst, overwriting existing files if they exist
|
|
see move command for full details
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
This doesn\[cq]t transfer unchanged files, testing by size and
|
|
modification time or MD5SUM.
|
|
src will be deleted on successful transfer.
|
|
.PP
|
|
\f[B]Important\f[R]: Since this can cause data loss, test first with the
|
|
\[en]dry\-run flag.
|
|
.PP
|
|
\f[B]Note\f[R]: Use the \f[C]\-P\f[R]/\f[C]\-\-progress\f[R] flag to
|
|
view real\-time transfer statistics.
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone moveto source:path dest:path [flags]
|
|
\f[R]
|
|
.fi
|
|
.SS Options
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
\-h, \-\-help help for moveto
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
See the global flags page (https://rclone.org/flags/) for global options
|
|
not listed here.
|
|
.SS SEE ALSO
|
|
.IP \[bu] 2
|
|
rclone (https://rclone.org/commands/rclone/) \- Show help for rclone
|
|
commands, flags and backends.
|
|
.SH rclone ncdu
|
|
.PP
|
|
Explore a remote with a text based user interface.
|
|
.SS Synopsis
|
|
.PP
|
|
This displays a text based user interface allowing the navigation of a
|
|
remote.
|
|
It is most useful for answering the question \- \[lq]What is using all
|
|
my disk space?\[rq].
|
|
.PP
|
|
To make the user interface it first scans the entire remote given and
|
|
builds an in memory representation.
|
|
rclone ncdu can be used during this scanning phase and you will see it
|
|
building up the directory structure as it goes along.
|
|
.PP
|
|
Here are the keys \- press `?' to toggle the help on and off
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
\[ua],\[da] or k,j to Move
|
|
\[->],l to enter
|
|
\[<-],h to return
|
|
c toggle counts
|
|
g toggle graph
|
|
n,s,C sort by name,size,count
|
|
d delete file/directory
|
|
y copy current path to clipbard
|
|
Y display current path
|
|
\[ha]L refresh screen
|
|
? to toggle help on and off
|
|
q/ESC/c\-C to quit
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
This an homage to the ncdu tool (https://dev.yorhel.nl/ncdu) but for
|
|
rclone remotes.
|
|
It is missing lots of features at the moment but is useful as it stands.
|
|
.PP
|
|
Note that it might take some time to delete big files/folders.
|
|
The UI won\[cq]t respond in the meantime since the deletion is done
|
|
synchronously.
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone ncdu remote:path [flags]
|
|
\f[R]
|
|
.fi
|
|
.SS Options
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
\-h, \-\-help help for ncdu
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
See the global flags page (https://rclone.org/flags/) for global options
|
|
not listed here.
|
|
.SS SEE ALSO
|
|
.IP \[bu] 2
|
|
rclone (https://rclone.org/commands/rclone/) \- Show help for rclone
|
|
commands, flags and backends.
|
|
.SH rclone obscure
|
|
.PP
|
|
Obscure password for use in the rclone.conf
|
|
.SS Synopsis
|
|
.PP
|
|
Obscure password for use in the rclone.conf
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone obscure password [flags]
|
|
\f[R]
|
|
.fi
|
|
.SS Options
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
\-h, \-\-help help for obscure
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
See the global flags page (https://rclone.org/flags/) for global options
|
|
not listed here.
|
|
.SS SEE ALSO
|
|
.IP \[bu] 2
|
|
rclone (https://rclone.org/commands/rclone/) \- Show help for rclone
|
|
commands, flags and backends.
|
|
.SH rclone rc
|
|
.PP
|
|
Run a command against a running rclone.
|
|
.SS Synopsis
|
|
.PP
|
|
This runs a command against a running rclone.
|
|
Use the \[en]url flag to specify an non default URL to connect on.
|
|
This can be either a \[lq]:port\[rq] which is taken to mean
|
|
\[lq]http://localhost:port\[rq] or a \[lq]host:port\[rq] which is taken
|
|
to mean \[lq]http://host:port\[rq]
|
|
.PP
|
|
A username and password can be passed in with \[en]user and \[en]pass.
|
|
.PP
|
|
Note that \[en]rc\-addr, \[en]rc\-user, \[en]rc\-pass will be read also
|
|
for \[en]url, \[en]user, \[en]pass.
|
|
.PP
|
|
Arguments should be passed in as parameter=value.
|
|
.PP
|
|
The result will be returned as a JSON object by default.
|
|
.PP
|
|
The \[en]json parameter can be used to pass in a JSON blob as an input
|
|
instead of key=value arguments.
|
|
This is the only way of passing in more complicated values.
|
|
.PP
|
|
The \-o/\[en]opt option can be used to set a key \[lq]opt\[rq] with key,
|
|
value options in the form \[lq]\-o key=value\[rq] or \[lq]\-o key\[rq].
|
|
It can be repeated as many times as required.
|
|
This is useful for rc commands which take the \[lq]opt\[rq] parameter
|
|
which by convention is a dictionary of strings.
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
\-o key=value \-o key2
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
Will place this in the \[lq]opt\[rq] value
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
{\[dq]key\[dq]:\[dq]value\[dq], \[dq]key2\[dq],\[dq]\[dq])
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
The \-a/\[en]arg option can be used to set strings in the \[lq]arg\[rq]
|
|
value.
|
|
It can be repeated as many times as required.
|
|
This is useful for rc commands which take the \[lq]arg\[rq] parameter
|
|
which by convention is a list of strings.
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
\-a value \-a value2
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
Will place this in the \[lq]arg\[rq] value
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
[\[dq]value\[dq], \[dq]value2\[dq]]
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
Use \[en]loopback to connect to the rclone instance running \[lq]rclone
|
|
rc\[rq].
|
|
This is very useful for testing commands without having to run an rclone
|
|
rc server, eg:
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone rc \-\-loopback operations/about fs=/
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
Use \[lq]rclone rc\[rq] to see a list of all possible commands.
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone rc commands parameter [flags]
|
|
\f[R]
|
|
.fi
|
|
.SS Options
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
\-a, \-\-arg stringArray Argument placed in the \[dq]arg\[dq] array.
|
|
\-h, \-\-help help for rc
|
|
\-\-json string Input JSON \- use instead of key=value args.
|
|
\-\-loopback If set connect to this rclone instance not via HTTP.
|
|
\-\-no\-output If set don\[aq]t output the JSON result.
|
|
\-o, \-\-opt stringArray Option in the form name=value or name placed in the \[dq]opt\[dq] array.
|
|
\-\-pass string Password to use to connect to rclone remote control.
|
|
\-\-url string URL to connect to rclone remote control. (default \[dq]http://localhost:5572/\[dq])
|
|
\-\-user string Username to use to rclone remote control.
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
See the global flags page (https://rclone.org/flags/) for global options
|
|
not listed here.
|
|
.SS SEE ALSO
|
|
.IP \[bu] 2
|
|
rclone (https://rclone.org/commands/rclone/) \- Show help for rclone
|
|
commands, flags and backends.
|
|
.SH rclone rcat
|
|
.PP
|
|
Copies standard input to file on remote.
|
|
.SS Synopsis
|
|
.PP
|
|
rclone rcat reads from standard input (stdin) and copies it to a single
|
|
remote file.
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
echo \[dq]hello world\[dq] | rclone rcat remote:path/to/file
|
|
ffmpeg \- | rclone rcat remote:path/to/file
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
If the remote file already exists, it will be overwritten.
|
|
.PP
|
|
rcat will try to upload small files in a single request, which is
|
|
usually more efficient than the streaming/chunked upload endpoints,
|
|
which use multiple requests.
|
|
Exact behaviour depends on the remote.
|
|
What is considered a small file may be set through
|
|
\f[C]\-\-streaming\-upload\-cutoff\f[R].
|
|
Uploading only starts after the cutoff is reached or if the file ends
|
|
before that.
|
|
The data must fit into RAM.
|
|
The cutoff needs to be small enough to adhere the limits of your remote,
|
|
please see there.
|
|
Generally speaking, setting this cutoff too high will decrease your
|
|
performance.
|
|
.PP
|
|
Note that the upload can also not be retried because the data is not
|
|
kept around until the upload succeeds.
|
|
If you need to transfer a lot of data, you\[cq]re better off caching
|
|
locally and then \f[C]rclone move\f[R] it to the destination.
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone rcat remote:path [flags]
|
|
\f[R]
|
|
.fi
|
|
.SS Options
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
\-h, \-\-help help for rcat
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
See the global flags page (https://rclone.org/flags/) for global options
|
|
not listed here.
|
|
.SS SEE ALSO
|
|
.IP \[bu] 2
|
|
rclone (https://rclone.org/commands/rclone/) \- Show help for rclone
|
|
commands, flags and backends.
|
|
.SH rclone rcd
|
|
.PP
|
|
Run rclone listening to remote control commands only.
|
|
.SS Synopsis
|
|
.PP
|
|
This runs rclone so that it only listens to remote control commands.
|
|
.PP
|
|
This is useful if you are controlling rclone via the rc API.
|
|
.PP
|
|
If you pass in a path to a directory, rclone will serve that directory
|
|
for GET requests on the URL passed in.
|
|
It will also open the URL in the browser when rclone is run.
|
|
.PP
|
|
See the rc documentation (https://rclone.org/rc/) for more info on the
|
|
rc flags.
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone rcd <path to files to serve>* [flags]
|
|
\f[R]
|
|
.fi
|
|
.SS Options
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
\-h, \-\-help help for rcd
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
See the global flags page (https://rclone.org/flags/) for global options
|
|
not listed here.
|
|
.SS SEE ALSO
|
|
.IP \[bu] 2
|
|
rclone (https://rclone.org/commands/rclone/) \- Show help for rclone
|
|
commands, flags and backends.
|
|
.SH rclone rmdirs
|
|
.PP
|
|
Remove empty directories under the path.
|
|
.SS Synopsis
|
|
.PP
|
|
This removes any empty directories (or directories that only contain
|
|
empty directories) under the path that it finds, including the path if
|
|
it has nothing in.
|
|
.PP
|
|
If you supply the \[en]leave\-root flag, it will not remove the root
|
|
directory.
|
|
.PP
|
|
This is useful for tidying up remotes that rclone has left a lot of
|
|
empty directories in.
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone rmdirs remote:path [flags]
|
|
\f[R]
|
|
.fi
|
|
.SS Options
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
\-h, \-\-help help for rmdirs
|
|
\-\-leave\-root Do not remove root directory if empty
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
See the global flags page (https://rclone.org/flags/) for global options
|
|
not listed here.
|
|
.SS SEE ALSO
|
|
.IP \[bu] 2
|
|
rclone (https://rclone.org/commands/rclone/) \- Show help for rclone
|
|
commands, flags and backends.
|
|
.SH rclone serve
|
|
.PP
|
|
Serve a remote over a protocol.
|
|
.SS Synopsis
|
|
.PP
|
|
rclone serve is used to serve a remote over a given protocol.
|
|
This command requires the use of a subcommand to specify the protocol,
|
|
eg
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone serve http remote:
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
Each subcommand has its own options which you can see in their help.
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone serve <protocol> [opts] <remote> [flags]
|
|
\f[R]
|
|
.fi
|
|
.SS Options
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
\-h, \-\-help help for serve
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
See the global flags page (https://rclone.org/flags/) for global options
|
|
not listed here.
|
|
.SS SEE ALSO
|
|
.IP \[bu] 2
|
|
rclone (https://rclone.org/commands/rclone/) \- Show help for rclone
|
|
commands, flags and backends.
|
|
.IP \[bu] 2
|
|
rclone serve dlna (https://rclone.org/commands/rclone_serve_dlna/) \-
|
|
Serve remote:path over DLNA
|
|
.IP \[bu] 2
|
|
rclone serve ftp (https://rclone.org/commands/rclone_serve_ftp/) \-
|
|
Serve remote:path over FTP.
|
|
.IP \[bu] 2
|
|
rclone serve http (https://rclone.org/commands/rclone_serve_http/) \-
|
|
Serve the remote over HTTP.
|
|
.IP \[bu] 2
|
|
rclone serve restic (https://rclone.org/commands/rclone_serve_restic/)
|
|
\- Serve the remote for restic\[cq]s REST API.
|
|
.IP \[bu] 2
|
|
rclone serve sftp (https://rclone.org/commands/rclone_serve_sftp/) \-
|
|
Serve the remote over SFTP.
|
|
.IP \[bu] 2
|
|
rclone serve webdav (https://rclone.org/commands/rclone_serve_webdav/)
|
|
\- Serve remote:path over webdav.
|
|
.SH rclone serve dlna
|
|
.PP
|
|
Serve remote:path over DLNA
|
|
.SS Synopsis
|
|
.PP
|
|
rclone serve dlna is a DLNA media server for media stored in an rclone
|
|
remote.
|
|
Many devices, such as the Xbox and PlayStation, can automatically
|
|
discover this server in the LAN and play audio/video from it.
|
|
VLC is also supported.
|
|
Service discovery uses UDP multicast packets (SSDP) and will thus only
|
|
work on LANs.
|
|
.PP
|
|
Rclone will list all files present in the remote, without filtering
|
|
based on media formats or file extensions.
|
|
Additionally, there is no media transcoding support.
|
|
This means that some players might show files that they are not able to
|
|
play back correctly.
|
|
.SS Server options
|
|
.PP
|
|
Use \[en]addr to specify which IP address and port the server should
|
|
listen on, eg \[en]addr 1.2.3.4:8000 or \[en]addr :8080 to listen to all
|
|
IPs.
|
|
.PP
|
|
Use \[en]name to choose the friendly server name, which is by default
|
|
\[lq]rclone (hostname)\[rq].
|
|
.PP
|
|
Use \[en]log\-trace in conjunction with \-vv to enable additional debug
|
|
logging of all UPNP traffic.
|
|
.SS Directory Cache
|
|
.PP
|
|
Using the \f[C]\-\-dir\-cache\-time\f[R] flag, you can set how long a
|
|
directory should be considered up to date and not refreshed from the
|
|
backend.
|
|
Changes made locally in the mount may appear immediately or invalidate
|
|
the cache.
|
|
However, changes done on the remote will only be picked up once the
|
|
cache expires if the backend configured does not support polling for
|
|
changes.
|
|
If the backend supports polling, changes will be picked up on within the
|
|
polling interval.
|
|
.PP
|
|
Alternatively, you can send a \f[C]SIGHUP\f[R] signal to rclone for it
|
|
to flush all directory caches, regardless of how old they are.
|
|
Assuming only one rclone instance is running, you can reset the cache
|
|
like this:
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
kill \-SIGHUP $(pidof rclone)
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
If you configure rclone with a remote control (/rc) then you can use
|
|
rclone rc to flush the whole directory cache:
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone rc vfs/forget
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
Or individual files or directories:
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone rc vfs/forget file=path/to/file dir=path/to/dir
|
|
\f[R]
|
|
.fi
|
|
.SS File Buffering
|
|
.PP
|
|
The \f[C]\-\-buffer\-size\f[R] flag determines the amount of memory,
|
|
that will be used to buffer data in advance.
|
|
.PP
|
|
Each open file descriptor will try to keep the specified amount of data
|
|
in memory at all times.
|
|
The buffered data is bound to one file descriptor and won\[cq]t be
|
|
shared between multiple open file descriptors of the same file.
|
|
.PP
|
|
This flag is a upper limit for the used memory per file descriptor.
|
|
The buffer will only use memory for data that is downloaded but not not
|
|
yet read.
|
|
If the buffer is empty, only a small amount of memory will be used.
|
|
The maximum memory used by rclone for buffering can be up to
|
|
\f[C]\-\-buffer\-size * open files\f[R].
|
|
.SS File Caching
|
|
.PP
|
|
These flags control the VFS file caching options.
|
|
The VFS layer is used by rclone mount to make a cloud storage system
|
|
work more like a normal file system.
|
|
.PP
|
|
You\[cq]ll need to enable VFS caching if you want, for example, to read
|
|
and write simultaneously to a file.
|
|
See below for more details.
|
|
.PP
|
|
Note that the VFS cache works in addition to the cache backend and you
|
|
may find that you need one or the other or both.
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
\-\-cache\-dir string Directory rclone will use for caching.
|
|
\-\-vfs\-cache\-max\-age duration Max age of objects in the cache. (default 1h0m0s)
|
|
\-\-vfs\-cache\-mode string Cache mode off|minimal|writes|full (default \[dq]off\[dq])
|
|
\-\-vfs\-cache\-poll\-interval duration Interval to poll the cache for stale objects. (default 1m0s)
|
|
\-\-vfs\-cache\-max\-size int Max total size of objects in the cache. (default off)
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
If run with \f[C]\-vv\f[R] rclone will print the location of the file
|
|
cache.
|
|
The files are stored in the user cache file area which is OS dependent
|
|
but can be controlled with \f[C]\-\-cache\-dir\f[R] or setting the
|
|
appropriate environment variable.
|
|
.PP
|
|
The cache has 4 different modes selected by
|
|
\f[C]\-\-vfs\-cache\-mode\f[R].
|
|
The higher the cache mode the more compatible rclone becomes at the cost
|
|
of using disk space.
|
|
.PP
|
|
Note that files are written back to the remote only when they are closed
|
|
so if rclone is quit or dies with open files then these won\[cq]t get
|
|
written back to the remote.
|
|
However they will still be in the on disk cache.
|
|
.PP
|
|
If using \[en]vfs\-cache\-max\-size note that the cache may exceed this
|
|
size for two reasons.
|
|
Firstly because it is only checked every
|
|
\[en]vfs\-cache\-poll\-interval.
|
|
Secondly because open files cannot be evicted from the cache.
|
|
.SS \[en]vfs\-cache\-mode off
|
|
.PP
|
|
In this mode the cache will read directly from the remote and write
|
|
directly to the remote without caching anything on disk.
|
|
.PP
|
|
This will mean some operations are not possible
|
|
.IP \[bu] 2
|
|
Files can\[cq]t be opened for both read AND write
|
|
.IP \[bu] 2
|
|
Files opened for write can\[cq]t be seeked
|
|
.IP \[bu] 2
|
|
Existing files opened for write must have O_TRUNC set
|
|
.IP \[bu] 2
|
|
Files open for read with O_TRUNC will be opened write only
|
|
.IP \[bu] 2
|
|
Files open for write only will behave as if O_TRUNC was supplied
|
|
.IP \[bu] 2
|
|
Open modes O_APPEND, O_TRUNC are ignored
|
|
.IP \[bu] 2
|
|
If an upload fails it can\[cq]t be retried
|
|
.SS \[en]vfs\-cache\-mode minimal
|
|
.PP
|
|
This is very similar to \[lq]off\[rq] except that files opened for read
|
|
AND write will be buffered to disks.
|
|
This means that files opened for write will be a lot more compatible,
|
|
but uses the minimal disk space.
|
|
.PP
|
|
These operations are not possible
|
|
.IP \[bu] 2
|
|
Files opened for write only can\[cq]t be seeked
|
|
.IP \[bu] 2
|
|
Existing files opened for write must have O_TRUNC set
|
|
.IP \[bu] 2
|
|
Files opened for write only will ignore O_APPEND, O_TRUNC
|
|
.IP \[bu] 2
|
|
If an upload fails it can\[cq]t be retried
|
|
.SS \[en]vfs\-cache\-mode writes
|
|
.PP
|
|
In this mode files opened for read only are still read directly from the
|
|
remote, write only and read/write files are buffered to disk first.
|
|
.PP
|
|
This mode should support all normal file system operations.
|
|
.PP
|
|
If an upload fails it will be retried up to \[en]low\-level\-retries
|
|
times.
|
|
.SS \[en]vfs\-cache\-mode full
|
|
.PP
|
|
In this mode all reads and writes are buffered to and from disk.
|
|
When a file is opened for read it will be downloaded in its entirety
|
|
first.
|
|
.PP
|
|
This may be appropriate for your needs, or you may prefer to look at the
|
|
cache backend which does a much more sophisticated job of caching,
|
|
including caching directory hierarchies and chunks of files.
|
|
.PP
|
|
In this mode, unlike the others, when a file is written to the disk, it
|
|
will be kept on the disk after it is written to the remote.
|
|
It will be purged on a schedule according to
|
|
\f[C]\-\-vfs\-cache\-max\-age\f[R].
|
|
.PP
|
|
This mode should support all normal file system operations.
|
|
.PP
|
|
If an upload or download fails it will be retried up to
|
|
\[en]low\-level\-retries times.
|
|
.SS Case Sensitivity
|
|
.PP
|
|
Linux file systems are case\-sensitive: two files can differ only by
|
|
case, and the exact case must be used when opening a file.
|
|
.PP
|
|
Windows is not like most other operating systems supported by rclone.
|
|
File systems in modern Windows are case\-insensitive but
|
|
case\-preserving: although existing files can be opened using any case,
|
|
the exact case used to create the file is preserved and available for
|
|
programs to query.
|
|
It is not allowed for two files in the same directory to differ only by
|
|
case.
|
|
.PP
|
|
Usually file systems on macOS are case\-insensitive.
|
|
It is possible to make macOS file systems case\-sensitive but that is
|
|
not the default
|
|
.PP
|
|
The \[lq]\[en]vfs\-case\-insensitive\[rq] mount flag controls how rclone
|
|
handles these two cases.
|
|
If its value is \[lq]false\[rq], rclone passes file names to the mounted
|
|
file system as is.
|
|
If the flag is \[lq]true\[rq] (or appears without a value on command
|
|
line), rclone may perform a \[lq]fixup\[rq] as explained below.
|
|
.PP
|
|
The user may specify a file name to open/delete/rename/etc with a case
|
|
different than what is stored on mounted file system.
|
|
If an argument refers to an existing file with exactly the same name,
|
|
then the case of the existing file on the disk will be used.
|
|
However, if a file name with exactly the same name is not found but a
|
|
name differing only by case exists, rclone will transparently fixup the
|
|
name.
|
|
This fixup happens only when an existing file is requested.
|
|
Case sensitivity of file names created anew by rclone is controlled by
|
|
an underlying mounted file system.
|
|
.PP
|
|
Note that case sensitivity of the operating system running rclone (the
|
|
target) may differ from case sensitivity of a file system mounted by
|
|
rclone (the source).
|
|
The flag controls whether \[lq]fixup\[rq] is performed to satisfy the
|
|
target.
|
|
.PP
|
|
If the flag is not provided on command line, then its default value
|
|
depends on the operating system where rclone runs: \[lq]true\[rq] on
|
|
Windows and macOS, \[lq]false\[rq] otherwise.
|
|
If the flag is provided without a value, then it is \[lq]true\[rq].
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone serve dlna remote:path [flags]
|
|
\f[R]
|
|
.fi
|
|
.SS Options
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
\-\-addr string ip:port or :port to bind the DLNA http server to. (default \[dq]:7879\[dq])
|
|
\-\-dir\-cache\-time duration Time to cache directory entries for. (default 5m0s)
|
|
\-\-dir\-perms FileMode Directory permissions (default 0777)
|
|
\-\-file\-perms FileMode File permissions (default 0666)
|
|
\-\-gid uint32 Override the gid field set by the filesystem. (default 1000)
|
|
\-h, \-\-help help for dlna
|
|
\-\-log\-trace enable trace logging of SOAP traffic
|
|
\-\-name string name of DLNA server
|
|
\-\-no\-checksum Don\[aq]t compare checksums on up/download.
|
|
\-\-no\-modtime Don\[aq]t read/write the modification time (can speed things up).
|
|
\-\-no\-seek Don\[aq]t allow seeking in files.
|
|
\-\-poll\-interval duration Time to wait between polling for changes. Must be smaller than dir\-cache\-time. Only on supported remotes. Set to 0 to disable. (default 1m0s)
|
|
\-\-read\-only Mount read\-only.
|
|
\-\-uid uint32 Override the uid field set by the filesystem. (default 1000)
|
|
\-\-umask int Override the permission bits set by the filesystem. (default 2)
|
|
\-\-vfs\-cache\-max\-age duration Max age of objects in the cache. (default 1h0m0s)
|
|
\-\-vfs\-cache\-max\-size SizeSuffix Max total size of objects in the cache. (default off)
|
|
\-\-vfs\-cache\-mode CacheMode Cache mode off|minimal|writes|full (default off)
|
|
\-\-vfs\-cache\-poll\-interval duration Interval to poll the cache for stale objects. (default 1m0s)
|
|
\-\-vfs\-case\-insensitive If a file name not found, find a case insensitive match.
|
|
\-\-vfs\-read\-chunk\-size SizeSuffix Read the source objects in chunks. (default 128M)
|
|
\-\-vfs\-read\-chunk\-size\-limit SizeSuffix If greater than \-\-vfs\-read\-chunk\-size, double the chunk size after each chunk read, until the limit is reached. \[aq]off\[aq] is unlimited. (default off)
|
|
\-\-vfs\-read\-wait duration Time to wait for in\-sequence read before seeking. (default 20ms)
|
|
\-\-vfs\-write\-wait duration Time to wait for in\-sequence write before giving error. (default 1s)
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
See the global flags page (https://rclone.org/flags/) for global options
|
|
not listed here.
|
|
.SS SEE ALSO
|
|
.IP \[bu] 2
|
|
rclone serve (https://rclone.org/commands/rclone_serve/) \- Serve a
|
|
remote over a protocol.
|
|
.SH rclone serve ftp
|
|
.PP
|
|
Serve remote:path over FTP.
|
|
.SS Synopsis
|
|
.PP
|
|
rclone serve ftp implements a basic ftp server to serve the remote over
|
|
FTP protocol.
|
|
This can be viewed with a ftp client or you can make a remote of type
|
|
ftp to read and write it.
|
|
.SS Server options
|
|
.PP
|
|
Use \[en]addr to specify which IP address and port the server should
|
|
listen on, eg \[en]addr 1.2.3.4:8000 or \[en]addr :8080 to listen to all
|
|
IPs.
|
|
By default it only listens on localhost.
|
|
You can use port :0 to let the OS choose an available port.
|
|
.PP
|
|
If you set \[en]addr to listen on a public or LAN accessible IP address
|
|
then using Authentication is advised \- see the next section for info.
|
|
.SS Authentication
|
|
.PP
|
|
By default this will serve files without needing a login.
|
|
.PP
|
|
You can set a single username and password with the \[en]user and
|
|
\[en]pass flags.
|
|
.SS Directory Cache
|
|
.PP
|
|
Using the \f[C]\-\-dir\-cache\-time\f[R] flag, you can set how long a
|
|
directory should be considered up to date and not refreshed from the
|
|
backend.
|
|
Changes made locally in the mount may appear immediately or invalidate
|
|
the cache.
|
|
However, changes done on the remote will only be picked up once the
|
|
cache expires if the backend configured does not support polling for
|
|
changes.
|
|
If the backend supports polling, changes will be picked up on within the
|
|
polling interval.
|
|
.PP
|
|
Alternatively, you can send a \f[C]SIGHUP\f[R] signal to rclone for it
|
|
to flush all directory caches, regardless of how old they are.
|
|
Assuming only one rclone instance is running, you can reset the cache
|
|
like this:
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
kill \-SIGHUP $(pidof rclone)
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
If you configure rclone with a remote control (/rc) then you can use
|
|
rclone rc to flush the whole directory cache:
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone rc vfs/forget
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
Or individual files or directories:
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone rc vfs/forget file=path/to/file dir=path/to/dir
|
|
\f[R]
|
|
.fi
|
|
.SS File Buffering
|
|
.PP
|
|
The \f[C]\-\-buffer\-size\f[R] flag determines the amount of memory,
|
|
that will be used to buffer data in advance.
|
|
.PP
|
|
Each open file descriptor will try to keep the specified amount of data
|
|
in memory at all times.
|
|
The buffered data is bound to one file descriptor and won\[cq]t be
|
|
shared between multiple open file descriptors of the same file.
|
|
.PP
|
|
This flag is a upper limit for the used memory per file descriptor.
|
|
The buffer will only use memory for data that is downloaded but not not
|
|
yet read.
|
|
If the buffer is empty, only a small amount of memory will be used.
|
|
The maximum memory used by rclone for buffering can be up to
|
|
\f[C]\-\-buffer\-size * open files\f[R].
|
|
.SS File Caching
|
|
.PP
|
|
These flags control the VFS file caching options.
|
|
The VFS layer is used by rclone mount to make a cloud storage system
|
|
work more like a normal file system.
|
|
.PP
|
|
You\[cq]ll need to enable VFS caching if you want, for example, to read
|
|
and write simultaneously to a file.
|
|
See below for more details.
|
|
.PP
|
|
Note that the VFS cache works in addition to the cache backend and you
|
|
may find that you need one or the other or both.
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
\-\-cache\-dir string Directory rclone will use for caching.
|
|
\-\-vfs\-cache\-max\-age duration Max age of objects in the cache. (default 1h0m0s)
|
|
\-\-vfs\-cache\-mode string Cache mode off|minimal|writes|full (default \[dq]off\[dq])
|
|
\-\-vfs\-cache\-poll\-interval duration Interval to poll the cache for stale objects. (default 1m0s)
|
|
\-\-vfs\-cache\-max\-size int Max total size of objects in the cache. (default off)
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
If run with \f[C]\-vv\f[R] rclone will print the location of the file
|
|
cache.
|
|
The files are stored in the user cache file area which is OS dependent
|
|
but can be controlled with \f[C]\-\-cache\-dir\f[R] or setting the
|
|
appropriate environment variable.
|
|
.PP
|
|
The cache has 4 different modes selected by
|
|
\f[C]\-\-vfs\-cache\-mode\f[R].
|
|
The higher the cache mode the more compatible rclone becomes at the cost
|
|
of using disk space.
|
|
.PP
|
|
Note that files are written back to the remote only when they are closed
|
|
so if rclone is quit or dies with open files then these won\[cq]t get
|
|
written back to the remote.
|
|
However they will still be in the on disk cache.
|
|
.PP
|
|
If using \[en]vfs\-cache\-max\-size note that the cache may exceed this
|
|
size for two reasons.
|
|
Firstly because it is only checked every
|
|
\[en]vfs\-cache\-poll\-interval.
|
|
Secondly because open files cannot be evicted from the cache.
|
|
.SS \[en]vfs\-cache\-mode off
|
|
.PP
|
|
In this mode the cache will read directly from the remote and write
|
|
directly to the remote without caching anything on disk.
|
|
.PP
|
|
This will mean some operations are not possible
|
|
.IP \[bu] 2
|
|
Files can\[cq]t be opened for both read AND write
|
|
.IP \[bu] 2
|
|
Files opened for write can\[cq]t be seeked
|
|
.IP \[bu] 2
|
|
Existing files opened for write must have O_TRUNC set
|
|
.IP \[bu] 2
|
|
Files open for read with O_TRUNC will be opened write only
|
|
.IP \[bu] 2
|
|
Files open for write only will behave as if O_TRUNC was supplied
|
|
.IP \[bu] 2
|
|
Open modes O_APPEND, O_TRUNC are ignored
|
|
.IP \[bu] 2
|
|
If an upload fails it can\[cq]t be retried
|
|
.SS \[en]vfs\-cache\-mode minimal
|
|
.PP
|
|
This is very similar to \[lq]off\[rq] except that files opened for read
|
|
AND write will be buffered to disks.
|
|
This means that files opened for write will be a lot more compatible,
|
|
but uses the minimal disk space.
|
|
.PP
|
|
These operations are not possible
|
|
.IP \[bu] 2
|
|
Files opened for write only can\[cq]t be seeked
|
|
.IP \[bu] 2
|
|
Existing files opened for write must have O_TRUNC set
|
|
.IP \[bu] 2
|
|
Files opened for write only will ignore O_APPEND, O_TRUNC
|
|
.IP \[bu] 2
|
|
If an upload fails it can\[cq]t be retried
|
|
.SS \[en]vfs\-cache\-mode writes
|
|
.PP
|
|
In this mode files opened for read only are still read directly from the
|
|
remote, write only and read/write files are buffered to disk first.
|
|
.PP
|
|
This mode should support all normal file system operations.
|
|
.PP
|
|
If an upload fails it will be retried up to \[en]low\-level\-retries
|
|
times.
|
|
.SS \[en]vfs\-cache\-mode full
|
|
.PP
|
|
In this mode all reads and writes are buffered to and from disk.
|
|
When a file is opened for read it will be downloaded in its entirety
|
|
first.
|
|
.PP
|
|
This may be appropriate for your needs, or you may prefer to look at the
|
|
cache backend which does a much more sophisticated job of caching,
|
|
including caching directory hierarchies and chunks of files.
|
|
.PP
|
|
In this mode, unlike the others, when a file is written to the disk, it
|
|
will be kept on the disk after it is written to the remote.
|
|
It will be purged on a schedule according to
|
|
\f[C]\-\-vfs\-cache\-max\-age\f[R].
|
|
.PP
|
|
This mode should support all normal file system operations.
|
|
.PP
|
|
If an upload or download fails it will be retried up to
|
|
\[en]low\-level\-retries times.
|
|
.SS Case Sensitivity
|
|
.PP
|
|
Linux file systems are case\-sensitive: two files can differ only by
|
|
case, and the exact case must be used when opening a file.
|
|
.PP
|
|
Windows is not like most other operating systems supported by rclone.
|
|
File systems in modern Windows are case\-insensitive but
|
|
case\-preserving: although existing files can be opened using any case,
|
|
the exact case used to create the file is preserved and available for
|
|
programs to query.
|
|
It is not allowed for two files in the same directory to differ only by
|
|
case.
|
|
.PP
|
|
Usually file systems on macOS are case\-insensitive.
|
|
It is possible to make macOS file systems case\-sensitive but that is
|
|
not the default
|
|
.PP
|
|
The \[lq]\[en]vfs\-case\-insensitive\[rq] mount flag controls how rclone
|
|
handles these two cases.
|
|
If its value is \[lq]false\[rq], rclone passes file names to the mounted
|
|
file system as is.
|
|
If the flag is \[lq]true\[rq] (or appears without a value on command
|
|
line), rclone may perform a \[lq]fixup\[rq] as explained below.
|
|
.PP
|
|
The user may specify a file name to open/delete/rename/etc with a case
|
|
different than what is stored on mounted file system.
|
|
If an argument refers to an existing file with exactly the same name,
|
|
then the case of the existing file on the disk will be used.
|
|
However, if a file name with exactly the same name is not found but a
|
|
name differing only by case exists, rclone will transparently fixup the
|
|
name.
|
|
This fixup happens only when an existing file is requested.
|
|
Case sensitivity of file names created anew by rclone is controlled by
|
|
an underlying mounted file system.
|
|
.PP
|
|
Note that case sensitivity of the operating system running rclone (the
|
|
target) may differ from case sensitivity of a file system mounted by
|
|
rclone (the source).
|
|
The flag controls whether \[lq]fixup\[rq] is performed to satisfy the
|
|
target.
|
|
.PP
|
|
If the flag is not provided on command line, then its default value
|
|
depends on the operating system where rclone runs: \[lq]true\[rq] on
|
|
Windows and macOS, \[lq]false\[rq] otherwise.
|
|
If the flag is provided without a value, then it is \[lq]true\[rq].
|
|
.SS Auth Proxy
|
|
.PP
|
|
If you supply the parameter \f[C]\-\-auth\-proxy /path/to/program\f[R]
|
|
then rclone will use that program to generate backends on the fly which
|
|
then are used to authenticate incoming requests.
|
|
This uses a simple JSON based protocl with input on STDIN and output on
|
|
STDOUT.
|
|
.PP
|
|
\f[B]PLEASE NOTE:\f[R] \f[C]\-\-auth\-proxy\f[R] and
|
|
\f[C]\-\-authorized\-keys\f[R] cannot be used together, if
|
|
\f[C]\-\-auth\-proxy\f[R] is set the authorized keys option will be
|
|
ignored.
|
|
.PP
|
|
There is an example program
|
|
bin/test_proxy.py (https://github.com/rclone/rclone/blob/master/test_proxy.py)
|
|
in the rclone source code.
|
|
.PP
|
|
The program\[cq]s job is to take a \f[C]user\f[R] and \f[C]pass\f[R] on
|
|
the input and turn those into the config for a backend on STDOUT in JSON
|
|
format.
|
|
This config will have any default parameters for the backend added, but
|
|
it won\[cq]t use configuration from environment variables or command
|
|
line options \- it is the job of the proxy program to make a complete
|
|
config.
|
|
.PP
|
|
This config generated must have this extra parameter \- \f[C]_root\f[R]
|
|
\- root to use for the backend
|
|
.PP
|
|
And it may have this parameter \- \f[C]_obscure\f[R] \- comma separated
|
|
strings for parameters to obscure
|
|
.PP
|
|
If password authentication was used by the client, input to the proxy
|
|
process (on STDIN) would look similar to this:
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
{
|
|
\[dq]user\[dq]: \[dq]me\[dq],
|
|
\[dq]pass\[dq]: \[dq]mypassword\[dq]
|
|
}
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
If public\-key authentication was used by the client, input to the proxy
|
|
process (on STDIN) would look similar to this:
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
{
|
|
\[dq]user\[dq]: \[dq]me\[dq],
|
|
\[dq]public_key\[dq]: \[dq]AAAAB3NzaC1yc2EAAAADAQABAAABAQDuwESFdAe14hVS6omeyX7edc...JQdf\[dq]
|
|
}
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
And as an example return this on STDOUT
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
{
|
|
\[dq]type\[dq]: \[dq]sftp\[dq],
|
|
\[dq]_root\[dq]: \[dq]\[dq],
|
|
\[dq]_obscure\[dq]: \[dq]pass\[dq],
|
|
\[dq]user\[dq]: \[dq]me\[dq],
|
|
\[dq]pass\[dq]: \[dq]mypassword\[dq],
|
|
\[dq]host\[dq]: \[dq]sftp.example.com\[dq]
|
|
}
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
This would mean that an SFTP backend would be created on the fly for the
|
|
\f[C]user\f[R] and \f[C]pass\f[R]/\f[C]public_key\f[R] returned in the
|
|
output to the host given.
|
|
Note that since \f[C]_obscure\f[R] is set to \f[C]pass\f[R], rclone will
|
|
obscure the \f[C]pass\f[R] parameter before creating the backend (which
|
|
is required for sftp backends).
|
|
.PP
|
|
The program can manipulate the supplied \f[C]user\f[R] in any way, for
|
|
example to make proxy to many different sftp backends, you could make
|
|
the \f[C]user\f[R] be \f[C]user\[at]example.com\f[R] and then set the
|
|
\f[C]host\f[R] to \f[C]example.com\f[R] in the output and the user to
|
|
\f[C]user\f[R].
|
|
For security you\[cq]d probably want to restrict the \f[C]host\f[R] to a
|
|
limited list.
|
|
.PP
|
|
Note that an internal cache is keyed on \f[C]user\f[R] so only use that
|
|
for configuration, don\[cq]t use \f[C]pass\f[R] or \f[C]public_key\f[R].
|
|
This also means that if a user\[cq]s password or public\-key is changed
|
|
the cache will need to expire (which takes 5 mins) before it takes
|
|
effect.
|
|
.PP
|
|
This can be used to build general purpose proxies to any kind of backend
|
|
that rclone supports.
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone serve ftp remote:path [flags]
|
|
\f[R]
|
|
.fi
|
|
.SS Options
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
\-\-addr string IPaddress:Port or :Port to bind server to. (default \[dq]localhost:2121\[dq])
|
|
\-\-auth\-proxy string A program to use to create the backend from the auth.
|
|
\-\-dir\-cache\-time duration Time to cache directory entries for. (default 5m0s)
|
|
\-\-dir\-perms FileMode Directory permissions (default 0777)
|
|
\-\-file\-perms FileMode File permissions (default 0666)
|
|
\-\-gid uint32 Override the gid field set by the filesystem. (default 1000)
|
|
\-h, \-\-help help for ftp
|
|
\-\-no\-checksum Don\[aq]t compare checksums on up/download.
|
|
\-\-no\-modtime Don\[aq]t read/write the modification time (can speed things up).
|
|
\-\-no\-seek Don\[aq]t allow seeking in files.
|
|
\-\-pass string Password for authentication. (empty value allow every password)
|
|
\-\-passive\-port string Passive port range to use. (default \[dq]30000\-32000\[dq])
|
|
\-\-poll\-interval duration Time to wait between polling for changes. Must be smaller than dir\-cache\-time. Only on supported remotes. Set to 0 to disable. (default 1m0s)
|
|
\-\-public\-ip string Public IP address to advertise for passive connections.
|
|
\-\-read\-only Mount read\-only.
|
|
\-\-uid uint32 Override the uid field set by the filesystem. (default 1000)
|
|
\-\-umask int Override the permission bits set by the filesystem. (default 2)
|
|
\-\-user string User name for authentication. (default \[dq]anonymous\[dq])
|
|
\-\-vfs\-cache\-max\-age duration Max age of objects in the cache. (default 1h0m0s)
|
|
\-\-vfs\-cache\-max\-size SizeSuffix Max total size of objects in the cache. (default off)
|
|
\-\-vfs\-cache\-mode CacheMode Cache mode off|minimal|writes|full (default off)
|
|
\-\-vfs\-cache\-poll\-interval duration Interval to poll the cache for stale objects. (default 1m0s)
|
|
\-\-vfs\-case\-insensitive If a file name not found, find a case insensitive match.
|
|
\-\-vfs\-read\-chunk\-size SizeSuffix Read the source objects in chunks. (default 128M)
|
|
\-\-vfs\-read\-chunk\-size\-limit SizeSuffix If greater than \-\-vfs\-read\-chunk\-size, double the chunk size after each chunk read, until the limit is reached. \[aq]off\[aq] is unlimited. (default off)
|
|
\-\-vfs\-read\-wait duration Time to wait for in\-sequence read before seeking. (default 20ms)
|
|
\-\-vfs\-write\-wait duration Time to wait for in\-sequence write before giving error. (default 1s)
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
See the global flags page (https://rclone.org/flags/) for global options
|
|
not listed here.
|
|
.SS SEE ALSO
|
|
.IP \[bu] 2
|
|
rclone serve (https://rclone.org/commands/rclone_serve/) \- Serve a
|
|
remote over a protocol.
|
|
.SH rclone serve http
|
|
.PP
|
|
Serve the remote over HTTP.
|
|
.SS Synopsis
|
|
.PP
|
|
rclone serve http implements a basic web server to serve the remote over
|
|
HTTP.
|
|
This can be viewed in a web browser or you can make a remote of type
|
|
http read from it.
|
|
.PP
|
|
You can use the filter flags (eg \[en]include, \[en]exclude) to control
|
|
what is served.
|
|
.PP
|
|
The server will log errors.
|
|
Use \-v to see access logs.
|
|
.PP
|
|
\[en]bwlimit will be respected for file transfers.
|
|
Use \[en]stats to control the stats printing.
|
|
.SS Server options
|
|
.PP
|
|
Use \[en]addr to specify which IP address and port the server should
|
|
listen on, eg \[en]addr 1.2.3.4:8000 or \[en]addr :8080 to listen to all
|
|
IPs.
|
|
By default it only listens on localhost.
|
|
You can use port :0 to let the OS choose an available port.
|
|
.PP
|
|
If you set \[en]addr to listen on a public or LAN accessible IP address
|
|
then using Authentication is advised \- see the next section for info.
|
|
.PP
|
|
\[en]server\-read\-timeout and \[en]server\-write\-timeout can be used
|
|
to control the timeouts on the server.
|
|
Note that this is the total time for a transfer.
|
|
.PP
|
|
\[en]max\-header\-bytes controls the maximum number of bytes the server
|
|
will accept in the HTTP header.
|
|
.PP
|
|
\[en]baseurl controls the URL prefix that rclone serves from.
|
|
By default rclone will serve from the root.
|
|
If you used \[en]baseurl \[lq]/rclone\[rq] then rclone would serve from
|
|
a URL starting with \[lq]/rclone/\[rq].
|
|
This is useful if you wish to proxy rclone serve.
|
|
Rclone automatically inserts leading and trailing \[lq]/\[rq] on
|
|
\[en]baseurl, so \[en]baseurl \[lq]rclone\[rq], \[en]baseurl
|
|
\[lq]/rclone\[rq] and \[en]baseurl \[lq]/rclone/\[rq] are all treated
|
|
identically.
|
|
.PP
|
|
\[en]template allows a user to specify a custom markup template for http
|
|
and webdav serve functions.
|
|
The server exports the following markup to be used within the template
|
|
to server pages:
|
|
.PP
|
|
.TS
|
|
tab(@);
|
|
lw(35.0n) lw(35.0n).
|
|
T{
|
|
Parameter
|
|
T}@T{
|
|
Description
|
|
T}
|
|
_
|
|
T{
|
|
\&.Name
|
|
T}@T{
|
|
The full path of a file/directory.
|
|
T}
|
|
T{
|
|
\&.Title
|
|
T}@T{
|
|
Directory listing of .Name
|
|
T}
|
|
T{
|
|
\&.Sort
|
|
T}@T{
|
|
The current sort used.
|
|
This is changeable via ?sort= parameter
|
|
T}
|
|
T{
|
|
T}@T{
|
|
Sort Options: namedirfist,name,size,time (default namedirfirst)
|
|
T}
|
|
T{
|
|
\&.Order
|
|
T}@T{
|
|
The current ordering used.
|
|
This is changeable via ?order= parameter
|
|
T}
|
|
T{
|
|
T}@T{
|
|
Order Options: asc,desc (default asc)
|
|
T}
|
|
T{
|
|
\&.Query
|
|
T}@T{
|
|
Currently unused.
|
|
T}
|
|
T{
|
|
\&.Breadcrumb
|
|
T}@T{
|
|
Allows for creating a relative navigation
|
|
T}
|
|
T{
|
|
\[en] .Link
|
|
T}@T{
|
|
The relative to the root link of the Text.
|
|
T}
|
|
T{
|
|
\[en] .Text
|
|
T}@T{
|
|
The Name of the directory.
|
|
T}
|
|
T{
|
|
\&.Entries
|
|
T}@T{
|
|
Information about a specific file/directory.
|
|
T}
|
|
T{
|
|
\[en] .URL
|
|
T}@T{
|
|
The `url' of an entry.
|
|
T}
|
|
T{
|
|
\[en] .Leaf
|
|
T}@T{
|
|
Currently same as `URL' but intended to be `just' the name.
|
|
T}
|
|
T{
|
|
\[en] .IsDir
|
|
T}@T{
|
|
Boolean for if an entry is a directory or not.
|
|
T}
|
|
T{
|
|
\[en] .Size
|
|
T}@T{
|
|
Size in Bytes of the entry.
|
|
T}
|
|
T{
|
|
\[en] .ModTime
|
|
T}@T{
|
|
The UTC timestamp of an entry.
|
|
T}
|
|
.TE
|
|
.SS Authentication
|
|
.PP
|
|
By default this will serve files without needing a login.
|
|
.PP
|
|
You can either use an htpasswd file which can take lots of users, or set
|
|
a single username and password with the \[en]user and \[en]pass flags.
|
|
.PP
|
|
Use \[en]htpasswd /path/to/htpasswd to provide an htpasswd file.
|
|
This is in standard apache format and supports MD5, SHA1 and BCrypt for
|
|
basic authentication.
|
|
Bcrypt is recommended.
|
|
.PP
|
|
To create an htpasswd file:
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
touch htpasswd
|
|
htpasswd \-B htpasswd user
|
|
htpasswd \-B htpasswd anotherUser
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
The password file can be updated while rclone is running.
|
|
.PP
|
|
Use \[en]realm to set the authentication realm.
|
|
.SS SSL/TLS
|
|
.PP
|
|
By default this will serve over http.
|
|
If you want you can serve over https.
|
|
You will need to supply the \[en]cert and \[en]key flags.
|
|
If you wish to do client side certificate validation then you will need
|
|
to supply \[en]client\-ca also.
|
|
.PP
|
|
\[en]cert should be either a PEM encoded certificate or a concatenation
|
|
of that with the CA certificate.
|
|
\[en]key should be the PEM encoded private key and \[en]client\-ca
|
|
should be the PEM encoded client certificate authority certificate.
|
|
.SS Directory Cache
|
|
.PP
|
|
Using the \f[C]\-\-dir\-cache\-time\f[R] flag, you can set how long a
|
|
directory should be considered up to date and not refreshed from the
|
|
backend.
|
|
Changes made locally in the mount may appear immediately or invalidate
|
|
the cache.
|
|
However, changes done on the remote will only be picked up once the
|
|
cache expires if the backend configured does not support polling for
|
|
changes.
|
|
If the backend supports polling, changes will be picked up on within the
|
|
polling interval.
|
|
.PP
|
|
Alternatively, you can send a \f[C]SIGHUP\f[R] signal to rclone for it
|
|
to flush all directory caches, regardless of how old they are.
|
|
Assuming only one rclone instance is running, you can reset the cache
|
|
like this:
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
kill \-SIGHUP $(pidof rclone)
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
If you configure rclone with a remote control (/rc) then you can use
|
|
rclone rc to flush the whole directory cache:
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone rc vfs/forget
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
Or individual files or directories:
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone rc vfs/forget file=path/to/file dir=path/to/dir
|
|
\f[R]
|
|
.fi
|
|
.SS File Buffering
|
|
.PP
|
|
The \f[C]\-\-buffer\-size\f[R] flag determines the amount of memory,
|
|
that will be used to buffer data in advance.
|
|
.PP
|
|
Each open file descriptor will try to keep the specified amount of data
|
|
in memory at all times.
|
|
The buffered data is bound to one file descriptor and won\[cq]t be
|
|
shared between multiple open file descriptors of the same file.
|
|
.PP
|
|
This flag is a upper limit for the used memory per file descriptor.
|
|
The buffer will only use memory for data that is downloaded but not not
|
|
yet read.
|
|
If the buffer is empty, only a small amount of memory will be used.
|
|
The maximum memory used by rclone for buffering can be up to
|
|
\f[C]\-\-buffer\-size * open files\f[R].
|
|
.SS File Caching
|
|
.PP
|
|
These flags control the VFS file caching options.
|
|
The VFS layer is used by rclone mount to make a cloud storage system
|
|
work more like a normal file system.
|
|
.PP
|
|
You\[cq]ll need to enable VFS caching if you want, for example, to read
|
|
and write simultaneously to a file.
|
|
See below for more details.
|
|
.PP
|
|
Note that the VFS cache works in addition to the cache backend and you
|
|
may find that you need one or the other or both.
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
\-\-cache\-dir string Directory rclone will use for caching.
|
|
\-\-vfs\-cache\-max\-age duration Max age of objects in the cache. (default 1h0m0s)
|
|
\-\-vfs\-cache\-mode string Cache mode off|minimal|writes|full (default \[dq]off\[dq])
|
|
\-\-vfs\-cache\-poll\-interval duration Interval to poll the cache for stale objects. (default 1m0s)
|
|
\-\-vfs\-cache\-max\-size int Max total size of objects in the cache. (default off)
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
If run with \f[C]\-vv\f[R] rclone will print the location of the file
|
|
cache.
|
|
The files are stored in the user cache file area which is OS dependent
|
|
but can be controlled with \f[C]\-\-cache\-dir\f[R] or setting the
|
|
appropriate environment variable.
|
|
.PP
|
|
The cache has 4 different modes selected by
|
|
\f[C]\-\-vfs\-cache\-mode\f[R].
|
|
The higher the cache mode the more compatible rclone becomes at the cost
|
|
of using disk space.
|
|
.PP
|
|
Note that files are written back to the remote only when they are closed
|
|
so if rclone is quit or dies with open files then these won\[cq]t get
|
|
written back to the remote.
|
|
However they will still be in the on disk cache.
|
|
.PP
|
|
If using \[en]vfs\-cache\-max\-size note that the cache may exceed this
|
|
size for two reasons.
|
|
Firstly because it is only checked every
|
|
\[en]vfs\-cache\-poll\-interval.
|
|
Secondly because open files cannot be evicted from the cache.
|
|
.SS \[en]vfs\-cache\-mode off
|
|
.PP
|
|
In this mode the cache will read directly from the remote and write
|
|
directly to the remote without caching anything on disk.
|
|
.PP
|
|
This will mean some operations are not possible
|
|
.IP \[bu] 2
|
|
Files can\[cq]t be opened for both read AND write
|
|
.IP \[bu] 2
|
|
Files opened for write can\[cq]t be seeked
|
|
.IP \[bu] 2
|
|
Existing files opened for write must have O_TRUNC set
|
|
.IP \[bu] 2
|
|
Files open for read with O_TRUNC will be opened write only
|
|
.IP \[bu] 2
|
|
Files open for write only will behave as if O_TRUNC was supplied
|
|
.IP \[bu] 2
|
|
Open modes O_APPEND, O_TRUNC are ignored
|
|
.IP \[bu] 2
|
|
If an upload fails it can\[cq]t be retried
|
|
.SS \[en]vfs\-cache\-mode minimal
|
|
.PP
|
|
This is very similar to \[lq]off\[rq] except that files opened for read
|
|
AND write will be buffered to disks.
|
|
This means that files opened for write will be a lot more compatible,
|
|
but uses the minimal disk space.
|
|
.PP
|
|
These operations are not possible
|
|
.IP \[bu] 2
|
|
Files opened for write only can\[cq]t be seeked
|
|
.IP \[bu] 2
|
|
Existing files opened for write must have O_TRUNC set
|
|
.IP \[bu] 2
|
|
Files opened for write only will ignore O_APPEND, O_TRUNC
|
|
.IP \[bu] 2
|
|
If an upload fails it can\[cq]t be retried
|
|
.SS \[en]vfs\-cache\-mode writes
|
|
.PP
|
|
In this mode files opened for read only are still read directly from the
|
|
remote, write only and read/write files are buffered to disk first.
|
|
.PP
|
|
This mode should support all normal file system operations.
|
|
.PP
|
|
If an upload fails it will be retried up to \[en]low\-level\-retries
|
|
times.
|
|
.SS \[en]vfs\-cache\-mode full
|
|
.PP
|
|
In this mode all reads and writes are buffered to and from disk.
|
|
When a file is opened for read it will be downloaded in its entirety
|
|
first.
|
|
.PP
|
|
This may be appropriate for your needs, or you may prefer to look at the
|
|
cache backend which does a much more sophisticated job of caching,
|
|
including caching directory hierarchies and chunks of files.
|
|
.PP
|
|
In this mode, unlike the others, when a file is written to the disk, it
|
|
will be kept on the disk after it is written to the remote.
|
|
It will be purged on a schedule according to
|
|
\f[C]\-\-vfs\-cache\-max\-age\f[R].
|
|
.PP
|
|
This mode should support all normal file system operations.
|
|
.PP
|
|
If an upload or download fails it will be retried up to
|
|
\[en]low\-level\-retries times.
|
|
.SS Case Sensitivity
|
|
.PP
|
|
Linux file systems are case\-sensitive: two files can differ only by
|
|
case, and the exact case must be used when opening a file.
|
|
.PP
|
|
Windows is not like most other operating systems supported by rclone.
|
|
File systems in modern Windows are case\-insensitive but
|
|
case\-preserving: although existing files can be opened using any case,
|
|
the exact case used to create the file is preserved and available for
|
|
programs to query.
|
|
It is not allowed for two files in the same directory to differ only by
|
|
case.
|
|
.PP
|
|
Usually file systems on macOS are case\-insensitive.
|
|
It is possible to make macOS file systems case\-sensitive but that is
|
|
not the default
|
|
.PP
|
|
The \[lq]\[en]vfs\-case\-insensitive\[rq] mount flag controls how rclone
|
|
handles these two cases.
|
|
If its value is \[lq]false\[rq], rclone passes file names to the mounted
|
|
file system as is.
|
|
If the flag is \[lq]true\[rq] (or appears without a value on command
|
|
line), rclone may perform a \[lq]fixup\[rq] as explained below.
|
|
.PP
|
|
The user may specify a file name to open/delete/rename/etc with a case
|
|
different than what is stored on mounted file system.
|
|
If an argument refers to an existing file with exactly the same name,
|
|
then the case of the existing file on the disk will be used.
|
|
However, if a file name with exactly the same name is not found but a
|
|
name differing only by case exists, rclone will transparently fixup the
|
|
name.
|
|
This fixup happens only when an existing file is requested.
|
|
Case sensitivity of file names created anew by rclone is controlled by
|
|
an underlying mounted file system.
|
|
.PP
|
|
Note that case sensitivity of the operating system running rclone (the
|
|
target) may differ from case sensitivity of a file system mounted by
|
|
rclone (the source).
|
|
The flag controls whether \[lq]fixup\[rq] is performed to satisfy the
|
|
target.
|
|
.PP
|
|
If the flag is not provided on command line, then its default value
|
|
depends on the operating system where rclone runs: \[lq]true\[rq] on
|
|
Windows and macOS, \[lq]false\[rq] otherwise.
|
|
If the flag is provided without a value, then it is \[lq]true\[rq].
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone serve http remote:path [flags]
|
|
\f[R]
|
|
.fi
|
|
.SS Options
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
\-\-addr string IPaddress:Port or :Port to bind server to. (default \[dq]localhost:8080\[dq])
|
|
\-\-baseurl string Prefix for URLs \- leave blank for root.
|
|
\-\-cert string SSL PEM key (concatenation of certificate and CA certificate)
|
|
\-\-client\-ca string Client certificate authority to verify clients with
|
|
\-\-dir\-cache\-time duration Time to cache directory entries for. (default 5m0s)
|
|
\-\-dir\-perms FileMode Directory permissions (default 0777)
|
|
\-\-file\-perms FileMode File permissions (default 0666)
|
|
\-\-gid uint32 Override the gid field set by the filesystem. (default 1000)
|
|
\-h, \-\-help help for http
|
|
\-\-htpasswd string htpasswd file \- if not provided no authentication is done
|
|
\-\-key string SSL PEM Private key
|
|
\-\-max\-header\-bytes int Maximum size of request header (default 4096)
|
|
\-\-no\-checksum Don\[aq]t compare checksums on up/download.
|
|
\-\-no\-modtime Don\[aq]t read/write the modification time (can speed things up).
|
|
\-\-no\-seek Don\[aq]t allow seeking in files.
|
|
\-\-pass string Password for authentication.
|
|
\-\-poll\-interval duration Time to wait between polling for changes. Must be smaller than dir\-cache\-time. Only on supported remotes. Set to 0 to disable. (default 1m0s)
|
|
\-\-read\-only Mount read\-only.
|
|
\-\-realm string realm for authentication (default \[dq]rclone\[dq])
|
|
\-\-server\-read\-timeout duration Timeout for server reading data (default 1h0m0s)
|
|
\-\-server\-write\-timeout duration Timeout for server writing data (default 1h0m0s)
|
|
\-\-template string User Specified Template.
|
|
\-\-uid uint32 Override the uid field set by the filesystem. (default 1000)
|
|
\-\-umask int Override the permission bits set by the filesystem. (default 2)
|
|
\-\-user string User name for authentication.
|
|
\-\-vfs\-cache\-max\-age duration Max age of objects in the cache. (default 1h0m0s)
|
|
\-\-vfs\-cache\-max\-size SizeSuffix Max total size of objects in the cache. (default off)
|
|
\-\-vfs\-cache\-mode CacheMode Cache mode off|minimal|writes|full (default off)
|
|
\-\-vfs\-cache\-poll\-interval duration Interval to poll the cache for stale objects. (default 1m0s)
|
|
\-\-vfs\-case\-insensitive If a file name not found, find a case insensitive match.
|
|
\-\-vfs\-read\-chunk\-size SizeSuffix Read the source objects in chunks. (default 128M)
|
|
\-\-vfs\-read\-chunk\-size\-limit SizeSuffix If greater than \-\-vfs\-read\-chunk\-size, double the chunk size after each chunk read, until the limit is reached. \[aq]off\[aq] is unlimited. (default off)
|
|
\-\-vfs\-read\-wait duration Time to wait for in\-sequence read before seeking. (default 20ms)
|
|
\-\-vfs\-write\-wait duration Time to wait for in\-sequence write before giving error. (default 1s)
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
See the global flags page (https://rclone.org/flags/) for global options
|
|
not listed here.
|
|
.SS SEE ALSO
|
|
.IP \[bu] 2
|
|
rclone serve (https://rclone.org/commands/rclone_serve/) \- Serve a
|
|
remote over a protocol.
|
|
.SH rclone serve restic
|
|
.PP
|
|
Serve the remote for restic\[cq]s REST API.
|
|
.SS Synopsis
|
|
.PP
|
|
rclone serve restic implements restic\[cq]s REST backend API over HTTP.
|
|
This allows restic to use rclone as a data storage mechanism for cloud
|
|
providers that restic does not support directly.
|
|
.PP
|
|
Restic (https://restic.net/) is a command line program for doing
|
|
backups.
|
|
.PP
|
|
The server will log errors.
|
|
Use \-v to see access logs.
|
|
.PP
|
|
\[en]bwlimit will be respected for file transfers.
|
|
Use \[en]stats to control the stats printing.
|
|
.SS Setting up rclone for use by restic
|
|
.PP
|
|
First set up a remote for your chosen cloud
|
|
provider (https://rclone.org/docs/#configure).
|
|
.PP
|
|
Once you have set up the remote, check it is working with, for example
|
|
\[lq]rclone lsd remote:\[rq].
|
|
You may have called the remote something other than \[lq]remote:\[rq] \-
|
|
just substitute whatever you called it in the following instructions.
|
|
.PP
|
|
Now start the rclone restic server
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone serve restic \-v remote:backup
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
Where you can replace \[lq]backup\[rq] in the above by whatever path in
|
|
the remote you wish to use.
|
|
.PP
|
|
By default this will serve on \[lq]localhost:8080\[rq] you can change
|
|
this with use of the \[lq]\[en]addr\[rq] flag.
|
|
.PP
|
|
You might wish to start this server on boot.
|
|
.SS Setting up restic to use rclone
|
|
.PP
|
|
Now you can follow the restic
|
|
instructions (http://restic.readthedocs.io/en/latest/030_preparing_a_new_repo.html#rest-server)
|
|
on setting up restic.
|
|
.PP
|
|
Note that you will need restic 0.8.2 or later to interoperate with
|
|
rclone.
|
|
.PP
|
|
For the example above you will want to use
|
|
\[lq]http://localhost:8080/\[rq] as the URL for the REST server.
|
|
.PP
|
|
For example:
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
$ export RESTIC_REPOSITORY=rest:http://localhost:8080/
|
|
$ export RESTIC_PASSWORD=yourpassword
|
|
$ restic init
|
|
created restic backend 8b1a4b56ae at rest:http://localhost:8080/
|
|
|
|
Please note that knowledge of your password is required to access
|
|
the repository. Losing your password means that your data is
|
|
irrecoverably lost.
|
|
$ restic backup /path/to/files/to/backup
|
|
scan [/path/to/files/to/backup]
|
|
scanned 189 directories, 312 files in 0:00
|
|
[0:00] 100.00% 38.128 MiB / 38.128 MiB 501 / 501 items 0 errors ETA 0:00
|
|
duration: 0:00
|
|
snapshot 45c8fdd8 saved
|
|
\f[R]
|
|
.fi
|
|
.SS Multiple repositories
|
|
.PP
|
|
Note that you can use the endpoint to host multiple repositories.
|
|
Do this by adding a directory name or path after the URL.
|
|
Note that these \f[B]must\f[R] end with /.
|
|
Eg
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
$ export RESTIC_REPOSITORY=rest:http://localhost:8080/user1repo/
|
|
# backup user1 stuff
|
|
$ export RESTIC_REPOSITORY=rest:http://localhost:8080/user2repo/
|
|
# backup user2 stuff
|
|
\f[R]
|
|
.fi
|
|
.SS Private repositories
|
|
.PP
|
|
The \[lq]\[en]private\-repos\[rq] flag can be used to limit users to
|
|
repositories starting with a path of \f[C]/<username>/\f[R].
|
|
.SS Server options
|
|
.PP
|
|
Use \[en]addr to specify which IP address and port the server should
|
|
listen on, eg \[en]addr 1.2.3.4:8000 or \[en]addr :8080 to listen to all
|
|
IPs.
|
|
By default it only listens on localhost.
|
|
You can use port :0 to let the OS choose an available port.
|
|
.PP
|
|
If you set \[en]addr to listen on a public or LAN accessible IP address
|
|
then using Authentication is advised \- see the next section for info.
|
|
.PP
|
|
\[en]server\-read\-timeout and \[en]server\-write\-timeout can be used
|
|
to control the timeouts on the server.
|
|
Note that this is the total time for a transfer.
|
|
.PP
|
|
\[en]max\-header\-bytes controls the maximum number of bytes the server
|
|
will accept in the HTTP header.
|
|
.PP
|
|
\[en]baseurl controls the URL prefix that rclone serves from.
|
|
By default rclone will serve from the root.
|
|
If you used \[en]baseurl \[lq]/rclone\[rq] then rclone would serve from
|
|
a URL starting with \[lq]/rclone/\[rq].
|
|
This is useful if you wish to proxy rclone serve.
|
|
Rclone automatically inserts leading and trailing \[lq]/\[rq] on
|
|
\[en]baseurl, so \[en]baseurl \[lq]rclone\[rq], \[en]baseurl
|
|
\[lq]/rclone\[rq] and \[en]baseurl \[lq]/rclone/\[rq] are all treated
|
|
identically.
|
|
.PP
|
|
\[en]template allows a user to specify a custom markup template for http
|
|
and webdav serve functions.
|
|
The server exports the following markup to be used within the template
|
|
to server pages:
|
|
.PP
|
|
.TS
|
|
tab(@);
|
|
lw(35.0n) lw(35.0n).
|
|
T{
|
|
Parameter
|
|
T}@T{
|
|
Description
|
|
T}
|
|
_
|
|
T{
|
|
\&.Name
|
|
T}@T{
|
|
The full path of a file/directory.
|
|
T}
|
|
T{
|
|
\&.Title
|
|
T}@T{
|
|
Directory listing of .Name
|
|
T}
|
|
T{
|
|
\&.Sort
|
|
T}@T{
|
|
The current sort used.
|
|
This is changeable via ?sort= parameter
|
|
T}
|
|
T{
|
|
T}@T{
|
|
Sort Options: namedirfist,name,size,time (default namedirfirst)
|
|
T}
|
|
T{
|
|
\&.Order
|
|
T}@T{
|
|
The current ordering used.
|
|
This is changeable via ?order= parameter
|
|
T}
|
|
T{
|
|
T}@T{
|
|
Order Options: asc,desc (default asc)
|
|
T}
|
|
T{
|
|
\&.Query
|
|
T}@T{
|
|
Currently unused.
|
|
T}
|
|
T{
|
|
\&.Breadcrumb
|
|
T}@T{
|
|
Allows for creating a relative navigation
|
|
T}
|
|
T{
|
|
\[en] .Link
|
|
T}@T{
|
|
The relative to the root link of the Text.
|
|
T}
|
|
T{
|
|
\[en] .Text
|
|
T}@T{
|
|
The Name of the directory.
|
|
T}
|
|
T{
|
|
\&.Entries
|
|
T}@T{
|
|
Information about a specific file/directory.
|
|
T}
|
|
T{
|
|
\[en] .URL
|
|
T}@T{
|
|
The `url' of an entry.
|
|
T}
|
|
T{
|
|
\[en] .Leaf
|
|
T}@T{
|
|
Currently same as `URL' but intended to be `just' the name.
|
|
T}
|
|
T{
|
|
\[en] .IsDir
|
|
T}@T{
|
|
Boolean for if an entry is a directory or not.
|
|
T}
|
|
T{
|
|
\[en] .Size
|
|
T}@T{
|
|
Size in Bytes of the entry.
|
|
T}
|
|
T{
|
|
\[en] .ModTime
|
|
T}@T{
|
|
The UTC timestamp of an entry.
|
|
T}
|
|
.TE
|
|
.SS Authentication
|
|
.PP
|
|
By default this will serve files without needing a login.
|
|
.PP
|
|
You can either use an htpasswd file which can take lots of users, or set
|
|
a single username and password with the \[en]user and \[en]pass flags.
|
|
.PP
|
|
Use \[en]htpasswd /path/to/htpasswd to provide an htpasswd file.
|
|
This is in standard apache format and supports MD5, SHA1 and BCrypt for
|
|
basic authentication.
|
|
Bcrypt is recommended.
|
|
.PP
|
|
To create an htpasswd file:
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
touch htpasswd
|
|
htpasswd \-B htpasswd user
|
|
htpasswd \-B htpasswd anotherUser
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
The password file can be updated while rclone is running.
|
|
.PP
|
|
Use \[en]realm to set the authentication realm.
|
|
.SS SSL/TLS
|
|
.PP
|
|
By default this will serve over http.
|
|
If you want you can serve over https.
|
|
You will need to supply the \[en]cert and \[en]key flags.
|
|
If you wish to do client side certificate validation then you will need
|
|
to supply \[en]client\-ca also.
|
|
.PP
|
|
\[en]cert should be either a PEM encoded certificate or a concatenation
|
|
of that with the CA certificate.
|
|
\[en]key should be the PEM encoded private key and \[en]client\-ca
|
|
should be the PEM encoded client certificate authority certificate.
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone serve restic remote:path [flags]
|
|
\f[R]
|
|
.fi
|
|
.SS Options
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
\-\-addr string IPaddress:Port or :Port to bind server to. (default \[dq]localhost:8080\[dq])
|
|
\-\-append\-only disallow deletion of repository data
|
|
\-\-baseurl string Prefix for URLs \- leave blank for root.
|
|
\-\-cert string SSL PEM key (concatenation of certificate and CA certificate)
|
|
\-\-client\-ca string Client certificate authority to verify clients with
|
|
\-h, \-\-help help for restic
|
|
\-\-htpasswd string htpasswd file \- if not provided no authentication is done
|
|
\-\-key string SSL PEM Private key
|
|
\-\-max\-header\-bytes int Maximum size of request header (default 4096)
|
|
\-\-pass string Password for authentication.
|
|
\-\-private\-repos users can only access their private repo
|
|
\-\-realm string realm for authentication (default \[dq]rclone\[dq])
|
|
\-\-server\-read\-timeout duration Timeout for server reading data (default 1h0m0s)
|
|
\-\-server\-write\-timeout duration Timeout for server writing data (default 1h0m0s)
|
|
\-\-stdio run an HTTP2 server on stdin/stdout
|
|
\-\-template string User Specified Template.
|
|
\-\-user string User name for authentication.
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
See the global flags page (https://rclone.org/flags/) for global options
|
|
not listed here.
|
|
.SS SEE ALSO
|
|
.IP \[bu] 2
|
|
rclone serve (https://rclone.org/commands/rclone_serve/) \- Serve a
|
|
remote over a protocol.
|
|
.SH rclone serve sftp
|
|
.PP
|
|
Serve the remote over SFTP.
|
|
.SS Synopsis
|
|
.PP
|
|
rclone serve sftp implements an SFTP server to serve the remote over
|
|
SFTP.
|
|
This can be used with an SFTP client or you can make a remote of type
|
|
sftp to use with it.
|
|
.PP
|
|
You can use the filter flags (eg \[en]include, \[en]exclude) to control
|
|
what is served.
|
|
.PP
|
|
The server will log errors.
|
|
Use \-v to see access logs.
|
|
.PP
|
|
\[en]bwlimit will be respected for file transfers.
|
|
Use \[en]stats to control the stats printing.
|
|
.PP
|
|
You must provide some means of authentication, either with
|
|
\[en]user/\[en]pass, an authorized keys file (specify location with
|
|
\[en]authorized\-keys \- the default is the same as ssh), an
|
|
\[en]auth\-proxy, or set the \[en]no\-auth flag for no authentication
|
|
when logging in.
|
|
.PP
|
|
Note that this also implements a small number of shell commands so that
|
|
it can provide md5sum/sha1sum/df information for the rclone sftp
|
|
backend.
|
|
This means that is can support SHA1SUMs, MD5SUMs and the about command
|
|
when paired with the rclone sftp backend.
|
|
.PP
|
|
If you don\[cq]t supply a \[en]key then rclone will generate one and
|
|
cache it for later use.
|
|
.PP
|
|
By default the server binds to localhost:2022 \- if you want it to be
|
|
reachable externally then supply \[lq]\[en]addr :2022\[rq] for example.
|
|
.PP
|
|
Note that the default of \[lq]\[en]vfs\-cache\-mode off\[rq] is fine for
|
|
the rclone sftp backend, but it may not be with other SFTP clients.
|
|
.SS Directory Cache
|
|
.PP
|
|
Using the \f[C]\-\-dir\-cache\-time\f[R] flag, you can set how long a
|
|
directory should be considered up to date and not refreshed from the
|
|
backend.
|
|
Changes made locally in the mount may appear immediately or invalidate
|
|
the cache.
|
|
However, changes done on the remote will only be picked up once the
|
|
cache expires if the backend configured does not support polling for
|
|
changes.
|
|
If the backend supports polling, changes will be picked up on within the
|
|
polling interval.
|
|
.PP
|
|
Alternatively, you can send a \f[C]SIGHUP\f[R] signal to rclone for it
|
|
to flush all directory caches, regardless of how old they are.
|
|
Assuming only one rclone instance is running, you can reset the cache
|
|
like this:
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
kill \-SIGHUP $(pidof rclone)
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
If you configure rclone with a remote control (/rc) then you can use
|
|
rclone rc to flush the whole directory cache:
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone rc vfs/forget
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
Or individual files or directories:
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone rc vfs/forget file=path/to/file dir=path/to/dir
|
|
\f[R]
|
|
.fi
|
|
.SS File Buffering
|
|
.PP
|
|
The \f[C]\-\-buffer\-size\f[R] flag determines the amount of memory,
|
|
that will be used to buffer data in advance.
|
|
.PP
|
|
Each open file descriptor will try to keep the specified amount of data
|
|
in memory at all times.
|
|
The buffered data is bound to one file descriptor and won\[cq]t be
|
|
shared between multiple open file descriptors of the same file.
|
|
.PP
|
|
This flag is a upper limit for the used memory per file descriptor.
|
|
The buffer will only use memory for data that is downloaded but not not
|
|
yet read.
|
|
If the buffer is empty, only a small amount of memory will be used.
|
|
The maximum memory used by rclone for buffering can be up to
|
|
\f[C]\-\-buffer\-size * open files\f[R].
|
|
.SS File Caching
|
|
.PP
|
|
These flags control the VFS file caching options.
|
|
The VFS layer is used by rclone mount to make a cloud storage system
|
|
work more like a normal file system.
|
|
.PP
|
|
You\[cq]ll need to enable VFS caching if you want, for example, to read
|
|
and write simultaneously to a file.
|
|
See below for more details.
|
|
.PP
|
|
Note that the VFS cache works in addition to the cache backend and you
|
|
may find that you need one or the other or both.
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
\-\-cache\-dir string Directory rclone will use for caching.
|
|
\-\-vfs\-cache\-max\-age duration Max age of objects in the cache. (default 1h0m0s)
|
|
\-\-vfs\-cache\-mode string Cache mode off|minimal|writes|full (default \[dq]off\[dq])
|
|
\-\-vfs\-cache\-poll\-interval duration Interval to poll the cache for stale objects. (default 1m0s)
|
|
\-\-vfs\-cache\-max\-size int Max total size of objects in the cache. (default off)
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
If run with \f[C]\-vv\f[R] rclone will print the location of the file
|
|
cache.
|
|
The files are stored in the user cache file area which is OS dependent
|
|
but can be controlled with \f[C]\-\-cache\-dir\f[R] or setting the
|
|
appropriate environment variable.
|
|
.PP
|
|
The cache has 4 different modes selected by
|
|
\f[C]\-\-vfs\-cache\-mode\f[R].
|
|
The higher the cache mode the more compatible rclone becomes at the cost
|
|
of using disk space.
|
|
.PP
|
|
Note that files are written back to the remote only when they are closed
|
|
so if rclone is quit or dies with open files then these won\[cq]t get
|
|
written back to the remote.
|
|
However they will still be in the on disk cache.
|
|
.PP
|
|
If using \[en]vfs\-cache\-max\-size note that the cache may exceed this
|
|
size for two reasons.
|
|
Firstly because it is only checked every
|
|
\[en]vfs\-cache\-poll\-interval.
|
|
Secondly because open files cannot be evicted from the cache.
|
|
.SS \[en]vfs\-cache\-mode off
|
|
.PP
|
|
In this mode the cache will read directly from the remote and write
|
|
directly to the remote without caching anything on disk.
|
|
.PP
|
|
This will mean some operations are not possible
|
|
.IP \[bu] 2
|
|
Files can\[cq]t be opened for both read AND write
|
|
.IP \[bu] 2
|
|
Files opened for write can\[cq]t be seeked
|
|
.IP \[bu] 2
|
|
Existing files opened for write must have O_TRUNC set
|
|
.IP \[bu] 2
|
|
Files open for read with O_TRUNC will be opened write only
|
|
.IP \[bu] 2
|
|
Files open for write only will behave as if O_TRUNC was supplied
|
|
.IP \[bu] 2
|
|
Open modes O_APPEND, O_TRUNC are ignored
|
|
.IP \[bu] 2
|
|
If an upload fails it can\[cq]t be retried
|
|
.SS \[en]vfs\-cache\-mode minimal
|
|
.PP
|
|
This is very similar to \[lq]off\[rq] except that files opened for read
|
|
AND write will be buffered to disks.
|
|
This means that files opened for write will be a lot more compatible,
|
|
but uses the minimal disk space.
|
|
.PP
|
|
These operations are not possible
|
|
.IP \[bu] 2
|
|
Files opened for write only can\[cq]t be seeked
|
|
.IP \[bu] 2
|
|
Existing files opened for write must have O_TRUNC set
|
|
.IP \[bu] 2
|
|
Files opened for write only will ignore O_APPEND, O_TRUNC
|
|
.IP \[bu] 2
|
|
If an upload fails it can\[cq]t be retried
|
|
.SS \[en]vfs\-cache\-mode writes
|
|
.PP
|
|
In this mode files opened for read only are still read directly from the
|
|
remote, write only and read/write files are buffered to disk first.
|
|
.PP
|
|
This mode should support all normal file system operations.
|
|
.PP
|
|
If an upload fails it will be retried up to \[en]low\-level\-retries
|
|
times.
|
|
.SS \[en]vfs\-cache\-mode full
|
|
.PP
|
|
In this mode all reads and writes are buffered to and from disk.
|
|
When a file is opened for read it will be downloaded in its entirety
|
|
first.
|
|
.PP
|
|
This may be appropriate for your needs, or you may prefer to look at the
|
|
cache backend which does a much more sophisticated job of caching,
|
|
including caching directory hierarchies and chunks of files.
|
|
.PP
|
|
In this mode, unlike the others, when a file is written to the disk, it
|
|
will be kept on the disk after it is written to the remote.
|
|
It will be purged on a schedule according to
|
|
\f[C]\-\-vfs\-cache\-max\-age\f[R].
|
|
.PP
|
|
This mode should support all normal file system operations.
|
|
.PP
|
|
If an upload or download fails it will be retried up to
|
|
\[en]low\-level\-retries times.
|
|
.SS Case Sensitivity
|
|
.PP
|
|
Linux file systems are case\-sensitive: two files can differ only by
|
|
case, and the exact case must be used when opening a file.
|
|
.PP
|
|
Windows is not like most other operating systems supported by rclone.
|
|
File systems in modern Windows are case\-insensitive but
|
|
case\-preserving: although existing files can be opened using any case,
|
|
the exact case used to create the file is preserved and available for
|
|
programs to query.
|
|
It is not allowed for two files in the same directory to differ only by
|
|
case.
|
|
.PP
|
|
Usually file systems on macOS are case\-insensitive.
|
|
It is possible to make macOS file systems case\-sensitive but that is
|
|
not the default
|
|
.PP
|
|
The \[lq]\[en]vfs\-case\-insensitive\[rq] mount flag controls how rclone
|
|
handles these two cases.
|
|
If its value is \[lq]false\[rq], rclone passes file names to the mounted
|
|
file system as is.
|
|
If the flag is \[lq]true\[rq] (or appears without a value on command
|
|
line), rclone may perform a \[lq]fixup\[rq] as explained below.
|
|
.PP
|
|
The user may specify a file name to open/delete/rename/etc with a case
|
|
different than what is stored on mounted file system.
|
|
If an argument refers to an existing file with exactly the same name,
|
|
then the case of the existing file on the disk will be used.
|
|
However, if a file name with exactly the same name is not found but a
|
|
name differing only by case exists, rclone will transparently fixup the
|
|
name.
|
|
This fixup happens only when an existing file is requested.
|
|
Case sensitivity of file names created anew by rclone is controlled by
|
|
an underlying mounted file system.
|
|
.PP
|
|
Note that case sensitivity of the operating system running rclone (the
|
|
target) may differ from case sensitivity of a file system mounted by
|
|
rclone (the source).
|
|
The flag controls whether \[lq]fixup\[rq] is performed to satisfy the
|
|
target.
|
|
.PP
|
|
If the flag is not provided on command line, then its default value
|
|
depends on the operating system where rclone runs: \[lq]true\[rq] on
|
|
Windows and macOS, \[lq]false\[rq] otherwise.
|
|
If the flag is provided without a value, then it is \[lq]true\[rq].
|
|
.SS Auth Proxy
|
|
.PP
|
|
If you supply the parameter \f[C]\-\-auth\-proxy /path/to/program\f[R]
|
|
then rclone will use that program to generate backends on the fly which
|
|
then are used to authenticate incoming requests.
|
|
This uses a simple JSON based protocl with input on STDIN and output on
|
|
STDOUT.
|
|
.PP
|
|
\f[B]PLEASE NOTE:\f[R] \f[C]\-\-auth\-proxy\f[R] and
|
|
\f[C]\-\-authorized\-keys\f[R] cannot be used together, if
|
|
\f[C]\-\-auth\-proxy\f[R] is set the authorized keys option will be
|
|
ignored.
|
|
.PP
|
|
There is an example program
|
|
bin/test_proxy.py (https://github.com/rclone/rclone/blob/master/test_proxy.py)
|
|
in the rclone source code.
|
|
.PP
|
|
The program\[cq]s job is to take a \f[C]user\f[R] and \f[C]pass\f[R] on
|
|
the input and turn those into the config for a backend on STDOUT in JSON
|
|
format.
|
|
This config will have any default parameters for the backend added, but
|
|
it won\[cq]t use configuration from environment variables or command
|
|
line options \- it is the job of the proxy program to make a complete
|
|
config.
|
|
.PP
|
|
This config generated must have this extra parameter \- \f[C]_root\f[R]
|
|
\- root to use for the backend
|
|
.PP
|
|
And it may have this parameter \- \f[C]_obscure\f[R] \- comma separated
|
|
strings for parameters to obscure
|
|
.PP
|
|
If password authentication was used by the client, input to the proxy
|
|
process (on STDIN) would look similar to this:
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
{
|
|
\[dq]user\[dq]: \[dq]me\[dq],
|
|
\[dq]pass\[dq]: \[dq]mypassword\[dq]
|
|
}
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
If public\-key authentication was used by the client, input to the proxy
|
|
process (on STDIN) would look similar to this:
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
{
|
|
\[dq]user\[dq]: \[dq]me\[dq],
|
|
\[dq]public_key\[dq]: \[dq]AAAAB3NzaC1yc2EAAAADAQABAAABAQDuwESFdAe14hVS6omeyX7edc...JQdf\[dq]
|
|
}
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
And as an example return this on STDOUT
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
{
|
|
\[dq]type\[dq]: \[dq]sftp\[dq],
|
|
\[dq]_root\[dq]: \[dq]\[dq],
|
|
\[dq]_obscure\[dq]: \[dq]pass\[dq],
|
|
\[dq]user\[dq]: \[dq]me\[dq],
|
|
\[dq]pass\[dq]: \[dq]mypassword\[dq],
|
|
\[dq]host\[dq]: \[dq]sftp.example.com\[dq]
|
|
}
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
This would mean that an SFTP backend would be created on the fly for the
|
|
\f[C]user\f[R] and \f[C]pass\f[R]/\f[C]public_key\f[R] returned in the
|
|
output to the host given.
|
|
Note that since \f[C]_obscure\f[R] is set to \f[C]pass\f[R], rclone will
|
|
obscure the \f[C]pass\f[R] parameter before creating the backend (which
|
|
is required for sftp backends).
|
|
.PP
|
|
The program can manipulate the supplied \f[C]user\f[R] in any way, for
|
|
example to make proxy to many different sftp backends, you could make
|
|
the \f[C]user\f[R] be \f[C]user\[at]example.com\f[R] and then set the
|
|
\f[C]host\f[R] to \f[C]example.com\f[R] in the output and the user to
|
|
\f[C]user\f[R].
|
|
For security you\[cq]d probably want to restrict the \f[C]host\f[R] to a
|
|
limited list.
|
|
.PP
|
|
Note that an internal cache is keyed on \f[C]user\f[R] so only use that
|
|
for configuration, don\[cq]t use \f[C]pass\f[R] or \f[C]public_key\f[R].
|
|
This also means that if a user\[cq]s password or public\-key is changed
|
|
the cache will need to expire (which takes 5 mins) before it takes
|
|
effect.
|
|
.PP
|
|
This can be used to build general purpose proxies to any kind of backend
|
|
that rclone supports.
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone serve sftp remote:path [flags]
|
|
\f[R]
|
|
.fi
|
|
.SS Options
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
\-\-addr string IPaddress:Port or :Port to bind server to. (default \[dq]localhost:2022\[dq])
|
|
\-\-auth\-proxy string A program to use to create the backend from the auth.
|
|
\-\-authorized\-keys string Authorized keys file (default \[dq]\[ti]/.ssh/authorized_keys\[dq])
|
|
\-\-dir\-cache\-time duration Time to cache directory entries for. (default 5m0s)
|
|
\-\-dir\-perms FileMode Directory permissions (default 0777)
|
|
\-\-file\-perms FileMode File permissions (default 0666)
|
|
\-\-gid uint32 Override the gid field set by the filesystem. (default 1000)
|
|
\-h, \-\-help help for sftp
|
|
\-\-key stringArray SSH private host key file (Can be multi\-valued, leave blank to auto generate)
|
|
\-\-no\-auth Allow connections with no authentication if set.
|
|
\-\-no\-checksum Don\[aq]t compare checksums on up/download.
|
|
\-\-no\-modtime Don\[aq]t read/write the modification time (can speed things up).
|
|
\-\-no\-seek Don\[aq]t allow seeking in files.
|
|
\-\-pass string Password for authentication.
|
|
\-\-poll\-interval duration Time to wait between polling for changes. Must be smaller than dir\-cache\-time. Only on supported remotes. Set to 0 to disable. (default 1m0s)
|
|
\-\-read\-only Mount read\-only.
|
|
\-\-uid uint32 Override the uid field set by the filesystem. (default 1000)
|
|
\-\-umask int Override the permission bits set by the filesystem. (default 2)
|
|
\-\-user string User name for authentication.
|
|
\-\-vfs\-cache\-max\-age duration Max age of objects in the cache. (default 1h0m0s)
|
|
\-\-vfs\-cache\-max\-size SizeSuffix Max total size of objects in the cache. (default off)
|
|
\-\-vfs\-cache\-mode CacheMode Cache mode off|minimal|writes|full (default off)
|
|
\-\-vfs\-cache\-poll\-interval duration Interval to poll the cache for stale objects. (default 1m0s)
|
|
\-\-vfs\-case\-insensitive If a file name not found, find a case insensitive match.
|
|
\-\-vfs\-read\-chunk\-size SizeSuffix Read the source objects in chunks. (default 128M)
|
|
\-\-vfs\-read\-chunk\-size\-limit SizeSuffix If greater than \-\-vfs\-read\-chunk\-size, double the chunk size after each chunk read, until the limit is reached. \[aq]off\[aq] is unlimited. (default off)
|
|
\-\-vfs\-read\-wait duration Time to wait for in\-sequence read before seeking. (default 20ms)
|
|
\-\-vfs\-write\-wait duration Time to wait for in\-sequence write before giving error. (default 1s)
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
See the global flags page (https://rclone.org/flags/) for global options
|
|
not listed here.
|
|
.SS SEE ALSO
|
|
.IP \[bu] 2
|
|
rclone serve (https://rclone.org/commands/rclone_serve/) \- Serve a
|
|
remote over a protocol.
|
|
.SH rclone serve webdav
|
|
.PP
|
|
Serve remote:path over webdav.
|
|
.SS Synopsis
|
|
.PP
|
|
rclone serve webdav implements a basic webdav server to serve the remote
|
|
over HTTP via the webdav protocol.
|
|
This can be viewed with a webdav client, through a web browser, or you
|
|
can make a remote of type webdav to read and write it.
|
|
.SS Webdav options
|
|
.SS \[en]etag\-hash
|
|
.PP
|
|
This controls the ETag header.
|
|
Without this flag the ETag will be based on the ModTime and Size of the
|
|
object.
|
|
.PP
|
|
If this flag is set to \[lq]auto\[rq] then rclone will choose the first
|
|
supported hash on the backend or you can use a named hash such as
|
|
\[lq]MD5\[rq] or \[lq]SHA\-1\[rq].
|
|
.PP
|
|
Use \[lq]rclone hashsum\[rq] to see the full list.
|
|
.SS Server options
|
|
.PP
|
|
Use \[en]addr to specify which IP address and port the server should
|
|
listen on, eg \[en]addr 1.2.3.4:8000 or \[en]addr :8080 to listen to all
|
|
IPs.
|
|
By default it only listens on localhost.
|
|
You can use port :0 to let the OS choose an available port.
|
|
.PP
|
|
If you set \[en]addr to listen on a public or LAN accessible IP address
|
|
then using Authentication is advised \- see the next section for info.
|
|
.PP
|
|
\[en]server\-read\-timeout and \[en]server\-write\-timeout can be used
|
|
to control the timeouts on the server.
|
|
Note that this is the total time for a transfer.
|
|
.PP
|
|
\[en]max\-header\-bytes controls the maximum number of bytes the server
|
|
will accept in the HTTP header.
|
|
.PP
|
|
\[en]baseurl controls the URL prefix that rclone serves from.
|
|
By default rclone will serve from the root.
|
|
If you used \[en]baseurl \[lq]/rclone\[rq] then rclone would serve from
|
|
a URL starting with \[lq]/rclone/\[rq].
|
|
This is useful if you wish to proxy rclone serve.
|
|
Rclone automatically inserts leading and trailing \[lq]/\[rq] on
|
|
\[en]baseurl, so \[en]baseurl \[lq]rclone\[rq], \[en]baseurl
|
|
\[lq]/rclone\[rq] and \[en]baseurl \[lq]/rclone/\[rq] are all treated
|
|
identically.
|
|
.PP
|
|
\[en]template allows a user to specify a custom markup template for http
|
|
and webdav serve functions.
|
|
The server exports the following markup to be used within the template
|
|
to server pages:
|
|
.PP
|
|
.TS
|
|
tab(@);
|
|
lw(35.0n) lw(35.0n).
|
|
T{
|
|
Parameter
|
|
T}@T{
|
|
Description
|
|
T}
|
|
_
|
|
T{
|
|
\&.Name
|
|
T}@T{
|
|
The full path of a file/directory.
|
|
T}
|
|
T{
|
|
\&.Title
|
|
T}@T{
|
|
Directory listing of .Name
|
|
T}
|
|
T{
|
|
\&.Sort
|
|
T}@T{
|
|
The current sort used.
|
|
This is changeable via ?sort= parameter
|
|
T}
|
|
T{
|
|
T}@T{
|
|
Sort Options: namedirfist,name,size,time (default namedirfirst)
|
|
T}
|
|
T{
|
|
\&.Order
|
|
T}@T{
|
|
The current ordering used.
|
|
This is changeable via ?order= parameter
|
|
T}
|
|
T{
|
|
T}@T{
|
|
Order Options: asc,desc (default asc)
|
|
T}
|
|
T{
|
|
\&.Query
|
|
T}@T{
|
|
Currently unused.
|
|
T}
|
|
T{
|
|
\&.Breadcrumb
|
|
T}@T{
|
|
Allows for creating a relative navigation
|
|
T}
|
|
T{
|
|
\[en] .Link
|
|
T}@T{
|
|
The relative to the root link of the Text.
|
|
T}
|
|
T{
|
|
\[en] .Text
|
|
T}@T{
|
|
The Name of the directory.
|
|
T}
|
|
T{
|
|
\&.Entries
|
|
T}@T{
|
|
Information about a specific file/directory.
|
|
T}
|
|
T{
|
|
\[en] .URL
|
|
T}@T{
|
|
The `url' of an entry.
|
|
T}
|
|
T{
|
|
\[en] .Leaf
|
|
T}@T{
|
|
Currently same as `URL' but intended to be `just' the name.
|
|
T}
|
|
T{
|
|
\[en] .IsDir
|
|
T}@T{
|
|
Boolean for if an entry is a directory or not.
|
|
T}
|
|
T{
|
|
\[en] .Size
|
|
T}@T{
|
|
Size in Bytes of the entry.
|
|
T}
|
|
T{
|
|
\[en] .ModTime
|
|
T}@T{
|
|
The UTC timestamp of an entry.
|
|
T}
|
|
.TE
|
|
.SS Authentication
|
|
.PP
|
|
By default this will serve files without needing a login.
|
|
.PP
|
|
You can either use an htpasswd file which can take lots of users, or set
|
|
a single username and password with the \[en]user and \[en]pass flags.
|
|
.PP
|
|
Use \[en]htpasswd /path/to/htpasswd to provide an htpasswd file.
|
|
This is in standard apache format and supports MD5, SHA1 and BCrypt for
|
|
basic authentication.
|
|
Bcrypt is recommended.
|
|
.PP
|
|
To create an htpasswd file:
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
touch htpasswd
|
|
htpasswd \-B htpasswd user
|
|
htpasswd \-B htpasswd anotherUser
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
The password file can be updated while rclone is running.
|
|
.PP
|
|
Use \[en]realm to set the authentication realm.
|
|
.SS SSL/TLS
|
|
.PP
|
|
By default this will serve over http.
|
|
If you want you can serve over https.
|
|
You will need to supply the \[en]cert and \[en]key flags.
|
|
If you wish to do client side certificate validation then you will need
|
|
to supply \[en]client\-ca also.
|
|
.PP
|
|
\[en]cert should be either a PEM encoded certificate or a concatenation
|
|
of that with the CA certificate.
|
|
\[en]key should be the PEM encoded private key and \[en]client\-ca
|
|
should be the PEM encoded client certificate authority certificate.
|
|
.SS Directory Cache
|
|
.PP
|
|
Using the \f[C]\-\-dir\-cache\-time\f[R] flag, you can set how long a
|
|
directory should be considered up to date and not refreshed from the
|
|
backend.
|
|
Changes made locally in the mount may appear immediately or invalidate
|
|
the cache.
|
|
However, changes done on the remote will only be picked up once the
|
|
cache expires if the backend configured does not support polling for
|
|
changes.
|
|
If the backend supports polling, changes will be picked up on within the
|
|
polling interval.
|
|
.PP
|
|
Alternatively, you can send a \f[C]SIGHUP\f[R] signal to rclone for it
|
|
to flush all directory caches, regardless of how old they are.
|
|
Assuming only one rclone instance is running, you can reset the cache
|
|
like this:
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
kill \-SIGHUP $(pidof rclone)
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
If you configure rclone with a remote control (/rc) then you can use
|
|
rclone rc to flush the whole directory cache:
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone rc vfs/forget
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
Or individual files or directories:
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone rc vfs/forget file=path/to/file dir=path/to/dir
|
|
\f[R]
|
|
.fi
|
|
.SS File Buffering
|
|
.PP
|
|
The \f[C]\-\-buffer\-size\f[R] flag determines the amount of memory,
|
|
that will be used to buffer data in advance.
|
|
.PP
|
|
Each open file descriptor will try to keep the specified amount of data
|
|
in memory at all times.
|
|
The buffered data is bound to one file descriptor and won\[cq]t be
|
|
shared between multiple open file descriptors of the same file.
|
|
.PP
|
|
This flag is a upper limit for the used memory per file descriptor.
|
|
The buffer will only use memory for data that is downloaded but not not
|
|
yet read.
|
|
If the buffer is empty, only a small amount of memory will be used.
|
|
The maximum memory used by rclone for buffering can be up to
|
|
\f[C]\-\-buffer\-size * open files\f[R].
|
|
.SS File Caching
|
|
.PP
|
|
These flags control the VFS file caching options.
|
|
The VFS layer is used by rclone mount to make a cloud storage system
|
|
work more like a normal file system.
|
|
.PP
|
|
You\[cq]ll need to enable VFS caching if you want, for example, to read
|
|
and write simultaneously to a file.
|
|
See below for more details.
|
|
.PP
|
|
Note that the VFS cache works in addition to the cache backend and you
|
|
may find that you need one or the other or both.
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
\-\-cache\-dir string Directory rclone will use for caching.
|
|
\-\-vfs\-cache\-max\-age duration Max age of objects in the cache. (default 1h0m0s)
|
|
\-\-vfs\-cache\-mode string Cache mode off|minimal|writes|full (default \[dq]off\[dq])
|
|
\-\-vfs\-cache\-poll\-interval duration Interval to poll the cache for stale objects. (default 1m0s)
|
|
\-\-vfs\-cache\-max\-size int Max total size of objects in the cache. (default off)
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
If run with \f[C]\-vv\f[R] rclone will print the location of the file
|
|
cache.
|
|
The files are stored in the user cache file area which is OS dependent
|
|
but can be controlled with \f[C]\-\-cache\-dir\f[R] or setting the
|
|
appropriate environment variable.
|
|
.PP
|
|
The cache has 4 different modes selected by
|
|
\f[C]\-\-vfs\-cache\-mode\f[R].
|
|
The higher the cache mode the more compatible rclone becomes at the cost
|
|
of using disk space.
|
|
.PP
|
|
Note that files are written back to the remote only when they are closed
|
|
so if rclone is quit or dies with open files then these won\[cq]t get
|
|
written back to the remote.
|
|
However they will still be in the on disk cache.
|
|
.PP
|
|
If using \[en]vfs\-cache\-max\-size note that the cache may exceed this
|
|
size for two reasons.
|
|
Firstly because it is only checked every
|
|
\[en]vfs\-cache\-poll\-interval.
|
|
Secondly because open files cannot be evicted from the cache.
|
|
.SS \[en]vfs\-cache\-mode off
|
|
.PP
|
|
In this mode the cache will read directly from the remote and write
|
|
directly to the remote without caching anything on disk.
|
|
.PP
|
|
This will mean some operations are not possible
|
|
.IP \[bu] 2
|
|
Files can\[cq]t be opened for both read AND write
|
|
.IP \[bu] 2
|
|
Files opened for write can\[cq]t be seeked
|
|
.IP \[bu] 2
|
|
Existing files opened for write must have O_TRUNC set
|
|
.IP \[bu] 2
|
|
Files open for read with O_TRUNC will be opened write only
|
|
.IP \[bu] 2
|
|
Files open for write only will behave as if O_TRUNC was supplied
|
|
.IP \[bu] 2
|
|
Open modes O_APPEND, O_TRUNC are ignored
|
|
.IP \[bu] 2
|
|
If an upload fails it can\[cq]t be retried
|
|
.SS \[en]vfs\-cache\-mode minimal
|
|
.PP
|
|
This is very similar to \[lq]off\[rq] except that files opened for read
|
|
AND write will be buffered to disks.
|
|
This means that files opened for write will be a lot more compatible,
|
|
but uses the minimal disk space.
|
|
.PP
|
|
These operations are not possible
|
|
.IP \[bu] 2
|
|
Files opened for write only can\[cq]t be seeked
|
|
.IP \[bu] 2
|
|
Existing files opened for write must have O_TRUNC set
|
|
.IP \[bu] 2
|
|
Files opened for write only will ignore O_APPEND, O_TRUNC
|
|
.IP \[bu] 2
|
|
If an upload fails it can\[cq]t be retried
|
|
.SS \[en]vfs\-cache\-mode writes
|
|
.PP
|
|
In this mode files opened for read only are still read directly from the
|
|
remote, write only and read/write files are buffered to disk first.
|
|
.PP
|
|
This mode should support all normal file system operations.
|
|
.PP
|
|
If an upload fails it will be retried up to \[en]low\-level\-retries
|
|
times.
|
|
.SS \[en]vfs\-cache\-mode full
|
|
.PP
|
|
In this mode all reads and writes are buffered to and from disk.
|
|
When a file is opened for read it will be downloaded in its entirety
|
|
first.
|
|
.PP
|
|
This may be appropriate for your needs, or you may prefer to look at the
|
|
cache backend which does a much more sophisticated job of caching,
|
|
including caching directory hierarchies and chunks of files.
|
|
.PP
|
|
In this mode, unlike the others, when a file is written to the disk, it
|
|
will be kept on the disk after it is written to the remote.
|
|
It will be purged on a schedule according to
|
|
\f[C]\-\-vfs\-cache\-max\-age\f[R].
|
|
.PP
|
|
This mode should support all normal file system operations.
|
|
.PP
|
|
If an upload or download fails it will be retried up to
|
|
\[en]low\-level\-retries times.
|
|
.SS Case Sensitivity
|
|
.PP
|
|
Linux file systems are case\-sensitive: two files can differ only by
|
|
case, and the exact case must be used when opening a file.
|
|
.PP
|
|
Windows is not like most other operating systems supported by rclone.
|
|
File systems in modern Windows are case\-insensitive but
|
|
case\-preserving: although existing files can be opened using any case,
|
|
the exact case used to create the file is preserved and available for
|
|
programs to query.
|
|
It is not allowed for two files in the same directory to differ only by
|
|
case.
|
|
.PP
|
|
Usually file systems on macOS are case\-insensitive.
|
|
It is possible to make macOS file systems case\-sensitive but that is
|
|
not the default
|
|
.PP
|
|
The \[lq]\[en]vfs\-case\-insensitive\[rq] mount flag controls how rclone
|
|
handles these two cases.
|
|
If its value is \[lq]false\[rq], rclone passes file names to the mounted
|
|
file system as is.
|
|
If the flag is \[lq]true\[rq] (or appears without a value on command
|
|
line), rclone may perform a \[lq]fixup\[rq] as explained below.
|
|
.PP
|
|
The user may specify a file name to open/delete/rename/etc with a case
|
|
different than what is stored on mounted file system.
|
|
If an argument refers to an existing file with exactly the same name,
|
|
then the case of the existing file on the disk will be used.
|
|
However, if a file name with exactly the same name is not found but a
|
|
name differing only by case exists, rclone will transparently fixup the
|
|
name.
|
|
This fixup happens only when an existing file is requested.
|
|
Case sensitivity of file names created anew by rclone is controlled by
|
|
an underlying mounted file system.
|
|
.PP
|
|
Note that case sensitivity of the operating system running rclone (the
|
|
target) may differ from case sensitivity of a file system mounted by
|
|
rclone (the source).
|
|
The flag controls whether \[lq]fixup\[rq] is performed to satisfy the
|
|
target.
|
|
.PP
|
|
If the flag is not provided on command line, then its default value
|
|
depends on the operating system where rclone runs: \[lq]true\[rq] on
|
|
Windows and macOS, \[lq]false\[rq] otherwise.
|
|
If the flag is provided without a value, then it is \[lq]true\[rq].
|
|
.SS Auth Proxy
|
|
.PP
|
|
If you supply the parameter \f[C]\-\-auth\-proxy /path/to/program\f[R]
|
|
then rclone will use that program to generate backends on the fly which
|
|
then are used to authenticate incoming requests.
|
|
This uses a simple JSON based protocl with input on STDIN and output on
|
|
STDOUT.
|
|
.PP
|
|
\f[B]PLEASE NOTE:\f[R] \f[C]\-\-auth\-proxy\f[R] and
|
|
\f[C]\-\-authorized\-keys\f[R] cannot be used together, if
|
|
\f[C]\-\-auth\-proxy\f[R] is set the authorized keys option will be
|
|
ignored.
|
|
.PP
|
|
There is an example program
|
|
bin/test_proxy.py (https://github.com/rclone/rclone/blob/master/test_proxy.py)
|
|
in the rclone source code.
|
|
.PP
|
|
The program\[cq]s job is to take a \f[C]user\f[R] and \f[C]pass\f[R] on
|
|
the input and turn those into the config for a backend on STDOUT in JSON
|
|
format.
|
|
This config will have any default parameters for the backend added, but
|
|
it won\[cq]t use configuration from environment variables or command
|
|
line options \- it is the job of the proxy program to make a complete
|
|
config.
|
|
.PP
|
|
This config generated must have this extra parameter \- \f[C]_root\f[R]
|
|
\- root to use for the backend
|
|
.PP
|
|
And it may have this parameter \- \f[C]_obscure\f[R] \- comma separated
|
|
strings for parameters to obscure
|
|
.PP
|
|
If password authentication was used by the client, input to the proxy
|
|
process (on STDIN) would look similar to this:
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
{
|
|
\[dq]user\[dq]: \[dq]me\[dq],
|
|
\[dq]pass\[dq]: \[dq]mypassword\[dq]
|
|
}
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
If public\-key authentication was used by the client, input to the proxy
|
|
process (on STDIN) would look similar to this:
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
{
|
|
\[dq]user\[dq]: \[dq]me\[dq],
|
|
\[dq]public_key\[dq]: \[dq]AAAAB3NzaC1yc2EAAAADAQABAAABAQDuwESFdAe14hVS6omeyX7edc...JQdf\[dq]
|
|
}
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
And as an example return this on STDOUT
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
{
|
|
\[dq]type\[dq]: \[dq]sftp\[dq],
|
|
\[dq]_root\[dq]: \[dq]\[dq],
|
|
\[dq]_obscure\[dq]: \[dq]pass\[dq],
|
|
\[dq]user\[dq]: \[dq]me\[dq],
|
|
\[dq]pass\[dq]: \[dq]mypassword\[dq],
|
|
\[dq]host\[dq]: \[dq]sftp.example.com\[dq]
|
|
}
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
This would mean that an SFTP backend would be created on the fly for the
|
|
\f[C]user\f[R] and \f[C]pass\f[R]/\f[C]public_key\f[R] returned in the
|
|
output to the host given.
|
|
Note that since \f[C]_obscure\f[R] is set to \f[C]pass\f[R], rclone will
|
|
obscure the \f[C]pass\f[R] parameter before creating the backend (which
|
|
is required for sftp backends).
|
|
.PP
|
|
The program can manipulate the supplied \f[C]user\f[R] in any way, for
|
|
example to make proxy to many different sftp backends, you could make
|
|
the \f[C]user\f[R] be \f[C]user\[at]example.com\f[R] and then set the
|
|
\f[C]host\f[R] to \f[C]example.com\f[R] in the output and the user to
|
|
\f[C]user\f[R].
|
|
For security you\[cq]d probably want to restrict the \f[C]host\f[R] to a
|
|
limited list.
|
|
.PP
|
|
Note that an internal cache is keyed on \f[C]user\f[R] so only use that
|
|
for configuration, don\[cq]t use \f[C]pass\f[R] or \f[C]public_key\f[R].
|
|
This also means that if a user\[cq]s password or public\-key is changed
|
|
the cache will need to expire (which takes 5 mins) before it takes
|
|
effect.
|
|
.PP
|
|
This can be used to build general purpose proxies to any kind of backend
|
|
that rclone supports.
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone serve webdav remote:path [flags]
|
|
\f[R]
|
|
.fi
|
|
.SS Options
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
\-\-addr string IPaddress:Port or :Port to bind server to. (default \[dq]localhost:8080\[dq])
|
|
\-\-auth\-proxy string A program to use to create the backend from the auth.
|
|
\-\-baseurl string Prefix for URLs \- leave blank for root.
|
|
\-\-cert string SSL PEM key (concatenation of certificate and CA certificate)
|
|
\-\-client\-ca string Client certificate authority to verify clients with
|
|
\-\-dir\-cache\-time duration Time to cache directory entries for. (default 5m0s)
|
|
\-\-dir\-perms FileMode Directory permissions (default 0777)
|
|
\-\-disable\-dir\-list Disable HTML directory list on GET request for a directory
|
|
\-\-etag\-hash string Which hash to use for the ETag, or auto or blank for off
|
|
\-\-file\-perms FileMode File permissions (default 0666)
|
|
\-\-gid uint32 Override the gid field set by the filesystem. (default 1000)
|
|
\-h, \-\-help help for webdav
|
|
\-\-htpasswd string htpasswd file \- if not provided no authentication is done
|
|
\-\-key string SSL PEM Private key
|
|
\-\-max\-header\-bytes int Maximum size of request header (default 4096)
|
|
\-\-no\-checksum Don\[aq]t compare checksums on up/download.
|
|
\-\-no\-modtime Don\[aq]t read/write the modification time (can speed things up).
|
|
\-\-no\-seek Don\[aq]t allow seeking in files.
|
|
\-\-pass string Password for authentication.
|
|
\-\-poll\-interval duration Time to wait between polling for changes. Must be smaller than dir\-cache\-time. Only on supported remotes. Set to 0 to disable. (default 1m0s)
|
|
\-\-read\-only Mount read\-only.
|
|
\-\-realm string realm for authentication (default \[dq]rclone\[dq])
|
|
\-\-server\-read\-timeout duration Timeout for server reading data (default 1h0m0s)
|
|
\-\-server\-write\-timeout duration Timeout for server writing data (default 1h0m0s)
|
|
\-\-template string User Specified Template.
|
|
\-\-uid uint32 Override the uid field set by the filesystem. (default 1000)
|
|
\-\-umask int Override the permission bits set by the filesystem. (default 2)
|
|
\-\-user string User name for authentication.
|
|
\-\-vfs\-cache\-max\-age duration Max age of objects in the cache. (default 1h0m0s)
|
|
\-\-vfs\-cache\-max\-size SizeSuffix Max total size of objects in the cache. (default off)
|
|
\-\-vfs\-cache\-mode CacheMode Cache mode off|minimal|writes|full (default off)
|
|
\-\-vfs\-cache\-poll\-interval duration Interval to poll the cache for stale objects. (default 1m0s)
|
|
\-\-vfs\-case\-insensitive If a file name not found, find a case insensitive match.
|
|
\-\-vfs\-read\-chunk\-size SizeSuffix Read the source objects in chunks. (default 128M)
|
|
\-\-vfs\-read\-chunk\-size\-limit SizeSuffix If greater than \-\-vfs\-read\-chunk\-size, double the chunk size after each chunk read, until the limit is reached. \[aq]off\[aq] is unlimited. (default off)
|
|
\-\-vfs\-read\-wait duration Time to wait for in\-sequence read before seeking. (default 20ms)
|
|
\-\-vfs\-write\-wait duration Time to wait for in\-sequence write before giving error. (default 1s)
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
See the global flags page (https://rclone.org/flags/) for global options
|
|
not listed here.
|
|
.SS SEE ALSO
|
|
.IP \[bu] 2
|
|
rclone serve (https://rclone.org/commands/rclone_serve/) \- Serve a
|
|
remote over a protocol.
|
|
.SH rclone settier
|
|
.PP
|
|
Changes storage class/tier of objects in remote.
|
|
.SS Synopsis
|
|
.PP
|
|
rclone settier changes storage tier or class at remote if supported.
|
|
Few cloud storage services provides different storage classes on
|
|
objects, for example AWS S3 and Glacier, Azure Blob storage \- Hot, Cool
|
|
and Archive, Google Cloud Storage, Regional Storage, Nearline, Coldline
|
|
etc.
|
|
.PP
|
|
Note that, certain tier changes make objects not available to access
|
|
immediately.
|
|
For example tiering to archive in azure blob storage makes objects in
|
|
frozen state, user can restore by setting tier to Hot/Cool, similarly S3
|
|
to Glacier makes object inaccessible.true
|
|
.PP
|
|
You can use it to tier single object
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone settier Cool remote:path/file
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
Or use rclone filters to set tier on only specific files
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone \-\-include \[dq]*.txt\[dq] settier Hot remote:path/dir
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
Or just provide remote directory and all files in directory will be
|
|
tiered
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone settier tier remote:path/dir
|
|
\f[R]
|
|
.fi
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone settier tier remote:path [flags]
|
|
\f[R]
|
|
.fi
|
|
.SS Options
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
\-h, \-\-help help for settier
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
See the global flags page (https://rclone.org/flags/) for global options
|
|
not listed here.
|
|
.SS SEE ALSO
|
|
.IP \[bu] 2
|
|
rclone (https://rclone.org/commands/rclone/) \- Show help for rclone
|
|
commands, flags and backends.
|
|
.SH rclone touch
|
|
.PP
|
|
Create new file or change file modification time.
|
|
.SS Synopsis
|
|
.PP
|
|
Set the modification time on object(s) as specified by remote:path to
|
|
have the current time.
|
|
.PP
|
|
If remote:path does not exist then a zero sized object will be created
|
|
unless the \[en]no\-create flag is provided.
|
|
.PP
|
|
If \[en]timestamp is used then it will set the modification time to that
|
|
time instead of the current time.
|
|
Times may be specified as one of:
|
|
.IP \[bu] 2
|
|
`YYMMDD' \- eg.
|
|
17.10.30
|
|
.IP \[bu] 2
|
|
`YYYY\-MM\-DDTHH:MM:SS' \- eg.
|
|
2006\-01\-02T15:04:05
|
|
.PP
|
|
Note that \[en]timestamp is in UTC if you want local time then add the
|
|
\[en]localtime flag.
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone touch remote:path [flags]
|
|
\f[R]
|
|
.fi
|
|
.SS Options
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
\-h, \-\-help help for touch
|
|
\-\-localtime Use localtime for timestamp, not UTC.
|
|
\-C, \-\-no\-create Do not create the file if it does not exist.
|
|
\-t, \-\-timestamp string Use specified time instead of the current time of day.
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
See the global flags page (https://rclone.org/flags/) for global options
|
|
not listed here.
|
|
.SS SEE ALSO
|
|
.IP \[bu] 2
|
|
rclone (https://rclone.org/commands/rclone/) \- Show help for rclone
|
|
commands, flags and backends.
|
|
.SH rclone tree
|
|
.PP
|
|
List the contents of the remote in a tree like fashion.
|
|
.SS Synopsis
|
|
.PP
|
|
rclone tree lists the contents of a remote in a similar way to the unix
|
|
tree command.
|
|
.PP
|
|
For example
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
$ rclone tree remote:path
|
|
/
|
|
\[u251C]\[u2500]\[u2500] file1
|
|
\[u251C]\[u2500]\[u2500] file2
|
|
\[u251C]\[u2500]\[u2500] file3
|
|
\[u2514]\[u2500]\[u2500] subdir
|
|
\[u251C]\[u2500]\[u2500] file4
|
|
\[u2514]\[u2500]\[u2500] file5
|
|
|
|
1 directories, 5 files
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
You can use any of the filtering options with the tree command (eg
|
|
\[en]include and \[en]exclude).
|
|
You can also use \[en]fast\-list.
|
|
.PP
|
|
The tree command has many options for controlling the listing which are
|
|
compatible with the tree command.
|
|
Note that not all of them have short options as they conflict with
|
|
rclone\[cq]s short options.
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone tree remote:path [flags]
|
|
\f[R]
|
|
.fi
|
|
.SS Options
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
\-a, \-\-all All files are listed (list . files too).
|
|
\-C, \-\-color Turn colorization on always.
|
|
\-d, \-\-dirs\-only List directories only.
|
|
\-\-dirsfirst List directories before files (\-U disables).
|
|
\-\-full\-path Print the full path prefix for each file.
|
|
\-h, \-\-help help for tree
|
|
\-\-human Print the size in a more human readable way.
|
|
\-\-level int Descend only level directories deep.
|
|
\-D, \-\-modtime Print the date of last modification.
|
|
\-i, \-\-noindent Don\[aq]t print indentation lines.
|
|
\-\-noreport Turn off file/directory count at end of tree listing.
|
|
\-o, \-\-output string Output to file instead of stdout.
|
|
\-p, \-\-protections Print the protections for each file.
|
|
\-Q, \-\-quote Quote filenames with double quotes.
|
|
\-s, \-\-size Print the size in bytes of each file.
|
|
\-\-sort string Select sort: name,version,size,mtime,ctime.
|
|
\-\-sort\-ctime Sort files by last status change time.
|
|
\-t, \-\-sort\-modtime Sort files by last modification time.
|
|
\-r, \-\-sort\-reverse Reverse the order of the sort.
|
|
\-U, \-\-unsorted Leave files unsorted.
|
|
\-\-version Sort files alphanumerically by version.
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
See the global flags page (https://rclone.org/flags/) for global options
|
|
not listed here.
|
|
.SS SEE ALSO
|
|
.IP \[bu] 2
|
|
rclone (https://rclone.org/commands/rclone/) \- Show help for rclone
|
|
commands, flags and backends.
|
|
.SS Copying single files
|
|
.PP
|
|
rclone normally syncs or copies directories.
|
|
However, if the source remote points to a file, rclone will just copy
|
|
that file.
|
|
The destination remote must point to a directory \- rclone will give the
|
|
error
|
|
\f[C]Failed to create file system for \[dq]remote:file\[dq]: is a file not a directory\f[R]
|
|
if it isn\[cq]t.
|
|
.PP
|
|
For example, suppose you have a remote with a file in called
|
|
\f[C]test.jpg\f[R], then you could copy just that file like this
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone copy remote:test.jpg /tmp/download
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
The file \f[C]test.jpg\f[R] will be placed inside
|
|
\f[C]/tmp/download\f[R].
|
|
.PP
|
|
This is equivalent to specifying
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone copy \-\-files\-from /tmp/files remote: /tmp/download
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
Where \f[C]/tmp/files\f[R] contains the single line
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
test.jpg
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
It is recommended to use \f[C]copy\f[R] when copying individual files,
|
|
not \f[C]sync\f[R].
|
|
They have pretty much the same effect but \f[C]copy\f[R] will use a lot
|
|
less memory.
|
|
.SS Syntax of remote paths
|
|
.PP
|
|
The syntax of the paths passed to the rclone command are as follows.
|
|
.SS /path/to/dir
|
|
.PP
|
|
This refers to the local file system.
|
|
.PP
|
|
On Windows only \f[C]\[rs]\f[R] may be used instead of \f[C]/\f[R] in
|
|
local paths \f[B]only\f[R], non local paths must use \f[C]/\f[R].
|
|
.PP
|
|
These paths needn\[cq]t start with a leading \f[C]/\f[R] \- if they
|
|
don\[cq]t then they will be relative to the current directory.
|
|
.SS remote:path/to/dir
|
|
.PP
|
|
This refers to a directory \f[C]path/to/dir\f[R] on \f[C]remote:\f[R] as
|
|
defined in the config file (configured with \f[C]rclone config\f[R]).
|
|
.SS remote:/path/to/dir
|
|
.PP
|
|
On most backends this is refers to the same directory as
|
|
\f[C]remote:path/to/dir\f[R] and that format should be preferred.
|
|
On a very small number of remotes (FTP, SFTP, Dropbox for business) this
|
|
will refer to a different directory.
|
|
On these, paths without a leading \f[C]/\f[R] will refer to your
|
|
\[lq]home\[rq] directory and paths with a leading \f[C]/\f[R] will refer
|
|
to the root.
|
|
.SS :backend:path/to/dir
|
|
.PP
|
|
This is an advanced form for creating remotes on the fly.
|
|
\f[C]backend\f[R] should be the name or prefix of a backend (the
|
|
\f[C]type\f[R] in the config file) and all the configuration for the
|
|
backend should be provided on the command line (or in environment
|
|
variables).
|
|
.PP
|
|
Here are some examples:
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone lsd \-\-http\-url https://pub.rclone.org :http:
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
To list all the directories in the root of
|
|
\f[C]https://pub.rclone.org/\f[R].
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone lsf \-\-http\-url https://example.com :http:path/to/dir
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
To list files and directories in
|
|
\f[C]https://example.com/path/to/dir/\f[R]
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone copy \-\-http\-url https://example.com :http:path/to/dir /tmp/dir
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
To copy files and directories in
|
|
\f[C]https://example.com/path/to/dir\f[R] to \f[C]/tmp/dir\f[R].
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone copy \-\-sftp\-host example.com :sftp:path/to/dir /tmp/dir
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
To copy files and directories from \f[C]example.com\f[R] in the relative
|
|
directory \f[C]path/to/dir\f[R] to \f[C]/tmp/dir\f[R] using sftp.
|
|
.SS Quoting and the shell
|
|
.PP
|
|
When you are typing commands to your computer you are using something
|
|
called the command line shell.
|
|
This interprets various characters in an OS specific way.
|
|
.PP
|
|
Here are some gotchas which may help users unfamiliar with the shell
|
|
rules
|
|
.SS Linux / OSX
|
|
.PP
|
|
If your names have spaces or shell metacharacters (eg \f[C]*\f[R],
|
|
\f[C]?\f[R], \f[C]$\f[R], \f[C]\[aq]\f[R], \f[C]\[dq]\f[R] etc) then you
|
|
must quote them.
|
|
Use single quotes \f[C]\[aq]\f[R] by default.
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone copy \[aq]Important files?\[aq] remote:backup
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
If you want to send a \f[C]\[aq]\f[R] you will need to use
|
|
\f[C]\[dq]\f[R], eg
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone copy \[dq]O\[aq]Reilly Reviews\[dq] remote:backup
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
The rules for quoting metacharacters are complicated and if you want the
|
|
full details you\[cq]ll have to consult the manual page for your shell.
|
|
.SS Windows
|
|
.PP
|
|
If your names have spaces in you need to put them in \f[C]\[dq]\f[R], eg
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone copy \[dq]E:\[rs]folder name\[rs]folder name\[rs]folder name\[dq] remote:backup
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
If you are using the root directory on its own then don\[cq]t quote it
|
|
(see #464 (https://github.com/rclone/rclone/issues/464) for why), eg
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone copy E:\[rs] remote:backup
|
|
\f[R]
|
|
.fi
|
|
.SS Copying files or directories with \f[C]:\f[R] in the names
|
|
.PP
|
|
rclone uses \f[C]:\f[R] to mark a remote name.
|
|
This is, however, a valid filename component in non\-Windows OSes.
|
|
The remote name parser will only search for a \f[C]:\f[R] up to the
|
|
first \f[C]/\f[R] so if you need to act on a file or directory like this
|
|
then use the full path starting with a \f[C]/\f[R], or use \f[C]./\f[R]
|
|
as a current directory prefix.
|
|
.PP
|
|
So to sync a directory called \f[C]sync:me\f[R] to a remote called
|
|
\f[C]remote:\f[R] use
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone sync ./sync:me remote:path
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
or
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone sync /full/path/to/sync:me remote:path
|
|
\f[R]
|
|
.fi
|
|
.SS Server Side Copy
|
|
.PP
|
|
Most remotes (but not all \- see the
|
|
overview (https://rclone.org/overview/#optional-features)) support
|
|
server side copy.
|
|
.PP
|
|
This means if you want to copy one folder to another then rclone
|
|
won\[cq]t download all the files and re\-upload them; it will instruct
|
|
the server to copy them in place.
|
|
.PP
|
|
Eg
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone copy s3:oldbucket s3:newbucket
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
Will copy the contents of \f[C]oldbucket\f[R] to \f[C]newbucket\f[R]
|
|
without downloading and re\-uploading.
|
|
.PP
|
|
Remotes which don\[cq]t support server side copy \f[B]will\f[R] download
|
|
and re\-upload in this case.
|
|
.PP
|
|
Server side copies are used with \f[C]sync\f[R] and \f[C]copy\f[R] and
|
|
will be identified in the log when using the \f[C]\-v\f[R] flag.
|
|
The \f[C]move\f[R] command may also use them if remote doesn\[cq]t
|
|
support server side move directly.
|
|
This is done by issuing a server side copy then a delete which is much
|
|
quicker than a download and re\-upload.
|
|
.PP
|
|
Server side copies will only be attempted if the remote names are the
|
|
same.
|
|
.PP
|
|
This can be used when scripting to make aged backups efficiently, eg
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone sync remote:current\-backup remote:previous\-backup
|
|
rclone sync /path/to/files remote:current\-backup
|
|
\f[R]
|
|
.fi
|
|
.SS Options
|
|
.PP
|
|
Rclone has a number of options to control its behaviour.
|
|
.PP
|
|
Options that take parameters can have the values passed in two ways,
|
|
\f[C]\-\-option=value\f[R] or \f[C]\-\-option value\f[R].
|
|
However boolean (true/false) options behave slightly differently to the
|
|
other options in that \f[C]\-\-boolean\f[R] sets the option to
|
|
\f[C]true\f[R] and the absence of the flag sets it to \f[C]false\f[R].
|
|
It is also possible to specify \f[C]\-\-boolean=false\f[R] or
|
|
\f[C]\-\-boolean=true\f[R].
|
|
Note that \f[C]\-\-boolean false\f[R] is not valid \- this is parsed as
|
|
\f[C]\-\-boolean\f[R] and the \f[C]false\f[R] is parsed as an extra
|
|
command line argument for rclone.
|
|
.PP
|
|
Options which use TIME use the go time parser.
|
|
A duration string is a possibly signed sequence of decimal numbers, each
|
|
with optional fraction and a unit suffix, such as \[lq]300ms\[rq],
|
|
\[lq]\-1.5h\[rq] or \[lq]2h45m\[rq].
|
|
Valid time units are \[lq]ns\[rq], \[lq]us\[rq] (or \[lq]\[mc]s\[rq]),
|
|
\[lq]ms\[rq], \[lq]s\[rq], \[lq]m\[rq], \[lq]h\[rq].
|
|
.PP
|
|
Options which use SIZE use kByte by default.
|
|
However, a suffix of \f[C]b\f[R] for bytes, \f[C]k\f[R] for kBytes,
|
|
\f[C]M\f[R] for MBytes, \f[C]G\f[R] for GBytes, \f[C]T\f[R] for TBytes
|
|
and \f[C]P\f[R] for PBytes may be used.
|
|
These are the binary units, eg 1, 2**10, 2**20, 2**30 respectively.
|
|
.SS \[en]backup\-dir=DIR
|
|
.PP
|
|
When using \f[C]sync\f[R], \f[C]copy\f[R] or \f[C]move\f[R] any files
|
|
which would have been overwritten or deleted are moved in their original
|
|
hierarchy into this directory.
|
|
.PP
|
|
If \f[C]\-\-suffix\f[R] is set, then the moved files will have the
|
|
suffix added to them.
|
|
If there is a file with the same path (after the suffix has been added)
|
|
in DIR, then it will be overwritten.
|
|
.PP
|
|
The remote in use must support server side move or copy and you must use
|
|
the same remote as the destination of the sync.
|
|
The backup directory must not overlap the destination directory.
|
|
.PP
|
|
For example
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone sync /path/to/local remote:current \-\-backup\-dir remote:old
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
will sync \f[C]/path/to/local\f[R] to \f[C]remote:current\f[R], but for
|
|
any files which would have been updated or deleted will be stored in
|
|
\f[C]remote:old\f[R].
|
|
.PP
|
|
If running rclone from a script you might want to use today\[cq]s date
|
|
as the directory name passed to \f[C]\-\-backup\-dir\f[R] to store the
|
|
old files, or you might want to pass \f[C]\-\-suffix\f[R] with
|
|
today\[cq]s date.
|
|
.PP
|
|
See \f[C]\-\-compare\-dest\f[R] and \f[C]\-\-copy\-dest\f[R].
|
|
.SS \[en]bind string
|
|
.PP
|
|
Local address to bind to for outgoing connections.
|
|
This can be an IPv4 address (1.2.3.4), an IPv6 address (1234::789A) or
|
|
host name.
|
|
If the host name doesn\[cq]t resolve or resolves to more than one IP
|
|
address it will give an error.
|
|
.SS \[en]bwlimit=BANDWIDTH_SPEC
|
|
.PP
|
|
This option controls the bandwidth limit.
|
|
Limits can be specified in two ways: As a single limit, or as a
|
|
timetable.
|
|
.PP
|
|
Single limits last for the duration of the session.
|
|
To use a single limit, specify the desired bandwidth in kBytes/s, or use
|
|
a suffix b|k|M|G.
|
|
The default is \f[C]0\f[R] which means to not limit bandwidth.
|
|
.PP
|
|
For example, to limit bandwidth usage to 10 MBytes/s use
|
|
\f[C]\-\-bwlimit 10M\f[R]
|
|
.PP
|
|
It is also possible to specify a \[lq]timetable\[rq] of limits, which
|
|
will cause certain limits to be applied at certain times.
|
|
To specify a timetable, format your entries as
|
|
\f[C]WEEKDAY\-HH:MM,BANDWIDTH WEEKDAY\-HH:MM,BANDWIDTH...\f[R] where:
|
|
\f[C]WEEKDAY\f[R] is optional element.
|
|
It could be written as whole world or only using 3 first characters.
|
|
\f[C]HH:MM\f[R] is an hour from 00:00 to 23:59.
|
|
.PP
|
|
An example of a typical timetable to avoid link saturation during
|
|
daytime working hours could be:
|
|
.PP
|
|
\f[C]\-\-bwlimit \[dq]08:00,512 12:00,10M 13:00,512 18:00,30M 23:00,off\[dq]\f[R]
|
|
.PP
|
|
In this example, the transfer bandwidth will be every day set to
|
|
512kBytes/sec at 8am.
|
|
At noon, it will raise to 10Mbytes/s, and drop back to 512kBytes/sec at
|
|
1pm.
|
|
At 6pm, the bandwidth limit will be set to 30MBytes/s, and at 11pm it
|
|
will be completely disabled (full speed).
|
|
Anything between 11pm and 8am will remain unlimited.
|
|
.PP
|
|
An example of timetable with \f[C]WEEKDAY\f[R] could be:
|
|
.PP
|
|
\f[C]\-\-bwlimit \[dq]Mon\-00:00,512 Fri\-23:59,10M Sat\-10:00,1M Sun\-20:00,off\[dq]\f[R]
|
|
.PP
|
|
It mean that, the transfer bandwidth will be set to 512kBytes/sec on
|
|
Monday.
|
|
It will raise to 10Mbytes/s before the end of Friday.
|
|
At 10:00 on Sunday it will be set to 1Mbyte/s.
|
|
From 20:00 at Sunday will be unlimited.
|
|
.PP
|
|
Timeslots without weekday are extended to whole week.
|
|
So this one example:
|
|
.PP
|
|
\f[C]\-\-bwlimit \[dq]Mon\-00:00,512 12:00,1M Sun\-20:00,off\[dq]\f[R]
|
|
.PP
|
|
Is equal to this:
|
|
.PP
|
|
\f[C]\-\-bwlimit \[dq]Mon\-00:00,512Mon\-12:00,1M Tue\-12:00,1M Wed\-12:00,1M Thu\-12:00,1M Fri\-12:00,1M Sat\-12:00,1M Sun\-12:00,1M Sun\-20:00,off\[dq]\f[R]
|
|
.PP
|
|
Bandwidth limits only apply to the data transfer.
|
|
They don\[cq]t apply to the bandwidth of the directory listings etc.
|
|
.PP
|
|
Note that the units are Bytes/s, not Bits/s.
|
|
Typically connections are measured in Bits/s \- to convert divide by 8.
|
|
For example, let\[cq]s say you have a 10 Mbit/s connection and you wish
|
|
rclone to use half of it \- 5 Mbit/s.
|
|
This is 5/8 = 0.625MByte/s so you would use a
|
|
\f[C]\-\-bwlimit 0.625M\f[R] parameter for rclone.
|
|
.PP
|
|
On Unix systems (Linux, macOS, \&...) the bandwidth limiter can be
|
|
toggled by sending a \f[C]SIGUSR2\f[R] signal to rclone.
|
|
This allows to remove the limitations of a long running rclone transfer
|
|
and to restore it back to the value specified with \f[C]\-\-bwlimit\f[R]
|
|
quickly when needed.
|
|
Assuming there is only one rclone instance running, you can toggle the
|
|
limiter like this:
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
kill \-SIGUSR2 $(pidof rclone)
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
If you configure rclone with a remote control (/rc) then you can use
|
|
change the bwlimit dynamically:
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone rc core/bwlimit rate=1M
|
|
\f[R]
|
|
.fi
|
|
.SS \[en]buffer\-size=SIZE
|
|
.PP
|
|
Use this sized buffer to speed up file transfers.
|
|
Each \f[C]\-\-transfer\f[R] will use this much memory for buffering.
|
|
.PP
|
|
When using \f[C]mount\f[R] or \f[C]cmount\f[R] each open file descriptor
|
|
will use this much memory for buffering.
|
|
See the mount (https://rclone.org/commands/rclone_mount/#file-buffering)
|
|
documentation for more details.
|
|
.PP
|
|
Set to \f[C]0\f[R] to disable the buffering for the minimum memory
|
|
usage.
|
|
.PP
|
|
Note that the memory allocation of the buffers is influenced by the
|
|
\[en]use\-mmap flag.
|
|
.SS \[en]check\-first
|
|
.PP
|
|
If this flag is set then in a \f[C]sync\f[R], \f[C]copy\f[R] or
|
|
\f[C]move\f[R], rclone will do all the checks to see whether files need
|
|
to be transferred before doing any of the transfers.
|
|
Normally rclone would start running transfers as soon as possible.
|
|
.PP
|
|
This flag can be useful on IO limited systems where transfers interfere
|
|
with checking.
|
|
.PP
|
|
Using this flag can use more memory as it effectively sets
|
|
\f[C]\-\-max\-backlog\f[R] to infinite.
|
|
This means that all the info on the objects to transfer is held in
|
|
memory before the transfers start.
|
|
.SS \[en]checkers=N
|
|
.PP
|
|
The number of checkers to run in parallel.
|
|
Checkers do the equality checking of files during a sync.
|
|
For some storage systems (eg S3, Swift, Dropbox) this can take a
|
|
significant amount of time so they are run in parallel.
|
|
.PP
|
|
The default is to run 8 checkers in parallel.
|
|
.SS \-c, \[en]checksum
|
|
.PP
|
|
Normally rclone will look at modification time and size of files to see
|
|
if they are equal.
|
|
If you set this flag then rclone will check the file hash and size to
|
|
determine if files are equal.
|
|
.PP
|
|
This is useful when the remote doesn\[cq]t support setting modified time
|
|
and a more accurate sync is desired than just checking the file size.
|
|
.PP
|
|
This is very useful when transferring between remotes which store the
|
|
same hash type on the object, eg Drive and Swift.
|
|
For details of which remotes support which hash type see the table in
|
|
the overview section (https://rclone.org/overview/).
|
|
.PP
|
|
Eg \f[C]rclone \-\-checksum sync s3:/bucket swift:/bucket\f[R] would run
|
|
much quicker than without the \f[C]\-\-checksum\f[R] flag.
|
|
.PP
|
|
When using this flag, rclone won\[cq]t update mtimes of remote files if
|
|
they are incorrect as it would normally.
|
|
.SS \[en]compare\-dest=DIR
|
|
.PP
|
|
When using \f[C]sync\f[R], \f[C]copy\f[R] or \f[C]move\f[R] DIR is
|
|
checked in addition to the destination for files.
|
|
If a file identical to the source is found that file is NOT copied from
|
|
source.
|
|
This is useful to copy just files that have changed since the last
|
|
backup.
|
|
.PP
|
|
You must use the same remote as the destination of the sync.
|
|
The compare directory must not overlap the destination directory.
|
|
.PP
|
|
See \f[C]\-\-copy\-dest\f[R] and \f[C]\-\-backup\-dir\f[R].
|
|
.SS \[en]config=CONFIG_FILE
|
|
.PP
|
|
Specify the location of the rclone config file.
|
|
.PP
|
|
Normally the config file is in your home directory as a file called
|
|
\f[C].config/rclone/rclone.conf\f[R] (or \f[C].rclone.conf\f[R] if
|
|
created with an older version).
|
|
If \f[C]$XDG_CONFIG_HOME\f[R] is set it will be at
|
|
\f[C]$XDG_CONFIG_HOME/rclone/rclone.conf\f[R].
|
|
.PP
|
|
If there is a file \f[C]rclone.conf\f[R] in the same directory as the
|
|
rclone executable it will be preferred.
|
|
This file must be created manually for Rclone to use it, it will never
|
|
be created automatically.
|
|
.PP
|
|
If you run \f[C]rclone config file\f[R] you will see where the default
|
|
location is for you.
|
|
.PP
|
|
Use this flag to override the config location, eg
|
|
\f[C]rclone \-\-config=\[dq].myconfig\[dq] .config\f[R].
|
|
.SS \[en]contimeout=TIME
|
|
.PP
|
|
Set the connection timeout.
|
|
This should be in go time format which looks like \f[C]5s\f[R] for 5
|
|
seconds, \f[C]10m\f[R] for 10 minutes, or \f[C]3h30m\f[R].
|
|
.PP
|
|
The connection timeout is the amount of time rclone will wait for a
|
|
connection to go through to a remote object storage system.
|
|
It is \f[C]1m\f[R] by default.
|
|
.SS \[en]copy\-dest=DIR
|
|
.PP
|
|
When using \f[C]sync\f[R], \f[C]copy\f[R] or \f[C]move\f[R] DIR is
|
|
checked in addition to the destination for files.
|
|
If a file identical to the source is found that file is server side
|
|
copied from DIR to the destination.
|
|
This is useful for incremental backup.
|
|
.PP
|
|
The remote in use must support server side copy and you must use the
|
|
same remote as the destination of the sync.
|
|
The compare directory must not overlap the destination directory.
|
|
.PP
|
|
See \f[C]\-\-compare\-dest\f[R] and \f[C]\-\-backup\-dir\f[R].
|
|
.SS \[en]dedupe\-mode MODE
|
|
.PP
|
|
Mode to run dedupe command in.
|
|
One of \f[C]interactive\f[R], \f[C]skip\f[R], \f[C]first\f[R],
|
|
\f[C]newest\f[R], \f[C]oldest\f[R], \f[C]rename\f[R].
|
|
The default is \f[C]interactive\f[R].
|
|
See the dedupe command for more information as to what these options
|
|
mean.
|
|
.SS \[en]disable FEATURE,FEATURE,\&...
|
|
.PP
|
|
This disables a comma separated list of optional features.
|
|
For example to disable server side move and server side copy use:
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
\-\-disable move,copy
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
The features can be put in any case.
|
|
.PP
|
|
To see a list of which features can be disabled use:
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
\-\-disable help
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
See the overview features (https://rclone.org/overview/#features) and
|
|
optional features (https://rclone.org/overview/#optional-features) to
|
|
get an idea of which feature does what.
|
|
.PP
|
|
This flag can be useful for debugging and in exceptional circumstances
|
|
(eg Google Drive limiting the total volume of Server Side Copies to
|
|
100GB/day).
|
|
.SS \-n, \[en]dry\-run
|
|
.PP
|
|
Do a trial run with no permanent changes.
|
|
Use this to see what rclone would do without actually doing it.
|
|
Useful when setting up the \f[C]sync\f[R] command which deletes files in
|
|
the destination.
|
|
.SS \[en]expect\-continue\-timeout=TIME
|
|
.PP
|
|
This specifies the amount of time to wait for a server\[cq]s first
|
|
response headers after fully writing the request headers if the request
|
|
has an \[lq]Expect: 100\-continue\[rq] header.
|
|
Not all backends support using this.
|
|
.PP
|
|
Zero means no timeout and causes the body to be sent immediately,
|
|
without waiting for the server to approve.
|
|
This time does not include the time to send the request header.
|
|
.PP
|
|
The default is \f[C]1s\f[R].
|
|
Set to \f[C]0\f[R] to disable.
|
|
.SS \[en]error\-on\-no\-transfer
|
|
.PP
|
|
By default, rclone will exit with return code 0 if there were no errors.
|
|
.PP
|
|
This option allows rclone to return exit code 9 if no files were
|
|
transferred between the source and destination.
|
|
This allows using rclone in scripts, and triggering follow\-on actions
|
|
if data was copied, or skipping if not.
|
|
.PP
|
|
NB: Enabling this option turns a usually non\-fatal error into a
|
|
potentially fatal one \- please check and adjust your scripts
|
|
accordingly!
|
|
.SS \[en]header
|
|
.PP
|
|
Add an HTTP header for all transactions.
|
|
The flag can be repeated to add multiple headers.
|
|
.PP
|
|
If you want to add headers only for uploads use
|
|
\f[C]\-\-header\-upload\f[R] and if you want to add headers only for
|
|
downloads use \f[C]\-\-header\-download\f[R].
|
|
.PP
|
|
This flag is supported for all HTTP based backends even those not
|
|
supported by \f[C]\-\-header\-upload\f[R] and
|
|
\f[C]\-\-header\-download\f[R] so may be used as a workaround for those
|
|
with care.
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone ls remote:test \-\-header \[dq]X\-Rclone: Foo\[dq] \-\-header \[dq]X\-LetMeIn: Yes\[dq]
|
|
\f[R]
|
|
.fi
|
|
.SS \[en]header\-download
|
|
.PP
|
|
Add an HTTP header for all download transactions.
|
|
The flag can be repeated to add multiple headers.
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone sync s3:test/src \[ti]/dst \-\-header\-download \[dq]X\-Amz\-Meta\-Test: Foo\[dq] \-\-header\-download \[dq]X\-Amz\-Meta\-Test2: Bar\[dq]
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
See the GitHub issue here (https://github.com/rclone/rclone/issues/59)
|
|
for currently supported backends.
|
|
.SS \[en]header\-upload
|
|
.PP
|
|
Add an HTTP header for all upload transactions.
|
|
The flag can be repeated to add multiple headers.
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone sync \[ti]/src s3:test/dst \-\-header\-upload \[dq]Content\-Disposition: attachment; filename=\[aq]cool.html\[aq]\[dq] \-\-header\-upload \[dq]X\-Amz\-Meta\-Test: FooBar\[dq]
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
See the GitHub issue here (https://github.com/rclone/rclone/issues/59)
|
|
for currently supported backends.
|
|
.SS \[en]ignore\-case\-sync
|
|
.PP
|
|
Using this option will cause rclone to ignore the case of the files when
|
|
synchronizing so files will not be copied/synced when the existing
|
|
filenames are the same, even if the casing is different.
|
|
.SS \[en]ignore\-checksum
|
|
.PP
|
|
Normally rclone will check that the checksums of transferred files
|
|
match, and give an error \[lq]corrupted on transfer\[rq] if they
|
|
don\[cq]t.
|
|
.PP
|
|
You can use this option to skip that check.
|
|
You should only use it if you have had the \[lq]corrupted on
|
|
transfer\[rq] error message and you are sure you might want to transfer
|
|
potentially corrupted data.
|
|
.SS \[en]ignore\-existing
|
|
.PP
|
|
Using this option will make rclone unconditionally skip all files that
|
|
exist on the destination, no matter the content of these files.
|
|
.PP
|
|
While this isn\[cq]t a generally recommended option, it can be useful in
|
|
cases where your files change due to encryption.
|
|
However, it cannot correct partial transfers in case a transfer was
|
|
interrupted.
|
|
.SS \[en]ignore\-size
|
|
.PP
|
|
Normally rclone will look at modification time and size of files to see
|
|
if they are equal.
|
|
If you set this flag then rclone will check only the modification time.
|
|
If \f[C]\-\-checksum\f[R] is set then it only checks the checksum.
|
|
.PP
|
|
It will also cause rclone to skip verifying the sizes are the same after
|
|
transfer.
|
|
.PP
|
|
This can be useful for transferring files to and from OneDrive which
|
|
occasionally misreports the size of image files (see
|
|
#399 (https://github.com/rclone/rclone/issues/399) for more info).
|
|
.SS \-I, \[en]ignore\-times
|
|
.PP
|
|
Using this option will cause rclone to unconditionally upload all files
|
|
regardless of the state of files on the destination.
|
|
.PP
|
|
Normally rclone would skip any files that have the same modification
|
|
time and are the same size (or have the same checksum if using
|
|
\f[C]\-\-checksum\f[R]).
|
|
.SS \[en]immutable
|
|
.PP
|
|
Treat source and destination files as immutable and disallow
|
|
modification.
|
|
.PP
|
|
With this option set, files will be created and deleted as requested,
|
|
but existing files will never be updated.
|
|
If an existing file does not match between the source and destination,
|
|
rclone will give the error
|
|
\f[C]Source and destination exist but do not match: immutable file modified\f[R].
|
|
.PP
|
|
Note that only commands which transfer files (e.g.\ \f[C]sync\f[R],
|
|
\f[C]copy\f[R], \f[C]move\f[R]) are affected by this behavior, and only
|
|
modification is disallowed.
|
|
Files may still be deleted explicitly (e.g.\ \f[C]delete\f[R],
|
|
\f[C]purge\f[R]) or implicitly (e.g.\ \f[C]sync\f[R], \f[C]move\f[R]).
|
|
Use \f[C]copy \-\-immutable\f[R] if it is desired to avoid deletion as
|
|
well as modification.
|
|
.PP
|
|
This can be useful as an additional layer of protection for immutable or
|
|
append\-only data sets (notably backup archives), where modification
|
|
implies corruption and should not be propagated.
|
|
.SS \[en]leave\-root
|
|
.PP
|
|
During rmdirs it will not remove root directory, even if it\[cq]s empty.
|
|
.SS \[en]log\-file=FILE
|
|
.PP
|
|
Log all of rclone\[cq]s output to FILE.
|
|
This is not active by default.
|
|
This can be useful for tracking down problems with syncs in combination
|
|
with the \f[C]\-v\f[R] flag.
|
|
See the Logging section for more info.
|
|
.PP
|
|
Note that if you are using the \f[C]logrotate\f[R] program to manage
|
|
rclone\[cq]s logs, then you should use the \f[C]copytruncate\f[R] option
|
|
as rclone doesn\[cq]t have a signal to rotate logs.
|
|
.SS \[en]log\-format LIST
|
|
.PP
|
|
Comma separated list of log format options.
|
|
\f[C]date\f[R], \f[C]time\f[R], \f[C]microseconds\f[R],
|
|
\f[C]longfile\f[R], \f[C]shortfile\f[R], \f[C]UTC\f[R].
|
|
The default is \[lq]\f[C]date\f[R],\f[C]time\f[R]\[rq].
|
|
.SS \[en]log\-level LEVEL
|
|
.PP
|
|
This sets the log level for rclone.
|
|
The default log level is \f[C]NOTICE\f[R].
|
|
.PP
|
|
\f[C]DEBUG\f[R] is equivalent to \f[C]\-vv\f[R].
|
|
It outputs lots of debug info \- useful for bug reports and really
|
|
finding out what rclone is doing.
|
|
.PP
|
|
\f[C]INFO\f[R] is equivalent to \f[C]\-v\f[R].
|
|
It outputs information about each transfer and prints stats once a
|
|
minute by default.
|
|
.PP
|
|
\f[C]NOTICE\f[R] is the default log level if no logging flags are
|
|
supplied.
|
|
It outputs very little when things are working normally.
|
|
It outputs warnings and significant events.
|
|
.PP
|
|
\f[C]ERROR\f[R] is equivalent to \f[C]\-q\f[R].
|
|
It only outputs error messages.
|
|
.SS \[en]use\-json\-log
|
|
.PP
|
|
This switches the log format to JSON for rclone.
|
|
The fields of json log are level, msg, source, time.
|
|
.SS \[en]low\-level\-retries NUMBER
|
|
.PP
|
|
This controls the number of low level retries rclone does.
|
|
.PP
|
|
A low level retry is used to retry a failing operation \- typically one
|
|
HTTP request.
|
|
This might be uploading a chunk of a big file for example.
|
|
You will see low level retries in the log with the \f[C]\-v\f[R] flag.
|
|
.PP
|
|
This shouldn\[cq]t need to be changed from the default in normal
|
|
operations.
|
|
However, if you get a lot of low level retries you may wish to reduce
|
|
the value so rclone moves on to a high level retry (see the
|
|
\f[C]\-\-retries\f[R] flag) quicker.
|
|
.PP
|
|
Disable low level retries with \f[C]\-\-low\-level\-retries 1\f[R].
|
|
.SS \[en]max\-backlog=N
|
|
.PP
|
|
This is the maximum allowable backlog of files in a sync/copy/move
|
|
queued for being checked or transferred.
|
|
.PP
|
|
This can be set arbitrarily large.
|
|
It will only use memory when the queue is in use.
|
|
Note that it will use in the order of N kB of memory when the backlog is
|
|
in use.
|
|
.PP
|
|
Setting this large allows rclone to calculate how many files are pending
|
|
more accurately, give a more accurate estimated finish time and make
|
|
\f[C]\-\-order\-by\f[R] work more accurately.
|
|
.PP
|
|
Setting this small will make rclone more synchronous to the listings of
|
|
the remote which may be desirable.
|
|
.PP
|
|
Setting this to a negative number will make the backlog as large as
|
|
possible.
|
|
.SS \[en]max\-delete=N
|
|
.PP
|
|
This tells rclone not to delete more than N files.
|
|
If that limit is exceeded then a fatal error will be generated and
|
|
rclone will stop the operation in progress.
|
|
.SS \[en]max\-depth=N
|
|
.PP
|
|
This modifies the recursion depth for all the commands except purge.
|
|
.PP
|
|
So if you do \f[C]rclone \-\-max\-depth 1 ls remote:path\f[R] you will
|
|
see only the files in the top level directory.
|
|
Using \f[C]\-\-max\-depth 2\f[R] means you will see all the files in
|
|
first two directory levels and so on.
|
|
.PP
|
|
For historical reasons the \f[C]lsd\f[R] command defaults to using a
|
|
\f[C]\-\-max\-depth\f[R] of 1 \- you can override this with the command
|
|
line flag.
|
|
.PP
|
|
You can use this command to disable recursion (with
|
|
\f[C]\-\-max\-depth 1\f[R]).
|
|
.PP
|
|
Note that if you use this with \f[C]sync\f[R] and
|
|
\f[C]\-\-delete\-excluded\f[R] the files not recursed through are
|
|
considered excluded and will be deleted on the destination.
|
|
Test first with \f[C]\-\-dry\-run\f[R] if you are not sure what will
|
|
happen.
|
|
.SS \[en]max\-duration=TIME
|
|
.PP
|
|
Rclone will stop scheduling new transfers when it has run for the
|
|
duration specified.
|
|
.PP
|
|
Defaults to off.
|
|
.PP
|
|
When the limit is reached any existing transfers will complete.
|
|
.PP
|
|
Rclone won\[cq]t exit with an error if the transfer limit is reached.
|
|
.SS \[en]max\-transfer=SIZE
|
|
.PP
|
|
Rclone will stop transferring when it has reached the size specified.
|
|
Defaults to off.
|
|
.PP
|
|
When the limit is reached all transfers will stop immediately.
|
|
.PP
|
|
Rclone will exit with exit code 8 if the transfer limit is reached.
|
|
.SS \[en]cutoff\-mode=hard|soft|cautious
|
|
.PP
|
|
This modifies the behavior of \f[C]\-\-max\-transfer\f[R] Defaults to
|
|
\f[C]\-\-cutoff\-mode=hard\f[R].
|
|
.PP
|
|
Specifying \f[C]\-\-cutoff\-mode=hard\f[R] will stop transferring
|
|
immediately when Rclone reaches the limit.
|
|
.PP
|
|
Specifying \f[C]\-\-cutoff\-mode=soft\f[R] will stop starting new
|
|
transfers when Rclone reaches the limit.
|
|
.PP
|
|
Specifying \f[C]\-\-cutoff\-mode=cautious\f[R] will try to prevent
|
|
Rclone from reaching the limit.
|
|
.SS \[en]modify\-window=TIME
|
|
.PP
|
|
When checking whether a file has been modified, this is the maximum
|
|
allowed time difference that a file can have and still be considered
|
|
equivalent.
|
|
.PP
|
|
The default is \f[C]1ns\f[R] unless this is overridden by a remote.
|
|
For example OS X only stores modification times to the nearest second so
|
|
if you are reading and writing to an OS X filing system this will be
|
|
\f[C]1s\f[R] by default.
|
|
.PP
|
|
This command line flag allows you to override that computed default.
|
|
.SS \[en]multi\-thread\-cutoff=SIZE
|
|
.PP
|
|
When downloading files to the local backend above this size, rclone will
|
|
use multiple threads to download the file (default 250M).
|
|
.PP
|
|
Rclone preallocates the file (using
|
|
\f[C]fallocate(FALLOC_FL_KEEP_SIZE)\f[R] on unix or
|
|
\f[C]NTSetInformationFile\f[R] on Windows both of which takes no time)
|
|
then each thread writes directly into the file at the correct place.
|
|
This means that rclone won\[cq]t create fragmented or sparse files and
|
|
there won\[cq]t be any assembly time at the end of the transfer.
|
|
.PP
|
|
The number of threads used to download is controlled by
|
|
\f[C]\-\-multi\-thread\-streams\f[R].
|
|
.PP
|
|
Use \f[C]\-vv\f[R] if you wish to see info about the threads.
|
|
.PP
|
|
This will work with the \f[C]sync\f[R]/\f[C]copy\f[R]/\f[C]move\f[R]
|
|
commands and friends \f[C]copyto\f[R]/\f[C]moveto\f[R].
|
|
Multi thread downloads will be used with \f[C]rclone mount\f[R] and
|
|
\f[C]rclone serve\f[R] if \f[C]\-\-vfs\-cache\-mode\f[R] is set to
|
|
\f[C]writes\f[R] or above.
|
|
.PP
|
|
\f[B]NB\f[R] that this \f[B]only\f[R] works for a local destination but
|
|
will work with any source.
|
|
.PP
|
|
\f[B]NB\f[R] that multi thread copies are disabled for local to local
|
|
copies as they are faster without unless
|
|
\f[C]\-\-multi\-thread\-streams\f[R] is set explicitly.
|
|
.PP
|
|
\f[B]NB\f[R] on Windows using multi\-thread downloads will cause the
|
|
resulting files to be
|
|
sparse (https://en.wikipedia.org/wiki/Sparse_file).
|
|
Use \f[C]\-\-local\-no\-sparse\f[R] to disable sparse files (which may
|
|
cause long delays at the start of downloads) or disable multi\-thread
|
|
downloads with \f[C]\-\-multi\-thread\-streams 0\f[R]
|
|
.SS \[en]multi\-thread\-streams=N
|
|
.PP
|
|
When using multi thread downloads (see above
|
|
\f[C]\-\-multi\-thread\-cutoff\f[R]) this sets the maximum number of
|
|
streams to use.
|
|
Set to \f[C]0\f[R] to disable multi thread downloads (Default 4).
|
|
.PP
|
|
Exactly how many streams rclone uses for the download depends on the
|
|
size of the file.
|
|
To calculate the number of download streams Rclone divides the size of
|
|
the file by the \f[C]\-\-multi\-thread\-cutoff\f[R] and rounds up, up to
|
|
the maximum set with \f[C]\-\-multi\-thread\-streams\f[R].
|
|
.PP
|
|
So if \f[C]\-\-multi\-thread\-cutoff 250MB\f[R] and
|
|
\f[C]\-\-multi\-thread\-streams 4\f[R] are in effect (the defaults):
|
|
.IP \[bu] 2
|
|
0MB..250MB files will be downloaded with 1 stream
|
|
.IP \[bu] 2
|
|
250MB..500MB files will be downloaded with 2 streams
|
|
.IP \[bu] 2
|
|
500MB..750MB files will be downloaded with 3 streams
|
|
.IP \[bu] 2
|
|
750MB+ files will be downloaded with 4 streams
|
|
.SS \[en]no\-check\-dest
|
|
.PP
|
|
The \f[C]\-\-no\-check\-dest\f[R] can be used with \f[C]move\f[R] or
|
|
\f[C]copy\f[R] and it causes rclone not to check the destination at all
|
|
when copying files.
|
|
.PP
|
|
This means that:
|
|
.IP \[bu] 2
|
|
the destination is not listed minimising the API calls
|
|
.IP \[bu] 2
|
|
files are always transferred
|
|
.IP \[bu] 2
|
|
this can cause duplicates on remotes which allow it (eg Google Drive)
|
|
.IP \[bu] 2
|
|
\f[C]\-\-retries 1\f[R] is recommended otherwise you\[cq]ll transfer
|
|
everything again on a retry
|
|
.PP
|
|
This flag is useful to minimise the transactions if you know that none
|
|
of the files are on the destination.
|
|
.PP
|
|
This is a specialized flag which should be ignored by most users!
|
|
.SS \[en]no\-gzip\-encoding
|
|
.PP
|
|
Don\[cq]t set \f[C]Accept\-Encoding: gzip\f[R].
|
|
This means that rclone won\[cq]t ask the server for compressed files
|
|
automatically.
|
|
Useful if you\[cq]ve set the server to return files with
|
|
\f[C]Content\-Encoding: gzip\f[R] but you uploaded compressed files.
|
|
.PP
|
|
There is no need to set this in normal operation, and doing so will
|
|
decrease the network transfer efficiency of rclone.
|
|
.SS \[en]no\-traverse
|
|
.PP
|
|
The \f[C]\-\-no\-traverse\f[R] flag controls whether the destination
|
|
file system is traversed when using the \f[C]copy\f[R] or \f[C]move\f[R]
|
|
commands.
|
|
\f[C]\-\-no\-traverse\f[R] is not compatible with \f[C]sync\f[R] and
|
|
will be ignored if you supply it with \f[C]sync\f[R].
|
|
.PP
|
|
If you are only copying a small number of files (or are filtering most
|
|
of the files) and/or have a large number of files on the destination
|
|
then \f[C]\-\-no\-traverse\f[R] will stop rclone listing the destination
|
|
and save time.
|
|
.PP
|
|
However, if you are copying a large number of files, especially if you
|
|
are doing a copy where lots of the files under consideration haven\[cq]t
|
|
changed and won\[cq]t need copying then you shouldn\[cq]t use
|
|
\f[C]\-\-no\-traverse\f[R].
|
|
.PP
|
|
See rclone copy (https://rclone.org/commands/rclone_copy/) for an
|
|
example of how to use it.
|
|
.SS \[en]no\-unicode\-normalization
|
|
.PP
|
|
Don\[cq]t normalize unicode characters in filenames during the sync
|
|
routine.
|
|
.PP
|
|
Sometimes, an operating system will store filenames containing unicode
|
|
parts in their decomposed form (particularly macOS).
|
|
Some cloud storage systems will then recompose the unicode, resulting in
|
|
duplicate files if the data is ever copied back to a local filesystem.
|
|
.PP
|
|
Using this flag will disable that functionality, treating each unicode
|
|
character as unique.
|
|
For example, by default e\[u0301] and \['e] will be normalized into the
|
|
same character.
|
|
With \f[C]\-\-no\-unicode\-normalization\f[R] they will be treated as
|
|
unique characters.
|
|
.SS \[en]no\-update\-modtime
|
|
.PP
|
|
When using this flag, rclone won\[cq]t update modification times of
|
|
remote files if they are incorrect as it would normally.
|
|
.PP
|
|
This can be used if the remote is being synced with another tool also
|
|
(eg the Google Drive client).
|
|
.SS \[en]order\-by string
|
|
.PP
|
|
The \f[C]\-\-order\-by\f[R] flag controls the order in which files in
|
|
the backlog are processed in \f[C]rclone sync\f[R],
|
|
\f[C]rclone copy\f[R] and \f[C]rclone move\f[R].
|
|
.PP
|
|
The order by string is constructed like this.
|
|
The first part describes what aspect is being measured:
|
|
.IP \[bu] 2
|
|
\f[C]size\f[R] \- order by the size of the files
|
|
.IP \[bu] 2
|
|
\f[C]name\f[R] \- order by the full path of the files
|
|
.IP \[bu] 2
|
|
\f[C]modtime\f[R] \- order by the modification date of the files
|
|
.PP
|
|
This can have a modifier appended with a comma:
|
|
.IP \[bu] 2
|
|
\f[C]ascending\f[R] or \f[C]asc\f[R] \- order so that the smallest (or
|
|
oldest) is processed first
|
|
.IP \[bu] 2
|
|
\f[C]descending\f[R] or \f[C]desc\f[R] \- order so that the largest (or
|
|
newest) is processed first
|
|
.IP \[bu] 2
|
|
\f[C]mixed\f[R] \- order so that the smallest is processed first for
|
|
some threads and the largest for others
|
|
.PP
|
|
If the modifier is \f[C]mixed\f[R] then it can have an optional
|
|
percentage (which defaults to \f[C]50\f[R]), eg \f[C]size,mixed,25\f[R]
|
|
which means that 25% of the threads should be taking the smallest items
|
|
and 75% the largest.
|
|
The threads which take the smallest first will always take the smallest
|
|
first and likewise the largest first threads.
|
|
The \f[C]mixed\f[R] mode can be useful to minimise the transfer time
|
|
when you are transferring a mixture of large and small files \- the
|
|
large files are guaranteed upload threads and bandwidth and the small
|
|
files will be processed continuously.
|
|
.PP
|
|
If no modifier is supplied then the order is \f[C]ascending\f[R].
|
|
.PP
|
|
For example
|
|
.IP \[bu] 2
|
|
\f[C]\-\-order\-by size,desc\f[R] \- send the largest files first
|
|
.IP \[bu] 2
|
|
\f[C]\-\-order\-by modtime,ascending\f[R] \- send the oldest files first
|
|
.IP \[bu] 2
|
|
\f[C]\-\-order\-by name\f[R] \- send the files with alphabetically by
|
|
path first
|
|
.PP
|
|
If the \f[C]\-\-order\-by\f[R] flag is not supplied or it is supplied
|
|
with an empty string then the default ordering will be used which is as
|
|
scanned.
|
|
With \f[C]\-\-checkers 1\f[R] this is mostly alphabetical, however with
|
|
the default \f[C]\-\-checkers 8\f[R] it is somewhat random.
|
|
.SS Limitations
|
|
.PP
|
|
The \f[C]\-\-order\-by\f[R] flag does not do a separate pass over the
|
|
data.
|
|
This means that it may transfer some files out of the order specified if
|
|
.IP \[bu] 2
|
|
there are no files in the backlog or the source has not been fully
|
|
scanned yet
|
|
.IP \[bu] 2
|
|
there are more than \[en]max\-backlog files in the backlog
|
|
.PP
|
|
Rclone will do its best to transfer the best file it has so in practice
|
|
this should not cause a problem.
|
|
Think of \f[C]\-\-order\-by\f[R] as being more of a best efforts flag
|
|
rather than a perfect ordering.
|
|
.SS \[en]password\-command SpaceSepList
|
|
.PP
|
|
This flag supplies a program which should supply the config password
|
|
when run.
|
|
This is an alternative to rclone prompting for the password or setting
|
|
the \f[C]RCLONE_CONFIG_PASS\f[R] variable.
|
|
.PP
|
|
The argument to this should be a command with a space separated list of
|
|
arguments.
|
|
If one of the arguments has a space in then enclose it in
|
|
\f[C]\[dq]\f[R], if you want a literal \f[C]\[dq]\f[R] in an argument
|
|
then enclose the argument in \f[C]\[dq]\f[R] and double the
|
|
\f[C]\[dq]\f[R].
|
|
See CSV encoding (https://godoc.org/encoding/csv) for more info.
|
|
.PP
|
|
Eg
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
\-\-password\-command echo hello
|
|
\-\-password\-command echo \[dq]hello with space\[dq]
|
|
\-\-password\-command echo \[dq]hello with \[dq]\[dq]quotes\[dq]\[dq] and space\[dq]
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
See the Configuration Encryption for more info.
|
|
.PP
|
|
See a Windows PowerShell example on the
|
|
Wiki (https://github.com/rclone/rclone/wiki/Windows-Powershell-use-rclone-password-command-for-Config-file-password).
|
|
.SS \-P, \[en]progress
|
|
.PP
|
|
This flag makes rclone update the stats in a static block in the
|
|
terminal providing a realtime overview of the transfer.
|
|
.PP
|
|
Any log messages will scroll above the static block.
|
|
Log messages will push the static block down to the bottom of the
|
|
terminal where it will stay.
|
|
.PP
|
|
Normally this is updated every 500mS but this period can be overridden
|
|
with the \f[C]\-\-stats\f[R] flag.
|
|
.PP
|
|
This can be used with the \f[C]\-\-stats\-one\-line\f[R] flag for a
|
|
simpler display.
|
|
.PP
|
|
Note: On Windows until this
|
|
bug (https://github.com/Azure/go-ansiterm/issues/26) is fixed all
|
|
non\-ASCII characters will be replaced with \f[C].\f[R] when
|
|
\f[C]\-\-progress\f[R] is in use.
|
|
.SS \-q, \[en]quiet
|
|
.PP
|
|
This flag will limit rclone\[cq]s output to error messages only.
|
|
.SS \[en]retries int
|
|
.PP
|
|
Retry the entire sync if it fails this many times it fails (default 3).
|
|
.PP
|
|
Some remotes can be unreliable and a few retries help pick up the files
|
|
which didn\[cq]t get transferred because of errors.
|
|
.PP
|
|
Disable retries with \f[C]\-\-retries 1\f[R].
|
|
.SS \[en]retries\-sleep=TIME
|
|
.PP
|
|
This sets the interval between each retry specified by
|
|
\f[C]\-\-retries\f[R]
|
|
.PP
|
|
The default is \f[C]0\f[R].
|
|
Use \f[C]0\f[R] to disable.
|
|
.SS \[en]size\-only
|
|
.PP
|
|
Normally rclone will look at modification time and size of files to see
|
|
if they are equal.
|
|
If you set this flag then rclone will check only the size.
|
|
.PP
|
|
This can be useful transferring files from Dropbox which have been
|
|
modified by the desktop sync client which doesn\[cq]t set checksums of
|
|
modification times in the same way as rclone.
|
|
.SS \[en]stats=TIME
|
|
.PP
|
|
Commands which transfer data (\f[C]sync\f[R], \f[C]copy\f[R],
|
|
\f[C]copyto\f[R], \f[C]move\f[R], \f[C]moveto\f[R]) will print data
|
|
transfer stats at regular intervals to show their progress.
|
|
.PP
|
|
This sets the interval.
|
|
.PP
|
|
The default is \f[C]1m\f[R].
|
|
Use \f[C]0\f[R] to disable.
|
|
.PP
|
|
If you set the stats interval then all commands can show stats.
|
|
This can be useful when running other commands, \f[C]check\f[R] or
|
|
\f[C]mount\f[R] for example.
|
|
.PP
|
|
Stats are logged at \f[C]INFO\f[R] level by default which means they
|
|
won\[cq]t show at default log level \f[C]NOTICE\f[R].
|
|
Use \f[C]\-\-stats\-log\-level NOTICE\f[R] or \f[C]\-v\f[R] to make them
|
|
show.
|
|
See the Logging section for more info on log levels.
|
|
.PP
|
|
Note that on macOS you can send a SIGINFO (which is normally ctrl\-T in
|
|
the terminal) to make the stats print immediately.
|
|
.SS \[en]stats\-file\-name\-length integer
|
|
.PP
|
|
By default, the \f[C]\-\-stats\f[R] output will truncate file names and
|
|
paths longer than 40 characters.
|
|
This is equivalent to providing
|
|
\f[C]\-\-stats\-file\-name\-length 40\f[R].
|
|
Use \f[C]\-\-stats\-file\-name\-length 0\f[R] to disable any truncation
|
|
of file names printed by stats.
|
|
.SS \[en]stats\-log\-level string
|
|
.PP
|
|
Log level to show \f[C]\-\-stats\f[R] output at.
|
|
This can be \f[C]DEBUG\f[R], \f[C]INFO\f[R], \f[C]NOTICE\f[R], or
|
|
\f[C]ERROR\f[R].
|
|
The default is \f[C]INFO\f[R].
|
|
This means at the default level of logging which is \f[C]NOTICE\f[R] the
|
|
stats won\[cq]t show \- if you want them to then use
|
|
\f[C]\-\-stats\-log\-level NOTICE\f[R].
|
|
See the Logging section for more info on log levels.
|
|
.SS \[en]stats\-one\-line
|
|
.PP
|
|
When this is specified, rclone condenses the stats into a single line
|
|
showing the most important stats only.
|
|
.SS \[en]stats\-one\-line\-date
|
|
.PP
|
|
When this is specified, rclone enables the single\-line stats and
|
|
prepends the display with a date string.
|
|
The default is \f[C]2006/01/02 15:04:05 \-\f[R]
|
|
.SS \[en]stats\-one\-line\-date\-format
|
|
.PP
|
|
When this is specified, rclone enables the single\-line stats and
|
|
prepends the display with a user\-supplied date string.
|
|
The date string MUST be enclosed in quotes.
|
|
Follow golang specs (https://golang.org/pkg/time/#Time.Format) for date
|
|
formatting syntax.
|
|
.SS \[en]stats\-unit=bits|bytes
|
|
.PP
|
|
By default, data transfer rates will be printed in bytes/second.
|
|
.PP
|
|
This option allows the data rate to be printed in bits/second.
|
|
.PP
|
|
Data transfer volume will still be reported in bytes.
|
|
.PP
|
|
The rate is reported as a binary unit, not SI unit.
|
|
So 1 Mbit/s equals 1,048,576 bits/s and not 1,000,000 bits/s.
|
|
.PP
|
|
The default is \f[C]bytes\f[R].
|
|
.SS \[en]suffix=SUFFIX
|
|
.PP
|
|
When using \f[C]sync\f[R], \f[C]copy\f[R] or \f[C]move\f[R] any files
|
|
which would have been overwritten or deleted will have the suffix added
|
|
to them.
|
|
If there is a file with the same path (after the suffix has been added),
|
|
then it will be overwritten.
|
|
.PP
|
|
The remote in use must support server side move or copy and you must use
|
|
the same remote as the destination of the sync.
|
|
.PP
|
|
This is for use with files to add the suffix in the current directory or
|
|
with \f[C]\-\-backup\-dir\f[R].
|
|
See \f[C]\-\-backup\-dir\f[R] for more info.
|
|
.PP
|
|
For example
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone sync /path/to/local/file remote:current \-\-suffix .bak
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
will sync \f[C]/path/to/local\f[R] to \f[C]remote:current\f[R], but for
|
|
any files which would have been updated or deleted have .bak added.
|
|
.SS \[en]suffix\-keep\-extension
|
|
.PP
|
|
When using \f[C]\-\-suffix\f[R], setting this causes rclone put the
|
|
SUFFIX before the extension of the files that it backs up rather than
|
|
after.
|
|
.PP
|
|
So let\[cq]s say we had \f[C]\-\-suffix \-2019\-01\-01\f[R], without the
|
|
flag \f[C]file.txt\f[R] would be backed up to
|
|
\f[C]file.txt\-2019\-01\-01\f[R] and with the flag it would be backed up
|
|
to \f[C]file\-2019\-01\-01.txt\f[R].
|
|
This can be helpful to make sure the suffixed files can still be opened.
|
|
.SS \[en]syslog
|
|
.PP
|
|
On capable OSes (not Windows or Plan9) send all log output to syslog.
|
|
.PP
|
|
This can be useful for running rclone in a script or
|
|
\f[C]rclone mount\f[R].
|
|
.SS \[en]syslog\-facility string
|
|
.PP
|
|
If using \f[C]\-\-syslog\f[R] this sets the syslog facility (eg
|
|
\f[C]KERN\f[R], \f[C]USER\f[R]).
|
|
See \f[C]man syslog\f[R] for a list of possible facilities.
|
|
The default facility is \f[C]DAEMON\f[R].
|
|
.SS \[en]tpslimit float
|
|
.PP
|
|
Limit HTTP transactions per second to this.
|
|
Default is 0 which is used to mean unlimited transactions per second.
|
|
.PP
|
|
For example to limit rclone to 10 HTTP transactions per second use
|
|
\f[C]\-\-tpslimit 10\f[R], or to 1 transaction every 2 seconds use
|
|
\f[C]\-\-tpslimit 0.5\f[R].
|
|
.PP
|
|
Use this when the number of transactions per second from rclone is
|
|
causing a problem with the cloud storage provider (eg getting you banned
|
|
or rate limited).
|
|
.PP
|
|
This can be very useful for \f[C]rclone mount\f[R] to control the
|
|
behaviour of applications using it.
|
|
.PP
|
|
See also \f[C]\-\-tpslimit\-burst\f[R].
|
|
.SS \[en]tpslimit\-burst int
|
|
.PP
|
|
Max burst of transactions for \f[C]\-\-tpslimit\f[R] (default
|
|
\f[C]1\f[R]).
|
|
.PP
|
|
Normally \f[C]\-\-tpslimit\f[R] will do exactly the number of
|
|
transaction per second specified.
|
|
However if you supply \f[C]\-\-tps\-burst\f[R] then rclone can save up
|
|
some transactions from when it was idle giving a burst of up to the
|
|
parameter supplied.
|
|
.PP
|
|
For example if you provide \f[C]\-\-tpslimit\-burst 10\f[R] then if
|
|
rclone has been idle for more than 10*\f[C]\-\-tpslimit\f[R] then it can
|
|
do 10 transactions very quickly before they are limited again.
|
|
.PP
|
|
This may be used to increase performance of \f[C]\-\-tpslimit\f[R]
|
|
without changing the long term average number of transactions per
|
|
second.
|
|
.SS \[en]track\-renames
|
|
.PP
|
|
By default, rclone doesn\[cq]t keep track of renamed files, so if you
|
|
rename a file locally then sync it to a remote, rclone will delete the
|
|
old file on the remote and upload a new copy.
|
|
.PP
|
|
If you use this flag, and the remote supports server side copy or server
|
|
side move, and the source and destination have a compatible hash, then
|
|
this will track renames during \f[C]sync\f[R] operations and perform
|
|
renaming server\-side.
|
|
.PP
|
|
Files will be matched by size and hash \- if both match then a rename
|
|
will be considered.
|
|
.PP
|
|
If the destination does not support server\-side copy or move, rclone
|
|
will fall back to the default behaviour and log an error level message
|
|
to the console.
|
|
Note: Encrypted destinations are not supported by
|
|
\f[C]\-\-track\-renames\f[R].
|
|
.PP
|
|
Note that \f[C]\-\-track\-renames\f[R] is incompatible with
|
|
\f[C]\-\-no\-traverse\f[R] and that it uses extra memory to keep track
|
|
of all the rename candidates.
|
|
.PP
|
|
Note also that \f[C]\-\-track\-renames\f[R] is incompatible with
|
|
\f[C]\-\-delete\-before\f[R] and will select \f[C]\-\-delete\-after\f[R]
|
|
instead of \f[C]\-\-delete\-during\f[R].
|
|
.SS \[en]track\-renames\-strategy (hash,modtime)
|
|
.PP
|
|
This option changes the matching criteria for
|
|
\f[C]\-\-track\-renames\f[R] to match by any combination of modtime,
|
|
hash, size.
|
|
Matching by size is always enabled no matter what option is selected
|
|
here.
|
|
This also means that it enables \f[C]\-\-track\-renames\f[R] support for
|
|
encrypted destinations.
|
|
If nothing is specified, the default option is matching by hashes.
|
|
.SS \[en]delete\-(before,during,after)
|
|
.PP
|
|
This option allows you to specify when files on your destination are
|
|
deleted when you sync folders.
|
|
.PP
|
|
Specifying the value \f[C]\-\-delete\-before\f[R] will delete all files
|
|
present on the destination, but not on the source \f[I]before\f[R]
|
|
starting the transfer of any new or updated files.
|
|
This uses two passes through the file systems, one for the deletions and
|
|
one for the copies.
|
|
.PP
|
|
Specifying \f[C]\-\-delete\-during\f[R] will delete files while checking
|
|
and uploading files.
|
|
This is the fastest option and uses the least memory.
|
|
.PP
|
|
Specifying \f[C]\-\-delete\-after\f[R] (the default value) will delay
|
|
deletion of files until all new/updated files have been successfully
|
|
transferred.
|
|
The files to be deleted are collected in the copy pass then deleted
|
|
after the copy pass has completed successfully.
|
|
The files to be deleted are held in memory so this mode may use more
|
|
memory.
|
|
This is the safest mode as it will only delete files if there have been
|
|
no errors subsequent to that.
|
|
If there have been errors before the deletions start then you will get
|
|
the message \f[C]not deleting files as there were IO errors\f[R].
|
|
.SS \[en]fast\-list
|
|
.PP
|
|
When doing anything which involves a directory listing (eg
|
|
\f[C]sync\f[R], \f[C]copy\f[R], \f[C]ls\f[R] \- in fact nearly every
|
|
command), rclone normally lists a directory and processes it before
|
|
using more directory lists to process any subdirectories.
|
|
This can be parallelised and works very quickly using the least amount
|
|
of memory.
|
|
.PP
|
|
However, some remotes have a way of listing all files beneath a
|
|
directory in one (or a small number) of transactions.
|
|
These tend to be the bucket based remotes (eg S3, B2, GCS, Swift,
|
|
Hubic).
|
|
.PP
|
|
If you use the \f[C]\-\-fast\-list\f[R] flag then rclone will use this
|
|
method for listing directories.
|
|
This will have the following consequences for the listing:
|
|
.IP \[bu] 2
|
|
It \f[B]will\f[R] use fewer transactions (important if you pay for them)
|
|
.IP \[bu] 2
|
|
It \f[B]will\f[R] use more memory.
|
|
Rclone has to load the whole listing into memory.
|
|
.IP \[bu] 2
|
|
It \f[I]may\f[R] be faster because it uses fewer transactions
|
|
.IP \[bu] 2
|
|
It \f[I]may\f[R] be slower because it can\[cq]t be parallelized
|
|
.PP
|
|
rclone should always give identical results with and without
|
|
\f[C]\-\-fast\-list\f[R].
|
|
.PP
|
|
If you pay for transactions and can fit your entire sync listing into
|
|
memory then \f[C]\-\-fast\-list\f[R] is recommended.
|
|
If you have a very big sync to do then don\[cq]t use
|
|
\f[C]\-\-fast\-list\f[R] otherwise you will run out of memory.
|
|
.PP
|
|
If you use \f[C]\-\-fast\-list\f[R] on a remote which doesn\[cq]t
|
|
support it, then rclone will just ignore it.
|
|
.SS \[en]timeout=TIME
|
|
.PP
|
|
This sets the IO idle timeout.
|
|
If a transfer has started but then becomes idle for this long it is
|
|
considered broken and disconnected.
|
|
.PP
|
|
The default is \f[C]5m\f[R].
|
|
Set to \f[C]0\f[R] to disable.
|
|
.SS \[en]transfers=N
|
|
.PP
|
|
The number of file transfers to run in parallel.
|
|
It can sometimes be useful to set this to a smaller number if the remote
|
|
is giving a lot of timeouts or bigger if you have lots of bandwidth and
|
|
a fast remote.
|
|
.PP
|
|
The default is to run 4 file transfers in parallel.
|
|
.SS \-u, \[en]update
|
|
.PP
|
|
This forces rclone to skip any files which exist on the destination and
|
|
have a modified time that is newer than the source file.
|
|
.PP
|
|
This can be useful when transferring to a remote which doesn\[cq]t
|
|
support mod times directly (or when using
|
|
\f[C]\-\-use\-server\-modtime\f[R] to avoid extra API calls) as it is
|
|
more accurate than a \f[C]\-\-size\-only\f[R] check and faster than
|
|
using \f[C]\-\-checksum\f[R].
|
|
.PP
|
|
If an existing destination file has a modification time equal (within
|
|
the computed modify window precision) to the source file\[cq]s, it will
|
|
be updated if the sizes are different.
|
|
If \f[C]\-\-checksum\f[R] is set then rclone will update the destination
|
|
if the checksums differ too.
|
|
.PP
|
|
If an existing destination file is older than the source file then it
|
|
will be updated if the size or checksum differs from the source file.
|
|
.PP
|
|
On remotes which don\[cq]t support mod time directly (or when using
|
|
\f[C]\-\-use\-server\-modtime\f[R]) the time checked will be the
|
|
uploaded time.
|
|
This means that if uploading to one of these remotes, rclone will skip
|
|
any files which exist on the destination and have an uploaded time that
|
|
is newer than the modification time of the source file.
|
|
.SS \[en]use\-mmap
|
|
.PP
|
|
If this flag is set then rclone will use anonymous memory allocated by
|
|
mmap on Unix based platforms and VirtualAlloc on Windows for its
|
|
transfer buffers (size controlled by \f[C]\-\-buffer\-size\f[R]).
|
|
Memory allocated like this does not go on the Go heap and can be
|
|
returned to the OS immediately when it is finished with.
|
|
.PP
|
|
If this flag is not set then rclone will allocate and free the buffers
|
|
using the Go memory allocator which may use more memory as memory pages
|
|
are returned less aggressively to the OS.
|
|
.PP
|
|
It is possible this does not work well on all platforms so it is
|
|
disabled by default; in the future it may be enabled by default.
|
|
.SS \[en]use\-server\-modtime
|
|
.PP
|
|
Some object\-store backends (e.g, Swift, S3) do not preserve file
|
|
modification times (modtime).
|
|
On these backends, rclone stores the original modtime as additional
|
|
metadata on the object.
|
|
By default it will make an API call to retrieve the metadata when the
|
|
modtime is needed by an operation.
|
|
.PP
|
|
Use this flag to disable the extra API call and rely instead on the
|
|
server\[cq]s modified time.
|
|
In cases such as a local to remote sync using \f[C]\-\-update\f[R],
|
|
knowing the local file is newer than the time it was last uploaded to
|
|
the remote is sufficient.
|
|
In those cases, this flag can speed up the process and reduce the number
|
|
of API calls necessary.
|
|
.PP
|
|
Using this flag on a sync operation without also using
|
|
\f[C]\-\-update\f[R] would cause all files modified at any time other
|
|
than the last upload time to be uploaded again, which is probably not
|
|
what you want.
|
|
.SS \-v, \-vv, \[en]verbose
|
|
.PP
|
|
With \f[C]\-v\f[R] rclone will tell you about each file that is
|
|
transferred and a small number of significant events.
|
|
.PP
|
|
With \f[C]\-vv\f[R] rclone will become very verbose telling you about
|
|
every file it considers and transfers.
|
|
Please send bug reports with a log with this setting.
|
|
.SS \-V, \[en]version
|
|
.PP
|
|
Prints the version number
|
|
.SS SSL/TLS options
|
|
.PP
|
|
The outgoing SSL/TLS connections rclone makes can be controlled with
|
|
these options.
|
|
For example this can be very useful with the HTTP or WebDAV backends.
|
|
Rclone HTTP servers have their own set of configuration for SSL/TLS
|
|
which you can find in their documentation.
|
|
.SS \[en]ca\-cert string
|
|
.PP
|
|
This loads the PEM encoded certificate authority certificate and uses it
|
|
to verify the certificates of the servers rclone connects to.
|
|
.PP
|
|
If you have generated certificates signed with a local CA then you will
|
|
need this flag to connect to servers using those certificates.
|
|
.SS \[en]client\-cert string
|
|
.PP
|
|
This loads the PEM encoded client side certificate.
|
|
.PP
|
|
This is used for mutual TLS
|
|
authentication (https://en.wikipedia.org/wiki/Mutual_authentication).
|
|
.PP
|
|
The \f[C]\-\-client\-key\f[R] flag is required too when using this.
|
|
.SS \[en]client\-key string
|
|
.PP
|
|
This loads the PEM encoded client side private key used for mutual TLS
|
|
authentication.
|
|
Used in conjunction with \f[C]\-\-client\-cert\f[R].
|
|
.SS \[en]no\-check\-certificate=true/false
|
|
.PP
|
|
\f[C]\-\-no\-check\-certificate\f[R] controls whether a client verifies
|
|
the server\[cq]s certificate chain and host name.
|
|
If \f[C]\-\-no\-check\-certificate\f[R] is true, TLS accepts any
|
|
certificate presented by the server and any host name in that
|
|
certificate.
|
|
In this mode, TLS is susceptible to man\-in\-the\-middle attacks.
|
|
.PP
|
|
This option defaults to \f[C]false\f[R].
|
|
.PP
|
|
\f[B]This should be used only for testing.\f[R]
|
|
.SS Configuration Encryption
|
|
.PP
|
|
Your configuration file contains information for logging in to your
|
|
cloud services.
|
|
This means that you should keep your \f[C].rclone.conf\f[R] file in a
|
|
secure location.
|
|
.PP
|
|
If you are in an environment where that isn\[cq]t possible, you can add
|
|
a password to your configuration.
|
|
This means that you will have to supply the password every time you
|
|
start rclone.
|
|
.PP
|
|
To add a password to your rclone configuration, execute
|
|
\f[C]rclone config\f[R].
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
>rclone config
|
|
Current remotes:
|
|
|
|
e) Edit existing remote
|
|
n) New remote
|
|
d) Delete remote
|
|
s) Set configuration password
|
|
q) Quit config
|
|
e/n/d/s/q>
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
Go into \f[C]s\f[R], Set configuration password:
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
e/n/d/s/q> s
|
|
Your configuration is not encrypted.
|
|
If you add a password, you will protect your login information to cloud services.
|
|
a) Add Password
|
|
q) Quit to main menu
|
|
a/q> a
|
|
Enter NEW configuration password:
|
|
password:
|
|
Confirm NEW password:
|
|
password:
|
|
Password set
|
|
Your configuration is encrypted.
|
|
c) Change Password
|
|
u) Unencrypt configuration
|
|
q) Quit to main menu
|
|
c/u/q>
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
Your configuration is now encrypted, and every time you start rclone you
|
|
will have to supply the password.
|
|
See below for details.
|
|
In the same menu, you can change the password or completely remove
|
|
encryption from your configuration.
|
|
.PP
|
|
There is no way to recover the configuration if you lose your password.
|
|
.PP
|
|
rclone uses nacl
|
|
secretbox (https://godoc.org/golang.org/x/crypto/nacl/secretbox) which
|
|
in turn uses XSalsa20 and Poly1305 to encrypt and authenticate your
|
|
configuration with secret\-key cryptography.
|
|
The password is SHA\-256 hashed, which produces the key for secretbox.
|
|
The hashed password is not stored.
|
|
.PP
|
|
While this provides very good security, we do not recommend storing your
|
|
encrypted rclone configuration in public if it contains sensitive
|
|
information, maybe except if you use a very strong password.
|
|
.PP
|
|
If it is safe in your environment, you can set the
|
|
\f[C]RCLONE_CONFIG_PASS\f[R] environment variable to contain your
|
|
password, in which case it will be used for decrypting the
|
|
configuration.
|
|
.PP
|
|
You can set this for a session from a script.
|
|
For unix like systems save this to a file called
|
|
\f[C]set\-rclone\-password\f[R]:
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
#!/bin/echo Source this file don\[aq]t run it
|
|
|
|
read \-s RCLONE_CONFIG_PASS
|
|
export RCLONE_CONFIG_PASS
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
Then source the file when you want to use it.
|
|
From the shell you would do \f[C]source set\-rclone\-password\f[R].
|
|
It will then ask you for the password and set it in the environment
|
|
variable.
|
|
.PP
|
|
An alternate means of supplying the password is to provide a script
|
|
which will retrieve the password and print on standard output.
|
|
This script should have a fully specified path name and not rely on any
|
|
environment variables.
|
|
The script is supplied either via
|
|
\f[C]\-\-password\-command=\[dq]...\[dq]\f[R] command line argument or
|
|
via the \f[C]RCLONE_PASSWORD_COMMAND\f[R] environment variable.
|
|
.PP
|
|
One useful example of this is using the \f[C]passwordstore\f[R]
|
|
application to retrieve the password:
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
export RCLONE_PASSWORD_COMMAND=\[dq]pass rclone/config\[dq]
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
If the \f[C]passwordstore\f[R] password manager holds the password for
|
|
the rclone configuration, using the script method means the password is
|
|
primarily protected by the \f[C]passwordstore\f[R] system, and is never
|
|
embedded in the clear in scripts, nor available for examination using
|
|
the standard commands available.
|
|
It is quite possible with long running rclone sessions for copies of
|
|
passwords to be innocently captured in log files or terminal scroll
|
|
buffers, etc.
|
|
Using the script method of supplying the password enhances the security
|
|
of the config password considerably.
|
|
.PP
|
|
If you are running rclone inside a script, unless you are using the
|
|
\f[C]\-\-password\-command\f[R] method, you might want to disable
|
|
password prompts.
|
|
To do that, pass the parameter \f[C]\-\-ask\-password=false\f[R] to
|
|
rclone.
|
|
This will make rclone fail instead of asking for a password if
|
|
\f[C]RCLONE_CONFIG_PASS\f[R] doesn\[cq]t contain a valid password, and
|
|
\f[C]\-\-password\-command\f[R] has not been supplied.
|
|
.SS Developer options
|
|
.PP
|
|
These options are useful when developing or debugging rclone.
|
|
There are also some more remote specific options which aren\[cq]t
|
|
documented here which are used for testing.
|
|
These start with remote name eg \f[C]\-\-drive\-test\-option\f[R] \- see
|
|
the docs for the remote in question.
|
|
.SS \[en]cpuprofile=FILE
|
|
.PP
|
|
Write CPU profile to file.
|
|
This can be analysed with \f[C]go tool pprof\f[R].
|
|
.SS \[en]dump flag,flag,flag
|
|
.PP
|
|
The \f[C]\-\-dump\f[R] flag takes a comma separated list of flags to
|
|
dump info about.
|
|
.PP
|
|
Note that some headers including \f[C]Accept\-Encoding\f[R] as shown may
|
|
not be correct in the request and the response may not show
|
|
\f[C]Content\-Encoding\f[R] if the go standard libraries auto gzip
|
|
encoding was in effect.
|
|
In this case the body of the request will be gunzipped before showing
|
|
it.
|
|
.PP
|
|
The available flags are:
|
|
.SS \[en]dump headers
|
|
.PP
|
|
Dump HTTP headers with \f[C]Authorization:\f[R] lines removed.
|
|
May still contain sensitive info.
|
|
Can be very verbose.
|
|
Useful for debugging only.
|
|
.PP
|
|
Use \f[C]\-\-dump auth\f[R] if you do want the \f[C]Authorization:\f[R]
|
|
headers.
|
|
.SS \[en]dump bodies
|
|
.PP
|
|
Dump HTTP headers and bodies \- may contain sensitive info.
|
|
Can be very verbose.
|
|
Useful for debugging only.
|
|
.PP
|
|
Note that the bodies are buffered in memory so don\[cq]t use this for
|
|
enormous files.
|
|
.SS \[en]dump requests
|
|
.PP
|
|
Like \f[C]\-\-dump bodies\f[R] but dumps the request bodies and the
|
|
response headers.
|
|
Useful for debugging download problems.
|
|
.SS \[en]dump responses
|
|
.PP
|
|
Like \f[C]\-\-dump bodies\f[R] but dumps the response bodies and the
|
|
request headers.
|
|
Useful for debugging upload problems.
|
|
.SS \[en]dump auth
|
|
.PP
|
|
Dump HTTP headers \- will contain sensitive info such as
|
|
\f[C]Authorization:\f[R] headers \- use \f[C]\-\-dump headers\f[R] to
|
|
dump without \f[C]Authorization:\f[R] headers.
|
|
Can be very verbose.
|
|
Useful for debugging only.
|
|
.SS \[en]dump filters
|
|
.PP
|
|
Dump the filters to the output.
|
|
Useful to see exactly what include and exclude options are filtering on.
|
|
.SS \[en]dump goroutines
|
|
.PP
|
|
This dumps a list of the running go\-routines at the end of the command
|
|
to standard output.
|
|
.SS \[en]dump openfiles
|
|
.PP
|
|
This dumps a list of the open files at the end of the command.
|
|
It uses the \f[C]lsof\f[R] command to do that so you\[cq]ll need that
|
|
installed to use it.
|
|
.SS \[en]memprofile=FILE
|
|
.PP
|
|
Write memory profile to file.
|
|
This can be analysed with \f[C]go tool pprof\f[R].
|
|
.SS Filtering
|
|
.PP
|
|
For the filtering options
|
|
.IP \[bu] 2
|
|
\f[C]\-\-delete\-excluded\f[R]
|
|
.IP \[bu] 2
|
|
\f[C]\-\-filter\f[R]
|
|
.IP \[bu] 2
|
|
\f[C]\-\-filter\-from\f[R]
|
|
.IP \[bu] 2
|
|
\f[C]\-\-exclude\f[R]
|
|
.IP \[bu] 2
|
|
\f[C]\-\-exclude\-from\f[R]
|
|
.IP \[bu] 2
|
|
\f[C]\-\-include\f[R]
|
|
.IP \[bu] 2
|
|
\f[C]\-\-include\-from\f[R]
|
|
.IP \[bu] 2
|
|
\f[C]\-\-files\-from\f[R]
|
|
.IP \[bu] 2
|
|
\f[C]\-\-files\-from\-raw\f[R]
|
|
.IP \[bu] 2
|
|
\f[C]\-\-min\-size\f[R]
|
|
.IP \[bu] 2
|
|
\f[C]\-\-max\-size\f[R]
|
|
.IP \[bu] 2
|
|
\f[C]\-\-min\-age\f[R]
|
|
.IP \[bu] 2
|
|
\f[C]\-\-max\-age\f[R]
|
|
.IP \[bu] 2
|
|
\f[C]\-\-dump filters\f[R]
|
|
.PP
|
|
See the filtering section (https://rclone.org/filtering/).
|
|
.SS Remote control
|
|
.PP
|
|
For the remote control options and for instructions on how to remote
|
|
control rclone
|
|
.IP \[bu] 2
|
|
\f[C]\-\-rc\f[R]
|
|
.IP \[bu] 2
|
|
and anything starting with \f[C]\-\-rc\-\f[R]
|
|
.PP
|
|
See the remote control section (https://rclone.org/rc/).
|
|
.SS Logging
|
|
.PP
|
|
rclone has 4 levels of logging, \f[C]ERROR\f[R], \f[C]NOTICE\f[R],
|
|
\f[C]INFO\f[R] and \f[C]DEBUG\f[R].
|
|
.PP
|
|
By default, rclone logs to standard error.
|
|
This means you can redirect standard error and still see the normal
|
|
output of rclone commands (eg \f[C]rclone ls\f[R]).
|
|
.PP
|
|
By default, rclone will produce \f[C]Error\f[R] and \f[C]Notice\f[R]
|
|
level messages.
|
|
.PP
|
|
If you use the \f[C]\-q\f[R] flag, rclone will only produce
|
|
\f[C]Error\f[R] messages.
|
|
.PP
|
|
If you use the \f[C]\-v\f[R] flag, rclone will produce \f[C]Error\f[R],
|
|
\f[C]Notice\f[R] and \f[C]Info\f[R] messages.
|
|
.PP
|
|
If you use the \f[C]\-vv\f[R] flag, rclone will produce \f[C]Error\f[R],
|
|
\f[C]Notice\f[R], \f[C]Info\f[R] and \f[C]Debug\f[R] messages.
|
|
.PP
|
|
You can also control the log levels with the \f[C]\-\-log\-level\f[R]
|
|
flag.
|
|
.PP
|
|
If you use the \f[C]\-\-log\-file=FILE\f[R] option, rclone will redirect
|
|
\f[C]Error\f[R], \f[C]Info\f[R] and \f[C]Debug\f[R] messages along with
|
|
standard error to FILE.
|
|
.PP
|
|
If you use the \f[C]\-\-syslog\f[R] flag then rclone will log to syslog
|
|
and the \f[C]\-\-syslog\-facility\f[R] control which facility it uses.
|
|
.PP
|
|
Rclone prefixes all log messages with their level in capitals, eg INFO
|
|
which makes it easy to grep the log file for different kinds of
|
|
information.
|
|
.SS Exit Code
|
|
.PP
|
|
If any errors occur during the command execution, rclone will exit with
|
|
a non\-zero exit code.
|
|
This allows scripts to detect when rclone operations have failed.
|
|
.PP
|
|
During the startup phase, rclone will exit immediately if an error is
|
|
detected in the configuration.
|
|
There will always be a log message immediately before exiting.
|
|
.PP
|
|
When rclone is running it will accumulate errors as it goes along, and
|
|
only exit with a non\-zero exit code if (after retries) there were still
|
|
failed transfers.
|
|
For every error counted there will be a high priority log message
|
|
(visible with \f[C]\-q\f[R]) showing the message and which file caused
|
|
the problem.
|
|
A high priority message is also shown when starting a retry so the user
|
|
can see that any previous error messages may not be valid after the
|
|
retry.
|
|
If rclone has done a retry it will log a high priority message if the
|
|
retry was successful.
|
|
.SS List of exit codes
|
|
.IP \[bu] 2
|
|
\f[C]0\f[R] \- success
|
|
.IP \[bu] 2
|
|
\f[C]1\f[R] \- Syntax or usage error
|
|
.IP \[bu] 2
|
|
\f[C]2\f[R] \- Error not otherwise categorised
|
|
.IP \[bu] 2
|
|
\f[C]3\f[R] \- Directory not found
|
|
.IP \[bu] 2
|
|
\f[C]4\f[R] \- File not found
|
|
.IP \[bu] 2
|
|
\f[C]5\f[R] \- Temporary error (one that more retries might fix) (Retry
|
|
errors)
|
|
.IP \[bu] 2
|
|
\f[C]6\f[R] \- Less serious errors (like 461 errors from dropbox)
|
|
(NoRetry errors)
|
|
.IP \[bu] 2
|
|
\f[C]7\f[R] \- Fatal error (one that more retries won\[cq]t fix, like
|
|
account suspended) (Fatal errors)
|
|
.IP \[bu] 2
|
|
\f[C]8\f[R] \- Transfer exceeded \- limit set by \[en]max\-transfer
|
|
reached
|
|
.IP \[bu] 2
|
|
\f[C]9\f[R] \- Operation successful, but no files transferred
|
|
.SS Environment Variables
|
|
.PP
|
|
Rclone can be configured entirely using environment variables.
|
|
These can be used to set defaults for options or config file entries.
|
|
.SS Options
|
|
.PP
|
|
Every option in rclone can have its default set by environment variable.
|
|
.PP
|
|
To find the name of the environment variable, first, take the long
|
|
option name, strip the leading \f[C]\-\-\f[R], change \f[C]\-\f[R] to
|
|
\f[C]_\f[R], make upper case and prepend \f[C]RCLONE_\f[R].
|
|
.PP
|
|
For example, to always set \f[C]\-\-stats 5s\f[R], set the environment
|
|
variable \f[C]RCLONE_STATS=5s\f[R].
|
|
If you set stats on the command line this will override the environment
|
|
variable setting.
|
|
.PP
|
|
Or to always use the trash in drive \f[C]\-\-drive\-use\-trash\f[R], set
|
|
\f[C]RCLONE_DRIVE_USE_TRASH=true\f[R].
|
|
.PP
|
|
The same parser is used for the options and the environment variables so
|
|
they take exactly the same form.
|
|
.SS Config file
|
|
.PP
|
|
You can set defaults for values in the config file on an individual
|
|
remote basis.
|
|
If you want to use this feature, you will need to discover the name of
|
|
the config items that you want.
|
|
The easiest way is to run through \f[C]rclone config\f[R] by hand, then
|
|
look in the config file to see what the values are (the config file can
|
|
be found by looking at the help for \f[C]\-\-config\f[R] in
|
|
\f[C]rclone help\f[R]).
|
|
.PP
|
|
To find the name of the environment variable, you need to set, take
|
|
\f[C]RCLONE_CONFIG_\f[R] + name of remote + \f[C]_\f[R] + name of config
|
|
file option and make it all uppercase.
|
|
.PP
|
|
For example, to configure an S3 remote named \f[C]mys3:\f[R] without a
|
|
config file (using unix ways of setting environment variables):
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
$ export RCLONE_CONFIG_MYS3_TYPE=s3
|
|
$ export RCLONE_CONFIG_MYS3_ACCESS_KEY_ID=XXX
|
|
$ export RCLONE_CONFIG_MYS3_SECRET_ACCESS_KEY=XXX
|
|
$ rclone lsd MYS3:
|
|
\-1 2016\-09\-21 12:54:21 \-1 my\-bucket
|
|
$ rclone listremotes | grep mys3
|
|
mys3:
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
Note that if you want to create a remote using environment variables you
|
|
must create the \f[C]..._TYPE\f[R] variable as above.
|
|
.SS Other environment variables
|
|
.IP \[bu] 2
|
|
\f[C]RCLONE_CONFIG_PASS\f[R] set to contain your config file password
|
|
(see Configuration Encryption section)
|
|
.IP \[bu] 2
|
|
\f[C]HTTP_PROXY\f[R], \f[C]HTTPS_PROXY\f[R] and \f[C]NO_PROXY\f[R] (or
|
|
the lowercase versions thereof).
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
\f[C]HTTPS_PROXY\f[R] takes precedence over \f[C]HTTP_PROXY\f[R] for
|
|
https requests.
|
|
.IP \[bu] 2
|
|
The environment values may be either a complete URL or a
|
|
\[lq]host[:port]\[rq] for, in which case the \[lq]http\[rq] scheme is
|
|
assumed.
|
|
.RE
|
|
.SH Configuring rclone on a remote / headless machine
|
|
.PP
|
|
Some of the configurations (those involving oauth2) require an Internet
|
|
connected web browser.
|
|
.PP
|
|
If you are trying to set rclone up on a remote or headless box with no
|
|
browser available on it (eg a NAS or a server in a datacenter) then you
|
|
will need to use an alternative means of configuration.
|
|
There are two ways of doing it, described below.
|
|
.SS Configuring using rclone authorize
|
|
.PP
|
|
On the headless box run \f[C]rclone\f[R] config but answer \f[C]N\f[R]
|
|
to the \f[C]Use auto config?\f[R] question.
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
\&...
|
|
Remote config
|
|
Use auto config?
|
|
* Say Y if not sure
|
|
* Say N if you are working on a remote or headless machine
|
|
y) Yes (default)
|
|
n) No
|
|
y/n> n
|
|
For this to work, you will need rclone available on a machine that has
|
|
a web browser available.
|
|
|
|
For more help and alternate methods see: https://rclone.org/remote_setup/
|
|
|
|
Execute the following on the machine with the web browser (same rclone
|
|
version recommended):
|
|
|
|
rclone authorize \[dq]amazon cloud drive\[dq]
|
|
|
|
Then paste the result below:
|
|
result>
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
Then on your main desktop machine
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone authorize \[dq]amazon cloud drive\[dq]
|
|
If your browser doesn\[aq]t open automatically go to the following link: http://127.0.0.1:53682/auth
|
|
Log in and authorize rclone for access
|
|
Waiting for code...
|
|
Got code
|
|
Paste the following into your remote machine \-\-\->
|
|
SECRET_TOKEN
|
|
<\-\-\-End paste
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
Then back to the headless box, paste in the code
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
result> SECRET_TOKEN
|
|
\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-
|
|
[acd12]
|
|
client_id =
|
|
client_secret =
|
|
token = SECRET_TOKEN
|
|
\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-
|
|
y) Yes this is OK
|
|
e) Edit this remote
|
|
d) Delete this remote
|
|
y/e/d>
|
|
\f[R]
|
|
.fi
|
|
.SS Configuring by copying the config file
|
|
.PP
|
|
Rclone stores all of its config in a single configuration file.
|
|
This can easily be copied to configure a remote rclone.
|
|
.PP
|
|
So first configure rclone on your desktop machine with
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone config
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
to set up the config file.
|
|
.PP
|
|
Find the config file by running \f[C]rclone config file\f[R], for
|
|
example
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
$ rclone config file
|
|
Configuration file is stored at:
|
|
/home/user/.rclone.conf
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
Now transfer it to the remote box (scp, cut paste, ftp, sftp etc) and
|
|
place it in the correct place (use \f[C]rclone config file\f[R] on the
|
|
remote box to find out where).
|
|
.SH Filtering, includes and excludes
|
|
.PP
|
|
Rclone has a sophisticated set of include and exclude rules.
|
|
Some of these are based on patterns and some on other things like file
|
|
size.
|
|
.PP
|
|
The filters are applied for the \f[C]copy\f[R], \f[C]sync\f[R],
|
|
\f[C]move\f[R], \f[C]ls\f[R], \f[C]lsl\f[R], \f[C]md5sum\f[R],
|
|
\f[C]sha1sum\f[R], \f[C]size\f[R], \f[C]delete\f[R] and \f[C]check\f[R]
|
|
operations.
|
|
Note that \f[C]purge\f[R] does not obey the filters.
|
|
.PP
|
|
Each path as it passes through rclone is matched against the include and
|
|
exclude rules like \f[C]\-\-include\f[R], \f[C]\-\-exclude\f[R],
|
|
\f[C]\-\-include\-from\f[R], \f[C]\-\-exclude\-from\f[R],
|
|
\f[C]\-\-filter\f[R], or \f[C]\-\-filter\-from\f[R].
|
|
The simplest way to try them out is using the \f[C]ls\f[R] command, or
|
|
\f[C]\-\-dry\-run\f[R] together with \f[C]\-v\f[R].
|
|
\f[C]\-\-filter\-from\f[R], \f[C]\-\-exclude\-from\f[R],
|
|
\f[C]\-\-include\-from\f[R], \f[C]\-\-files\-from\f[R],
|
|
\f[C]\-\-files\-from\-raw\f[R] understand \f[C]\-\f[R] as a file name to
|
|
mean read from standard input.
|
|
.SS Patterns
|
|
.PP
|
|
The patterns used to match files for inclusion or exclusion are based on
|
|
\[lq]file globs\[rq] as used by the unix shell.
|
|
.PP
|
|
If the pattern starts with a \f[C]/\f[R] then it only matches at the top
|
|
level of the directory tree, \f[B]relative to the root of the
|
|
remote\f[R] (not necessarily the root of the local drive).
|
|
If it doesn\[cq]t start with \f[C]/\f[R] then it is matched starting at
|
|
the \f[B]end of the path\f[R], but it will only match a complete path
|
|
element:
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
file.jpg \- matches \[dq]file.jpg\[dq]
|
|
\- matches \[dq]directory/file.jpg\[dq]
|
|
\- doesn\[aq]t match \[dq]afile.jpg\[dq]
|
|
\- doesn\[aq]t match \[dq]directory/afile.jpg\[dq]
|
|
/file.jpg \- matches \[dq]file.jpg\[dq] in the root directory of the remote
|
|
\- doesn\[aq]t match \[dq]afile.jpg\[dq]
|
|
\- doesn\[aq]t match \[dq]directory/file.jpg\[dq]
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
\f[B]Important\f[R] Note that you must use \f[C]/\f[R] in patterns and
|
|
not \f[C]\[rs]\f[R] even if running on Windows.
|
|
.PP
|
|
A \f[C]*\f[R] matches anything but not a \f[C]/\f[R].
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
*.jpg \- matches \[dq]file.jpg\[dq]
|
|
\- matches \[dq]directory/file.jpg\[dq]
|
|
\- doesn\[aq]t match \[dq]file.jpg/something\[dq]
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
Use \f[C]**\f[R] to match anything, including slashes (\f[C]/\f[R]).
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
dir/** \- matches \[dq]dir/file.jpg\[dq]
|
|
\- matches \[dq]dir/dir1/dir2/file.jpg\[dq]
|
|
\- doesn\[aq]t match \[dq]directory/file.jpg\[dq]
|
|
\- doesn\[aq]t match \[dq]adir/file.jpg\[dq]
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
A \f[C]?\f[R] matches any character except a slash \f[C]/\f[R].
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
l?ss \- matches \[dq]less\[dq]
|
|
\- matches \[dq]lass\[dq]
|
|
\- doesn\[aq]t match \[dq]floss\[dq]
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
A \f[C][\f[R] and \f[C]]\f[R] together make a character class, such as
|
|
\f[C][a\-z]\f[R] or \f[C][aeiou]\f[R] or \f[C][[:alpha:]]\f[R].
|
|
See the go regexp docs (https://golang.org/pkg/regexp/syntax/) for more
|
|
info on these.
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
h[ae]llo \- matches \[dq]hello\[dq]
|
|
\- matches \[dq]hallo\[dq]
|
|
\- doesn\[aq]t match \[dq]hullo\[dq]
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
A \f[C]{\f[R] and \f[C]}\f[R] define a choice between elements.
|
|
It should contain a comma separated list of patterns, any of which might
|
|
match.
|
|
These patterns can contain wildcards.
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
{one,two}_potato \- matches \[dq]one_potato\[dq]
|
|
\- matches \[dq]two_potato\[dq]
|
|
\- doesn\[aq]t match \[dq]three_potato\[dq]
|
|
\- doesn\[aq]t match \[dq]_potato\[dq]
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
Special characters can be escaped with a \f[C]\[rs]\f[R] before them.
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
\[rs]*.jpg \- matches \[dq]*.jpg\[dq]
|
|
\[rs]\[rs].jpg \- matches \[dq]\[rs].jpg\[dq]
|
|
\[rs][one\[rs]].jpg \- matches \[dq][one].jpg\[dq]
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
Patterns are case sensitive unless the \f[C]\-\-ignore\-case\f[R] flag
|
|
is used.
|
|
.PP
|
|
Without \f[C]\-\-ignore\-case\f[R] (default)
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
potato \- matches \[dq]potato\[dq]
|
|
\- doesn\[aq]t match \[dq]POTATO\[dq]
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
With \f[C]\-\-ignore\-case\f[R]
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
potato \- matches \[dq]potato\[dq]
|
|
\- matches \[dq]POTATO\[dq]
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
Note also that rclone filter globs can only be used in one of the filter
|
|
command line flags, not in the specification of the remote, so
|
|
\f[C]rclone copy \[dq]remote:dir*.jpg\[dq] /path/to/dir\f[R] won\[cq]t
|
|
work \- what is required is
|
|
\f[C]rclone \-\-include \[dq]*.jpg\[dq] copy remote:dir /path/to/dir\f[R]
|
|
.SS Directories
|
|
.PP
|
|
Rclone keeps track of directories that could match any file patterns.
|
|
.PP
|
|
Eg if you add the include rule
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
/a/*.jpg
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
Rclone will synthesize the directory include rule
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
/a/
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
If you put any rules which end in \f[C]/\f[R] then it will only match
|
|
directories.
|
|
.PP
|
|
Directory matches are \f[B]only\f[R] used to optimise directory access
|
|
patterns \- you must still match the files that you want to match.
|
|
Directory matches won\[cq]t optimise anything on bucket based remotes
|
|
(eg s3, swift, google compute storage, b2) which don\[cq]t have a
|
|
concept of directory.
|
|
.SS Differences between rsync and rclone patterns
|
|
.PP
|
|
Rclone implements bash style \f[C]{a,b,c}\f[R] glob matching which rsync
|
|
doesn\[cq]t.
|
|
.PP
|
|
Rclone always does a wildcard match so \f[C]\[rs]\f[R] must always
|
|
escape a \f[C]\[rs]\f[R].
|
|
.SS How the rules are used
|
|
.PP
|
|
Rclone maintains a combined list of include rules and exclude rules.
|
|
.PP
|
|
Each file is matched in order, starting from the top, against the rule
|
|
in the list until it finds a match.
|
|
The file is then included or excluded according to the rule type.
|
|
.PP
|
|
If the matcher fails to find a match after testing against all the
|
|
entries in the list then the path is included.
|
|
.PP
|
|
For example given the following rules, \f[C]+\f[R] being include,
|
|
\f[C]\-\f[R] being exclude,
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
\- secret*.jpg
|
|
+ *.jpg
|
|
+ *.png
|
|
+ file2.avi
|
|
\- *
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
This would include
|
|
.IP \[bu] 2
|
|
\f[C]file1.jpg\f[R]
|
|
.IP \[bu] 2
|
|
\f[C]file3.png\f[R]
|
|
.IP \[bu] 2
|
|
\f[C]file2.avi\f[R]
|
|
.PP
|
|
This would exclude
|
|
.IP \[bu] 2
|
|
\f[C]secret17.jpg\f[R]
|
|
.IP \[bu] 2
|
|
non \f[C]*.jpg\f[R] and \f[C]*.png\f[R]
|
|
.PP
|
|
A similar process is done on directory entries before recursing into
|
|
them.
|
|
This only works on remotes which have a concept of directory (Eg local,
|
|
google drive, onedrive, amazon drive) and not on bucket based remotes
|
|
(eg s3, swift, google compute storage, b2).
|
|
.SS Adding filtering rules
|
|
.PP
|
|
Filtering rules are added with the following command line flags.
|
|
.SS Repeating options
|
|
.PP
|
|
You can repeat the following options to add more than one rule of that
|
|
type.
|
|
.IP \[bu] 2
|
|
\f[C]\-\-include\f[R]
|
|
.IP \[bu] 2
|
|
\f[C]\-\-include\-from\f[R]
|
|
.IP \[bu] 2
|
|
\f[C]\-\-exclude\f[R]
|
|
.IP \[bu] 2
|
|
\f[C]\-\-exclude\-from\f[R]
|
|
.IP \[bu] 2
|
|
\f[C]\-\-filter\f[R]
|
|
.IP \[bu] 2
|
|
\f[C]\-\-filter\-from\f[R]
|
|
.IP \[bu] 2
|
|
\f[C]\-\-filter\-from\-raw\f[R]
|
|
.PP
|
|
\f[B]Important\f[R] You should not use \f[C]\-\-include*\f[R] together
|
|
with \f[C]\-\-exclude*\f[R].
|
|
It may produce different results than you expected.
|
|
In that case try to use: \f[C]\-\-filter*\f[R].
|
|
.PP
|
|
Note that all the options of the same type are processed together in the
|
|
order above, regardless of what order they were placed on the command
|
|
line.
|
|
.PP
|
|
So all \f[C]\-\-include\f[R] options are processed first in the order
|
|
they appeared on the command line, then all \f[C]\-\-include\-from\f[R]
|
|
options etc.
|
|
.PP
|
|
To mix up the order includes and excludes, the \f[C]\-\-filter\f[R] flag
|
|
can be used.
|
|
.SS \f[C]\-\-exclude\f[R] \- Exclude files matching pattern
|
|
.PP
|
|
Add a single exclude rule with \f[C]\-\-exclude\f[R].
|
|
.PP
|
|
This flag can be repeated.
|
|
See above for the order the flags are processed in.
|
|
.PP
|
|
Eg \f[C]\-\-exclude *.bak\f[R] to exclude all bak files from the sync.
|
|
.SS \f[C]\-\-exclude\-from\f[R] \- Read exclude patterns from file
|
|
.PP
|
|
Add exclude rules from a file.
|
|
.PP
|
|
This flag can be repeated.
|
|
See above for the order the flags are processed in.
|
|
.PP
|
|
Prepare a file like this \f[C]exclude\-file.txt\f[R]
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
# a sample exclude rule file
|
|
*.bak
|
|
file2.jpg
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
Then use as \f[C]\-\-exclude\-from exclude\-file.txt\f[R].
|
|
This will sync all files except those ending in \f[C]bak\f[R] and
|
|
\f[C]file2.jpg\f[R].
|
|
.PP
|
|
This is useful if you have a lot of rules.
|
|
.SS \f[C]\-\-include\f[R] \- Include files matching pattern
|
|
.PP
|
|
Add a single include rule with \f[C]\-\-include\f[R].
|
|
.PP
|
|
This flag can be repeated.
|
|
See above for the order the flags are processed in.
|
|
.PP
|
|
Eg \f[C]\-\-include *.{png,jpg}\f[R] to include all \f[C]png\f[R] and
|
|
\f[C]jpg\f[R] files in the backup and no others.
|
|
.PP
|
|
This adds an implicit \f[C]\-\-exclude *\f[R] at the very end of the
|
|
filter list.
|
|
This means you can mix \f[C]\-\-include\f[R] and
|
|
\f[C]\-\-include\-from\f[R] with the other filters (eg
|
|
\f[C]\-\-exclude\f[R]) but you must include all the files you want in
|
|
the include statement.
|
|
If this doesn\[cq]t provide enough flexibility then you must use
|
|
\f[C]\-\-filter\-from\f[R].
|
|
.SS \f[C]\-\-include\-from\f[R] \- Read include patterns from file
|
|
.PP
|
|
Add include rules from a file.
|
|
.PP
|
|
This flag can be repeated.
|
|
See above for the order the flags are processed in.
|
|
.PP
|
|
Prepare a file like this \f[C]include\-file.txt\f[R]
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
# a sample include rule file
|
|
*.jpg
|
|
*.png
|
|
file2.avi
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
Then use as \f[C]\-\-include\-from include\-file.txt\f[R].
|
|
This will sync all \f[C]jpg\f[R], \f[C]png\f[R] files and
|
|
\f[C]file2.avi\f[R].
|
|
.PP
|
|
This is useful if you have a lot of rules.
|
|
.PP
|
|
This adds an implicit \f[C]\-\-exclude *\f[R] at the very end of the
|
|
filter list.
|
|
This means you can mix \f[C]\-\-include\f[R] and
|
|
\f[C]\-\-include\-from\f[R] with the other filters (eg
|
|
\f[C]\-\-exclude\f[R]) but you must include all the files you want in
|
|
the include statement.
|
|
If this doesn\[cq]t provide enough flexibility then you must use
|
|
\f[C]\-\-filter\-from\f[R].
|
|
.SS \f[C]\-\-filter\f[R] \- Add a file\-filtering rule
|
|
.PP
|
|
This can be used to add a single include or exclude rule.
|
|
Include rules start with \f[C]+\f[R] and exclude rules start with
|
|
\f[C]\-\f[R].
|
|
A special rule called \f[C]!\f[R] can be used to clear the existing
|
|
rules.
|
|
.PP
|
|
This flag can be repeated.
|
|
See above for the order the flags are processed in.
|
|
.PP
|
|
Eg \f[C]\-\-filter \[dq]\- *.bak\[dq]\f[R] to exclude all bak files from
|
|
the sync.
|
|
.SS \f[C]\-\-filter\-from\f[R] \- Read filtering patterns from a file
|
|
.PP
|
|
Add include/exclude rules from a file.
|
|
.PP
|
|
This flag can be repeated.
|
|
See above for the order the flags are processed in.
|
|
.PP
|
|
Prepare a file like this \f[C]filter\-file.txt\f[R]
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
# a sample filter rule file
|
|
\- secret*.jpg
|
|
+ *.jpg
|
|
+ *.png
|
|
+ file2.avi
|
|
\- /dir/Trash/**
|
|
+ /dir/**
|
|
# exclude everything else
|
|
\- *
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
Then use as \f[C]\-\-filter\-from filter\-file.txt\f[R].
|
|
The rules are processed in the order that they are defined.
|
|
.PP
|
|
This example will include all \f[C]jpg\f[R] and \f[C]png\f[R] files,
|
|
exclude any files matching \f[C]secret*.jpg\f[R] and include
|
|
\f[C]file2.avi\f[R].
|
|
It will also include everything in the directory \f[C]dir\f[R] at the
|
|
root of the sync, except \f[C]dir/Trash\f[R] which it will exclude.
|
|
Everything else will be excluded from the sync.
|
|
.SS \f[C]\-\-files\-from\f[R] \- Read list of source\-file names
|
|
.PP
|
|
This reads a list of file names from the file passed in and
|
|
\f[B]only\f[R] these files are transferred.
|
|
The \f[B]filtering rules are ignored\f[R] completely if you use this
|
|
option.
|
|
.PP
|
|
\f[C]\-\-files\-from\f[R] expects a list of files as its input.
|
|
Leading / trailing whitespace is stripped from the input lines and lines
|
|
starting with \f[C]#\f[R] and \f[C];\f[R] are ignored.
|
|
.PP
|
|
Rclone will traverse the file system if you use
|
|
\f[C]\-\-files\-from\f[R], effectively using the files in
|
|
\f[C]\-\-files\-from\f[R] as a set of filters.
|
|
Rclone will not error if any of the files are missing.
|
|
.PP
|
|
If you use \f[C]\-\-no\-traverse\f[R] as well as
|
|
\f[C]\-\-files\-from\f[R] then rclone will not traverse the destination
|
|
file system, it will find each file individually using approximately 1
|
|
API call.
|
|
This can be more efficient for small lists of files.
|
|
.PP
|
|
This option can be repeated to read from more than one file.
|
|
These are read in the order that they are placed on the command line.
|
|
.PP
|
|
Paths within the \f[C]\-\-files\-from\f[R] file will be interpreted as
|
|
starting with the root specified in the command.
|
|
Leading \f[C]/\f[R] characters are ignored.
|
|
See \[en]files\-from\-raw if you need the input to be processed in a raw
|
|
manner.
|
|
.PP
|
|
For example, suppose you had \f[C]files\-from.txt\f[R] with this
|
|
content:
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
# comment
|
|
file1.jpg
|
|
subdir/file2.jpg
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
You could then use it like this:
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone copy \-\-files\-from files\-from.txt /home/me/pics remote:pics
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
This will transfer these files only (if they exist)
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
/home/me/pics/file1.jpg \[->] remote:pics/file1.jpg
|
|
/home/me/pics/subdir/file2.jpg \[->] remote:pics/subdir/file2.jpg
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
To take a more complicated example, let\[cq]s say you had a few files
|
|
you want to back up regularly with these absolute paths:
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
/home/user1/important
|
|
/home/user1/dir/file
|
|
/home/user2/stuff
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
To copy these you\[cq]d find a common subdirectory \- in this case
|
|
\f[C]/home\f[R] and put the remaining files in \f[C]files\-from.txt\f[R]
|
|
with or without leading \f[C]/\f[R], eg
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
user1/important
|
|
user1/dir/file
|
|
user2/stuff
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
You could then copy these to a remote like this
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone copy \-\-files\-from files\-from.txt /home remote:backup
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
The 3 files will arrive in \f[C]remote:backup\f[R] with the paths as in
|
|
the \f[C]files\-from.txt\f[R] like this:
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
/home/user1/important \[->] remote:backup/user1/important
|
|
/home/user1/dir/file \[->] remote:backup/user1/dir/file
|
|
/home/user2/stuff \[->] remote:backup/user2/stuff
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
You could of course choose \f[C]/\f[R] as the root too in which case
|
|
your \f[C]files\-from.txt\f[R] might look like this.
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
/home/user1/important
|
|
/home/user1/dir/file
|
|
/home/user2/stuff
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
And you would transfer it like this
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone copy \-\-files\-from files\-from.txt / remote:backup
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
In this case there will be an extra \f[C]home\f[R] directory on the
|
|
remote:
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
/home/user1/important \[->] remote:backup/home/user1/important
|
|
/home/user1/dir/file \[->] remote:backup/home/user1/dir/file
|
|
/home/user2/stuff \[->] remote:backup/home/user2/stuff
|
|
\f[R]
|
|
.fi
|
|
.SS \f[C]\-\-files\-from\-raw\f[R] \- Read list of source\-file names without any processing
|
|
.PP
|
|
This option is same as \f[C]\-\-files\-from\f[R] with the only
|
|
difference being that the input is read in a raw manner.
|
|
This means that lines with leading/trailing whitespace and lines
|
|
starting with \f[C];\f[R] or \f[C]#\f[R] are read without any
|
|
processing.
|
|
rclone lsf (https://rclone.org/commands/rclone_lsf/) has a compatible
|
|
format that can be used to export file lists from remotes, which can
|
|
then be used as an input to \f[C]\-\-files\-from\-raw\f[R].
|
|
.SS \f[C]\-\-min\-size\f[R] \- Don\[cq]t transfer any file smaller than this
|
|
.PP
|
|
This option controls the minimum size file which will be transferred.
|
|
This defaults to \f[C]kBytes\f[R] but a suffix of \f[C]k\f[R],
|
|
\f[C]M\f[R], or \f[C]G\f[R] can be used.
|
|
.PP
|
|
For example \f[C]\-\-min\-size 50k\f[R] means no files smaller than
|
|
50kByte will be transferred.
|
|
.SS \f[C]\-\-max\-size\f[R] \- Don\[cq]t transfer any file larger than this
|
|
.PP
|
|
This option controls the maximum size file which will be transferred.
|
|
This defaults to \f[C]kBytes\f[R] but a suffix of \f[C]k\f[R],
|
|
\f[C]M\f[R], or \f[C]G\f[R] can be used.
|
|
.PP
|
|
For example \f[C]\-\-max\-size 1G\f[R] means no files larger than 1GByte
|
|
will be transferred.
|
|
.SS \f[C]\-\-max\-age\f[R] \- Don\[cq]t transfer any file older than this
|
|
.PP
|
|
This option controls the maximum age of files to transfer.
|
|
Give in seconds or with a suffix of:
|
|
.IP \[bu] 2
|
|
\f[C]ms\f[R] \- Milliseconds
|
|
.IP \[bu] 2
|
|
\f[C]s\f[R] \- Seconds
|
|
.IP \[bu] 2
|
|
\f[C]m\f[R] \- Minutes
|
|
.IP \[bu] 2
|
|
\f[C]h\f[R] \- Hours
|
|
.IP \[bu] 2
|
|
\f[C]d\f[R] \- Days
|
|
.IP \[bu] 2
|
|
\f[C]w\f[R] \- Weeks
|
|
.IP \[bu] 2
|
|
\f[C]M\f[R] \- Months
|
|
.IP \[bu] 2
|
|
\f[C]y\f[R] \- Years
|
|
.PP
|
|
For example \f[C]\-\-max\-age 2d\f[R] means no files older than 2 days
|
|
will be transferred.
|
|
.PP
|
|
This can also be an absolute time in one of these formats
|
|
.IP \[bu] 2
|
|
RFC3339 \- eg \[lq]2006\-01\-02T15:04:05Z07:00\[rq]
|
|
.IP \[bu] 2
|
|
ISO8601 Date and time, local timezone \- \[lq]2006\-01\-02T15:04:05\[rq]
|
|
.IP \[bu] 2
|
|
ISO8601 Date and time, local timezone \- \[lq]2006\-01\-02 15:04:05\[rq]
|
|
.IP \[bu] 2
|
|
ISO8601 Date \- \[lq]2006\-01\-02\[rq] (YYYY\-MM\-DD)
|
|
.SS \f[C]\-\-min\-age\f[R] \- Don\[cq]t transfer any file younger than this
|
|
.PP
|
|
This option controls the minimum age of files to transfer.
|
|
Give in seconds or with a suffix (see \f[C]\-\-max\-age\f[R] for list of
|
|
suffixes)
|
|
.PP
|
|
For example \f[C]\-\-min\-age 2d\f[R] means no files younger than 2 days
|
|
will be transferred.
|
|
.SS \f[C]\-\-delete\-excluded\f[R] \- Delete files on dest excluded from sync
|
|
.PP
|
|
\f[B]Important\f[R] this flag is dangerous \- use with
|
|
\f[C]\-\-dry\-run\f[R] and \f[C]\-v\f[R] first.
|
|
.PP
|
|
When doing \f[C]rclone sync\f[R] this will delete any files which are
|
|
excluded from the sync on the destination.
|
|
.PP
|
|
If for example you did a sync from \f[C]A\f[R] to \f[C]B\f[R] without
|
|
the \f[C]\-\-min\-size 50k\f[R] flag
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone sync A: B:
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
Then you repeated it like this with the \f[C]\-\-delete\-excluded\f[R]
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone \-\-min\-size 50k \-\-delete\-excluded sync A: B:
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
This would delete all files on \f[C]B\f[R] which are less than 50 kBytes
|
|
as these are now excluded from the sync.
|
|
.PP
|
|
Always test first with \f[C]\-\-dry\-run\f[R] and \f[C]\-v\f[R] before
|
|
using this flag.
|
|
.SS \f[C]\-\-dump filters\f[R] \- dump the filters to the output
|
|
.PP
|
|
This dumps the defined filters to the output as regular expressions.
|
|
.PP
|
|
Useful for debugging.
|
|
.SS \f[C]\-\-ignore\-case\f[R] \- make searches case insensitive
|
|
.PP
|
|
Normally filter patterns are case sensitive.
|
|
If this flag is supplied then filter patterns become case insensitive.
|
|
.PP
|
|
Normally a \f[C]\-\-include \[dq]file.txt\[dq]\f[R] will not match a
|
|
file called \f[C]FILE.txt\f[R].
|
|
However if you use the \f[C]\-\-ignore\-case\f[R] flag then
|
|
\f[C]\-\-include \[dq]file.txt\[dq]\f[R] this will match a file called
|
|
\f[C]FILE.txt\f[R].
|
|
.SS Quoting shell metacharacters
|
|
.PP
|
|
The examples above may not work verbatim in your shell as they have
|
|
shell metacharacters in them (eg \f[C]*\f[R]), and may require quoting.
|
|
.PP
|
|
Eg linux, OSX
|
|
.IP \[bu] 2
|
|
\f[C]\-\-include \[rs]*.jpg\f[R]
|
|
.IP \[bu] 2
|
|
\f[C]\-\-include \[aq]*.jpg\[aq]\f[R]
|
|
.IP \[bu] 2
|
|
\f[C]\-\-include=\[aq]*.jpg\[aq]\f[R]
|
|
.PP
|
|
In Windows the expansion is done by the command not the shell so this
|
|
should work fine
|
|
.IP \[bu] 2
|
|
\f[C]\-\-include *.jpg\f[R]
|
|
.SS Exclude directory based on a file
|
|
.PP
|
|
It is possible to exclude a directory based on a file, which is present
|
|
in this directory.
|
|
Filename should be specified using the
|
|
\f[C]\-\-exclude\-if\-present\f[R] flag.
|
|
This flag has a priority over the other filtering flags.
|
|
.PP
|
|
Imagine, you have the following directory structure:
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
dir1/file1
|
|
dir1/dir2/file2
|
|
dir1/dir2/dir3/file3
|
|
dir1/dir2/dir3/.ignore
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
You can exclude \f[C]dir3\f[R] from sync by running the following
|
|
command:
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone sync \-\-exclude\-if\-present .ignore dir1 remote:backup
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
Currently only one filename is supported,
|
|
i.e.\ \f[C]\-\-exclude\-if\-present\f[R] should not be used multiple
|
|
times.
|
|
.SH GUI (Experimental)
|
|
.PP
|
|
Rclone can serve a web based GUI (graphical user interface).
|
|
This is somewhat experimental at the moment so things may be subject to
|
|
change.
|
|
.PP
|
|
Run this command in a terminal and rclone will download and then display
|
|
the GUI in a web browser.
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone rcd \-\-rc\-web\-gui
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
This will produce logs like this and rclone needs to continue to run to
|
|
serve the GUI:
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
2019/08/25 11:40:14 NOTICE: A new release for gui is present at https://github.com/rclone/rclone\-webui\-react/releases/download/v0.0.6/currentbuild.zip
|
|
2019/08/25 11:40:14 NOTICE: Downloading webgui binary. Please wait. [Size: 3813937, Path : /home/USER/.cache/rclone/webgui/v0.0.6.zip]
|
|
2019/08/25 11:40:16 NOTICE: Unzipping
|
|
2019/08/25 11:40:16 NOTICE: Serving remote control on http://127.0.0.1:5572/
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
This assumes you are running rclone locally on your machine.
|
|
It is possible to separate the rclone and the GUI \- see below for
|
|
details.
|
|
.PP
|
|
If you wish to check for updates then you can add
|
|
\f[C]\-\-rc\-web\-gui\-update\f[R] to the command line.
|
|
.PP
|
|
If you find your GUI broken, you may force it to update by add
|
|
\f[C]\-\-rc\-web\-gui\-force\-update\f[R].
|
|
.PP
|
|
By default, rclone will open your browser.
|
|
Add \f[C]\-\-rc\-web\-gui\-no\-open\-browser\f[R] to disable this
|
|
feature.
|
|
.SS Using the GUI
|
|
.PP
|
|
Once the GUI opens, you will be looking at the dashboard which has an
|
|
overall overview.
|
|
.PP
|
|
On the left hand side you will see a series of view buttons you can
|
|
click on:
|
|
.IP \[bu] 2
|
|
Dashboard \- main overview
|
|
.IP \[bu] 2
|
|
Configs \- examine and create new configurations
|
|
.IP \[bu] 2
|
|
Explorer \- view, download and upload files to the cloud storage systems
|
|
.IP \[bu] 2
|
|
Backend \- view or alter the backend config
|
|
.IP \[bu] 2
|
|
Log out
|
|
.PP
|
|
(More docs and walkthrough video to come!)
|
|
.SS How it works
|
|
.PP
|
|
When you run the \f[C]rclone rcd \-\-rc\-web\-gui\f[R] this is what
|
|
happens
|
|
.IP \[bu] 2
|
|
Rclone starts but only runs the remote control API (\[lq]rc\[rq]).
|
|
.IP \[bu] 2
|
|
The API is bound to localhost with an auto generated username and
|
|
password.
|
|
.IP \[bu] 2
|
|
If the API bundle is missing then rclone will download it.
|
|
.IP \[bu] 2
|
|
rclone will start serving the files from the API bundle over the same
|
|
port as the API
|
|
.IP \[bu] 2
|
|
rclone will open the browser with a \f[C]login_token\f[R] so it can log
|
|
straight in.
|
|
.SS Advanced use
|
|
.PP
|
|
The \f[C]rclone rcd\f[R] may use any of the flags documented on the rc
|
|
page (https://rclone.org/rc/#supported-parameters).
|
|
.PP
|
|
The flag \f[C]\-\-rc\-web\-gui\f[R] is shorthand for
|
|
.IP \[bu] 2
|
|
Download the web GUI if necessary
|
|
.IP \[bu] 2
|
|
Check we are using some authentication
|
|
.IP \[bu] 2
|
|
\f[C]\-\-rc\-user gui\f[R]
|
|
.IP \[bu] 2
|
|
\f[C]\-\-rc\-pass <random password>\f[R]
|
|
.IP \[bu] 2
|
|
\f[C]\-\-rc\-serve\f[R]
|
|
.PP
|
|
These flags can be overridden as desired.
|
|
.PP
|
|
See also the rclone rcd
|
|
documentation (https://rclone.org/commands/rclone_rcd/).
|
|
.SS Example: Running a public GUI
|
|
.PP
|
|
For example the GUI could be served on a public port over SSL using an
|
|
htpasswd file using the following flags:
|
|
.IP \[bu] 2
|
|
\f[C]\-\-rc\-web\-gui\f[R]
|
|
.IP \[bu] 2
|
|
\f[C]\-\-rc\-addr :443\f[R]
|
|
.IP \[bu] 2
|
|
\f[C]\-\-rc\-htpasswd /path/to/htpasswd\f[R]
|
|
.IP \[bu] 2
|
|
\f[C]\-\-rc\-cert /path/to/ssl.crt\f[R]
|
|
.IP \[bu] 2
|
|
\f[C]\-\-rc\-key /path/to/ssl.key\f[R]
|
|
.SS Example: Running a GUI behind a proxy
|
|
.PP
|
|
If you want to run the GUI behind a proxy at \f[C]/rclone\f[R] you could
|
|
use these flags:
|
|
.IP \[bu] 2
|
|
\f[C]\-\-rc\-web\-gui\f[R]
|
|
.IP \[bu] 2
|
|
\f[C]\-\-rc\-baseurl rclone\f[R]
|
|
.IP \[bu] 2
|
|
\f[C]\-\-rc\-htpasswd /path/to/htpasswd\f[R]
|
|
.PP
|
|
Or instead of htpasswd if you just want a single user and password:
|
|
.IP \[bu] 2
|
|
\f[C]\-\-rc\-user me\f[R]
|
|
.IP \[bu] 2
|
|
\f[C]\-\-rc\-pass mypassword\f[R]
|
|
.SS Project
|
|
.PP
|
|
The GUI is being developed in the: rclone/rclone\-webui\-react
|
|
repository (https://github.com/rclone/rclone-webui-react).
|
|
.PP
|
|
Bug reports and contributions are very welcome :\-)
|
|
.PP
|
|
If you have questions then please ask them on the rclone
|
|
forum (https://forum.rclone.org/).
|
|
.SH Remote controlling rclone with its API
|
|
.PP
|
|
If rclone is run with the \f[C]\-\-rc\f[R] flag then it starts an http
|
|
server which can be used to remote control rclone using its API.
|
|
.PP
|
|
If you just want to run a remote control then see the rcd
|
|
command (https://rclone.org/commands/rclone_rcd/).
|
|
.SS Supported parameters
|
|
.SS \[en]rc
|
|
.PP
|
|
Flag to start the http server listen on remote requests
|
|
.SS \[en]rc\-addr=IP
|
|
.PP
|
|
IPaddress:Port or :Port to bind server to.
|
|
(default \[lq]localhost:5572\[rq])
|
|
.SS \[en]rc\-cert=KEY
|
|
.PP
|
|
SSL PEM key (concatenation of certificate and CA certificate)
|
|
.SS \[en]rc\-client\-ca=PATH
|
|
.PP
|
|
Client certificate authority to verify clients with
|
|
.SS \[en]rc\-htpasswd=PATH
|
|
.PP
|
|
htpasswd file \- if not provided no authentication is done
|
|
.SS \[en]rc\-key=PATH
|
|
.PP
|
|
SSL PEM Private key
|
|
.SS \[en]rc\-max\-header\-bytes=VALUE
|
|
.PP
|
|
Maximum size of request header (default 4096)
|
|
.SS \[en]rc\-user=VALUE
|
|
.PP
|
|
User name for authentication.
|
|
.SS \[en]rc\-pass=VALUE
|
|
.PP
|
|
Password for authentication.
|
|
.SS \[en]rc\-realm=VALUE
|
|
.PP
|
|
Realm for authentication (default \[lq]rclone\[rq])
|
|
.SS \[en]rc\-server\-read\-timeout=DURATION
|
|
.PP
|
|
Timeout for server reading data (default 1h0m0s)
|
|
.SS \[en]rc\-server\-write\-timeout=DURATION
|
|
.PP
|
|
Timeout for server writing data (default 1h0m0s)
|
|
.SS \[en]rc\-serve
|
|
.PP
|
|
Enable the serving of remote objects via the HTTP interface.
|
|
This means objects will be accessible at http://127.0.0.1:5572/ by
|
|
default, so you can browse to http://127.0.0.1:5572/ or
|
|
http://127.0.0.1:5572/* to see a listing of the remotes.
|
|
Objects may be requested from remotes using this syntax
|
|
http://127.0.0.1:5572/[remote:path]/path/to/object
|
|
.PP
|
|
Default Off.
|
|
.SS \[en]rc\-files /path/to/directory
|
|
.PP
|
|
Path to local files to serve on the HTTP server.
|
|
.PP
|
|
If this is set then rclone will serve the files in that directory.
|
|
It will also open the root in the web browser if specified.
|
|
This is for implementing browser based GUIs for rclone functions.
|
|
.PP
|
|
If \f[C]\-\-rc\-user\f[R] or \f[C]\-\-rc\-pass\f[R] is set then the URL
|
|
that is opened will have the authorization in the URL in the
|
|
\f[C]http://user:pass\[at]localhost/\f[R] style.
|
|
.PP
|
|
Default Off.
|
|
.SS \[en]rc\-enable\-metrics
|
|
.PP
|
|
Enable OpenMetrics/Prometheus compatible endpoint at \f[C]/metrics\f[R].
|
|
.PP
|
|
Default Off.
|
|
.SS \[en]rc\-web\-gui
|
|
.PP
|
|
Set this flag to serve the default web gui on the same port as rclone.
|
|
.PP
|
|
Default Off.
|
|
.SS \[en]rc\-allow\-origin
|
|
.PP
|
|
Set the allowed Access\-Control\-Allow\-Origin for rc requests.
|
|
.PP
|
|
Can be used with \[en]rc\-web\-gui if the rclone is running on different
|
|
IP than the web\-gui.
|
|
.PP
|
|
Default is IP address on which rc is running.
|
|
.SS \[en]rc\-web\-fetch\-url
|
|
.PP
|
|
Set the URL to fetch the rclone\-web\-gui files from.
|
|
.PP
|
|
Default
|
|
https://api.github.com/repos/rclone/rclone\-webui\-react/releases/latest.
|
|
.SS \[en]rc\-web\-gui\-update
|
|
.PP
|
|
Set this flag to check and update rclone\-webui\-react from the
|
|
rc\-web\-fetch\-url.
|
|
.PP
|
|
Default Off.
|
|
.SS \[en]rc\-web\-gui\-force\-update
|
|
.PP
|
|
Set this flag to force update rclone\-webui\-react from the
|
|
rc\-web\-fetch\-url.
|
|
.PP
|
|
Default Off.
|
|
.SS \[en]rc\-web\-gui\-no\-open\-browser
|
|
.PP
|
|
Set this flag to disable opening browser automatically when using
|
|
web\-gui.
|
|
.PP
|
|
Default Off.
|
|
.SS \[en]rc\-job\-expire\-duration=DURATION
|
|
.PP
|
|
Expire finished async jobs older than DURATION (default 60s).
|
|
.SS \[en]rc\-job\-expire\-interval=DURATION
|
|
.PP
|
|
Interval duration to check for expired async jobs (default 10s).
|
|
.SS \[en]rc\-no\-auth
|
|
.PP
|
|
By default rclone will require authorisation to have been set up on the
|
|
rc interface in order to use any methods which access any rclone
|
|
remotes.
|
|
Eg \f[C]operations/list\f[R] is denied as it involved creating a remote
|
|
as is \f[C]sync/copy\f[R].
|
|
.PP
|
|
If this is set then no authorisation will be required on the server to
|
|
use these methods.
|
|
The alternative is to use \f[C]\-\-rc\-user\f[R] and
|
|
\f[C]\-\-rc\-pass\f[R] and use these credentials in the request.
|
|
.PP
|
|
Default Off.
|
|
.SS Accessing the remote control via the rclone rc command
|
|
.PP
|
|
Rclone itself implements the remote control protocol in its
|
|
\f[C]rclone rc\f[R] command.
|
|
.PP
|
|
You can use it like this
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
$ rclone rc rc/noop param1=one param2=two
|
|
{
|
|
\[dq]param1\[dq]: \[dq]one\[dq],
|
|
\[dq]param2\[dq]: \[dq]two\[dq]
|
|
}
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
Run \f[C]rclone rc\f[R] on its own to see the help for the installed
|
|
remote control commands.
|
|
.PP
|
|
\f[C]rclone rc\f[R] also supports a \f[C]\-\-json\f[R] flag which can be
|
|
used to send more complicated input parameters.
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
$ rclone rc \-\-json \[aq]{ \[dq]p1\[dq]: [1,\[dq]2\[dq],null,4], \[dq]p2\[dq]: { \[dq]a\[dq]:1, \[dq]b\[dq]:2 } }\[aq] rc/noop
|
|
{
|
|
\[dq]p1\[dq]: [
|
|
1,
|
|
\[dq]2\[dq],
|
|
null,
|
|
4
|
|
],
|
|
\[dq]p2\[dq]: {
|
|
\[dq]a\[dq]: 1,
|
|
\[dq]b\[dq]: 2
|
|
}
|
|
}
|
|
\f[R]
|
|
.fi
|
|
.SS Special parameters
|
|
.PP
|
|
The rc interface supports some special parameters which apply to
|
|
\f[B]all\f[R] commands.
|
|
These start with \f[C]_\f[R] to show they are different.
|
|
.SS Running asynchronous jobs with _async = true
|
|
.PP
|
|
Each rc call is classified as a job and it is assigned its own id.
|
|
By default jobs are executed immediately as they are created or
|
|
synchronously.
|
|
.PP
|
|
If \f[C]_async\f[R] has a true value when supplied to an rc call then it
|
|
will return immediately with a job id and the task will be run in the
|
|
background.
|
|
The \f[C]job/status\f[R] call can be used to get information of the
|
|
background job.
|
|
The job can be queried for up to 1 minute after it has finished.
|
|
.PP
|
|
It is recommended that potentially long running jobs, eg
|
|
\f[C]sync/sync\f[R], \f[C]sync/copy\f[R], \f[C]sync/move\f[R],
|
|
\f[C]operations/purge\f[R] are run with the \f[C]_async\f[R] flag to
|
|
avoid any potential problems with the HTTP request and response timing
|
|
out.
|
|
.PP
|
|
Starting a job with the \f[C]_async\f[R] flag:
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
$ rclone rc \-\-json \[aq]{ \[dq]p1\[dq]: [1,\[dq]2\[dq],null,4], \[dq]p2\[dq]: { \[dq]a\[dq]:1, \[dq]b\[dq]:2 }, \[dq]_async\[dq]: true }\[aq] rc/noop
|
|
{
|
|
\[dq]jobid\[dq]: 2
|
|
}
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
Query the status to see if the job has finished.
|
|
For more information on the meaning of these return parameters see the
|
|
\f[C]job/status\f[R] call.
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
$ rclone rc \-\-json \[aq]{ \[dq]jobid\[dq]:2 }\[aq] job/status
|
|
{
|
|
\[dq]duration\[dq]: 0.000124163,
|
|
\[dq]endTime\[dq]: \[dq]2018\-10\-27T11:38:07.911245881+01:00\[dq],
|
|
\[dq]error\[dq]: \[dq]\[dq],
|
|
\[dq]finished\[dq]: true,
|
|
\[dq]id\[dq]: 2,
|
|
\[dq]output\[dq]: {
|
|
\[dq]_async\[dq]: true,
|
|
\[dq]p1\[dq]: [
|
|
1,
|
|
\[dq]2\[dq],
|
|
null,
|
|
4
|
|
],
|
|
\[dq]p2\[dq]: {
|
|
\[dq]a\[dq]: 1,
|
|
\[dq]b\[dq]: 2
|
|
}
|
|
},
|
|
\[dq]startTime\[dq]: \[dq]2018\-10\-27T11:38:07.911121728+01:00\[dq],
|
|
\[dq]success\[dq]: true
|
|
}
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
\f[C]job/list\f[R] can be used to show the running or recently completed
|
|
jobs
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
$ rclone rc job/list
|
|
{
|
|
\[dq]jobids\[dq]: [
|
|
2
|
|
]
|
|
}
|
|
\f[R]
|
|
.fi
|
|
.SS Assigning operations to groups with _group = value
|
|
.PP
|
|
Each rc call has its own stats group for tracking its metrics.
|
|
By default grouping is done by the composite group name from prefix
|
|
\f[C]job/\f[R] and id of the job like so \f[C]job/1\f[R].
|
|
.PP
|
|
If \f[C]_group\f[R] has a value then stats for that request will be
|
|
grouped under that value.
|
|
This allows caller to group stats under their own name.
|
|
.PP
|
|
Stats for specific group can be accessed by passing \f[C]group\f[R] to
|
|
\f[C]core/stats\f[R]:
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
$ rclone rc \-\-json \[aq]{ \[dq]group\[dq]: \[dq]job/1\[dq] }\[aq] core/stats
|
|
{
|
|
\[dq]speed\[dq]: 12345
|
|
...
|
|
}
|
|
\f[R]
|
|
.fi
|
|
.SS Supported commands
|
|
.SS backend/command: Runs a backend command.
|
|
.PP
|
|
This takes the following parameters
|
|
.IP \[bu] 2
|
|
command \- a string with the command name
|
|
.IP \[bu] 2
|
|
fs \- a remote name string eg \[lq]drive:\[rq]
|
|
.IP \[bu] 2
|
|
arg \- a list of arguments for the backend command
|
|
.IP \[bu] 2
|
|
opt \- a map of string to string of options
|
|
.PP
|
|
Returns
|
|
.IP \[bu] 2
|
|
result \- result from the backend command
|
|
.PP
|
|
For example
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone rc backend/command command=noop fs=. \-o echo=yes \-o blue \-a path1 \-a path2
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
Returns
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
{
|
|
\[dq]result\[dq]: {
|
|
\[dq]arg\[dq]: [
|
|
\[dq]path1\[dq],
|
|
\[dq]path2\[dq]
|
|
],
|
|
\[dq]name\[dq]: \[dq]noop\[dq],
|
|
\[dq]opt\[dq]: {
|
|
\[dq]blue\[dq]: \[dq]\[dq],
|
|
\[dq]echo\[dq]: \[dq]yes\[dq]
|
|
}
|
|
}
|
|
}
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
Note that this is the direct equivalent of using this \[lq]backend\[rq]
|
|
command:
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone backend noop . \-o echo=yes \-o blue path1 path2
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
Note that arguments must be preceded by the \[lq]\-a\[rq] flag
|
|
.PP
|
|
See the backend (https://rclone.org/commands/rclone_backend/) command
|
|
for more information.
|
|
.PP
|
|
\f[B]Authentication is required for this call.\f[R]
|
|
.SS cache/expire: Purge a remote from cache
|
|
.PP
|
|
Purge a remote from the cache backend.
|
|
Supports either a directory or a file.
|
|
Params: \- remote = path to remote (required) \- withData = true/false
|
|
to delete cached data (chunks) as well (optional)
|
|
.PP
|
|
Eg
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone rc cache/expire remote=path/to/sub/folder/
|
|
rclone rc cache/expire remote=/ withData=true
|
|
\f[R]
|
|
.fi
|
|
.SS cache/fetch: Fetch file chunks
|
|
.PP
|
|
Ensure the specified file chunks are cached on disk.
|
|
.PP
|
|
The chunks= parameter specifies the file chunks to check.
|
|
It takes a comma separated list of array slice indices.
|
|
The slice indices are similar to Python slices: start[:end]
|
|
.PP
|
|
start is the 0 based chunk number from the beginning of the file to
|
|
fetch inclusive.
|
|
end is 0 based chunk number from the beginning of the file to fetch
|
|
exclusive.
|
|
Both values can be negative, in which case they count from the back of
|
|
the file.
|
|
The value \[lq]\-5:\[rq] represents the last 5 chunks of a file.
|
|
.PP
|
|
Some valid examples are: \[lq]:5,\-5:\[rq] \-> the first and last five
|
|
chunks \[lq]0,\-2\[rq] \-> the first and the second last chunk
|
|
\[lq]0:10\[rq] \-> the first ten chunks
|
|
.PP
|
|
Any parameter with a key that starts with \[lq]file\[rq] can be used to
|
|
specify files to fetch, eg
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone rc cache/fetch chunks=0 file=hello file2=home/goodbye
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
File names will automatically be encrypted when the a crypt remote is
|
|
used on top of the cache.
|
|
.SS cache/stats: Get cache stats
|
|
.PP
|
|
Show statistics for the cache remote.
|
|
.SS config/create: create the config for a remote.
|
|
.PP
|
|
This takes the following parameters
|
|
.IP \[bu] 2
|
|
name \- name of remote
|
|
.IP \[bu] 2
|
|
parameters \- a map of { \[lq]key\[rq]: \[lq]value\[rq] } pairs
|
|
.IP \[bu] 2
|
|
type \- type of the new remote
|
|
.IP \[bu] 2
|
|
obscure \- optional bool \- forces obscuring of passwords
|
|
.IP \[bu] 2
|
|
noObscure \- optional bool \- forces passwords not to be obscured
|
|
.PP
|
|
See the config create
|
|
command (https://rclone.org/commands/rclone_config_create/) command for
|
|
more information on the above.
|
|
.PP
|
|
\f[B]Authentication is required for this call.\f[R]
|
|
.SS config/delete: Delete a remote in the config file.
|
|
.PP
|
|
Parameters:
|
|
.IP \[bu] 2
|
|
name \- name of remote to delete
|
|
.PP
|
|
See the config delete
|
|
command (https://rclone.org/commands/rclone_config_delete/) command for
|
|
more information on the above.
|
|
.PP
|
|
\f[B]Authentication is required for this call.\f[R]
|
|
.SS config/dump: Dumps the config file.
|
|
.PP
|
|
Returns a JSON object: \- key: value
|
|
.PP
|
|
Where keys are remote names and values are the config parameters.
|
|
.PP
|
|
See the config dump
|
|
command (https://rclone.org/commands/rclone_config_dump/) command for
|
|
more information on the above.
|
|
.PP
|
|
\f[B]Authentication is required for this call.\f[R]
|
|
.SS config/get: Get a remote in the config file.
|
|
.PP
|
|
Parameters:
|
|
.IP \[bu] 2
|
|
name \- name of remote to get
|
|
.PP
|
|
See the config dump
|
|
command (https://rclone.org/commands/rclone_config_dump/) command for
|
|
more information on the above.
|
|
.PP
|
|
\f[B]Authentication is required for this call.\f[R]
|
|
.SS config/listremotes: Lists the remotes in the config file.
|
|
.PP
|
|
Returns \- remotes \- array of remote names
|
|
.PP
|
|
See the listremotes
|
|
command (https://rclone.org/commands/rclone_listremotes/) command for
|
|
more information on the above.
|
|
.PP
|
|
\f[B]Authentication is required for this call.\f[R]
|
|
.SS config/password: password the config for a remote.
|
|
.PP
|
|
This takes the following parameters
|
|
.IP \[bu] 2
|
|
name \- name of remote
|
|
.IP \[bu] 2
|
|
parameters \- a map of { \[lq]key\[rq]: \[lq]value\[rq] } pairs
|
|
.PP
|
|
See the config password
|
|
command (https://rclone.org/commands/rclone_config_password/) command
|
|
for more information on the above.
|
|
.PP
|
|
\f[B]Authentication is required for this call.\f[R]
|
|
.SS config/providers: Shows how providers are configured in the config file.
|
|
.PP
|
|
Returns a JSON object: \- providers \- array of objects
|
|
.PP
|
|
See the config providers
|
|
command (https://rclone.org/commands/rclone_config_providers/) command
|
|
for more information on the above.
|
|
.PP
|
|
\f[B]Authentication is required for this call.\f[R]
|
|
.SS config/update: update the config for a remote.
|
|
.PP
|
|
This takes the following parameters
|
|
.IP \[bu] 2
|
|
name \- name of remote
|
|
.IP \[bu] 2
|
|
parameters \- a map of { \[lq]key\[rq]: \[lq]value\[rq] } pairs
|
|
.IP \[bu] 2
|
|
obscure \- optional bool \- forces obscuring of passwords
|
|
.IP \[bu] 2
|
|
noObscure \- optional bool \- forces passwords not to be obscured
|
|
.PP
|
|
See the config update
|
|
command (https://rclone.org/commands/rclone_config_update/) command for
|
|
more information on the above.
|
|
.PP
|
|
\f[B]Authentication is required for this call.\f[R]
|
|
.SS core/bwlimit: Set the bandwidth limit.
|
|
.PP
|
|
This sets the bandwidth limit to that passed in.
|
|
.PP
|
|
Eg
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone rc core/bwlimit rate=off
|
|
{
|
|
\[dq]bytesPerSecond\[dq]: \-1,
|
|
\[dq]rate\[dq]: \[dq]off\[dq]
|
|
}
|
|
rclone rc core/bwlimit rate=1M
|
|
{
|
|
\[dq]bytesPerSecond\[dq]: 1048576,
|
|
\[dq]rate\[dq]: \[dq]1M\[dq]
|
|
}
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
If the rate parameter is not supplied then the bandwidth is queried
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone rc core/bwlimit
|
|
{
|
|
\[dq]bytesPerSecond\[dq]: 1048576,
|
|
\[dq]rate\[dq]: \[dq]1M\[dq]
|
|
}
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
The format of the parameter is exactly the same as passed to
|
|
\[en]bwlimit except only one bandwidth may be specified.
|
|
.PP
|
|
In either case \[lq]rate\[rq] is returned as a human readable string,
|
|
and \[lq]bytesPerSecond\[rq] is returned as a number.
|
|
.SS core/gc: Runs a garbage collection.
|
|
.PP
|
|
This tells the go runtime to do a garbage collection run.
|
|
It isn\[cq]t necessary to call this normally, but it can be useful for
|
|
debugging memory problems.
|
|
.SS core/group\-list: Returns list of stats.
|
|
.PP
|
|
This returns list of stats groups currently in memory.
|
|
.PP
|
|
Returns the following values:
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
{
|
|
\[dq]groups\[dq]: an array of group names:
|
|
[
|
|
\[dq]group1\[dq],
|
|
\[dq]group2\[dq],
|
|
...
|
|
]
|
|
}
|
|
\f[R]
|
|
.fi
|
|
.SS core/memstats: Returns the memory statistics
|
|
.PP
|
|
This returns the memory statistics of the running program.
|
|
What the values mean are explained in the go docs:
|
|
https://golang.org/pkg/runtime/#MemStats
|
|
.PP
|
|
The most interesting values for most people are:
|
|
.IP \[bu] 2
|
|
HeapAlloc: This is the amount of memory rclone is actually using
|
|
.IP \[bu] 2
|
|
HeapSys: This is the amount of memory rclone has obtained from the OS
|
|
.IP \[bu] 2
|
|
Sys: this is the total amount of memory requested from the OS
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
It is virtual memory so may include unused memory
|
|
.RE
|
|
.SS core/obscure: Obscures a string passed in.
|
|
.PP
|
|
Pass a clear string and rclone will obscure it for the config file: \-
|
|
clear \- string
|
|
.PP
|
|
Returns \- obscured \- string
|
|
.SS core/pid: Return PID of current process
|
|
.PP
|
|
This returns PID of current process.
|
|
Useful for stopping rclone process.
|
|
.SS core/quit: Terminates the app.
|
|
.PP
|
|
(optional) Pass an exit code to be used for terminating the app: \-
|
|
exitCode \- int
|
|
.SS core/stats: Returns stats about current transfers.
|
|
.PP
|
|
This returns all available stats:
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone rc core/stats
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
If group is not provided then summed up stats for all groups will be
|
|
returned.
|
|
.PP
|
|
Parameters
|
|
.IP \[bu] 2
|
|
group \- name of the stats group (string)
|
|
.PP
|
|
Returns the following values:
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
{
|
|
\[dq]speed\[dq]: average speed in bytes/sec since start of the process,
|
|
\[dq]bytes\[dq]: total transferred bytes since the start of the process,
|
|
\[dq]errors\[dq]: number of errors,
|
|
\[dq]fatalError\[dq]: whether there has been at least one FatalError,
|
|
\[dq]retryError\[dq]: whether there has been at least one non\-NoRetryError,
|
|
\[dq]checks\[dq]: number of checked files,
|
|
\[dq]transfers\[dq]: number of transferred files,
|
|
\[dq]deletes\[dq] : number of deleted files,
|
|
\[dq]renames\[dq] : number of renamed files,
|
|
\[dq]elapsedTime\[dq]: time in seconds since the start of the process,
|
|
\[dq]lastError\[dq]: last occurred error,
|
|
\[dq]transferring\[dq]: an array of currently active file transfers:
|
|
[
|
|
{
|
|
\[dq]bytes\[dq]: total transferred bytes for this file,
|
|
\[dq]eta\[dq]: estimated time in seconds until file transfer completion
|
|
\[dq]name\[dq]: name of the file,
|
|
\[dq]percentage\[dq]: progress of the file transfer in percent,
|
|
\[dq]speed\[dq]: speed in bytes/sec,
|
|
\[dq]speedAvg\[dq]: speed in bytes/sec as an exponentially weighted moving average,
|
|
\[dq]size\[dq]: size of the file in bytes
|
|
}
|
|
],
|
|
\[dq]checking\[dq]: an array of names of currently active file checks
|
|
[]
|
|
}
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
Values for \[lq]transferring\[rq], \[lq]checking\[rq] and
|
|
\[lq]lastError\[rq] are only assigned if data is available.
|
|
The value for \[lq]eta\[rq] is null if an eta cannot be determined.
|
|
.SS core/stats\-delete: Delete stats group.
|
|
.PP
|
|
This deletes entire stats group
|
|
.PP
|
|
Parameters
|
|
.IP \[bu] 2
|
|
group \- name of the stats group (string)
|
|
.SS core/stats\-reset: Reset stats.
|
|
.PP
|
|
This clears counters, errors and finished transfers for all stats or
|
|
specific stats group if group is provided.
|
|
.PP
|
|
Parameters
|
|
.IP \[bu] 2
|
|
group \- name of the stats group (string)
|
|
.SS core/transferred: Returns stats about completed transfers.
|
|
.PP
|
|
This returns stats about completed transfers:
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone rc core/transferred
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
If group is not provided then completed transfers for all groups will be
|
|
returned.
|
|
.PP
|
|
Note only the last 100 completed transfers are returned.
|
|
.PP
|
|
Parameters
|
|
.IP \[bu] 2
|
|
group \- name of the stats group (string)
|
|
.PP
|
|
Returns the following values:
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
{
|
|
\[dq]transferred\[dq]: an array of completed transfers (including failed ones):
|
|
[
|
|
{
|
|
\[dq]name\[dq]: name of the file,
|
|
\[dq]size\[dq]: size of the file in bytes,
|
|
\[dq]bytes\[dq]: total transferred bytes for this file,
|
|
\[dq]checked\[dq]: if the transfer is only checked (skipped, deleted),
|
|
\[dq]timestamp\[dq]: integer representing millisecond unix epoch,
|
|
\[dq]error\[dq]: string description of the error (empty if successful),
|
|
\[dq]jobid\[dq]: id of the job that this transfer belongs to
|
|
}
|
|
]
|
|
}
|
|
\f[R]
|
|
.fi
|
|
.SS core/version: Shows the current version of rclone and the go runtime.
|
|
.PP
|
|
This shows the current version of go and the go runtime
|
|
.IP \[bu] 2
|
|
version \- rclone version, eg \[lq]v1.44\[rq]
|
|
.IP \[bu] 2
|
|
decomposed \- version number as [major, minor, patch, subpatch]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
note patch and subpatch will be 999 for a git compiled version
|
|
.RE
|
|
.IP \[bu] 2
|
|
isGit \- boolean \- true if this was compiled from the git version
|
|
.IP \[bu] 2
|
|
os \- OS in use as according to Go
|
|
.IP \[bu] 2
|
|
arch \- cpu architecture in use according to Go
|
|
.IP \[bu] 2
|
|
goVersion \- version of Go runtime in use
|
|
.SS debug/set\-block\-profile\-rate: Set runtime.SetBlockProfileRate for blocking profiling.
|
|
.PP
|
|
SetBlockProfileRate controls the fraction of goroutine blocking events
|
|
that are reported in the blocking profile.
|
|
The profiler aims to sample an average of one blocking event per rate
|
|
nanoseconds spent blocked.
|
|
.PP
|
|
To include every blocking event in the profile, pass rate = 1.
|
|
To turn off profiling entirely, pass rate <= 0.
|
|
.PP
|
|
After calling this you can use this to see the blocking profile:
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
go tool pprof http://localhost:5572/debug/pprof/block
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
Parameters
|
|
.IP \[bu] 2
|
|
rate \- int
|
|
.SS debug/set\-mutex\-profile\-fraction: Set runtime.SetMutexProfileFraction for mutex profiling.
|
|
.PP
|
|
SetMutexProfileFraction controls the fraction of mutex contention events
|
|
that are reported in the mutex profile.
|
|
On average 1/rate events are reported.
|
|
The previous rate is returned.
|
|
.PP
|
|
To turn off profiling entirely, pass rate 0.
|
|
To just read the current rate, pass rate < 0.
|
|
(For n>1 the details of sampling may change.)
|
|
.PP
|
|
Once this is set you can look use this to profile the mutex contention:
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
go tool pprof http://localhost:5572/debug/pprof/mutex
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
Parameters
|
|
.IP \[bu] 2
|
|
rate \- int
|
|
.PP
|
|
Results
|
|
.IP \[bu] 2
|
|
previousRate \- int
|
|
.SS job/list: Lists the IDs of the running jobs
|
|
.PP
|
|
Parameters \- None
|
|
.PP
|
|
Results
|
|
.IP \[bu] 2
|
|
jobids \- array of integer job ids
|
|
.SS job/status: Reads the status of the job ID
|
|
.PP
|
|
Parameters
|
|
.IP \[bu] 2
|
|
jobid \- id of the job (integer)
|
|
.PP
|
|
Results
|
|
.IP \[bu] 2
|
|
finished \- boolean
|
|
.IP \[bu] 2
|
|
duration \- time in seconds that the job ran for
|
|
.IP \[bu] 2
|
|
endTime \- time the job finished (eg
|
|
\[lq]2018\-10\-26T18:50:20.528746884+01:00\[rq])
|
|
.IP \[bu] 2
|
|
error \- error from the job or empty string for no error
|
|
.IP \[bu] 2
|
|
finished \- boolean whether the job has finished or not
|
|
.IP \[bu] 2
|
|
id \- as passed in above
|
|
.IP \[bu] 2
|
|
startTime \- time the job started (eg
|
|
\[lq]2018\-10\-26T18:50:20.528336039+01:00\[rq])
|
|
.IP \[bu] 2
|
|
success \- boolean \- true for success false otherwise
|
|
.IP \[bu] 2
|
|
output \- output of the job as would have been returned if called
|
|
synchronously
|
|
.IP \[bu] 2
|
|
progress \- output of the progress related to the underlying job
|
|
.SS job/stop: Stop the running job
|
|
.PP
|
|
Parameters
|
|
.IP \[bu] 2
|
|
jobid \- id of the job (integer)
|
|
.SS mount/mount: Create a new mount point
|
|
.PP
|
|
rclone allows Linux, FreeBSD, macOS and Windows to mount any of
|
|
Rclone\[cq]s cloud storage systems as a file system with FUSE.
|
|
.PP
|
|
If no mountType is provided, the priority is given as follows: 1.
|
|
mount 2.cmount 3.mount2
|
|
.PP
|
|
This takes the following parameters
|
|
.IP \[bu] 2
|
|
fs \- a remote path to be mounted (required)
|
|
.IP \[bu] 2
|
|
mountPoint: valid path on the local machine (required)
|
|
.IP \[bu] 2
|
|
mountType: One of the values (mount, cmount, mount2) specifies the mount
|
|
implementation to use
|
|
.PP
|
|
Eg
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone rc mount/mount fs=mydrive: mountPoint=/home/<user>/mountPoint
|
|
rclone rc mount/mount fs=mydrive: mountPoint=/home/<user>/mountPoint mountType=mount
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
\f[B]Authentication is required for this call.\f[R]
|
|
.SS mount/types: Show all possible mount types
|
|
.PP
|
|
This shows all possible mount types and returns them as a list.
|
|
.PP
|
|
This takes no parameters and returns
|
|
.IP \[bu] 2
|
|
mountTypes: list of mount types
|
|
.PP
|
|
The mount types are strings like \[lq]mount\[rq], \[lq]mount2\[rq],
|
|
\[lq]cmount\[rq] and can be passed to mount/mount as the mountType
|
|
parameter.
|
|
.PP
|
|
Eg
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone rc mount/types
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
\f[B]Authentication is required for this call.\f[R]
|
|
.SS mount/unmount: Unmount all active mounts
|
|
.PP
|
|
rclone allows Linux, FreeBSD, macOS and Windows to mount any of
|
|
Rclone\[cq]s cloud storage systems as a file system with FUSE.
|
|
.PP
|
|
This takes the following parameters
|
|
.IP \[bu] 2
|
|
mountPoint: valid path on the local machine where the mount was created
|
|
(required)
|
|
.PP
|
|
Eg
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone rc mount/unmount mountPoint=/home/<user>/mountPoint
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
\f[B]Authentication is required for this call.\f[R]
|
|
.SS operations/about: Return the space used on the remote
|
|
.PP
|
|
This takes the following parameters
|
|
.IP \[bu] 2
|
|
fs \- a remote name string eg \[lq]drive:\[rq]
|
|
.PP
|
|
The result is as returned from rclone about \[en]json
|
|
.PP
|
|
See the about command (https://rclone.org/commands/rclone_size/) command
|
|
for more information on the above.
|
|
.PP
|
|
\f[B]Authentication is required for this call.\f[R]
|
|
.SS operations/cleanup: Remove trashed files in the remote or path
|
|
.PP
|
|
This takes the following parameters
|
|
.IP \[bu] 2
|
|
fs \- a remote name string eg \[lq]drive:\[rq]
|
|
.PP
|
|
See the cleanup command (https://rclone.org/commands/rclone_cleanup/)
|
|
command for more information on the above.
|
|
.PP
|
|
\f[B]Authentication is required for this call.\f[R]
|
|
.SS operations/copyfile: Copy a file from source remote to destination remote
|
|
.PP
|
|
This takes the following parameters
|
|
.IP \[bu] 2
|
|
srcFs \- a remote name string eg \[lq]drive:\[rq] for the source
|
|
.IP \[bu] 2
|
|
srcRemote \- a path within that remote eg \[lq]file.txt\[rq] for the
|
|
source
|
|
.IP \[bu] 2
|
|
dstFs \- a remote name string eg \[lq]drive2:\[rq] for the destination
|
|
.IP \[bu] 2
|
|
dstRemote \- a path within that remote eg \[lq]file2.txt\[rq] for the
|
|
destination
|
|
.PP
|
|
\f[B]Authentication is required for this call.\f[R]
|
|
.SS operations/copyurl: Copy the URL to the object
|
|
.PP
|
|
This takes the following parameters
|
|
.IP \[bu] 2
|
|
fs \- a remote name string eg \[lq]drive:\[rq]
|
|
.IP \[bu] 2
|
|
remote \- a path within that remote eg \[lq]dir\[rq]
|
|
.IP \[bu] 2
|
|
url \- string, URL to read from
|
|
.IP \[bu] 2
|
|
autoFilename \- boolean, set to true to retrieve destination file name
|
|
from url See the copyurl
|
|
command (https://rclone.org/commands/rclone_copyurl/) command for more
|
|
information on the above.
|
|
.PP
|
|
\f[B]Authentication is required for this call.\f[R]
|
|
.SS operations/delete: Remove files in the path
|
|
.PP
|
|
This takes the following parameters
|
|
.IP \[bu] 2
|
|
fs \- a remote name string eg \[lq]drive:\[rq]
|
|
.PP
|
|
See the delete command (https://rclone.org/commands/rclone_delete/)
|
|
command for more information on the above.
|
|
.PP
|
|
\f[B]Authentication is required for this call.\f[R]
|
|
.SS operations/deletefile: Remove the single file pointed to
|
|
.PP
|
|
This takes the following parameters
|
|
.IP \[bu] 2
|
|
fs \- a remote name string eg \[lq]drive:\[rq]
|
|
.IP \[bu] 2
|
|
remote \- a path within that remote eg \[lq]dir\[rq]
|
|
.PP
|
|
See the deletefile
|
|
command (https://rclone.org/commands/rclone_deletefile/) command for
|
|
more information on the above.
|
|
.PP
|
|
\f[B]Authentication is required for this call.\f[R]
|
|
.SS operations/fsinfo: Return information about the remote
|
|
.PP
|
|
This takes the following parameters
|
|
.IP \[bu] 2
|
|
fs \- a remote name string eg \[lq]drive:\[rq]
|
|
.PP
|
|
This returns info about the remote passed in;
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
{
|
|
// optional features and whether they are available or not
|
|
\[dq]Features\[dq]: {
|
|
\[dq]About\[dq]: true,
|
|
\[dq]BucketBased\[dq]: false,
|
|
\[dq]CanHaveEmptyDirectories\[dq]: true,
|
|
\[dq]CaseInsensitive\[dq]: false,
|
|
\[dq]ChangeNotify\[dq]: false,
|
|
\[dq]CleanUp\[dq]: false,
|
|
\[dq]Copy\[dq]: false,
|
|
\[dq]DirCacheFlush\[dq]: false,
|
|
\[dq]DirMove\[dq]: true,
|
|
\[dq]DuplicateFiles\[dq]: false,
|
|
\[dq]GetTier\[dq]: false,
|
|
\[dq]ListR\[dq]: false,
|
|
\[dq]MergeDirs\[dq]: false,
|
|
\[dq]Move\[dq]: true,
|
|
\[dq]OpenWriterAt\[dq]: true,
|
|
\[dq]PublicLink\[dq]: false,
|
|
\[dq]Purge\[dq]: true,
|
|
\[dq]PutStream\[dq]: true,
|
|
\[dq]PutUnchecked\[dq]: false,
|
|
\[dq]ReadMimeType\[dq]: false,
|
|
\[dq]ServerSideAcrossConfigs\[dq]: false,
|
|
\[dq]SetTier\[dq]: false,
|
|
\[dq]SetWrapper\[dq]: false,
|
|
\[dq]UnWrap\[dq]: false,
|
|
\[dq]WrapFs\[dq]: false,
|
|
\[dq]WriteMimeType\[dq]: false
|
|
},
|
|
// Names of hashes available
|
|
\[dq]Hashes\[dq]: [
|
|
\[dq]MD5\[dq],
|
|
\[dq]SHA\-1\[dq],
|
|
\[dq]DropboxHash\[dq],
|
|
\[dq]QuickXorHash\[dq]
|
|
],
|
|
\[dq]Name\[dq]: \[dq]local\[dq], // Name as created
|
|
\[dq]Precision\[dq]: 1, // Precision of timestamps in ns
|
|
\[dq]Root\[dq]: \[dq]/\[dq], // Path as created
|
|
\[dq]String\[dq]: \[dq]Local file system at /\[dq] // how the remote will appear in logs
|
|
}
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
This command does not have a command line equivalent so use this
|
|
instead:
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone rc \-\-loopback operations/fsinfo fs=remote:
|
|
\f[R]
|
|
.fi
|
|
.SS operations/list: List the given remote and path in JSON format
|
|
.PP
|
|
This takes the following parameters
|
|
.IP \[bu] 2
|
|
fs \- a remote name string eg \[lq]drive:\[rq]
|
|
.IP \[bu] 2
|
|
remote \- a path within that remote eg \[lq]dir\[rq]
|
|
.IP \[bu] 2
|
|
opt \- a dictionary of options to control the listing (optional)
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
recurse \- If set recurse directories
|
|
.IP \[bu] 2
|
|
noModTime \- If set return modification time
|
|
.IP \[bu] 2
|
|
showEncrypted \- If set show decrypted names
|
|
.IP \[bu] 2
|
|
showOrigIDs \- If set show the IDs for each item if known
|
|
.IP \[bu] 2
|
|
showHash \- If set return a dictionary of hashes
|
|
.RE
|
|
.PP
|
|
The result is
|
|
.IP \[bu] 2
|
|
list
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
This is an array of objects as described in the lsjson command
|
|
.RE
|
|
.PP
|
|
See the lsjson command (https://rclone.org/commands/rclone_lsjson/) for
|
|
more information on the above and examples.
|
|
.PP
|
|
\f[B]Authentication is required for this call.\f[R]
|
|
.SS operations/mkdir: Make a destination directory or container
|
|
.PP
|
|
This takes the following parameters
|
|
.IP \[bu] 2
|
|
fs \- a remote name string eg \[lq]drive:\[rq]
|
|
.IP \[bu] 2
|
|
remote \- a path within that remote eg \[lq]dir\[rq]
|
|
.PP
|
|
See the mkdir command (https://rclone.org/commands/rclone_mkdir/)
|
|
command for more information on the above.
|
|
.PP
|
|
\f[B]Authentication is required for this call.\f[R]
|
|
.SS operations/movefile: Move a file from source remote to destination remote
|
|
.PP
|
|
This takes the following parameters
|
|
.IP \[bu] 2
|
|
srcFs \- a remote name string eg \[lq]drive:\[rq] for the source
|
|
.IP \[bu] 2
|
|
srcRemote \- a path within that remote eg \[lq]file.txt\[rq] for the
|
|
source
|
|
.IP \[bu] 2
|
|
dstFs \- a remote name string eg \[lq]drive2:\[rq] for the destination
|
|
.IP \[bu] 2
|
|
dstRemote \- a path within that remote eg \[lq]file2.txt\[rq] for the
|
|
destination
|
|
.PP
|
|
\f[B]Authentication is required for this call.\f[R]
|
|
.SS operations/publiclink: Create or retrieve a public link to the given file or folder.
|
|
.PP
|
|
This takes the following parameters
|
|
.IP \[bu] 2
|
|
fs \- a remote name string eg \[lq]drive:\[rq]
|
|
.IP \[bu] 2
|
|
remote \- a path within that remote eg \[lq]dir\[rq]
|
|
.PP
|
|
Returns
|
|
.IP \[bu] 2
|
|
url \- URL of the resource
|
|
.PP
|
|
See the link command (https://rclone.org/commands/rclone_link/) command
|
|
for more information on the above.
|
|
.PP
|
|
\f[B]Authentication is required for this call.\f[R]
|
|
.SS operations/purge: Remove a directory or container and all of its contents
|
|
.PP
|
|
This takes the following parameters
|
|
.IP \[bu] 2
|
|
fs \- a remote name string eg \[lq]drive:\[rq]
|
|
.IP \[bu] 2
|
|
remote \- a path within that remote eg \[lq]dir\[rq]
|
|
.PP
|
|
See the purge command (https://rclone.org/commands/rclone_purge/)
|
|
command for more information on the above.
|
|
.PP
|
|
\f[B]Authentication is required for this call.\f[R]
|
|
.SS operations/rmdir: Remove an empty directory or container
|
|
.PP
|
|
This takes the following parameters
|
|
.IP \[bu] 2
|
|
fs \- a remote name string eg \[lq]drive:\[rq]
|
|
.IP \[bu] 2
|
|
remote \- a path within that remote eg \[lq]dir\[rq]
|
|
.PP
|
|
See the rmdir command (https://rclone.org/commands/rclone_rmdir/)
|
|
command for more information on the above.
|
|
.PP
|
|
\f[B]Authentication is required for this call.\f[R]
|
|
.SS operations/rmdirs: Remove all the empty directories in the path
|
|
.PP
|
|
This takes the following parameters
|
|
.IP \[bu] 2
|
|
fs \- a remote name string eg \[lq]drive:\[rq]
|
|
.IP \[bu] 2
|
|
remote \- a path within that remote eg \[lq]dir\[rq]
|
|
.IP \[bu] 2
|
|
leaveRoot \- boolean, set to true not to delete the root
|
|
.PP
|
|
See the rmdirs command (https://rclone.org/commands/rclone_rmdirs/)
|
|
command for more information on the above.
|
|
.PP
|
|
\f[B]Authentication is required for this call.\f[R]
|
|
.SS operations/size: Count the number of bytes and files in remote
|
|
.PP
|
|
This takes the following parameters
|
|
.IP \[bu] 2
|
|
fs \- a remote name string eg \[lq]drive:path/to/dir\[rq]
|
|
.PP
|
|
Returns
|
|
.IP \[bu] 2
|
|
count \- number of files
|
|
.IP \[bu] 2
|
|
bytes \- number of bytes in those files
|
|
.PP
|
|
See the size command (https://rclone.org/commands/rclone_size/) command
|
|
for more information on the above.
|
|
.PP
|
|
\f[B]Authentication is required for this call.\f[R]
|
|
.SS options/blocks: List all the option blocks
|
|
.PP
|
|
Returns \- options \- a list of the options block names
|
|
.SS options/get: Get all the options
|
|
.PP
|
|
Returns an object where keys are option block names and values are an
|
|
object with the current option values in.
|
|
.PP
|
|
This shows the internal names of the option within rclone which should
|
|
map to the external options very easily with a few exceptions.
|
|
.SS options/set: Set an option
|
|
.PP
|
|
Parameters
|
|
.IP \[bu] 2
|
|
option block name containing an object with
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
key: value
|
|
.RE
|
|
.PP
|
|
Repeated as often as required.
|
|
.PP
|
|
Only supply the options you wish to change.
|
|
If an option is unknown it will be silently ignored.
|
|
Not all options will have an effect when changed like this.
|
|
.PP
|
|
For example:
|
|
.PP
|
|
This sets DEBUG level logs (\-vv)
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone rc options/set \-\-json \[aq]{\[dq]main\[dq]: {\[dq]LogLevel\[dq]: 8}}\[aq]
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
And this sets INFO level logs (\-v)
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone rc options/set \-\-json \[aq]{\[dq]main\[dq]: {\[dq]LogLevel\[dq]: 7}}\[aq]
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
And this sets NOTICE level logs (normal without \-v)
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone rc options/set \-\-json \[aq]{\[dq]main\[dq]: {\[dq]LogLevel\[dq]: 6}}\[aq]
|
|
\f[R]
|
|
.fi
|
|
.SS rc/error: This returns an error
|
|
.PP
|
|
This returns an error with the input as part of its error string.
|
|
Useful for testing error handling.
|
|
.SS rc/list: List all the registered remote control commands
|
|
.PP
|
|
This lists all the registered remote control commands as a JSON map in
|
|
the commands response.
|
|
.SS rc/noop: Echo the input to the output parameters
|
|
.PP
|
|
This echoes the input parameters to the output parameters for testing
|
|
purposes.
|
|
It can be used to check that rclone is still alive and to check that
|
|
parameter passing is working properly.
|
|
.SS rc/noopauth: Echo the input to the output parameters requiring auth
|
|
.PP
|
|
This echoes the input parameters to the output parameters for testing
|
|
purposes.
|
|
It can be used to check that rclone is still alive and to check that
|
|
parameter passing is working properly.
|
|
.PP
|
|
\f[B]Authentication is required for this call.\f[R]
|
|
.SS sync/copy: copy a directory from source remote to destination remote
|
|
.PP
|
|
This takes the following parameters
|
|
.IP \[bu] 2
|
|
srcFs \- a remote name string eg \[lq]drive:src\[rq] for the source
|
|
.IP \[bu] 2
|
|
dstFs \- a remote name string eg \[lq]drive:dst\[rq] for the destination
|
|
.PP
|
|
See the copy command (https://rclone.org/commands/rclone_copy/) command
|
|
for more information on the above.
|
|
.PP
|
|
\f[B]Authentication is required for this call.\f[R]
|
|
.SS sync/move: move a directory from source remote to destination remote
|
|
.PP
|
|
This takes the following parameters
|
|
.IP \[bu] 2
|
|
srcFs \- a remote name string eg \[lq]drive:src\[rq] for the source
|
|
.IP \[bu] 2
|
|
dstFs \- a remote name string eg \[lq]drive:dst\[rq] for the destination
|
|
.IP \[bu] 2
|
|
deleteEmptySrcDirs \- delete empty src directories if set
|
|
.PP
|
|
See the move command (https://rclone.org/commands/rclone_move/) command
|
|
for more information on the above.
|
|
.PP
|
|
\f[B]Authentication is required for this call.\f[R]
|
|
.SS sync/sync: sync a directory from source remote to destination remote
|
|
.PP
|
|
This takes the following parameters
|
|
.IP \[bu] 2
|
|
srcFs \- a remote name string eg \[lq]drive:src\[rq] for the source
|
|
.IP \[bu] 2
|
|
dstFs \- a remote name string eg \[lq]drive:dst\[rq] for the destination
|
|
.PP
|
|
See the sync command (https://rclone.org/commands/rclone_sync/) command
|
|
for more information on the above.
|
|
.PP
|
|
\f[B]Authentication is required for this call.\f[R]
|
|
.SS vfs/forget: Forget files or directories in the directory cache.
|
|
.PP
|
|
This forgets the paths in the directory cache causing them to be
|
|
re\-read from the remote when needed.
|
|
.PP
|
|
If no paths are passed in then it will forget all the paths in the
|
|
directory cache.
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone rc vfs/forget
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
Otherwise pass files or dirs in as file=path or dir=path.
|
|
Any parameter key starting with file will forget that file and any
|
|
starting with dir will forget that dir, eg
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone rc vfs/forget file=hello file2=goodbye dir=home/junk
|
|
\f[R]
|
|
.fi
|
|
.SS vfs/poll\-interval: Get the status or update the value of the poll\-interval option.
|
|
.PP
|
|
Without any parameter given this returns the current status of the
|
|
poll\-interval setting.
|
|
.PP
|
|
When the interval=duration parameter is set, the poll\-interval value is
|
|
updated and the polling function is notified.
|
|
Setting interval=0 disables poll\-interval.
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone rc vfs/poll\-interval interval=5m
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
The timeout=duration parameter can be used to specify a time to wait for
|
|
the current poll function to apply the new value.
|
|
If timeout is less or equal 0, which is the default, wait indefinitely.
|
|
.PP
|
|
The new poll\-interval value will only be active when the timeout is not
|
|
reached.
|
|
.PP
|
|
If poll\-interval is updated or disabled temporarily, some changes might
|
|
not get picked up by the polling function, depending on the used remote.
|
|
.SS vfs/refresh: Refresh the directory cache.
|
|
.PP
|
|
This reads the directories for the specified paths and freshens the
|
|
directory cache.
|
|
.PP
|
|
If no paths are passed in then it will refresh the root directory.
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone rc vfs/refresh
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
Otherwise pass directories in as dir=path.
|
|
Any parameter key starting with dir will refresh that directory, eg
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone rc vfs/refresh dir=home/junk dir2=data/misc
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
If the parameter recursive=true is given the whole directory tree will
|
|
get refreshed.
|
|
This refresh will use \[en]fast\-list if enabled.
|
|
.SS Accessing the remote control via HTTP
|
|
.PP
|
|
Rclone implements a simple HTTP based protocol.
|
|
.PP
|
|
Each endpoint takes an JSON object and returns a JSON object or an
|
|
error.
|
|
The JSON objects are essentially a map of string names to values.
|
|
.PP
|
|
All calls must made using POST.
|
|
.PP
|
|
The input objects can be supplied using URL parameters, POST parameters
|
|
or by supplying \[lq]Content\-Type: application/json\[rq] and a JSON
|
|
blob in the body.
|
|
There are examples of these below using \f[C]curl\f[R].
|
|
.PP
|
|
The response will be a JSON blob in the body of the response.
|
|
This is formatted to be reasonably human readable.
|
|
.SS Error returns
|
|
.PP
|
|
If an error occurs then there will be an HTTP error status (eg 500) and
|
|
the body of the response will contain a JSON encoded error object, eg
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
{
|
|
\[dq]error\[dq]: \[dq]Expecting string value for key \[rs]\[dq]remote\[rs]\[dq] (was float64)\[dq],
|
|
\[dq]input\[dq]: {
|
|
\[dq]fs\[dq]: \[dq]/tmp\[dq],
|
|
\[dq]remote\[dq]: 3
|
|
},
|
|
\[dq]status\[dq]: 400
|
|
\[dq]path\[dq]: \[dq]operations/rmdir\[dq],
|
|
}
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
The keys in the error response are \- error \- error string \- input \-
|
|
the input parameters to the call \- status \- the HTTP status code \-
|
|
path \- the path of the call
|
|
.SS CORS
|
|
.PP
|
|
The sever implements basic CORS support and allows all origins for that.
|
|
The response to a preflight OPTIONS request will echo the requested
|
|
\[lq]Access\-Control\-Request\-Headers\[rq] back.
|
|
.SS Using POST with URL parameters only
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
curl \-X POST \[aq]http://localhost:5572/rc/noop?potato=1&sausage=2\[aq]
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
Response
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
{
|
|
\[dq]potato\[dq]: \[dq]1\[dq],
|
|
\[dq]sausage\[dq]: \[dq]2\[dq]
|
|
}
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
Here is what an error response looks like:
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
curl \-X POST \[aq]http://localhost:5572/rc/error?potato=1&sausage=2\[aq]
|
|
\f[R]
|
|
.fi
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
{
|
|
\[dq]error\[dq]: \[dq]arbitrary error on input map[potato:1 sausage:2]\[dq],
|
|
\[dq]input\[dq]: {
|
|
\[dq]potato\[dq]: \[dq]1\[dq],
|
|
\[dq]sausage\[dq]: \[dq]2\[dq]
|
|
}
|
|
}
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
Note that curl doesn\[cq]t return errors to the shell unless you use the
|
|
\f[C]\-f\f[R] option
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
$ curl \-f \-X POST \[aq]http://localhost:5572/rc/error?potato=1&sausage=2\[aq]
|
|
curl: (22) The requested URL returned error: 400 Bad Request
|
|
$ echo $?
|
|
22
|
|
\f[R]
|
|
.fi
|
|
.SS Using POST with a form
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
curl \-\-data \[dq]potato=1\[dq] \-\-data \[dq]sausage=2\[dq] http://localhost:5572/rc/noop
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
Response
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
{
|
|
\[dq]potato\[dq]: \[dq]1\[dq],
|
|
\[dq]sausage\[dq]: \[dq]2\[dq]
|
|
}
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
Note that you can combine these with URL parameters too with the POST
|
|
parameters taking precedence.
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
curl \-\-data \[dq]potato=1\[dq] \-\-data \[dq]sausage=2\[dq] \[dq]http://localhost:5572/rc/noop?rutabaga=3&sausage=4\[dq]
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
Response
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
{
|
|
\[dq]potato\[dq]: \[dq]1\[dq],
|
|
\[dq]rutabaga\[dq]: \[dq]3\[dq],
|
|
\[dq]sausage\[dq]: \[dq]4\[dq]
|
|
}
|
|
\f[R]
|
|
.fi
|
|
.SS Using POST with a JSON blob
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
curl \-H \[dq]Content\-Type: application/json\[dq] \-X POST \-d \[aq]{\[dq]potato\[dq]:2,\[dq]sausage\[dq]:1}\[aq] http://localhost:5572/rc/noop
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
response
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
{
|
|
\[dq]password\[dq]: \[dq]xyz\[dq],
|
|
\[dq]username\[dq]: \[dq]xyz\[dq]
|
|
}
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
This can be combined with URL parameters too if required.
|
|
The JSON blob takes precedence.
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
curl \-H \[dq]Content\-Type: application/json\[dq] \-X POST \-d \[aq]{\[dq]potato\[dq]:2,\[dq]sausage\[dq]:1}\[aq] \[aq]http://localhost:5572/rc/noop?rutabaga=3&potato=4\[aq]
|
|
\f[R]
|
|
.fi
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
{
|
|
\[dq]potato\[dq]: 2,
|
|
\[dq]rutabaga\[dq]: \[dq]3\[dq],
|
|
\[dq]sausage\[dq]: 1
|
|
}
|
|
\f[R]
|
|
.fi
|
|
.SS Debugging rclone with pprof
|
|
.PP
|
|
If you use the \f[C]\-\-rc\f[R] flag this will also enable the use of
|
|
the go profiling tools on the same port.
|
|
.PP
|
|
To use these, first install go (https://golang.org/doc/install).
|
|
.SS Debugging memory use
|
|
.PP
|
|
To profile rclone\[cq]s memory use you can run:
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
go tool pprof \-web http://localhost:5572/debug/pprof/heap
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
This should open a page in your browser showing what is using what
|
|
memory.
|
|
.PP
|
|
You can also use the \f[C]\-text\f[R] flag to produce a textual summary
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
$ go tool pprof \-text http://localhost:5572/debug/pprof/heap
|
|
Showing nodes accounting for 1537.03kB, 100% of 1537.03kB total
|
|
flat flat% sum% cum cum%
|
|
1024.03kB 66.62% 66.62% 1024.03kB 66.62% github.com/rclone/rclone/vendor/golang.org/x/net/http2/hpack.addDecoderNode
|
|
513kB 33.38% 100% 513kB 33.38% net/http.newBufioWriterSize
|
|
0 0% 100% 1024.03kB 66.62% github.com/rclone/rclone/cmd/all.init
|
|
0 0% 100% 1024.03kB 66.62% github.com/rclone/rclone/cmd/serve.init
|
|
0 0% 100% 1024.03kB 66.62% github.com/rclone/rclone/cmd/serve/restic.init
|
|
0 0% 100% 1024.03kB 66.62% github.com/rclone/rclone/vendor/golang.org/x/net/http2.init
|
|
0 0% 100% 1024.03kB 66.62% github.com/rclone/rclone/vendor/golang.org/x/net/http2/hpack.init
|
|
0 0% 100% 1024.03kB 66.62% github.com/rclone/rclone/vendor/golang.org/x/net/http2/hpack.init.0
|
|
0 0% 100% 1024.03kB 66.62% main.init
|
|
0 0% 100% 513kB 33.38% net/http.(*conn).readRequest
|
|
0 0% 100% 513kB 33.38% net/http.(*conn).serve
|
|
0 0% 100% 1024.03kB 66.62% runtime.main
|
|
\f[R]
|
|
.fi
|
|
.SS Debugging go routine leaks
|
|
.PP
|
|
Memory leaks are most often caused by go routine leaks keeping memory
|
|
alive which should have been garbage collected.
|
|
.PP
|
|
See all active go routines using
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
curl http://localhost:5572/debug/pprof/goroutine?debug=1
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
Or go to http://localhost:5572/debug/pprof/goroutine?debug=1 in your
|
|
browser.
|
|
.SS Other profiles to look at
|
|
.PP
|
|
You can see a summary of profiles available at
|
|
http://localhost:5572/debug/pprof/
|
|
.PP
|
|
Here is how to use some of them:
|
|
.IP \[bu] 2
|
|
Memory: \f[C]go tool pprof http://localhost:5572/debug/pprof/heap\f[R]
|
|
.IP \[bu] 2
|
|
Go routines:
|
|
\f[C]curl http://localhost:5572/debug/pprof/goroutine?debug=1\f[R]
|
|
.IP \[bu] 2
|
|
30\-second CPU profile:
|
|
\f[C]go tool pprof http://localhost:5572/debug/pprof/profile\f[R]
|
|
.IP \[bu] 2
|
|
5\-second execution trace:
|
|
\f[C]wget http://localhost:5572/debug/pprof/trace?seconds=5\f[R]
|
|
.IP \[bu] 2
|
|
Goroutine blocking profile
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Enable first with:
|
|
\f[C]rclone rc debug/set\-block\-profile\-rate rate=1\f[R] (docs)
|
|
.IP \[bu] 2
|
|
\f[C]go tool pprof http://localhost:5572/debug/pprof/block\f[R]
|
|
.RE
|
|
.IP \[bu] 2
|
|
Contended mutexes:
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Enable first with:
|
|
\f[C]rclone rc debug/set\-mutex\-profile\-fraction rate=1\f[R] (docs)
|
|
.IP \[bu] 2
|
|
\f[C]go tool pprof http://localhost:5572/debug/pprof/mutex\f[R]
|
|
.RE
|
|
.PP
|
|
See the net/http/pprof docs (https://golang.org/pkg/net/http/pprof/) for
|
|
more info on how to use the profiling and for a general overview see the
|
|
Go team\[cq]s blog post on profiling go
|
|
programs (https://blog.golang.org/profiling-go-programs).
|
|
.PP
|
|
The profiling hook is zero overhead unless it is
|
|
used (https://stackoverflow.com/q/26545159/164234).
|
|
.SH Overview of cloud storage systems
|
|
.PP
|
|
Each cloud storage system is slightly different.
|
|
Rclone attempts to provide a unified interface to them, but some
|
|
underlying differences show through.
|
|
.SS Features
|
|
.PP
|
|
Here is an overview of the major features of each cloud storage system.
|
|
.PP
|
|
.TS
|
|
tab(@);
|
|
l c c c c c.
|
|
T{
|
|
Name
|
|
T}@T{
|
|
Hash
|
|
T}@T{
|
|
ModTime
|
|
T}@T{
|
|
Case Insensitive
|
|
T}@T{
|
|
Duplicate Files
|
|
T}@T{
|
|
MIME Type
|
|
T}
|
|
_
|
|
T{
|
|
1Fichier
|
|
T}@T{
|
|
Whirlpool
|
|
T}@T{
|
|
No
|
|
T}@T{
|
|
No
|
|
T}@T{
|
|
Yes
|
|
T}@T{
|
|
R
|
|
T}
|
|
T{
|
|
Amazon Drive
|
|
T}@T{
|
|
MD5
|
|
T}@T{
|
|
No
|
|
T}@T{
|
|
Yes
|
|
T}@T{
|
|
No
|
|
T}@T{
|
|
R
|
|
T}
|
|
T{
|
|
Amazon S3
|
|
T}@T{
|
|
MD5
|
|
T}@T{
|
|
Yes
|
|
T}@T{
|
|
No
|
|
T}@T{
|
|
No
|
|
T}@T{
|
|
R/W
|
|
T}
|
|
T{
|
|
Backblaze B2
|
|
T}@T{
|
|
SHA1
|
|
T}@T{
|
|
Yes
|
|
T}@T{
|
|
No
|
|
T}@T{
|
|
No
|
|
T}@T{
|
|
R/W
|
|
T}
|
|
T{
|
|
Box
|
|
T}@T{
|
|
SHA1
|
|
T}@T{
|
|
Yes
|
|
T}@T{
|
|
Yes
|
|
T}@T{
|
|
No
|
|
T}@T{
|
|
\-
|
|
T}
|
|
T{
|
|
Citrix ShareFile
|
|
T}@T{
|
|
MD5
|
|
T}@T{
|
|
Yes
|
|
T}@T{
|
|
Yes
|
|
T}@T{
|
|
No
|
|
T}@T{
|
|
\-
|
|
T}
|
|
T{
|
|
Dropbox
|
|
T}@T{
|
|
DBHASH \[dg]
|
|
T}@T{
|
|
Yes
|
|
T}@T{
|
|
Yes
|
|
T}@T{
|
|
No
|
|
T}@T{
|
|
\-
|
|
T}
|
|
T{
|
|
FTP
|
|
T}@T{
|
|
\-
|
|
T}@T{
|
|
No
|
|
T}@T{
|
|
No
|
|
T}@T{
|
|
No
|
|
T}@T{
|
|
\-
|
|
T}
|
|
T{
|
|
Google Cloud Storage
|
|
T}@T{
|
|
MD5
|
|
T}@T{
|
|
Yes
|
|
T}@T{
|
|
No
|
|
T}@T{
|
|
No
|
|
T}@T{
|
|
R/W
|
|
T}
|
|
T{
|
|
Google Drive
|
|
T}@T{
|
|
MD5
|
|
T}@T{
|
|
Yes
|
|
T}@T{
|
|
No
|
|
T}@T{
|
|
Yes
|
|
T}@T{
|
|
R/W
|
|
T}
|
|
T{
|
|
Google Photos
|
|
T}@T{
|
|
\-
|
|
T}@T{
|
|
No
|
|
T}@T{
|
|
No
|
|
T}@T{
|
|
Yes
|
|
T}@T{
|
|
R
|
|
T}
|
|
T{
|
|
HTTP
|
|
T}@T{
|
|
\-
|
|
T}@T{
|
|
No
|
|
T}@T{
|
|
No
|
|
T}@T{
|
|
No
|
|
T}@T{
|
|
R
|
|
T}
|
|
T{
|
|
Hubic
|
|
T}@T{
|
|
MD5
|
|
T}@T{
|
|
Yes
|
|
T}@T{
|
|
No
|
|
T}@T{
|
|
No
|
|
T}@T{
|
|
R/W
|
|
T}
|
|
T{
|
|
Jottacloud
|
|
T}@T{
|
|
MD5
|
|
T}@T{
|
|
Yes
|
|
T}@T{
|
|
Yes
|
|
T}@T{
|
|
No
|
|
T}@T{
|
|
R/W
|
|
T}
|
|
T{
|
|
Koofr
|
|
T}@T{
|
|
MD5
|
|
T}@T{
|
|
No
|
|
T}@T{
|
|
Yes
|
|
T}@T{
|
|
No
|
|
T}@T{
|
|
\-
|
|
T}
|
|
T{
|
|
Mail.ru Cloud
|
|
T}@T{
|
|
Mailru \[dd]\[dd]\[dd]
|
|
T}@T{
|
|
Yes
|
|
T}@T{
|
|
Yes
|
|
T}@T{
|
|
No
|
|
T}@T{
|
|
\-
|
|
T}
|
|
T{
|
|
Mega
|
|
T}@T{
|
|
\-
|
|
T}@T{
|
|
No
|
|
T}@T{
|
|
No
|
|
T}@T{
|
|
Yes
|
|
T}@T{
|
|
\-
|
|
T}
|
|
T{
|
|
Memory
|
|
T}@T{
|
|
MD5
|
|
T}@T{
|
|
Yes
|
|
T}@T{
|
|
No
|
|
T}@T{
|
|
No
|
|
T}@T{
|
|
\-
|
|
T}
|
|
T{
|
|
Microsoft Azure Blob Storage
|
|
T}@T{
|
|
MD5
|
|
T}@T{
|
|
Yes
|
|
T}@T{
|
|
No
|
|
T}@T{
|
|
No
|
|
T}@T{
|
|
R/W
|
|
T}
|
|
T{
|
|
Microsoft OneDrive
|
|
T}@T{
|
|
SHA1 \[dd]\[dd]
|
|
T}@T{
|
|
Yes
|
|
T}@T{
|
|
Yes
|
|
T}@T{
|
|
No
|
|
T}@T{
|
|
R
|
|
T}
|
|
T{
|
|
OpenDrive
|
|
T}@T{
|
|
MD5
|
|
T}@T{
|
|
Yes
|
|
T}@T{
|
|
Yes
|
|
T}@T{
|
|
No
|
|
T}@T{
|
|
\-
|
|
T}
|
|
T{
|
|
OpenStack Swift
|
|
T}@T{
|
|
MD5
|
|
T}@T{
|
|
Yes
|
|
T}@T{
|
|
No
|
|
T}@T{
|
|
No
|
|
T}@T{
|
|
R/W
|
|
T}
|
|
T{
|
|
pCloud
|
|
T}@T{
|
|
MD5, SHA1
|
|
T}@T{
|
|
Yes
|
|
T}@T{
|
|
No
|
|
T}@T{
|
|
No
|
|
T}@T{
|
|
W
|
|
T}
|
|
T{
|
|
premiumize.me
|
|
T}@T{
|
|
\-
|
|
T}@T{
|
|
No
|
|
T}@T{
|
|
Yes
|
|
T}@T{
|
|
No
|
|
T}@T{
|
|
R
|
|
T}
|
|
T{
|
|
put.io
|
|
T}@T{
|
|
CRC\-32
|
|
T}@T{
|
|
Yes
|
|
T}@T{
|
|
No
|
|
T}@T{
|
|
Yes
|
|
T}@T{
|
|
R
|
|
T}
|
|
T{
|
|
QingStor
|
|
T}@T{
|
|
MD5
|
|
T}@T{
|
|
No
|
|
T}@T{
|
|
No
|
|
T}@T{
|
|
No
|
|
T}@T{
|
|
R/W
|
|
T}
|
|
T{
|
|
Seafile
|
|
T}@T{
|
|
\-
|
|
T}@T{
|
|
No
|
|
T}@T{
|
|
No
|
|
T}@T{
|
|
No
|
|
T}@T{
|
|
\-
|
|
T}
|
|
T{
|
|
SFTP
|
|
T}@T{
|
|
MD5, SHA1 \[dd]
|
|
T}@T{
|
|
Yes
|
|
T}@T{
|
|
Depends
|
|
T}@T{
|
|
No
|
|
T}@T{
|
|
\-
|
|
T}
|
|
T{
|
|
SugarSync
|
|
T}@T{
|
|
\-
|
|
T}@T{
|
|
No
|
|
T}@T{
|
|
No
|
|
T}@T{
|
|
No
|
|
T}@T{
|
|
\-
|
|
T}
|
|
T{
|
|
Tardigrade
|
|
T}@T{
|
|
\-
|
|
T}@T{
|
|
Yes
|
|
T}@T{
|
|
No
|
|
T}@T{
|
|
No
|
|
T}@T{
|
|
\-
|
|
T}
|
|
T{
|
|
WebDAV
|
|
T}@T{
|
|
MD5, SHA1 \[dg]\[dg]
|
|
T}@T{
|
|
Yes \[dg]\[dg]\[dg]
|
|
T}@T{
|
|
Depends
|
|
T}@T{
|
|
No
|
|
T}@T{
|
|
\-
|
|
T}
|
|
T{
|
|
Yandex Disk
|
|
T}@T{
|
|
MD5
|
|
T}@T{
|
|
Yes
|
|
T}@T{
|
|
No
|
|
T}@T{
|
|
No
|
|
T}@T{
|
|
R/W
|
|
T}
|
|
T{
|
|
The local filesystem
|
|
T}@T{
|
|
All
|
|
T}@T{
|
|
Yes
|
|
T}@T{
|
|
Depends
|
|
T}@T{
|
|
No
|
|
T}@T{
|
|
\-
|
|
T}
|
|
.TE
|
|
.SS Hash
|
|
.PP
|
|
The cloud storage system supports various hash types of the objects.
|
|
The hashes are used when transferring data as an integrity check and can
|
|
be specifically used with the \f[C]\-\-checksum\f[R] flag in syncs and
|
|
in the \f[C]check\f[R] command.
|
|
.PP
|
|
To use the verify checksums when transferring between cloud storage
|
|
systems they must support a common hash type.
|
|
.PP
|
|
\[dg] Note that Dropbox supports its own custom
|
|
hash (https://www.dropbox.com/developers/reference/content-hash).
|
|
This is an SHA256 sum of all the 4MB block SHA256s.
|
|
.PP
|
|
\[dd] SFTP supports checksums if the same login has shell access and
|
|
\f[C]md5sum\f[R] or \f[C]sha1sum\f[R] as well as \f[C]echo\f[R] are in
|
|
the remote\[cq]s PATH.
|
|
.PP
|
|
\[dg]\[dg] WebDAV supports hashes when used with Owncloud and Nextcloud
|
|
only.
|
|
.PP
|
|
\[dg]\[dg]\[dg] WebDAV supports modtimes when used with Owncloud and
|
|
Nextcloud only.
|
|
.PP
|
|
\[dd]\[dd] Microsoft OneDrive Personal supports SHA1 hashes, whereas
|
|
OneDrive for business and SharePoint server support Microsoft\[cq]s own
|
|
QuickXorHash (https://docs.microsoft.com/en-us/onedrive/developer/code-snippets/quickxorhash).
|
|
.PP
|
|
\[dd]\[dd]\[dd] Mail.ru uses its own modified SHA1 hash
|
|
.SS ModTime
|
|
.PP
|
|
The cloud storage system supports setting modification times on objects.
|
|
If it does then this enables a using the modification times as part of
|
|
the sync.
|
|
If not then only the size will be checked by default, though the MD5SUM
|
|
can be checked with the \f[C]\-\-checksum\f[R] flag.
|
|
.PP
|
|
All cloud storage systems support some kind of date on the object and
|
|
these will be set when transferring from the cloud storage system.
|
|
.SS Case Insensitive
|
|
.PP
|
|
If a cloud storage systems is case sensitive then it is possible to have
|
|
two files which differ only in case, eg \f[C]file.txt\f[R] and
|
|
\f[C]FILE.txt\f[R].
|
|
If a cloud storage system is case insensitive then that isn\[cq]t
|
|
possible.
|
|
.PP
|
|
This can cause problems when syncing between a case insensitive system
|
|
and a case sensitive system.
|
|
The symptom of this is that no matter how many times you run the sync it
|
|
never completes fully.
|
|
.PP
|
|
The local filesystem and SFTP may or may not be case sensitive depending
|
|
on OS.
|
|
.IP \[bu] 2
|
|
Windows \- usually case insensitive, though case is preserved
|
|
.IP \[bu] 2
|
|
OSX \- usually case insensitive, though it is possible to format case
|
|
sensitive
|
|
.IP \[bu] 2
|
|
Linux \- usually case sensitive, but there are case insensitive file
|
|
systems (eg FAT formatted USB keys)
|
|
.PP
|
|
Most of the time this doesn\[cq]t cause any problems as people tend to
|
|
avoid files whose name differs only by case even on case sensitive
|
|
systems.
|
|
.SS Duplicate files
|
|
.PP
|
|
If a cloud storage system allows duplicate files then it can have two
|
|
objects with the same name.
|
|
.PP
|
|
This confuses rclone greatly when syncing \- use the
|
|
\f[C]rclone dedupe\f[R] command to rename or remove duplicates.
|
|
.SS Restricted filenames
|
|
.PP
|
|
Some cloud storage systems might have restrictions on the characters
|
|
that are usable in file or directory names.
|
|
When \f[C]rclone\f[R] detects such a name during a file upload, it will
|
|
transparently replace the restricted characters with similar looking
|
|
Unicode characters.
|
|
.PP
|
|
This process is designed to avoid ambiguous file names as much as
|
|
possible and allow to move files between many cloud storage systems
|
|
transparently.
|
|
.PP
|
|
The name shown by \f[C]rclone\f[R] to the user or during log output will
|
|
only contain a minimal set of replaced characters to ensure correct
|
|
formatting and not necessarily the actual name used on the cloud
|
|
storage.
|
|
.PP
|
|
This transformation is reversed when downloading a file or parsing
|
|
\f[C]rclone\f[R] arguments.
|
|
For example, when uploading a file named \f[C]my file?.txt\f[R] to
|
|
Onedrive will be displayed as \f[C]my file?.txt\f[R] on the console, but
|
|
stored as \f[C]my file\[uFF1F].txt\f[R] (the \f[C]?\f[R] gets replaced
|
|
by the similar looking \f[C]\[uFF1F]\f[R] character) to Onedrive.
|
|
The reverse transformation allows to read a
|
|
file\f[C]unusual/name.txt\f[R] from Google Drive, by passing the name
|
|
\f[C]unusual\[uFF0F]name.txt\f[R] (the \f[C]/\f[R] needs to be replaced
|
|
by the similar looking \f[C]\[uFF0F]\f[R] character) on the command
|
|
line.
|
|
.SS Default restricted characters
|
|
.PP
|
|
The table below shows the characters that are replaced by default.
|
|
.PP
|
|
When a replacement character is found in a filename, this character will
|
|
be escaped with the \f[C]\[u201B]\f[R] character to avoid ambiguous file
|
|
names.
|
|
(e.g.\ a file named \f[C]\[u2400].txt\f[R] would shown as
|
|
\f[C]\[u201B]\[u2400].txt\f[R])
|
|
.PP
|
|
Each cloud storage backend can use a different set of characters, which
|
|
will be specified in the documentation for each backend.
|
|
.PP
|
|
.TS
|
|
tab(@);
|
|
l c c.
|
|
T{
|
|
Character
|
|
T}@T{
|
|
Value
|
|
T}@T{
|
|
Replacement
|
|
T}
|
|
_
|
|
T{
|
|
NUL
|
|
T}@T{
|
|
0x00
|
|
T}@T{
|
|
\[u2400]
|
|
T}
|
|
T{
|
|
SOH
|
|
T}@T{
|
|
0x01
|
|
T}@T{
|
|
\[u2401]
|
|
T}
|
|
T{
|
|
STX
|
|
T}@T{
|
|
0x02
|
|
T}@T{
|
|
\[u2402]
|
|
T}
|
|
T{
|
|
ETX
|
|
T}@T{
|
|
0x03
|
|
T}@T{
|
|
\[u2403]
|
|
T}
|
|
T{
|
|
EOT
|
|
T}@T{
|
|
0x04
|
|
T}@T{
|
|
\[u2404]
|
|
T}
|
|
T{
|
|
ENQ
|
|
T}@T{
|
|
0x05
|
|
T}@T{
|
|
\[u2405]
|
|
T}
|
|
T{
|
|
ACK
|
|
T}@T{
|
|
0x06
|
|
T}@T{
|
|
\[u2406]
|
|
T}
|
|
T{
|
|
BEL
|
|
T}@T{
|
|
0x07
|
|
T}@T{
|
|
\[u2407]
|
|
T}
|
|
T{
|
|
BS
|
|
T}@T{
|
|
0x08
|
|
T}@T{
|
|
\[u2408]
|
|
T}
|
|
T{
|
|
HT
|
|
T}@T{
|
|
0x09
|
|
T}@T{
|
|
\[u2409]
|
|
T}
|
|
T{
|
|
LF
|
|
T}@T{
|
|
0x0A
|
|
T}@T{
|
|
\[u240A]
|
|
T}
|
|
T{
|
|
VT
|
|
T}@T{
|
|
0x0B
|
|
T}@T{
|
|
\[u240B]
|
|
T}
|
|
T{
|
|
FF
|
|
T}@T{
|
|
0x0C
|
|
T}@T{
|
|
\[u240C]
|
|
T}
|
|
T{
|
|
CR
|
|
T}@T{
|
|
0x0D
|
|
T}@T{
|
|
\[u240D]
|
|
T}
|
|
T{
|
|
SO
|
|
T}@T{
|
|
0x0E
|
|
T}@T{
|
|
\[u240E]
|
|
T}
|
|
T{
|
|
SI
|
|
T}@T{
|
|
0x0F
|
|
T}@T{
|
|
\[u240F]
|
|
T}
|
|
T{
|
|
DLE
|
|
T}@T{
|
|
0x10
|
|
T}@T{
|
|
\[u2410]
|
|
T}
|
|
T{
|
|
DC1
|
|
T}@T{
|
|
0x11
|
|
T}@T{
|
|
\[u2411]
|
|
T}
|
|
T{
|
|
DC2
|
|
T}@T{
|
|
0x12
|
|
T}@T{
|
|
\[u2412]
|
|
T}
|
|
T{
|
|
DC3
|
|
T}@T{
|
|
0x13
|
|
T}@T{
|
|
\[u2413]
|
|
T}
|
|
T{
|
|
DC4
|
|
T}@T{
|
|
0x14
|
|
T}@T{
|
|
\[u2414]
|
|
T}
|
|
T{
|
|
NAK
|
|
T}@T{
|
|
0x15
|
|
T}@T{
|
|
\[u2415]
|
|
T}
|
|
T{
|
|
SYN
|
|
T}@T{
|
|
0x16
|
|
T}@T{
|
|
\[u2416]
|
|
T}
|
|
T{
|
|
ETB
|
|
T}@T{
|
|
0x17
|
|
T}@T{
|
|
\[u2417]
|
|
T}
|
|
T{
|
|
CAN
|
|
T}@T{
|
|
0x18
|
|
T}@T{
|
|
\[u2418]
|
|
T}
|
|
T{
|
|
EM
|
|
T}@T{
|
|
0x19
|
|
T}@T{
|
|
\[u2419]
|
|
T}
|
|
T{
|
|
SUB
|
|
T}@T{
|
|
0x1A
|
|
T}@T{
|
|
\[u241A]
|
|
T}
|
|
T{
|
|
ESC
|
|
T}@T{
|
|
0x1B
|
|
T}@T{
|
|
\[u241B]
|
|
T}
|
|
T{
|
|
FS
|
|
T}@T{
|
|
0x1C
|
|
T}@T{
|
|
\[u241C]
|
|
T}
|
|
T{
|
|
GS
|
|
T}@T{
|
|
0x1D
|
|
T}@T{
|
|
\[u241D]
|
|
T}
|
|
T{
|
|
RS
|
|
T}@T{
|
|
0x1E
|
|
T}@T{
|
|
\[u241E]
|
|
T}
|
|
T{
|
|
US
|
|
T}@T{
|
|
0x1F
|
|
T}@T{
|
|
\[u241F]
|
|
T}
|
|
T{
|
|
/
|
|
T}@T{
|
|
0x2F
|
|
T}@T{
|
|
\[uFF0F]
|
|
T}
|
|
T{
|
|
DEL
|
|
T}@T{
|
|
0x7F
|
|
T}@T{
|
|
\[u2421]
|
|
T}
|
|
.TE
|
|
.PP
|
|
The default encoding will also encode these file names as they are
|
|
problematic with many cloud storage systems.
|
|
.PP
|
|
.TS
|
|
tab(@);
|
|
l c.
|
|
T{
|
|
File name
|
|
T}@T{
|
|
Replacement
|
|
T}
|
|
_
|
|
T{
|
|
\&.
|
|
T}@T{
|
|
\[uFF0E]
|
|
T}
|
|
T{
|
|
\&..
|
|
T}@T{
|
|
\[uFF0E]\[uFF0E]
|
|
T}
|
|
.TE
|
|
.SS Invalid UTF\-8 bytes
|
|
.PP
|
|
Some backends only support a sequence of well formed UTF\-8 bytes as
|
|
file or directory names.
|
|
.PP
|
|
In this case all invalid UTF\-8 bytes will be replaced with a quoted
|
|
representation of the byte value to allow uploading a file to such a
|
|
backend.
|
|
For example, the invalid byte \f[C]0xFE\f[R] will be encoded as
|
|
\f[C]\[u201B]FE\f[R].
|
|
.PP
|
|
A common source of invalid UTF\-8 bytes are local filesystems, that
|
|
store names in a different encoding than UTF\-8 or UTF\-16, like latin1.
|
|
See the local filenames (https://rclone.org/local/#filenames) section
|
|
for details.
|
|
.SS Encoding option
|
|
.PP
|
|
Most backends have an encoding options, specified as a flag
|
|
\f[C]\-\-backend\-encoding\f[R] where \f[C]backend\f[R] is the name of
|
|
the backend, or as a config parameter \f[C]encoding\f[R] (you\[cq]ll
|
|
need to select the Advanced config in \f[C]rclone config\f[R] to see
|
|
it).
|
|
.PP
|
|
This will have default value which encodes and decodes characters in
|
|
such a way as to preserve the maximum number of characters (see above).
|
|
.PP
|
|
However this can be incorrect in some scenarios, for example if you have
|
|
a Windows file system with characters such as \f[C]\[uFF0A]\f[R] and
|
|
\f[C]\[uFF1F]\f[R] that you want to remain as those characters on the
|
|
remote rather than being translated to \f[C]*\f[R] and \f[C]?\f[R].
|
|
.PP
|
|
The \f[C]\-\-backend\-encoding\f[R] flags allow you to change that.
|
|
You can disable the encoding completely with
|
|
\f[C]\-\-backend\-encoding None\f[R] or set \f[C]encoding = None\f[R] in
|
|
the config file.
|
|
.PP
|
|
Encoding takes a comma separated list of encodings.
|
|
You can see the list of all available characters by passing an invalid
|
|
value to this flag, eg \f[C]\-\-local\-encoding \[dq]help\[dq]\f[R] and
|
|
\f[C]rclone help flags encoding\f[R] will show you the defaults for the
|
|
backends.
|
|
.PP
|
|
.TS
|
|
tab(@);
|
|
l l.
|
|
T{
|
|
Encoding
|
|
T}@T{
|
|
Characters
|
|
T}
|
|
_
|
|
T{
|
|
Asterisk
|
|
T}@T{
|
|
\f[C]*\f[R]
|
|
T}
|
|
T{
|
|
BackQuote
|
|
T}@T{
|
|
\f[C]\[ga]\f[R]
|
|
T}
|
|
T{
|
|
BackSlash
|
|
T}@T{
|
|
\f[C]\[rs]\f[R]
|
|
T}
|
|
T{
|
|
Colon
|
|
T}@T{
|
|
\f[C]:\f[R]
|
|
T}
|
|
T{
|
|
CrLf
|
|
T}@T{
|
|
CR 0x0D, LF 0x0A
|
|
T}
|
|
T{
|
|
Ctl
|
|
T}@T{
|
|
All control characters 0x00\-0x1F
|
|
T}
|
|
T{
|
|
Del
|
|
T}@T{
|
|
DEL 0x7F
|
|
T}
|
|
T{
|
|
Dollar
|
|
T}@T{
|
|
\f[C]$\f[R]
|
|
T}
|
|
T{
|
|
Dot
|
|
T}@T{
|
|
\f[C].\f[R]
|
|
T}
|
|
T{
|
|
DoubleQuote
|
|
T}@T{
|
|
\f[C]\[dq]\f[R]
|
|
T}
|
|
T{
|
|
Hash
|
|
T}@T{
|
|
\f[C]#\f[R]
|
|
T}
|
|
T{
|
|
InvalidUtf8
|
|
T}@T{
|
|
An invalid UTF\-8 character (eg latin1)
|
|
T}
|
|
T{
|
|
LeftCrLfHtVt
|
|
T}@T{
|
|
CR 0x0D, LF 0x0A,HT 0x09, VT 0x0B on the left of a string
|
|
T}
|
|
T{
|
|
LeftPeriod
|
|
T}@T{
|
|
\f[C].\f[R] on the left of a string
|
|
T}
|
|
T{
|
|
LeftSpace
|
|
T}@T{
|
|
SPACE on the left of a string
|
|
T}
|
|
T{
|
|
LeftTilde
|
|
T}@T{
|
|
\f[C]\[ti]\f[R] on the left of a string
|
|
T}
|
|
T{
|
|
LtGt
|
|
T}@T{
|
|
\f[C]<\f[R], \f[C]>\f[R]
|
|
T}
|
|
T{
|
|
None
|
|
T}@T{
|
|
No characters are encoded
|
|
T}
|
|
T{
|
|
Percent
|
|
T}@T{
|
|
\f[C]%\f[R]
|
|
T}
|
|
T{
|
|
Pipe
|
|
T}@T{
|
|
|
|
|
T}
|
|
T{
|
|
Question
|
|
T}@T{
|
|
\f[C]?\f[R]
|
|
T}
|
|
T{
|
|
RightCrLfHtVt
|
|
T}@T{
|
|
CR 0x0D, LF 0x0A, HT 0x09, VT 0x0B on the right of a string
|
|
T}
|
|
T{
|
|
RightPeriod
|
|
T}@T{
|
|
\f[C].\f[R] on the right of a string
|
|
T}
|
|
T{
|
|
RightSpace
|
|
T}@T{
|
|
SPACE on the right of a string
|
|
T}
|
|
T{
|
|
SingleQuote
|
|
T}@T{
|
|
\f[C]\[aq]\f[R]
|
|
T}
|
|
T{
|
|
Slash
|
|
T}@T{
|
|
\f[C]/\f[R]
|
|
T}
|
|
.TE
|
|
.PP
|
|
To take a specific example, the FTP backend\[cq]s default encoding is
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
\-\-ftp\-encoding \[dq]Slash,Del,Ctl,RightSpace,Dot\[dq]
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
However, let\[cq]s say the FTP server is running on Windows and
|
|
can\[cq]t have any of the invalid Windows characters in file names.
|
|
You are backing up Linux servers to this FTP server which do have those
|
|
characters in file names.
|
|
So you would add the Windows set which are
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
Slash,LtGt,DoubleQuote,Colon,Question,Asterisk,Pipe,BackSlash,Ctl,RightSpace,RightPeriod,InvalidUtf8,Dot
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
to the existing ones, giving:
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
Slash,LtGt,DoubleQuote,Colon,Question,Asterisk,Pipe,BackSlash,Ctl,RightSpace,RightPeriod,InvalidUtf8,Dot,Del,RightSpace
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
This can be specified using the \f[C]\-\-ftp\-encoding\f[R] flag or
|
|
using an \f[C]encoding\f[R] parameter in the config file.
|
|
.PP
|
|
Or let\[cq]s say you have a Windows server but you want to preserve
|
|
\f[C]\[uFF0A]\f[R] and \f[C]\[uFF1F]\f[R], you would then have this as
|
|
the encoding (the Windows encoding minus \f[C]Asterisk\f[R] and
|
|
\f[C]Question\f[R]).
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
Slash,LtGt,DoubleQuote,Colon,Pipe,BackSlash,Ctl,RightSpace,RightPeriod,InvalidUtf8,Dot
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
This can be specified using the \f[C]\-\-local\-encoding\f[R] flag or
|
|
using an \f[C]encoding\f[R] parameter in the config file.
|
|
.SS MIME Type
|
|
.PP
|
|
MIME types (also known as media types) classify types of documents using
|
|
a simple text classification, eg \f[C]text/html\f[R] or
|
|
\f[C]application/pdf\f[R].
|
|
.PP
|
|
Some cloud storage systems support reading (\f[C]R\f[R]) the MIME type
|
|
of objects and some support writing (\f[C]W\f[R]) the MIME type of
|
|
objects.
|
|
.PP
|
|
The MIME type can be important if you are serving files directly to HTTP
|
|
from the storage system.
|
|
.PP
|
|
If you are copying from a remote which supports reading (\f[C]R\f[R]) to
|
|
a remote which supports writing (\f[C]W\f[R]) then rclone will preserve
|
|
the MIME types.
|
|
Otherwise they will be guessed from the extension, or the remote itself
|
|
may assign the MIME type.
|
|
.SS Optional Features
|
|
.PP
|
|
All the remotes support a basic set of features, but there are some
|
|
optional features supported by some remotes used to make some operations
|
|
more efficient.
|
|
.PP
|
|
.TS
|
|
tab(@);
|
|
l c c c c c c c c c c.
|
|
T{
|
|
Name
|
|
T}@T{
|
|
Purge
|
|
T}@T{
|
|
Copy
|
|
T}@T{
|
|
Move
|
|
T}@T{
|
|
DirMove
|
|
T}@T{
|
|
CleanUp
|
|
T}@T{
|
|
ListR
|
|
T}@T{
|
|
StreamUpload
|
|
T}@T{
|
|
LinkSharing
|
|
T}@T{
|
|
About
|
|
T}@T{
|
|
EmptyDir
|
|
T}
|
|
_
|
|
T{
|
|
1Fichier
|
|
T}@T{
|
|
No
|
|
T}@T{
|
|
No
|
|
T}@T{
|
|
No
|
|
T}@T{
|
|
No
|
|
T}@T{
|
|
No
|
|
T}@T{
|
|
No
|
|
T}@T{
|
|
No
|
|
T}@T{
|
|
No
|
|
T}@T{
|
|
No
|
|
T}@T{
|
|
Yes
|
|
T}
|
|
T{
|
|
Amazon Drive
|
|
T}@T{
|
|
Yes
|
|
T}@T{
|
|
No
|
|
T}@T{
|
|
Yes
|
|
T}@T{
|
|
Yes
|
|
T}@T{
|
|
No #575 (https://github.com/rclone/rclone/issues/575)
|
|
T}@T{
|
|
No
|
|
T}@T{
|
|
No
|
|
T}@T{
|
|
No #2178 (https://github.com/rclone/rclone/issues/2178)
|
|
T}@T{
|
|
No
|
|
T}@T{
|
|
Yes
|
|
T}
|
|
T{
|
|
Amazon S3
|
|
T}@T{
|
|
No
|
|
T}@T{
|
|
Yes
|
|
T}@T{
|
|
No
|
|
T}@T{
|
|
No
|
|
T}@T{
|
|
No
|
|
T}@T{
|
|
Yes
|
|
T}@T{
|
|
Yes
|
|
T}@T{
|
|
No #2178 (https://github.com/rclone/rclone/issues/2178)
|
|
T}@T{
|
|
No
|
|
T}@T{
|
|
No
|
|
T}
|
|
T{
|
|
Backblaze B2
|
|
T}@T{
|
|
No
|
|
T}@T{
|
|
Yes
|
|
T}@T{
|
|
No
|
|
T}@T{
|
|
No
|
|
T}@T{
|
|
Yes
|
|
T}@T{
|
|
Yes
|
|
T}@T{
|
|
Yes
|
|
T}@T{
|
|
Yes
|
|
T}@T{
|
|
No
|
|
T}@T{
|
|
No
|
|
T}
|
|
T{
|
|
Box
|
|
T}@T{
|
|
Yes
|
|
T}@T{
|
|
Yes
|
|
T}@T{
|
|
Yes
|
|
T}@T{
|
|
Yes
|
|
T}@T{
|
|
No #575 (https://github.com/rclone/rclone/issues/575)
|
|
T}@T{
|
|
No
|
|
T}@T{
|
|
Yes
|
|
T}@T{
|
|
Yes
|
|
T}@T{
|
|
No
|
|
T}@T{
|
|
Yes
|
|
T}
|
|
T{
|
|
Citrix ShareFile
|
|
T}@T{
|
|
Yes
|
|
T}@T{
|
|
Yes
|
|
T}@T{
|
|
Yes
|
|
T}@T{
|
|
Yes
|
|
T}@T{
|
|
No
|
|
T}@T{
|
|
No
|
|
T}@T{
|
|
Yes
|
|
T}@T{
|
|
No
|
|
T}@T{
|
|
No
|
|
T}@T{
|
|
Yes
|
|
T}
|
|
T{
|
|
Dropbox
|
|
T}@T{
|
|
Yes
|
|
T}@T{
|
|
Yes
|
|
T}@T{
|
|
Yes
|
|
T}@T{
|
|
Yes
|
|
T}@T{
|
|
No #575 (https://github.com/rclone/rclone/issues/575)
|
|
T}@T{
|
|
No
|
|
T}@T{
|
|
Yes
|
|
T}@T{
|
|
Yes
|
|
T}@T{
|
|
Yes
|
|
T}@T{
|
|
Yes
|
|
T}
|
|
T{
|
|
FTP
|
|
T}@T{
|
|
No
|
|
T}@T{
|
|
No
|
|
T}@T{
|
|
Yes
|
|
T}@T{
|
|
Yes
|
|
T}@T{
|
|
No
|
|
T}@T{
|
|
No
|
|
T}@T{
|
|
Yes
|
|
T}@T{
|
|
No #2178 (https://github.com/rclone/rclone/issues/2178)
|
|
T}@T{
|
|
No
|
|
T}@T{
|
|
Yes
|
|
T}
|
|
T{
|
|
Google Cloud Storage
|
|
T}@T{
|
|
Yes
|
|
T}@T{
|
|
Yes
|
|
T}@T{
|
|
No
|
|
T}@T{
|
|
No
|
|
T}@T{
|
|
No
|
|
T}@T{
|
|
Yes
|
|
T}@T{
|
|
Yes
|
|
T}@T{
|
|
No #2178 (https://github.com/rclone/rclone/issues/2178)
|
|
T}@T{
|
|
No
|
|
T}@T{
|
|
No
|
|
T}
|
|
T{
|
|
Google Drive
|
|
T}@T{
|
|
Yes
|
|
T}@T{
|
|
Yes
|
|
T}@T{
|
|
Yes
|
|
T}@T{
|
|
Yes
|
|
T}@T{
|
|
Yes
|
|
T}@T{
|
|
Yes
|
|
T}@T{
|
|
Yes
|
|
T}@T{
|
|
Yes
|
|
T}@T{
|
|
Yes
|
|
T}@T{
|
|
Yes
|
|
T}
|
|
T{
|
|
Google Photos
|
|
T}@T{
|
|
No
|
|
T}@T{
|
|
No
|
|
T}@T{
|
|
No
|
|
T}@T{
|
|
No
|
|
T}@T{
|
|
No
|
|
T}@T{
|
|
No
|
|
T}@T{
|
|
No
|
|
T}@T{
|
|
No
|
|
T}@T{
|
|
No
|
|
T}@T{
|
|
No
|
|
T}
|
|
T{
|
|
HTTP
|
|
T}@T{
|
|
No
|
|
T}@T{
|
|
No
|
|
T}@T{
|
|
No
|
|
T}@T{
|
|
No
|
|
T}@T{
|
|
No
|
|
T}@T{
|
|
No
|
|
T}@T{
|
|
No
|
|
T}@T{
|
|
No #2178 (https://github.com/rclone/rclone/issues/2178)
|
|
T}@T{
|
|
No
|
|
T}@T{
|
|
Yes
|
|
T}
|
|
T{
|
|
Hubic
|
|
T}@T{
|
|
Yes \[dg]
|
|
T}@T{
|
|
Yes
|
|
T}@T{
|
|
No
|
|
T}@T{
|
|
No
|
|
T}@T{
|
|
No
|
|
T}@T{
|
|
Yes
|
|
T}@T{
|
|
Yes
|
|
T}@T{
|
|
No #2178 (https://github.com/rclone/rclone/issues/2178)
|
|
T}@T{
|
|
Yes
|
|
T}@T{
|
|
No
|
|
T}
|
|
T{
|
|
Jottacloud
|
|
T}@T{
|
|
Yes
|
|
T}@T{
|
|
Yes
|
|
T}@T{
|
|
Yes
|
|
T}@T{
|
|
Yes
|
|
T}@T{
|
|
Yes
|
|
T}@T{
|
|
Yes
|
|
T}@T{
|
|
No
|
|
T}@T{
|
|
Yes
|
|
T}@T{
|
|
Yes
|
|
T}@T{
|
|
Yes
|
|
T}
|
|
T{
|
|
Mail.ru Cloud
|
|
T}@T{
|
|
Yes
|
|
T}@T{
|
|
Yes
|
|
T}@T{
|
|
Yes
|
|
T}@T{
|
|
Yes
|
|
T}@T{
|
|
Yes
|
|
T}@T{
|
|
No
|
|
T}@T{
|
|
No
|
|
T}@T{
|
|
Yes
|
|
T}@T{
|
|
Yes
|
|
T}@T{
|
|
Yes
|
|
T}
|
|
T{
|
|
Mega
|
|
T}@T{
|
|
Yes
|
|
T}@T{
|
|
No
|
|
T}@T{
|
|
Yes
|
|
T}@T{
|
|
Yes
|
|
T}@T{
|
|
Yes
|
|
T}@T{
|
|
No
|
|
T}@T{
|
|
No
|
|
T}@T{
|
|
No #2178 (https://github.com/rclone/rclone/issues/2178)
|
|
T}@T{
|
|
Yes
|
|
T}@T{
|
|
Yes
|
|
T}
|
|
T{
|
|
Memory
|
|
T}@T{
|
|
No
|
|
T}@T{
|
|
Yes
|
|
T}@T{
|
|
No
|
|
T}@T{
|
|
No
|
|
T}@T{
|
|
No
|
|
T}@T{
|
|
Yes
|
|
T}@T{
|
|
Yes
|
|
T}@T{
|
|
No
|
|
T}@T{
|
|
No
|
|
T}@T{
|
|
No
|
|
T}
|
|
T{
|
|
Microsoft Azure Blob Storage
|
|
T}@T{
|
|
Yes
|
|
T}@T{
|
|
Yes
|
|
T}@T{
|
|
No
|
|
T}@T{
|
|
No
|
|
T}@T{
|
|
No
|
|
T}@T{
|
|
Yes
|
|
T}@T{
|
|
Yes
|
|
T}@T{
|
|
No #2178 (https://github.com/rclone/rclone/issues/2178)
|
|
T}@T{
|
|
No
|
|
T}@T{
|
|
No
|
|
T}
|
|
T{
|
|
Microsoft OneDrive
|
|
T}@T{
|
|
Yes
|
|
T}@T{
|
|
Yes
|
|
T}@T{
|
|
Yes
|
|
T}@T{
|
|
Yes
|
|
T}@T{
|
|
No #575 (https://github.com/rclone/rclone/issues/575)
|
|
T}@T{
|
|
No
|
|
T}@T{
|
|
No
|
|
T}@T{
|
|
Yes
|
|
T}@T{
|
|
Yes
|
|
T}@T{
|
|
Yes
|
|
T}
|
|
T{
|
|
OpenDrive
|
|
T}@T{
|
|
Yes
|
|
T}@T{
|
|
Yes
|
|
T}@T{
|
|
Yes
|
|
T}@T{
|
|
Yes
|
|
T}@T{
|
|
No
|
|
T}@T{
|
|
No
|
|
T}@T{
|
|
No
|
|
T}@T{
|
|
No
|
|
T}@T{
|
|
No
|
|
T}@T{
|
|
Yes
|
|
T}
|
|
T{
|
|
OpenStack Swift
|
|
T}@T{
|
|
Yes \[dg]
|
|
T}@T{
|
|
Yes
|
|
T}@T{
|
|
No
|
|
T}@T{
|
|
No
|
|
T}@T{
|
|
No
|
|
T}@T{
|
|
Yes
|
|
T}@T{
|
|
Yes
|
|
T}@T{
|
|
No #2178 (https://github.com/rclone/rclone/issues/2178)
|
|
T}@T{
|
|
Yes
|
|
T}@T{
|
|
No
|
|
T}
|
|
T{
|
|
pCloud
|
|
T}@T{
|
|
Yes
|
|
T}@T{
|
|
Yes
|
|
T}@T{
|
|
Yes
|
|
T}@T{
|
|
Yes
|
|
T}@T{
|
|
Yes
|
|
T}@T{
|
|
No
|
|
T}@T{
|
|
No
|
|
T}@T{
|
|
No #2178 (https://github.com/rclone/rclone/issues/2178)
|
|
T}@T{
|
|
Yes
|
|
T}@T{
|
|
Yes
|
|
T}
|
|
T{
|
|
premiumize.me
|
|
T}@T{
|
|
Yes
|
|
T}@T{
|
|
No
|
|
T}@T{
|
|
Yes
|
|
T}@T{
|
|
Yes
|
|
T}@T{
|
|
No
|
|
T}@T{
|
|
No
|
|
T}@T{
|
|
No
|
|
T}@T{
|
|
Yes
|
|
T}@T{
|
|
Yes
|
|
T}@T{
|
|
Yes
|
|
T}
|
|
T{
|
|
put.io
|
|
T}@T{
|
|
Yes
|
|
T}@T{
|
|
No
|
|
T}@T{
|
|
Yes
|
|
T}@T{
|
|
Yes
|
|
T}@T{
|
|
Yes
|
|
T}@T{
|
|
No
|
|
T}@T{
|
|
Yes
|
|
T}@T{
|
|
No #2178 (https://github.com/rclone/rclone/issues/2178)
|
|
T}@T{
|
|
Yes
|
|
T}@T{
|
|
Yes
|
|
T}
|
|
T{
|
|
QingStor
|
|
T}@T{
|
|
No
|
|
T}@T{
|
|
Yes
|
|
T}@T{
|
|
No
|
|
T}@T{
|
|
No
|
|
T}@T{
|
|
Yes
|
|
T}@T{
|
|
Yes
|
|
T}@T{
|
|
No
|
|
T}@T{
|
|
No #2178 (https://github.com/rclone/rclone/issues/2178)
|
|
T}@T{
|
|
No
|
|
T}@T{
|
|
No
|
|
T}
|
|
T{
|
|
Seafile
|
|
T}@T{
|
|
Yes
|
|
T}@T{
|
|
Yes
|
|
T}@T{
|
|
Yes
|
|
T}@T{
|
|
Yes
|
|
T}@T{
|
|
Yes
|
|
T}@T{
|
|
Yes
|
|
T}@T{
|
|
Yes
|
|
T}@T{
|
|
Yes
|
|
T}@T{
|
|
Yes
|
|
T}@T{
|
|
Yes
|
|
T}
|
|
T{
|
|
SFTP
|
|
T}@T{
|
|
No
|
|
T}@T{
|
|
No
|
|
T}@T{
|
|
Yes
|
|
T}@T{
|
|
Yes
|
|
T}@T{
|
|
No
|
|
T}@T{
|
|
No
|
|
T}@T{
|
|
Yes
|
|
T}@T{
|
|
No #2178 (https://github.com/rclone/rclone/issues/2178)
|
|
T}@T{
|
|
Yes
|
|
T}@T{
|
|
Yes
|
|
T}
|
|
T{
|
|
SugarSync
|
|
T}@T{
|
|
Yes
|
|
T}@T{
|
|
Yes
|
|
T}@T{
|
|
Yes
|
|
T}@T{
|
|
Yes
|
|
T}@T{
|
|
No
|
|
T}@T{
|
|
No
|
|
T}@T{
|
|
Yes
|
|
T}@T{
|
|
Yes
|
|
T}@T{
|
|
No
|
|
T}@T{
|
|
Yes
|
|
T}
|
|
T{
|
|
Tardigrade
|
|
T}@T{
|
|
Yes \[dg]
|
|
T}@T{
|
|
No
|
|
T}@T{
|
|
No
|
|
T}@T{
|
|
No
|
|
T}@T{
|
|
No
|
|
T}@T{
|
|
Yes
|
|
T}@T{
|
|
Yes
|
|
T}@T{
|
|
No
|
|
T}@T{
|
|
No
|
|
T}@T{
|
|
No
|
|
T}
|
|
T{
|
|
WebDAV
|
|
T}@T{
|
|
Yes
|
|
T}@T{
|
|
Yes
|
|
T}@T{
|
|
Yes
|
|
T}@T{
|
|
Yes
|
|
T}@T{
|
|
No
|
|
T}@T{
|
|
No
|
|
T}@T{
|
|
Yes \[dd]
|
|
T}@T{
|
|
No #2178 (https://github.com/rclone/rclone/issues/2178)
|
|
T}@T{
|
|
Yes
|
|
T}@T{
|
|
Yes
|
|
T}
|
|
T{
|
|
Yandex Disk
|
|
T}@T{
|
|
Yes
|
|
T}@T{
|
|
Yes
|
|
T}@T{
|
|
Yes
|
|
T}@T{
|
|
Yes
|
|
T}@T{
|
|
Yes
|
|
T}@T{
|
|
No
|
|
T}@T{
|
|
Yes
|
|
T}@T{
|
|
Yes
|
|
T}@T{
|
|
Yes
|
|
T}@T{
|
|
Yes
|
|
T}
|
|
T{
|
|
The local filesystem
|
|
T}@T{
|
|
Yes
|
|
T}@T{
|
|
No
|
|
T}@T{
|
|
Yes
|
|
T}@T{
|
|
Yes
|
|
T}@T{
|
|
No
|
|
T}@T{
|
|
No
|
|
T}@T{
|
|
Yes
|
|
T}@T{
|
|
No
|
|
T}@T{
|
|
Yes
|
|
T}@T{
|
|
Yes
|
|
T}
|
|
.TE
|
|
.SS Purge
|
|
.PP
|
|
This deletes a directory quicker than just deleting all the files in the
|
|
directory.
|
|
.PP
|
|
\[dg] Note Swift, Hubic, and Tardigrade implement this in order to
|
|
delete directory markers but they don\[cq]t actually have a quicker way
|
|
of deleting files other than deleting them individually.
|
|
.PP
|
|
\[dd] StreamUpload is not supported with Nextcloud
|
|
.SS Copy
|
|
.PP
|
|
Used when copying an object to and from the same remote.
|
|
This known as a server side copy so you can copy a file without
|
|
downloading it and uploading it again.
|
|
It is used if you use \f[C]rclone copy\f[R] or \f[C]rclone move\f[R] if
|
|
the remote doesn\[cq]t support \f[C]Move\f[R] directly.
|
|
.PP
|
|
If the server doesn\[cq]t support \f[C]Copy\f[R] directly then for copy
|
|
operations the file is downloaded then re\-uploaded.
|
|
.SS Move
|
|
.PP
|
|
Used when moving/renaming an object on the same remote.
|
|
This is known as a server side move of a file.
|
|
This is used in \f[C]rclone move\f[R] if the server doesn\[cq]t support
|
|
\f[C]DirMove\f[R].
|
|
.PP
|
|
If the server isn\[cq]t capable of \f[C]Move\f[R] then rclone simulates
|
|
it with \f[C]Copy\f[R] then delete.
|
|
If the server doesn\[cq]t support \f[C]Copy\f[R] then rclone will
|
|
download the file and re\-upload it.
|
|
.SS DirMove
|
|
.PP
|
|
This is used to implement \f[C]rclone move\f[R] to move a directory if
|
|
possible.
|
|
If it isn\[cq]t then it will use \f[C]Move\f[R] on each file (which
|
|
falls back to \f[C]Copy\f[R] then download and upload \- see
|
|
\f[C]Move\f[R] section).
|
|
.SS CleanUp
|
|
.PP
|
|
This is used for emptying the trash for a remote by
|
|
\f[C]rclone cleanup\f[R].
|
|
.PP
|
|
If the server can\[cq]t do \f[C]CleanUp\f[R] then
|
|
\f[C]rclone cleanup\f[R] will return an error.
|
|
.SS ListR
|
|
.PP
|
|
The remote supports a recursive list to list all the contents beneath a
|
|
directory quickly.
|
|
This enables the \f[C]\-\-fast\-list\f[R] flag to work.
|
|
See the rclone docs (https://rclone.org/docs/#fast-list) for more
|
|
details.
|
|
.SS StreamUpload
|
|
.PP
|
|
Some remotes allow files to be uploaded without knowing the file size in
|
|
advance.
|
|
This allows certain operations to work without spooling the file to
|
|
local disk first, e.g.\ \f[C]rclone rcat\f[R].
|
|
.SS LinkSharing
|
|
.PP
|
|
Sets the necessary permissions on a file or folder and prints a link
|
|
that allows others to access them, even if they don\[cq]t have an
|
|
account on the particular cloud provider.
|
|
.SS About
|
|
.PP
|
|
This is used to fetch quota information from the remote, like bytes
|
|
used/free/quota and bytes used in the trash.
|
|
.PP
|
|
This is also used to return the space used, available for
|
|
\f[C]rclone mount\f[R].
|
|
.PP
|
|
If the server can\[cq]t do \f[C]About\f[R] then \f[C]rclone about\f[R]
|
|
will return an error.
|
|
.SS EmptyDir
|
|
.PP
|
|
The remote supports empty directories.
|
|
See Limitations (https://rclone.org/bugs/#limitations) for details.
|
|
Most Object/Bucket based remotes do not support this.
|
|
.SH Global Flags
|
|
.PP
|
|
This describes the global flags available to every rclone command split
|
|
into two groups, non backend and backend flags.
|
|
.SS Non Backend Flags
|
|
.PP
|
|
These flags are available for every command.
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
\-\-ask\-password Allow prompt for password for encrypted configuration. (default true)
|
|
\-\-auto\-confirm If enabled, do not request console confirmation.
|
|
\-\-backup\-dir string Make backups into hierarchy based in DIR.
|
|
\-\-bind string Local address to bind to for outgoing connections, IPv4, IPv6 or name.
|
|
\-\-buffer\-size SizeSuffix In memory buffer size when reading files for each \-\-transfer. (default 16M)
|
|
\-\-bwlimit BwTimetable Bandwidth limit in kBytes/s, or use suffix b|k|M|G or a full timetable.
|
|
\-\-ca\-cert string CA certificate used to verify servers
|
|
\-\-cache\-dir string Directory rclone will use for caching. (default \[dq]$HOME/.cache/rclone\[dq])
|
|
\-\-check\-first Do all the checks before starting transfers.
|
|
\-\-checkers int Number of checkers to run in parallel. (default 8)
|
|
\-c, \-\-checksum Skip based on checksum (if available) & size, not mod\-time & size
|
|
\-\-client\-cert string Client SSL certificate (PEM) for mutual TLS auth
|
|
\-\-client\-key string Client SSL private key (PEM) for mutual TLS auth
|
|
\-\-compare\-dest string Include additional server\-side path during comparison.
|
|
\-\-config string Config file. (default \[dq]$HOME/.config/rclone/rclone.conf\[dq])
|
|
\-\-contimeout duration Connect timeout (default 1m0s)
|
|
\-\-copy\-dest string Implies \-\-compare\-dest but also copies files from path into destination.
|
|
\-\-cpuprofile string Write cpu profile to file
|
|
\-\-cutoff\-mode string Mode to stop transfers when reaching the max transfer limit HARD|SOFT|CAUTIOUS (default \[dq]HARD\[dq])
|
|
\-\-delete\-after When synchronizing, delete files on destination after transferring (default)
|
|
\-\-delete\-before When synchronizing, delete files on destination before transferring
|
|
\-\-delete\-during When synchronizing, delete files during transfer
|
|
\-\-delete\-excluded Delete files on dest excluded from sync
|
|
\-\-disable string Disable a comma separated list of features. Use help to see a list.
|
|
\-n, \-\-dry\-run Do a trial run with no permanent changes
|
|
\-\-dump DumpFlags List of items to dump from: headers,bodies,requests,responses,auth,filters,goroutines,openfiles
|
|
\-\-dump\-bodies Dump HTTP headers and bodies \- may contain sensitive info
|
|
\-\-dump\-headers Dump HTTP headers \- may contain sensitive info
|
|
\-\-error\-on\-no\-transfer Sets exit code 9 if no files are transferred, useful in scripts
|
|
\-\-exclude stringArray Exclude files matching pattern
|
|
\-\-exclude\-from stringArray Read exclude patterns from file (use \- to read from stdin)
|
|
\-\-exclude\-if\-present string Exclude directories if filename is present
|
|
\-\-expect\-continue\-timeout duration Timeout when using expect / 100\-continue in HTTP (default 1s)
|
|
\-\-fast\-list Use recursive list if available. Uses more memory but fewer transactions.
|
|
\-\-files\-from stringArray Read list of source\-file names from file (use \- to read from stdin)
|
|
\-\-files\-from\-raw stringArray Read list of source\-file names from file without any processing of lines (use \- to read from stdin)
|
|
\-f, \-\-filter stringArray Add a file\-filtering rule
|
|
\-\-filter\-from stringArray Read filtering patterns from a file (use \- to read from stdin)
|
|
\-\-header stringArray Set HTTP header for all transactions
|
|
\-\-header\-download stringArray Set HTTP header for download transactions
|
|
\-\-header\-upload stringArray Set HTTP header for upload transactions
|
|
\-\-ignore\-case Ignore case in filters (case insensitive)
|
|
\-\-ignore\-case\-sync Ignore case when synchronizing
|
|
\-\-ignore\-checksum Skip post copy check of checksums.
|
|
\-\-ignore\-errors delete even if there are I/O errors
|
|
\-\-ignore\-existing Skip all files that exist on destination
|
|
\-\-ignore\-size Ignore size when skipping use mod\-time or checksum.
|
|
\-I, \-\-ignore\-times Don\[aq]t skip files that match size and time \- transfer all files
|
|
\-\-immutable Do not modify files. Fail if existing files have been modified.
|
|
\-\-include stringArray Include files matching pattern
|
|
\-\-include\-from stringArray Read include patterns from file (use \- to read from stdin)
|
|
\-\-log\-file string Log everything to this file
|
|
\-\-log\-format string Comma separated list of log format options (default \[dq]date,time\[dq])
|
|
\-\-log\-level string Log level DEBUG|INFO|NOTICE|ERROR (default \[dq]NOTICE\[dq])
|
|
\-\-low\-level\-retries int Number of low level retries to do. (default 10)
|
|
\-\-max\-age Duration Only transfer files younger than this in s or suffix ms|s|m|h|d|w|M|y (default off)
|
|
\-\-max\-backlog int Maximum number of objects in sync or check backlog. (default 10000)
|
|
\-\-max\-delete int When synchronizing, limit the number of deletes (default \-1)
|
|
\-\-max\-depth int If set limits the recursion depth to this. (default \-1)
|
|
\-\-max\-duration duration Maximum duration rclone will transfer data for.
|
|
\-\-max\-size SizeSuffix Only transfer files smaller than this in k or suffix b|k|M|G (default off)
|
|
\-\-max\-stats\-groups int Maximum number of stats groups to keep in memory. On max oldest is discarded. (default 1000)
|
|
\-\-max\-transfer SizeSuffix Maximum size of data to transfer. (default off)
|
|
\-\-memprofile string Write memory profile to file
|
|
\-\-min\-age Duration Only transfer files older than this in s or suffix ms|s|m|h|d|w|M|y (default off)
|
|
\-\-min\-size SizeSuffix Only transfer files bigger than this in k or suffix b|k|M|G (default off)
|
|
\-\-modify\-window duration Max time diff to be considered the same (default 1ns)
|
|
\-\-multi\-thread\-cutoff SizeSuffix Use multi\-thread downloads for files above this size. (default 250M)
|
|
\-\-multi\-thread\-streams int Max number of streams to use for multi\-thread downloads. (default 4)
|
|
\-\-no\-check\-certificate Do not verify the server SSL certificate. Insecure.
|
|
\-\-no\-check\-dest Don\[aq]t check the destination, copy regardless.
|
|
\-\-no\-gzip\-encoding Don\[aq]t set Accept\-Encoding: gzip.
|
|
\-\-no\-traverse Don\[aq]t traverse destination file system on copy.
|
|
\-\-no\-unicode\-normalization Don\[aq]t normalize unicode characters in filenames.
|
|
\-\-no\-update\-modtime Don\[aq]t update destination mod\-time if files identical.
|
|
\-\-order\-by string Instructions on how to order the transfers, eg \[aq]size,descending\[aq]
|
|
\-\-password\-command SpaceSepList Command for supplying password for encrypted configuration.
|
|
\-P, \-\-progress Show progress during transfer.
|
|
\-q, \-\-quiet Print as little stuff as possible
|
|
\-\-rc Enable the remote control server.
|
|
\-\-rc\-addr string IPaddress:Port or :Port to bind server to. (default \[dq]localhost:5572\[dq])
|
|
\-\-rc\-allow\-origin string Set the allowed origin for CORS.
|
|
\-\-rc\-baseurl string Prefix for URLs \- leave blank for root.
|
|
\-\-rc\-cert string SSL PEM key (concatenation of certificate and CA certificate)
|
|
\-\-rc\-client\-ca string Client certificate authority to verify clients with
|
|
\-\-rc\-enable\-metrics Enable prometheus metrics on /metrics
|
|
\-\-rc\-files string Path to local files to serve on the HTTP server.
|
|
\-\-rc\-htpasswd string htpasswd file \- if not provided no authentication is done
|
|
\-\-rc\-job\-expire\-duration duration expire finished async jobs older than this value (default 1m0s)
|
|
\-\-rc\-job\-expire\-interval duration interval to check for expired async jobs (default 10s)
|
|
\-\-rc\-key string SSL PEM Private key
|
|
\-\-rc\-max\-header\-bytes int Maximum size of request header (default 4096)
|
|
\-\-rc\-no\-auth Don\[aq]t require auth for certain methods.
|
|
\-\-rc\-pass string Password for authentication.
|
|
\-\-rc\-realm string realm for authentication (default \[dq]rclone\[dq])
|
|
\-\-rc\-serve Enable the serving of remote objects.
|
|
\-\-rc\-server\-read\-timeout duration Timeout for server reading data (default 1h0m0s)
|
|
\-\-rc\-server\-write\-timeout duration Timeout for server writing data (default 1h0m0s)
|
|
\-\-rc\-template string User Specified Template.
|
|
\-\-rc\-user string User name for authentication.
|
|
\-\-rc\-web\-fetch\-url string URL to fetch the releases for webgui. (default \[dq]https://api.github.com/repos/rclone/rclone\-webui\-react/releases/latest\[dq])
|
|
\-\-rc\-web\-gui Launch WebGUI on localhost
|
|
\-\-rc\-web\-gui\-force\-update Force update to latest version of web gui
|
|
\-\-rc\-web\-gui\-no\-open\-browser Don\[aq]t open the browser automatically
|
|
\-\-rc\-web\-gui\-update Check and update to latest version of web gui
|
|
\-\-retries int Retry operations this many times if they fail (default 3)
|
|
\-\-retries\-sleep duration Interval between retrying operations if they fail, e.g 500ms, 60s, 5m. (0 to disable)
|
|
\-\-size\-only Skip based on size only, not mod\-time or checksum
|
|
\-\-stats duration Interval between printing stats, e.g 500ms, 60s, 5m. (0 to disable) (default 1m0s)
|
|
\-\-stats\-file\-name\-length int Max file name length in stats. 0 for no limit (default 45)
|
|
\-\-stats\-log\-level string Log level to show \-\-stats output DEBUG|INFO|NOTICE|ERROR (default \[dq]INFO\[dq])
|
|
\-\-stats\-one\-line Make the stats fit on one line.
|
|
\-\-stats\-one\-line\-date Enables \-\-stats\-one\-line and add current date/time prefix.
|
|
\-\-stats\-one\-line\-date\-format string Enables \-\-stats\-one\-line\-date and uses custom formatted date. Enclose date string in double quotes (\[dq]). See https://golang.org/pkg/time/#Time.Format
|
|
\-\-stats\-unit string Show data rate in stats as either \[aq]bits\[aq] or \[aq]bytes\[aq]/s (default \[dq]bytes\[dq])
|
|
\-\-streaming\-upload\-cutoff SizeSuffix Cutoff for switching to chunked upload if file size is unknown. Upload starts after reaching cutoff or when file ends. (default 100k)
|
|
\-\-suffix string Suffix to add to changed files.
|
|
\-\-suffix\-keep\-extension Preserve the extension when using \-\-suffix.
|
|
\-\-syslog Use Syslog for logging
|
|
\-\-syslog\-facility string Facility for syslog, eg KERN,USER,... (default \[dq]DAEMON\[dq])
|
|
\-\-timeout duration IO idle timeout (default 5m0s)
|
|
\-\-tpslimit float Limit HTTP transactions per second to this.
|
|
\-\-tpslimit\-burst int Max burst of transactions for \-\-tpslimit. (default 1)
|
|
\-\-track\-renames When synchronizing, track file renames and do a server side move if possible
|
|
\-\-track\-renames\-strategy string Strategies to use when synchronizing using track\-renames hash|modtime (default \[dq]hash\[dq])
|
|
\-\-transfers int Number of file transfers to run in parallel. (default 4)
|
|
\-u, \-\-update Skip files that are newer on the destination.
|
|
\-\-use\-cookies Enable session cookiejar.
|
|
\-\-use\-json\-log Use json log format.
|
|
\-\-use\-mmap Use mmap allocator (see docs).
|
|
\-\-use\-server\-modtime Use server modified time instead of object metadata
|
|
\-\-user\-agent string Set the user\-agent to a specified string. The default is rclone/ version (default \[dq]rclone/v1.52.0\[dq])
|
|
\-v, \-\-verbose count Print lots more stuff (repeat for more)
|
|
\f[R]
|
|
.fi
|
|
.SS Backend Flags
|
|
.PP
|
|
These flags are available for every command.
|
|
They control the backends and may be set in the config file.
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
\-\-acd\-auth\-url string Auth server URL.
|
|
\-\-acd\-client\-id string Amazon Application Client ID.
|
|
\-\-acd\-client\-secret string Amazon Application Client Secret.
|
|
\-\-acd\-encoding MultiEncoder This sets the encoding for the backend. (default Slash,InvalidUtf8,Dot)
|
|
\-\-acd\-templink\-threshold SizeSuffix Files >= this size will be downloaded via their tempLink. (default 9G)
|
|
\-\-acd\-token\-url string Token server url.
|
|
\-\-acd\-upload\-wait\-per\-gb Duration Additional time per GB to wait after a failed complete upload to see if it appears. (default 3m0s)
|
|
\-\-alias\-remote string Remote or path to alias.
|
|
\-\-azureblob\-access\-tier string Access tier of blob: hot, cool or archive.
|
|
\-\-azureblob\-account string Storage Account Name (leave blank to use SAS URL or Emulator)
|
|
\-\-azureblob\-chunk\-size SizeSuffix Upload chunk size (<= 100MB). (default 4M)
|
|
\-\-azureblob\-disable\-checksum Don\[aq]t store MD5 checksum with object metadata.
|
|
\-\-azureblob\-encoding MultiEncoder This sets the encoding for the backend. (default Slash,BackSlash,Del,Ctl,RightPeriod,InvalidUtf8)
|
|
\-\-azureblob\-endpoint string Endpoint for the service
|
|
\-\-azureblob\-key string Storage Account Key (leave blank to use SAS URL or Emulator)
|
|
\-\-azureblob\-list\-chunk int Size of blob list. (default 5000)
|
|
\-\-azureblob\-memory\-pool\-flush\-time Duration How often internal memory buffer pools will be flushed. (default 1m0s)
|
|
\-\-azureblob\-memory\-pool\-use\-mmap Whether to use mmap buffers in internal memory pool.
|
|
\-\-azureblob\-sas\-url string SAS URL for container level access only
|
|
\-\-azureblob\-upload\-cutoff SizeSuffix Cutoff for switching to chunked upload (<= 256MB). (default 256M)
|
|
\-\-azureblob\-use\-emulator Uses local storage emulator if provided as \[aq]true\[aq] (leave blank if using real azure storage endpoint)
|
|
\-\-b2\-account string Account ID or Application Key ID
|
|
\-\-b2\-chunk\-size SizeSuffix Upload chunk size. Must fit in memory. (default 96M)
|
|
\-\-b2\-disable\-checksum Disable checksums for large (> upload cutoff) files
|
|
\-\-b2\-download\-auth\-duration Duration Time before the authorization token will expire in s or suffix ms|s|m|h|d. (default 1w)
|
|
\-\-b2\-download\-url string Custom endpoint for downloads.
|
|
\-\-b2\-encoding MultiEncoder This sets the encoding for the backend. (default Slash,BackSlash,Del,Ctl,InvalidUtf8,Dot)
|
|
\-\-b2\-endpoint string Endpoint for the service.
|
|
\-\-b2\-hard\-delete Permanently delete files on remote removal, otherwise hide files.
|
|
\-\-b2\-key string Application Key
|
|
\-\-b2\-test\-mode string A flag string for X\-Bz\-Test\-Mode header for debugging.
|
|
\-\-b2\-upload\-cutoff SizeSuffix Cutoff for switching to chunked upload. (default 200M)
|
|
\-\-b2\-versions Include old versions in directory listings.
|
|
\-\-box\-box\-config\-file string Box App config.json location
|
|
\-\-box\-box\-sub\-type string (default \[dq]user\[dq])
|
|
\-\-box\-client\-id string Box App Client Id.
|
|
\-\-box\-client\-secret string Box App Client Secret
|
|
\-\-box\-commit\-retries int Max number of times to try committing a multipart file. (default 100)
|
|
\-\-box\-encoding MultiEncoder This sets the encoding for the backend. (default Slash,BackSlash,Del,Ctl,RightSpace,InvalidUtf8,Dot)
|
|
\-\-box\-root\-folder\-id string Fill in for rclone to use a non root folder as its starting point.
|
|
\-\-box\-upload\-cutoff SizeSuffix Cutoff for switching to multipart upload (>= 50MB). (default 50M)
|
|
\-\-cache\-chunk\-clean\-interval Duration How often should the cache perform cleanups of the chunk storage. (default 1m0s)
|
|
\-\-cache\-chunk\-no\-memory Disable the in\-memory cache for storing chunks during streaming.
|
|
\-\-cache\-chunk\-path string Directory to cache chunk files. (default \[dq]$HOME/.cache/rclone/cache\-backend\[dq])
|
|
\-\-cache\-chunk\-size SizeSuffix The size of a chunk (partial file data). (default 5M)
|
|
\-\-cache\-chunk\-total\-size SizeSuffix The total size that the chunks can take up on the local disk. (default 10G)
|
|
\-\-cache\-db\-path string Directory to store file structure metadata DB. (default \[dq]$HOME/.cache/rclone/cache\-backend\[dq])
|
|
\-\-cache\-db\-purge Clear all the cached data for this remote on start.
|
|
\-\-cache\-db\-wait\-time Duration How long to wait for the DB to be available \- 0 is unlimited (default 1s)
|
|
\-\-cache\-info\-age Duration How long to cache file structure information (directory listings, file size, times etc). (default 6h0m0s)
|
|
\-\-cache\-plex\-insecure string Skip all certificate verification when connecting to the Plex server
|
|
\-\-cache\-plex\-password string The password of the Plex user
|
|
\-\-cache\-plex\-url string The URL of the Plex server
|
|
\-\-cache\-plex\-username string The username of the Plex user
|
|
\-\-cache\-read\-retries int How many times to retry a read from a cache storage. (default 10)
|
|
\-\-cache\-remote string Remote to cache.
|
|
\-\-cache\-rps int Limits the number of requests per second to the source FS (\-1 to disable) (default \-1)
|
|
\-\-cache\-tmp\-upload\-path string Directory to keep temporary files until they are uploaded.
|
|
\-\-cache\-tmp\-wait\-time Duration How long should files be stored in local cache before being uploaded (default 15s)
|
|
\-\-cache\-workers int How many workers should run in parallel to download chunks. (default 4)
|
|
\-\-cache\-writes Cache file data on writes through the FS
|
|
\-\-chunker\-chunk\-size SizeSuffix Files larger than chunk size will be split in chunks. (default 2G)
|
|
\-\-chunker\-fail\-hard Choose how chunker should handle files with missing or invalid chunks.
|
|
\-\-chunker\-hash\-type string Choose how chunker handles hash sums. All modes but \[dq]none\[dq] require metadata. (default \[dq]md5\[dq])
|
|
\-\-chunker\-meta\-format string Format of the metadata object or \[dq]none\[dq]. By default \[dq]simplejson\[dq]. (default \[dq]simplejson\[dq])
|
|
\-\-chunker\-name\-format string String format of chunk file names. (default \[dq]*.rclone_chunk.###\[dq])
|
|
\-\-chunker\-remote string Remote to chunk/unchunk.
|
|
\-\-chunker\-start\-from int Minimum valid chunk number. Usually 0 or 1. (default 1)
|
|
\-L, \-\-copy\-links Follow symlinks and copy the pointed to item.
|
|
\-\-crypt\-directory\-name\-encryption Option to either encrypt directory names or leave them intact. (default true)
|
|
\-\-crypt\-filename\-encryption string How to encrypt the filenames. (default \[dq]standard\[dq])
|
|
\-\-crypt\-password string Password or pass phrase for encryption.
|
|
\-\-crypt\-password2 string Password or pass phrase for salt. Optional but recommended.
|
|
\-\-crypt\-remote string Remote to encrypt/decrypt.
|
|
\-\-crypt\-show\-mapping For all files listed show how the names encrypt.
|
|
\-\-drive\-acknowledge\-abuse Set to allow files which return cannotDownloadAbusiveFile to be downloaded.
|
|
\-\-drive\-allow\-import\-name\-change Allow the filetype to change when uploading Google docs (e.g. file.doc to file.docx). This will confuse sync and reupload every time.
|
|
\-\-drive\-alternate\-export Use alternate export URLs for google documents export.,
|
|
\-\-drive\-auth\-owner\-only Only consider files owned by the authenticated user.
|
|
\-\-drive\-chunk\-size SizeSuffix Upload chunk size. Must a power of 2 >= 256k. (default 8M)
|
|
\-\-drive\-client\-id string Google Application Client Id
|
|
\-\-drive\-client\-secret string Google Application Client Secret
|
|
\-\-drive\-disable\-http2 Disable drive using http2 (default true)
|
|
\-\-drive\-encoding MultiEncoder This sets the encoding for the backend. (default InvalidUtf8)
|
|
\-\-drive\-export\-formats string Comma separated list of preferred formats for downloading Google docs. (default \[dq]docx,xlsx,pptx,svg\[dq])
|
|
\-\-drive\-formats string Deprecated: see export_formats
|
|
\-\-drive\-impersonate string Impersonate this user when using a service account.
|
|
\-\-drive\-import\-formats string Comma separated list of preferred formats for uploading Google docs.
|
|
\-\-drive\-keep\-revision\-forever Keep new head revision of each file forever.
|
|
\-\-drive\-list\-chunk int Size of listing chunk 100\-1000. 0 to disable. (default 1000)
|
|
\-\-drive\-pacer\-burst int Number of API calls to allow without sleeping. (default 100)
|
|
\-\-drive\-pacer\-min\-sleep Duration Minimum time to sleep between API calls. (default 100ms)
|
|
\-\-drive\-root\-folder\-id string ID of the root folder
|
|
\-\-drive\-scope string Scope that rclone should use when requesting access from drive.
|
|
\-\-drive\-server\-side\-across\-configs Allow server side operations (eg copy) to work across different drive configs.
|
|
\-\-drive\-service\-account\-credentials string Service Account Credentials JSON blob
|
|
\-\-drive\-service\-account\-file string Service Account Credentials JSON file path
|
|
\-\-drive\-shared\-with\-me Only show files that are shared with me.
|
|
\-\-drive\-size\-as\-quota Show sizes as storage quota usage, not actual size.
|
|
\-\-drive\-skip\-checksum\-gphotos Skip MD5 checksum on Google photos and videos only.
|
|
\-\-drive\-skip\-gdocs Skip google documents in all listings.
|
|
\-\-drive\-skip\-shortcuts If set skip shortcut files
|
|
\-\-drive\-stop\-on\-upload\-limit Make upload limit errors be fatal
|
|
\-\-drive\-team\-drive string ID of the Team Drive
|
|
\-\-drive\-trashed\-only Only show files that are in the trash.
|
|
\-\-drive\-upload\-cutoff SizeSuffix Cutoff for switching to chunked upload (default 8M)
|
|
\-\-drive\-use\-created\-date Use file created date instead of modified date.,
|
|
\-\-drive\-use\-shared\-date Use date file was shared instead of modified date.
|
|
\-\-drive\-use\-trash Send files to the trash instead of deleting permanently. (default true)
|
|
\-\-drive\-v2\-download\-min\-size SizeSuffix If Object\[aq]s are greater, use drive v2 API to download. (default off)
|
|
\-\-dropbox\-chunk\-size SizeSuffix Upload chunk size. (< 150M). (default 48M)
|
|
\-\-dropbox\-client\-id string Dropbox App Client Id
|
|
\-\-dropbox\-client\-secret string Dropbox App Client Secret
|
|
\-\-dropbox\-encoding MultiEncoder This sets the encoding for the backend. (default Slash,BackSlash,Del,RightSpace,InvalidUtf8,Dot)
|
|
\-\-dropbox\-impersonate string Impersonate this user when using a business account.
|
|
\-\-fichier\-api\-key string Your API Key, get it from https://1fichier.com/console/params.pl
|
|
\-\-fichier\-encoding MultiEncoder This sets the encoding for the backend. (default Slash,LtGt,DoubleQuote,SingleQuote,BackQuote,Dollar,BackSlash,Del,Ctl,LeftSpace,RightSpace,InvalidUtf8,Dot)
|
|
\-\-fichier\-shared\-folder string If you want to download a shared folder, add this parameter
|
|
\-\-ftp\-concurrency int Maximum number of FTP simultaneous connections, 0 for unlimited
|
|
\-\-ftp\-disable\-epsv Disable using EPSV even if server advertises support
|
|
\-\-ftp\-encoding MultiEncoder This sets the encoding for the backend. (default Slash,Del,Ctl,RightSpace,Dot)
|
|
\-\-ftp\-host string FTP host to connect to
|
|
\-\-ftp\-no\-check\-certificate Do not verify the TLS certificate of the server
|
|
\-\-ftp\-pass string FTP password
|
|
\-\-ftp\-port string FTP port, leave blank to use default (21)
|
|
\-\-ftp\-tls Use FTP over TLS (Implicit)
|
|
\-\-ftp\-user string FTP username, leave blank for current username, $USER
|
|
\-\-gcs\-bucket\-acl string Access Control List for new buckets.
|
|
\-\-gcs\-bucket\-policy\-only Access checks should use bucket\-level IAM policies.
|
|
\-\-gcs\-client\-id string Google Application Client Id
|
|
\-\-gcs\-client\-secret string Google Application Client Secret
|
|
\-\-gcs\-encoding MultiEncoder This sets the encoding for the backend. (default Slash,CrLf,InvalidUtf8,Dot)
|
|
\-\-gcs\-location string Location for the newly created buckets.
|
|
\-\-gcs\-object\-acl string Access Control List for new objects.
|
|
\-\-gcs\-project\-number string Project number.
|
|
\-\-gcs\-service\-account\-file string Service Account Credentials JSON file path
|
|
\-\-gcs\-storage\-class string The storage class to use when storing objects in Google Cloud Storage.
|
|
\-\-gphotos\-client\-id string Google Application Client Id
|
|
\-\-gphotos\-client\-secret string Google Application Client Secret
|
|
\-\-gphotos\-read\-only Set to make the Google Photos backend read only.
|
|
\-\-gphotos\-read\-size Set to read the size of media items.
|
|
\-\-gphotos\-start\-year int Year limits the photos to be downloaded to those which are uploaded after the given year (default 2000)
|
|
\-\-http\-headers CommaSepList Set HTTP headers for all transactions
|
|
\-\-http\-no\-head Don\[aq]t use HEAD requests to find file sizes in dir listing
|
|
\-\-http\-no\-slash Set this if the site doesn\[aq]t end directories with /
|
|
\-\-http\-url string URL of http host to connect to
|
|
\-\-hubic\-chunk\-size SizeSuffix Above this size files will be chunked into a _segments container. (default 5G)
|
|
\-\-hubic\-client\-id string Hubic Client Id
|
|
\-\-hubic\-client\-secret string Hubic Client Secret
|
|
\-\-hubic\-encoding MultiEncoder This sets the encoding for the backend. (default Slash,InvalidUtf8)
|
|
\-\-hubic\-no\-chunk Don\[aq]t chunk files during streaming upload.
|
|
\-\-jottacloud\-encoding MultiEncoder This sets the encoding for the backend. (default Slash,LtGt,DoubleQuote,Colon,Question,Asterisk,Pipe,Del,Ctl,InvalidUtf8,Dot)
|
|
\-\-jottacloud\-hard\-delete Delete files permanently rather than putting them into the trash.
|
|
\-\-jottacloud\-md5\-memory\-limit SizeSuffix Files bigger than this will be cached on disk to calculate the MD5 if required. (default 10M)
|
|
\-\-jottacloud\-trashed\-only Only show files that are in the trash.
|
|
\-\-jottacloud\-unlink Remove existing public link to file/folder with link command rather than creating.
|
|
\-\-jottacloud\-upload\-resume\-limit SizeSuffix Files bigger than this can be resumed if the upload fail\[aq]s. (default 10M)
|
|
\-\-koofr\-encoding MultiEncoder This sets the encoding for the backend. (default Slash,BackSlash,Del,Ctl,InvalidUtf8,Dot)
|
|
\-\-koofr\-endpoint string The Koofr API endpoint to use (default \[dq]https://app.koofr.net\[dq])
|
|
\-\-koofr\-mountid string Mount ID of the mount to use. If omitted, the primary mount is used.
|
|
\-\-koofr\-password string Your Koofr password for rclone (generate one at https://app.koofr.net/app/admin/preferences/password)
|
|
\-\-koofr\-setmtime Does the backend support setting modification time. Set this to false if you use a mount ID that points to a Dropbox or Amazon Drive backend. (default true)
|
|
\-\-koofr\-user string Your Koofr user name
|
|
\-l, \-\-links Translate symlinks to/from regular files with a \[aq].rclonelink\[aq] extension
|
|
\-\-local\-case\-insensitive Force the filesystem to report itself as case insensitive
|
|
\-\-local\-case\-sensitive Force the filesystem to report itself as case sensitive.
|
|
\-\-local\-encoding MultiEncoder This sets the encoding for the backend. (default Slash,Dot)
|
|
\-\-local\-no\-check\-updated Don\[aq]t check to see if the files change during upload
|
|
\-\-local\-no\-sparse Disable sparse files for multi\-thread downloads
|
|
\-\-local\-no\-unicode\-normalization Don\[aq]t apply unicode normalization to paths and filenames (Deprecated)
|
|
\-\-local\-nounc string Disable UNC (long path names) conversion on Windows
|
|
\-\-mailru\-check\-hash What should copy do if file checksum is mismatched or invalid (default true)
|
|
\-\-mailru\-encoding MultiEncoder This sets the encoding for the backend. (default Slash,LtGt,DoubleQuote,Colon,Question,Asterisk,Pipe,BackSlash,Del,Ctl,InvalidUtf8,Dot)
|
|
\-\-mailru\-pass string Password
|
|
\-\-mailru\-speedup\-enable Skip full upload if there is another file with same data hash. (default true)
|
|
\-\-mailru\-speedup\-file\-patterns string Comma separated list of file name patterns eligible for speedup (put by hash). (default \[dq]*.mkv,*.avi,*.mp4,*.mp3,*.zip,*.gz,*.rar,*.pdf\[dq])
|
|
\-\-mailru\-speedup\-max\-disk SizeSuffix This option allows you to disable speedup (put by hash) for large files (default 3G)
|
|
\-\-mailru\-speedup\-max\-memory SizeSuffix Files larger than the size given below will always be hashed on disk. (default 32M)
|
|
\-\-mailru\-user string User name (usually email)
|
|
\-\-mega\-debug Output more debug from Mega.
|
|
\-\-mega\-encoding MultiEncoder This sets the encoding for the backend. (default Slash,InvalidUtf8,Dot)
|
|
\-\-mega\-hard\-delete Delete files permanently rather than putting them into the trash.
|
|
\-\-mega\-pass string Password.
|
|
\-\-mega\-user string User name
|
|
\-x, \-\-one\-file\-system Don\[aq]t cross filesystem boundaries (unix/macOS only).
|
|
\-\-onedrive\-chunk\-size SizeSuffix Chunk size to upload files with \- must be multiple of 320k (327,680 bytes). (default 10M)
|
|
\-\-onedrive\-client\-id string Microsoft App Client Id
|
|
\-\-onedrive\-client\-secret string Microsoft App Client Secret
|
|
\-\-onedrive\-drive\-id string The ID of the drive to use
|
|
\-\-onedrive\-drive\-type string The type of the drive ( personal | business | documentLibrary )
|
|
\-\-onedrive\-encoding MultiEncoder This sets the encoding for the backend. (default Slash,LtGt,DoubleQuote,Colon,Question,Asterisk,Pipe,Hash,Percent,BackSlash,Del,Ctl,LeftSpace,LeftTilde,RightSpace,RightPeriod,InvalidUtf8,Dot)
|
|
\-\-onedrive\-expose\-onenote\-files Set to make OneNote files show up in directory listings.
|
|
\-\-onedrive\-server\-side\-across\-configs Allow server side operations (eg copy) to work across different onedrive configs.
|
|
\-\-opendrive\-chunk\-size SizeSuffix Files will be uploaded in chunks this size. (default 10M)
|
|
\-\-opendrive\-encoding MultiEncoder This sets the encoding for the backend. (default Slash,LtGt,DoubleQuote,Colon,Question,Asterisk,Pipe,BackSlash,LeftSpace,LeftCrLfHtVt,RightSpace,RightCrLfHtVt,InvalidUtf8,Dot)
|
|
\-\-opendrive\-password string Password.
|
|
\-\-opendrive\-username string Username
|
|
\-\-pcloud\-client\-id string Pcloud App Client Id
|
|
\-\-pcloud\-client\-secret string Pcloud App Client Secret
|
|
\-\-pcloud\-encoding MultiEncoder This sets the encoding for the backend. (default Slash,BackSlash,Del,Ctl,InvalidUtf8,Dot)
|
|
\-\-pcloud\-root\-folder\-id string Fill in for rclone to use a non root folder as its starting point. (default \[dq]d0\[dq])
|
|
\-\-premiumizeme\-encoding MultiEncoder This sets the encoding for the backend. (default Slash,DoubleQuote,BackSlash,Del,Ctl,InvalidUtf8,Dot)
|
|
\-\-putio\-encoding MultiEncoder This sets the encoding for the backend. (default Slash,BackSlash,Del,Ctl,InvalidUtf8,Dot)
|
|
\-\-qingstor\-access\-key\-id string QingStor Access Key ID
|
|
\-\-qingstor\-chunk\-size SizeSuffix Chunk size to use for uploading. (default 4M)
|
|
\-\-qingstor\-connection\-retries int Number of connection retries. (default 3)
|
|
\-\-qingstor\-encoding MultiEncoder This sets the encoding for the backend. (default Slash,Ctl,InvalidUtf8)
|
|
\-\-qingstor\-endpoint string Enter an endpoint URL to connection QingStor API.
|
|
\-\-qingstor\-env\-auth Get QingStor credentials from runtime. Only applies if access_key_id and secret_access_key is blank.
|
|
\-\-qingstor\-secret\-access\-key string QingStor Secret Access Key (password)
|
|
\-\-qingstor\-upload\-concurrency int Concurrency for multipart uploads. (default 1)
|
|
\-\-qingstor\-upload\-cutoff SizeSuffix Cutoff for switching to chunked upload (default 200M)
|
|
\-\-qingstor\-zone string Zone to connect to.
|
|
\-\-s3\-access\-key\-id string AWS Access Key ID.
|
|
\-\-s3\-acl string Canned ACL used when creating buckets and storing or copying objects.
|
|
\-\-s3\-bucket\-acl string Canned ACL used when creating buckets.
|
|
\-\-s3\-chunk\-size SizeSuffix Chunk size to use for uploading. (default 5M)
|
|
\-\-s3\-copy\-cutoff SizeSuffix Cutoff for switching to multipart copy (default 5G)
|
|
\-\-s3\-disable\-checksum Don\[aq]t store MD5 checksum with object metadata
|
|
\-\-s3\-encoding MultiEncoder This sets the encoding for the backend. (default Slash,InvalidUtf8,Dot)
|
|
\-\-s3\-endpoint string Endpoint for S3 API.
|
|
\-\-s3\-env\-auth Get AWS credentials from runtime (environment variables or EC2/ECS meta data if no env vars).
|
|
\-\-s3\-force\-path\-style If true use path style access if false use virtual hosted style. (default true)
|
|
\-\-s3\-leave\-parts\-on\-error If true avoid calling abort upload on a failure, leaving all successfully uploaded parts on S3 for manual recovery.
|
|
\-\-s3\-list\-chunk int Size of listing chunk (response list for each ListObject S3 request). (default 1000)
|
|
\-\-s3\-location\-constraint string Location constraint \- must be set to match the Region.
|
|
\-\-s3\-memory\-pool\-flush\-time Duration How often internal memory buffer pools will be flushed. (default 1m0s)
|
|
\-\-s3\-memory\-pool\-use\-mmap Whether to use mmap buffers in internal memory pool.
|
|
\-\-s3\-provider string Choose your S3 provider.
|
|
\-\-s3\-region string Region to connect to.
|
|
\-\-s3\-secret\-access\-key string AWS Secret Access Key (password)
|
|
\-\-s3\-server\-side\-encryption string The server\-side encryption algorithm used when storing this object in S3.
|
|
\-\-s3\-session\-token string An AWS session token
|
|
\-\-s3\-sse\-customer\-algorithm string If using SSE\-C, the server\-side encryption algorithm used when storing this object in S3.
|
|
\-\-s3\-sse\-customer\-key string If using SSE\-C you must provide the secret encryption key used to encrypt/decrypt your data.
|
|
\-\-s3\-sse\-customer\-key\-md5 string If using SSE\-C you must provide the secret encryption key MD5 checksum.
|
|
\-\-s3\-sse\-kms\-key\-id string If using KMS ID you must provide the ARN of Key.
|
|
\-\-s3\-storage\-class string The storage class to use when storing new objects in S3.
|
|
\-\-s3\-upload\-concurrency int Concurrency for multipart uploads. (default 4)
|
|
\-\-s3\-upload\-cutoff SizeSuffix Cutoff for switching to chunked upload (default 200M)
|
|
\-\-s3\-use\-accelerate\-endpoint If true use the AWS S3 accelerated endpoint.
|
|
\-\-s3\-v2\-auth If true use v2 authentication.
|
|
\-\-seafile\-2fa Two\-factor authentication (\[aq]true\[aq] if the account has 2FA enabled)
|
|
\-\-seafile\-create\-library Should rclone create a library if it doesn\[aq]t exist
|
|
\-\-seafile\-encoding MultiEncoder This sets the encoding for the backend. (default Slash,DoubleQuote,BackSlash,Ctl,InvalidUtf8)
|
|
\-\-seafile\-library string Name of the library. Leave blank to access all non\-encrypted libraries.
|
|
\-\-seafile\-library\-key string Library password (for encrypted libraries only). Leave blank if you pass it through the command line.
|
|
\-\-seafile\-pass string Password
|
|
\-\-seafile\-url string URL of seafile host to connect to
|
|
\-\-seafile\-user string User name (usually email address)
|
|
\-\-sftp\-ask\-password Allow asking for SFTP password when needed.
|
|
\-\-sftp\-disable\-hashcheck Disable the execution of SSH commands to determine if remote file hashing is available.
|
|
\-\-sftp\-host string SSH host to connect to
|
|
\-\-sftp\-key\-file string Path to PEM\-encoded private key file, leave blank or set key\-use\-agent to use ssh\-agent.
|
|
\-\-sftp\-key\-file\-pass string The passphrase to decrypt the PEM\-encoded private key file.
|
|
\-\-sftp\-key\-pem string Raw PEM\-encoded private key, If specified, will override key_file parameter.
|
|
\-\-sftp\-key\-use\-agent When set forces the usage of the ssh\-agent.
|
|
\-\-sftp\-md5sum\-command string The command used to read md5 hashes. Leave blank for autodetect.
|
|
\-\-sftp\-pass string SSH password, leave blank to use ssh\-agent.
|
|
\-\-sftp\-path\-override string Override path used by SSH connection.
|
|
\-\-sftp\-port string SSH port, leave blank to use default (22)
|
|
\-\-sftp\-set\-modtime Set the modified time on the remote if set. (default true)
|
|
\-\-sftp\-sha1sum\-command string The command used to read sha1 hashes. Leave blank for autodetect.
|
|
\-\-sftp\-skip\-links Set to skip any symlinks and any other non regular files.
|
|
\-\-sftp\-use\-insecure\-cipher Enable the use of insecure ciphers and key exchange methods.
|
|
\-\-sftp\-user string SSH username, leave blank for current username, ncw
|
|
\-\-sharefile\-chunk\-size SizeSuffix Upload chunk size. Must a power of 2 >= 256k. (default 64M)
|
|
\-\-sharefile\-encoding MultiEncoder This sets the encoding for the backend. (default Slash,LtGt,DoubleQuote,Colon,Question,Asterisk,Pipe,BackSlash,Ctl,LeftSpace,LeftPeriod,RightSpace,RightPeriod,InvalidUtf8,Dot)
|
|
\-\-sharefile\-endpoint string Endpoint for API calls.
|
|
\-\-sharefile\-root\-folder\-id string ID of the root folder
|
|
\-\-sharefile\-upload\-cutoff SizeSuffix Cutoff for switching to multipart upload. (default 128M)
|
|
\-\-skip\-links Don\[aq]t warn about skipped symlinks.
|
|
\-\-sugarsync\-access\-key\-id string Sugarsync Access Key ID.
|
|
\-\-sugarsync\-app\-id string Sugarsync App ID.
|
|
\-\-sugarsync\-authorization string Sugarsync authorization
|
|
\-\-sugarsync\-authorization\-expiry string Sugarsync authorization expiry
|
|
\-\-sugarsync\-deleted\-id string Sugarsync deleted folder id
|
|
\-\-sugarsync\-encoding MultiEncoder This sets the encoding for the backend. (default Slash,Ctl,InvalidUtf8,Dot)
|
|
\-\-sugarsync\-hard\-delete Permanently delete files if true
|
|
\-\-sugarsync\-private\-access\-key string Sugarsync Private Access Key
|
|
\-\-sugarsync\-refresh\-token string Sugarsync refresh token
|
|
\-\-sugarsync\-root\-id string Sugarsync root id
|
|
\-\-sugarsync\-user string Sugarsync user
|
|
\-\-swift\-application\-credential\-id string Application Credential ID (OS_APPLICATION_CREDENTIAL_ID)
|
|
\-\-swift\-application\-credential\-name string Application Credential Name (OS_APPLICATION_CREDENTIAL_NAME)
|
|
\-\-swift\-application\-credential\-secret string Application Credential Secret (OS_APPLICATION_CREDENTIAL_SECRET)
|
|
\-\-swift\-auth string Authentication URL for server (OS_AUTH_URL).
|
|
\-\-swift\-auth\-token string Auth Token from alternate authentication \- optional (OS_AUTH_TOKEN)
|
|
\-\-swift\-auth\-version int AuthVersion \- optional \- set to (1,2,3) if your auth URL has no version (ST_AUTH_VERSION)
|
|
\-\-swift\-chunk\-size SizeSuffix Above this size files will be chunked into a _segments container. (default 5G)
|
|
\-\-swift\-domain string User domain \- optional (v3 auth) (OS_USER_DOMAIN_NAME)
|
|
\-\-swift\-encoding MultiEncoder This sets the encoding for the backend. (default Slash,InvalidUtf8)
|
|
\-\-swift\-endpoint\-type string Endpoint type to choose from the service catalogue (OS_ENDPOINT_TYPE) (default \[dq]public\[dq])
|
|
\-\-swift\-env\-auth Get swift credentials from environment variables in standard OpenStack form.
|
|
\-\-swift\-key string API key or password (OS_PASSWORD).
|
|
\-\-swift\-no\-chunk Don\[aq]t chunk files during streaming upload.
|
|
\-\-swift\-region string Region name \- optional (OS_REGION_NAME)
|
|
\-\-swift\-storage\-policy string The storage policy to use when creating a new container
|
|
\-\-swift\-storage\-url string Storage URL \- optional (OS_STORAGE_URL)
|
|
\-\-swift\-tenant string Tenant name \- optional for v1 auth, this or tenant_id required otherwise (OS_TENANT_NAME or OS_PROJECT_NAME)
|
|
\-\-swift\-tenant\-domain string Tenant domain \- optional (v3 auth) (OS_PROJECT_DOMAIN_NAME)
|
|
\-\-swift\-tenant\-id string Tenant ID \- optional for v1 auth, this or tenant required otherwise (OS_TENANT_ID)
|
|
\-\-swift\-user string User name to log in (OS_USERNAME).
|
|
\-\-swift\-user\-id string User ID to log in \- optional \- most swift systems use user and leave this blank (v3 auth) (OS_USER_ID).
|
|
\-\-tardigrade\-access\-grant string Access Grant.
|
|
\-\-tardigrade\-api\-key string API Key.
|
|
\-\-tardigrade\-passphrase string Encryption Passphrase. To access existing objects enter passphrase used for uploading.
|
|
\-\-tardigrade\-provider string Choose an authentication method. (default \[dq]existing\[dq])
|
|
\-\-tardigrade\-satellite\-address <nodeid>\[at]<address>:<port> Satellite Address. Custom satellite address should match the format: <nodeid>\[at]<address>:<port>. (default \[dq]us\-central\-1.tardigrade.io\[dq])
|
|
\-\-union\-action\-policy string Policy to choose upstream on ACTION category. (default \[dq]epall\[dq])
|
|
\-\-union\-cache\-time int Cache time of usage and free space (in seconds). This option is only useful when a path preserving policy is used. (default 120)
|
|
\-\-union\-create\-policy string Policy to choose upstream on CREATE category. (default \[dq]epmfs\[dq])
|
|
\-\-union\-search\-policy string Policy to choose upstream on SEARCH category. (default \[dq]ff\[dq])
|
|
\-\-union\-upstreams string List of space separated upstreams.
|
|
\-\-webdav\-bearer\-token string Bearer token instead of user/pass (eg a Macaroon)
|
|
\-\-webdav\-bearer\-token\-command string Command to run to get a bearer token
|
|
\-\-webdav\-pass string Password.
|
|
\-\-webdav\-url string URL of http host to connect to
|
|
\-\-webdav\-user string User name
|
|
\-\-webdav\-vendor string Name of the Webdav site/service/software you are using
|
|
\-\-yandex\-client\-id string Yandex Client Id
|
|
\-\-yandex\-client\-secret string Yandex Client Secret
|
|
\-\-yandex\-encoding MultiEncoder This sets the encoding for the backend. (default Slash,Del,Ctl,InvalidUtf8,Dot)
|
|
\-\-yandex\-unlink Remove existing public link to file/folder with link command rather than creating.
|
|
\f[R]
|
|
.fi
|
|
.SS 1Fichier
|
|
.PP
|
|
This is a backend for the 1fichier (https://1fichier.com) cloud storage
|
|
service.
|
|
Note that a Premium subscription is required to use the API.
|
|
.PP
|
|
Paths are specified as \f[C]remote:path\f[R]
|
|
.PP
|
|
Paths may be as deep as required, eg
|
|
\f[C]remote:directory/subdirectory\f[R].
|
|
.PP
|
|
The initial setup for 1Fichier involves getting the API key from the
|
|
website which you need to do in your browser.
|
|
.PP
|
|
Here is an example of how to make a remote called \f[C]remote\f[R].
|
|
First run:
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone config
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
This will guide you through an interactive setup process:
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
No remotes found \- make a new one
|
|
n) New remote
|
|
s) Set configuration password
|
|
q) Quit config
|
|
n/s/q> n
|
|
name> remote
|
|
Type of storage to configure.
|
|
Enter a string value. Press Enter for the default (\[dq]\[dq]).
|
|
Choose a number from below, or type in your own value
|
|
[snip]
|
|
XX / 1Fichier
|
|
\[rs] \[dq]fichier\[dq]
|
|
[snip]
|
|
Storage> fichier
|
|
** See help for fichier backend at: https://rclone.org/fichier/ **
|
|
|
|
Your API Key, get it from https://1fichier.com/console/params.pl
|
|
Enter a string value. Press Enter for the default (\[dq]\[dq]).
|
|
api_key> example_key
|
|
|
|
Edit advanced config? (y/n)
|
|
y) Yes
|
|
n) No
|
|
y/n>
|
|
Remote config
|
|
\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-
|
|
[remote]
|
|
type = fichier
|
|
api_key = example_key
|
|
\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-
|
|
y) Yes this is OK
|
|
e) Edit this remote
|
|
d) Delete this remote
|
|
y/e/d> y
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
Once configured you can then use \f[C]rclone\f[R] like this,
|
|
.PP
|
|
List directories in top level of your 1Fichier account
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone lsd remote:
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
List all the files in your 1Fichier account
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone ls remote:
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
To copy a local directory to a 1Fichier directory called backup
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone copy /home/source remote:backup
|
|
\f[R]
|
|
.fi
|
|
.SS Modified time and hashes
|
|
.PP
|
|
1Fichier does not support modification times.
|
|
It supports the Whirlpool hash algorithm.
|
|
.SS Duplicated files
|
|
.PP
|
|
1Fichier can have two files with exactly the same name and path (unlike
|
|
a normal file system).
|
|
.PP
|
|
Duplicated files cause problems with the syncing and you will see
|
|
messages in the log about duplicates.
|
|
.SS Restricted filename characters
|
|
.PP
|
|
In addition to the default restricted characters
|
|
set (https://rclone.org/overview/#restricted-characters) the following
|
|
characters are also replaced:
|
|
.PP
|
|
.TS
|
|
tab(@);
|
|
l c c.
|
|
T{
|
|
Character
|
|
T}@T{
|
|
Value
|
|
T}@T{
|
|
Replacement
|
|
T}
|
|
_
|
|
T{
|
|
\[rs]
|
|
T}@T{
|
|
0x5C
|
|
T}@T{
|
|
\[uFF3C]
|
|
T}
|
|
T{
|
|
<
|
|
T}@T{
|
|
0x3C
|
|
T}@T{
|
|
\[uFF1C]
|
|
T}
|
|
T{
|
|
>
|
|
T}@T{
|
|
0x3E
|
|
T}@T{
|
|
\[uFF1E]
|
|
T}
|
|
T{
|
|
\[dq]
|
|
T}@T{
|
|
0x22
|
|
T}@T{
|
|
\[uFF02]
|
|
T}
|
|
T{
|
|
$
|
|
T}@T{
|
|
0x24
|
|
T}@T{
|
|
\[uFF04]
|
|
T}
|
|
T{
|
|
\[ga]
|
|
T}@T{
|
|
0x60
|
|
T}@T{
|
|
\[uFF40]
|
|
T}
|
|
T{
|
|
\[cq]
|
|
T}@T{
|
|
0x27
|
|
T}@T{
|
|
\[uFF07]
|
|
T}
|
|
.TE
|
|
.PP
|
|
File names can also not start or end with the following characters.
|
|
These only get replaced if they are the first or last character in the
|
|
name:
|
|
.PP
|
|
.TS
|
|
tab(@);
|
|
l c c.
|
|
T{
|
|
Character
|
|
T}@T{
|
|
Value
|
|
T}@T{
|
|
Replacement
|
|
T}
|
|
_
|
|
T{
|
|
SP
|
|
T}@T{
|
|
0x20
|
|
T}@T{
|
|
\[u2420]
|
|
T}
|
|
.TE
|
|
.PP
|
|
Invalid UTF\-8 bytes will also be
|
|
replaced (https://rclone.org/overview/#invalid-utf8), as they can\[cq]t
|
|
be used in JSON strings.
|
|
.SS Standard Options
|
|
.PP
|
|
Here are the standard options specific to fichier (1Fichier).
|
|
.SS \[en]fichier\-api\-key
|
|
.PP
|
|
Your API Key, get it from https://1fichier.com/console/params.pl
|
|
.IP \[bu] 2
|
|
Config: api_key
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_FICHIER_API_KEY
|
|
.IP \[bu] 2
|
|
Type: string
|
|
.IP \[bu] 2
|
|
Default: \[dq]\[dq]
|
|
.SS Advanced Options
|
|
.PP
|
|
Here are the advanced options specific to fichier (1Fichier).
|
|
.SS \[en]fichier\-shared\-folder
|
|
.PP
|
|
If you want to download a shared folder, add this parameter
|
|
.IP \[bu] 2
|
|
Config: shared_folder
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_FICHIER_SHARED_FOLDER
|
|
.IP \[bu] 2
|
|
Type: string
|
|
.IP \[bu] 2
|
|
Default: \[dq]\[dq]
|
|
.SS \[en]fichier\-encoding
|
|
.PP
|
|
This sets the encoding for the backend.
|
|
.PP
|
|
See: the encoding section in the
|
|
overview (https://rclone.org/overview/#encoding) for more info.
|
|
.IP \[bu] 2
|
|
Config: encoding
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_FICHIER_ENCODING
|
|
.IP \[bu] 2
|
|
Type: MultiEncoder
|
|
.IP \[bu] 2
|
|
Default:
|
|
Slash,LtGt,DoubleQuote,SingleQuote,BackQuote,Dollar,BackSlash,Del,Ctl,LeftSpace,RightSpace,InvalidUtf8,Dot
|
|
.SS Alias
|
|
.PP
|
|
The \f[C]alias\f[R] remote provides a new name for another remote.
|
|
.PP
|
|
Paths may be as deep as required or a local path, eg
|
|
\f[C]remote:directory/subdirectory\f[R] or
|
|
\f[C]/directory/subdirectory\f[R].
|
|
.PP
|
|
During the initial setup with \f[C]rclone config\f[R] you will specify
|
|
the target remote.
|
|
The target remote can either be a local path or another remote.
|
|
.PP
|
|
Subfolders can be used in target remote.
|
|
Assume an alias remote named \f[C]backup\f[R] with the target
|
|
\f[C]mydrive:private/backup\f[R].
|
|
Invoking \f[C]rclone mkdir backup:desktop\f[R] is exactly the same as
|
|
invoking \f[C]rclone mkdir mydrive:private/backup/desktop\f[R].
|
|
.PP
|
|
There will be no special handling of paths containing \f[C]..\f[R]
|
|
segments.
|
|
Invoking \f[C]rclone mkdir backup:../desktop\f[R] is exactly the same as
|
|
invoking \f[C]rclone mkdir mydrive:private/backup/../desktop\f[R].
|
|
The empty path is not allowed as a remote.
|
|
To alias the current directory use \f[C].\f[R] instead.
|
|
.PP
|
|
Here is an example of how to make an alias called \f[C]remote\f[R] for
|
|
local folder.
|
|
First run:
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone config
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
This will guide you through an interactive setup process:
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
No remotes found \- make a new one
|
|
n) New remote
|
|
s) Set configuration password
|
|
q) Quit config
|
|
n/s/q> n
|
|
name> remote
|
|
Type of storage to configure.
|
|
Choose a number from below, or type in your own value
|
|
[snip]
|
|
XX / Alias for an existing remote
|
|
\[rs] \[dq]alias\[dq]
|
|
[snip]
|
|
Storage> alias
|
|
Remote or path to alias.
|
|
Can be \[dq]myremote:path/to/dir\[dq], \[dq]myremote:bucket\[dq], \[dq]myremote:\[dq] or \[dq]/local/path\[dq].
|
|
remote> /mnt/storage/backup
|
|
Remote config
|
|
\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-
|
|
[remote]
|
|
remote = /mnt/storage/backup
|
|
\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-
|
|
y) Yes this is OK
|
|
e) Edit this remote
|
|
d) Delete this remote
|
|
y/e/d> y
|
|
Current remotes:
|
|
|
|
Name Type
|
|
==== ====
|
|
remote alias
|
|
|
|
e) Edit existing remote
|
|
n) New remote
|
|
d) Delete remote
|
|
r) Rename remote
|
|
c) Copy remote
|
|
s) Set configuration password
|
|
q) Quit config
|
|
e/n/d/r/c/s/q> q
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
Once configured you can then use \f[C]rclone\f[R] like this,
|
|
.PP
|
|
List directories in top level in \f[C]/mnt/storage/backup\f[R]
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone lsd remote:
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
List all the files in \f[C]/mnt/storage/backup\f[R]
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone ls remote:
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
Copy another local directory to the alias directory called source
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone copy /home/source remote:source
|
|
\f[R]
|
|
.fi
|
|
.SS Standard Options
|
|
.PP
|
|
Here are the standard options specific to alias (Alias for an existing
|
|
remote).
|
|
.SS \[en]alias\-remote
|
|
.PP
|
|
Remote or path to alias.
|
|
Can be \[lq]myremote:path/to/dir\[rq], \[lq]myremote:bucket\[rq],
|
|
\[lq]myremote:\[rq] or \[lq]/local/path\[rq].
|
|
.IP \[bu] 2
|
|
Config: remote
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_ALIAS_REMOTE
|
|
.IP \[bu] 2
|
|
Type: string
|
|
.IP \[bu] 2
|
|
Default: \[dq]\[dq]
|
|
.SS Amazon Drive
|
|
.PP
|
|
Amazon Drive, formerly known as Amazon Cloud Drive, is a cloud storage
|
|
service run by Amazon for consumers.
|
|
.SS Status
|
|
.PP
|
|
\f[B]Important:\f[R] rclone supports Amazon Drive only if you have your
|
|
own set of API keys.
|
|
Unfortunately the Amazon Drive developer
|
|
program (https://developer.amazon.com/amazon-drive) is now closed to new
|
|
entries so if you don\[cq]t already have your own set of keys you will
|
|
not be able to use rclone with Amazon Drive.
|
|
.PP
|
|
For the history on why rclone no longer has a set of Amazon Drive API
|
|
keys see the
|
|
forum (https://forum.rclone.org/t/rclone-has-been-banned-from-amazon-drive/2314).
|
|
.PP
|
|
If you happen to know anyone who works at Amazon then please ask them to
|
|
re\-instate rclone into the Amazon Drive developer program \- thanks!
|
|
.SS Setup
|
|
.PP
|
|
The initial setup for Amazon Drive involves getting a token from Amazon
|
|
which you need to do in your browser.
|
|
\f[C]rclone config\f[R] walks you through it.
|
|
.PP
|
|
The configuration process for Amazon Drive may involve using an oauth
|
|
proxy (https://github.com/ncw/oauthproxy).
|
|
This is used to keep the Amazon credentials out of the source code.
|
|
The proxy runs in Google\[cq]s very secure App Engine environment and
|
|
doesn\[cq]t store any credentials which pass through it.
|
|
.PP
|
|
Since rclone doesn\[cq]t currently have its own Amazon Drive credentials
|
|
so you will either need to have your own \f[C]client_id\f[R] and
|
|
\f[C]client_secret\f[R] with Amazon Drive, or use a third party oauth
|
|
proxy in which case you will need to enter \f[C]client_id\f[R],
|
|
\f[C]client_secret\f[R], \f[C]auth_url\f[R] and \f[C]token_url\f[R].
|
|
.PP
|
|
Note also if you are not using Amazon\[cq]s \f[C]auth_url\f[R] and
|
|
\f[C]token_url\f[R], (ie you filled in something for those) then if
|
|
setting up on a remote machine you can only use the copying the config
|
|
method of
|
|
configuration (https://rclone.org/remote_setup/#configuring-by-copying-the-config-file)
|
|
\- \f[C]rclone authorize\f[R] will not work.
|
|
.PP
|
|
Here is an example of how to make a remote called \f[C]remote\f[R].
|
|
First run:
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone config
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
This will guide you through an interactive setup process:
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
No remotes found \- make a new one
|
|
n) New remote
|
|
r) Rename remote
|
|
c) Copy remote
|
|
s) Set configuration password
|
|
q) Quit config
|
|
n/r/c/s/q> n
|
|
name> remote
|
|
Type of storage to configure.
|
|
Choose a number from below, or type in your own value
|
|
[snip]
|
|
XX / Amazon Drive
|
|
\[rs] \[dq]amazon cloud drive\[dq]
|
|
[snip]
|
|
Storage> amazon cloud drive
|
|
Amazon Application Client Id \- required.
|
|
client_id> your client ID goes here
|
|
Amazon Application Client Secret \- required.
|
|
client_secret> your client secret goes here
|
|
Auth server URL \- leave blank to use Amazon\[aq]s.
|
|
auth_url> Optional auth URL
|
|
Token server url \- leave blank to use Amazon\[aq]s.
|
|
token_url> Optional token URL
|
|
Remote config
|
|
Make sure your Redirect URL is set to \[dq]http://127.0.0.1:53682/\[dq] in your custom config.
|
|
Use auto config?
|
|
* Say Y if not sure
|
|
* Say N if you are working on a remote or headless machine
|
|
y) Yes
|
|
n) No
|
|
y/n> y
|
|
If your browser doesn\[aq]t open automatically go to the following link: http://127.0.0.1:53682/auth
|
|
Log in and authorize rclone for access
|
|
Waiting for code...
|
|
Got code
|
|
\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-
|
|
[remote]
|
|
client_id = your client ID goes here
|
|
client_secret = your client secret goes here
|
|
auth_url = Optional auth URL
|
|
token_url = Optional token URL
|
|
token = {\[dq]access_token\[dq]:\[dq]xxxxxxxxxxxxxxxxxxxxxxx\[dq],\[dq]token_type\[dq]:\[dq]bearer\[dq],\[dq]refresh_token\[dq]:\[dq]xxxxxxxxxxxxxxxxxx\[dq],\[dq]expiry\[dq]:\[dq]2015\-09\-06T16:07:39.658438471+01:00\[dq]}
|
|
\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-
|
|
y) Yes this is OK
|
|
e) Edit this remote
|
|
d) Delete this remote
|
|
y/e/d> y
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
See the remote setup docs (https://rclone.org/remote_setup/) for how to
|
|
set it up on a machine with no Internet browser available.
|
|
.PP
|
|
Note that rclone runs a webserver on your local machine to collect the
|
|
token as returned from Amazon.
|
|
This only runs from the moment it opens your browser to the moment you
|
|
get back the verification code.
|
|
This is on \f[C]http://127.0.0.1:53682/\f[R] and this it may require you
|
|
to unblock it temporarily if you are running a host firewall.
|
|
.PP
|
|
Once configured you can then use \f[C]rclone\f[R] like this,
|
|
.PP
|
|
List directories in top level of your Amazon Drive
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone lsd remote:
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
List all the files in your Amazon Drive
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone ls remote:
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
To copy a local directory to an Amazon Drive directory called backup
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone copy /home/source remote:backup
|
|
\f[R]
|
|
.fi
|
|
.SS Modified time and MD5SUMs
|
|
.PP
|
|
Amazon Drive doesn\[cq]t allow modification times to be changed via the
|
|
API so these won\[cq]t be accurate or used for syncing.
|
|
.PP
|
|
It does store MD5SUMs so for a more accurate sync, you can use the
|
|
\f[C]\-\-checksum\f[R] flag.
|
|
.SS Restricted filename characters
|
|
.PP
|
|
.TS
|
|
tab(@);
|
|
l c c.
|
|
T{
|
|
Character
|
|
T}@T{
|
|
Value
|
|
T}@T{
|
|
Replacement
|
|
T}
|
|
_
|
|
T{
|
|
NUL
|
|
T}@T{
|
|
0x00
|
|
T}@T{
|
|
\[u2400]
|
|
T}
|
|
T{
|
|
/
|
|
T}@T{
|
|
0x2F
|
|
T}@T{
|
|
\[uFF0F]
|
|
T}
|
|
.TE
|
|
.PP
|
|
Invalid UTF\-8 bytes will also be
|
|
replaced (https://rclone.org/overview/#invalid-utf8), as they can\[cq]t
|
|
be used in JSON strings.
|
|
.SS Deleting files
|
|
.PP
|
|
Any files you delete with rclone will end up in the trash.
|
|
Amazon don\[cq]t provide an API to permanently delete files, nor to
|
|
empty the trash, so you will have to do that with one of Amazon\[cq]s
|
|
apps or via the Amazon Drive website.
|
|
As of November 17, 2016, files are automatically deleted by Amazon from
|
|
the trash after 30 days.
|
|
.SS Using with non \f[C].com\f[R] Amazon accounts
|
|
.PP
|
|
Let\[cq]s say you usually use \f[C]amazon.co.uk\f[R].
|
|
When you authenticate with rclone it will take you to an
|
|
\f[C]amazon.com\f[R] page to log in.
|
|
Your \f[C]amazon.co.uk\f[R] email and password should work here just
|
|
fine.
|
|
.SS Standard Options
|
|
.PP
|
|
Here are the standard options specific to amazon cloud drive (Amazon
|
|
Drive).
|
|
.SS \[en]acd\-client\-id
|
|
.PP
|
|
Amazon Application Client ID.
|
|
.IP \[bu] 2
|
|
Config: client_id
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_ACD_CLIENT_ID
|
|
.IP \[bu] 2
|
|
Type: string
|
|
.IP \[bu] 2
|
|
Default: \[dq]\[dq]
|
|
.SS \[en]acd\-client\-secret
|
|
.PP
|
|
Amazon Application Client Secret.
|
|
.IP \[bu] 2
|
|
Config: client_secret
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_ACD_CLIENT_SECRET
|
|
.IP \[bu] 2
|
|
Type: string
|
|
.IP \[bu] 2
|
|
Default: \[dq]\[dq]
|
|
.SS Advanced Options
|
|
.PP
|
|
Here are the advanced options specific to amazon cloud drive (Amazon
|
|
Drive).
|
|
.SS \[en]acd\-auth\-url
|
|
.PP
|
|
Auth server URL.
|
|
Leave blank to use Amazon\[cq]s.
|
|
.IP \[bu] 2
|
|
Config: auth_url
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_ACD_AUTH_URL
|
|
.IP \[bu] 2
|
|
Type: string
|
|
.IP \[bu] 2
|
|
Default: \[dq]\[dq]
|
|
.SS \[en]acd\-token\-url
|
|
.PP
|
|
Token server url.
|
|
leave blank to use Amazon\[cq]s.
|
|
.IP \[bu] 2
|
|
Config: token_url
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_ACD_TOKEN_URL
|
|
.IP \[bu] 2
|
|
Type: string
|
|
.IP \[bu] 2
|
|
Default: \[dq]\[dq]
|
|
.SS \[en]acd\-checkpoint
|
|
.PP
|
|
Checkpoint for internal polling (debug).
|
|
.IP \[bu] 2
|
|
Config: checkpoint
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_ACD_CHECKPOINT
|
|
.IP \[bu] 2
|
|
Type: string
|
|
.IP \[bu] 2
|
|
Default: \[dq]\[dq]
|
|
.SS \[en]acd\-upload\-wait\-per\-gb
|
|
.PP
|
|
Additional time per GB to wait after a failed complete upload to see if
|
|
it appears.
|
|
.PP
|
|
Sometimes Amazon Drive gives an error when a file has been fully
|
|
uploaded but the file appears anyway after a little while.
|
|
This happens sometimes for files over 1GB in size and nearly every time
|
|
for files bigger than 10GB.
|
|
This parameter controls the time rclone waits for the file to appear.
|
|
.PP
|
|
The default value for this parameter is 3 minutes per GB, so by default
|
|
it will wait 3 minutes for every GB uploaded to see if the file appears.
|
|
.PP
|
|
You can disable this feature by setting it to 0.
|
|
This may cause conflict errors as rclone retries the failed upload but
|
|
the file will most likely appear correctly eventually.
|
|
.PP
|
|
These values were determined empirically by observing lots of uploads of
|
|
big files for a range of file sizes.
|
|
.PP
|
|
Upload with the \[lq]\-v\[rq] flag to see more info about what rclone is
|
|
doing in this situation.
|
|
.IP \[bu] 2
|
|
Config: upload_wait_per_gb
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_ACD_UPLOAD_WAIT_PER_GB
|
|
.IP \[bu] 2
|
|
Type: Duration
|
|
.IP \[bu] 2
|
|
Default: 3m0s
|
|
.SS \[en]acd\-templink\-threshold
|
|
.PP
|
|
Files >= this size will be downloaded via their tempLink.
|
|
.PP
|
|
Files this size or more will be downloaded via their \[lq]tempLink\[rq].
|
|
This is to work around a problem with Amazon Drive which blocks
|
|
downloads of files bigger than about 10GB.
|
|
The default for this is 9GB which shouldn\[cq]t need to be changed.
|
|
.PP
|
|
To download files above this threshold, rclone requests a
|
|
\[lq]tempLink\[rq] which downloads the file through a temporary URL
|
|
directly from the underlying S3 storage.
|
|
.IP \[bu] 2
|
|
Config: templink_threshold
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_ACD_TEMPLINK_THRESHOLD
|
|
.IP \[bu] 2
|
|
Type: SizeSuffix
|
|
.IP \[bu] 2
|
|
Default: 9G
|
|
.SS \[en]acd\-encoding
|
|
.PP
|
|
This sets the encoding for the backend.
|
|
.PP
|
|
See: the encoding section in the
|
|
overview (https://rclone.org/overview/#encoding) for more info.
|
|
.IP \[bu] 2
|
|
Config: encoding
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_ACD_ENCODING
|
|
.IP \[bu] 2
|
|
Type: MultiEncoder
|
|
.IP \[bu] 2
|
|
Default: Slash,InvalidUtf8,Dot
|
|
.SS Limitations
|
|
.PP
|
|
Note that Amazon Drive is case insensitive so you can\[cq]t have a file
|
|
called \[lq]Hello.doc\[rq] and one called \[lq]hello.doc\[rq].
|
|
.PP
|
|
Amazon Drive has rate limiting so you may notice errors in the sync (429
|
|
errors).
|
|
rclone will automatically retry the sync up to 3 times by default (see
|
|
\f[C]\-\-retries\f[R] flag) which should hopefully work around this
|
|
problem.
|
|
.PP
|
|
Amazon Drive has an internal limit of file sizes that can be uploaded to
|
|
the service.
|
|
This limit is not officially published, but all files larger than this
|
|
will fail.
|
|
.PP
|
|
At the time of writing (Jan 2016) is in the area of 50GB per file.
|
|
This means that larger files are likely to fail.
|
|
.PP
|
|
Unfortunately there is no way for rclone to see that this failure is
|
|
because of file size, so it will retry the operation, as any other
|
|
failure.
|
|
To avoid this problem, use \f[C]\-\-max\-size 50000M\f[R] option to
|
|
limit the maximum size of uploaded files.
|
|
Note that \f[C]\-\-max\-size\f[R] does not split files into segments, it
|
|
only ignores files over this size.
|
|
.SS Amazon S3 Storage Providers
|
|
.PP
|
|
The S3 backend can be used with a number of different providers:
|
|
.IP \[bu] 2
|
|
AWS S3
|
|
.IP \[bu] 2
|
|
Alibaba Cloud (Aliyun) Object Storage System (OSS)
|
|
.IP \[bu] 2
|
|
Ceph
|
|
.IP \[bu] 2
|
|
DigitalOcean Spaces
|
|
.IP \[bu] 2
|
|
Dreamhost
|
|
.IP \[bu] 2
|
|
IBM COS S3
|
|
.IP \[bu] 2
|
|
Minio
|
|
.IP \[bu] 2
|
|
Scaleway
|
|
.IP \[bu] 2
|
|
StackPath
|
|
.IP \[bu] 2
|
|
Wasabi
|
|
.PP
|
|
Paths are specified as \f[C]remote:bucket\f[R] (or \f[C]remote:\f[R] for
|
|
the \f[C]lsd\f[R] command.) You may put subdirectories in too, eg
|
|
\f[C]remote:bucket/path/to/dir\f[R].
|
|
.PP
|
|
Once you have made a remote (see the provider specific section above)
|
|
you can use it like this:
|
|
.PP
|
|
See all buckets
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone lsd remote:
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
Make a new bucket
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone mkdir remote:bucket
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
List the contents of a bucket
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone ls remote:bucket
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
Sync \f[C]/home/local/directory\f[R] to the remote bucket, deleting any
|
|
excess files in the bucket.
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone sync /home/local/directory remote:bucket
|
|
\f[R]
|
|
.fi
|
|
.SS AWS S3
|
|
.PP
|
|
Here is an example of making an s3 configuration.
|
|
First run
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone config
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
This will guide you through an interactive setup process.
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
No remotes found \- make a new one
|
|
n) New remote
|
|
s) Set configuration password
|
|
q) Quit config
|
|
n/s/q> n
|
|
name> remote
|
|
Type of storage to configure.
|
|
Choose a number from below, or type in your own value
|
|
[snip]
|
|
XX / Amazon S3 Compliant Storage Providers (AWS, Ceph, Dreamhost, IBM COS, Minio)
|
|
\[rs] \[dq]s3\[dq]
|
|
[snip]
|
|
Storage> s3
|
|
Choose your S3 provider.
|
|
Choose a number from below, or type in your own value
|
|
1 / Amazon Web Services (AWS) S3
|
|
\[rs] \[dq]AWS\[dq]
|
|
2 / Ceph Object Storage
|
|
\[rs] \[dq]Ceph\[dq]
|
|
3 / Digital Ocean Spaces
|
|
\[rs] \[dq]DigitalOcean\[dq]
|
|
4 / Dreamhost DreamObjects
|
|
\[rs] \[dq]Dreamhost\[dq]
|
|
5 / IBM COS S3
|
|
\[rs] \[dq]IBMCOS\[dq]
|
|
6 / Minio Object Storage
|
|
\[rs] \[dq]Minio\[dq]
|
|
7 / Wasabi Object Storage
|
|
\[rs] \[dq]Wasabi\[dq]
|
|
8 / Any other S3 compatible provider
|
|
\[rs] \[dq]Other\[dq]
|
|
provider> 1
|
|
Get AWS credentials from runtime (environment variables or EC2/ECS meta data if no env vars). Only applies if access_key_id and secret_access_key is blank.
|
|
Choose a number from below, or type in your own value
|
|
1 / Enter AWS credentials in the next step
|
|
\[rs] \[dq]false\[dq]
|
|
2 / Get AWS credentials from the environment (env vars or IAM)
|
|
\[rs] \[dq]true\[dq]
|
|
env_auth> 1
|
|
AWS Access Key ID \- leave blank for anonymous access or runtime credentials.
|
|
access_key_id> XXX
|
|
AWS Secret Access Key (password) \- leave blank for anonymous access or runtime credentials.
|
|
secret_access_key> YYY
|
|
Region to connect to.
|
|
Choose a number from below, or type in your own value
|
|
/ The default endpoint \- a good choice if you are unsure.
|
|
1 | US Region, Northern Virginia or Pacific Northwest.
|
|
| Leave location constraint empty.
|
|
\[rs] \[dq]us\-east\-1\[dq]
|
|
/ US East (Ohio) Region
|
|
2 | Needs location constraint us\-east\-2.
|
|
\[rs] \[dq]us\-east\-2\[dq]
|
|
/ US West (Oregon) Region
|
|
3 | Needs location constraint us\-west\-2.
|
|
\[rs] \[dq]us\-west\-2\[dq]
|
|
/ US West (Northern California) Region
|
|
4 | Needs location constraint us\-west\-1.
|
|
\[rs] \[dq]us\-west\-1\[dq]
|
|
/ Canada (Central) Region
|
|
5 | Needs location constraint ca\-central\-1.
|
|
\[rs] \[dq]ca\-central\-1\[dq]
|
|
/ EU (Ireland) Region
|
|
6 | Needs location constraint EU or eu\-west\-1.
|
|
\[rs] \[dq]eu\-west\-1\[dq]
|
|
/ EU (London) Region
|
|
7 | Needs location constraint eu\-west\-2.
|
|
\[rs] \[dq]eu\-west\-2\[dq]
|
|
/ EU (Frankfurt) Region
|
|
8 | Needs location constraint eu\-central\-1.
|
|
\[rs] \[dq]eu\-central\-1\[dq]
|
|
/ Asia Pacific (Singapore) Region
|
|
9 | Needs location constraint ap\-southeast\-1.
|
|
\[rs] \[dq]ap\-southeast\-1\[dq]
|
|
/ Asia Pacific (Sydney) Region
|
|
10 | Needs location constraint ap\-southeast\-2.
|
|
\[rs] \[dq]ap\-southeast\-2\[dq]
|
|
/ Asia Pacific (Tokyo) Region
|
|
11 | Needs location constraint ap\-northeast\-1.
|
|
\[rs] \[dq]ap\-northeast\-1\[dq]
|
|
/ Asia Pacific (Seoul)
|
|
12 | Needs location constraint ap\-northeast\-2.
|
|
\[rs] \[dq]ap\-northeast\-2\[dq]
|
|
/ Asia Pacific (Mumbai)
|
|
13 | Needs location constraint ap\-south\-1.
|
|
\[rs] \[dq]ap\-south\-1\[dq]
|
|
/ Asia Patific (Hong Kong) Region
|
|
14 | Needs location constraint ap\-east\-1.
|
|
\[rs] \[dq]ap\-east\-1\[dq]
|
|
/ South America (Sao Paulo) Region
|
|
15 | Needs location constraint sa\-east\-1.
|
|
\[rs] \[dq]sa\-east\-1\[dq]
|
|
region> 1
|
|
Endpoint for S3 API.
|
|
Leave blank if using AWS to use the default endpoint for the region.
|
|
endpoint>
|
|
Location constraint \- must be set to match the Region. Used when creating buckets only.
|
|
Choose a number from below, or type in your own value
|
|
1 / Empty for US Region, Northern Virginia or Pacific Northwest.
|
|
\[rs] \[dq]\[dq]
|
|
2 / US East (Ohio) Region.
|
|
\[rs] \[dq]us\-east\-2\[dq]
|
|
3 / US West (Oregon) Region.
|
|
\[rs] \[dq]us\-west\-2\[dq]
|
|
4 / US West (Northern California) Region.
|
|
\[rs] \[dq]us\-west\-1\[dq]
|
|
5 / Canada (Central) Region.
|
|
\[rs] \[dq]ca\-central\-1\[dq]
|
|
6 / EU (Ireland) Region.
|
|
\[rs] \[dq]eu\-west\-1\[dq]
|
|
7 / EU (London) Region.
|
|
\[rs] \[dq]eu\-west\-2\[dq]
|
|
8 / EU Region.
|
|
\[rs] \[dq]EU\[dq]
|
|
9 / Asia Pacific (Singapore) Region.
|
|
\[rs] \[dq]ap\-southeast\-1\[dq]
|
|
10 / Asia Pacific (Sydney) Region.
|
|
\[rs] \[dq]ap\-southeast\-2\[dq]
|
|
11 / Asia Pacific (Tokyo) Region.
|
|
\[rs] \[dq]ap\-northeast\-1\[dq]
|
|
12 / Asia Pacific (Seoul)
|
|
\[rs] \[dq]ap\-northeast\-2\[dq]
|
|
13 / Asia Pacific (Mumbai)
|
|
\[rs] \[dq]ap\-south\-1\[dq]
|
|
14 / Asia Pacific (Hong Kong)
|
|
\[rs] \[dq]ap\-east\-1\[dq]
|
|
15 / South America (Sao Paulo) Region.
|
|
\[rs] \[dq]sa\-east\-1\[dq]
|
|
location_constraint> 1
|
|
Canned ACL used when creating buckets and/or storing objects in S3.
|
|
For more info visit https://docs.aws.amazon.com/AmazonS3/latest/dev/acl\-overview.html#canned\-acl
|
|
Choose a number from below, or type in your own value
|
|
1 / Owner gets FULL_CONTROL. No one else has access rights (default).
|
|
\[rs] \[dq]private\[dq]
|
|
2 / Owner gets FULL_CONTROL. The AllUsers group gets READ access.
|
|
\[rs] \[dq]public\-read\[dq]
|
|
/ Owner gets FULL_CONTROL. The AllUsers group gets READ and WRITE access.
|
|
3 | Granting this on a bucket is generally not recommended.
|
|
\[rs] \[dq]public\-read\-write\[dq]
|
|
4 / Owner gets FULL_CONTROL. The AuthenticatedUsers group gets READ access.
|
|
\[rs] \[dq]authenticated\-read\[dq]
|
|
/ Object owner gets FULL_CONTROL. Bucket owner gets READ access.
|
|
5 | If you specify this canned ACL when creating a bucket, Amazon S3 ignores it.
|
|
\[rs] \[dq]bucket\-owner\-read\[dq]
|
|
/ Both the object owner and the bucket owner get FULL_CONTROL over the object.
|
|
6 | If you specify this canned ACL when creating a bucket, Amazon S3 ignores it.
|
|
\[rs] \[dq]bucket\-owner\-full\-control\[dq]
|
|
acl> 1
|
|
The server\-side encryption algorithm used when storing this object in S3.
|
|
Choose a number from below, or type in your own value
|
|
1 / None
|
|
\[rs] \[dq]\[dq]
|
|
2 / AES256
|
|
\[rs] \[dq]AES256\[dq]
|
|
server_side_encryption> 1
|
|
The storage class to use when storing objects in S3.
|
|
Choose a number from below, or type in your own value
|
|
1 / Default
|
|
\[rs] \[dq]\[dq]
|
|
2 / Standard storage class
|
|
\[rs] \[dq]STANDARD\[dq]
|
|
3 / Reduced redundancy storage class
|
|
\[rs] \[dq]REDUCED_REDUNDANCY\[dq]
|
|
4 / Standard Infrequent Access storage class
|
|
\[rs] \[dq]STANDARD_IA\[dq]
|
|
5 / One Zone Infrequent Access storage class
|
|
\[rs] \[dq]ONEZONE_IA\[dq]
|
|
6 / Glacier storage class
|
|
\[rs] \[dq]GLACIER\[dq]
|
|
7 / Glacier Deep Archive storage class
|
|
\[rs] \[dq]DEEP_ARCHIVE\[dq]
|
|
8 / Intelligent\-Tiering storage class
|
|
\[rs] \[dq]INTELLIGENT_TIERING\[dq]
|
|
storage_class> 1
|
|
Remote config
|
|
\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-
|
|
[remote]
|
|
type = s3
|
|
provider = AWS
|
|
env_auth = false
|
|
access_key_id = XXX
|
|
secret_access_key = YYY
|
|
region = us\-east\-1
|
|
endpoint =
|
|
location_constraint =
|
|
acl = private
|
|
server_side_encryption =
|
|
storage_class =
|
|
\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-
|
|
y) Yes this is OK
|
|
e) Edit this remote
|
|
d) Delete this remote
|
|
y/e/d>
|
|
\f[R]
|
|
.fi
|
|
.SS \[en]fast\-list
|
|
.PP
|
|
This remote supports \f[C]\-\-fast\-list\f[R] which allows you to use
|
|
fewer transactions in exchange for more memory.
|
|
See the rclone docs (https://rclone.org/docs/#fast-list) for more
|
|
details.
|
|
.SS \[en]update and \[en]use\-server\-modtime
|
|
.PP
|
|
As noted below, the modified time is stored on metadata on the object.
|
|
It is used by default for all operations that require checking the time
|
|
a file was last updated.
|
|
It allows rclone to treat the remote more like a true filesystem, but it
|
|
is inefficient because it requires an extra API call to retrieve the
|
|
metadata.
|
|
.PP
|
|
For many operations, the time the object was last uploaded to the remote
|
|
is sufficient to determine if it is \[lq]dirty\[rq].
|
|
By using \f[C]\-\-update\f[R] along with
|
|
\f[C]\-\-use\-server\-modtime\f[R], you can avoid the extra API call and
|
|
simply upload files whose local modtime is newer than the time it was
|
|
last uploaded.
|
|
.SS Modified time
|
|
.PP
|
|
The modified time is stored as metadata on the object as
|
|
\f[C]X\-Amz\-Meta\-Mtime\f[R] as floating point since the epoch accurate
|
|
to 1 ns.
|
|
.PP
|
|
If the modification time needs to be updated rclone will attempt to
|
|
perform a server side copy to update the modification if the object can
|
|
be copied in a single part.
|
|
In the case the object is larger than 5Gb or is in Glacier or Glacier
|
|
Deep Archive storage the object will be uploaded rather than copied.
|
|
.SS Restricted filename characters
|
|
.PP
|
|
S3 allows any valid UTF\-8 string as a key.
|
|
.PP
|
|
Invalid UTF\-8 bytes will be
|
|
replaced (https://rclone.org/overview/#invalid-utf8), as they can\[cq]t
|
|
be used in XML.
|
|
.PP
|
|
The following characters are replaced since these are problematic when
|
|
dealing with the REST API:
|
|
.PP
|
|
.TS
|
|
tab(@);
|
|
l c c.
|
|
T{
|
|
Character
|
|
T}@T{
|
|
Value
|
|
T}@T{
|
|
Replacement
|
|
T}
|
|
_
|
|
T{
|
|
NUL
|
|
T}@T{
|
|
0x00
|
|
T}@T{
|
|
\[u2400]
|
|
T}
|
|
T{
|
|
/
|
|
T}@T{
|
|
0x2F
|
|
T}@T{
|
|
\[uFF0F]
|
|
T}
|
|
.TE
|
|
.PP
|
|
The encoding will also encode these file names as they don\[cq]t seem to
|
|
work with the SDK properly:
|
|
.PP
|
|
.TS
|
|
tab(@);
|
|
l c.
|
|
T{
|
|
File name
|
|
T}@T{
|
|
Replacement
|
|
T}
|
|
_
|
|
T{
|
|
\&.
|
|
T}@T{
|
|
\[uFF0E]
|
|
T}
|
|
T{
|
|
\&..
|
|
T}@T{
|
|
\[uFF0E]\[uFF0E]
|
|
T}
|
|
.TE
|
|
.SS Multipart uploads
|
|
.PP
|
|
rclone supports multipart uploads with S3 which means that it can upload
|
|
files bigger than 5GB.
|
|
.PP
|
|
Note that files uploaded \f[I]both\f[R] with multipart upload
|
|
\f[I]and\f[R] through crypt remotes do not have MD5 sums.
|
|
.PP
|
|
rclone switches from single part uploads to multipart uploads at the
|
|
point specified by \f[C]\-\-s3\-upload\-cutoff\f[R].
|
|
This can be a maximum of 5GB and a minimum of 0 (ie always upload
|
|
multipart files).
|
|
.PP
|
|
The chunk sizes used in the multipart upload are specified by
|
|
\f[C]\-\-s3\-chunk\-size\f[R] and the number of chunks uploaded
|
|
concurrently is specified by \f[C]\-\-s3\-upload\-concurrency\f[R].
|
|
.PP
|
|
Multipart uploads will use \f[C]\-\-transfers\f[R] *
|
|
\f[C]\-\-s3\-upload\-concurrency\f[R] * \f[C]\-\-s3\-chunk\-size\f[R]
|
|
extra memory.
|
|
Single part uploads to not use extra memory.
|
|
.PP
|
|
Single part transfers can be faster than multipart transfers or slower
|
|
depending on your latency from S3 \- the more latency, the more likely
|
|
single part transfers will be faster.
|
|
.PP
|
|
Increasing \f[C]\-\-s3\-upload\-concurrency\f[R] will increase
|
|
throughput (8 would be a sensible value) and increasing
|
|
\f[C]\-\-s3\-chunk\-size\f[R] also increases throughput (16M would be
|
|
sensible).
|
|
Increasing either of these will use more memory.
|
|
The default values are high enough to gain most of the possible
|
|
performance without using too much memory.
|
|
.SS Buckets and Regions
|
|
.PP
|
|
With Amazon S3 you can list buckets (\f[C]rclone lsd\f[R]) using any
|
|
region, but you can only access the content of a bucket from the region
|
|
it was created in.
|
|
If you attempt to access a bucket from the wrong region, you will get an
|
|
error,
|
|
\f[C]incorrect region, the bucket is not in \[aq]XXX\[aq] region\f[R].
|
|
.SS Authentication
|
|
.PP
|
|
There are a number of ways to supply \f[C]rclone\f[R] with a set of AWS
|
|
credentials, with and without using the environment.
|
|
.PP
|
|
The different authentication methods are tried in this order:
|
|
.IP \[bu] 2
|
|
Directly in the rclone configuration file (\f[C]env_auth = false\f[R] in
|
|
the config file):
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
\f[C]access_key_id\f[R] and \f[C]secret_access_key\f[R] are required.
|
|
.IP \[bu] 2
|
|
\f[C]session_token\f[R] can be optionally set when using AWS STS.
|
|
.RE
|
|
.IP \[bu] 2
|
|
Runtime configuration (\f[C]env_auth = true\f[R] in the config file):
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Export the following environment variables before running
|
|
\f[C]rclone\f[R]:
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Access Key ID: \f[C]AWS_ACCESS_KEY_ID\f[R] or \f[C]AWS_ACCESS_KEY\f[R]
|
|
.IP \[bu] 2
|
|
Secret Access Key: \f[C]AWS_SECRET_ACCESS_KEY\f[R] or
|
|
\f[C]AWS_SECRET_KEY\f[R]
|
|
.IP \[bu] 2
|
|
Session Token: \f[C]AWS_SESSION_TOKEN\f[R] (optional)
|
|
.RE
|
|
.IP \[bu] 2
|
|
Or, use a named
|
|
profile (https://docs.aws.amazon.com/cli/latest/userguide/cli-multiple-profiles.html):
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Profile files are standard files used by AWS CLI tools
|
|
.IP \[bu] 2
|
|
By default it will use the profile in your home directory (eg
|
|
\f[C]\[ti]/.aws/credentials\f[R] on unix based systems) file and the
|
|
\[lq]default\[rq] profile, to change set these environment variables:
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
\f[C]AWS_SHARED_CREDENTIALS_FILE\f[R] to control which file.
|
|
.IP \[bu] 2
|
|
\f[C]AWS_PROFILE\f[R] to control which profile to use.
|
|
.RE
|
|
.RE
|
|
.IP \[bu] 2
|
|
Or, run \f[C]rclone\f[R] in an ECS task with an IAM role (AWS only).
|
|
.IP \[bu] 2
|
|
Or, run \f[C]rclone\f[R] on an EC2 instance with an IAM role (AWS only).
|
|
.IP \[bu] 2
|
|
Or, run \f[C]rclone\f[R] in an EKS pod with an IAM role that is
|
|
associated with a service account (AWS only).
|
|
.RE
|
|
.PP
|
|
If none of these option actually end up providing \f[C]rclone\f[R] with
|
|
AWS credentials then S3 interaction will be non\-authenticated (see
|
|
below).
|
|
.SS S3 Permissions
|
|
.PP
|
|
When using the \f[C]sync\f[R] subcommand of \f[C]rclone\f[R] the
|
|
following minimum permissions are required to be available on the bucket
|
|
being written to:
|
|
.IP \[bu] 2
|
|
\f[C]ListBucket\f[R]
|
|
.IP \[bu] 2
|
|
\f[C]DeleteObject\f[R]
|
|
.IP \[bu] 2
|
|
\f[C]GetObject\f[R]
|
|
.IP \[bu] 2
|
|
\f[C]PutObject\f[R]
|
|
.IP \[bu] 2
|
|
\f[C]PutObjectACL\f[R]
|
|
.PP
|
|
When using the \f[C]lsd\f[R] subcommand, the \f[C]ListAllMyBuckets\f[R]
|
|
permission is required.
|
|
.PP
|
|
Example policy:
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
{
|
|
\[dq]Version\[dq]: \[dq]2012\-10\-17\[dq],
|
|
\[dq]Statement\[dq]: [
|
|
{
|
|
\[dq]Effect\[dq]: \[dq]Allow\[dq],
|
|
\[dq]Principal\[dq]: {
|
|
\[dq]AWS\[dq]: \[dq]arn:aws:iam::USER_SID:user/USER_NAME\[dq]
|
|
},
|
|
\[dq]Action\[dq]: [
|
|
\[dq]s3:ListBucket\[dq],
|
|
\[dq]s3:DeleteObject\[dq],
|
|
\[dq]s3:GetObject\[dq],
|
|
\[dq]s3:PutObject\[dq],
|
|
\[dq]s3:PutObjectAcl\[dq]
|
|
],
|
|
\[dq]Resource\[dq]: [
|
|
\[dq]arn:aws:s3:::BUCKET_NAME/*\[dq],
|
|
\[dq]arn:aws:s3:::BUCKET_NAME\[dq]
|
|
]
|
|
},
|
|
{
|
|
\[dq]Effect\[dq]: \[dq]Allow\[dq],
|
|
\[dq]Action\[dq]: \[dq]s3:ListAllMyBuckets\[dq],
|
|
\[dq]Resource\[dq]: \[dq]arn:aws:s3:::*\[dq]
|
|
}
|
|
]
|
|
}
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
Notes on above:
|
|
.IP "1." 3
|
|
This is a policy that can be used when creating bucket.
|
|
It assumes that \f[C]USER_NAME\f[R] has been created.
|
|
.IP "2." 3
|
|
The Resource entry must include both resource ARNs, as one implies the
|
|
bucket and the other implies the bucket\[cq]s objects.
|
|
.PP
|
|
For reference, here\[cq]s an Ansible
|
|
script (https://gist.github.com/ebridges/ebfc9042dd7c756cd101cfa807b7ae2b)
|
|
that will generate one or more buckets that will work with
|
|
\f[C]rclone sync\f[R].
|
|
.SS Key Management System (KMS)
|
|
.PP
|
|
If you are using server side encryption with KMS then you will find you
|
|
can\[cq]t transfer small objects.
|
|
As a work\-around you can use the \f[C]\-\-ignore\-checksum\f[R] flag.
|
|
.PP
|
|
A proper fix is being worked on in issue
|
|
#1824 (https://github.com/rclone/rclone/issues/1824).
|
|
.SS Glacier and Glacier Deep Archive
|
|
.PP
|
|
You can upload objects using the glacier storage class or transition
|
|
them to glacier using a lifecycle
|
|
policy (http://docs.aws.amazon.com/AmazonS3/latest/user-guide/create-lifecycle.html).
|
|
The bucket can still be synced or copied into normally, but if rclone
|
|
tries to access data from the glacier storage class you will see an
|
|
error like below.
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
2017/09/11 19:07:43 Failed to sync: failed to open source object: Object in GLACIER, restore first: path/to/file
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
In this case you need to
|
|
restore (http://docs.aws.amazon.com/AmazonS3/latest/user-guide/restore-archived-objects.html)
|
|
the object(s) in question before using rclone.
|
|
.PP
|
|
Note that rclone only speaks the S3 API it does not speak the Glacier
|
|
Vault API, so rclone cannot directly access Glacier Vaults.
|
|
.SS Standard Options
|
|
.PP
|
|
Here are the standard options specific to s3 (Amazon S3 Compliant
|
|
Storage Provider (AWS, Alibaba, Ceph, Digital Ocean, Dreamhost, IBM COS,
|
|
Minio, etc)).
|
|
.SS \[en]s3\-provider
|
|
.PP
|
|
Choose your S3 provider.
|
|
.IP \[bu] 2
|
|
Config: provider
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_S3_PROVIDER
|
|
.IP \[bu] 2
|
|
Type: string
|
|
.IP \[bu] 2
|
|
Default: \[dq]\[dq]
|
|
.IP \[bu] 2
|
|
Examples:
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
\[lq]AWS\[rq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Amazon Web Services (AWS) S3
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[lq]Alibaba\[rq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Alibaba Cloud Object Storage System (OSS) formerly Aliyun
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[lq]Ceph\[rq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Ceph Object Storage
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[lq]DigitalOcean\[rq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Digital Ocean Spaces
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[lq]Dreamhost\[rq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Dreamhost DreamObjects
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[lq]IBMCOS\[rq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
IBM COS S3
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[lq]Minio\[rq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Minio Object Storage
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[lq]Netease\[rq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Netease Object Storage (NOS)
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[lq]StackPath\[rq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
StackPath Object Storage
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[lq]Wasabi\[rq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Wasabi Object Storage
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[lq]Other\[rq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Any other S3 compatible provider
|
|
.RE
|
|
.RE
|
|
.SS \[en]s3\-env\-auth
|
|
.PP
|
|
Get AWS credentials from runtime (environment variables or EC2/ECS meta
|
|
data if no env vars).
|
|
Only applies if access_key_id and secret_access_key is blank.
|
|
.IP \[bu] 2
|
|
Config: env_auth
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_S3_ENV_AUTH
|
|
.IP \[bu] 2
|
|
Type: bool
|
|
.IP \[bu] 2
|
|
Default: false
|
|
.IP \[bu] 2
|
|
Examples:
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
\[lq]false\[rq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Enter AWS credentials in the next step
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[lq]true\[rq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Get AWS credentials from the environment (env vars or IAM)
|
|
.RE
|
|
.RE
|
|
.SS \[en]s3\-access\-key\-id
|
|
.PP
|
|
AWS Access Key ID.
|
|
Leave blank for anonymous access or runtime credentials.
|
|
.IP \[bu] 2
|
|
Config: access_key_id
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_S3_ACCESS_KEY_ID
|
|
.IP \[bu] 2
|
|
Type: string
|
|
.IP \[bu] 2
|
|
Default: \[dq]\[dq]
|
|
.SS \[en]s3\-secret\-access\-key
|
|
.PP
|
|
AWS Secret Access Key (password) Leave blank for anonymous access or
|
|
runtime credentials.
|
|
.IP \[bu] 2
|
|
Config: secret_access_key
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_S3_SECRET_ACCESS_KEY
|
|
.IP \[bu] 2
|
|
Type: string
|
|
.IP \[bu] 2
|
|
Default: \[dq]\[dq]
|
|
.SS \[en]s3\-region
|
|
.PP
|
|
Region to connect to.
|
|
.IP \[bu] 2
|
|
Config: region
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_S3_REGION
|
|
.IP \[bu] 2
|
|
Type: string
|
|
.IP \[bu] 2
|
|
Default: \[dq]\[dq]
|
|
.IP \[bu] 2
|
|
Examples:
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
\[lq]us\-east\-1\[rq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
The default endpoint \- a good choice if you are unsure.
|
|
.IP \[bu] 2
|
|
US Region, Northern Virginia or Pacific Northwest.
|
|
.IP \[bu] 2
|
|
Leave location constraint empty.
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[lq]us\-east\-2\[rq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
US East (Ohio) Region
|
|
.IP \[bu] 2
|
|
Needs location constraint us\-east\-2.
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[lq]us\-west\-2\[rq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
US West (Oregon) Region
|
|
.IP \[bu] 2
|
|
Needs location constraint us\-west\-2.
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[lq]us\-west\-1\[rq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
US West (Northern California) Region
|
|
.IP \[bu] 2
|
|
Needs location constraint us\-west\-1.
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[lq]ca\-central\-1\[rq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Canada (Central) Region
|
|
.IP \[bu] 2
|
|
Needs location constraint ca\-central\-1.
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[lq]eu\-west\-1\[rq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
EU (Ireland) Region
|
|
.IP \[bu] 2
|
|
Needs location constraint EU or eu\-west\-1.
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[lq]eu\-west\-2\[rq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
EU (London) Region
|
|
.IP \[bu] 2
|
|
Needs location constraint eu\-west\-2.
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[lq]eu\-north\-1\[rq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
EU (Stockholm) Region
|
|
.IP \[bu] 2
|
|
Needs location constraint eu\-north\-1.
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[lq]eu\-central\-1\[rq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
EU (Frankfurt) Region
|
|
.IP \[bu] 2
|
|
Needs location constraint eu\-central\-1.
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[lq]ap\-southeast\-1\[rq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Asia Pacific (Singapore) Region
|
|
.IP \[bu] 2
|
|
Needs location constraint ap\-southeast\-1.
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[lq]ap\-southeast\-2\[rq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Asia Pacific (Sydney) Region
|
|
.IP \[bu] 2
|
|
Needs location constraint ap\-southeast\-2.
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[lq]ap\-northeast\-1\[rq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Asia Pacific (Tokyo) Region
|
|
.IP \[bu] 2
|
|
Needs location constraint ap\-northeast\-1.
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[lq]ap\-northeast\-2\[rq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Asia Pacific (Seoul)
|
|
.IP \[bu] 2
|
|
Needs location constraint ap\-northeast\-2.
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[lq]ap\-south\-1\[rq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Asia Pacific (Mumbai)
|
|
.IP \[bu] 2
|
|
Needs location constraint ap\-south\-1.
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[lq]ap\-east\-1\[rq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Asia Patific (Hong Kong) Region
|
|
.IP \[bu] 2
|
|
Needs location constraint ap\-east\-1.
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[lq]sa\-east\-1\[rq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
South America (Sao Paulo) Region
|
|
.IP \[bu] 2
|
|
Needs location constraint sa\-east\-1.
|
|
.RE
|
|
.RE
|
|
.SS \[en]s3\-region
|
|
.PP
|
|
Region to connect to.
|
|
Leave blank if you are using an S3 clone and you don\[cq]t have a
|
|
region.
|
|
.IP \[bu] 2
|
|
Config: region
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_S3_REGION
|
|
.IP \[bu] 2
|
|
Type: string
|
|
.IP \[bu] 2
|
|
Default: \[dq]\[dq]
|
|
.IP \[bu] 2
|
|
Examples:
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
\[dq]\[dq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Use this if unsure.
|
|
Will use v4 signatures and an empty region.
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[lq]other\-v2\-signature\[rq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Use this only if v4 signatures don\[cq]t work, eg pre Jewel/v10 CEPH.
|
|
.RE
|
|
.RE
|
|
.SS \[en]s3\-endpoint
|
|
.PP
|
|
Endpoint for S3 API.
|
|
Leave blank if using AWS to use the default endpoint for the region.
|
|
.IP \[bu] 2
|
|
Config: endpoint
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_S3_ENDPOINT
|
|
.IP \[bu] 2
|
|
Type: string
|
|
.IP \[bu] 2
|
|
Default: \[dq]\[dq]
|
|
.SS \[en]s3\-endpoint
|
|
.PP
|
|
Endpoint for IBM COS S3 API.
|
|
Specify if using an IBM COS On Premise.
|
|
.IP \[bu] 2
|
|
Config: endpoint
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_S3_ENDPOINT
|
|
.IP \[bu] 2
|
|
Type: string
|
|
.IP \[bu] 2
|
|
Default: \[dq]\[dq]
|
|
.IP \[bu] 2
|
|
Examples:
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
\[lq]s3\-api.us\-geo.objectstorage.softlayer.net\[rq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
US Cross Region Endpoint
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[lq]s3\-api.dal.us\-geo.objectstorage.softlayer.net\[rq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
US Cross Region Dallas Endpoint
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[lq]s3\-api.wdc\-us\-geo.objectstorage.softlayer.net\[rq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
US Cross Region Washington DC Endpoint
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[lq]s3\-api.sjc\-us\-geo.objectstorage.softlayer.net\[rq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
US Cross Region San Jose Endpoint
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[lq]s3\-api.us\-geo.objectstorage.service.networklayer.com\[rq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
US Cross Region Private Endpoint
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[lq]s3\-api.dal\-us\-geo.objectstorage.service.networklayer.com\[rq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
US Cross Region Dallas Private Endpoint
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[lq]s3\-api.wdc\-us\-geo.objectstorage.service.networklayer.com\[rq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
US Cross Region Washington DC Private Endpoint
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[lq]s3\-api.sjc\-us\-geo.objectstorage.service.networklayer.com\[rq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
US Cross Region San Jose Private Endpoint
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[lq]s3.us\-east.objectstorage.softlayer.net\[rq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
US Region East Endpoint
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[lq]s3.us\-east.objectstorage.service.networklayer.com\[rq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
US Region East Private Endpoint
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[lq]s3.us\-south.objectstorage.softlayer.net\[rq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
US Region South Endpoint
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[lq]s3.us\-south.objectstorage.service.networklayer.com\[rq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
US Region South Private Endpoint
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[lq]s3.eu\-geo.objectstorage.softlayer.net\[rq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
EU Cross Region Endpoint
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[lq]s3.fra\-eu\-geo.objectstorage.softlayer.net\[rq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
EU Cross Region Frankfurt Endpoint
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[lq]s3.mil\-eu\-geo.objectstorage.softlayer.net\[rq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
EU Cross Region Milan Endpoint
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[lq]s3.ams\-eu\-geo.objectstorage.softlayer.net\[rq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
EU Cross Region Amsterdam Endpoint
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[lq]s3.eu\-geo.objectstorage.service.networklayer.com\[rq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
EU Cross Region Private Endpoint
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[lq]s3.fra\-eu\-geo.objectstorage.service.networklayer.com\[rq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
EU Cross Region Frankfurt Private Endpoint
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[lq]s3.mil\-eu\-geo.objectstorage.service.networklayer.com\[rq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
EU Cross Region Milan Private Endpoint
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[lq]s3.ams\-eu\-geo.objectstorage.service.networklayer.com\[rq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
EU Cross Region Amsterdam Private Endpoint
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[lq]s3.eu\-gb.objectstorage.softlayer.net\[rq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Great Britain Endpoint
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[lq]s3.eu\-gb.objectstorage.service.networklayer.com\[rq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Great Britain Private Endpoint
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[lq]s3.ap\-geo.objectstorage.softlayer.net\[rq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
APAC Cross Regional Endpoint
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[lq]s3.tok\-ap\-geo.objectstorage.softlayer.net\[rq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
APAC Cross Regional Tokyo Endpoint
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[lq]s3.hkg\-ap\-geo.objectstorage.softlayer.net\[rq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
APAC Cross Regional HongKong Endpoint
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[lq]s3.seo\-ap\-geo.objectstorage.softlayer.net\[rq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
APAC Cross Regional Seoul Endpoint
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[lq]s3.ap\-geo.objectstorage.service.networklayer.com\[rq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
APAC Cross Regional Private Endpoint
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[lq]s3.tok\-ap\-geo.objectstorage.service.networklayer.com\[rq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
APAC Cross Regional Tokyo Private Endpoint
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[lq]s3.hkg\-ap\-geo.objectstorage.service.networklayer.com\[rq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
APAC Cross Regional HongKong Private Endpoint
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[lq]s3.seo\-ap\-geo.objectstorage.service.networklayer.com\[rq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
APAC Cross Regional Seoul Private Endpoint
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[lq]s3.mel01.objectstorage.softlayer.net\[rq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Melbourne Single Site Endpoint
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[lq]s3.mel01.objectstorage.service.networklayer.com\[rq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Melbourne Single Site Private Endpoint
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[lq]s3.tor01.objectstorage.softlayer.net\[rq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Toronto Single Site Endpoint
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[lq]s3.tor01.objectstorage.service.networklayer.com\[rq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Toronto Single Site Private Endpoint
|
|
.RE
|
|
.RE
|
|
.SS \[en]s3\-endpoint
|
|
.PP
|
|
Endpoint for OSS API.
|
|
.IP \[bu] 2
|
|
Config: endpoint
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_S3_ENDPOINT
|
|
.IP \[bu] 2
|
|
Type: string
|
|
.IP \[bu] 2
|
|
Default: \[dq]\[dq]
|
|
.IP \[bu] 2
|
|
Examples:
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
\[lq]oss\-cn\-hangzhou.aliyuncs.com\[rq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
East China 1 (Hangzhou)
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[lq]oss\-cn\-shanghai.aliyuncs.com\[rq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
East China 2 (Shanghai)
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[lq]oss\-cn\-qingdao.aliyuncs.com\[rq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
North China 1 (Qingdao)
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[lq]oss\-cn\-beijing.aliyuncs.com\[rq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
North China 2 (Beijing)
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[lq]oss\-cn\-zhangjiakou.aliyuncs.com\[rq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
North China 3 (Zhangjiakou)
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[lq]oss\-cn\-huhehaote.aliyuncs.com\[rq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
North China 5 (Huhehaote)
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[lq]oss\-cn\-shenzhen.aliyuncs.com\[rq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
South China 1 (Shenzhen)
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[lq]oss\-cn\-hongkong.aliyuncs.com\[rq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Hong Kong (Hong Kong)
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[lq]oss\-us\-west\-1.aliyuncs.com\[rq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
US West 1 (Silicon Valley)
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[lq]oss\-us\-east\-1.aliyuncs.com\[rq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
US East 1 (Virginia)
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[lq]oss\-ap\-southeast\-1.aliyuncs.com\[rq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Southeast Asia Southeast 1 (Singapore)
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[lq]oss\-ap\-southeast\-2.aliyuncs.com\[rq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Asia Pacific Southeast 2 (Sydney)
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[lq]oss\-ap\-southeast\-3.aliyuncs.com\[rq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Southeast Asia Southeast 3 (Kuala Lumpur)
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[lq]oss\-ap\-southeast\-5.aliyuncs.com\[rq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Asia Pacific Southeast 5 (Jakarta)
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[lq]oss\-ap\-northeast\-1.aliyuncs.com\[rq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Asia Pacific Northeast 1 (Japan)
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[lq]oss\-ap\-south\-1.aliyuncs.com\[rq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Asia Pacific South 1 (Mumbai)
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[lq]oss\-eu\-central\-1.aliyuncs.com\[rq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Central Europe 1 (Frankfurt)
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[lq]oss\-eu\-west\-1.aliyuncs.com\[rq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
West Europe (London)
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[lq]oss\-me\-east\-1.aliyuncs.com\[rq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Middle East 1 (Dubai)
|
|
.RE
|
|
.RE
|
|
.SS \[en]s3\-endpoint
|
|
.PP
|
|
Endpoint for StackPath Object Storage.
|
|
.IP \[bu] 2
|
|
Config: endpoint
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_S3_ENDPOINT
|
|
.IP \[bu] 2
|
|
Type: string
|
|
.IP \[bu] 2
|
|
Default: \[dq]\[dq]
|
|
.IP \[bu] 2
|
|
Examples:
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
\[lq]s3.us\-east\-2.stackpathstorage.com\[rq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
US East Endpoint
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[lq]s3.us\-west\-1.stackpathstorage.com\[rq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
US West Endpoint
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[lq]s3.eu\-central\-1.stackpathstorage.com\[rq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
EU Endpoint
|
|
.RE
|
|
.RE
|
|
.SS \[en]s3\-endpoint
|
|
.PP
|
|
Endpoint for S3 API.
|
|
Required when using an S3 clone.
|
|
.IP \[bu] 2
|
|
Config: endpoint
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_S3_ENDPOINT
|
|
.IP \[bu] 2
|
|
Type: string
|
|
.IP \[bu] 2
|
|
Default: \[dq]\[dq]
|
|
.IP \[bu] 2
|
|
Examples:
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
\[lq]objects\-us\-east\-1.dream.io\[rq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Dream Objects endpoint
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[lq]nyc3.digitaloceanspaces.com\[rq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Digital Ocean Spaces New York 3
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[lq]ams3.digitaloceanspaces.com\[rq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Digital Ocean Spaces Amsterdam 3
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[lq]sgp1.digitaloceanspaces.com\[rq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Digital Ocean Spaces Singapore 1
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[lq]s3.wasabisys.com\[rq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Wasabi US East endpoint
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[lq]s3.us\-west\-1.wasabisys.com\[rq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Wasabi US West endpoint
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[lq]s3.eu\-central\-1.wasabisys.com\[rq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Wasabi EU Central endpoint
|
|
.RE
|
|
.RE
|
|
.SS \[en]s3\-location\-constraint
|
|
.PP
|
|
Location constraint \- must be set to match the Region.
|
|
Used when creating buckets only.
|
|
.IP \[bu] 2
|
|
Config: location_constraint
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_S3_LOCATION_CONSTRAINT
|
|
.IP \[bu] 2
|
|
Type: string
|
|
.IP \[bu] 2
|
|
Default: \[dq]\[dq]
|
|
.IP \[bu] 2
|
|
Examples:
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
\[dq]\[dq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Empty for US Region, Northern Virginia or Pacific Northwest.
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[lq]us\-east\-2\[rq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
US East (Ohio) Region.
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[lq]us\-west\-2\[rq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
US West (Oregon) Region.
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[lq]us\-west\-1\[rq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
US West (Northern California) Region.
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[lq]ca\-central\-1\[rq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Canada (Central) Region.
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[lq]eu\-west\-1\[rq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
EU (Ireland) Region.
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[lq]eu\-west\-2\[rq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
EU (London) Region.
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[lq]eu\-north\-1\[rq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
EU (Stockholm) Region.
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[lq]EU\[rq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
EU Region.
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[lq]ap\-southeast\-1\[rq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Asia Pacific (Singapore) Region.
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[lq]ap\-southeast\-2\[rq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Asia Pacific (Sydney) Region.
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[lq]ap\-northeast\-1\[rq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Asia Pacific (Tokyo) Region.
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[lq]ap\-northeast\-2\[rq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Asia Pacific (Seoul)
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[lq]ap\-south\-1\[rq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Asia Pacific (Mumbai)
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[lq]ap\-east\-1\[rq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Asia Pacific (Hong Kong)
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[lq]sa\-east\-1\[rq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
South America (Sao Paulo) Region.
|
|
.RE
|
|
.RE
|
|
.SS \[en]s3\-location\-constraint
|
|
.PP
|
|
Location constraint \- must match endpoint when using IBM Cloud Public.
|
|
For on\-prem COS, do not make a selection from this list, hit enter
|
|
.IP \[bu] 2
|
|
Config: location_constraint
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_S3_LOCATION_CONSTRAINT
|
|
.IP \[bu] 2
|
|
Type: string
|
|
.IP \[bu] 2
|
|
Default: \[dq]\[dq]
|
|
.IP \[bu] 2
|
|
Examples:
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
\[lq]us\-standard\[rq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
US Cross Region Standard
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[lq]us\-vault\[rq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
US Cross Region Vault
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[lq]us\-cold\[rq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
US Cross Region Cold
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[lq]us\-flex\[rq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
US Cross Region Flex
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[lq]us\-east\-standard\[rq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
US East Region Standard
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[lq]us\-east\-vault\[rq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
US East Region Vault
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[lq]us\-east\-cold\[rq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
US East Region Cold
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[lq]us\-east\-flex\[rq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
US East Region Flex
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[lq]us\-south\-standard\[rq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
US South Region Standard
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[lq]us\-south\-vault\[rq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
US South Region Vault
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[lq]us\-south\-cold\[rq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
US South Region Cold
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[lq]us\-south\-flex\[rq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
US South Region Flex
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[lq]eu\-standard\[rq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
EU Cross Region Standard
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[lq]eu\-vault\[rq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
EU Cross Region Vault
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[lq]eu\-cold\[rq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
EU Cross Region Cold
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[lq]eu\-flex\[rq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
EU Cross Region Flex
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[lq]eu\-gb\-standard\[rq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Great Britain Standard
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[lq]eu\-gb\-vault\[rq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Great Britain Vault
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[lq]eu\-gb\-cold\[rq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Great Britain Cold
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[lq]eu\-gb\-flex\[rq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Great Britain Flex
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[lq]ap\-standard\[rq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
APAC Standard
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[lq]ap\-vault\[rq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
APAC Vault
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[lq]ap\-cold\[rq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
APAC Cold
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[lq]ap\-flex\[rq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
APAC Flex
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[lq]mel01\-standard\[rq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Melbourne Standard
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[lq]mel01\-vault\[rq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Melbourne Vault
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[lq]mel01\-cold\[rq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Melbourne Cold
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[lq]mel01\-flex\[rq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Melbourne Flex
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[lq]tor01\-standard\[rq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Toronto Standard
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[lq]tor01\-vault\[rq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Toronto Vault
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[lq]tor01\-cold\[rq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Toronto Cold
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[lq]tor01\-flex\[rq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Toronto Flex
|
|
.RE
|
|
.RE
|
|
.SS \[en]s3\-location\-constraint
|
|
.PP
|
|
Location constraint \- must be set to match the Region.
|
|
Leave blank if not sure.
|
|
Used when creating buckets only.
|
|
.IP \[bu] 2
|
|
Config: location_constraint
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_S3_LOCATION_CONSTRAINT
|
|
.IP \[bu] 2
|
|
Type: string
|
|
.IP \[bu] 2
|
|
Default: \[dq]\[dq]
|
|
.SS \[en]s3\-acl
|
|
.PP
|
|
Canned ACL used when creating buckets and storing or copying objects.
|
|
.PP
|
|
This ACL is used for creating objects and if bucket_acl isn\[cq]t set,
|
|
for creating buckets too.
|
|
.PP
|
|
For more info visit
|
|
https://docs.aws.amazon.com/AmazonS3/latest/dev/acl\-overview.html#canned\-acl
|
|
.PP
|
|
Note that this ACL is applied when server side copying objects as S3
|
|
doesn\[cq]t copy the ACL from the source but rather writes a fresh one.
|
|
.IP \[bu] 2
|
|
Config: acl
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_S3_ACL
|
|
.IP \[bu] 2
|
|
Type: string
|
|
.IP \[bu] 2
|
|
Default: \[dq]\[dq]
|
|
.IP \[bu] 2
|
|
Examples:
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
\[lq]private\[rq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Owner gets FULL_CONTROL.
|
|
No one else has access rights (default).
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[lq]public\-read\[rq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Owner gets FULL_CONTROL.
|
|
The AllUsers group gets READ access.
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[lq]public\-read\-write\[rq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Owner gets FULL_CONTROL.
|
|
The AllUsers group gets READ and WRITE access.
|
|
.IP \[bu] 2
|
|
Granting this on a bucket is generally not recommended.
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[lq]authenticated\-read\[rq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Owner gets FULL_CONTROL.
|
|
The AuthenticatedUsers group gets READ access.
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[lq]bucket\-owner\-read\[rq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Object owner gets FULL_CONTROL.
|
|
Bucket owner gets READ access.
|
|
.IP \[bu] 2
|
|
If you specify this canned ACL when creating a bucket, Amazon S3 ignores
|
|
it.
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[lq]bucket\-owner\-full\-control\[rq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Both the object owner and the bucket owner get FULL_CONTROL over the
|
|
object.
|
|
.IP \[bu] 2
|
|
If you specify this canned ACL when creating a bucket, Amazon S3 ignores
|
|
it.
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[lq]private\[rq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Owner gets FULL_CONTROL.
|
|
No one else has access rights (default).
|
|
This acl is available on IBM Cloud (Infra), IBM Cloud (Storage),
|
|
On\-Premise COS
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[lq]public\-read\[rq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Owner gets FULL_CONTROL.
|
|
The AllUsers group gets READ access.
|
|
This acl is available on IBM Cloud (Infra), IBM Cloud (Storage),
|
|
On\-Premise IBM COS
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[lq]public\-read\-write\[rq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Owner gets FULL_CONTROL.
|
|
The AllUsers group gets READ and WRITE access.
|
|
This acl is available on IBM Cloud (Infra), On\-Premise IBM COS
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[lq]authenticated\-read\[rq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Owner gets FULL_CONTROL.
|
|
The AuthenticatedUsers group gets READ access.
|
|
Not supported on Buckets.
|
|
This acl is available on IBM Cloud (Infra) and On\-Premise IBM COS
|
|
.RE
|
|
.RE
|
|
.SS \[en]s3\-server\-side\-encryption
|
|
.PP
|
|
The server\-side encryption algorithm used when storing this object in
|
|
S3.
|
|
.IP \[bu] 2
|
|
Config: server_side_encryption
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_S3_SERVER_SIDE_ENCRYPTION
|
|
.IP \[bu] 2
|
|
Type: string
|
|
.IP \[bu] 2
|
|
Default: \[dq]\[dq]
|
|
.IP \[bu] 2
|
|
Examples:
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
\[dq]\[dq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
None
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[lq]AES256\[rq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
AES256
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[lq]aws:kms\[rq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
aws:kms
|
|
.RE
|
|
.RE
|
|
.SS \[en]s3\-sse\-kms\-key\-id
|
|
.PP
|
|
If using KMS ID you must provide the ARN of Key.
|
|
.IP \[bu] 2
|
|
Config: sse_kms_key_id
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_S3_SSE_KMS_KEY_ID
|
|
.IP \[bu] 2
|
|
Type: string
|
|
.IP \[bu] 2
|
|
Default: \[dq]\[dq]
|
|
.IP \[bu] 2
|
|
Examples:
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
\[dq]\[dq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
None
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[dq]arn:aws:kms:us\-east\-1:*\[dq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
arn:aws:kms:*
|
|
.RE
|
|
.RE
|
|
.SS \[en]s3\-storage\-class
|
|
.PP
|
|
The storage class to use when storing new objects in S3.
|
|
.IP \[bu] 2
|
|
Config: storage_class
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_S3_STORAGE_CLASS
|
|
.IP \[bu] 2
|
|
Type: string
|
|
.IP \[bu] 2
|
|
Default: \[dq]\[dq]
|
|
.IP \[bu] 2
|
|
Examples:
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
\[dq]\[dq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Default
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[lq]STANDARD\[rq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Standard storage class
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[lq]REDUCED_REDUNDANCY\[rq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Reduced redundancy storage class
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[lq]STANDARD_IA\[rq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Standard Infrequent Access storage class
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[lq]ONEZONE_IA\[rq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
One Zone Infrequent Access storage class
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[lq]GLACIER\[rq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Glacier storage class
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[lq]DEEP_ARCHIVE\[rq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Glacier Deep Archive storage class
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[lq]INTELLIGENT_TIERING\[rq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Intelligent\-Tiering storage class
|
|
.RE
|
|
.RE
|
|
.SS \[en]s3\-storage\-class
|
|
.PP
|
|
The storage class to use when storing new objects in OSS.
|
|
.IP \[bu] 2
|
|
Config: storage_class
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_S3_STORAGE_CLASS
|
|
.IP \[bu] 2
|
|
Type: string
|
|
.IP \[bu] 2
|
|
Default: \[dq]\[dq]
|
|
.IP \[bu] 2
|
|
Examples:
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
\[dq]\[dq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Default
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[lq]STANDARD\[rq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Standard storage class
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[lq]GLACIER\[rq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Archive storage mode.
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[lq]STANDARD_IA\[rq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Infrequent access storage mode.
|
|
.RE
|
|
.RE
|
|
.SS Advanced Options
|
|
.PP
|
|
Here are the advanced options specific to s3 (Amazon S3 Compliant
|
|
Storage Provider (AWS, Alibaba, Ceph, Digital Ocean, Dreamhost, IBM COS,
|
|
Minio, etc)).
|
|
.SS \[en]s3\-bucket\-acl
|
|
.PP
|
|
Canned ACL used when creating buckets.
|
|
.PP
|
|
For more info visit
|
|
https://docs.aws.amazon.com/AmazonS3/latest/dev/acl\-overview.html#canned\-acl
|
|
.PP
|
|
Note that this ACL is applied when only when creating buckets.
|
|
If it isn\[cq]t set then \[lq]acl\[rq] is used instead.
|
|
.IP \[bu] 2
|
|
Config: bucket_acl
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_S3_BUCKET_ACL
|
|
.IP \[bu] 2
|
|
Type: string
|
|
.IP \[bu] 2
|
|
Default: \[dq]\[dq]
|
|
.IP \[bu] 2
|
|
Examples:
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
\[lq]private\[rq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Owner gets FULL_CONTROL.
|
|
No one else has access rights (default).
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[lq]public\-read\[rq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Owner gets FULL_CONTROL.
|
|
The AllUsers group gets READ access.
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[lq]public\-read\-write\[rq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Owner gets FULL_CONTROL.
|
|
The AllUsers group gets READ and WRITE access.
|
|
.IP \[bu] 2
|
|
Granting this on a bucket is generally not recommended.
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[lq]authenticated\-read\[rq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Owner gets FULL_CONTROL.
|
|
The AuthenticatedUsers group gets READ access.
|
|
.RE
|
|
.RE
|
|
.SS \[en]s3\-sse\-customer\-algorithm
|
|
.PP
|
|
If using SSE\-C, the server\-side encryption algorithm used when storing
|
|
this object in S3.
|
|
.IP \[bu] 2
|
|
Config: sse_customer_algorithm
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_S3_SSE_CUSTOMER_ALGORITHM
|
|
.IP \[bu] 2
|
|
Type: string
|
|
.IP \[bu] 2
|
|
Default: \[dq]\[dq]
|
|
.IP \[bu] 2
|
|
Examples:
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
\[dq]\[dq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
None
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[lq]AES256\[rq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
AES256
|
|
.RE
|
|
.RE
|
|
.SS \[en]s3\-sse\-customer\-key
|
|
.PP
|
|
If using SSE\-C you must provide the secret encryption key used to
|
|
encrypt/decrypt your data.
|
|
.IP \[bu] 2
|
|
Config: sse_customer_key
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_S3_SSE_CUSTOMER_KEY
|
|
.IP \[bu] 2
|
|
Type: string
|
|
.IP \[bu] 2
|
|
Default: \[dq]\[dq]
|
|
.IP \[bu] 2
|
|
Examples:
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
\[dq]\[dq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
None
|
|
.RE
|
|
.RE
|
|
.SS \[en]s3\-sse\-customer\-key\-md5
|
|
.PP
|
|
If using SSE\-C you must provide the secret encryption key MD5 checksum.
|
|
.IP \[bu] 2
|
|
Config: sse_customer_key_md5
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_S3_SSE_CUSTOMER_KEY_MD5
|
|
.IP \[bu] 2
|
|
Type: string
|
|
.IP \[bu] 2
|
|
Default: \[dq]\[dq]
|
|
.IP \[bu] 2
|
|
Examples:
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
\[dq]\[dq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
None
|
|
.RE
|
|
.RE
|
|
.SS \[en]s3\-upload\-cutoff
|
|
.PP
|
|
Cutoff for switching to chunked upload
|
|
.PP
|
|
Any files larger than this will be uploaded in chunks of chunk_size.
|
|
The minimum is 0 and the maximum is 5GB.
|
|
.IP \[bu] 2
|
|
Config: upload_cutoff
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_S3_UPLOAD_CUTOFF
|
|
.IP \[bu] 2
|
|
Type: SizeSuffix
|
|
.IP \[bu] 2
|
|
Default: 200M
|
|
.SS \[en]s3\-chunk\-size
|
|
.PP
|
|
Chunk size to use for uploading.
|
|
.PP
|
|
When uploading files larger than upload_cutoff or files with unknown
|
|
size (eg from \[lq]rclone rcat\[rq] or uploaded with \[lq]rclone
|
|
mount\[rq] or google photos or google docs) they will be uploaded as
|
|
multipart uploads using this chunk size.
|
|
.PP
|
|
Note that \[lq]\[en]s3\-upload\-concurrency\[rq] chunks of this size are
|
|
buffered in memory per transfer.
|
|
.PP
|
|
If you are transferring large files over high speed links and you have
|
|
enough memory, then increasing this will speed up the transfers.
|
|
.PP
|
|
Rclone will automatically increase the chunk size when uploading a large
|
|
file of known size to stay below the 10,000 chunks limit.
|
|
.PP
|
|
Files of unknown size are uploaded with the configured chunk_size.
|
|
Since the default chunk size is 5MB and there can be at most 10,000
|
|
chunks, this means that by default the maximum size of file you can
|
|
stream upload is 48GB.
|
|
If you wish to stream upload larger files then you will need to increase
|
|
chunk_size.
|
|
.IP \[bu] 2
|
|
Config: chunk_size
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_S3_CHUNK_SIZE
|
|
.IP \[bu] 2
|
|
Type: SizeSuffix
|
|
.IP \[bu] 2
|
|
Default: 5M
|
|
.SS \[en]s3\-copy\-cutoff
|
|
.PP
|
|
Cutoff for switching to multipart copy
|
|
.PP
|
|
Any files larger than this that need to be server side copied will be
|
|
copied in chunks of this size.
|
|
.PP
|
|
The minimum is 0 and the maximum is 5GB.
|
|
.IP \[bu] 2
|
|
Config: copy_cutoff
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_S3_COPY_CUTOFF
|
|
.IP \[bu] 2
|
|
Type: SizeSuffix
|
|
.IP \[bu] 2
|
|
Default: 5G
|
|
.SS \[en]s3\-disable\-checksum
|
|
.PP
|
|
Don\[cq]t store MD5 checksum with object metadata
|
|
.PP
|
|
Normally rclone will calculate the MD5 checksum of the input before
|
|
uploading it so it can add it to metadata on the object.
|
|
This is great for data integrity checking but can cause long delays for
|
|
large files to start uploading.
|
|
.IP \[bu] 2
|
|
Config: disable_checksum
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_S3_DISABLE_CHECKSUM
|
|
.IP \[bu] 2
|
|
Type: bool
|
|
.IP \[bu] 2
|
|
Default: false
|
|
.SS \[en]s3\-session\-token
|
|
.PP
|
|
An AWS session token
|
|
.IP \[bu] 2
|
|
Config: session_token
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_S3_SESSION_TOKEN
|
|
.IP \[bu] 2
|
|
Type: string
|
|
.IP \[bu] 2
|
|
Default: \[dq]\[dq]
|
|
.SS \[en]s3\-upload\-concurrency
|
|
.PP
|
|
Concurrency for multipart uploads.
|
|
.PP
|
|
This is the number of chunks of the same file that are uploaded
|
|
concurrently.
|
|
.PP
|
|
If you are uploading small numbers of large file over high speed link
|
|
and these uploads do not fully utilize your bandwidth, then increasing
|
|
this may help to speed up the transfers.
|
|
.IP \[bu] 2
|
|
Config: upload_concurrency
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_S3_UPLOAD_CONCURRENCY
|
|
.IP \[bu] 2
|
|
Type: int
|
|
.IP \[bu] 2
|
|
Default: 4
|
|
.SS \[en]s3\-force\-path\-style
|
|
.PP
|
|
If true use path style access if false use virtual hosted style.
|
|
.PP
|
|
If this is true (the default) then rclone will use path style access, if
|
|
false then rclone will use virtual path style.
|
|
See the AWS S3
|
|
docs (https://docs.aws.amazon.com/AmazonS3/latest/dev/UsingBucket.html#access-bucket-intro)
|
|
for more info.
|
|
.PP
|
|
Some providers (eg AWS, Aliyun OSS or Netease COS) require this set to
|
|
false \- rclone will do this automatically based on the provider
|
|
setting.
|
|
.IP \[bu] 2
|
|
Config: force_path_style
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_S3_FORCE_PATH_STYLE
|
|
.IP \[bu] 2
|
|
Type: bool
|
|
.IP \[bu] 2
|
|
Default: true
|
|
.SS \[en]s3\-v2\-auth
|
|
.PP
|
|
If true use v2 authentication.
|
|
.PP
|
|
If this is false (the default) then rclone will use v4 authentication.
|
|
If it is set then rclone will use v2 authentication.
|
|
.PP
|
|
Use this only if v4 signatures don\[cq]t work, eg pre Jewel/v10 CEPH.
|
|
.IP \[bu] 2
|
|
Config: v2_auth
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_S3_V2_AUTH
|
|
.IP \[bu] 2
|
|
Type: bool
|
|
.IP \[bu] 2
|
|
Default: false
|
|
.SS \[en]s3\-use\-accelerate\-endpoint
|
|
.PP
|
|
If true use the AWS S3 accelerated endpoint.
|
|
.PP
|
|
See: AWS S3 Transfer
|
|
acceleration (https://docs.aws.amazon.com/AmazonS3/latest/dev/transfer-acceleration-examples.html)
|
|
.IP \[bu] 2
|
|
Config: use_accelerate_endpoint
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_S3_USE_ACCELERATE_ENDPOINT
|
|
.IP \[bu] 2
|
|
Type: bool
|
|
.IP \[bu] 2
|
|
Default: false
|
|
.SS \[en]s3\-leave\-parts\-on\-error
|
|
.PP
|
|
If true avoid calling abort upload on a failure, leaving all
|
|
successfully uploaded parts on S3 for manual recovery.
|
|
.PP
|
|
It should be set to true for resuming uploads across different sessions.
|
|
.PP
|
|
WARNING: Storing parts of an incomplete multipart upload counts towards
|
|
space usage on S3 and will add additional costs if not cleaned up.
|
|
.IP \[bu] 2
|
|
Config: leave_parts_on_error
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_S3_LEAVE_PARTS_ON_ERROR
|
|
.IP \[bu] 2
|
|
Type: bool
|
|
.IP \[bu] 2
|
|
Default: false
|
|
.SS \[en]s3\-list\-chunk
|
|
.PP
|
|
Size of listing chunk (response list for each ListObject S3 request).
|
|
.PP
|
|
This option is also known as \[lq]MaxKeys\[rq], \[lq]max\-items\[rq], or
|
|
\[lq]page\-size\[rq] from the AWS S3 specification.
|
|
Most services truncate the response list to 1000 objects even if
|
|
requested more than that.
|
|
In AWS S3 this is a global maximum and cannot be changed, see AWS
|
|
S3 (https://docs.aws.amazon.com/cli/latest/reference/s3/ls.html).
|
|
In Ceph, this can be increased with the \[lq]rgw list buckets max
|
|
chunk\[rq] option.
|
|
.IP \[bu] 2
|
|
Config: list_chunk
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_S3_LIST_CHUNK
|
|
.IP \[bu] 2
|
|
Type: int
|
|
.IP \[bu] 2
|
|
Default: 1000
|
|
.SS \[en]s3\-encoding
|
|
.PP
|
|
This sets the encoding for the backend.
|
|
.PP
|
|
See: the encoding section in the
|
|
overview (https://rclone.org/overview/#encoding) for more info.
|
|
.IP \[bu] 2
|
|
Config: encoding
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_S3_ENCODING
|
|
.IP \[bu] 2
|
|
Type: MultiEncoder
|
|
.IP \[bu] 2
|
|
Default: Slash,InvalidUtf8,Dot
|
|
.SS \[en]s3\-memory\-pool\-flush\-time
|
|
.PP
|
|
How often internal memory buffer pools will be flushed.
|
|
Uploads which requires additional buffers (f.e multipart) will use
|
|
memory pool for allocations.
|
|
This option controls how often unused buffers will be removed from the
|
|
pool.
|
|
.IP \[bu] 2
|
|
Config: memory_pool_flush_time
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_S3_MEMORY_POOL_FLUSH_TIME
|
|
.IP \[bu] 2
|
|
Type: Duration
|
|
.IP \[bu] 2
|
|
Default: 1m0s
|
|
.SS \[en]s3\-memory\-pool\-use\-mmap
|
|
.PP
|
|
Whether to use mmap buffers in internal memory pool.
|
|
.IP \[bu] 2
|
|
Config: memory_pool_use_mmap
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_S3_MEMORY_POOL_USE_MMAP
|
|
.IP \[bu] 2
|
|
Type: bool
|
|
.IP \[bu] 2
|
|
Default: false
|
|
.SS Anonymous access to public buckets
|
|
.PP
|
|
If you want to use rclone to access a public bucket, configure with a
|
|
blank \f[C]access_key_id\f[R] and \f[C]secret_access_key\f[R].
|
|
Your config should end up looking like this:
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
[anons3]
|
|
type = s3
|
|
provider = AWS
|
|
env_auth = false
|
|
access_key_id =
|
|
secret_access_key =
|
|
region = us\-east\-1
|
|
endpoint =
|
|
location_constraint =
|
|
acl = private
|
|
server_side_encryption =
|
|
storage_class =
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
Then use it as normal with the name of the public bucket, eg
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone lsd anons3:1000genomes
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
You will be able to list and copy data but not upload it.
|
|
.SS Ceph
|
|
.PP
|
|
Ceph (https://ceph.com/) is an open source unified, distributed storage
|
|
system designed for excellent performance, reliability and scalability.
|
|
It has an S3 compatible object storage interface.
|
|
.PP
|
|
To use rclone with Ceph, configure as above but leave the region blank
|
|
and set the endpoint.
|
|
You should end up with something like this in your config:
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
[ceph]
|
|
type = s3
|
|
provider = Ceph
|
|
env_auth = false
|
|
access_key_id = XXX
|
|
secret_access_key = YYY
|
|
region =
|
|
endpoint = https://ceph.endpoint.example.com
|
|
location_constraint =
|
|
acl =
|
|
server_side_encryption =
|
|
storage_class =
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
If you are using an older version of CEPH, eg 10.2.x Jewel, then you may
|
|
need to supply the parameter \f[C]\-\-s3\-upload\-cutoff 0\f[R] or put
|
|
this in the config file as \f[C]upload_cutoff 0\f[R] to work around a
|
|
bug which causes uploading of small files to fail.
|
|
.PP
|
|
Note also that Ceph sometimes puts \f[C]/\f[R] in the passwords it gives
|
|
users.
|
|
If you read the secret access key using the command line tools you will
|
|
get a JSON blob with the \f[C]/\f[R] escaped as \f[C]\[rs]/\f[R].
|
|
Make sure you only write \f[C]/\f[R] in the secret access key.
|
|
.PP
|
|
Eg the dump from Ceph looks something like this (irrelevant keys
|
|
removed).
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
{
|
|
\[dq]user_id\[dq]: \[dq]xxx\[dq],
|
|
\[dq]display_name\[dq]: \[dq]xxxx\[dq],
|
|
\[dq]keys\[dq]: [
|
|
{
|
|
\[dq]user\[dq]: \[dq]xxx\[dq],
|
|
\[dq]access_key\[dq]: \[dq]xxxxxx\[dq],
|
|
\[dq]secret_key\[dq]: \[dq]xxxxxx\[rs]/xxxx\[dq]
|
|
}
|
|
],
|
|
}
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
Because this is a json dump, it is encoding the \f[C]/\f[R] as
|
|
\f[C]\[rs]/\f[R], so if you use the secret key as \f[C]xxxxxx/xxxx\f[R]
|
|
it will work fine.
|
|
.SS Dreamhost
|
|
.PP
|
|
Dreamhost DreamObjects (https://www.dreamhost.com/cloud/storage/) is an
|
|
object storage system based on CEPH.
|
|
.PP
|
|
To use rclone with Dreamhost, configure as above but leave the region
|
|
blank and set the endpoint.
|
|
You should end up with something like this in your config:
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
[dreamobjects]
|
|
type = s3
|
|
provider = DreamHost
|
|
env_auth = false
|
|
access_key_id = your_access_key
|
|
secret_access_key = your_secret_key
|
|
region =
|
|
endpoint = objects\-us\-west\-1.dream.io
|
|
location_constraint =
|
|
acl = private
|
|
server_side_encryption =
|
|
storage_class =
|
|
\f[R]
|
|
.fi
|
|
.SS DigitalOcean Spaces
|
|
.PP
|
|
Spaces (https://www.digitalocean.com/products/object-storage/) is an
|
|
S3\-interoperable (https://developers.digitalocean.com/documentation/spaces/)
|
|
object storage service from cloud provider DigitalOcean.
|
|
.PP
|
|
To connect to DigitalOcean Spaces you will need an access key and secret
|
|
key.
|
|
These can be retrieved on the \[lq]Applications &
|
|
API (https://cloud.digitalocean.com/settings/api/tokens)\[rq] page of
|
|
the DigitalOcean control panel.
|
|
They will be needed when prompted by \f[C]rclone config\f[R] for your
|
|
\f[C]access_key_id\f[R] and \f[C]secret_access_key\f[R].
|
|
.PP
|
|
When prompted for a \f[C]region\f[R] or \f[C]location_constraint\f[R],
|
|
press enter to use the default value.
|
|
The region must be included in the \f[C]endpoint\f[R] setting
|
|
(e.g.\ \f[C]nyc3.digitaloceanspaces.com\f[R]).
|
|
The default values can be used for other settings.
|
|
.PP
|
|
Going through the whole process of creating a new remote by running
|
|
\f[C]rclone config\f[R], each prompt should be answered as shown below:
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
Storage> s3
|
|
env_auth> 1
|
|
access_key_id> YOUR_ACCESS_KEY
|
|
secret_access_key> YOUR_SECRET_KEY
|
|
region>
|
|
endpoint> nyc3.digitaloceanspaces.com
|
|
location_constraint>
|
|
acl>
|
|
storage_class>
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
The resulting configuration file should look like:
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
[spaces]
|
|
type = s3
|
|
provider = DigitalOcean
|
|
env_auth = false
|
|
access_key_id = YOUR_ACCESS_KEY
|
|
secret_access_key = YOUR_SECRET_KEY
|
|
region =
|
|
endpoint = nyc3.digitaloceanspaces.com
|
|
location_constraint =
|
|
acl =
|
|
server_side_encryption =
|
|
storage_class =
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
Once configured, you can create a new Space and begin copying files.
|
|
For example:
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone mkdir spaces:my\-new\-space
|
|
rclone copy /path/to/files spaces:my\-new\-space
|
|
\f[R]
|
|
.fi
|
|
.SS IBM COS (S3)
|
|
.PP
|
|
Information stored with IBM Cloud Object Storage is encrypted and
|
|
dispersed across multiple geographic locations, and accessed through an
|
|
implementation of the S3 API.
|
|
This service makes use of the distributed storage technologies provided
|
|
by IBM\[cq]s Cloud Object Storage System (formerly Cleversafe).
|
|
For more information visit: (http://www.ibm.com/cloud/object\-storage)
|
|
.PP
|
|
To configure access to IBM COS S3, follow the steps below:
|
|
.IP "1." 3
|
|
Run rclone config and select n for a new remote.
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
2018/02/14 14:13:11 NOTICE: Config file \[dq]C:\[rs]\[rs]Users\[rs]\[rs]a\[rs]\[rs].config\[rs]\[rs]rclone\[rs]\[rs]rclone.conf\[dq] not found \- using defaults
|
|
No remotes found \- make a new one
|
|
n) New remote
|
|
s) Set configuration password
|
|
q) Quit config
|
|
n/s/q> n
|
|
\f[R]
|
|
.fi
|
|
.IP "2." 3
|
|
Enter the name for the configuration
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
name> <YOUR NAME>
|
|
\f[R]
|
|
.fi
|
|
.IP "3." 3
|
|
Select \[lq]s3\[rq] storage.
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
Choose a number from below, or type in your own value
|
|
1 / Alias for an existing remote
|
|
\[rs] \[dq]alias\[dq]
|
|
2 / Amazon Drive
|
|
\[rs] \[dq]amazon cloud drive\[dq]
|
|
3 / Amazon S3 Complaint Storage Providers (Dreamhost, Ceph, Minio, IBM COS)
|
|
\[rs] \[dq]s3\[dq]
|
|
4 / Backblaze B2
|
|
\[rs] \[dq]b2\[dq]
|
|
[snip]
|
|
23 / http Connection
|
|
\[rs] \[dq]http\[dq]
|
|
Storage> 3
|
|
\f[R]
|
|
.fi
|
|
.IP "4." 3
|
|
Select IBM COS as the S3 Storage Provider.
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
Choose the S3 provider.
|
|
Choose a number from below, or type in your own value
|
|
1 / Choose this option to configure Storage to AWS S3
|
|
\[rs] \[dq]AWS\[dq]
|
|
2 / Choose this option to configure Storage to Ceph Systems
|
|
\[rs] \[dq]Ceph\[dq]
|
|
3 / Choose this option to configure Storage to Dreamhost
|
|
\[rs] \[dq]Dreamhost\[dq]
|
|
4 / Choose this option to the configure Storage to IBM COS S3
|
|
\[rs] \[dq]IBMCOS\[dq]
|
|
5 / Choose this option to the configure Storage to Minio
|
|
\[rs] \[dq]Minio\[dq]
|
|
Provider>4
|
|
\f[R]
|
|
.fi
|
|
.IP "5." 3
|
|
Enter the Access Key and Secret.
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
AWS Access Key ID \- leave blank for anonymous access or runtime credentials.
|
|
access_key_id> <>
|
|
AWS Secret Access Key (password) \- leave blank for anonymous access or runtime credentials.
|
|
secret_access_key> <>
|
|
\f[R]
|
|
.fi
|
|
.IP "6." 3
|
|
Specify the endpoint for IBM COS.
|
|
For Public IBM COS, choose from the option below.
|
|
For On Premise IBM COS, enter an enpoint address.
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
Endpoint for IBM COS S3 API.
|
|
Specify if using an IBM COS On Premise.
|
|
Choose a number from below, or type in your own value
|
|
1 / US Cross Region Endpoint
|
|
\[rs] \[dq]s3\-api.us\-geo.objectstorage.softlayer.net\[dq]
|
|
2 / US Cross Region Dallas Endpoint
|
|
\[rs] \[dq]s3\-api.dal.us\-geo.objectstorage.softlayer.net\[dq]
|
|
3 / US Cross Region Washington DC Endpoint
|
|
\[rs] \[dq]s3\-api.wdc\-us\-geo.objectstorage.softlayer.net\[dq]
|
|
4 / US Cross Region San Jose Endpoint
|
|
\[rs] \[dq]s3\-api.sjc\-us\-geo.objectstorage.softlayer.net\[dq]
|
|
5 / US Cross Region Private Endpoint
|
|
\[rs] \[dq]s3\-api.us\-geo.objectstorage.service.networklayer.com\[dq]
|
|
6 / US Cross Region Dallas Private Endpoint
|
|
\[rs] \[dq]s3\-api.dal\-us\-geo.objectstorage.service.networklayer.com\[dq]
|
|
7 / US Cross Region Washington DC Private Endpoint
|
|
\[rs] \[dq]s3\-api.wdc\-us\-geo.objectstorage.service.networklayer.com\[dq]
|
|
8 / US Cross Region San Jose Private Endpoint
|
|
\[rs] \[dq]s3\-api.sjc\-us\-geo.objectstorage.service.networklayer.com\[dq]
|
|
9 / US Region East Endpoint
|
|
\[rs] \[dq]s3.us\-east.objectstorage.softlayer.net\[dq]
|
|
10 / US Region East Private Endpoint
|
|
\[rs] \[dq]s3.us\-east.objectstorage.service.networklayer.com\[dq]
|
|
11 / US Region South Endpoint
|
|
[snip]
|
|
34 / Toronto Single Site Private Endpoint
|
|
\[rs] \[dq]s3.tor01.objectstorage.service.networklayer.com\[dq]
|
|
endpoint>1
|
|
\f[R]
|
|
.fi
|
|
.IP "7." 3
|
|
Specify a IBM COS Location Constraint.
|
|
The location constraint must match endpoint when using IBM Cloud Public.
|
|
For on\-prem COS, do not make a selection from this list, hit enter
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
1 / US Cross Region Standard
|
|
\[rs] \[dq]us\-standard\[dq]
|
|
2 / US Cross Region Vault
|
|
\[rs] \[dq]us\-vault\[dq]
|
|
3 / US Cross Region Cold
|
|
\[rs] \[dq]us\-cold\[dq]
|
|
4 / US Cross Region Flex
|
|
\[rs] \[dq]us\-flex\[dq]
|
|
5 / US East Region Standard
|
|
\[rs] \[dq]us\-east\-standard\[dq]
|
|
6 / US East Region Vault
|
|
\[rs] \[dq]us\-east\-vault\[dq]
|
|
7 / US East Region Cold
|
|
\[rs] \[dq]us\-east\-cold\[dq]
|
|
8 / US East Region Flex
|
|
\[rs] \[dq]us\-east\-flex\[dq]
|
|
9 / US South Region Standard
|
|
\[rs] \[dq]us\-south\-standard\[dq]
|
|
10 / US South Region Vault
|
|
\[rs] \[dq]us\-south\-vault\[dq]
|
|
[snip]
|
|
32 / Toronto Flex
|
|
\[rs] \[dq]tor01\-flex\[dq]
|
|
location_constraint>1
|
|
\f[R]
|
|
.fi
|
|
.IP "9." 3
|
|
Specify a canned ACL.
|
|
IBM Cloud (Strorage) supports \[lq]public\-read\[rq] and
|
|
\[lq]private\[rq].
|
|
IBM Cloud(Infra) supports all the canned ACLs.
|
|
On\-Premise COS supports all the canned ACLs.
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
Canned ACL used when creating buckets and/or storing objects in S3.
|
|
For more info visit https://docs.aws.amazon.com/AmazonS3/latest/dev/acl\-overview.html#canned\-acl
|
|
Choose a number from below, or type in your own value
|
|
1 / Owner gets FULL_CONTROL. No one else has access rights (default). This acl is available on IBM Cloud (Infra), IBM Cloud (Storage), On\-Premise COS
|
|
\[rs] \[dq]private\[dq]
|
|
2 / Owner gets FULL_CONTROL. The AllUsers group gets READ access. This acl is available on IBM Cloud (Infra), IBM Cloud (Storage), On\-Premise IBM COS
|
|
\[rs] \[dq]public\-read\[dq]
|
|
3 / Owner gets FULL_CONTROL. The AllUsers group gets READ and WRITE access. This acl is available on IBM Cloud (Infra), On\-Premise IBM COS
|
|
\[rs] \[dq]public\-read\-write\[dq]
|
|
4 / Owner gets FULL_CONTROL. The AuthenticatedUsers group gets READ access. Not supported on Buckets. This acl is available on IBM Cloud (Infra) and On\-Premise IBM COS
|
|
\[rs] \[dq]authenticated\-read\[dq]
|
|
acl> 1
|
|
\f[R]
|
|
.fi
|
|
.IP "12." 4
|
|
Review the displayed configuration and accept to save the
|
|
\[lq]remote\[rq] then quit.
|
|
The config file should look like this
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
[xxx]
|
|
type = s3
|
|
Provider = IBMCOS
|
|
access_key_id = xxx
|
|
secret_access_key = yyy
|
|
endpoint = s3\-api.us\-geo.objectstorage.softlayer.net
|
|
location_constraint = us\-standard
|
|
acl = private
|
|
\f[R]
|
|
.fi
|
|
.IP "13." 4
|
|
Execute rclone commands
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
1) Create a bucket.
|
|
rclone mkdir IBM\-COS\-XREGION:newbucket
|
|
2) List available buckets.
|
|
rclone lsd IBM\-COS\-XREGION:
|
|
\-1 2017\-11\-08 21:16:22 \-1 test
|
|
\-1 2018\-02\-14 20:16:39 \-1 newbucket
|
|
3) List contents of a bucket.
|
|
rclone ls IBM\-COS\-XREGION:newbucket
|
|
18685952 test.exe
|
|
4) Copy a file from local to remote.
|
|
rclone copy /Users/file.txt IBM\-COS\-XREGION:newbucket
|
|
5) Copy a file from remote to local.
|
|
rclone copy IBM\-COS\-XREGION:newbucket/file.txt .
|
|
6) Delete a file on remote.
|
|
rclone delete IBM\-COS\-XREGION:newbucket/file.txt
|
|
\f[R]
|
|
.fi
|
|
.SS Minio
|
|
.PP
|
|
Minio (https://minio.io/) is an object storage server built for cloud
|
|
application developers and devops.
|
|
.PP
|
|
It is very easy to install and provides an S3 compatible server which
|
|
can be used by rclone.
|
|
.PP
|
|
To use it, install Minio following the instructions
|
|
here (https://docs.minio.io/docs/minio-quickstart-guide).
|
|
.PP
|
|
When it configures itself Minio will print something like this
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
Endpoint: http://192.168.1.106:9000 http://172.23.0.1:9000
|
|
AccessKey: USWUXHGYZQYFYFFIT3RE
|
|
SecretKey: MOJRH0mkL1IPauahWITSVvyDrQbEEIwljvmxdq03
|
|
Region: us\-east\-1
|
|
SQS ARNs: arn:minio:sqs:us\-east\-1:1:redis arn:minio:sqs:us\-east\-1:2:redis
|
|
|
|
Browser Access:
|
|
http://192.168.1.106:9000 http://172.23.0.1:9000
|
|
|
|
Command\-line Access: https://docs.minio.io/docs/minio\-client\-quickstart\-guide
|
|
$ mc config host add myminio http://192.168.1.106:9000 USWUXHGYZQYFYFFIT3RE MOJRH0mkL1IPauahWITSVvyDrQbEEIwljvmxdq03
|
|
|
|
Object API (Amazon S3 compatible):
|
|
Go: https://docs.minio.io/docs/golang\-client\-quickstart\-guide
|
|
Java: https://docs.minio.io/docs/java\-client\-quickstart\-guide
|
|
Python: https://docs.minio.io/docs/python\-client\-quickstart\-guide
|
|
JavaScript: https://docs.minio.io/docs/javascript\-client\-quickstart\-guide
|
|
.NET: https://docs.minio.io/docs/dotnet\-client\-quickstart\-guide
|
|
|
|
Drive Capacity: 26 GiB Free, 165 GiB Total
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
These details need to go into \f[C]rclone config\f[R] like this.
|
|
Note that it is important to put the region in as stated above.
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
env_auth> 1
|
|
access_key_id> USWUXHGYZQYFYFFIT3RE
|
|
secret_access_key> MOJRH0mkL1IPauahWITSVvyDrQbEEIwljvmxdq03
|
|
region> us\-east\-1
|
|
endpoint> http://192.168.1.106:9000
|
|
location_constraint>
|
|
server_side_encryption>
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
Which makes the config file look like this
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
[minio]
|
|
type = s3
|
|
provider = Minio
|
|
env_auth = false
|
|
access_key_id = USWUXHGYZQYFYFFIT3RE
|
|
secret_access_key = MOJRH0mkL1IPauahWITSVvyDrQbEEIwljvmxdq03
|
|
region = us\-east\-1
|
|
endpoint = http://192.168.1.106:9000
|
|
location_constraint =
|
|
server_side_encryption =
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
So once set up, for example to copy files into a bucket
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone copy /path/to/files minio:bucket
|
|
\f[R]
|
|
.fi
|
|
.SS Scaleway
|
|
.PP
|
|
Scaleway (https://www.scaleway.com/object-storage/) The Object Storage
|
|
platform allows you to store anything from backups, logs and web assets
|
|
to documents and photos.
|
|
Files can be dropped from the Scaleway console or transferred through
|
|
our API and CLI or using any S3\-compatible tool.
|
|
.PP
|
|
Scaleway provides an S3 interface which can be configured for use with
|
|
rclone like this:
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
[scaleway]
|
|
type = s3
|
|
env_auth = false
|
|
endpoint = s3.nl\-ams.scw.cloud
|
|
access_key_id = SCWXXXXXXXXXXXXXX
|
|
secret_access_key = 1111111\-2222\-3333\-44444\-55555555555555
|
|
region = nl\-ams
|
|
location_constraint =
|
|
acl = private
|
|
force_path_style = false
|
|
server_side_encryption =
|
|
storage_class =
|
|
\f[R]
|
|
.fi
|
|
.SS Wasabi
|
|
.PP
|
|
Wasabi (https://wasabi.com) is a cloud\-based object storage service for
|
|
a broad range of applications and use cases.
|
|
Wasabi is designed for individuals and organizations that require a
|
|
high\-performance, reliable, and secure data storage infrastructure at
|
|
minimal cost.
|
|
.PP
|
|
Wasabi provides an S3 interface which can be configured for use with
|
|
rclone like this.
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
No remotes found \- make a new one
|
|
n) New remote
|
|
s) Set configuration password
|
|
n/s> n
|
|
name> wasabi
|
|
Type of storage to configure.
|
|
Choose a number from below, or type in your own value
|
|
[snip]
|
|
XX / Amazon S3 (also Dreamhost, Ceph, Minio)
|
|
\[rs] \[dq]s3\[dq]
|
|
[snip]
|
|
Storage> s3
|
|
Get AWS credentials from runtime (environment variables or EC2/ECS meta data if no env vars). Only applies if access_key_id and secret_access_key is blank.
|
|
Choose a number from below, or type in your own value
|
|
1 / Enter AWS credentials in the next step
|
|
\[rs] \[dq]false\[dq]
|
|
2 / Get AWS credentials from the environment (env vars or IAM)
|
|
\[rs] \[dq]true\[dq]
|
|
env_auth> 1
|
|
AWS Access Key ID \- leave blank for anonymous access or runtime credentials.
|
|
access_key_id> YOURACCESSKEY
|
|
AWS Secret Access Key (password) \- leave blank for anonymous access or runtime credentials.
|
|
secret_access_key> YOURSECRETACCESSKEY
|
|
Region to connect to.
|
|
Choose a number from below, or type in your own value
|
|
/ The default endpoint \- a good choice if you are unsure.
|
|
1 | US Region, Northern Virginia or Pacific Northwest.
|
|
| Leave location constraint empty.
|
|
\[rs] \[dq]us\-east\-1\[dq]
|
|
[snip]
|
|
region> us\-east\-1
|
|
Endpoint for S3 API.
|
|
Leave blank if using AWS to use the default endpoint for the region.
|
|
Specify if using an S3 clone such as Ceph.
|
|
endpoint> s3.wasabisys.com
|
|
Location constraint \- must be set to match the Region. Used when creating buckets only.
|
|
Choose a number from below, or type in your own value
|
|
1 / Empty for US Region, Northern Virginia or Pacific Northwest.
|
|
\[rs] \[dq]\[dq]
|
|
[snip]
|
|
location_constraint>
|
|
Canned ACL used when creating buckets and/or storing objects in S3.
|
|
For more info visit https://docs.aws.amazon.com/AmazonS3/latest/dev/acl\-overview.html#canned\-acl
|
|
Choose a number from below, or type in your own value
|
|
1 / Owner gets FULL_CONTROL. No one else has access rights (default).
|
|
\[rs] \[dq]private\[dq]
|
|
[snip]
|
|
acl>
|
|
The server\-side encryption algorithm used when storing this object in S3.
|
|
Choose a number from below, or type in your own value
|
|
1 / None
|
|
\[rs] \[dq]\[dq]
|
|
2 / AES256
|
|
\[rs] \[dq]AES256\[dq]
|
|
server_side_encryption>
|
|
The storage class to use when storing objects in S3.
|
|
Choose a number from below, or type in your own value
|
|
1 / Default
|
|
\[rs] \[dq]\[dq]
|
|
2 / Standard storage class
|
|
\[rs] \[dq]STANDARD\[dq]
|
|
3 / Reduced redundancy storage class
|
|
\[rs] \[dq]REDUCED_REDUNDANCY\[dq]
|
|
4 / Standard Infrequent Access storage class
|
|
\[rs] \[dq]STANDARD_IA\[dq]
|
|
storage_class>
|
|
Remote config
|
|
\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-
|
|
[wasabi]
|
|
env_auth = false
|
|
access_key_id = YOURACCESSKEY
|
|
secret_access_key = YOURSECRETACCESSKEY
|
|
region = us\-east\-1
|
|
endpoint = s3.wasabisys.com
|
|
location_constraint =
|
|
acl =
|
|
server_side_encryption =
|
|
storage_class =
|
|
\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-
|
|
y) Yes this is OK
|
|
e) Edit this remote
|
|
d) Delete this remote
|
|
y/e/d> y
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
This will leave the config file looking like this.
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
[wasabi]
|
|
type = s3
|
|
provider = Wasabi
|
|
env_auth = false
|
|
access_key_id = YOURACCESSKEY
|
|
secret_access_key = YOURSECRETACCESSKEY
|
|
region =
|
|
endpoint = s3.wasabisys.com
|
|
location_constraint =
|
|
acl =
|
|
server_side_encryption =
|
|
storage_class =
|
|
\f[R]
|
|
.fi
|
|
.SS Alibaba OSS
|
|
.PP
|
|
Here is an example of making an Alibaba Cloud (Aliyun)
|
|
OSS (https://www.alibabacloud.com/product/oss/) configuration.
|
|
First run:
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone config
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
This will guide you through an interactive setup process.
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
No remotes found \- make a new one
|
|
n) New remote
|
|
s) Set configuration password
|
|
q) Quit config
|
|
n/s/q> n
|
|
name> oss
|
|
Type of storage to configure.
|
|
Enter a string value. Press Enter for the default (\[dq]\[dq]).
|
|
Choose a number from below, or type in your own value
|
|
[snip]
|
|
4 / Amazon S3 Compliant Storage Provider (AWS, Alibaba, Ceph, Digital Ocean, Dreamhost, IBM COS, Minio, etc)
|
|
\[rs] \[dq]s3\[dq]
|
|
[snip]
|
|
Storage> s3
|
|
Choose your S3 provider.
|
|
Enter a string value. Press Enter for the default (\[dq]\[dq]).
|
|
Choose a number from below, or type in your own value
|
|
1 / Amazon Web Services (AWS) S3
|
|
\[rs] \[dq]AWS\[dq]
|
|
2 / Alibaba Cloud Object Storage System (OSS) formerly Aliyun
|
|
\[rs] \[dq]Alibaba\[dq]
|
|
3 / Ceph Object Storage
|
|
\[rs] \[dq]Ceph\[dq]
|
|
[snip]
|
|
provider> Alibaba
|
|
Get AWS credentials from runtime (environment variables or EC2/ECS meta data if no env vars).
|
|
Only applies if access_key_id and secret_access_key is blank.
|
|
Enter a boolean value (true or false). Press Enter for the default (\[dq]false\[dq]).
|
|
Choose a number from below, or type in your own value
|
|
1 / Enter AWS credentials in the next step
|
|
\[rs] \[dq]false\[dq]
|
|
2 / Get AWS credentials from the environment (env vars or IAM)
|
|
\[rs] \[dq]true\[dq]
|
|
env_auth> 1
|
|
AWS Access Key ID.
|
|
Leave blank for anonymous access or runtime credentials.
|
|
Enter a string value. Press Enter for the default (\[dq]\[dq]).
|
|
access_key_id> accesskeyid
|
|
AWS Secret Access Key (password)
|
|
Leave blank for anonymous access or runtime credentials.
|
|
Enter a string value. Press Enter for the default (\[dq]\[dq]).
|
|
secret_access_key> secretaccesskey
|
|
Endpoint for OSS API.
|
|
Enter a string value. Press Enter for the default (\[dq]\[dq]).
|
|
Choose a number from below, or type in your own value
|
|
1 / East China 1 (Hangzhou)
|
|
\[rs] \[dq]oss\-cn\-hangzhou.aliyuncs.com\[dq]
|
|
2 / East China 2 (Shanghai)
|
|
\[rs] \[dq]oss\-cn\-shanghai.aliyuncs.com\[dq]
|
|
3 / North China 1 (Qingdao)
|
|
\[rs] \[dq]oss\-cn\-qingdao.aliyuncs.com\[dq]
|
|
[snip]
|
|
endpoint> 1
|
|
Canned ACL used when creating buckets and storing or copying objects.
|
|
|
|
Note that this ACL is applied when server side copying objects as S3
|
|
doesn\[aq]t copy the ACL from the source but rather writes a fresh one.
|
|
Enter a string value. Press Enter for the default (\[dq]\[dq]).
|
|
Choose a number from below, or type in your own value
|
|
1 / Owner gets FULL_CONTROL. No one else has access rights (default).
|
|
\[rs] \[dq]private\[dq]
|
|
2 / Owner gets FULL_CONTROL. The AllUsers group gets READ access.
|
|
\[rs] \[dq]public\-read\[dq]
|
|
/ Owner gets FULL_CONTROL. The AllUsers group gets READ and WRITE access.
|
|
[snip]
|
|
acl> 1
|
|
The storage class to use when storing new objects in OSS.
|
|
Enter a string value. Press Enter for the default (\[dq]\[dq]).
|
|
Choose a number from below, or type in your own value
|
|
1 / Default
|
|
\[rs] \[dq]\[dq]
|
|
2 / Standard storage class
|
|
\[rs] \[dq]STANDARD\[dq]
|
|
3 / Archive storage mode.
|
|
\[rs] \[dq]GLACIER\[dq]
|
|
4 / Infrequent access storage mode.
|
|
\[rs] \[dq]STANDARD_IA\[dq]
|
|
storage_class> 1
|
|
Edit advanced config? (y/n)
|
|
y) Yes
|
|
n) No
|
|
y/n> n
|
|
Remote config
|
|
\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-
|
|
[oss]
|
|
type = s3
|
|
provider = Alibaba
|
|
env_auth = false
|
|
access_key_id = accesskeyid
|
|
secret_access_key = secretaccesskey
|
|
endpoint = oss\-cn\-hangzhou.aliyuncs.com
|
|
acl = private
|
|
storage_class = Standard
|
|
\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-
|
|
y) Yes this is OK
|
|
e) Edit this remote
|
|
d) Delete this remote
|
|
y/e/d> y
|
|
\f[R]
|
|
.fi
|
|
.SS Netease NOS
|
|
.PP
|
|
For Netease NOS configure as per the configurator
|
|
\f[C]rclone config\f[R] setting the provider \f[C]Netease\f[R].
|
|
This will automatically set \f[C]force_path_style = false\f[R] which is
|
|
necessary for it to run properly.
|
|
.SS Backblaze B2
|
|
.PP
|
|
B2 is Backblaze\[cq]s cloud storage
|
|
system (https://www.backblaze.com/b2/).
|
|
.PP
|
|
Paths are specified as \f[C]remote:bucket\f[R] (or \f[C]remote:\f[R] for
|
|
the \f[C]lsd\f[R] command.) You may put subdirectories in too, eg
|
|
\f[C]remote:bucket/path/to/dir\f[R].
|
|
.PP
|
|
Here is an example of making a b2 configuration.
|
|
First run
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone config
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
This will guide you through an interactive setup process.
|
|
To authenticate you will either need your Account ID (a short hex
|
|
number) and Master Application Key (a long hex number) OR an Application
|
|
Key, which is the recommended method.
|
|
See below for further details on generating and using an Application
|
|
Key.
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
No remotes found \- make a new one
|
|
n) New remote
|
|
q) Quit config
|
|
n/q> n
|
|
name> remote
|
|
Type of storage to configure.
|
|
Choose a number from below, or type in your own value
|
|
[snip]
|
|
XX / Backblaze B2
|
|
\[rs] \[dq]b2\[dq]
|
|
[snip]
|
|
Storage> b2
|
|
Account ID or Application Key ID
|
|
account> 123456789abc
|
|
Application Key
|
|
key> 0123456789abcdef0123456789abcdef0123456789
|
|
Endpoint for the service \- leave blank normally.
|
|
endpoint>
|
|
Remote config
|
|
\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-
|
|
[remote]
|
|
account = 123456789abc
|
|
key = 0123456789abcdef0123456789abcdef0123456789
|
|
endpoint =
|
|
\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-
|
|
y) Yes this is OK
|
|
e) Edit this remote
|
|
d) Delete this remote
|
|
y/e/d> y
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
This remote is called \f[C]remote\f[R] and can now be used like this
|
|
.PP
|
|
See all buckets
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone lsd remote:
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
Create a new bucket
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone mkdir remote:bucket
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
List the contents of a bucket
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone ls remote:bucket
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
Sync \f[C]/home/local/directory\f[R] to the remote bucket, deleting any
|
|
excess files in the bucket.
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone sync /home/local/directory remote:bucket
|
|
\f[R]
|
|
.fi
|
|
.SS Application Keys
|
|
.PP
|
|
B2 supports multiple Application Keys for different access permission to
|
|
B2 Buckets (https://www.backblaze.com/b2/docs/application_keys.html).
|
|
.PP
|
|
You can use these with rclone too; you will need to use rclone version
|
|
1.43 or later.
|
|
.PP
|
|
Follow Backblaze\[cq]s docs to create an Application Key with the
|
|
required permission and add the \f[C]applicationKeyId\f[R] as the
|
|
\f[C]account\f[R] and the \f[C]Application Key\f[R] itself as the
|
|
\f[C]key\f[R].
|
|
.PP
|
|
Note that you must put the \f[I]applicationKeyId\f[R] as the
|
|
\f[C]account\f[R] \[en] you can\[cq]t use the master Account ID.
|
|
If you try then B2 will return 401 errors.
|
|
.SS \[en]fast\-list
|
|
.PP
|
|
This remote supports \f[C]\-\-fast\-list\f[R] which allows you to use
|
|
fewer transactions in exchange for more memory.
|
|
See the rclone docs (https://rclone.org/docs/#fast-list) for more
|
|
details.
|
|
.SS Modified time
|
|
.PP
|
|
The modified time is stored as metadata on the object as
|
|
\f[C]X\-Bz\-Info\-src_last_modified_millis\f[R] as milliseconds since
|
|
1970\-01\-01 in the Backblaze standard.
|
|
Other tools should be able to use this as a modified time.
|
|
.PP
|
|
Modified times are used in syncing and are fully supported.
|
|
Note that if a modification time needs to be updated on an object then
|
|
it will create a new version of the object.
|
|
.SS Restricted filename characters
|
|
.PP
|
|
In addition to the default restricted characters
|
|
set (https://rclone.org/overview/#restricted-characters) the following
|
|
characters are also replaced:
|
|
.PP
|
|
.TS
|
|
tab(@);
|
|
l c c.
|
|
T{
|
|
Character
|
|
T}@T{
|
|
Value
|
|
T}@T{
|
|
Replacement
|
|
T}
|
|
_
|
|
T{
|
|
\[rs]
|
|
T}@T{
|
|
0x5C
|
|
T}@T{
|
|
\[uFF3C]
|
|
T}
|
|
.TE
|
|
.PP
|
|
Invalid UTF\-8 bytes will also be
|
|
replaced (https://rclone.org/overview/#invalid-utf8), as they can\[cq]t
|
|
be used in JSON strings.
|
|
.SS SHA1 checksums
|
|
.PP
|
|
The SHA1 checksums of the files are checked on upload and download and
|
|
will be used in the syncing process.
|
|
.PP
|
|
Large files (bigger than the limit in \f[C]\-\-b2\-upload\-cutoff\f[R])
|
|
which are uploaded in chunks will store their SHA1 on the object as
|
|
\f[C]X\-Bz\-Info\-large_file_sha1\f[R] as recommended by Backblaze.
|
|
.PP
|
|
For a large file to be uploaded with an SHA1 checksum, the source needs
|
|
to support SHA1 checksums.
|
|
The local disk supports SHA1 checksums so large file transfers from
|
|
local disk will have an SHA1.
|
|
See the overview (https://rclone.org/overview/#features) for exactly
|
|
which remotes support SHA1.
|
|
.PP
|
|
Sources which don\[cq]t support SHA1, in particular \f[C]crypt\f[R] will
|
|
upload large files without SHA1 checksums.
|
|
This may be fixed in the future (see
|
|
#1767 (https://github.com/rclone/rclone/issues/1767)).
|
|
.PP
|
|
Files sizes below \f[C]\-\-b2\-upload\-cutoff\f[R] will always have an
|
|
SHA1 regardless of the source.
|
|
.SS Transfers
|
|
.PP
|
|
Backblaze recommends that you do lots of transfers simultaneously for
|
|
maximum speed.
|
|
In tests from my SSD equipped laptop the optimum setting is about
|
|
\f[C]\-\-transfers 32\f[R] though higher numbers may be used for a
|
|
slight speed improvement.
|
|
The optimum number for you may vary depending on your hardware, how big
|
|
the files are, how much you want to load your computer, etc.
|
|
The default of \f[C]\-\-transfers 4\f[R] is definitely too low for
|
|
Backblaze B2 though.
|
|
.PP
|
|
Note that uploading big files (bigger than 200 MB by default) will use a
|
|
96 MB RAM buffer by default.
|
|
There can be at most \f[C]\-\-transfers\f[R] of these in use at any
|
|
moment, so this sets the upper limit on the memory used.
|
|
.SS Versions
|
|
.PP
|
|
When rclone uploads a new version of a file it creates a new version of
|
|
it (https://www.backblaze.com/b2/docs/file_versions.html).
|
|
Likewise when you delete a file, the old version will be marked hidden
|
|
and still be available.
|
|
Conversely, you may opt in to a \[lq]hard delete\[rq] of files with the
|
|
\f[C]\-\-b2\-hard\-delete\f[R] flag which would permanently remove the
|
|
file instead of hiding it.
|
|
.PP
|
|
Old versions of files, where available, are visible using the
|
|
\f[C]\-\-b2\-versions\f[R] flag.
|
|
.PP
|
|
\f[B]NB\f[R] Note that \f[C]\-\-b2\-versions\f[R] does not work with
|
|
crypt at the moment
|
|
#1627 (https://github.com/rclone/rclone/issues/1627).
|
|
Using \[en]backup\-dir (https://rclone.org/docs/#backup-dir-dir) with
|
|
rclone is the recommended way of working around this.
|
|
.PP
|
|
If you wish to remove all the old versions then you can use the
|
|
\f[C]rclone cleanup remote:bucket\f[R] command which will delete all the
|
|
old versions of files, leaving the current ones intact.
|
|
You can also supply a path and only old versions under that path will be
|
|
deleted, eg \f[C]rclone cleanup remote:bucket/path/to/stuff\f[R].
|
|
.PP
|
|
Note that \f[C]cleanup\f[R] will remove partially uploaded files from
|
|
the bucket if they are more than a day old.
|
|
.PP
|
|
When you \f[C]purge\f[R] a bucket, the current and the old versions will
|
|
be deleted then the bucket will be deleted.
|
|
.PP
|
|
However \f[C]delete\f[R] will cause the current versions of the files to
|
|
become hidden old versions.
|
|
.PP
|
|
Here is a session showing the listing and retrieval of an old version
|
|
followed by a \f[C]cleanup\f[R] of the old versions.
|
|
.PP
|
|
Show current version and all the versions with
|
|
\f[C]\-\-b2\-versions\f[R] flag.
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
$ rclone \-q ls b2:cleanup\-test
|
|
9 one.txt
|
|
|
|
$ rclone \-q \-\-b2\-versions ls b2:cleanup\-test
|
|
9 one.txt
|
|
8 one\-v2016\-07\-04\-141032\-000.txt
|
|
16 one\-v2016\-07\-04\-141003\-000.txt
|
|
15 one\-v2016\-07\-02\-155621\-000.txt
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
Retrieve an old version
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
$ rclone \-q \-\-b2\-versions copy b2:cleanup\-test/one\-v2016\-07\-04\-141003\-000.txt /tmp
|
|
|
|
$ ls \-l /tmp/one\-v2016\-07\-04\-141003\-000.txt
|
|
\-rw\-rw\-r\-\- 1 ncw ncw 16 Jul 2 17:46 /tmp/one\-v2016\-07\-04\-141003\-000.txt
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
Clean up all the old versions and show that they\[cq]ve gone.
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
$ rclone \-q cleanup b2:cleanup\-test
|
|
|
|
$ rclone \-q ls b2:cleanup\-test
|
|
9 one.txt
|
|
|
|
$ rclone \-q \-\-b2\-versions ls b2:cleanup\-test
|
|
9 one.txt
|
|
\f[R]
|
|
.fi
|
|
.SS Data usage
|
|
.PP
|
|
It is useful to know how many requests are sent to the server in
|
|
different scenarios.
|
|
.PP
|
|
All copy commands send the following 4 requests:
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
/b2api/v1/b2_authorize_account
|
|
/b2api/v1/b2_create_bucket
|
|
/b2api/v1/b2_list_buckets
|
|
/b2api/v1/b2_list_file_names
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
The \f[C]b2_list_file_names\f[R] request will be sent once for every 1k
|
|
files in the remote path, providing the checksum and modification time
|
|
of the listed files.
|
|
As of version 1.33 issue
|
|
#818 (https://github.com/rclone/rclone/issues/818) causes extra requests
|
|
to be sent when using B2 with Crypt.
|
|
When a copy operation does not require any files to be uploaded, no more
|
|
requests will be sent.
|
|
.PP
|
|
Uploading files that do not require chunking, will send 2 requests per
|
|
file upload:
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
/b2api/v1/b2_get_upload_url
|
|
/b2api/v1/b2_upload_file/
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
Uploading files requiring chunking, will send 2 requests (one each to
|
|
start and finish the upload) and another 2 requests for each chunk:
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
/b2api/v1/b2_start_large_file
|
|
/b2api/v1/b2_get_upload_part_url
|
|
/b2api/v1/b2_upload_part/
|
|
/b2api/v1/b2_finish_large_file
|
|
\f[R]
|
|
.fi
|
|
.SS Versions
|
|
.PP
|
|
Versions can be viewed with the \f[C]\-\-b2\-versions\f[R] flag.
|
|
When it is set rclone will show and act on older versions of files.
|
|
For example
|
|
.PP
|
|
Listing without \f[C]\-\-b2\-versions\f[R]
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
$ rclone \-q ls b2:cleanup\-test
|
|
9 one.txt
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
And with
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
$ rclone \-q \-\-b2\-versions ls b2:cleanup\-test
|
|
9 one.txt
|
|
8 one\-v2016\-07\-04\-141032\-000.txt
|
|
16 one\-v2016\-07\-04\-141003\-000.txt
|
|
15 one\-v2016\-07\-02\-155621\-000.txt
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
Showing that the current version is unchanged but older versions can be
|
|
seen.
|
|
These have the UTC date that they were uploaded to the server to the
|
|
nearest millisecond appended to them.
|
|
.PP
|
|
Note that when using \f[C]\-\-b2\-versions\f[R] no file write operations
|
|
are permitted, so you can\[cq]t upload files or delete them.
|
|
.SS B2 and rclone link
|
|
.PP
|
|
Rclone supports generating file share links for private B2 buckets.
|
|
They can either be for a file for example:
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
\&./rclone link B2:bucket/path/to/file.txt
|
|
https://f002.backblazeb2.com/file/bucket/path/to/file.txt?Authorization=xxxxxxxx
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
or if run on a directory you will get:
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
\&./rclone link B2:bucket/path
|
|
https://f002.backblazeb2.com/file/bucket/path?Authorization=xxxxxxxx
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
you can then use the authorization token (the part of the url from the
|
|
\f[C]?Authorization=\f[R] on) on any file path under that directory.
|
|
For example:
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
https://f002.backblazeb2.com/file/bucket/path/to/file1?Authorization=xxxxxxxx
|
|
https://f002.backblazeb2.com/file/bucket/path/file2?Authorization=xxxxxxxx
|
|
https://f002.backblazeb2.com/file/bucket/path/folder/file3?Authorization=xxxxxxxx
|
|
\f[R]
|
|
.fi
|
|
.SS Standard Options
|
|
.PP
|
|
Here are the standard options specific to b2 (Backblaze B2).
|
|
.SS \[en]b2\-account
|
|
.PP
|
|
Account ID or Application Key ID
|
|
.IP \[bu] 2
|
|
Config: account
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_B2_ACCOUNT
|
|
.IP \[bu] 2
|
|
Type: string
|
|
.IP \[bu] 2
|
|
Default: \[dq]\[dq]
|
|
.SS \[en]b2\-key
|
|
.PP
|
|
Application Key
|
|
.IP \[bu] 2
|
|
Config: key
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_B2_KEY
|
|
.IP \[bu] 2
|
|
Type: string
|
|
.IP \[bu] 2
|
|
Default: \[dq]\[dq]
|
|
.SS \[en]b2\-hard\-delete
|
|
.PP
|
|
Permanently delete files on remote removal, otherwise hide files.
|
|
.IP \[bu] 2
|
|
Config: hard_delete
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_B2_HARD_DELETE
|
|
.IP \[bu] 2
|
|
Type: bool
|
|
.IP \[bu] 2
|
|
Default: false
|
|
.SS Advanced Options
|
|
.PP
|
|
Here are the advanced options specific to b2 (Backblaze B2).
|
|
.SS \[en]b2\-endpoint
|
|
.PP
|
|
Endpoint for the service.
|
|
Leave blank normally.
|
|
.IP \[bu] 2
|
|
Config: endpoint
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_B2_ENDPOINT
|
|
.IP \[bu] 2
|
|
Type: string
|
|
.IP \[bu] 2
|
|
Default: \[dq]\[dq]
|
|
.SS \[en]b2\-test\-mode
|
|
.PP
|
|
A flag string for X\-Bz\-Test\-Mode header for debugging.
|
|
.PP
|
|
This is for debugging purposes only.
|
|
Setting it to one of the strings below will cause b2 to return specific
|
|
errors:
|
|
.IP \[bu] 2
|
|
\[lq]fail_some_uploads\[rq]
|
|
.IP \[bu] 2
|
|
\[lq]expire_some_account_authorization_tokens\[rq]
|
|
.IP \[bu] 2
|
|
\[lq]force_cap_exceeded\[rq]
|
|
.PP
|
|
These will be set in the \[lq]X\-Bz\-Test\-Mode\[rq] header which is
|
|
documented in the b2 integrations
|
|
checklist (https://www.backblaze.com/b2/docs/integration_checklist.html).
|
|
.IP \[bu] 2
|
|
Config: test_mode
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_B2_TEST_MODE
|
|
.IP \[bu] 2
|
|
Type: string
|
|
.IP \[bu] 2
|
|
Default: \[dq]\[dq]
|
|
.SS \[en]b2\-versions
|
|
.PP
|
|
Include old versions in directory listings.
|
|
Note that when using this no file write operations are permitted, so you
|
|
can\[cq]t upload files or delete them.
|
|
.IP \[bu] 2
|
|
Config: versions
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_B2_VERSIONS
|
|
.IP \[bu] 2
|
|
Type: bool
|
|
.IP \[bu] 2
|
|
Default: false
|
|
.SS \[en]b2\-upload\-cutoff
|
|
.PP
|
|
Cutoff for switching to chunked upload.
|
|
.PP
|
|
Files above this size will be uploaded in chunks of
|
|
\[lq]\[en]b2\-chunk\-size\[rq].
|
|
.PP
|
|
This value should be set no larger than 4.657GiB (== 5GB).
|
|
.IP \[bu] 2
|
|
Config: upload_cutoff
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_B2_UPLOAD_CUTOFF
|
|
.IP \[bu] 2
|
|
Type: SizeSuffix
|
|
.IP \[bu] 2
|
|
Default: 200M
|
|
.SS \[en]b2\-chunk\-size
|
|
.PP
|
|
Upload chunk size.
|
|
Must fit in memory.
|
|
.PP
|
|
When uploading large files, chunk the file into this size.
|
|
Note that these chunks are buffered in memory and there might a maximum
|
|
of \[lq]\[en]transfers\[rq] chunks in progress at once.
|
|
5,000,000 Bytes is the minimum size.
|
|
.IP \[bu] 2
|
|
Config: chunk_size
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_B2_CHUNK_SIZE
|
|
.IP \[bu] 2
|
|
Type: SizeSuffix
|
|
.IP \[bu] 2
|
|
Default: 96M
|
|
.SS \[en]b2\-disable\-checksum
|
|
.PP
|
|
Disable checksums for large (> upload cutoff) files
|
|
.PP
|
|
Normally rclone will calculate the SHA1 checksum of the input before
|
|
uploading it so it can add it to metadata on the object.
|
|
This is great for data integrity checking but can cause long delays for
|
|
large files to start uploading.
|
|
.IP \[bu] 2
|
|
Config: disable_checksum
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_B2_DISABLE_CHECKSUM
|
|
.IP \[bu] 2
|
|
Type: bool
|
|
.IP \[bu] 2
|
|
Default: false
|
|
.SS \[en]b2\-download\-url
|
|
.PP
|
|
Custom endpoint for downloads.
|
|
.PP
|
|
This is usually set to a Cloudflare CDN URL as Backblaze offers free
|
|
egress for data downloaded through the Cloudflare network.
|
|
This is probably only useful for a public bucket.
|
|
Leave blank if you want to use the endpoint provided by Backblaze.
|
|
.IP \[bu] 2
|
|
Config: download_url
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_B2_DOWNLOAD_URL
|
|
.IP \[bu] 2
|
|
Type: string
|
|
.IP \[bu] 2
|
|
Default: \[dq]\[dq]
|
|
.SS \[en]b2\-download\-auth\-duration
|
|
.PP
|
|
Time before the authorization token will expire in s or suffix
|
|
ms|s|m|h|d.
|
|
.PP
|
|
The duration before the download authorization token will expire.
|
|
The minimum value is 1 second.
|
|
The maximum value is one week.
|
|
.IP \[bu] 2
|
|
Config: download_auth_duration
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_B2_DOWNLOAD_AUTH_DURATION
|
|
.IP \[bu] 2
|
|
Type: Duration
|
|
.IP \[bu] 2
|
|
Default: 1w
|
|
.SS \[en]b2\-encoding
|
|
.PP
|
|
This sets the encoding for the backend.
|
|
.PP
|
|
See: the encoding section in the
|
|
overview (https://rclone.org/overview/#encoding) for more info.
|
|
.IP \[bu] 2
|
|
Config: encoding
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_B2_ENCODING
|
|
.IP \[bu] 2
|
|
Type: MultiEncoder
|
|
.IP \[bu] 2
|
|
Default: Slash,BackSlash,Del,Ctl,InvalidUtf8,Dot
|
|
.SS Box
|
|
.PP
|
|
Paths are specified as \f[C]remote:path\f[R]
|
|
.PP
|
|
Paths may be as deep as required, eg
|
|
\f[C]remote:directory/subdirectory\f[R].
|
|
.PP
|
|
The initial setup for Box involves getting a token from Box which you
|
|
can do either in your browser, or with a config.json downloaded from Box
|
|
to use JWT authentication.
|
|
\f[C]rclone config\f[R] walks you through it.
|
|
.PP
|
|
Here is an example of how to make a remote called \f[C]remote\f[R].
|
|
First run:
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone config
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
This will guide you through an interactive setup process:
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
No remotes found \- make a new one
|
|
n) New remote
|
|
s) Set configuration password
|
|
q) Quit config
|
|
n/s/q> n
|
|
name> remote
|
|
Type of storage to configure.
|
|
Choose a number from below, or type in your own value
|
|
[snip]
|
|
XX / Box
|
|
\[rs] \[dq]box\[dq]
|
|
[snip]
|
|
Storage> box
|
|
Box App Client Id \- leave blank normally.
|
|
client_id>
|
|
Box App Client Secret \- leave blank normally.
|
|
client_secret>
|
|
Box App config.json location
|
|
Leave blank normally.
|
|
Enter a string value. Press Enter for the default (\[dq]\[dq]).
|
|
config_json>
|
|
\[aq]enterprise\[aq] or \[aq]user\[aq] depending on the type of token being requested.
|
|
Enter a string value. Press Enter for the default (\[dq]user\[dq]).
|
|
box_sub_type>
|
|
Remote config
|
|
Use auto config?
|
|
* Say Y if not sure
|
|
* Say N if you are working on a remote or headless machine
|
|
y) Yes
|
|
n) No
|
|
y/n> y
|
|
If your browser doesn\[aq]t open automatically go to the following link: http://127.0.0.1:53682/auth
|
|
Log in and authorize rclone for access
|
|
Waiting for code...
|
|
Got code
|
|
\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-
|
|
[remote]
|
|
client_id =
|
|
client_secret =
|
|
token = {\[dq]access_token\[dq]:\[dq]XXX\[dq],\[dq]token_type\[dq]:\[dq]bearer\[dq],\[dq]refresh_token\[dq]:\[dq]XXX\[dq],\[dq]expiry\[dq]:\[dq]XXX\[dq]}
|
|
\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-
|
|
y) Yes this is OK
|
|
e) Edit this remote
|
|
d) Delete this remote
|
|
y/e/d> y
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
See the remote setup docs (https://rclone.org/remote_setup/) for how to
|
|
set it up on a machine with no Internet browser available.
|
|
.PP
|
|
Note that rclone runs a webserver on your local machine to collect the
|
|
token as returned from Box.
|
|
This only runs from the moment it opens your browser to the moment you
|
|
get back the verification code.
|
|
This is on \f[C]http://127.0.0.1:53682/\f[R] and this it may require you
|
|
to unblock it temporarily if you are running a host firewall.
|
|
.PP
|
|
Once configured you can then use \f[C]rclone\f[R] like this,
|
|
.PP
|
|
List directories in top level of your Box
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone lsd remote:
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
List all the files in your Box
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone ls remote:
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
To copy a local directory to an Box directory called backup
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone copy /home/source remote:backup
|
|
\f[R]
|
|
.fi
|
|
.SS Using rclone with an Enterprise account with SSO
|
|
.PP
|
|
If you have an \[lq]Enterprise\[rq] account type with Box with single
|
|
sign on (SSO), you need to create a password to use Box with rclone.
|
|
This can be done at your Enterprise Box account by going to Settings,
|
|
\[lq]Account\[rq] Tab, and then set the password in the
|
|
\[lq]Authentication\[rq] field.
|
|
.PP
|
|
Once you have done this, you can setup your Enterprise Box account using
|
|
the same procedure detailed above in the, using the password you have
|
|
just set.
|
|
.SS Invalid refresh token
|
|
.PP
|
|
According to the box
|
|
docs (https://developer.box.com/v2.0/docs/oauth-20#section-6-using-the-access-and-refresh-tokens):
|
|
.RS
|
|
.PP
|
|
Each refresh_token is valid for one use in 60 days.
|
|
.RE
|
|
.PP
|
|
This means that if you
|
|
.IP \[bu] 2
|
|
Don\[cq]t use the box remote for 60 days
|
|
.IP \[bu] 2
|
|
Copy the config file with a box refresh token in and use it in two
|
|
places
|
|
.IP \[bu] 2
|
|
Get an error on a token refresh
|
|
.PP
|
|
then rclone will return an error which includes the text
|
|
\f[C]Invalid refresh token\f[R].
|
|
.PP
|
|
To fix this you will need to use oauth2 again to update the refresh
|
|
token.
|
|
You can use the methods in the remote setup
|
|
docs (https://rclone.org/remote_setup/), bearing in mind that if you use
|
|
the copy the config file method, you should not use that remote on the
|
|
computer you did the authentication on.
|
|
.PP
|
|
Here is how to do it.
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
$ rclone config
|
|
Current remotes:
|
|
|
|
Name Type
|
|
==== ====
|
|
remote box
|
|
|
|
e) Edit existing remote
|
|
n) New remote
|
|
d) Delete remote
|
|
r) Rename remote
|
|
c) Copy remote
|
|
s) Set configuration password
|
|
q) Quit config
|
|
e/n/d/r/c/s/q> e
|
|
Choose a number from below, or type in an existing value
|
|
1 > remote
|
|
remote> remote
|
|
\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-
|
|
[remote]
|
|
type = box
|
|
token = {\[dq]access_token\[dq]:\[dq]XXX\[dq],\[dq]token_type\[dq]:\[dq]bearer\[dq],\[dq]refresh_token\[dq]:\[dq]XXX\[dq],\[dq]expiry\[dq]:\[dq]2017\-07\-08T23:40:08.059167677+01:00\[dq]}
|
|
\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-
|
|
Edit remote
|
|
Value \[dq]client_id\[dq] = \[dq]\[dq]
|
|
Edit? (y/n)>
|
|
y) Yes
|
|
n) No
|
|
y/n> n
|
|
Value \[dq]client_secret\[dq] = \[dq]\[dq]
|
|
Edit? (y/n)>
|
|
y) Yes
|
|
n) No
|
|
y/n> n
|
|
Remote config
|
|
Already have a token \- refresh?
|
|
y) Yes
|
|
n) No
|
|
y/n> y
|
|
Use auto config?
|
|
* Say Y if not sure
|
|
* Say N if you are working on a remote or headless machine
|
|
y) Yes
|
|
n) No
|
|
y/n> y
|
|
If your browser doesn\[aq]t open automatically go to the following link: http://127.0.0.1:53682/auth
|
|
Log in and authorize rclone for access
|
|
Waiting for code...
|
|
Got code
|
|
\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-
|
|
[remote]
|
|
type = box
|
|
token = {\[dq]access_token\[dq]:\[dq]YYY\[dq],\[dq]token_type\[dq]:\[dq]bearer\[dq],\[dq]refresh_token\[dq]:\[dq]YYY\[dq],\[dq]expiry\[dq]:\[dq]2017\-07\-23T12:22:29.259137901+01:00\[dq]}
|
|
\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-
|
|
y) Yes this is OK
|
|
e) Edit this remote
|
|
d) Delete this remote
|
|
y/e/d> y
|
|
\f[R]
|
|
.fi
|
|
.SS Modified time and hashes
|
|
.PP
|
|
Box allows modification times to be set on objects accurate to 1 second.
|
|
These will be used to detect whether objects need syncing or not.
|
|
.PP
|
|
Box supports SHA1 type hashes, so you can use the \f[C]\-\-checksum\f[R]
|
|
flag.
|
|
.SS Restricted filename characters
|
|
.PP
|
|
In addition to the default restricted characters
|
|
set (https://rclone.org/overview/#restricted-characters) the following
|
|
characters are also replaced:
|
|
.PP
|
|
.TS
|
|
tab(@);
|
|
l c c.
|
|
T{
|
|
Character
|
|
T}@T{
|
|
Value
|
|
T}@T{
|
|
Replacement
|
|
T}
|
|
_
|
|
T{
|
|
\[rs]
|
|
T}@T{
|
|
0x5C
|
|
T}@T{
|
|
\[uFF3C]
|
|
T}
|
|
.TE
|
|
.PP
|
|
File names can also not end with the following characters.
|
|
These only get replaced if they are the last character in the name:
|
|
.PP
|
|
.TS
|
|
tab(@);
|
|
l c c.
|
|
T{
|
|
Character
|
|
T}@T{
|
|
Value
|
|
T}@T{
|
|
Replacement
|
|
T}
|
|
_
|
|
T{
|
|
SP
|
|
T}@T{
|
|
0x20
|
|
T}@T{
|
|
\[u2420]
|
|
T}
|
|
.TE
|
|
.PP
|
|
Invalid UTF\-8 bytes will also be
|
|
replaced (https://rclone.org/overview/#invalid-utf8), as they can\[cq]t
|
|
be used in JSON strings.
|
|
.SS Transfers
|
|
.PP
|
|
For files above 50MB rclone will use a chunked transfer.
|
|
Rclone will upload up to \f[C]\-\-transfers\f[R] chunks at the same time
|
|
(shared among all the multipart uploads).
|
|
Chunks are buffered in memory and are normally 8MB so increasing
|
|
\f[C]\-\-transfers\f[R] will increase memory use.
|
|
.SS Deleting files
|
|
.PP
|
|
Depending on the enterprise settings for your user, the item will either
|
|
be actually deleted from Box or moved to the trash.
|
|
.SS Root folder ID
|
|
.PP
|
|
You can set the \f[C]root_folder_id\f[R] for rclone.
|
|
This is the directory (identified by its \f[C]Folder ID\f[R]) that
|
|
rclone considers to be the root of your Box drive.
|
|
.PP
|
|
Normally you will leave this blank and rclone will determine the correct
|
|
root to use itself.
|
|
.PP
|
|
However you can set this to restrict rclone to a specific folder
|
|
hierarchy.
|
|
.PP
|
|
In order to do this you will have to find the \f[C]Folder ID\f[R] of the
|
|
directory you wish rclone to display.
|
|
This will be the last segment of the URL when you open the relevant
|
|
folder in the Box web interface.
|
|
.PP
|
|
So if the folder you want rclone to use has a URL which looks like
|
|
\f[C]https://app.box.com/folder/11xxxxxxxxx8\f[R] in the browser, then
|
|
you use \f[C]11xxxxxxxxx8\f[R] as the \f[C]root_folder_id\f[R] in the
|
|
config.
|
|
.SS Standard Options
|
|
.PP
|
|
Here are the standard options specific to box (Box).
|
|
.SS \[en]box\-client\-id
|
|
.PP
|
|
Box App Client Id.
|
|
Leave blank normally.
|
|
.IP \[bu] 2
|
|
Config: client_id
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_BOX_CLIENT_ID
|
|
.IP \[bu] 2
|
|
Type: string
|
|
.IP \[bu] 2
|
|
Default: \[dq]\[dq]
|
|
.SS \[en]box\-client\-secret
|
|
.PP
|
|
Box App Client Secret Leave blank normally.
|
|
.IP \[bu] 2
|
|
Config: client_secret
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_BOX_CLIENT_SECRET
|
|
.IP \[bu] 2
|
|
Type: string
|
|
.IP \[bu] 2
|
|
Default: \[dq]\[dq]
|
|
.SS \[en]box\-box\-config\-file
|
|
.PP
|
|
Box App config.json location Leave blank normally.
|
|
.IP \[bu] 2
|
|
Config: box_config_file
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_BOX_BOX_CONFIG_FILE
|
|
.IP \[bu] 2
|
|
Type: string
|
|
.IP \[bu] 2
|
|
Default: \[dq]\[dq]
|
|
.SS \[en]box\-box\-sub\-type
|
|
.IP \[bu] 2
|
|
Config: box_sub_type
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_BOX_BOX_SUB_TYPE
|
|
.IP \[bu] 2
|
|
Type: string
|
|
.IP \[bu] 2
|
|
Default: \[lq]user\[rq]
|
|
.IP \[bu] 2
|
|
Examples:
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
\[lq]user\[rq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Rclone should act on behalf of a user
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[lq]enterprise\[rq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Rclone should act on behalf of a service account
|
|
.RE
|
|
.RE
|
|
.SS Advanced Options
|
|
.PP
|
|
Here are the advanced options specific to box (Box).
|
|
.SS \[en]box\-root\-folder\-id
|
|
.PP
|
|
Fill in for rclone to use a non root folder as its starting point.
|
|
.IP \[bu] 2
|
|
Config: root_folder_id
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_BOX_ROOT_FOLDER_ID
|
|
.IP \[bu] 2
|
|
Type: string
|
|
.IP \[bu] 2
|
|
Default: \[lq]0\[rq]
|
|
.SS \[en]box\-upload\-cutoff
|
|
.PP
|
|
Cutoff for switching to multipart upload (>= 50MB).
|
|
.IP \[bu] 2
|
|
Config: upload_cutoff
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_BOX_UPLOAD_CUTOFF
|
|
.IP \[bu] 2
|
|
Type: SizeSuffix
|
|
.IP \[bu] 2
|
|
Default: 50M
|
|
.SS \[en]box\-commit\-retries
|
|
.PP
|
|
Max number of times to try committing a multipart file.
|
|
.IP \[bu] 2
|
|
Config: commit_retries
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_BOX_COMMIT_RETRIES
|
|
.IP \[bu] 2
|
|
Type: int
|
|
.IP \[bu] 2
|
|
Default: 100
|
|
.SS \[en]box\-encoding
|
|
.PP
|
|
This sets the encoding for the backend.
|
|
.PP
|
|
See: the encoding section in the
|
|
overview (https://rclone.org/overview/#encoding) for more info.
|
|
.IP \[bu] 2
|
|
Config: encoding
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_BOX_ENCODING
|
|
.IP \[bu] 2
|
|
Type: MultiEncoder
|
|
.IP \[bu] 2
|
|
Default: Slash,BackSlash,Del,Ctl,RightSpace,InvalidUtf8,Dot
|
|
.SS Limitations
|
|
.PP
|
|
Note that Box is case insensitive so you can\[cq]t have a file called
|
|
\[lq]Hello.doc\[rq] and one called \[lq]hello.doc\[rq].
|
|
.PP
|
|
Box file names can\[cq]t have the \f[C]\[rs]\f[R] character in.
|
|
rclone maps this to and from an identical looking unicode equivalent
|
|
\f[C]\[uFF3C]\f[R].
|
|
.PP
|
|
Box only supports filenames up to 255 characters in length.
|
|
.SS Cache (BETA)
|
|
.PP
|
|
The \f[C]cache\f[R] remote wraps another existing remote and stores file
|
|
structure and its data for long running tasks like
|
|
\f[C]rclone mount\f[R].
|
|
.SS Status
|
|
.PP
|
|
The cache backend code is working but it currently doesn\[cq]t have a
|
|
maintainer so there are outstanding
|
|
bugs (https://github.com/rclone/rclone/issues?q=is%3Aopen+is%3Aissue+label%3Abug+label%3A%22Remote%3A+Cache%22)
|
|
which aren\[cq]t getting fixed.
|
|
.PP
|
|
The cache backend is due to be phased out in favour of the VFS caching
|
|
layer eventually which is more tightly integrated into rclone.
|
|
.PP
|
|
Until this happens we recommend only using the cache backend if you find
|
|
you can\[cq]t work without it.
|
|
There are many docs online describing the use of the cache backend to
|
|
minimize API hits and by\-and\-large these are out of date and the cache
|
|
backend isn\[cq]t needed in those scenarios any more.
|
|
.SS Setup
|
|
.PP
|
|
To get started you just need to have an existing remote which can be
|
|
configured with \f[C]cache\f[R].
|
|
.PP
|
|
Here is an example of how to make a remote called \f[C]test\-cache\f[R].
|
|
First run:
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone config
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
This will guide you through an interactive setup process:
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
No remotes found \- make a new one
|
|
n) New remote
|
|
r) Rename remote
|
|
c) Copy remote
|
|
s) Set configuration password
|
|
q) Quit config
|
|
n/r/c/s/q> n
|
|
name> test\-cache
|
|
Type of storage to configure.
|
|
Choose a number from below, or type in your own value
|
|
[snip]
|
|
XX / Cache a remote
|
|
\[rs] \[dq]cache\[dq]
|
|
[snip]
|
|
Storage> cache
|
|
Remote to cache.
|
|
Normally should contain a \[aq]:\[aq] and a path, eg \[dq]myremote:path/to/dir\[dq],
|
|
\[dq]myremote:bucket\[dq] or maybe \[dq]myremote:\[dq] (not recommended).
|
|
remote> local:/test
|
|
Optional: The URL of the Plex server
|
|
plex_url> http://127.0.0.1:32400
|
|
Optional: The username of the Plex user
|
|
plex_username> dummyusername
|
|
Optional: The password of the Plex user
|
|
y) Yes type in my own password
|
|
g) Generate random password
|
|
n) No leave this optional password blank
|
|
y/g/n> y
|
|
Enter the password:
|
|
password:
|
|
Confirm the password:
|
|
password:
|
|
The size of a chunk. Lower value good for slow connections but can affect seamless reading.
|
|
Default: 5M
|
|
Choose a number from below, or type in your own value
|
|
1 / 1MB
|
|
\[rs] \[dq]1m\[dq]
|
|
2 / 5 MB
|
|
\[rs] \[dq]5M\[dq]
|
|
3 / 10 MB
|
|
\[rs] \[dq]10M\[dq]
|
|
chunk_size> 2
|
|
How much time should object info (file size, file hashes etc) be stored in cache. Use a very high value if you don\[aq]t plan on changing the source FS from outside the cache.
|
|
Accepted units are: \[dq]s\[dq], \[dq]m\[dq], \[dq]h\[dq].
|
|
Default: 5m
|
|
Choose a number from below, or type in your own value
|
|
1 / 1 hour
|
|
\[rs] \[dq]1h\[dq]
|
|
2 / 24 hours
|
|
\[rs] \[dq]24h\[dq]
|
|
3 / 24 hours
|
|
\[rs] \[dq]48h\[dq]
|
|
info_age> 2
|
|
The maximum size of stored chunks. When the storage grows beyond this size, the oldest chunks will be deleted.
|
|
Default: 10G
|
|
Choose a number from below, or type in your own value
|
|
1 / 500 MB
|
|
\[rs] \[dq]500M\[dq]
|
|
2 / 1 GB
|
|
\[rs] \[dq]1G\[dq]
|
|
3 / 10 GB
|
|
\[rs] \[dq]10G\[dq]
|
|
chunk_total_size> 3
|
|
Remote config
|
|
\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-
|
|
[test\-cache]
|
|
remote = local:/test
|
|
plex_url = http://127.0.0.1:32400
|
|
plex_username = dummyusername
|
|
plex_password = *** ENCRYPTED ***
|
|
chunk_size = 5M
|
|
info_age = 48h
|
|
chunk_total_size = 10G
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
You can then use it like this,
|
|
.PP
|
|
List directories in top level of your drive
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone lsd test\-cache:
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
List all the files in your drive
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone ls test\-cache:
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
To start a cached mount
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone mount \-\-allow\-other test\-cache: /var/tmp/test\-cache
|
|
\f[R]
|
|
.fi
|
|
.SS Write Features
|
|
.SS Offline uploading
|
|
.PP
|
|
In an effort to make writing through cache more reliable, the backend
|
|
now supports this feature which can be activated by specifying a
|
|
\f[C]cache\-tmp\-upload\-path\f[R].
|
|
.PP
|
|
A files goes through these states when using this feature:
|
|
.IP "1." 3
|
|
An upload is started (usually by copying a file on the cache remote)
|
|
.IP "2." 3
|
|
When the copy to the temporary location is complete the file is part of
|
|
the cached remote and looks and behaves like any other file (reading
|
|
included)
|
|
.IP "3." 3
|
|
After \f[C]cache\-tmp\-wait\-time\f[R] passes and the file is next in
|
|
line, \f[C]rclone move\f[R] is used to move the file to the cloud
|
|
provider
|
|
.IP "4." 3
|
|
Reading the file still works during the upload but most modifications on
|
|
it will be prohibited
|
|
.IP "5." 3
|
|
Once the move is complete the file is unlocked for modifications as it
|
|
becomes as any other regular file
|
|
.IP "6." 3
|
|
If the file is being read through \f[C]cache\f[R] when it\[cq]s actually
|
|
deleted from the temporary path then \f[C]cache\f[R] will simply swap
|
|
the source to the cloud provider without interrupting the reading (small
|
|
blip can happen though)
|
|
.PP
|
|
Files are uploaded in sequence and only one file is uploaded at a time.
|
|
Uploads will be stored in a queue and be processed based on the order
|
|
they were added.
|
|
The queue and the temporary storage is persistent across restarts but
|
|
can be cleared on startup with the \f[C]\-\-cache\-db\-purge\f[R] flag.
|
|
.SS Write Support
|
|
.PP
|
|
Writes are supported through \f[C]cache\f[R].
|
|
One caveat is that a mounted cache remote does not add any retry or
|
|
fallback mechanism to the upload operation.
|
|
This will depend on the implementation of the wrapped remote.
|
|
Consider using \f[C]Offline uploading\f[R] for reliable writes.
|
|
.PP
|
|
One special case is covered with \f[C]cache\-writes\f[R] which will
|
|
cache the file data at the same time as the upload when it is enabled
|
|
making it available from the cache store immediately once the upload is
|
|
finished.
|
|
.SS Read Features
|
|
.SS Multiple connections
|
|
.PP
|
|
To counter the high latency between a local PC where rclone is running
|
|
and cloud providers, the cache remote can split multiple requests to the
|
|
cloud provider for smaller file chunks and combines them together
|
|
locally where they can be available almost immediately before the reader
|
|
usually needs them.
|
|
.PP
|
|
This is similar to buffering when media files are played online.
|
|
Rclone will stay around the current marker but always try its best to
|
|
stay ahead and prepare the data before.
|
|
.SS Plex Integration
|
|
.PP
|
|
There is a direct integration with Plex which allows cache to detect
|
|
during reading if the file is in playback or not.
|
|
This helps cache to adapt how it queries the cloud provider depending on
|
|
what is needed for.
|
|
.PP
|
|
Scans will have a minimum amount of workers (1) while in a confirmed
|
|
playback cache will deploy the configured number of workers.
|
|
.PP
|
|
This integration opens the doorway to additional performance
|
|
improvements which will be explored in the near future.
|
|
.PP
|
|
\f[B]Note:\f[R] If Plex options are not configured, \f[C]cache\f[R] will
|
|
function with its configured options without adapting any of its
|
|
settings.
|
|
.PP
|
|
How to enable?
|
|
Run \f[C]rclone config\f[R] and add all the Plex options (endpoint,
|
|
username and password) in your remote and it will be automatically
|
|
enabled.
|
|
.PP
|
|
Affected settings: \- \f[C]cache\-workers\f[R]: \f[I]Configured
|
|
value\f[R] during confirmed playback or \f[I]1\f[R] all the other times
|
|
.SS Certificate Validation
|
|
.PP
|
|
When the Plex server is configured to only accept secure connections, it
|
|
is possible to use \f[C].plex.direct\f[R] URLs to ensure certificate
|
|
validation succeeds.
|
|
These URLs are used by Plex internally to connect to the Plex server
|
|
securely.
|
|
.PP
|
|
The format for these URLs is the following:
|
|
.PP
|
|
https://ip\-with\-dots\-replaced.server\-hash.plex.direct:32400/
|
|
.PP
|
|
The \f[C]ip\-with\-dots\-replaced\f[R] part can be any IPv4 address,
|
|
where the dots have been replaced with dashes, e.g.\ \f[C]127.0.0.1\f[R]
|
|
becomes \f[C]127\-0\-0\-1\f[R].
|
|
.PP
|
|
To get the \f[C]server\-hash\f[R] part, the easiest way is to visit
|
|
.PP
|
|
https://plex.tv/api/resources?includeHttps=1&X\-Plex\-Token=your\-plex\-token
|
|
.PP
|
|
This page will list all the available Plex servers for your account with
|
|
at least one \f[C].plex.direct\f[R] link for each.
|
|
Copy one URL and replace the IP address with the desired address.
|
|
This can be used as the \f[C]plex_url\f[R] value.
|
|
.SS Known issues
|
|
.SS Mount and \[en]dir\-cache\-time
|
|
.PP
|
|
\[en]dir\-cache\-time controls the first layer of directory caching
|
|
which works at the mount layer.
|
|
Being an independent caching mechanism from the \f[C]cache\f[R] backend,
|
|
it will manage its own entries based on the configured time.
|
|
.PP
|
|
To avoid getting in a scenario where dir cache has obsolete data and
|
|
cache would have the correct one, try to set
|
|
\f[C]\-\-dir\-cache\-time\f[R] to a lower time than
|
|
\f[C]\-\-cache\-info\-age\f[R].
|
|
Default values are already configured in this way.
|
|
.SS Windows support \- Experimental
|
|
.PP
|
|
There are a couple of issues with Windows \f[C]mount\f[R] functionality
|
|
that still require some investigations.
|
|
It should be considered as experimental thus far as fixes come in for
|
|
this OS.
|
|
.PP
|
|
Most of the issues seem to be related to the difference between
|
|
filesystems on Linux flavors and Windows as cache is heavily dependent
|
|
on them.
|
|
.PP
|
|
Any reports or feedback on how cache behaves on this OS is greatly
|
|
appreciated.
|
|
.IP \[bu] 2
|
|
https://github.com/rclone/rclone/issues/1935
|
|
.IP \[bu] 2
|
|
https://github.com/rclone/rclone/issues/1907
|
|
.IP \[bu] 2
|
|
https://github.com/rclone/rclone/issues/1834
|
|
.SS Risk of throttling
|
|
.PP
|
|
Future iterations of the cache backend will make use of the pooling
|
|
functionality of the cloud provider to synchronize and at the same time
|
|
make writing through it more tolerant to failures.
|
|
.PP
|
|
There are a couple of enhancements in track to add these but in the
|
|
meantime there is a valid concern that the expiring cache listings can
|
|
lead to cloud provider throttles or bans due to repeated queries on it
|
|
for very large mounts.
|
|
.PP
|
|
Some recommendations: \- don\[cq]t use a very small interval for entry
|
|
information (\f[C]\-\-cache\-info\-age\f[R]) \- while writes aren\[cq]t
|
|
yet optimised, you can still write through \f[C]cache\f[R] which gives
|
|
you the advantage of adding the file in the cache at the same time if
|
|
configured to do so.
|
|
.PP
|
|
Future enhancements:
|
|
.IP \[bu] 2
|
|
https://github.com/rclone/rclone/issues/1937
|
|
.IP \[bu] 2
|
|
https://github.com/rclone/rclone/issues/1936
|
|
.SS cache and crypt
|
|
.PP
|
|
One common scenario is to keep your data encrypted in the cloud provider
|
|
using the \f[C]crypt\f[R] remote.
|
|
\f[C]crypt\f[R] uses a similar technique to wrap around an existing
|
|
remote and handles this translation in a seamless way.
|
|
.PP
|
|
There is an issue with wrapping the remotes in this order: \f[B]cloud
|
|
remote\f[R] \-> \f[B]crypt\f[R] \-> \f[B]cache\f[R]
|
|
.PP
|
|
During testing, I experienced a lot of bans with the remotes in this
|
|
order.
|
|
I suspect it might be related to how crypt opens files on the cloud
|
|
provider which makes it think we\[cq]re downloading the full file
|
|
instead of small chunks.
|
|
Organizing the remotes in this order yields better results: \f[B]cloud
|
|
remote\f[R] \-> \f[B]cache\f[R] \-> \f[B]crypt\f[R]
|
|
.SS absolute remote paths
|
|
.PP
|
|
\f[C]cache\f[R] can not differentiate between relative and absolute
|
|
paths for the wrapped remote.
|
|
Any path given in the \f[C]remote\f[R] config setting and on the command
|
|
line will be passed to the wrapped remote as is, but for storing the
|
|
chunks on disk the path will be made relative by removing any leading
|
|
\f[C]/\f[R] character.
|
|
.PP
|
|
This behavior is irrelevant for most backend types, but there are
|
|
backends where a leading \f[C]/\f[R] changes the effective directory,
|
|
e.g.\ in the \f[C]sftp\f[R] backend paths starting with a \f[C]/\f[R]
|
|
are relative to the root of the SSH server and paths without are
|
|
relative to the user home directory.
|
|
As a result \f[C]sftp:bin\f[R] and \f[C]sftp:/bin\f[R] will share the
|
|
same cache folder, even if they represent a different directory on the
|
|
SSH server.
|
|
.SS Cache and Remote Control (\[en]rc)
|
|
.PP
|
|
Cache supports the new \f[C]\-\-rc\f[R] mode in rclone and can be remote
|
|
controlled through the following end points: By default, the listener is
|
|
disabled if you do not add the flag.
|
|
.SS rc cache/expire
|
|
.PP
|
|
Purge a remote from the cache backend.
|
|
Supports either a directory or a file.
|
|
It supports both encrypted and unencrypted file names if cache is
|
|
wrapped by crypt.
|
|
.PP
|
|
Params: \- \f[B]remote\f[R] = path to remote \f[B](required)\f[R] \-
|
|
\f[B]withData\f[R] = true/false to delete cached data (chunks) as well
|
|
\f[I](optional, false by default)\f[R]
|
|
.SS Standard Options
|
|
.PP
|
|
Here are the standard options specific to cache (Cache a remote).
|
|
.SS \[en]cache\-remote
|
|
.PP
|
|
Remote to cache.
|
|
Normally should contain a `:' and a path, eg
|
|
\[lq]myremote:path/to/dir\[rq], \[lq]myremote:bucket\[rq] or maybe
|
|
\[lq]myremote:\[rq] (not recommended).
|
|
.IP \[bu] 2
|
|
Config: remote
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_CACHE_REMOTE
|
|
.IP \[bu] 2
|
|
Type: string
|
|
.IP \[bu] 2
|
|
Default: \[dq]\[dq]
|
|
.SS \[en]cache\-plex\-url
|
|
.PP
|
|
The URL of the Plex server
|
|
.IP \[bu] 2
|
|
Config: plex_url
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_CACHE_PLEX_URL
|
|
.IP \[bu] 2
|
|
Type: string
|
|
.IP \[bu] 2
|
|
Default: \[dq]\[dq]
|
|
.SS \[en]cache\-plex\-username
|
|
.PP
|
|
The username of the Plex user
|
|
.IP \[bu] 2
|
|
Config: plex_username
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_CACHE_PLEX_USERNAME
|
|
.IP \[bu] 2
|
|
Type: string
|
|
.IP \[bu] 2
|
|
Default: \[dq]\[dq]
|
|
.SS \[en]cache\-plex\-password
|
|
.PP
|
|
The password of the Plex user
|
|
.IP \[bu] 2
|
|
Config: plex_password
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_CACHE_PLEX_PASSWORD
|
|
.IP \[bu] 2
|
|
Type: string
|
|
.IP \[bu] 2
|
|
Default: \[dq]\[dq]
|
|
.SS \[en]cache\-chunk\-size
|
|
.PP
|
|
The size of a chunk (partial file data).
|
|
.PP
|
|
Use lower numbers for slower connections.
|
|
If the chunk size is changed, any downloaded chunks will be invalid and
|
|
cache\-chunk\-path will need to be cleared or unexpected EOF errors will
|
|
occur.
|
|
.IP \[bu] 2
|
|
Config: chunk_size
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_CACHE_CHUNK_SIZE
|
|
.IP \[bu] 2
|
|
Type: SizeSuffix
|
|
.IP \[bu] 2
|
|
Default: 5M
|
|
.IP \[bu] 2
|
|
Examples:
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
\[lq]1m\[rq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
1MB
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[lq]5M\[rq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
5 MB
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[lq]10M\[rq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
10 MB
|
|
.RE
|
|
.RE
|
|
.SS \[en]cache\-info\-age
|
|
.PP
|
|
How long to cache file structure information (directory listings, file
|
|
size, times etc).
|
|
If all write operations are done through the cache then you can safely
|
|
make this value very large as the cache store will also be updated in
|
|
real time.
|
|
.IP \[bu] 2
|
|
Config: info_age
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_CACHE_INFO_AGE
|
|
.IP \[bu] 2
|
|
Type: Duration
|
|
.IP \[bu] 2
|
|
Default: 6h0m0s
|
|
.IP \[bu] 2
|
|
Examples:
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
\[lq]1h\[rq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
1 hour
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[lq]24h\[rq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
24 hours
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[lq]48h\[rq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
48 hours
|
|
.RE
|
|
.RE
|
|
.SS \[en]cache\-chunk\-total\-size
|
|
.PP
|
|
The total size that the chunks can take up on the local disk.
|
|
.PP
|
|
If the cache exceeds this value then it will start to delete the oldest
|
|
chunks until it goes under this value.
|
|
.IP \[bu] 2
|
|
Config: chunk_total_size
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_CACHE_CHUNK_TOTAL_SIZE
|
|
.IP \[bu] 2
|
|
Type: SizeSuffix
|
|
.IP \[bu] 2
|
|
Default: 10G
|
|
.IP \[bu] 2
|
|
Examples:
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
\[lq]500M\[rq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
500 MB
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[lq]1G\[rq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
1 GB
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[lq]10G\[rq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
10 GB
|
|
.RE
|
|
.RE
|
|
.SS Advanced Options
|
|
.PP
|
|
Here are the advanced options specific to cache (Cache a remote).
|
|
.SS \[en]cache\-plex\-token
|
|
.PP
|
|
The plex token for authentication \- auto set normally
|
|
.IP \[bu] 2
|
|
Config: plex_token
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_CACHE_PLEX_TOKEN
|
|
.IP \[bu] 2
|
|
Type: string
|
|
.IP \[bu] 2
|
|
Default: \[dq]\[dq]
|
|
.SS \[en]cache\-plex\-insecure
|
|
.PP
|
|
Skip all certificate verification when connecting to the Plex server
|
|
.IP \[bu] 2
|
|
Config: plex_insecure
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_CACHE_PLEX_INSECURE
|
|
.IP \[bu] 2
|
|
Type: string
|
|
.IP \[bu] 2
|
|
Default: \[dq]\[dq]
|
|
.SS \[en]cache\-db\-path
|
|
.PP
|
|
Directory to store file structure metadata DB.
|
|
The remote name is used as the DB file name.
|
|
.IP \[bu] 2
|
|
Config: db_path
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_CACHE_DB_PATH
|
|
.IP \[bu] 2
|
|
Type: string
|
|
.IP \[bu] 2
|
|
Default: \[lq]$HOME/.cache/rclone/cache\-backend\[rq]
|
|
.SS \[en]cache\-chunk\-path
|
|
.PP
|
|
Directory to cache chunk files.
|
|
.PP
|
|
Path to where partial file data (chunks) are stored locally.
|
|
The remote name is appended to the final path.
|
|
.PP
|
|
This config follows the \[lq]\[en]cache\-db\-path\[rq].
|
|
If you specify a custom location for \[lq]\[en]cache\-db\-path\[rq] and
|
|
don\[cq]t specify one for \[lq]\[en]cache\-chunk\-path\[rq] then
|
|
\[lq]\[en]cache\-chunk\-path\[rq] will use the same path as
|
|
\[lq]\[en]cache\-db\-path\[rq].
|
|
.IP \[bu] 2
|
|
Config: chunk_path
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_CACHE_CHUNK_PATH
|
|
.IP \[bu] 2
|
|
Type: string
|
|
.IP \[bu] 2
|
|
Default: \[lq]$HOME/.cache/rclone/cache\-backend\[rq]
|
|
.SS \[en]cache\-db\-purge
|
|
.PP
|
|
Clear all the cached data for this remote on start.
|
|
.IP \[bu] 2
|
|
Config: db_purge
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_CACHE_DB_PURGE
|
|
.IP \[bu] 2
|
|
Type: bool
|
|
.IP \[bu] 2
|
|
Default: false
|
|
.SS \[en]cache\-chunk\-clean\-interval
|
|
.PP
|
|
How often should the cache perform cleanups of the chunk storage.
|
|
The default value should be ok for most people.
|
|
If you find that the cache goes over \[lq]cache\-chunk\-total\-size\[rq]
|
|
too often then try to lower this value to force it to perform cleanups
|
|
more often.
|
|
.IP \[bu] 2
|
|
Config: chunk_clean_interval
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_CACHE_CHUNK_CLEAN_INTERVAL
|
|
.IP \[bu] 2
|
|
Type: Duration
|
|
.IP \[bu] 2
|
|
Default: 1m0s
|
|
.SS \[en]cache\-read\-retries
|
|
.PP
|
|
How many times to retry a read from a cache storage.
|
|
.PP
|
|
Since reading from a cache stream is independent from downloading file
|
|
data, readers can get to a point where there\[cq]s no more data in the
|
|
cache.
|
|
Most of the times this can indicate a connectivity issue if cache
|
|
isn\[cq]t able to provide file data anymore.
|
|
.PP
|
|
For really slow connections, increase this to a point where the stream
|
|
is able to provide data but your experience will be very stuttering.
|
|
.IP \[bu] 2
|
|
Config: read_retries
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_CACHE_READ_RETRIES
|
|
.IP \[bu] 2
|
|
Type: int
|
|
.IP \[bu] 2
|
|
Default: 10
|
|
.SS \[en]cache\-workers
|
|
.PP
|
|
How many workers should run in parallel to download chunks.
|
|
.PP
|
|
Higher values will mean more parallel processing (better CPU needed) and
|
|
more concurrent requests on the cloud provider.
|
|
This impacts several aspects like the cloud provider API limits, more
|
|
stress on the hardware that rclone runs on but it also means that
|
|
streams will be more fluid and data will be available much more faster
|
|
to readers.
|
|
.PP
|
|
\f[B]Note\f[R]: If the optional Plex integration is enabled then this
|
|
setting will adapt to the type of reading performed and the value
|
|
specified here will be used as a maximum number of workers to use.
|
|
.IP \[bu] 2
|
|
Config: workers
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_CACHE_WORKERS
|
|
.IP \[bu] 2
|
|
Type: int
|
|
.IP \[bu] 2
|
|
Default: 4
|
|
.SS \[en]cache\-chunk\-no\-memory
|
|
.PP
|
|
Disable the in\-memory cache for storing chunks during streaming.
|
|
.PP
|
|
By default, cache will keep file data during streaming in RAM as well to
|
|
provide it to readers as fast as possible.
|
|
.PP
|
|
This transient data is evicted as soon as it is read and the number of
|
|
chunks stored doesn\[cq]t exceed the number of workers.
|
|
However, depending on other settings like \[lq]cache\-chunk\-size\[rq]
|
|
and \[lq]cache\-workers\[rq] this footprint can increase if there are
|
|
parallel streams too (multiple files being read at the same time).
|
|
.PP
|
|
If the hardware permits it, use this feature to provide an overall
|
|
better performance during streaming but it can also be disabled if RAM
|
|
is not available on the local machine.
|
|
.IP \[bu] 2
|
|
Config: chunk_no_memory
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_CACHE_CHUNK_NO_MEMORY
|
|
.IP \[bu] 2
|
|
Type: bool
|
|
.IP \[bu] 2
|
|
Default: false
|
|
.SS \[en]cache\-rps
|
|
.PP
|
|
Limits the number of requests per second to the source FS (\-1 to
|
|
disable)
|
|
.PP
|
|
This setting places a hard limit on the number of requests per second
|
|
that cache will be doing to the cloud provider remote and try to respect
|
|
that value by setting waits between reads.
|
|
.PP
|
|
If you find that you\[cq]re getting banned or limited on the cloud
|
|
provider through cache and know that a smaller number of requests per
|
|
second will allow you to work with it then you can use this setting for
|
|
that.
|
|
.PP
|
|
A good balance of all the other settings should make this setting
|
|
useless but it is available to set for more special cases.
|
|
.PP
|
|
\f[B]NOTE\f[R]: This will limit the number of requests during streams
|
|
but other API calls to the cloud provider like directory listings will
|
|
still pass.
|
|
.IP \[bu] 2
|
|
Config: rps
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_CACHE_RPS
|
|
.IP \[bu] 2
|
|
Type: int
|
|
.IP \[bu] 2
|
|
Default: \-1
|
|
.SS \[en]cache\-writes
|
|
.PP
|
|
Cache file data on writes through the FS
|
|
.PP
|
|
If you need to read files immediately after you upload them through
|
|
cache you can enable this flag to have their data stored in the cache
|
|
store at the same time during upload.
|
|
.IP \[bu] 2
|
|
Config: writes
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_CACHE_WRITES
|
|
.IP \[bu] 2
|
|
Type: bool
|
|
.IP \[bu] 2
|
|
Default: false
|
|
.SS \[en]cache\-tmp\-upload\-path
|
|
.PP
|
|
Directory to keep temporary files until they are uploaded.
|
|
.PP
|
|
This is the path where cache will use as a temporary storage for new
|
|
files that need to be uploaded to the cloud provider.
|
|
.PP
|
|
Specifying a value will enable this feature.
|
|
Without it, it is completely disabled and files will be uploaded
|
|
directly to the cloud provider
|
|
.IP \[bu] 2
|
|
Config: tmp_upload_path
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_CACHE_TMP_UPLOAD_PATH
|
|
.IP \[bu] 2
|
|
Type: string
|
|
.IP \[bu] 2
|
|
Default: \[dq]\[dq]
|
|
.SS \[en]cache\-tmp\-wait\-time
|
|
.PP
|
|
How long should files be stored in local cache before being uploaded
|
|
.PP
|
|
This is the duration that a file must wait in the temporary location
|
|
\f[I]cache\-tmp\-upload\-path\f[R] before it is selected for upload.
|
|
.PP
|
|
Note that only one file is uploaded at a time and it can take longer to
|
|
start the upload if a queue formed for this purpose.
|
|
.IP \[bu] 2
|
|
Config: tmp_wait_time
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_CACHE_TMP_WAIT_TIME
|
|
.IP \[bu] 2
|
|
Type: Duration
|
|
.IP \[bu] 2
|
|
Default: 15s
|
|
.SS \[en]cache\-db\-wait\-time
|
|
.PP
|
|
How long to wait for the DB to be available \- 0 is unlimited
|
|
.PP
|
|
Only one process can have the DB open at any one time, so rclone waits
|
|
for this duration for the DB to become available before it gives an
|
|
error.
|
|
.PP
|
|
If you set it to 0 then it will wait forever.
|
|
.IP \[bu] 2
|
|
Config: db_wait_time
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_CACHE_DB_WAIT_TIME
|
|
.IP \[bu] 2
|
|
Type: Duration
|
|
.IP \[bu] 2
|
|
Default: 1s
|
|
.SS Backend commands
|
|
.PP
|
|
Here are the commands specific to the cache backend.
|
|
.PP
|
|
Run them with
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone backend COMMAND remote:
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
The help below will explain what arguments each command takes.
|
|
.PP
|
|
See the \[lq]rclone backend\[rq]
|
|
command (https://rclone.org/commands/rclone_backend/) for more info on
|
|
how to pass options and arguments.
|
|
.PP
|
|
These can be run on a running backend using the rc command
|
|
backend/command (https://rclone.org/rc/#backend/command).
|
|
.SS stats
|
|
.PP
|
|
Print stats on the cache backend in JSON format.
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone backend stats remote: [options] [<arguments>+]
|
|
\f[R]
|
|
.fi
|
|
.SS Chunker (BETA)
|
|
.PP
|
|
The \f[C]chunker\f[R] overlay transparently splits large files into
|
|
smaller chunks during upload to wrapped remote and transparently
|
|
assembles them back when the file is downloaded.
|
|
This allows to effectively overcome size limits imposed by storage
|
|
providers.
|
|
.PP
|
|
To use it, first set up the underlying remote following the
|
|
configuration instructions for that remote.
|
|
You can also use a local pathname instead of a remote.
|
|
.PP
|
|
First check your chosen remote is working \- we\[cq]ll call it
|
|
\f[C]remote:path\f[R] here.
|
|
Note that anything inside \f[C]remote:path\f[R] will be chunked and
|
|
anything outside won\[cq]t.
|
|
This means that if you are using a bucket based remote (eg S3, B2,
|
|
swift) then you should probably put the bucket in the remote
|
|
\f[C]s3:bucket\f[R].
|
|
.PP
|
|
Now configure \f[C]chunker\f[R] using \f[C]rclone config\f[R].
|
|
We will call this one \f[C]overlay\f[R] to separate it from the
|
|
\f[C]remote\f[R] itself.
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
No remotes found \- make a new one
|
|
n) New remote
|
|
s) Set configuration password
|
|
q) Quit config
|
|
n/s/q> n
|
|
name> overlay
|
|
Type of storage to configure.
|
|
Choose a number from below, or type in your own value
|
|
[snip]
|
|
XX / Transparently chunk/split large files
|
|
\[rs] \[dq]chunker\[dq]
|
|
[snip]
|
|
Storage> chunker
|
|
Remote to chunk/unchunk.
|
|
Normally should contain a \[aq]:\[aq] and a path, eg \[dq]myremote:path/to/dir\[dq],
|
|
\[dq]myremote:bucket\[dq] or maybe \[dq]myremote:\[dq] (not recommended).
|
|
Enter a string value. Press Enter for the default (\[dq]\[dq]).
|
|
remote> remote:path
|
|
Files larger than chunk size will be split in chunks.
|
|
Enter a size with suffix k,M,G,T. Press Enter for the default (\[dq]2G\[dq]).
|
|
chunk_size> 100M
|
|
Choose how chunker handles hash sums. All modes but \[dq]none\[dq] require metadata.
|
|
Enter a string value. Press Enter for the default (\[dq]md5\[dq]).
|
|
Choose a number from below, or type in your own value
|
|
1 / Pass any hash supported by wrapped remote for non\-chunked files, return nothing otherwise
|
|
\[rs] \[dq]none\[dq]
|
|
2 / MD5 for composite files
|
|
\[rs] \[dq]md5\[dq]
|
|
3 / SHA1 for composite files
|
|
\[rs] \[dq]sha1\[dq]
|
|
4 / MD5 for all files
|
|
\[rs] \[dq]md5all\[dq]
|
|
5 / SHA1 for all files
|
|
\[rs] \[dq]sha1all\[dq]
|
|
6 / Copying a file to chunker will request MD5 from the source falling back to SHA1 if unsupported
|
|
\[rs] \[dq]md5quick\[dq]
|
|
7 / Similar to \[dq]md5quick\[dq] but prefers SHA1 over MD5
|
|
\[rs] \[dq]sha1quick\[dq]
|
|
hash_type> md5
|
|
Edit advanced config? (y/n)
|
|
y) Yes
|
|
n) No
|
|
y/n> n
|
|
Remote config
|
|
\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-
|
|
[overlay]
|
|
type = chunker
|
|
remote = remote:bucket
|
|
chunk_size = 100M
|
|
hash_type = md5
|
|
\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-
|
|
y) Yes this is OK
|
|
e) Edit this remote
|
|
d) Delete this remote
|
|
y/e/d> y
|
|
\f[R]
|
|
.fi
|
|
.SS Specifying the remote
|
|
.PP
|
|
In normal use, make sure the remote has a \f[C]:\f[R] in.
|
|
If you specify the remote without a \f[C]:\f[R] then rclone will use a
|
|
local directory of that name.
|
|
So if you use a remote of \f[C]/path/to/secret/files\f[R] then rclone
|
|
will chunk stuff in that directory.
|
|
If you use a remote of \f[C]name\f[R] then rclone will put files in a
|
|
directory called \f[C]name\f[R] in the current directory.
|
|
.SS Chunking
|
|
.PP
|
|
When rclone starts a file upload, chunker checks the file size.
|
|
If it doesn\[cq]t exceed the configured chunk size, chunker will just
|
|
pass the file to the wrapped remote.
|
|
If a file is large, chunker will transparently cut data in pieces with
|
|
temporary names and stream them one by one, on the fly.
|
|
Each data chunk will contain the specified number of bytes, except for
|
|
the last one which may have less data.
|
|
If file size is unknown in advance (this is called a streaming upload),
|
|
chunker will internally create a temporary copy, record its size and
|
|
repeat the above process.
|
|
.PP
|
|
When upload completes, temporary chunk files are finally renamed.
|
|
This scheme guarantees that operations can be run in parallel and look
|
|
from outside as atomic.
|
|
A similar method with hidden temporary chunks is used for other
|
|
operations (copy/move/rename etc).
|
|
If an operation fails, hidden chunks are normally destroyed, and the
|
|
target composite file stays intact.
|
|
.PP
|
|
When a composite file download is requested, chunker transparently
|
|
assembles it by concatenating data chunks in order.
|
|
As the split is trivial one could even manually concatenate data chunks
|
|
together to obtain the original content.
|
|
.PP
|
|
When the \f[C]list\f[R] rclone command scans a directory on wrapped
|
|
remote, the potential chunk files are accounted for, grouped and
|
|
assembled into composite directory entries.
|
|
Any temporary chunks are hidden.
|
|
.PP
|
|
List and other commands can sometimes come across composite files with
|
|
missing or invalid chunks, eg.
|
|
shadowed by like\-named directory or another file.
|
|
This usually means that wrapped file system has been directly tampered
|
|
with or damaged.
|
|
If chunker detects a missing chunk it will by default print warning,
|
|
skip the whole incomplete group of chunks but proceed with current
|
|
command.
|
|
You can set the \f[C]\-\-chunker\-fail\-hard\f[R] flag to have commands
|
|
abort with error message in such cases.
|
|
.SS Chunk names
|
|
.PP
|
|
The default chunk name format is \f[C]*.rclone_chunk.###\f[R], hence by
|
|
default chunk names are \f[C]BIG_FILE_NAME.rclone_chunk.001\f[R],
|
|
\f[C]BIG_FILE_NAME.rclone_chunk.002\f[R] etc.
|
|
You can configure another name format using the \f[C]name_format\f[R]
|
|
configuration file option.
|
|
The format uses asterisk \f[C]*\f[R] as a placeholder for the base file
|
|
name and one or more consecutive hash characters \f[C]#\f[R] as a
|
|
placeholder for sequential chunk number.
|
|
There must be one and only one asterisk.
|
|
The number of consecutive hash characters defines the minimum length of
|
|
a string representing a chunk number.
|
|
If decimal chunk number has less digits than the number of hashes, it is
|
|
left\-padded by zeros.
|
|
If the decimal string is longer, it is left intact.
|
|
By default numbering starts from 1 but there is another option that
|
|
allows user to start from 0, eg.
|
|
for compatibility with legacy software.
|
|
.PP
|
|
For example, if name format is \f[C]big_*\-##.part\f[R] and original
|
|
file name is \f[C]data.txt\f[R] and numbering starts from 0, then the
|
|
first chunk will be named \f[C]big_data.txt\-00.part\f[R], the 99th
|
|
chunk will be \f[C]big_data.txt\-98.part\f[R] and the 302nd chunk will
|
|
become \f[C]big_data.txt\-301.part\f[R].
|
|
.PP
|
|
Note that \f[C]list\f[R] assembles composite directory entries only when
|
|
chunk names match the configured format and treats non\-conforming file
|
|
names as normal non\-chunked files.
|
|
.SS Metadata
|
|
.PP
|
|
Besides data chunks chunker will by default create metadata object for a
|
|
composite file.
|
|
The object is named after the original file.
|
|
Chunker allows user to disable metadata completely (the \f[C]none\f[R]
|
|
format).
|
|
Note that metadata is normally not created for files smaller than the
|
|
configured chunk size.
|
|
This may change in future rclone releases.
|
|
.SS Simple JSON metadata format
|
|
.PP
|
|
This is the default format.
|
|
It supports hash sums and chunk validation for composite files.
|
|
Meta objects carry the following fields:
|
|
.IP \[bu] 2
|
|
\f[C]ver\f[R] \- version of format, currently \f[C]1\f[R]
|
|
.IP \[bu] 2
|
|
\f[C]size\f[R] \- total size of composite file
|
|
.IP \[bu] 2
|
|
\f[C]nchunks\f[R] \- number of data chunks in file
|
|
.IP \[bu] 2
|
|
\f[C]md5\f[R] \- MD5 hashsum of composite file (if present)
|
|
.IP \[bu] 2
|
|
\f[C]sha1\f[R] \- SHA1 hashsum (if present)
|
|
.PP
|
|
There is no field for composite file name as it\[cq]s simply equal to
|
|
the name of meta object on the wrapped remote.
|
|
Please refer to respective sections for details on hashsums and modified
|
|
time handling.
|
|
.SS No metadata
|
|
.PP
|
|
You can disable meta objects by setting the meta format option to
|
|
\f[C]none\f[R].
|
|
In this mode chunker will scan directory for all files that follow
|
|
configured chunk name format, group them by detecting chunks with the
|
|
same base name and show group names as virtual composite files.
|
|
This method is more prone to missing chunk errors (especially missing
|
|
last chunk) than format with metadata enabled.
|
|
.SS Hashsums
|
|
.PP
|
|
Chunker supports hashsums only when a compatible metadata is present.
|
|
Hence, if you choose metadata format of \f[C]none\f[R], chunker will
|
|
report hashsum as \f[C]UNSUPPORTED\f[R].
|
|
.PP
|
|
Please note that by default metadata is stored only for composite files.
|
|
If a file is smaller than configured chunk size, chunker will
|
|
transparently redirect hash requests to wrapped remote, so support
|
|
depends on that.
|
|
You will see the empty string as a hashsum of requested type for small
|
|
files if the wrapped remote doesn\[cq]t support it.
|
|
.PP
|
|
Many storage backends support MD5 and SHA1 hash types, so does chunker.
|
|
With chunker you can choose one or another but not both.
|
|
MD5 is set by default as the most supported type.
|
|
Since chunker keeps hashes for composite files and falls back to the
|
|
wrapped remote hash for non\-chunked ones, we advise you to choose the
|
|
same hash type as supported by wrapped remote so that your file listings
|
|
look coherent.
|
|
.PP
|
|
If your storage backend does not support MD5 or SHA1 but you need
|
|
consistent file hashing, configure chunker with \f[C]md5all\f[R] or
|
|
\f[C]sha1all\f[R].
|
|
These two modes guarantee given hash for all files.
|
|
If wrapped remote doesn\[cq]t support it, chunker will then add metadata
|
|
to all files, even small.
|
|
However, this can double the amount of small files in storage and incur
|
|
additional service charges.
|
|
You can even use chunker to force md5/sha1 support in any other remote
|
|
at expense of sidecar meta objects by setting eg.
|
|
\f[C]chunk_type=sha1all\f[R] to force hashsums and
|
|
\f[C]chunk_size=1P\f[R] to effectively disable chunking.
|
|
.PP
|
|
Normally, when a file is copied to chunker controlled remote, chunker
|
|
will ask the file source for compatible file hash and revert to
|
|
on\-the\-fly calculation if none is found.
|
|
This involves some CPU overhead but provides a guarantee that given
|
|
hashsum is available.
|
|
Also, chunker will reject a server\-side copy or move operation if
|
|
source and destination hashsum types are different resulting in the
|
|
extra network bandwidth, too.
|
|
In some rare cases this may be undesired, so chunker provides two
|
|
optional choices: \f[C]sha1quick\f[R] and \f[C]md5quick\f[R].
|
|
If the source does not support primary hash type and the quick mode is
|
|
enabled, chunker will try to fall back to the secondary type.
|
|
This will save CPU and bandwidth but can result in empty hashsums at
|
|
destination.
|
|
Beware of consequences: the \f[C]sync\f[R] command will revert
|
|
(sometimes silently) to time/size comparison if compatible hashsums
|
|
between source and target are not found.
|
|
.SS Modified time
|
|
.PP
|
|
Chunker stores modification times using the wrapped remote so support
|
|
depends on that.
|
|
For a small non\-chunked file the chunker overlay simply manipulates
|
|
modification time of the wrapped remote file.
|
|
For a composite file with metadata chunker will get and set modification
|
|
time of the metadata object on the wrapped remote.
|
|
If file is chunked but metadata format is \f[C]none\f[R] then chunker
|
|
will use modification time of the first data chunk.
|
|
.SS Migrations
|
|
.PP
|
|
The idiomatic way to migrate to a different chunk size, hash type or
|
|
chunk naming scheme is to:
|
|
.IP \[bu] 2
|
|
Collect all your chunked files under a directory and have your chunker
|
|
remote point to it.
|
|
.IP \[bu] 2
|
|
Create another directory (most probably on the same cloud storage) and
|
|
configure a new remote with desired metadata format, hash type, chunk
|
|
naming etc.
|
|
.IP \[bu] 2
|
|
Now run \f[C]rclone sync oldchunks: newchunks:\f[R] and all your data
|
|
will be transparently converted in transfer.
|
|
This may take some time, yet chunker will try server\-side copy if
|
|
possible.
|
|
.IP \[bu] 2
|
|
After checking data integrity you may remove configuration section of
|
|
the old remote.
|
|
.PP
|
|
If rclone gets killed during a long operation on a big composite file,
|
|
hidden temporary chunks may stay in the directory.
|
|
They will not be shown by the \f[C]list\f[R] command but will eat up
|
|
your account quota.
|
|
Please note that the \f[C]deletefile\f[R] command deletes only active
|
|
chunks of a file.
|
|
As a workaround, you can use remote of the wrapped file system to see
|
|
them.
|
|
An easy way to get rid of hidden garbage is to copy littered directory
|
|
somewhere using the chunker remote and purge the original directory.
|
|
The \f[C]copy\f[R] command will copy only active chunks while the
|
|
\f[C]purge\f[R] will remove everything including garbage.
|
|
.SS Caveats and Limitations
|
|
.PP
|
|
Chunker requires wrapped remote to support server side \f[C]move\f[R]
|
|
(or \f[C]copy\f[R] + \f[C]delete\f[R]) operations, otherwise it will
|
|
explicitly refuse to start.
|
|
This is because it internally renames temporary chunk files to their
|
|
final names when an operation completes successfully.
|
|
.PP
|
|
Chunker encodes chunk number in file name, so with default
|
|
\f[C]name_format\f[R] setting it adds 17 characters.
|
|
Also chunker adds 7 characters of temporary suffix during operations.
|
|
Many file systems limit base file name without path by 255 characters.
|
|
Using rclone\[cq]s crypt remote as a base file system limits file name
|
|
by 143 characters.
|
|
Thus, maximum name length is 231 for most files and 119 for
|
|
chunker\-over\-crypt.
|
|
A user in need can change name format to eg.
|
|
\f[C]*.rcc##\f[R] and save 10 characters (provided at most 99 chunks per
|
|
file).
|
|
.PP
|
|
Note that a move implemented using the copy\-and\-delete method may
|
|
incur double charging with some cloud storage providers.
|
|
.PP
|
|
Chunker will not automatically rename existing chunks when you run
|
|
\f[C]rclone config\f[R] on a live remote and change the chunk name
|
|
format.
|
|
Beware that in result of this some files which have been treated as
|
|
chunks before the change can pop up in directory listings as normal
|
|
files and vice versa.
|
|
The same warning holds for the chunk size.
|
|
If you desperately need to change critical chunking settings, you should
|
|
run data migration as described above.
|
|
.PP
|
|
If wrapped remote is case insensitive, the chunker overlay will inherit
|
|
that property (so you can\[cq]t have a file called \[lq]Hello.doc\[rq]
|
|
and \[lq]hello.doc\[rq] in the same directory).
|
|
.SS Standard Options
|
|
.PP
|
|
Here are the standard options specific to chunker (Transparently
|
|
chunk/split large files).
|
|
.SS \[en]chunker\-remote
|
|
.PP
|
|
Remote to chunk/unchunk.
|
|
Normally should contain a `:' and a path, eg
|
|
\[lq]myremote:path/to/dir\[rq], \[lq]myremote:bucket\[rq] or maybe
|
|
\[lq]myremote:\[rq] (not recommended).
|
|
.IP \[bu] 2
|
|
Config: remote
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_CHUNKER_REMOTE
|
|
.IP \[bu] 2
|
|
Type: string
|
|
.IP \[bu] 2
|
|
Default: \[dq]\[dq]
|
|
.SS \[en]chunker\-chunk\-size
|
|
.PP
|
|
Files larger than chunk size will be split in chunks.
|
|
.IP \[bu] 2
|
|
Config: chunk_size
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_CHUNKER_CHUNK_SIZE
|
|
.IP \[bu] 2
|
|
Type: SizeSuffix
|
|
.IP \[bu] 2
|
|
Default: 2G
|
|
.SS \[en]chunker\-hash\-type
|
|
.PP
|
|
Choose how chunker handles hash sums.
|
|
All modes but \[lq]none\[rq] require metadata.
|
|
.IP \[bu] 2
|
|
Config: hash_type
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_CHUNKER_HASH_TYPE
|
|
.IP \[bu] 2
|
|
Type: string
|
|
.IP \[bu] 2
|
|
Default: \[lq]md5\[rq]
|
|
.IP \[bu] 2
|
|
Examples:
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
\[lq]none\[rq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Pass any hash supported by wrapped remote for non\-chunked files, return
|
|
nothing otherwise
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[lq]md5\[rq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
MD5 for composite files
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[lq]sha1\[rq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
SHA1 for composite files
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[lq]md5all\[rq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
MD5 for all files
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[lq]sha1all\[rq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
SHA1 for all files
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[lq]md5quick\[rq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Copying a file to chunker will request MD5 from the source falling back
|
|
to SHA1 if unsupported
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[lq]sha1quick\[rq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Similar to \[lq]md5quick\[rq] but prefers SHA1 over MD5
|
|
.RE
|
|
.RE
|
|
.SS Advanced Options
|
|
.PP
|
|
Here are the advanced options specific to chunker (Transparently
|
|
chunk/split large files).
|
|
.SS \[en]chunker\-name\-format
|
|
.PP
|
|
String format of chunk file names.
|
|
The two placeholders are: base file name (*) and chunk number (#\&...).
|
|
There must be one and only one asterisk and one or more consecutive hash
|
|
characters.
|
|
If chunk number has less digits than the number of hashes, it is
|
|
left\-padded by zeros.
|
|
If there are more digits in the number, they are left as is.
|
|
Possible chunk files are ignored if their name does not match given
|
|
format.
|
|
.IP \[bu] 2
|
|
Config: name_format
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_CHUNKER_NAME_FORMAT
|
|
.IP \[bu] 2
|
|
Type: string
|
|
.IP \[bu] 2
|
|
Default: \[dq]*.rclone_chunk.###\[dq]
|
|
.SS \[en]chunker\-start\-from
|
|
.PP
|
|
Minimum valid chunk number.
|
|
Usually 0 or 1.
|
|
By default chunk numbers start from 1.
|
|
.IP \[bu] 2
|
|
Config: start_from
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_CHUNKER_START_FROM
|
|
.IP \[bu] 2
|
|
Type: int
|
|
.IP \[bu] 2
|
|
Default: 1
|
|
.SS \[en]chunker\-meta\-format
|
|
.PP
|
|
Format of the metadata object or \[lq]none\[rq].
|
|
By default \[lq]simplejson\[rq].
|
|
Metadata is a small JSON file named after the composite file.
|
|
.IP \[bu] 2
|
|
Config: meta_format
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_CHUNKER_META_FORMAT
|
|
.IP \[bu] 2
|
|
Type: string
|
|
.IP \[bu] 2
|
|
Default: \[lq]simplejson\[rq]
|
|
.IP \[bu] 2
|
|
Examples:
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
\[lq]none\[rq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Do not use metadata files at all.
|
|
Requires hash type \[lq]none\[rq].
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[lq]simplejson\[rq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Simple JSON supports hash sums and chunk validation.
|
|
.IP \[bu] 2
|
|
It has the following fields: ver, size, nchunks, md5, sha1.
|
|
.RE
|
|
.RE
|
|
.SS \[en]chunker\-fail\-hard
|
|
.PP
|
|
Choose how chunker should handle files with missing or invalid chunks.
|
|
.IP \[bu] 2
|
|
Config: fail_hard
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_CHUNKER_FAIL_HARD
|
|
.IP \[bu] 2
|
|
Type: bool
|
|
.IP \[bu] 2
|
|
Default: false
|
|
.IP \[bu] 2
|
|
Examples:
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
\[lq]true\[rq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Report errors and abort current command.
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[lq]false\[rq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Warn user, skip incomplete file and proceed.
|
|
.RE
|
|
.RE
|
|
.SS Citrix ShareFile
|
|
.PP
|
|
Citrix ShareFile (https://sharefile.com) is a secure file sharing and
|
|
transfer service aimed as business.
|
|
.PP
|
|
The initial setup for Citrix ShareFile involves getting a token from
|
|
Citrix ShareFile which you can in your browser.
|
|
\f[C]rclone config\f[R] walks you through it.
|
|
.PP
|
|
Here is an example of how to make a remote called \f[C]remote\f[R].
|
|
First run:
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone config
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
This will guide you through an interactive setup process:
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
No remotes found \- make a new one
|
|
n) New remote
|
|
s) Set configuration password
|
|
q) Quit config
|
|
n/s/q> n
|
|
name> remote
|
|
Type of storage to configure.
|
|
Enter a string value. Press Enter for the default (\[dq]\[dq]).
|
|
Choose a number from below, or type in your own value
|
|
XX / Citrix Sharefile
|
|
\[rs] \[dq]sharefile\[dq]
|
|
Storage> sharefile
|
|
** See help for sharefile backend at: https://rclone.org/sharefile/ **
|
|
|
|
ID of the root folder
|
|
|
|
Leave blank to access \[dq]Personal Folders\[dq]. You can use one of the
|
|
standard values here or any folder ID (long hex number ID).
|
|
Enter a string value. Press Enter for the default (\[dq]\[dq]).
|
|
Choose a number from below, or type in your own value
|
|
1 / Access the Personal Folders. (Default)
|
|
\[rs] \[dq]\[dq]
|
|
2 / Access the Favorites folder.
|
|
\[rs] \[dq]favorites\[dq]
|
|
3 / Access all the shared folders.
|
|
\[rs] \[dq]allshared\[dq]
|
|
4 / Access all the individual connectors.
|
|
\[rs] \[dq]connectors\[dq]
|
|
5 / Access the home, favorites, and shared folders as well as the connectors.
|
|
\[rs] \[dq]top\[dq]
|
|
root_folder_id>
|
|
Edit advanced config? (y/n)
|
|
y) Yes
|
|
n) No
|
|
y/n> n
|
|
Remote config
|
|
Use auto config?
|
|
* Say Y if not sure
|
|
* Say N if you are working on a remote or headless machine
|
|
y) Yes
|
|
n) No
|
|
y/n> y
|
|
If your browser doesn\[aq]t open automatically go to the following link: http://127.0.0.1:53682/auth?state=XXX
|
|
Log in and authorize rclone for access
|
|
Waiting for code...
|
|
Got code
|
|
\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-
|
|
[remote]
|
|
type = sharefile
|
|
endpoint = https://XXX.sharefile.com
|
|
token = {\[dq]access_token\[dq]:\[dq]XXX\[dq],\[dq]token_type\[dq]:\[dq]bearer\[dq],\[dq]refresh_token\[dq]:\[dq]XXX\[dq],\[dq]expiry\[dq]:\[dq]2019\-09\-30T19:41:45.878561877+01:00\[dq]}
|
|
\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-
|
|
y) Yes this is OK
|
|
e) Edit this remote
|
|
d) Delete this remote
|
|
y/e/d> y
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
See the remote setup docs (https://rclone.org/remote_setup/) for how to
|
|
set it up on a machine with no Internet browser available.
|
|
.PP
|
|
Note that rclone runs a webserver on your local machine to collect the
|
|
token as returned from Citrix ShareFile.
|
|
This only runs from the moment it opens your browser to the moment you
|
|
get back the verification code.
|
|
This is on \f[C]http://127.0.0.1:53682/\f[R] and this it may require you
|
|
to unblock it temporarily if you are running a host firewall.
|
|
.PP
|
|
Once configured you can then use \f[C]rclone\f[R] like this,
|
|
.PP
|
|
List directories in top level of your ShareFile
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone lsd remote:
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
List all the files in your ShareFile
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone ls remote:
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
To copy a local directory to an ShareFile directory called backup
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone copy /home/source remote:backup
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
Paths may be as deep as required, eg
|
|
\f[C]remote:directory/subdirectory\f[R].
|
|
.SS Modified time and hashes
|
|
.PP
|
|
ShareFile allows modification times to be set on objects accurate to 1
|
|
second.
|
|
These will be used to detect whether objects need syncing or not.
|
|
.PP
|
|
ShareFile supports MD5 type hashes, so you can use the
|
|
\f[C]\-\-checksum\f[R] flag.
|
|
.SS Transfers
|
|
.PP
|
|
For files above 128MB rclone will use a chunked transfer.
|
|
Rclone will upload up to \f[C]\-\-transfers\f[R] chunks at the same time
|
|
(shared among all the multipart uploads).
|
|
Chunks are buffered in memory and are normally 64MB so increasing
|
|
\f[C]\-\-transfers\f[R] will increase memory use.
|
|
.SS Limitations
|
|
.PP
|
|
Note that ShareFile is case insensitive so you can\[cq]t have a file
|
|
called \[lq]Hello.doc\[rq] and one called \[lq]hello.doc\[rq].
|
|
.PP
|
|
ShareFile only supports filenames up to 256 characters in length.
|
|
.SS Restricted filename characters
|
|
.PP
|
|
In addition to the default restricted characters
|
|
set (https://rclone.org/overview/#restricted-characters) the following
|
|
characters are also replaced:
|
|
.PP
|
|
.TS
|
|
tab(@);
|
|
l c c.
|
|
T{
|
|
Character
|
|
T}@T{
|
|
Value
|
|
T}@T{
|
|
Replacement
|
|
T}
|
|
_
|
|
T{
|
|
\[rs]
|
|
T}@T{
|
|
0x5C
|
|
T}@T{
|
|
\[uFF3C]
|
|
T}
|
|
T{
|
|
*
|
|
T}@T{
|
|
0x2A
|
|
T}@T{
|
|
\[uFF0A]
|
|
T}
|
|
T{
|
|
<
|
|
T}@T{
|
|
0x3C
|
|
T}@T{
|
|
\[uFF1C]
|
|
T}
|
|
T{
|
|
>
|
|
T}@T{
|
|
0x3E
|
|
T}@T{
|
|
\[uFF1E]
|
|
T}
|
|
T{
|
|
?
|
|
T}@T{
|
|
0x3F
|
|
T}@T{
|
|
\[uFF1F]
|
|
T}
|
|
T{
|
|
:
|
|
T}@T{
|
|
0x3A
|
|
T}@T{
|
|
\[uFF1A]
|
|
T}
|
|
T{
|
|
|
|
|
T}@T{
|
|
0x7C
|
|
T}@T{
|
|
\[uFF5C]
|
|
T}
|
|
T{
|
|
\[dq]
|
|
T}@T{
|
|
0x22
|
|
T}@T{
|
|
\[uFF02]
|
|
T}
|
|
.TE
|
|
.PP
|
|
File names can also not start or end with the following characters.
|
|
These only get replaced if they are the first or last character in the
|
|
name:
|
|
.PP
|
|
.TS
|
|
tab(@);
|
|
l c c.
|
|
T{
|
|
Character
|
|
T}@T{
|
|
Value
|
|
T}@T{
|
|
Replacement
|
|
T}
|
|
_
|
|
T{
|
|
SP
|
|
T}@T{
|
|
0x20
|
|
T}@T{
|
|
\[u2420]
|
|
T}
|
|
T{
|
|
\&.
|
|
T}@T{
|
|
0x2E
|
|
T}@T{
|
|
\[uFF0E]
|
|
T}
|
|
.TE
|
|
.PP
|
|
Invalid UTF\-8 bytes will also be
|
|
replaced (https://rclone.org/overview/#invalid-utf8), as they can\[cq]t
|
|
be used in JSON strings.
|
|
.SS Standard Options
|
|
.PP
|
|
Here are the standard options specific to sharefile (Citrix Sharefile).
|
|
.SS \[en]sharefile\-root\-folder\-id
|
|
.PP
|
|
ID of the root folder
|
|
.PP
|
|
Leave blank to access \[lq]Personal Folders\[rq].
|
|
You can use one of the standard values here or any folder ID (long hex
|
|
number ID).
|
|
.IP \[bu] 2
|
|
Config: root_folder_id
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_SHAREFILE_ROOT_FOLDER_ID
|
|
.IP \[bu] 2
|
|
Type: string
|
|
.IP \[bu] 2
|
|
Default: \[dq]\[dq]
|
|
.IP \[bu] 2
|
|
Examples:
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
\[dq]\[dq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Access the Personal Folders.
|
|
(Default)
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[lq]favorites\[rq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Access the Favorites folder.
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[lq]allshared\[rq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Access all the shared folders.
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[lq]connectors\[rq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Access all the individual connectors.
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[lq]top\[rq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Access the home, favorites, and shared folders as well as the
|
|
connectors.
|
|
.RE
|
|
.RE
|
|
.SS Advanced Options
|
|
.PP
|
|
Here are the advanced options specific to sharefile (Citrix Sharefile).
|
|
.SS \[en]sharefile\-upload\-cutoff
|
|
.PP
|
|
Cutoff for switching to multipart upload.
|
|
.IP \[bu] 2
|
|
Config: upload_cutoff
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_SHAREFILE_UPLOAD_CUTOFF
|
|
.IP \[bu] 2
|
|
Type: SizeSuffix
|
|
.IP \[bu] 2
|
|
Default: 128M
|
|
.SS \[en]sharefile\-chunk\-size
|
|
.PP
|
|
Upload chunk size.
|
|
Must a power of 2 >= 256k.
|
|
.PP
|
|
Making this larger will improve performance, but note that each chunk is
|
|
buffered in memory one per transfer.
|
|
.PP
|
|
Reducing this will reduce memory usage but decrease performance.
|
|
.IP \[bu] 2
|
|
Config: chunk_size
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_SHAREFILE_CHUNK_SIZE
|
|
.IP \[bu] 2
|
|
Type: SizeSuffix
|
|
.IP \[bu] 2
|
|
Default: 64M
|
|
.SS \[en]sharefile\-endpoint
|
|
.PP
|
|
Endpoint for API calls.
|
|
.PP
|
|
This is usually auto discovered as part of the oauth process, but can be
|
|
set manually to something like: https://XXX.sharefile.com
|
|
.IP \[bu] 2
|
|
Config: endpoint
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_SHAREFILE_ENDPOINT
|
|
.IP \[bu] 2
|
|
Type: string
|
|
.IP \[bu] 2
|
|
Default: \[dq]\[dq]
|
|
.SS \[en]sharefile\-encoding
|
|
.PP
|
|
This sets the encoding for the backend.
|
|
.PP
|
|
See: the encoding section in the
|
|
overview (https://rclone.org/overview/#encoding) for more info.
|
|
.IP \[bu] 2
|
|
Config: encoding
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_SHAREFILE_ENCODING
|
|
.IP \[bu] 2
|
|
Type: MultiEncoder
|
|
.IP \[bu] 2
|
|
Default:
|
|
Slash,LtGt,DoubleQuote,Colon,Question,Asterisk,Pipe,BackSlash,Ctl,LeftSpace,LeftPeriod,RightSpace,RightPeriod,InvalidUtf8,Dot
|
|
.SS Crypt
|
|
.PP
|
|
The \f[C]crypt\f[R] remote encrypts and decrypts another remote.
|
|
.PP
|
|
To use it first set up the underlying remote following the config
|
|
instructions for that remote.
|
|
You can also use a local pathname instead of a remote which will encrypt
|
|
and decrypt from that directory which might be useful for encrypting
|
|
onto a USB stick for example.
|
|
.PP
|
|
First check your chosen remote is working \- we\[cq]ll call it
|
|
\f[C]remote:path\f[R] in these docs.
|
|
Note that anything inside \f[C]remote:path\f[R] will be encrypted and
|
|
anything outside won\[cq]t.
|
|
This means that if you are using a bucket based remote (eg S3, B2,
|
|
swift) then you should probably put the bucket in the remote
|
|
\f[C]s3:bucket\f[R].
|
|
If you just use \f[C]s3:\f[R] then rclone will make encrypted bucket
|
|
names too (if using file name encryption) which may or may not be what
|
|
you want.
|
|
.PP
|
|
Now configure \f[C]crypt\f[R] using \f[C]rclone config\f[R].
|
|
We will call this one \f[C]secret\f[R] to differentiate it from the
|
|
\f[C]remote\f[R].
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
No remotes found \- make a new one
|
|
n) New remote
|
|
s) Set configuration password
|
|
q) Quit config
|
|
n/s/q> n
|
|
name> secret
|
|
Type of storage to configure.
|
|
Choose a number from below, or type in your own value
|
|
[snip]
|
|
XX / Encrypt/Decrypt a remote
|
|
\[rs] \[dq]crypt\[dq]
|
|
[snip]
|
|
Storage> crypt
|
|
Remote to encrypt/decrypt.
|
|
Normally should contain a \[aq]:\[aq] and a path, eg \[dq]myremote:path/to/dir\[dq],
|
|
\[dq]myremote:bucket\[dq] or maybe \[dq]myremote:\[dq] (not recommended).
|
|
remote> remote:path
|
|
How to encrypt the filenames.
|
|
Choose a number from below, or type in your own value
|
|
1 / Don\[aq]t encrypt the file names. Adds a \[dq].bin\[dq] extension only.
|
|
\[rs] \[dq]off\[dq]
|
|
2 / Encrypt the filenames see the docs for the details.
|
|
\[rs] \[dq]standard\[dq]
|
|
3 / Very simple filename obfuscation.
|
|
\[rs] \[dq]obfuscate\[dq]
|
|
filename_encryption> 2
|
|
Option to either encrypt directory names or leave them intact.
|
|
Choose a number from below, or type in your own value
|
|
1 / Encrypt directory names.
|
|
\[rs] \[dq]true\[dq]
|
|
2 / Don\[aq]t encrypt directory names, leave them intact.
|
|
\[rs] \[dq]false\[dq]
|
|
filename_encryption> 1
|
|
Password or pass phrase for encryption.
|
|
y) Yes type in my own password
|
|
g) Generate random password
|
|
y/g> y
|
|
Enter the password:
|
|
password:
|
|
Confirm the password:
|
|
password:
|
|
Password or pass phrase for salt. Optional but recommended.
|
|
Should be different to the previous password.
|
|
y) Yes type in my own password
|
|
g) Generate random password
|
|
n) No leave this optional password blank
|
|
y/g/n> g
|
|
Password strength in bits.
|
|
64 is just about memorable
|
|
128 is secure
|
|
1024 is the maximum
|
|
Bits> 128
|
|
Your password is: JAsJvRcgR\-_veXNfy_sGmQ
|
|
Use this password?
|
|
y) Yes
|
|
n) No
|
|
y/n> y
|
|
Remote config
|
|
\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-
|
|
[secret]
|
|
remote = remote:path
|
|
filename_encryption = standard
|
|
password = *** ENCRYPTED ***
|
|
password2 = *** ENCRYPTED ***
|
|
\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-
|
|
y) Yes this is OK
|
|
e) Edit this remote
|
|
d) Delete this remote
|
|
y/e/d> y
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
\f[B]Important\f[R] The password is stored in the config file is lightly
|
|
obscured so it isn\[cq]t immediately obvious what it is.
|
|
It is in no way secure unless you use config file encryption.
|
|
.PP
|
|
A long passphrase is recommended, or you can use a random one.
|
|
.PP
|
|
The obscured password is created by using AES\-CTR with a static key,
|
|
with the salt stored verbatim at the beginning of the obscured password.
|
|
This static key is shared by between all versions of rclone.
|
|
.PP
|
|
If you reconfigure rclone with the same passwords/passphrases elsewhere
|
|
it will be compatible, but the obscured version will be different due to
|
|
the different salt.
|
|
.PP
|
|
Note that rclone does not encrypt
|
|
.IP \[bu] 2
|
|
file length \- this can be calculated within 16 bytes
|
|
.IP \[bu] 2
|
|
modification time \- used for syncing
|
|
.SS Specifying the remote
|
|
.PP
|
|
In normal use, make sure the remote has a \f[C]:\f[R] in.
|
|
If you specify the remote without a \f[C]:\f[R] then rclone will use a
|
|
local directory of that name.
|
|
So if you use a remote of \f[C]/path/to/secret/files\f[R] then rclone
|
|
will encrypt stuff to that directory.
|
|
If you use a remote of \f[C]name\f[R] then rclone will put files in a
|
|
directory called \f[C]name\f[R] in the current directory.
|
|
.PP
|
|
If you specify the remote as \f[C]remote:path/to/dir\f[R] then rclone
|
|
will store encrypted files in \f[C]path/to/dir\f[R] on the remote.
|
|
If you are using file name encryption, then when you save files to
|
|
\f[C]secret:subdir/subfile\f[R] this will store them in the unencrypted
|
|
path \f[C]path/to/dir\f[R] but the \f[C]subdir/subpath\f[R] bit will be
|
|
encrypted.
|
|
.PP
|
|
Note that unless you want encrypted bucket names (which are difficult to
|
|
manage because you won\[cq]t know what directory they represent in web
|
|
interfaces etc), you should probably specify a bucket, eg
|
|
\f[C]remote:secretbucket\f[R] when using bucket based remotes such as
|
|
S3, Swift, Hubic, B2, GCS.
|
|
.SS Example
|
|
.PP
|
|
To test I made a little directory of files using \[lq]standard\[rq] file
|
|
name encryption.
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
plaintext/
|
|
\[u251C]\[u2500]\[u2500] file0.txt
|
|
\[u251C]\[u2500]\[u2500] file1.txt
|
|
\[u2514]\[u2500]\[u2500] subdir
|
|
\[u251C]\[u2500]\[u2500] file2.txt
|
|
\[u251C]\[u2500]\[u2500] file3.txt
|
|
\[u2514]\[u2500]\[u2500] subsubdir
|
|
\[u2514]\[u2500]\[u2500] file4.txt
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
Copy these to the remote and list them back
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
$ rclone \-q copy plaintext secret:
|
|
$ rclone \-q ls secret:
|
|
7 file1.txt
|
|
6 file0.txt
|
|
8 subdir/file2.txt
|
|
10 subdir/subsubdir/file4.txt
|
|
9 subdir/file3.txt
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
Now see what that looked like when encrypted
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
$ rclone \-q ls remote:path
|
|
55 hagjclgavj2mbiqm6u6cnjjqcg
|
|
54 v05749mltvv1tf4onltun46gls
|
|
57 86vhrsv86mpbtd3a0akjuqslj8/dlj7fkq4kdq72emafg7a7s41uo
|
|
58 86vhrsv86mpbtd3a0akjuqslj8/7uu829995du6o42n32otfhjqp4/b9pausrfansjth5ob3jkdqd4lc
|
|
56 86vhrsv86mpbtd3a0akjuqslj8/8njh1sk437gttmep3p70g81aps
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
Note that this retains the directory structure which means you can do
|
|
this
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
$ rclone \-q ls secret:subdir
|
|
8 file2.txt
|
|
9 file3.txt
|
|
10 subsubdir/file4.txt
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
If don\[cq]t use file name encryption then the remote will look like
|
|
this \- note the \f[C].bin\f[R] extensions added to prevent the cloud
|
|
provider attempting to interpret the data.
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
$ rclone \-q ls remote:path
|
|
54 file0.txt.bin
|
|
57 subdir/file3.txt.bin
|
|
56 subdir/file2.txt.bin
|
|
58 subdir/subsubdir/file4.txt.bin
|
|
55 file1.txt.bin
|
|
\f[R]
|
|
.fi
|
|
.SS File name encryption modes
|
|
.PP
|
|
Here are some of the features of the file name encryption modes
|
|
.PP
|
|
Off
|
|
.IP \[bu] 2
|
|
doesn\[cq]t hide file names or directory structure
|
|
.IP \[bu] 2
|
|
allows for longer file names (\[ti]246 characters)
|
|
.IP \[bu] 2
|
|
can use sub paths and copy single files
|
|
.PP
|
|
Standard
|
|
.IP \[bu] 2
|
|
file names encrypted
|
|
.IP \[bu] 2
|
|
file names can\[cq]t be as long (\[ti]143 characters)
|
|
.IP \[bu] 2
|
|
can use sub paths and copy single files
|
|
.IP \[bu] 2
|
|
directory structure visible
|
|
.IP \[bu] 2
|
|
identical files names will have identical uploaded names
|
|
.IP \[bu] 2
|
|
can use shortcuts to shorten the directory recursion
|
|
.PP
|
|
Obfuscation
|
|
.PP
|
|
This is a simple \[lq]rotate\[rq] of the filename, with each file having
|
|
a rot distance based on the filename.
|
|
We store the distance at the beginning of the filename.
|
|
So a file called \[lq]hello\[rq] may become \[lq]53.jgnnq\[rq].
|
|
.PP
|
|
This is not a strong encryption of filenames, but it may stop automated
|
|
scanning tools from picking up on filename patterns.
|
|
As such it\[cq]s an intermediate between \[lq]off\[rq] and
|
|
\[lq]standard\[rq].
|
|
The advantage is that it allows for longer path segment names.
|
|
.PP
|
|
There is a possibility with some unicode based filenames that the
|
|
obfuscation is weak and may map lower case characters to upper case
|
|
equivalents.
|
|
You can not rely on this for strong protection.
|
|
.IP \[bu] 2
|
|
file names very lightly obfuscated
|
|
.IP \[bu] 2
|
|
file names can be longer than standard encryption
|
|
.IP \[bu] 2
|
|
can use sub paths and copy single files
|
|
.IP \[bu] 2
|
|
directory structure visible
|
|
.IP \[bu] 2
|
|
identical files names will have identical uploaded names
|
|
.PP
|
|
Cloud storage systems have various limits on file name length and total
|
|
path length which you are more likely to hit using \[lq]Standard\[rq]
|
|
file name encryption.
|
|
If you keep your file names to below 156 characters in length then you
|
|
should be OK on all providers.
|
|
.PP
|
|
There may be an even more secure file name encryption mode in the future
|
|
which will address the long file name problem.
|
|
.SS Directory name encryption
|
|
.PP
|
|
Crypt offers the option of encrypting dir names or leaving them intact.
|
|
There are two options:
|
|
.PP
|
|
True
|
|
.PP
|
|
Encrypts the whole file path including directory names Example:
|
|
\f[C]1/12/123.txt\f[R] is encrypted to
|
|
\f[C]p0e52nreeaj0a5ea7s64m4j72s/l42g6771hnv3an9cgc8cr2n1ng/qgm4avr35m5loi1th53ato71v0\f[R]
|
|
.PP
|
|
False
|
|
.PP
|
|
Only encrypts file names, skips directory names Example:
|
|
\f[C]1/12/123.txt\f[R] is encrypted to
|
|
\f[C]1/12/qgm4avr35m5loi1th53ato71v0\f[R]
|
|
.SS Modified time and hashes
|
|
.PP
|
|
Crypt stores modification times using the underlying remote so support
|
|
depends on that.
|
|
.PP
|
|
Hashes are not stored for crypt.
|
|
However the data integrity is protected by an extremely strong crypto
|
|
authenticator.
|
|
.PP
|
|
Note that you should use the \f[C]rclone cryptcheck\f[R] command to
|
|
check the integrity of a crypted remote instead of
|
|
\f[C]rclone check\f[R] which can\[cq]t check the checksums properly.
|
|
.SS Standard Options
|
|
.PP
|
|
Here are the standard options specific to crypt (Encrypt/Decrypt a
|
|
remote).
|
|
.SS \[en]crypt\-remote
|
|
.PP
|
|
Remote to encrypt/decrypt.
|
|
Normally should contain a `:' and a path, eg
|
|
\[lq]myremote:path/to/dir\[rq], \[lq]myremote:bucket\[rq] or maybe
|
|
\[lq]myremote:\[rq] (not recommended).
|
|
.IP \[bu] 2
|
|
Config: remote
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_CRYPT_REMOTE
|
|
.IP \[bu] 2
|
|
Type: string
|
|
.IP \[bu] 2
|
|
Default: \[dq]\[dq]
|
|
.SS \[en]crypt\-filename\-encryption
|
|
.PP
|
|
How to encrypt the filenames.
|
|
.IP \[bu] 2
|
|
Config: filename_encryption
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_CRYPT_FILENAME_ENCRYPTION
|
|
.IP \[bu] 2
|
|
Type: string
|
|
.IP \[bu] 2
|
|
Default: \[lq]standard\[rq]
|
|
.IP \[bu] 2
|
|
Examples:
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
\[lq]standard\[rq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Encrypt the filenames see the docs for the details.
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[lq]obfuscate\[rq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Very simple filename obfuscation.
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[lq]off\[rq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Don\[cq]t encrypt the file names.
|
|
Adds a \[lq].bin\[rq] extension only.
|
|
.RE
|
|
.RE
|
|
.SS \[en]crypt\-directory\-name\-encryption
|
|
.PP
|
|
Option to either encrypt directory names or leave them intact.
|
|
.PP
|
|
NB If filename_encryption is \[lq]off\[rq] then this option will do
|
|
nothing.
|
|
.IP \[bu] 2
|
|
Config: directory_name_encryption
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_CRYPT_DIRECTORY_NAME_ENCRYPTION
|
|
.IP \[bu] 2
|
|
Type: bool
|
|
.IP \[bu] 2
|
|
Default: true
|
|
.IP \[bu] 2
|
|
Examples:
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
\[lq]true\[rq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Encrypt directory names.
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[lq]false\[rq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Don\[cq]t encrypt directory names, leave them intact.
|
|
.RE
|
|
.RE
|
|
.SS \[en]crypt\-password
|
|
.PP
|
|
Password or pass phrase for encryption.
|
|
.IP \[bu] 2
|
|
Config: password
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_CRYPT_PASSWORD
|
|
.IP \[bu] 2
|
|
Type: string
|
|
.IP \[bu] 2
|
|
Default: \[dq]\[dq]
|
|
.SS \[en]crypt\-password2
|
|
.PP
|
|
Password or pass phrase for salt.
|
|
Optional but recommended.
|
|
Should be different to the previous password.
|
|
.IP \[bu] 2
|
|
Config: password2
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_CRYPT_PASSWORD2
|
|
.IP \[bu] 2
|
|
Type: string
|
|
.IP \[bu] 2
|
|
Default: \[dq]\[dq]
|
|
.SS Advanced Options
|
|
.PP
|
|
Here are the advanced options specific to crypt (Encrypt/Decrypt a
|
|
remote).
|
|
.SS \[en]crypt\-show\-mapping
|
|
.PP
|
|
For all files listed show how the names encrypt.
|
|
.PP
|
|
If this flag is set then for each file that the remote is asked to list,
|
|
it will log (at level INFO) a line stating the decrypted file name and
|
|
the encrypted file name.
|
|
.PP
|
|
This is so you can work out which encrypted names are which decrypted
|
|
names just in case you need to do something with the encrypted file
|
|
names, or for debugging purposes.
|
|
.IP \[bu] 2
|
|
Config: show_mapping
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_CRYPT_SHOW_MAPPING
|
|
.IP \[bu] 2
|
|
Type: bool
|
|
.IP \[bu] 2
|
|
Default: false
|
|
.SS Backend commands
|
|
.PP
|
|
Here are the commands specific to the crypt backend.
|
|
.PP
|
|
Run them with
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone backend COMMAND remote:
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
The help below will explain what arguments each command takes.
|
|
.PP
|
|
See the \[lq]rclone backend\[rq]
|
|
command (https://rclone.org/commands/rclone_backend/) for more info on
|
|
how to pass options and arguments.
|
|
.PP
|
|
These can be run on a running backend using the rc command
|
|
backend/command (https://rclone.org/rc/#backend/command).
|
|
.SS encode
|
|
.PP
|
|
Encode the given filename(s)
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone backend encode remote: [options] [<arguments>+]
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
This encodes the filenames given as arguments returning a list of
|
|
strings of the encoded results.
|
|
.PP
|
|
Usage Example:
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone backend encode crypt: file1 [file2...]
|
|
rclone rc backend/command command=encode fs=crypt: file1 [file2...]
|
|
\f[R]
|
|
.fi
|
|
.SS decode
|
|
.PP
|
|
Decode the given filename(s)
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone backend decode remote: [options] [<arguments>+]
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
This decodes the filenames given as arguments returning a list of
|
|
strings of the decoded results.
|
|
It will return an error if any of the inputs are invalid.
|
|
.PP
|
|
Usage Example:
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone backend decode crypt: encryptedfile1 [encryptedfile2...]
|
|
rclone rc backend/command command=decode fs=crypt: encryptedfile1 [encryptedfile2...]
|
|
\f[R]
|
|
.fi
|
|
.SS Backing up a crypted remote
|
|
.PP
|
|
If you wish to backup a crypted remote, it is recommended that you use
|
|
\f[C]rclone sync\f[R] on the encrypted files, and make sure the
|
|
passwords are the same in the new encrypted remote.
|
|
.PP
|
|
This will have the following advantages
|
|
.IP \[bu] 2
|
|
\f[C]rclone sync\f[R] will check the checksums while copying
|
|
.IP \[bu] 2
|
|
you can use \f[C]rclone check\f[R] between the encrypted remotes
|
|
.IP \[bu] 2
|
|
you don\[cq]t decrypt and encrypt unnecessarily
|
|
.PP
|
|
For example, let\[cq]s say you have your original remote at
|
|
\f[C]remote:\f[R] with the encrypted version at \f[C]eremote:\f[R] with
|
|
path \f[C]remote:crypt\f[R].
|
|
You would then set up the new remote \f[C]remote2:\f[R] and then the
|
|
encrypted version \f[C]eremote2:\f[R] with path \f[C]remote2:crypt\f[R]
|
|
using the same passwords as \f[C]eremote:\f[R].
|
|
.PP
|
|
To sync the two remotes you would do
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone sync remote:crypt remote2:crypt
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
And to check the integrity you would do
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone check remote:crypt remote2:crypt
|
|
\f[R]
|
|
.fi
|
|
.SS File formats
|
|
.SS File encryption
|
|
.PP
|
|
Files are encrypted 1:1 source file to destination object.
|
|
The file has a header and is divided into chunks.
|
|
.SS Header
|
|
.IP \[bu] 2
|
|
8 bytes magic string \f[C]RCLONE\[rs]x00\[rs]x00\f[R]
|
|
.IP \[bu] 2
|
|
24 bytes Nonce (IV)
|
|
.PP
|
|
The initial nonce is generated from the operating systems crypto strong
|
|
random number generator.
|
|
The nonce is incremented for each chunk read making sure each nonce is
|
|
unique for each block written.
|
|
The chance of a nonce being re\-used is minuscule.
|
|
If you wrote an exabyte of data (10\[S1]\[u2078] bytes) you would have a
|
|
probability of approximately 2\[tmu]10\[u207B]\[S3]\[S2] of re\-using a
|
|
nonce.
|
|
.SS Chunk
|
|
.PP
|
|
Each chunk will contain 64kB of data, except for the last one which may
|
|
have less data.
|
|
The data chunk is in standard NACL secretbox format.
|
|
Secretbox uses XSalsa20 and Poly1305 to encrypt and authenticate
|
|
messages.
|
|
.PP
|
|
Each chunk contains:
|
|
.IP \[bu] 2
|
|
16 Bytes of Poly1305 authenticator
|
|
.IP \[bu] 2
|
|
1 \- 65536 bytes XSalsa20 encrypted data
|
|
.PP
|
|
64k chunk size was chosen as the best performing chunk size (the
|
|
authenticator takes too much time below this and the performance drops
|
|
off due to cache effects above this).
|
|
Note that these chunks are buffered in memory so they can\[cq]t be too
|
|
big.
|
|
.PP
|
|
This uses a 32 byte (256 bit key) key derived from the user password.
|
|
.SS Examples
|
|
.PP
|
|
1 byte file will encrypt to
|
|
.IP \[bu] 2
|
|
32 bytes header
|
|
.IP \[bu] 2
|
|
17 bytes data chunk
|
|
.PP
|
|
49 bytes total
|
|
.PP
|
|
1MB (1048576 bytes) file will encrypt to
|
|
.IP \[bu] 2
|
|
32 bytes header
|
|
.IP \[bu] 2
|
|
16 chunks of 65568 bytes
|
|
.PP
|
|
1049120 bytes total (a 0.05% overhead).
|
|
This is the overhead for big files.
|
|
.SS Name encryption
|
|
.PP
|
|
File names are encrypted segment by segment \- the path is broken up
|
|
into \f[C]/\f[R] separated strings and these are encrypted individually.
|
|
.PP
|
|
File segments are padded using PKCS#7 to a multiple of 16 bytes before
|
|
encryption.
|
|
.PP
|
|
They are then encrypted with EME using AES with 256 bit key.
|
|
EME (ECB\-Mix\-ECB) is a wide\-block encryption mode presented in the
|
|
2003 paper \[lq]A Parallelizable Enciphering Mode\[rq] by Halevi and
|
|
Rogaway.
|
|
.PP
|
|
This makes for deterministic encryption which is what we want \- the
|
|
same filename must encrypt to the same thing otherwise we can\[cq]t find
|
|
it on the cloud storage system.
|
|
.PP
|
|
This means that
|
|
.IP \[bu] 2
|
|
filenames with the same name will encrypt the same
|
|
.IP \[bu] 2
|
|
filenames which start the same won\[cq]t have a common prefix
|
|
.PP
|
|
This uses a 32 byte key (256 bits) and a 16 byte (128 bits) IV both of
|
|
which are derived from the user password.
|
|
.PP
|
|
After encryption they are written out using a modified version of
|
|
standard \f[C]base32\f[R] encoding as described in RFC4648.
|
|
The standard encoding is modified in two ways:
|
|
.IP \[bu] 2
|
|
it becomes lower case (no\-one likes upper case filenames!)
|
|
.IP \[bu] 2
|
|
we strip the padding character \f[C]=\f[R]
|
|
.PP
|
|
\f[C]base32\f[R] is used rather than the more efficient \f[C]base64\f[R]
|
|
so rclone can be used on case insensitive remotes (eg Windows, Amazon
|
|
Drive).
|
|
.SS Key derivation
|
|
.PP
|
|
Rclone uses \f[C]scrypt\f[R] with parameters \f[C]N=16384, r=8, p=1\f[R]
|
|
with an optional user supplied salt (password2) to derive the 32+32+16 =
|
|
80 bytes of key material required.
|
|
If the user doesn\[cq]t supply a salt then rclone uses an internal one.
|
|
.PP
|
|
\f[C]scrypt\f[R] makes it impractical to mount a dictionary attack on
|
|
rclone encrypted data.
|
|
For full protection against this you should always use a salt.
|
|
.SS Dropbox
|
|
.PP
|
|
Paths are specified as \f[C]remote:path\f[R]
|
|
.PP
|
|
Dropbox paths may be as deep as required, eg
|
|
\f[C]remote:directory/subdirectory\f[R].
|
|
.PP
|
|
The initial setup for dropbox involves getting a token from Dropbox
|
|
which you need to do in your browser.
|
|
\f[C]rclone config\f[R] walks you through it.
|
|
.PP
|
|
Here is an example of how to make a remote called \f[C]remote\f[R].
|
|
First run:
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone config
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
This will guide you through an interactive setup process:
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
n) New remote
|
|
d) Delete remote
|
|
q) Quit config
|
|
e/n/d/q> n
|
|
name> remote
|
|
Type of storage to configure.
|
|
Choose a number from below, or type in your own value
|
|
[snip]
|
|
XX / Dropbox
|
|
\[rs] \[dq]dropbox\[dq]
|
|
[snip]
|
|
Storage> dropbox
|
|
Dropbox App Key \- leave blank normally.
|
|
app_key>
|
|
Dropbox App Secret \- leave blank normally.
|
|
app_secret>
|
|
Remote config
|
|
Please visit:
|
|
https://www.dropbox.com/1/oauth2/authorize?client_id=XXXXXXXXXXXXXXX&response_type=code
|
|
Enter the code: XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX_XXXXXXXXXX
|
|
\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-
|
|
[remote]
|
|
app_key =
|
|
app_secret =
|
|
token = XXXXXXXXXXXXXXXXXXXXXXXXXXXXX_XXXX_XXXXXXXXXXXXXXXXXXXXXXXXXXXXX
|
|
\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-
|
|
y) Yes this is OK
|
|
e) Edit this remote
|
|
d) Delete this remote
|
|
y/e/d> y
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
You can then use it like this,
|
|
.PP
|
|
List directories in top level of your dropbox
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone lsd remote:
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
List all the files in your dropbox
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone ls remote:
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
To copy a local directory to a dropbox directory called backup
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone copy /home/source remote:backup
|
|
\f[R]
|
|
.fi
|
|
.SS Dropbox for business
|
|
.PP
|
|
Rclone supports Dropbox for business and Team Folders.
|
|
.PP
|
|
When using Dropbox for business \f[C]remote:\f[R] and
|
|
\f[C]remote:path/to/file\f[R] will refer to your personal folder.
|
|
.PP
|
|
If you wish to see Team Folders you must use a leading \f[C]/\f[R] in
|
|
the path, so \f[C]rclone lsd remote:/\f[R] will refer to the root and
|
|
show you all Team Folders and your User Folder.
|
|
.PP
|
|
You can then use team folders like this \f[C]remote:/TeamFolder\f[R] and
|
|
\f[C]remote:/TeamFolder/path/to/file\f[R].
|
|
.PP
|
|
A leading \f[C]/\f[R] for a Dropbox personal account will do nothing,
|
|
but it will take an extra HTTP transaction so it should be avoided.
|
|
.SS Modified time and Hashes
|
|
.PP
|
|
Dropbox supports modified times, but the only way to set a modification
|
|
time is to re\-upload the file.
|
|
.PP
|
|
This means that if you uploaded your data with an older version of
|
|
rclone which didn\[cq]t support the v2 API and modified times, rclone
|
|
will decide to upload all your old data to fix the modification times.
|
|
If you don\[cq]t want this to happen use \f[C]\-\-size\-only\f[R] or
|
|
\f[C]\-\-checksum\f[R] flag to stop it.
|
|
.PP
|
|
Dropbox supports its own hash
|
|
type (https://www.dropbox.com/developers/reference/content-hash) which
|
|
is checked for all transfers.
|
|
.SS Restricted filename characters
|
|
.PP
|
|
.TS
|
|
tab(@);
|
|
l c c.
|
|
T{
|
|
Character
|
|
T}@T{
|
|
Value
|
|
T}@T{
|
|
Replacement
|
|
T}
|
|
_
|
|
T{
|
|
NUL
|
|
T}@T{
|
|
0x00
|
|
T}@T{
|
|
\[u2400]
|
|
T}
|
|
T{
|
|
/
|
|
T}@T{
|
|
0x2F
|
|
T}@T{
|
|
\[uFF0F]
|
|
T}
|
|
T{
|
|
DEL
|
|
T}@T{
|
|
0x7F
|
|
T}@T{
|
|
\[u2421]
|
|
T}
|
|
T{
|
|
\[rs]
|
|
T}@T{
|
|
0x5C
|
|
T}@T{
|
|
\[uFF3C]
|
|
T}
|
|
.TE
|
|
.PP
|
|
File names can also not end with the following characters.
|
|
These only get replaced if they are the last character in the name:
|
|
.PP
|
|
.TS
|
|
tab(@);
|
|
l c c.
|
|
T{
|
|
Character
|
|
T}@T{
|
|
Value
|
|
T}@T{
|
|
Replacement
|
|
T}
|
|
_
|
|
T{
|
|
SP
|
|
T}@T{
|
|
0x20
|
|
T}@T{
|
|
\[u2420]
|
|
T}
|
|
.TE
|
|
.PP
|
|
Invalid UTF\-8 bytes will also be
|
|
replaced (https://rclone.org/overview/#invalid-utf8), as they can\[cq]t
|
|
be used in JSON strings.
|
|
.SS Standard Options
|
|
.PP
|
|
Here are the standard options specific to dropbox (Dropbox).
|
|
.SS \[en]dropbox\-client\-id
|
|
.PP
|
|
Dropbox App Client Id Leave blank normally.
|
|
.IP \[bu] 2
|
|
Config: client_id
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_DROPBOX_CLIENT_ID
|
|
.IP \[bu] 2
|
|
Type: string
|
|
.IP \[bu] 2
|
|
Default: \[dq]\[dq]
|
|
.SS \[en]dropbox\-client\-secret
|
|
.PP
|
|
Dropbox App Client Secret Leave blank normally.
|
|
.IP \[bu] 2
|
|
Config: client_secret
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_DROPBOX_CLIENT_SECRET
|
|
.IP \[bu] 2
|
|
Type: string
|
|
.IP \[bu] 2
|
|
Default: \[dq]\[dq]
|
|
.SS Advanced Options
|
|
.PP
|
|
Here are the advanced options specific to dropbox (Dropbox).
|
|
.SS \[en]dropbox\-chunk\-size
|
|
.PP
|
|
Upload chunk size.
|
|
(< 150M).
|
|
.PP
|
|
Any files larger than this will be uploaded in chunks of this size.
|
|
.PP
|
|
Note that chunks are buffered in memory (one at a time) so rclone can
|
|
deal with retries.
|
|
Setting this larger will increase the speed slightly (at most 10% for
|
|
128MB in tests) at the cost of using more memory.
|
|
It can be set smaller if you are tight on memory.
|
|
.IP \[bu] 2
|
|
Config: chunk_size
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_DROPBOX_CHUNK_SIZE
|
|
.IP \[bu] 2
|
|
Type: SizeSuffix
|
|
.IP \[bu] 2
|
|
Default: 48M
|
|
.SS \[en]dropbox\-impersonate
|
|
.PP
|
|
Impersonate this user when using a business account.
|
|
.IP \[bu] 2
|
|
Config: impersonate
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_DROPBOX_IMPERSONATE
|
|
.IP \[bu] 2
|
|
Type: string
|
|
.IP \[bu] 2
|
|
Default: \[dq]\[dq]
|
|
.SS \[en]dropbox\-encoding
|
|
.PP
|
|
This sets the encoding for the backend.
|
|
.PP
|
|
See: the encoding section in the
|
|
overview (https://rclone.org/overview/#encoding) for more info.
|
|
.IP \[bu] 2
|
|
Config: encoding
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_DROPBOX_ENCODING
|
|
.IP \[bu] 2
|
|
Type: MultiEncoder
|
|
.IP \[bu] 2
|
|
Default: Slash,BackSlash,Del,RightSpace,InvalidUtf8,Dot
|
|
.SS Limitations
|
|
.PP
|
|
Note that Dropbox is case insensitive so you can\[cq]t have a file
|
|
called \[lq]Hello.doc\[rq] and one called \[lq]hello.doc\[rq].
|
|
.PP
|
|
There are some file names such as \f[C]thumbs.db\f[R] which Dropbox
|
|
can\[cq]t store.
|
|
There is a full list of them in the \[lq]Ignored Files\[rq] section of
|
|
this document (https://www.dropbox.com/en/help/145).
|
|
Rclone will issue an error message
|
|
\f[C]File name disallowed \- not uploading\f[R] if it attempts to upload
|
|
one of those file names, but the sync won\[cq]t fail.
|
|
.PP
|
|
If you have more than 10,000 files in a directory then
|
|
\f[C]rclone purge dropbox:dir\f[R] will return the error
|
|
\f[C]Failed to purge: There are too many files involved in this operation\f[R].
|
|
As a work\-around do an \f[C]rclone delete dropbox:dir\f[R] followed by
|
|
an \f[C]rclone rmdir dropbox:dir\f[R].
|
|
.SS Get your own Dropbox App ID
|
|
.PP
|
|
When you use rclone with Dropbox in its default configuration you are
|
|
using rclone\[cq]s App ID.
|
|
This is shared between all the rclone users.
|
|
.PP
|
|
Here is how to create your own Dropbox App ID for rclone:
|
|
.IP "1." 3
|
|
Log into the Dropbox App
|
|
console (https://www.dropbox.com/developers/apps/create) with your
|
|
Dropbox Account (It need not to be the same account as the Dropbox you
|
|
want to access)
|
|
.IP "2." 3
|
|
Choose an API => Usually this should be \f[C]Dropbox API\f[R]
|
|
.IP "3." 3
|
|
Choose the type of access you want to use => \f[C]Full Dropbox\f[R] or
|
|
\f[C]App Folder\f[R]
|
|
.IP "4." 3
|
|
Name your App.
|
|
The app name is global, so you can\[cq]t use \f[C]rclone\f[R] for
|
|
example
|
|
.IP "5." 3
|
|
Click the button \f[C]Create App\f[R]
|
|
.IP "6." 3
|
|
Fill \f[C]Redirect URIs\f[R] as \f[C]http://localhost:53682/\f[R]
|
|
.IP "7." 3
|
|
Find the \f[C]App key\f[R] and \f[C]App secret\f[R] Use these values in
|
|
rclone config to add a new remote or edit an existing remote.
|
|
.SS FTP
|
|
.PP
|
|
FTP is the File Transfer Protocol.
|
|
FTP support is provided using the
|
|
github.com/jlaffaye/ftp (https://godoc.org/github.com/jlaffaye/ftp)
|
|
package.
|
|
.PP
|
|
Here is an example of making an FTP configuration.
|
|
First run
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone config
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
This will guide you through an interactive setup process.
|
|
An FTP remote only needs a host together with and a username and a
|
|
password.
|
|
With anonymous FTP server, you will need to use \f[C]anonymous\f[R] as
|
|
username and your email address as the password.
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
No remotes found \- make a new one
|
|
n) New remote
|
|
r) Rename remote
|
|
c) Copy remote
|
|
s) Set configuration password
|
|
q) Quit config
|
|
n/r/c/s/q> n
|
|
name> remote
|
|
Type of storage to configure.
|
|
Enter a string value. Press Enter for the default (\[dq]\[dq]).
|
|
Choose a number from below, or type in your own value
|
|
[snip]
|
|
XX / FTP Connection
|
|
\[rs] \[dq]ftp\[dq]
|
|
[snip]
|
|
Storage> ftp
|
|
** See help for ftp backend at: https://rclone.org/ftp/ **
|
|
|
|
FTP host to connect to
|
|
Enter a string value. Press Enter for the default (\[dq]\[dq]).
|
|
Choose a number from below, or type in your own value
|
|
1 / Connect to ftp.example.com
|
|
\[rs] \[dq]ftp.example.com\[dq]
|
|
host> ftp.example.com
|
|
FTP username, leave blank for current username, ncw
|
|
Enter a string value. Press Enter for the default (\[dq]\[dq]).
|
|
user>
|
|
FTP port, leave blank to use default (21)
|
|
Enter a string value. Press Enter for the default (\[dq]\[dq]).
|
|
port>
|
|
FTP password
|
|
y) Yes type in my own password
|
|
g) Generate random password
|
|
y/g> y
|
|
Enter the password:
|
|
password:
|
|
Confirm the password:
|
|
password:
|
|
Use FTP over TLS (Implicit)
|
|
Enter a boolean value (true or false). Press Enter for the default (\[dq]false\[dq]).
|
|
tls>
|
|
Remote config
|
|
\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-
|
|
[remote]
|
|
type = ftp
|
|
host = ftp.example.com
|
|
pass = *** ENCRYPTED ***
|
|
\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-
|
|
y) Yes this is OK
|
|
e) Edit this remote
|
|
d) Delete this remote
|
|
y/e/d> y
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
This remote is called \f[C]remote\f[R] and can now be used like this
|
|
.PP
|
|
See all directories in the home directory
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone lsd remote:
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
Make a new directory
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone mkdir remote:path/to/directory
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
List the contents of a directory
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone ls remote:path/to/directory
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
Sync \f[C]/home/local/directory\f[R] to the remote directory, deleting
|
|
any excess files in the directory.
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone sync /home/local/directory remote:directory
|
|
\f[R]
|
|
.fi
|
|
.SS Modified time
|
|
.PP
|
|
FTP does not support modified times.
|
|
Any times you see on the server will be time of upload.
|
|
.SS Checksums
|
|
.PP
|
|
FTP does not support any checksums.
|
|
.SS Usage without a config file
|
|
.PP
|
|
An example how to use the ftp remote without a config file:
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone lsf :ftp: \-\-ftp\-host=speedtest.tele2.net \-\-ftp\-user=anonymous \-\-ftp\-pass=\[ga]rclone obscure dummy\[ga]
|
|
\f[R]
|
|
.fi
|
|
.SS Restricted filename characters
|
|
.PP
|
|
In addition to the default restricted characters
|
|
set (https://rclone.org/overview/#restricted-characters) the following
|
|
characters are also replaced:
|
|
.PP
|
|
File names can also not end with the following characters.
|
|
These only get replaced if they are the last character in the name:
|
|
.PP
|
|
.TS
|
|
tab(@);
|
|
l c c.
|
|
T{
|
|
Character
|
|
T}@T{
|
|
Value
|
|
T}@T{
|
|
Replacement
|
|
T}
|
|
_
|
|
T{
|
|
SP
|
|
T}@T{
|
|
0x20
|
|
T}@T{
|
|
\[u2420]
|
|
T}
|
|
.TE
|
|
.PP
|
|
Note that not all FTP servers can have all characters in file names, for
|
|
example:
|
|
.PP
|
|
.TS
|
|
tab(@);
|
|
l c.
|
|
T{
|
|
FTP Server
|
|
T}@T{
|
|
Forbidden characters
|
|
T}
|
|
_
|
|
T{
|
|
proftpd
|
|
T}@T{
|
|
\f[C]*\f[R]
|
|
T}
|
|
T{
|
|
pureftpd
|
|
T}@T{
|
|
\f[C]\[rs] [ ]\f[R]
|
|
T}
|
|
.TE
|
|
.SS Implicit TLS
|
|
.PP
|
|
FTP supports implicit FTP over TLS servers (FTPS).
|
|
This has to be enabled in the config for the remote.
|
|
The default FTPS port is \f[C]990\f[R] so the port will likely have to
|
|
be explicitly set in the config for the remote.
|
|
.SS Standard Options
|
|
.PP
|
|
Here are the standard options specific to ftp (FTP Connection).
|
|
.SS \[en]ftp\-host
|
|
.PP
|
|
FTP host to connect to
|
|
.IP \[bu] 2
|
|
Config: host
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_FTP_HOST
|
|
.IP \[bu] 2
|
|
Type: string
|
|
.IP \[bu] 2
|
|
Default: \[dq]\[dq]
|
|
.IP \[bu] 2
|
|
Examples:
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
\[lq]ftp.example.com\[rq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Connect to ftp.example.com
|
|
.RE
|
|
.RE
|
|
.SS \[en]ftp\-user
|
|
.PP
|
|
FTP username, leave blank for current username, $USER
|
|
.IP \[bu] 2
|
|
Config: user
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_FTP_USER
|
|
.IP \[bu] 2
|
|
Type: string
|
|
.IP \[bu] 2
|
|
Default: \[dq]\[dq]
|
|
.SS \[en]ftp\-port
|
|
.PP
|
|
FTP port, leave blank to use default (21)
|
|
.IP \[bu] 2
|
|
Config: port
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_FTP_PORT
|
|
.IP \[bu] 2
|
|
Type: string
|
|
.IP \[bu] 2
|
|
Default: \[dq]\[dq]
|
|
.SS \[en]ftp\-pass
|
|
.PP
|
|
FTP password
|
|
.IP \[bu] 2
|
|
Config: pass
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_FTP_PASS
|
|
.IP \[bu] 2
|
|
Type: string
|
|
.IP \[bu] 2
|
|
Default: \[dq]\[dq]
|
|
.SS \[en]ftp\-tls
|
|
.PP
|
|
Use FTP over TLS (Implicit)
|
|
.IP \[bu] 2
|
|
Config: tls
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_FTP_TLS
|
|
.IP \[bu] 2
|
|
Type: bool
|
|
.IP \[bu] 2
|
|
Default: false
|
|
.SS Advanced Options
|
|
.PP
|
|
Here are the advanced options specific to ftp (FTP Connection).
|
|
.SS \[en]ftp\-concurrency
|
|
.PP
|
|
Maximum number of FTP simultaneous connections, 0 for unlimited
|
|
.IP \[bu] 2
|
|
Config: concurrency
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_FTP_CONCURRENCY
|
|
.IP \[bu] 2
|
|
Type: int
|
|
.IP \[bu] 2
|
|
Default: 0
|
|
.SS \[en]ftp\-no\-check\-certificate
|
|
.PP
|
|
Do not verify the TLS certificate of the server
|
|
.IP \[bu] 2
|
|
Config: no_check_certificate
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_FTP_NO_CHECK_CERTIFICATE
|
|
.IP \[bu] 2
|
|
Type: bool
|
|
.IP \[bu] 2
|
|
Default: false
|
|
.SS \[en]ftp\-disable\-epsv
|
|
.PP
|
|
Disable using EPSV even if server advertises support
|
|
.IP \[bu] 2
|
|
Config: disable_epsv
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_FTP_DISABLE_EPSV
|
|
.IP \[bu] 2
|
|
Type: bool
|
|
.IP \[bu] 2
|
|
Default: false
|
|
.SS \[en]ftp\-encoding
|
|
.PP
|
|
This sets the encoding for the backend.
|
|
.PP
|
|
See: the encoding section in the
|
|
overview (https://rclone.org/overview/#encoding) for more info.
|
|
.IP \[bu] 2
|
|
Config: encoding
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_FTP_ENCODING
|
|
.IP \[bu] 2
|
|
Type: MultiEncoder
|
|
.IP \[bu] 2
|
|
Default: Slash,Del,Ctl,RightSpace,Dot
|
|
.SS Limitations
|
|
.PP
|
|
Note that since FTP isn\[cq]t HTTP based the following flags don\[cq]t
|
|
work with it: \f[C]\-\-dump\-headers\f[R], \f[C]\-\-dump\-bodies\f[R],
|
|
\f[C]\-\-dump\-auth\f[R]
|
|
.PP
|
|
Note that \f[C]\-\-timeout\f[R] isn\[cq]t supported (but
|
|
\f[C]\-\-contimeout\f[R] is).
|
|
.PP
|
|
Note that \f[C]\-\-bind\f[R] isn\[cq]t supported.
|
|
.PP
|
|
FTP could support server side move but doesn\[cq]t yet.
|
|
.PP
|
|
Note that the ftp backend does not support the \f[C]ftp_proxy\f[R]
|
|
environment variable yet.
|
|
.PP
|
|
Note that while implicit FTP over TLS is supported, explicit FTP over
|
|
TLS is not.
|
|
.SS Google Cloud Storage
|
|
.PP
|
|
Paths are specified as \f[C]remote:bucket\f[R] (or \f[C]remote:\f[R] for
|
|
the \f[C]lsd\f[R] command.) You may put subdirectories in too, eg
|
|
\f[C]remote:bucket/path/to/dir\f[R].
|
|
.PP
|
|
The initial setup for google cloud storage involves getting a token from
|
|
Google Cloud Storage which you need to do in your browser.
|
|
\f[C]rclone config\f[R] walks you through it.
|
|
.PP
|
|
Here is an example of how to make a remote called \f[C]remote\f[R].
|
|
First run:
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone config
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
This will guide you through an interactive setup process:
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
n) New remote
|
|
d) Delete remote
|
|
q) Quit config
|
|
e/n/d/q> n
|
|
name> remote
|
|
Type of storage to configure.
|
|
Choose a number from below, or type in your own value
|
|
[snip]
|
|
XX / Google Cloud Storage (this is not Google Drive)
|
|
\[rs] \[dq]google cloud storage\[dq]
|
|
[snip]
|
|
Storage> google cloud storage
|
|
Google Application Client Id \- leave blank normally.
|
|
client_id>
|
|
Google Application Client Secret \- leave blank normally.
|
|
client_secret>
|
|
Project number optional \- needed only for list/create/delete buckets \- see your developer console.
|
|
project_number> 12345678
|
|
Service Account Credentials JSON file path \- needed only if you want use SA instead of interactive login.
|
|
service_account_file>
|
|
Access Control List for new objects.
|
|
Choose a number from below, or type in your own value
|
|
1 / Object owner gets OWNER access, and all Authenticated Users get READER access.
|
|
\[rs] \[dq]authenticatedRead\[dq]
|
|
2 / Object owner gets OWNER access, and project team owners get OWNER access.
|
|
\[rs] \[dq]bucketOwnerFullControl\[dq]
|
|
3 / Object owner gets OWNER access, and project team owners get READER access.
|
|
\[rs] \[dq]bucketOwnerRead\[dq]
|
|
4 / Object owner gets OWNER access [default if left blank].
|
|
\[rs] \[dq]private\[dq]
|
|
5 / Object owner gets OWNER access, and project team members get access according to their roles.
|
|
\[rs] \[dq]projectPrivate\[dq]
|
|
6 / Object owner gets OWNER access, and all Users get READER access.
|
|
\[rs] \[dq]publicRead\[dq]
|
|
object_acl> 4
|
|
Access Control List for new buckets.
|
|
Choose a number from below, or type in your own value
|
|
1 / Project team owners get OWNER access, and all Authenticated Users get READER access.
|
|
\[rs] \[dq]authenticatedRead\[dq]
|
|
2 / Project team owners get OWNER access [default if left blank].
|
|
\[rs] \[dq]private\[dq]
|
|
3 / Project team members get access according to their roles.
|
|
\[rs] \[dq]projectPrivate\[dq]
|
|
4 / Project team owners get OWNER access, and all Users get READER access.
|
|
\[rs] \[dq]publicRead\[dq]
|
|
5 / Project team owners get OWNER access, and all Users get WRITER access.
|
|
\[rs] \[dq]publicReadWrite\[dq]
|
|
bucket_acl> 2
|
|
Location for the newly created buckets.
|
|
Choose a number from below, or type in your own value
|
|
1 / Empty for default location (US).
|
|
\[rs] \[dq]\[dq]
|
|
2 / Multi\-regional location for Asia.
|
|
\[rs] \[dq]asia\[dq]
|
|
3 / Multi\-regional location for Europe.
|
|
\[rs] \[dq]eu\[dq]
|
|
4 / Multi\-regional location for United States.
|
|
\[rs] \[dq]us\[dq]
|
|
5 / Taiwan.
|
|
\[rs] \[dq]asia\-east1\[dq]
|
|
6 / Tokyo.
|
|
\[rs] \[dq]asia\-northeast1\[dq]
|
|
7 / Singapore.
|
|
\[rs] \[dq]asia\-southeast1\[dq]
|
|
8 / Sydney.
|
|
\[rs] \[dq]australia\-southeast1\[dq]
|
|
9 / Belgium.
|
|
\[rs] \[dq]europe\-west1\[dq]
|
|
10 / London.
|
|
\[rs] \[dq]europe\-west2\[dq]
|
|
11 / Iowa.
|
|
\[rs] \[dq]us\-central1\[dq]
|
|
12 / South Carolina.
|
|
\[rs] \[dq]us\-east1\[dq]
|
|
13 / Northern Virginia.
|
|
\[rs] \[dq]us\-east4\[dq]
|
|
14 / Oregon.
|
|
\[rs] \[dq]us\-west1\[dq]
|
|
location> 12
|
|
The storage class to use when storing objects in Google Cloud Storage.
|
|
Choose a number from below, or type in your own value
|
|
1 / Default
|
|
\[rs] \[dq]\[dq]
|
|
2 / Multi\-regional storage class
|
|
\[rs] \[dq]MULTI_REGIONAL\[dq]
|
|
3 / Regional storage class
|
|
\[rs] \[dq]REGIONAL\[dq]
|
|
4 / Nearline storage class
|
|
\[rs] \[dq]NEARLINE\[dq]
|
|
5 / Coldline storage class
|
|
\[rs] \[dq]COLDLINE\[dq]
|
|
6 / Durable reduced availability storage class
|
|
\[rs] \[dq]DURABLE_REDUCED_AVAILABILITY\[dq]
|
|
storage_class> 5
|
|
Remote config
|
|
Use auto config?
|
|
* Say Y if not sure
|
|
* Say N if you are working on a remote or headless machine or Y didn\[aq]t work
|
|
y) Yes
|
|
n) No
|
|
y/n> y
|
|
If your browser doesn\[aq]t open automatically go to the following link: http://127.0.0.1:53682/auth
|
|
Log in and authorize rclone for access
|
|
Waiting for code...
|
|
Got code
|
|
\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-
|
|
[remote]
|
|
type = google cloud storage
|
|
client_id =
|
|
client_secret =
|
|
token = {\[dq]AccessToken\[dq]:\[dq]xxxx.xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx\-xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx\[dq],\[dq]RefreshToken\[dq]:\[dq]x/xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx_xxxxxxxxx\[dq],\[dq]Expiry\[dq]:\[dq]2014\-07\-17T20:49:14.929208288+01:00\[dq],\[dq]Extra\[dq]:null}
|
|
project_number = 12345678
|
|
object_acl = private
|
|
bucket_acl = private
|
|
\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-
|
|
y) Yes this is OK
|
|
e) Edit this remote
|
|
d) Delete this remote
|
|
y/e/d> y
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
Note that rclone runs a webserver on your local machine to collect the
|
|
token as returned from Google if you use auto config mode.
|
|
This only runs from the moment it opens your browser to the moment you
|
|
get back the verification code.
|
|
This is on \f[C]http://127.0.0.1:53682/\f[R] and this it may require you
|
|
to unblock it temporarily if you are running a host firewall, or use
|
|
manual mode.
|
|
.PP
|
|
This remote is called \f[C]remote\f[R] and can now be used like this
|
|
.PP
|
|
See all the buckets in your project
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone lsd remote:
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
Make a new bucket
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone mkdir remote:bucket
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
List the contents of a bucket
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone ls remote:bucket
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
Sync \f[C]/home/local/directory\f[R] to the remote bucket, deleting any
|
|
excess files in the bucket.
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone sync /home/local/directory remote:bucket
|
|
\f[R]
|
|
.fi
|
|
.SS Service Account support
|
|
.PP
|
|
You can set up rclone with Google Cloud Storage in an unattended mode,
|
|
i.e.\ not tied to a specific end\-user Google account.
|
|
This is useful when you want to synchronise files onto machines that
|
|
don\[cq]t have actively logged\-in users, for example build machines.
|
|
.PP
|
|
To get credentials for Google Cloud Platform IAM Service
|
|
Accounts (https://cloud.google.com/iam/docs/service-accounts), please
|
|
head to the Service
|
|
Account (https://console.cloud.google.com/permissions/serviceaccounts)
|
|
section of the Google Developer Console.
|
|
Service Accounts behave just like normal \f[C]User\f[R] permissions in
|
|
Google Cloud Storage
|
|
ACLs (https://cloud.google.com/storage/docs/access-control), so you can
|
|
limit their access (e.g.\ make them read only).
|
|
After creating an account, a JSON file containing the Service
|
|
Account\[cq]s credentials will be downloaded onto your machines.
|
|
These credentials are what rclone will use for authentication.
|
|
.PP
|
|
To use a Service Account instead of OAuth2 token flow, enter the path to
|
|
your Service Account credentials at the \f[C]service_account_file\f[R]
|
|
prompt and rclone won\[cq]t use the browser based authentication flow.
|
|
If you\[cq]d rather stuff the contents of the credentials file into the
|
|
rclone config file, you can set \f[C]service_account_credentials\f[R]
|
|
with the actual contents of the file instead, or set the equivalent
|
|
environment variable.
|
|
.SS Application Default Credentials
|
|
.PP
|
|
If no other source of credentials is provided, rclone will fall back to
|
|
Application Default
|
|
Credentials (https://cloud.google.com/video-intelligence/docs/common/auth#authenticating_with_application_default_credentials)
|
|
this is useful both when you already have configured authentication for
|
|
your developer account, or in production when running on a google
|
|
compute host.
|
|
Note that if running in docker, you may need to run additional commands
|
|
on your google compute machine \- see this
|
|
page (https://cloud.google.com/container-registry/docs/advanced-authentication#gcloud_as_a_docker_credential_helper).
|
|
.PP
|
|
Note that in the case application default credentials are used, there is
|
|
no need to explicitly configure a project number.
|
|
.SS \[en]fast\-list
|
|
.PP
|
|
This remote supports \f[C]\-\-fast\-list\f[R] which allows you to use
|
|
fewer transactions in exchange for more memory.
|
|
See the rclone docs (https://rclone.org/docs/#fast-list) for more
|
|
details.
|
|
.SS Custom upload headers
|
|
.PP
|
|
You can set custom upload headers with the \f[C]\-\-header\-upload\f[R]
|
|
flag.
|
|
Google Cloud Storage supports the headers as described in the working
|
|
with metadata
|
|
documentation (https://cloud.google.com/storage/docs/gsutil/addlhelp/WorkingWithObjectMetadata)
|
|
.IP \[bu] 2
|
|
Cache\-Control
|
|
.IP \[bu] 2
|
|
Content\-Disposition
|
|
.IP \[bu] 2
|
|
Content\-Encoding
|
|
.IP \[bu] 2
|
|
Content\-Language
|
|
.IP \[bu] 2
|
|
Content\-Type
|
|
.IP \[bu] 2
|
|
X\-Goog\-Meta\-
|
|
.PP
|
|
Eg \f[C]\-\-header\-upload \[dq]Content\-Type text/potato\[dq]\f[R]
|
|
.PP
|
|
Note that the last of these is for setting custom metadata in the form
|
|
\f[C]\-\-header\-upload \[dq]x\-goog\-meta\-key: value\[dq]\f[R]
|
|
.SS Modified time
|
|
.PP
|
|
Google google cloud storage stores md5sums natively and rclone stores
|
|
modification times as metadata on the object, under the \[lq]mtime\[rq]
|
|
key in RFC3339 format accurate to 1ns.
|
|
.SS Restricted filename characters
|
|
.PP
|
|
.TS
|
|
tab(@);
|
|
l c c.
|
|
T{
|
|
Character
|
|
T}@T{
|
|
Value
|
|
T}@T{
|
|
Replacement
|
|
T}
|
|
_
|
|
T{
|
|
NUL
|
|
T}@T{
|
|
0x00
|
|
T}@T{
|
|
\[u2400]
|
|
T}
|
|
T{
|
|
LF
|
|
T}@T{
|
|
0x0A
|
|
T}@T{
|
|
\[u240A]
|
|
T}
|
|
T{
|
|
CR
|
|
T}@T{
|
|
0x0D
|
|
T}@T{
|
|
\[u240D]
|
|
T}
|
|
T{
|
|
/
|
|
T}@T{
|
|
0x2F
|
|
T}@T{
|
|
\[uFF0F]
|
|
T}
|
|
.TE
|
|
.PP
|
|
Invalid UTF\-8 bytes will also be
|
|
replaced (https://rclone.org/overview/#invalid-utf8), as they can\[cq]t
|
|
be used in JSON strings.
|
|
.SS Standard Options
|
|
.PP
|
|
Here are the standard options specific to google cloud storage (Google
|
|
Cloud Storage (this is not Google Drive)).
|
|
.SS \[en]gcs\-client\-id
|
|
.PP
|
|
Google Application Client Id Leave blank normally.
|
|
.IP \[bu] 2
|
|
Config: client_id
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_GCS_CLIENT_ID
|
|
.IP \[bu] 2
|
|
Type: string
|
|
.IP \[bu] 2
|
|
Default: \[dq]\[dq]
|
|
.SS \[en]gcs\-client\-secret
|
|
.PP
|
|
Google Application Client Secret Leave blank normally.
|
|
.IP \[bu] 2
|
|
Config: client_secret
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_GCS_CLIENT_SECRET
|
|
.IP \[bu] 2
|
|
Type: string
|
|
.IP \[bu] 2
|
|
Default: \[dq]\[dq]
|
|
.SS \[en]gcs\-project\-number
|
|
.PP
|
|
Project number.
|
|
Optional \- needed only for list/create/delete buckets \- see your
|
|
developer console.
|
|
.IP \[bu] 2
|
|
Config: project_number
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_GCS_PROJECT_NUMBER
|
|
.IP \[bu] 2
|
|
Type: string
|
|
.IP \[bu] 2
|
|
Default: \[dq]\[dq]
|
|
.SS \[en]gcs\-service\-account\-file
|
|
.PP
|
|
Service Account Credentials JSON file path Leave blank normally.
|
|
Needed only if you want use SA instead of interactive login.
|
|
.IP \[bu] 2
|
|
Config: service_account_file
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_GCS_SERVICE_ACCOUNT_FILE
|
|
.IP \[bu] 2
|
|
Type: string
|
|
.IP \[bu] 2
|
|
Default: \[dq]\[dq]
|
|
.SS \[en]gcs\-service\-account\-credentials
|
|
.PP
|
|
Service Account Credentials JSON blob Leave blank normally.
|
|
Needed only if you want use SA instead of interactive login.
|
|
.IP \[bu] 2
|
|
Config: service_account_credentials
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_GCS_SERVICE_ACCOUNT_CREDENTIALS
|
|
.IP \[bu] 2
|
|
Type: string
|
|
.IP \[bu] 2
|
|
Default: \[dq]\[dq]
|
|
.SS \[en]gcs\-object\-acl
|
|
.PP
|
|
Access Control List for new objects.
|
|
.IP \[bu] 2
|
|
Config: object_acl
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_GCS_OBJECT_ACL
|
|
.IP \[bu] 2
|
|
Type: string
|
|
.IP \[bu] 2
|
|
Default: \[dq]\[dq]
|
|
.IP \[bu] 2
|
|
Examples:
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
\[lq]authenticatedRead\[rq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Object owner gets OWNER access, and all Authenticated Users get READER
|
|
access.
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[lq]bucketOwnerFullControl\[rq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Object owner gets OWNER access, and project team owners get OWNER
|
|
access.
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[lq]bucketOwnerRead\[rq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Object owner gets OWNER access, and project team owners get READER
|
|
access.
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[lq]private\[rq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Object owner gets OWNER access [default if left blank].
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[lq]projectPrivate\[rq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Object owner gets OWNER access, and project team members get access
|
|
according to their roles.
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[lq]publicRead\[rq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Object owner gets OWNER access, and all Users get READER access.
|
|
.RE
|
|
.RE
|
|
.SS \[en]gcs\-bucket\-acl
|
|
.PP
|
|
Access Control List for new buckets.
|
|
.IP \[bu] 2
|
|
Config: bucket_acl
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_GCS_BUCKET_ACL
|
|
.IP \[bu] 2
|
|
Type: string
|
|
.IP \[bu] 2
|
|
Default: \[dq]\[dq]
|
|
.IP \[bu] 2
|
|
Examples:
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
\[lq]authenticatedRead\[rq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Project team owners get OWNER access, and all Authenticated Users get
|
|
READER access.
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[lq]private\[rq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Project team owners get OWNER access [default if left blank].
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[lq]projectPrivate\[rq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Project team members get access according to their roles.
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[lq]publicRead\[rq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Project team owners get OWNER access, and all Users get READER access.
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[lq]publicReadWrite\[rq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Project team owners get OWNER access, and all Users get WRITER access.
|
|
.RE
|
|
.RE
|
|
.SS \[en]gcs\-bucket\-policy\-only
|
|
.PP
|
|
Access checks should use bucket\-level IAM policies.
|
|
.PP
|
|
If you want to upload objects to a bucket with Bucket Policy Only set
|
|
then you will need to set this.
|
|
.PP
|
|
When it is set, rclone:
|
|
.IP \[bu] 2
|
|
ignores ACLs set on buckets
|
|
.IP \[bu] 2
|
|
ignores ACLs set on objects
|
|
.IP \[bu] 2
|
|
creates buckets with Bucket Policy Only set
|
|
.PP
|
|
Docs: https://cloud.google.com/storage/docs/bucket\-policy\-only
|
|
.IP \[bu] 2
|
|
Config: bucket_policy_only
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_GCS_BUCKET_POLICY_ONLY
|
|
.IP \[bu] 2
|
|
Type: bool
|
|
.IP \[bu] 2
|
|
Default: false
|
|
.SS \[en]gcs\-location
|
|
.PP
|
|
Location for the newly created buckets.
|
|
.IP \[bu] 2
|
|
Config: location
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_GCS_LOCATION
|
|
.IP \[bu] 2
|
|
Type: string
|
|
.IP \[bu] 2
|
|
Default: \[dq]\[dq]
|
|
.IP \[bu] 2
|
|
Examples:
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
\[dq]\[dq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Empty for default location (US).
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[lq]asia\[rq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Multi\-regional location for Asia.
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[lq]eu\[rq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Multi\-regional location for Europe.
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[lq]us\[rq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Multi\-regional location for United States.
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[lq]asia\-east1\[rq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Taiwan.
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[lq]asia\-east2\[rq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Hong Kong.
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[lq]asia\-northeast1\[rq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Tokyo.
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[lq]asia\-south1\[rq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Mumbai.
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[lq]asia\-southeast1\[rq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Singapore.
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[lq]australia\-southeast1\[rq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Sydney.
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[lq]europe\-north1\[rq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Finland.
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[lq]europe\-west1\[rq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Belgium.
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[lq]europe\-west2\[rq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
London.
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[lq]europe\-west3\[rq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Frankfurt.
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[lq]europe\-west4\[rq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Netherlands.
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[lq]us\-central1\[rq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Iowa.
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[lq]us\-east1\[rq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
South Carolina.
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[lq]us\-east4\[rq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Northern Virginia.
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[lq]us\-west1\[rq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Oregon.
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[lq]us\-west2\[rq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
California.
|
|
.RE
|
|
.RE
|
|
.SS \[en]gcs\-storage\-class
|
|
.PP
|
|
The storage class to use when storing objects in Google Cloud Storage.
|
|
.IP \[bu] 2
|
|
Config: storage_class
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_GCS_STORAGE_CLASS
|
|
.IP \[bu] 2
|
|
Type: string
|
|
.IP \[bu] 2
|
|
Default: \[dq]\[dq]
|
|
.IP \[bu] 2
|
|
Examples:
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
\[dq]\[dq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Default
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[lq]MULTI_REGIONAL\[rq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Multi\-regional storage class
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[lq]REGIONAL\[rq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Regional storage class
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[lq]NEARLINE\[rq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Nearline storage class
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[lq]COLDLINE\[rq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Coldline storage class
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[lq]ARCHIVE\[rq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Archive storage class
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[lq]DURABLE_REDUCED_AVAILABILITY\[rq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Durable reduced availability storage class
|
|
.RE
|
|
.RE
|
|
.SS Advanced Options
|
|
.PP
|
|
Here are the advanced options specific to google cloud storage (Google
|
|
Cloud Storage (this is not Google Drive)).
|
|
.SS \[en]gcs\-encoding
|
|
.PP
|
|
This sets the encoding for the backend.
|
|
.PP
|
|
See: the encoding section in the
|
|
overview (https://rclone.org/overview/#encoding) for more info.
|
|
.IP \[bu] 2
|
|
Config: encoding
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_GCS_ENCODING
|
|
.IP \[bu] 2
|
|
Type: MultiEncoder
|
|
.IP \[bu] 2
|
|
Default: Slash,CrLf,InvalidUtf8,Dot
|
|
.SS Google Drive
|
|
.PP
|
|
Paths are specified as \f[C]drive:path\f[R]
|
|
.PP
|
|
Drive paths may be as deep as required, eg
|
|
\f[C]drive:directory/subdirectory\f[R].
|
|
.PP
|
|
The initial setup for drive involves getting a token from Google drive
|
|
which you need to do in your browser.
|
|
\f[C]rclone config\f[R] walks you through it.
|
|
.PP
|
|
Here is an example of how to make a remote called \f[C]remote\f[R].
|
|
First run:
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone config
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
This will guide you through an interactive setup process:
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
No remotes found \- make a new one
|
|
n) New remote
|
|
r) Rename remote
|
|
c) Copy remote
|
|
s) Set configuration password
|
|
q) Quit config
|
|
n/r/c/s/q> n
|
|
name> remote
|
|
Type of storage to configure.
|
|
Choose a number from below, or type in your own value
|
|
[snip]
|
|
XX / Google Drive
|
|
\[rs] \[dq]drive\[dq]
|
|
[snip]
|
|
Storage> drive
|
|
Google Application Client Id \- leave blank normally.
|
|
client_id>
|
|
Google Application Client Secret \- leave blank normally.
|
|
client_secret>
|
|
Scope that rclone should use when requesting access from drive.
|
|
Choose a number from below, or type in your own value
|
|
1 / Full access all files, excluding Application Data Folder.
|
|
\[rs] \[dq]drive\[dq]
|
|
2 / Read\-only access to file metadata and file contents.
|
|
\[rs] \[dq]drive.readonly\[dq]
|
|
/ Access to files created by rclone only.
|
|
3 | These are visible in the drive website.
|
|
| File authorization is revoked when the user deauthorizes the app.
|
|
\[rs] \[dq]drive.file\[dq]
|
|
/ Allows read and write access to the Application Data folder.
|
|
4 | This is not visible in the drive website.
|
|
\[rs] \[dq]drive.appfolder\[dq]
|
|
/ Allows read\-only access to file metadata but
|
|
5 | does not allow any access to read or download file content.
|
|
\[rs] \[dq]drive.metadata.readonly\[dq]
|
|
scope> 1
|
|
ID of the root folder \- leave blank normally. Fill in to access \[dq]Computers\[dq] folders. (see docs).
|
|
root_folder_id>
|
|
Service Account Credentials JSON file path \- needed only if you want use SA instead of interactive login.
|
|
service_account_file>
|
|
Remote config
|
|
Use auto config?
|
|
* Say Y if not sure
|
|
* Say N if you are working on a remote or headless machine or Y didn\[aq]t work
|
|
y) Yes
|
|
n) No
|
|
y/n> y
|
|
If your browser doesn\[aq]t open automatically go to the following link: http://127.0.0.1:53682/auth
|
|
Log in and authorize rclone for access
|
|
Waiting for code...
|
|
Got code
|
|
Configure this as a team drive?
|
|
y) Yes
|
|
n) No
|
|
y/n> n
|
|
\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-
|
|
[remote]
|
|
client_id =
|
|
client_secret =
|
|
scope = drive
|
|
root_folder_id =
|
|
service_account_file =
|
|
token = {\[dq]access_token\[dq]:\[dq]XXX\[dq],\[dq]token_type\[dq]:\[dq]Bearer\[dq],\[dq]refresh_token\[dq]:\[dq]XXX\[dq],\[dq]expiry\[dq]:\[dq]2014\-03\-16T13:57:58.955387075Z\[dq]}
|
|
\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-
|
|
y) Yes this is OK
|
|
e) Edit this remote
|
|
d) Delete this remote
|
|
y/e/d> y
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
Note that rclone runs a webserver on your local machine to collect the
|
|
token as returned from Google if you use auto config mode.
|
|
This only runs from the moment it opens your browser to the moment you
|
|
get back the verification code.
|
|
This is on \f[C]http://127.0.0.1:53682/\f[R] and this it may require you
|
|
to unblock it temporarily if you are running a host firewall, or use
|
|
manual mode.
|
|
.PP
|
|
You can then use it like this,
|
|
.PP
|
|
List directories in top level of your drive
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone lsd remote:
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
List all the files in your drive
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone ls remote:
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
To copy a local directory to a drive directory called backup
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone copy /home/source remote:backup
|
|
\f[R]
|
|
.fi
|
|
.SS Scopes
|
|
.PP
|
|
Rclone allows you to select which scope you would like for rclone to
|
|
use.
|
|
This changes what type of token is granted to rclone.
|
|
The scopes are defined
|
|
here (https://developers.google.com/drive/v3/web/about-auth).
|
|
.PP
|
|
The scope are
|
|
.SS drive
|
|
.PP
|
|
This is the default scope and allows full access to all files, except
|
|
for the Application Data Folder (see below).
|
|
.PP
|
|
Choose this one if you aren\[cq]t sure.
|
|
.SS drive.readonly
|
|
.PP
|
|
This allows read only access to all files.
|
|
Files may be listed and downloaded but not uploaded, renamed or deleted.
|
|
.SS drive.file
|
|
.PP
|
|
With this scope rclone can read/view/modify only those files and folders
|
|
it creates.
|
|
.PP
|
|
So if you uploaded files to drive via the web interface (or any other
|
|
means) they will not be visible to rclone.
|
|
.PP
|
|
This can be useful if you are using rclone to backup data and you want
|
|
to be sure confidential data on your drive is not visible to rclone.
|
|
.PP
|
|
Files created with this scope are visible in the web interface.
|
|
.SS drive.appfolder
|
|
.PP
|
|
This gives rclone its own private area to store files.
|
|
Rclone will not be able to see any other files on your drive and you
|
|
won\[cq]t be able to see rclone\[cq]s files from the web interface
|
|
either.
|
|
.SS drive.metadata.readonly
|
|
.PP
|
|
This allows read only access to file names only.
|
|
It does not allow rclone to download or upload data, or rename or delete
|
|
files or directories.
|
|
.SS Root folder ID
|
|
.PP
|
|
You can set the \f[C]root_folder_id\f[R] for rclone.
|
|
This is the directory (identified by its \f[C]Folder ID\f[R]) that
|
|
rclone considers to be the root of your drive.
|
|
.PP
|
|
Normally you will leave this blank and rclone will determine the correct
|
|
root to use itself.
|
|
.PP
|
|
However you can set this to restrict rclone to a specific folder
|
|
hierarchy or to access data within the \[lq]Computers\[rq] tab on the
|
|
drive web interface (where files from Google\[cq]s Backup and Sync
|
|
desktop program go).
|
|
.PP
|
|
In order to do this you will have to find the \f[C]Folder ID\f[R] of the
|
|
directory you wish rclone to display.
|
|
This will be the last segment of the URL when you open the relevant
|
|
folder in the drive web interface.
|
|
.PP
|
|
So if the folder you want rclone to use has a URL which looks like
|
|
\f[C]https://drive.google.com/drive/folders/1XyfxxxxxxxxxxxxxxxxxxxxxxxxxKHCh\f[R]
|
|
in the browser, then you use \f[C]1XyfxxxxxxxxxxxxxxxxxxxxxxxxxKHCh\f[R]
|
|
as the \f[C]root_folder_id\f[R] in the config.
|
|
.PP
|
|
\f[B]NB\f[R] folders under the \[lq]Computers\[rq] tab seem to be read
|
|
only (drive gives a 500 error) when using rclone.
|
|
.PP
|
|
There doesn\[cq]t appear to be an API to discover the folder IDs of the
|
|
\[lq]Computers\[rq] tab \- please contact us if you know otherwise!
|
|
.PP
|
|
Note also that rclone can\[cq]t access any data under the
|
|
\[lq]Backups\[rq] tab on the google drive web interface yet.
|
|
.SS Service Account support
|
|
.PP
|
|
You can set up rclone with Google Drive in an unattended mode, i.e.\ not
|
|
tied to a specific end\-user Google account.
|
|
This is useful when you want to synchronise files onto machines that
|
|
don\[cq]t have actively logged\-in users, for example build machines.
|
|
.PP
|
|
To use a Service Account instead of OAuth2 token flow, enter the path to
|
|
your Service Account credentials at the \f[C]service_account_file\f[R]
|
|
prompt during \f[C]rclone config\f[R] and rclone won\[cq]t use the
|
|
browser based authentication flow.
|
|
If you\[cq]d rather stuff the contents of the credentials file into the
|
|
rclone config file, you can set \f[C]service_account_credentials\f[R]
|
|
with the actual contents of the file instead, or set the equivalent
|
|
environment variable.
|
|
.SS Use case \- Google Apps/G\-suite account and individual Drive
|
|
.PP
|
|
Let\[cq]s say that you are the administrator of a Google Apps (old) or
|
|
G\-suite account.
|
|
The goal is to store data on an individual\[cq]s Drive account, who IS a
|
|
member of the domain.
|
|
We\[cq]ll call the domain \f[B]example.com\f[R], and the user
|
|
\f[B]foo\[at]example.com\f[R].
|
|
.PP
|
|
There\[cq]s a few steps we need to go through to accomplish this:
|
|
.SS 1. Create a service account for example.com
|
|
.IP \[bu] 2
|
|
To create a service account and obtain its credentials, go to the Google
|
|
Developer Console (https://console.developers.google.com).
|
|
.IP \[bu] 2
|
|
You must have a project \- create one if you don\[cq]t.
|
|
.IP \[bu] 2
|
|
Then go to \[lq]IAM & admin\[rq] \-> \[lq]Service Accounts\[rq].
|
|
.IP \[bu] 2
|
|
Use the \[lq]Create Credentials\[rq] button.
|
|
Fill in \[lq]Service account name\[rq] with something that identifies
|
|
your client.
|
|
\[lq]Role\[rq] can be empty.
|
|
.IP \[bu] 2
|
|
Tick \[lq]Furnish a new private key\[rq] \- select \[lq]Key type
|
|
JSON\[rq].
|
|
.IP \[bu] 2
|
|
Tick \[lq]Enable G Suite Domain\-wide Delegation\[rq].
|
|
This option makes \[lq]impersonation\[rq] possible, as documented here:
|
|
Delegating domain\-wide authority to the service
|
|
account (https://developers.google.com/identity/protocols/OAuth2ServiceAccount#delegatingauthority)
|
|
.IP \[bu] 2
|
|
These credentials are what rclone will use for authentication.
|
|
If you ever need to remove access, press the \[lq]Delete service account
|
|
key\[rq] button.
|
|
.SS 2. Allowing API access to example.com Google Drive
|
|
.IP \[bu] 2
|
|
Go to example.com\[cq]s admin console
|
|
.IP \[bu] 2
|
|
Go into \[lq]Security\[rq] (or use the search bar)
|
|
.IP \[bu] 2
|
|
Select \[lq]Show more\[rq] and then \[lq]Advanced settings\[rq]
|
|
.IP \[bu] 2
|
|
Select \[lq]Manage API client access\[rq] in the
|
|
\[lq]Authentication\[rq] section
|
|
.IP \[bu] 2
|
|
In the \[lq]Client Name\[rq] field enter the service account\[cq]s
|
|
\[lq]Client ID\[rq] \- this can be found in the Developer Console under
|
|
\[lq]IAM & Admin\[rq] \-> \[lq]Service Accounts\[rq], then \[lq]View
|
|
Client ID\[rq] for the newly created service account.
|
|
It is a \[ti]21 character numerical string.
|
|
.IP \[bu] 2
|
|
In the next field, \[lq]One or More API Scopes\[rq], enter
|
|
\f[C]https://www.googleapis.com/auth/drive\f[R] to grant access to
|
|
Google Drive specifically.
|
|
.SS 3. Configure rclone, assuming a new install
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone config
|
|
|
|
n/s/q> n # New
|
|
name>gdrive # Gdrive is an example name
|
|
Storage> # Select the number shown for Google Drive
|
|
client_id> # Can be left blank
|
|
client_secret> # Can be left blank
|
|
scope> # Select your scope, 1 for example
|
|
root_folder_id> # Can be left blank
|
|
service_account_file> /home/foo/myJSONfile.json # This is where the JSON file goes!
|
|
y/n> # Auto config, y
|
|
\f[R]
|
|
.fi
|
|
.SS 4. Verify that it\[cq]s working
|
|
.IP \[bu] 2
|
|
\f[C]rclone \-v \-\-drive\-impersonate foo\[at]example.com lsf gdrive:backup\f[R]
|
|
.IP \[bu] 2
|
|
The arguments do:
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
\f[C]\-v\f[R] \- verbose logging
|
|
.IP \[bu] 2
|
|
\f[C]\-\-drive\-impersonate foo\[at]example.com\f[R] \- this is what
|
|
does the magic, pretending to be user foo.
|
|
.IP \[bu] 2
|
|
\f[C]lsf\f[R] \- list files in a parsing friendly way
|
|
.IP \[bu] 2
|
|
\f[C]gdrive:backup\f[R] \- use the remote called gdrive, work in the
|
|
folder named backup.
|
|
.RE
|
|
.SS Team drives
|
|
.PP
|
|
If you want to configure the remote to point to a Google Team Drive then
|
|
answer \f[C]y\f[R] to the question
|
|
\f[C]Configure this as a team drive?\f[R].
|
|
.PP
|
|
This will fetch the list of Team Drives from google and allow you to
|
|
configure which one you want to use.
|
|
You can also type in a team drive ID if you prefer.
|
|
.PP
|
|
For example:
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
Configure this as a team drive?
|
|
y) Yes
|
|
n) No
|
|
y/n> y
|
|
Fetching team drive list...
|
|
Choose a number from below, or type in your own value
|
|
1 / Rclone Test
|
|
\[rs] \[dq]xxxxxxxxxxxxxxxxxxxx\[dq]
|
|
2 / Rclone Test 2
|
|
\[rs] \[dq]yyyyyyyyyyyyyyyyyyyy\[dq]
|
|
3 / Rclone Test 3
|
|
\[rs] \[dq]zzzzzzzzzzzzzzzzzzzz\[dq]
|
|
Enter a Team Drive ID> 1
|
|
\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-
|
|
[remote]
|
|
client_id =
|
|
client_secret =
|
|
token = {\[dq]AccessToken\[dq]:\[dq]xxxx.x.xxxxx_xxxxxxxxxxx_xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx\[dq],\[dq]RefreshToken\[dq]:\[dq]1/xxxxxxxxxxxxxxxx_xxxxxxxxxxxxxxxxxxxxxxxxxx\[dq],\[dq]Expiry\[dq]:\[dq]2014\-03\-16T13:57:58.955387075Z\[dq],\[dq]Extra\[dq]:null}
|
|
team_drive = xxxxxxxxxxxxxxxxxxxx
|
|
\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-
|
|
y) Yes this is OK
|
|
e) Edit this remote
|
|
d) Delete this remote
|
|
y/e/d> y
|
|
\f[R]
|
|
.fi
|
|
.SS \[en]fast\-list
|
|
.PP
|
|
This remote supports \f[C]\-\-fast\-list\f[R] which allows you to use
|
|
fewer transactions in exchange for more memory.
|
|
See the rclone docs (https://rclone.org/docs/#fast-list) for more
|
|
details.
|
|
.PP
|
|
It does this by combining multiple \f[C]list\f[R] calls into a single
|
|
API request.
|
|
.PP
|
|
This works by combining many \f[C]\[aq]%s\[aq] in parents\f[R] filters
|
|
into one expression.
|
|
To list the contents of directories a, b and c, the following requests
|
|
will be send by the regular \f[C]List\f[R] function:
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
trashed=false and \[aq]a\[aq] in parents
|
|
trashed=false and \[aq]b\[aq] in parents
|
|
trashed=false and \[aq]c\[aq] in parents
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
These can now be combined into a single request:
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
trashed=false and (\[aq]a\[aq] in parents or \[aq]b\[aq] in parents or \[aq]c\[aq] in parents)
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
The implementation of \f[C]ListR\f[R] will put up to 50
|
|
\f[C]parents\f[R] filters into one request.
|
|
It will use the \f[C]\-\-checkers\f[R] value to specify the number of
|
|
requests to run in parallel.
|
|
.PP
|
|
In tests, these batch requests were up to 20x faster than the regular
|
|
method.
|
|
Running the following command against different sized folders gives:
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone lsjson \-vv \-R \-\-checkers=6 gdrive:folder
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
small folder (220 directories, 700 files):
|
|
.IP \[bu] 2
|
|
without \f[C]\-\-fast\-list\f[R]: 38s
|
|
.IP \[bu] 2
|
|
with \f[C]\-\-fast\-list\f[R]: 10s
|
|
.PP
|
|
large folder (10600 directories, 39000 files):
|
|
.IP \[bu] 2
|
|
without \f[C]\-\-fast\-list\f[R]: 22:05 min
|
|
.IP \[bu] 2
|
|
with \f[C]\-\-fast\-list\f[R]: 58s
|
|
.SS Modified time
|
|
.PP
|
|
Google drive stores modification times accurate to 1 ms.
|
|
.SS Restricted filename characters
|
|
.PP
|
|
Only Invalid UTF\-8 bytes will be
|
|
replaced (https://rclone.org/overview/#invalid-utf8), as they can\[cq]t
|
|
be used in JSON strings.
|
|
.PP
|
|
In contrast to other backends, \f[C]/\f[R] can also be used in names and
|
|
\f[C].\f[R] or \f[C]..\f[R] are valid names.
|
|
.SS Revisions
|
|
.PP
|
|
Google drive stores revisions of files.
|
|
When you upload a change to an existing file to google drive using
|
|
rclone it will create a new revision of that file.
|
|
.PP
|
|
Revisions follow the standard google policy which at time of writing was
|
|
.IP \[bu] 2
|
|
They are deleted after 30 days or 100 revisions (whatever comes first).
|
|
.IP \[bu] 2
|
|
They do not count towards a user storage quota.
|
|
.SS Deleting files
|
|
.PP
|
|
By default rclone will send all files to the trash when deleting files.
|
|
If deleting them permanently is required then use the
|
|
\f[C]\-\-drive\-use\-trash=false\f[R] flag, or set the equivalent
|
|
environment variable.
|
|
.SS Shortcuts
|
|
.PP
|
|
In March 2020 Google introduced a new feature in Google Drive called
|
|
drive shortcuts (https://support.google.com/drive/answer/9700156)
|
|
(API (https://developers.google.com/drive/api/v3/shortcuts)).
|
|
These will (by September 2020) replace the ability for files or folders
|
|
to be in multiple folders at
|
|
once (https://cloud.google.com/blog/products/g-suite/simplifying-google-drives-folder-structure-and-sharing-models).
|
|
.PP
|
|
Shortcuts are files that link to other files on Google Drive somewhat
|
|
like a symlink in unix, except they point to the underlying file data
|
|
(eg the inode in unix terms) so they don\[cq]t break if the source is
|
|
renamed or moved about.
|
|
.PP
|
|
Be default rclone treats these as follows.
|
|
.PP
|
|
For shortcuts pointing to files:
|
|
.IP \[bu] 2
|
|
When listing a file shortcut appears as the destination file.
|
|
.IP \[bu] 2
|
|
When downloading the contents of the destination file is downloaded.
|
|
.IP \[bu] 2
|
|
When updating shortcut file with a non shortcut file, the shortcut is
|
|
removed then a new file is uploaded in place of the shortcut.
|
|
.IP \[bu] 2
|
|
When server side moving (renaming) the shortcut is renamed, not the
|
|
destination file.
|
|
.IP \[bu] 2
|
|
When server side copying the shortcut is copied, not the contents of the
|
|
shortcut.
|
|
.IP \[bu] 2
|
|
When deleting the shortcut is deleted not the linked file.
|
|
.IP \[bu] 2
|
|
When setting the modification time, the modification time of the linked
|
|
file will be set.
|
|
.PP
|
|
For shortcuts pointing to folders:
|
|
.IP \[bu] 2
|
|
When listing the shortcut appears as a folder and that folder will
|
|
contain the contents of the linked folder appear (including any sub
|
|
folders)
|
|
.IP \[bu] 2
|
|
When downloading the contents of the linked folder and sub contents are
|
|
downloaded
|
|
.IP \[bu] 2
|
|
When uploading to a shortcut folder the file will be placed in the
|
|
linked folder
|
|
.IP \[bu] 2
|
|
When server side moving (renaming) the shortcut is renamed, not the
|
|
destination folder
|
|
.IP \[bu] 2
|
|
When server side copying the contents of the linked folder is copied,
|
|
not the shortcut.
|
|
.IP \[bu] 2
|
|
When deleting with \f[C]rclone rmdir\f[R] or \f[C]rclone purge\f[R] the
|
|
shortcut is deleted not the linked folder.
|
|
.IP \[bu] 2
|
|
\f[B]NB\f[R] When deleting with \f[C]rclone remove\f[R] or
|
|
\f[C]rclone mount\f[R] the contents of the linked folder will be
|
|
deleted.
|
|
.PP
|
|
It isn\[cq]t currently possible to create shortcuts with rclone.
|
|
.PP
|
|
Shortcuts can be completely ignored with the
|
|
\f[C]\-\-drive\-skip\-shortcuts\f[R] flag or the corresponding
|
|
\f[C]skip_shortcuts\f[R] configuration setting.
|
|
.SS Emptying trash
|
|
.PP
|
|
If you wish to empty your trash you can use the
|
|
\f[C]rclone cleanup remote:\f[R] command which will permanently delete
|
|
all your trashed files.
|
|
This command does not take any path arguments.
|
|
.PP
|
|
Note that Google Drive takes some time (minutes to days) to empty the
|
|
trash even though the command returns within a few seconds.
|
|
No output is echoed, so there will be no confirmation even using \-v or
|
|
\-vv.
|
|
.SS Quota information
|
|
.PP
|
|
To view your current quota you can use the
|
|
\f[C]rclone about remote:\f[R] command which will display your usage
|
|
limit (quota), the usage in Google Drive, the size of all files in the
|
|
Trash and the space used by other Google services such as Gmail.
|
|
This command does not take any path arguments.
|
|
.SS Import/Export of google documents
|
|
.PP
|
|
Google documents can be exported from and uploaded to Google Drive.
|
|
.PP
|
|
When rclone downloads a Google doc it chooses a format to download
|
|
depending upon the \f[C]\-\-drive\-export\-formats\f[R] setting.
|
|
By default the export formats are \f[C]docx,xlsx,pptx,svg\f[R] which are
|
|
a sensible default for an editable document.
|
|
.PP
|
|
When choosing a format, rclone runs down the list provided in order and
|
|
chooses the first file format the doc can be exported as from the list.
|
|
If the file can\[cq]t be exported to a format on the formats list, then
|
|
rclone will choose a format from the default list.
|
|
.PP
|
|
If you prefer an archive copy then you might use
|
|
\f[C]\-\-drive\-export\-formats pdf\f[R], or if you prefer
|
|
openoffice/libreoffice formats you might use
|
|
\f[C]\-\-drive\-export\-formats ods,odt,odp\f[R].
|
|
.PP
|
|
Note that rclone adds the extension to the google doc, so if it is
|
|
called \f[C]My Spreadsheet\f[R] on google docs, it will be exported as
|
|
\f[C]My Spreadsheet.xlsx\f[R] or \f[C]My Spreadsheet.pdf\f[R] etc.
|
|
.PP
|
|
When importing files into Google Drive, rclone will convert all files
|
|
with an extension in \f[C]\-\-drive\-import\-formats\f[R] to their
|
|
associated document type.
|
|
rclone will not convert any files by default, since the conversion is
|
|
lossy process.
|
|
.PP
|
|
The conversion must result in a file with the same extension when the
|
|
\f[C]\-\-drive\-export\-formats\f[R] rules are applied to the uploaded
|
|
document.
|
|
.PP
|
|
Here are some examples for allowed and prohibited conversions.
|
|
.PP
|
|
.TS
|
|
tab(@);
|
|
l l l l l.
|
|
T{
|
|
export\-formats
|
|
T}@T{
|
|
import\-formats
|
|
T}@T{
|
|
Upload Ext
|
|
T}@T{
|
|
Document Ext
|
|
T}@T{
|
|
Allowed
|
|
T}
|
|
_
|
|
T{
|
|
odt
|
|
T}@T{
|
|
odt
|
|
T}@T{
|
|
odt
|
|
T}@T{
|
|
odt
|
|
T}@T{
|
|
Yes
|
|
T}
|
|
T{
|
|
odt
|
|
T}@T{
|
|
docx,odt
|
|
T}@T{
|
|
odt
|
|
T}@T{
|
|
odt
|
|
T}@T{
|
|
Yes
|
|
T}
|
|
T{
|
|
T}@T{
|
|
docx
|
|
T}@T{
|
|
docx
|
|
T}@T{
|
|
docx
|
|
T}@T{
|
|
Yes
|
|
T}
|
|
T{
|
|
T}@T{
|
|
odt
|
|
T}@T{
|
|
odt
|
|
T}@T{
|
|
docx
|
|
T}@T{
|
|
No
|
|
T}
|
|
T{
|
|
odt,docx
|
|
T}@T{
|
|
docx,odt
|
|
T}@T{
|
|
docx
|
|
T}@T{
|
|
odt
|
|
T}@T{
|
|
No
|
|
T}
|
|
T{
|
|
docx,odt
|
|
T}@T{
|
|
docx,odt
|
|
T}@T{
|
|
docx
|
|
T}@T{
|
|
docx
|
|
T}@T{
|
|
Yes
|
|
T}
|
|
T{
|
|
docx,odt
|
|
T}@T{
|
|
docx,odt
|
|
T}@T{
|
|
odt
|
|
T}@T{
|
|
docx
|
|
T}@T{
|
|
No
|
|
T}
|
|
.TE
|
|
.PP
|
|
This limitation can be disabled by specifying
|
|
\f[C]\-\-drive\-allow\-import\-name\-change\f[R].
|
|
When using this flag, rclone can convert multiple files types resulting
|
|
in the same document type at once, eg with
|
|
\f[C]\-\-drive\-import\-formats docx,odt,txt\f[R], all files having
|
|
these extension would result in a document represented as a docx file.
|
|
This brings the additional risk of overwriting a document, if multiple
|
|
files have the same stem.
|
|
Many rclone operations will not handle this name change in any way.
|
|
They assume an equal name when copying files and might copy the file
|
|
again or delete them when the name changes.
|
|
.PP
|
|
Here are the possible export extensions with their corresponding mime
|
|
types.
|
|
Most of these can also be used for importing, but there more that are
|
|
not listed here.
|
|
Some of these additional ones might only be available when the operating
|
|
system provides the correct MIME type entries.
|
|
.PP
|
|
This list can be changed by Google Drive at any time and might not
|
|
represent the currently available conversions.
|
|
.PP
|
|
.TS
|
|
tab(@);
|
|
lw(19.7n) lw(24.1n) lw(26.2n).
|
|
T{
|
|
Extension
|
|
T}@T{
|
|
Mime Type
|
|
T}@T{
|
|
Description
|
|
T}
|
|
_
|
|
T{
|
|
csv
|
|
T}@T{
|
|
text/csv
|
|
T}@T{
|
|
Standard CSV format for Spreadsheets
|
|
T}
|
|
T{
|
|
docx
|
|
T}@T{
|
|
application/vnd.openxmlformats\-officedocument.wordprocessingml.document
|
|
T}@T{
|
|
Microsoft Office Document
|
|
T}
|
|
T{
|
|
epub
|
|
T}@T{
|
|
application/epub+zip
|
|
T}@T{
|
|
E\-book format
|
|
T}
|
|
T{
|
|
html
|
|
T}@T{
|
|
text/html
|
|
T}@T{
|
|
An HTML Document
|
|
T}
|
|
T{
|
|
jpg
|
|
T}@T{
|
|
image/jpeg
|
|
T}@T{
|
|
A JPEG Image File
|
|
T}
|
|
T{
|
|
json
|
|
T}@T{
|
|
application/vnd.google\-apps.script+json
|
|
T}@T{
|
|
JSON Text Format
|
|
T}
|
|
T{
|
|
odp
|
|
T}@T{
|
|
application/vnd.oasis.opendocument.presentation
|
|
T}@T{
|
|
Openoffice Presentation
|
|
T}
|
|
T{
|
|
ods
|
|
T}@T{
|
|
application/vnd.oasis.opendocument.spreadsheet
|
|
T}@T{
|
|
Openoffice Spreadsheet
|
|
T}
|
|
T{
|
|
ods
|
|
T}@T{
|
|
application/x\-vnd.oasis.opendocument.spreadsheet
|
|
T}@T{
|
|
Openoffice Spreadsheet
|
|
T}
|
|
T{
|
|
odt
|
|
T}@T{
|
|
application/vnd.oasis.opendocument.text
|
|
T}@T{
|
|
Openoffice Document
|
|
T}
|
|
T{
|
|
pdf
|
|
T}@T{
|
|
application/pdf
|
|
T}@T{
|
|
Adobe PDF Format
|
|
T}
|
|
T{
|
|
png
|
|
T}@T{
|
|
image/png
|
|
T}@T{
|
|
PNG Image Format
|
|
T}
|
|
T{
|
|
pptx
|
|
T}@T{
|
|
application/vnd.openxmlformats\-officedocument.presentationml.presentation
|
|
T}@T{
|
|
Microsoft Office Powerpoint
|
|
T}
|
|
T{
|
|
rtf
|
|
T}@T{
|
|
application/rtf
|
|
T}@T{
|
|
Rich Text Format
|
|
T}
|
|
T{
|
|
svg
|
|
T}@T{
|
|
image/svg+xml
|
|
T}@T{
|
|
Scalable Vector Graphics Format
|
|
T}
|
|
T{
|
|
tsv
|
|
T}@T{
|
|
text/tab\-separated\-values
|
|
T}@T{
|
|
Standard TSV format for spreadsheets
|
|
T}
|
|
T{
|
|
txt
|
|
T}@T{
|
|
text/plain
|
|
T}@T{
|
|
Plain Text
|
|
T}
|
|
T{
|
|
xlsx
|
|
T}@T{
|
|
application/vnd.openxmlformats\-officedocument.spreadsheetml.sheet
|
|
T}@T{
|
|
Microsoft Office Spreadsheet
|
|
T}
|
|
T{
|
|
zip
|
|
T}@T{
|
|
application/zip
|
|
T}@T{
|
|
A ZIP file of HTML, Images CSS
|
|
T}
|
|
.TE
|
|
.PP
|
|
Google documents can also be exported as link files.
|
|
These files will open a browser window for the Google Docs website of
|
|
that document when opened.
|
|
The link file extension has to be specified as a
|
|
\f[C]\-\-drive\-export\-formats\f[R] parameter.
|
|
They will match all available Google Documents.
|
|
.PP
|
|
.TS
|
|
tab(@);
|
|
l l l.
|
|
T{
|
|
Extension
|
|
T}@T{
|
|
Description
|
|
T}@T{
|
|
OS Support
|
|
T}
|
|
_
|
|
T{
|
|
desktop
|
|
T}@T{
|
|
freedesktop.org specified desktop entry
|
|
T}@T{
|
|
Linux
|
|
T}
|
|
T{
|
|
link.html
|
|
T}@T{
|
|
An HTML Document with a redirect
|
|
T}@T{
|
|
All
|
|
T}
|
|
T{
|
|
url
|
|
T}@T{
|
|
INI style link file
|
|
T}@T{
|
|
macOS, Windows
|
|
T}
|
|
T{
|
|
webloc
|
|
T}@T{
|
|
macOS specific XML format
|
|
T}@T{
|
|
macOS
|
|
T}
|
|
.TE
|
|
.SS Standard Options
|
|
.PP
|
|
Here are the standard options specific to drive (Google Drive).
|
|
.SS \[en]drive\-client\-id
|
|
.PP
|
|
Google Application Client Id Setting your own is recommended.
|
|
See https://rclone.org/drive/#making\-your\-own\-client\-id for how to
|
|
create your own.
|
|
If you leave this blank, it will use an internal key which is low
|
|
performance.
|
|
.IP \[bu] 2
|
|
Config: client_id
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_DRIVE_CLIENT_ID
|
|
.IP \[bu] 2
|
|
Type: string
|
|
.IP \[bu] 2
|
|
Default: \[dq]\[dq]
|
|
.SS \[en]drive\-client\-secret
|
|
.PP
|
|
Google Application Client Secret Setting your own is recommended.
|
|
.IP \[bu] 2
|
|
Config: client_secret
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_DRIVE_CLIENT_SECRET
|
|
.IP \[bu] 2
|
|
Type: string
|
|
.IP \[bu] 2
|
|
Default: \[dq]\[dq]
|
|
.SS \[en]drive\-scope
|
|
.PP
|
|
Scope that rclone should use when requesting access from drive.
|
|
.IP \[bu] 2
|
|
Config: scope
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_DRIVE_SCOPE
|
|
.IP \[bu] 2
|
|
Type: string
|
|
.IP \[bu] 2
|
|
Default: \[dq]\[dq]
|
|
.IP \[bu] 2
|
|
Examples:
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
\[lq]drive\[rq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Full access all files, excluding Application Data Folder.
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[lq]drive.readonly\[rq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Read\-only access to file metadata and file contents.
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[lq]drive.file\[rq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Access to files created by rclone only.
|
|
.IP \[bu] 2
|
|
These are visible in the drive website.
|
|
.IP \[bu] 2
|
|
File authorization is revoked when the user deauthorizes the app.
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[lq]drive.appfolder\[rq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Allows read and write access to the Application Data folder.
|
|
.IP \[bu] 2
|
|
This is not visible in the drive website.
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[lq]drive.metadata.readonly\[rq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Allows read\-only access to file metadata but
|
|
.IP \[bu] 2
|
|
does not allow any access to read or download file content.
|
|
.RE
|
|
.RE
|
|
.SS \[en]drive\-root\-folder\-id
|
|
.PP
|
|
ID of the root folder Leave blank normally.
|
|
.PP
|
|
Fill in to access \[lq]Computers\[rq] folders (see docs), or for rclone
|
|
to use a non root folder as its starting point.
|
|
.PP
|
|
Note that if this is blank, the first time rclone runs it will fill it
|
|
in with the ID of the root folder.
|
|
.IP \[bu] 2
|
|
Config: root_folder_id
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_DRIVE_ROOT_FOLDER_ID
|
|
.IP \[bu] 2
|
|
Type: string
|
|
.IP \[bu] 2
|
|
Default: \[dq]\[dq]
|
|
.SS \[en]drive\-service\-account\-file
|
|
.PP
|
|
Service Account Credentials JSON file path Leave blank normally.
|
|
Needed only if you want use SA instead of interactive login.
|
|
.IP \[bu] 2
|
|
Config: service_account_file
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_DRIVE_SERVICE_ACCOUNT_FILE
|
|
.IP \[bu] 2
|
|
Type: string
|
|
.IP \[bu] 2
|
|
Default: \[dq]\[dq]
|
|
.SS Advanced Options
|
|
.PP
|
|
Here are the advanced options specific to drive (Google Drive).
|
|
.SS \[en]drive\-service\-account\-credentials
|
|
.PP
|
|
Service Account Credentials JSON blob Leave blank normally.
|
|
Needed only if you want use SA instead of interactive login.
|
|
.IP \[bu] 2
|
|
Config: service_account_credentials
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_DRIVE_SERVICE_ACCOUNT_CREDENTIALS
|
|
.IP \[bu] 2
|
|
Type: string
|
|
.IP \[bu] 2
|
|
Default: \[dq]\[dq]
|
|
.SS \[en]drive\-team\-drive
|
|
.PP
|
|
ID of the Team Drive
|
|
.IP \[bu] 2
|
|
Config: team_drive
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_DRIVE_TEAM_DRIVE
|
|
.IP \[bu] 2
|
|
Type: string
|
|
.IP \[bu] 2
|
|
Default: \[dq]\[dq]
|
|
.SS \[en]drive\-auth\-owner\-only
|
|
.PP
|
|
Only consider files owned by the authenticated user.
|
|
.IP \[bu] 2
|
|
Config: auth_owner_only
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_DRIVE_AUTH_OWNER_ONLY
|
|
.IP \[bu] 2
|
|
Type: bool
|
|
.IP \[bu] 2
|
|
Default: false
|
|
.SS \[en]drive\-use\-trash
|
|
.PP
|
|
Send files to the trash instead of deleting permanently.
|
|
Defaults to true, namely sending files to the trash.
|
|
Use \f[C]\-\-drive\-use\-trash=false\f[R] to delete files permanently
|
|
instead.
|
|
.IP \[bu] 2
|
|
Config: use_trash
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_DRIVE_USE_TRASH
|
|
.IP \[bu] 2
|
|
Type: bool
|
|
.IP \[bu] 2
|
|
Default: true
|
|
.SS \[en]drive\-skip\-gdocs
|
|
.PP
|
|
Skip google documents in all listings.
|
|
If given, gdocs practically become invisible to rclone.
|
|
.IP \[bu] 2
|
|
Config: skip_gdocs
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_DRIVE_SKIP_GDOCS
|
|
.IP \[bu] 2
|
|
Type: bool
|
|
.IP \[bu] 2
|
|
Default: false
|
|
.SS \[en]drive\-skip\-checksum\-gphotos
|
|
.PP
|
|
Skip MD5 checksum on Google photos and videos only.
|
|
.PP
|
|
Use this if you get checksum errors when transferring Google photos or
|
|
videos.
|
|
.PP
|
|
Setting this flag will cause Google photos and videos to return a blank
|
|
MD5 checksum.
|
|
.PP
|
|
Google photos are identified by being in the \[lq]photos\[rq] space.
|
|
.PP
|
|
Corrupted checksums are caused by Google modifying the image/video but
|
|
not updating the checksum.
|
|
.IP \[bu] 2
|
|
Config: skip_checksum_gphotos
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_DRIVE_SKIP_CHECKSUM_GPHOTOS
|
|
.IP \[bu] 2
|
|
Type: bool
|
|
.IP \[bu] 2
|
|
Default: false
|
|
.SS \[en]drive\-shared\-with\-me
|
|
.PP
|
|
Only show files that are shared with me.
|
|
.PP
|
|
Instructs rclone to operate on your \[lq]Shared with me\[rq] folder
|
|
(where Google Drive lets you access the files and folders others have
|
|
shared with you).
|
|
.PP
|
|
This works both with the \[lq]list\[rq] (lsd, lsl, etc) and the
|
|
\[lq]copy\[rq] commands (copy, sync, etc), and with all other commands
|
|
too.
|
|
.IP \[bu] 2
|
|
Config: shared_with_me
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_DRIVE_SHARED_WITH_ME
|
|
.IP \[bu] 2
|
|
Type: bool
|
|
.IP \[bu] 2
|
|
Default: false
|
|
.SS \[en]drive\-trashed\-only
|
|
.PP
|
|
Only show files that are in the trash.
|
|
This will show trashed files in their original directory structure.
|
|
.IP \[bu] 2
|
|
Config: trashed_only
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_DRIVE_TRASHED_ONLY
|
|
.IP \[bu] 2
|
|
Type: bool
|
|
.IP \[bu] 2
|
|
Default: false
|
|
.SS \[en]drive\-formats
|
|
.PP
|
|
Deprecated: see export_formats
|
|
.IP \[bu] 2
|
|
Config: formats
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_DRIVE_FORMATS
|
|
.IP \[bu] 2
|
|
Type: string
|
|
.IP \[bu] 2
|
|
Default: \[dq]\[dq]
|
|
.SS \[en]drive\-export\-formats
|
|
.PP
|
|
Comma separated list of preferred formats for downloading Google docs.
|
|
.IP \[bu] 2
|
|
Config: export_formats
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_DRIVE_EXPORT_FORMATS
|
|
.IP \[bu] 2
|
|
Type: string
|
|
.IP \[bu] 2
|
|
Default: \[lq]docx,xlsx,pptx,svg\[rq]
|
|
.SS \[en]drive\-import\-formats
|
|
.PP
|
|
Comma separated list of preferred formats for uploading Google docs.
|
|
.IP \[bu] 2
|
|
Config: import_formats
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_DRIVE_IMPORT_FORMATS
|
|
.IP \[bu] 2
|
|
Type: string
|
|
.IP \[bu] 2
|
|
Default: \[dq]\[dq]
|
|
.SS \[en]drive\-allow\-import\-name\-change
|
|
.PP
|
|
Allow the filetype to change when uploading Google docs (e.g.\ file.doc
|
|
to file.docx).
|
|
This will confuse sync and reupload every time.
|
|
.IP \[bu] 2
|
|
Config: allow_import_name_change
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_DRIVE_ALLOW_IMPORT_NAME_CHANGE
|
|
.IP \[bu] 2
|
|
Type: bool
|
|
.IP \[bu] 2
|
|
Default: false
|
|
.SS \[en]drive\-use\-created\-date
|
|
.PP
|
|
Use file created date instead of modified date.,
|
|
.PP
|
|
Useful when downloading data and you want the creation date used in
|
|
place of the last modified date.
|
|
.PP
|
|
\f[B]WARNING\f[R]: This flag may have some unexpected consequences.
|
|
.PP
|
|
When uploading to your drive all files will be overwritten unless they
|
|
haven\[cq]t been modified since their creation.
|
|
And the inverse will occur while downloading.
|
|
This side effect can be avoided by using the \[lq]\[en]checksum\[rq]
|
|
flag.
|
|
.PP
|
|
This feature was implemented to retain photos capture date as recorded
|
|
by google photos.
|
|
You will first need to check the \[lq]Create a Google Photos folder\[rq]
|
|
option in your google drive settings.
|
|
You can then copy or move the photos locally and use the date the image
|
|
was taken (created) set as the modification date.
|
|
.IP \[bu] 2
|
|
Config: use_created_date
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_DRIVE_USE_CREATED_DATE
|
|
.IP \[bu] 2
|
|
Type: bool
|
|
.IP \[bu] 2
|
|
Default: false
|
|
.SS \[en]drive\-use\-shared\-date
|
|
.PP
|
|
Use date file was shared instead of modified date.
|
|
.PP
|
|
Note that, as with \[lq]\[en]drive\-use\-created\-date\[rq], this flag
|
|
may have unexpected consequences when uploading/downloading files.
|
|
.PP
|
|
If both this flag and \[lq]\[en]drive\-use\-created\-date\[rq] are set,
|
|
the created date is used.
|
|
.IP \[bu] 2
|
|
Config: use_shared_date
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_DRIVE_USE_SHARED_DATE
|
|
.IP \[bu] 2
|
|
Type: bool
|
|
.IP \[bu] 2
|
|
Default: false
|
|
.SS \[en]drive\-list\-chunk
|
|
.PP
|
|
Size of listing chunk 100\-1000.
|
|
0 to disable.
|
|
.IP \[bu] 2
|
|
Config: list_chunk
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_DRIVE_LIST_CHUNK
|
|
.IP \[bu] 2
|
|
Type: int
|
|
.IP \[bu] 2
|
|
Default: 1000
|
|
.SS \[en]drive\-impersonate
|
|
.PP
|
|
Impersonate this user when using a service account.
|
|
.IP \[bu] 2
|
|
Config: impersonate
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_DRIVE_IMPERSONATE
|
|
.IP \[bu] 2
|
|
Type: string
|
|
.IP \[bu] 2
|
|
Default: \[dq]\[dq]
|
|
.SS \[en]drive\-alternate\-export
|
|
.PP
|
|
Use alternate export URLs for google documents export.,
|
|
.PP
|
|
If this option is set this instructs rclone to use an alternate set of
|
|
export URLs for drive documents.
|
|
Users have reported that the official export URLs can\[cq]t export large
|
|
documents, whereas these unofficial ones can.
|
|
.PP
|
|
See rclone issue #2243 (https://github.com/rclone/rclone/issues/2243)
|
|
for background, this google drive
|
|
issue (https://issuetracker.google.com/issues/36761333) and this helpful
|
|
post (https://www.labnol.org/internet/direct-links-for-google-drive/28356/).
|
|
.IP \[bu] 2
|
|
Config: alternate_export
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_DRIVE_ALTERNATE_EXPORT
|
|
.IP \[bu] 2
|
|
Type: bool
|
|
.IP \[bu] 2
|
|
Default: false
|
|
.SS \[en]drive\-upload\-cutoff
|
|
.PP
|
|
Cutoff for switching to chunked upload
|
|
.IP \[bu] 2
|
|
Config: upload_cutoff
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_DRIVE_UPLOAD_CUTOFF
|
|
.IP \[bu] 2
|
|
Type: SizeSuffix
|
|
.IP \[bu] 2
|
|
Default: 8M
|
|
.SS \[en]drive\-chunk\-size
|
|
.PP
|
|
Upload chunk size.
|
|
Must a power of 2 >= 256k.
|
|
.PP
|
|
Making this larger will improve performance, but note that each chunk is
|
|
buffered in memory one per transfer.
|
|
.PP
|
|
Reducing this will reduce memory usage but decrease performance.
|
|
.IP \[bu] 2
|
|
Config: chunk_size
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_DRIVE_CHUNK_SIZE
|
|
.IP \[bu] 2
|
|
Type: SizeSuffix
|
|
.IP \[bu] 2
|
|
Default: 8M
|
|
.SS \[en]drive\-acknowledge\-abuse
|
|
.PP
|
|
Set to allow files which return cannotDownloadAbusiveFile to be
|
|
downloaded.
|
|
.PP
|
|
If downloading a file returns the error \[lq]This file has been
|
|
identified as malware or spam and cannot be downloaded\[rq] with the
|
|
error code \[lq]cannotDownloadAbusiveFile\[rq] then supply this flag to
|
|
rclone to indicate you acknowledge the risks of downloading the file and
|
|
rclone will download it anyway.
|
|
.IP \[bu] 2
|
|
Config: acknowledge_abuse
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_DRIVE_ACKNOWLEDGE_ABUSE
|
|
.IP \[bu] 2
|
|
Type: bool
|
|
.IP \[bu] 2
|
|
Default: false
|
|
.SS \[en]drive\-keep\-revision\-forever
|
|
.PP
|
|
Keep new head revision of each file forever.
|
|
.IP \[bu] 2
|
|
Config: keep_revision_forever
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_DRIVE_KEEP_REVISION_FOREVER
|
|
.IP \[bu] 2
|
|
Type: bool
|
|
.IP \[bu] 2
|
|
Default: false
|
|
.SS \[en]drive\-size\-as\-quota
|
|
.PP
|
|
Show sizes as storage quota usage, not actual size.
|
|
.PP
|
|
Show the size of a file as the storage quota used.
|
|
This is the current version plus any older versions that have been set
|
|
to keep forever.
|
|
.PP
|
|
\f[B]WARNING\f[R]: This flag may have some unexpected consequences.
|
|
.PP
|
|
It is not recommended to set this flag in your config \- the recommended
|
|
usage is using the flag form \[en]drive\-size\-as\-quota when doing
|
|
rclone ls/lsl/lsf/lsjson/etc only.
|
|
.PP
|
|
If you do use this flag for syncing (not recommended) then you will need
|
|
to use \[en]ignore size also.
|
|
.IP \[bu] 2
|
|
Config: size_as_quota
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_DRIVE_SIZE_AS_QUOTA
|
|
.IP \[bu] 2
|
|
Type: bool
|
|
.IP \[bu] 2
|
|
Default: false
|
|
.SS \[en]drive\-v2\-download\-min\-size
|
|
.PP
|
|
If Object\[cq]s are greater, use drive v2 API to download.
|
|
.IP \[bu] 2
|
|
Config: v2_download_min_size
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_DRIVE_V2_DOWNLOAD_MIN_SIZE
|
|
.IP \[bu] 2
|
|
Type: SizeSuffix
|
|
.IP \[bu] 2
|
|
Default: off
|
|
.SS \[en]drive\-pacer\-min\-sleep
|
|
.PP
|
|
Minimum time to sleep between API calls.
|
|
.IP \[bu] 2
|
|
Config: pacer_min_sleep
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_DRIVE_PACER_MIN_SLEEP
|
|
.IP \[bu] 2
|
|
Type: Duration
|
|
.IP \[bu] 2
|
|
Default: 100ms
|
|
.SS \[en]drive\-pacer\-burst
|
|
.PP
|
|
Number of API calls to allow without sleeping.
|
|
.IP \[bu] 2
|
|
Config: pacer_burst
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_DRIVE_PACER_BURST
|
|
.IP \[bu] 2
|
|
Type: int
|
|
.IP \[bu] 2
|
|
Default: 100
|
|
.SS \[en]drive\-server\-side\-across\-configs
|
|
.PP
|
|
Allow server side operations (eg copy) to work across different drive
|
|
configs.
|
|
.PP
|
|
This can be useful if you wish to do a server side copy between two
|
|
different Google drives.
|
|
Note that this isn\[cq]t enabled by default because it isn\[cq]t easy to
|
|
tell if it will work between any two configurations.
|
|
.IP \[bu] 2
|
|
Config: server_side_across_configs
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_DRIVE_SERVER_SIDE_ACROSS_CONFIGS
|
|
.IP \[bu] 2
|
|
Type: bool
|
|
.IP \[bu] 2
|
|
Default: false
|
|
.SS \[en]drive\-disable\-http2
|
|
.PP
|
|
Disable drive using http2
|
|
.PP
|
|
There is currently an unsolved issue with the google drive backend and
|
|
HTTP/2.
|
|
HTTP/2 is therefore disabled by default for the drive backend but can be
|
|
re\-enabled here.
|
|
When the issue is solved this flag will be removed.
|
|
.PP
|
|
See: https://github.com/rclone/rclone/issues/3631
|
|
.IP \[bu] 2
|
|
Config: disable_http2
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_DRIVE_DISABLE_HTTP2
|
|
.IP \[bu] 2
|
|
Type: bool
|
|
.IP \[bu] 2
|
|
Default: true
|
|
.SS \[en]drive\-stop\-on\-upload\-limit
|
|
.PP
|
|
Make upload limit errors be fatal
|
|
.PP
|
|
At the time of writing it is only possible to upload 750GB of data to
|
|
Google Drive a day (this is an undocumented limit).
|
|
When this limit is reached Google Drive produces a slightly different
|
|
error message.
|
|
When this flag is set it causes these errors to be fatal.
|
|
These will stop the in\-progress sync.
|
|
.PP
|
|
Note that this detection is relying on error message strings which
|
|
Google don\[cq]t document so it may break in the future.
|
|
.PP
|
|
See: https://github.com/rclone/rclone/issues/3857
|
|
.IP \[bu] 2
|
|
Config: stop_on_upload_limit
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_DRIVE_STOP_ON_UPLOAD_LIMIT
|
|
.IP \[bu] 2
|
|
Type: bool
|
|
.IP \[bu] 2
|
|
Default: false
|
|
.SS \[en]drive\-skip\-shortcuts
|
|
.PP
|
|
If set skip shortcut files
|
|
.PP
|
|
Normally rclone dereferences shortcut files making them appear as if
|
|
they are the original file (see the shortcuts section).
|
|
If this flag is set then rclone will ignore shortcut files completely.
|
|
.IP \[bu] 2
|
|
Config: skip_shortcuts
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_DRIVE_SKIP_SHORTCUTS
|
|
.IP \[bu] 2
|
|
Type: bool
|
|
.IP \[bu] 2
|
|
Default: false
|
|
.SS \[en]drive\-encoding
|
|
.PP
|
|
This sets the encoding for the backend.
|
|
.PP
|
|
See: the encoding section in the
|
|
overview (https://rclone.org/overview/#encoding) for more info.
|
|
.IP \[bu] 2
|
|
Config: encoding
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_DRIVE_ENCODING
|
|
.IP \[bu] 2
|
|
Type: MultiEncoder
|
|
.IP \[bu] 2
|
|
Default: InvalidUtf8
|
|
.SS Backend commands
|
|
.PP
|
|
Here are the commands specific to the drive backend.
|
|
.PP
|
|
Run them with
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone backend COMMAND remote:
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
The help below will explain what arguments each command takes.
|
|
.PP
|
|
See the \[lq]rclone backend\[rq]
|
|
command (https://rclone.org/commands/rclone_backend/) for more info on
|
|
how to pass options and arguments.
|
|
.PP
|
|
These can be run on a running backend using the rc command
|
|
backend/command (https://rclone.org/rc/#backend/command).
|
|
.SS get
|
|
.PP
|
|
Get command for fetching the drive config parameters
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone backend get remote: [options] [<arguments>+]
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
This is a get command which will be used to fetch the various drive
|
|
config parameters
|
|
.PP
|
|
Usage Examples:
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone backend get drive: [\-o service_account_file] [\-o chunk_size]
|
|
rclone rc backend/command command=get fs=drive: [\-o service_account_file] [\-o chunk_size]
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
Options:
|
|
.IP \[bu] 2
|
|
\[lq]chunk_size\[rq]: show the current upload chunk size
|
|
.IP \[bu] 2
|
|
\[lq]service_account_file\[rq]: show the current service account file
|
|
.SS set
|
|
.PP
|
|
Set command for updating the drive config parameters
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone backend set remote: [options] [<arguments>+]
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
This is a set command which will be used to update the various drive
|
|
config parameters
|
|
.PP
|
|
Usage Examples:
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone backend set drive: [\-o service_account_file=sa.json] [\-o chunk_size=67108864]
|
|
rclone rc backend/command command=set fs=drive: [\-o service_account_file=sa.json] [\-o chunk_size=67108864]
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
Options:
|
|
.IP \[bu] 2
|
|
\[lq]chunk_size\[rq]: update the current upload chunk size
|
|
.IP \[bu] 2
|
|
\[lq]service_account_file\[rq]: update the current service account file
|
|
.SS shortcut
|
|
.PP
|
|
Create shortcuts from files or directories
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone backend shortcut remote: [options] [<arguments>+]
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
This command creates shortcuts from files or directories.
|
|
.PP
|
|
Usage:
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone backend shortcut drive: source_item destination_shortcut
|
|
rclone backend shortcut drive: source_item \-o target=drive2: destination_shortcut
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
In the first example this creates a shortcut from the
|
|
\[lq]source_item\[rq] which can be a file or a directory to the
|
|
\[lq]destination_shortcut\[rq].
|
|
The \[lq]source_item\[rq] and the \[lq]destination_shortcut\[rq] should
|
|
be relative paths from \[lq]drive:\[rq]
|
|
.PP
|
|
In the second example this creates a shortcut from the
|
|
\[lq]source_item\[rq] relative to \[lq]drive:\[rq] to the
|
|
\[lq]destination_shortcut\[rq] relative to \[lq]drive2:\[rq].
|
|
This may fail with a permission error if the user authenticated with
|
|
\[lq]drive2:\[rq] can\[cq]t read files from \[lq]drive:\[rq].
|
|
.PP
|
|
Options:
|
|
.IP \[bu] 2
|
|
\[lq]target\[rq]: optional target remote for the shortcut destination
|
|
.SS Limitations
|
|
.PP
|
|
Drive has quite a lot of rate limiting.
|
|
This causes rclone to be limited to transferring about 2 files per
|
|
second only.
|
|
Individual files may be transferred much faster at 100s of MBytes/s but
|
|
lots of small files can take a long time.
|
|
.PP
|
|
Server side copies are also subject to a separate rate limit.
|
|
If you see User rate limit exceeded errors, wait at least 24 hours and
|
|
retry.
|
|
You can disable server side copies with \f[C]\-\-disable copy\f[R] to
|
|
download and upload the files if you prefer.
|
|
.SS Limitations of Google Docs
|
|
.PP
|
|
Google docs will appear as size \-1 in \f[C]rclone ls\f[R] and as size 0
|
|
in anything which uses the VFS layer, eg \f[C]rclone mount\f[R],
|
|
\f[C]rclone serve\f[R].
|
|
.PP
|
|
This is because rclone can\[cq]t find out the size of the Google docs
|
|
without downloading them.
|
|
.PP
|
|
Google docs will transfer correctly with \f[C]rclone sync\f[R],
|
|
\f[C]rclone copy\f[R] etc as rclone knows to ignore the size when doing
|
|
the transfer.
|
|
.PP
|
|
However an unfortunate consequence of this is that you may not be able
|
|
to download Google docs using \f[C]rclone mount\f[R].
|
|
If it doesn\[cq]t work you will get a 0 sized file.
|
|
If you try again the doc may gain its correct size and be downloadable.
|
|
Whether it will work on not depends on the application accessing the
|
|
mount and the OS you are running \- experiment to find out if it does
|
|
work for you!
|
|
.SS Duplicated files
|
|
.PP
|
|
Sometimes, for no reason I\[cq]ve been able to track down, drive will
|
|
duplicate a file that rclone uploads.
|
|
Drive unlike all the other remotes can have duplicated files.
|
|
.PP
|
|
Duplicated files cause problems with the syncing and you will see
|
|
messages in the log about duplicates.
|
|
.PP
|
|
Use \f[C]rclone dedupe\f[R] to fix duplicated files.
|
|
.PP
|
|
Note that this isn\[cq]t just a problem with rclone, even Google Photos
|
|
on Android duplicates files on drive sometimes.
|
|
.SS Rclone appears to be re\-copying files it shouldn\[cq]t
|
|
.PP
|
|
The most likely cause of this is the duplicated file issue above \- run
|
|
\f[C]rclone dedupe\f[R] and check your logs for duplicate object or
|
|
directory messages.
|
|
.PP
|
|
This can also be caused by a delay/caching on google drive\[cq]s end
|
|
when comparing directory listings.
|
|
Specifically with team drives used in combination with \[en]fast\-list.
|
|
Files that were uploaded recently may not appear on the directory list
|
|
sent to rclone when using \[en]fast\-list.
|
|
.PP
|
|
Waiting a moderate period of time between attempts (estimated to be
|
|
approximately 1 hour) and/or not using \[en]fast\-list both seem to be
|
|
effective in preventing the problem.
|
|
.SS Making your own client_id
|
|
.PP
|
|
When you use rclone with Google drive in its default configuration you
|
|
are using rclone\[cq]s client_id.
|
|
This is shared between all the rclone users.
|
|
There is a global rate limit on the number of queries per second that
|
|
each client_id can do set by Google.
|
|
rclone already has a high quota and I will continue to make sure it is
|
|
high enough by contacting Google.
|
|
.PP
|
|
It is strongly recommended to use your own client ID as the default
|
|
rclone ID is heavily used.
|
|
If you have multiple services running, it is recommended to use an API
|
|
key for each service.
|
|
The default Google quota is 10 transactions per second so it is
|
|
recommended to stay under that number as if you use more than that, it
|
|
will cause rclone to rate limit and make things slower.
|
|
.PP
|
|
Here is how to create your own Google Drive client ID for rclone:
|
|
.IP "1." 3
|
|
Log into the Google API Console (https://console.developers.google.com/)
|
|
with your Google account.
|
|
It doesn\[cq]t matter what Google account you use.
|
|
(It need not be the same account as the Google Drive you want to access)
|
|
.IP "2." 3
|
|
Select a project or create a new project.
|
|
.IP "3." 3
|
|
Under \[lq]ENABLE APIS AND SERVICES\[rq] search for \[lq]Drive\[rq], and
|
|
enable the \[lq]Google Drive API\[rq].
|
|
.IP "4." 3
|
|
Click \[lq]Credentials\[rq] in the left\-side panel (not \[lq]Create
|
|
credentials\[rq], which opens the wizard), then \[lq]Create
|
|
credentials\[rq]
|
|
.IP "5." 3
|
|
If you already configured an \[lq]Oauth Consent Screen\[rq], then skip
|
|
to the next step; if not, click on \[lq]CONFIGURE CONSENT SCREEN\[rq]
|
|
button (near the top right corner of the right panel), then select
|
|
\[lq]External\[rq] and click on \[lq]CREATE\[rq]; on the next screen,
|
|
enter an \[lq]Application name\[rq] (\[lq]rclone\[rq] is OK) then click
|
|
on \[lq]Save\[rq] (all other data is optional).
|
|
Click again on \[lq]Credentials\[rq] on the left panel to go back to the
|
|
\[lq]Credentials\[rq] screen.
|
|
.PP
|
|
(PS: if you are a GSuite user, you could also select \[lq]Internal\[rq]
|
|
instead of \[lq]External\[rq] above, but this has not been
|
|
tested/documented so far).
|
|
.IP "6." 3
|
|
Click on the \[lq]+ CREATE CREDENTIALS\[rq] button at the top of the
|
|
screen, then select \[lq]OAuth client ID\[rq].
|
|
.IP "7." 3
|
|
Choose an application type of \[lq]Desktop app\[rq] if you using a
|
|
Google account or \[lq]Other\[rq] if you using a GSuite account and
|
|
click \[lq]Create\[rq].
|
|
(the default name is fine)
|
|
.IP "8." 3
|
|
It will show you a client ID and client secret.
|
|
Use these values in rclone config to add a new remote or edit an
|
|
existing remote.
|
|
.PP
|
|
Be aware that, due to the \[lq]enhanced security\[rq] recently
|
|
introduced by Google, you are theoretically expected to \[lq]submit your
|
|
app for verification\[rq] and then wait a few weeks(!) for their
|
|
response; in practice, you can go right ahead and use the client ID and
|
|
client secret with rclone, the only issue will be a very scary
|
|
confirmation screen shown when you connect via your browser for rclone
|
|
to be able to get its token\-id (but as this only happens during the
|
|
remote configuration, it\[cq]s not such a big deal).
|
|
.PP
|
|
(Thanks to \[at]balazer on github for these instructions.)
|
|
.SS Google Photos
|
|
.PP
|
|
The rclone backend for Google
|
|
Photos (https://www.google.com/photos/about/) is a specialized backend
|
|
for transferring photos and videos to and from Google Photos.
|
|
.PP
|
|
\f[B]NB\f[R] The Google Photos API which rclone uses has quite a few
|
|
limitations, so please read the limitations section carefully to make
|
|
sure it is suitable for your use.
|
|
.SS Configuring Google Photos
|
|
.PP
|
|
The initial setup for google cloud storage involves getting a token from
|
|
Google Photos which you need to do in your browser.
|
|
\f[C]rclone config\f[R] walks you through it.
|
|
.PP
|
|
Here is an example of how to make a remote called \f[C]remote\f[R].
|
|
First run:
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone config
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
This will guide you through an interactive setup process:
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
No remotes found \- make a new one
|
|
n) New remote
|
|
s) Set configuration password
|
|
q) Quit config
|
|
n/s/q> n
|
|
name> remote
|
|
Type of storage to configure.
|
|
Enter a string value. Press Enter for the default (\[dq]\[dq]).
|
|
Choose a number from below, or type in your own value
|
|
[snip]
|
|
XX / Google Photos
|
|
\[rs] \[dq]google photos\[dq]
|
|
[snip]
|
|
Storage> google photos
|
|
** See help for google photos backend at: https://rclone.org/googlephotos/ **
|
|
|
|
Google Application Client Id
|
|
Leave blank normally.
|
|
Enter a string value. Press Enter for the default (\[dq]\[dq]).
|
|
client_id>
|
|
Google Application Client Secret
|
|
Leave blank normally.
|
|
Enter a string value. Press Enter for the default (\[dq]\[dq]).
|
|
client_secret>
|
|
Set to make the Google Photos backend read only.
|
|
|
|
If you choose read only then rclone will only request read only access
|
|
to your photos, otherwise rclone will request full access.
|
|
Enter a boolean value (true or false). Press Enter for the default (\[dq]false\[dq]).
|
|
read_only>
|
|
Edit advanced config? (y/n)
|
|
y) Yes
|
|
n) No
|
|
y/n> n
|
|
Remote config
|
|
Use auto config?
|
|
* Say Y if not sure
|
|
* Say N if you are working on a remote or headless machine
|
|
y) Yes
|
|
n) No
|
|
y/n> y
|
|
If your browser doesn\[aq]t open automatically go to the following link: http://127.0.0.1:53682/auth
|
|
Log in and authorize rclone for access
|
|
Waiting for code...
|
|
Got code
|
|
|
|
*** IMPORTANT: All media items uploaded to Google Photos with rclone
|
|
*** are stored in full resolution at original quality. These uploads
|
|
*** will count towards storage in your Google Account.
|
|
|
|
\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-
|
|
[remote]
|
|
type = google photos
|
|
token = {\[dq]access_token\[dq]:\[dq]XXX\[dq],\[dq]token_type\[dq]:\[dq]Bearer\[dq],\[dq]refresh_token\[dq]:\[dq]XXX\[dq],\[dq]expiry\[dq]:\[dq]2019\-06\-28T17:38:04.644930156+01:00\[dq]}
|
|
\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-
|
|
y) Yes this is OK
|
|
e) Edit this remote
|
|
d) Delete this remote
|
|
y/e/d> y
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
Note that rclone runs a webserver on your local machine to collect the
|
|
token as returned from Google if you use auto config mode.
|
|
This only runs from the moment it opens your browser to the moment you
|
|
get back the verification code.
|
|
This is on \f[C]http://127.0.0.1:53682/\f[R] and this may require you to
|
|
unblock it temporarily if you are running a host firewall, or use manual
|
|
mode.
|
|
.PP
|
|
This remote is called \f[C]remote\f[R] and can now be used like this
|
|
.PP
|
|
See all the albums in your photos
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone lsd remote:album
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
Make a new album
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone mkdir remote:album/newAlbum
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
List the contents of an album
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone ls remote:album/newAlbum
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
Sync \f[C]/home/local/images\f[R] to the Google Photos, removing any
|
|
excess files in the album.
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone sync /home/local/image remote:album/newAlbum
|
|
\f[R]
|
|
.fi
|
|
.SS Layout
|
|
.PP
|
|
As Google Photos is not a general purpose cloud storage system the
|
|
backend is laid out to help you navigate it.
|
|
.PP
|
|
The directories under \f[C]media\f[R] show different ways of
|
|
categorizing the media.
|
|
Each file will appear multiple times.
|
|
So if you want to make a backup of your google photos you might choose
|
|
to backup \f[C]remote:media/by\-month\f[R].
|
|
(\f[B]NB\f[R] \f[C]remote:media/by\-day\f[R] is rather slow at the
|
|
moment so avoid for syncing.)
|
|
.PP
|
|
Note that all your photos and videos will appear somewhere under
|
|
\f[C]media\f[R], but they may not appear under \f[C]album\f[R] unless
|
|
you\[cq]ve put them into albums.
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
/
|
|
\- upload
|
|
\- file1.jpg
|
|
\- file2.jpg
|
|
\- ...
|
|
\- media
|
|
\- all
|
|
\- file1.jpg
|
|
\- file2.jpg
|
|
\- ...
|
|
\- by\-year
|
|
\- 2000
|
|
\- file1.jpg
|
|
\- ...
|
|
\- 2001
|
|
\- file2.jpg
|
|
\- ...
|
|
\- ...
|
|
\- by\-month
|
|
\- 2000
|
|
\- 2000\-01
|
|
\- file1.jpg
|
|
\- ...
|
|
\- 2000\-02
|
|
\- file2.jpg
|
|
\- ...
|
|
\- ...
|
|
\- by\-day
|
|
\- 2000
|
|
\- 2000\-01\-01
|
|
\- file1.jpg
|
|
\- ...
|
|
\- 2000\-01\-02
|
|
\- file2.jpg
|
|
\- ...
|
|
\- ...
|
|
\- album
|
|
\- album name
|
|
\- album name/sub
|
|
\- shared\-album
|
|
\- album name
|
|
\- album name/sub
|
|
\- feature
|
|
\- favorites
|
|
\- file1.jpg
|
|
\- file2.jpg
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
There are two writable parts of the tree, the \f[C]upload\f[R] directory
|
|
and sub directories of the \f[C]album\f[R] directory.
|
|
.PP
|
|
The \f[C]upload\f[R] directory is for uploading files you don\[cq]t want
|
|
to put into albums.
|
|
This will be empty to start with and will contain the files you\[cq]ve
|
|
uploaded for one rclone session only, becoming empty again when you
|
|
restart rclone.
|
|
The use case for this would be if you have a load of files you just want
|
|
to once off dump into Google Photos.
|
|
For repeated syncing, uploading to \f[C]album\f[R] will work better.
|
|
.PP
|
|
Directories within the \f[C]album\f[R] directory are also writeable and
|
|
you may create new directories (albums) under \f[C]album\f[R].
|
|
If you copy files with a directory hierarchy in there then rclone will
|
|
create albums with the \f[C]/\f[R] character in them.
|
|
For example if you do
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone copy /path/to/images remote:album/images
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
and the images directory contains
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
images
|
|
\- file1.jpg
|
|
dir
|
|
file2.jpg
|
|
dir2
|
|
dir3
|
|
file3.jpg
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
Then rclone will create the following albums with the following files in
|
|
.IP \[bu] 2
|
|
images
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
file1.jpg
|
|
.RE
|
|
.IP \[bu] 2
|
|
images/dir
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
file2.jpg
|
|
.RE
|
|
.IP \[bu] 2
|
|
images/dir2/dir3
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
file3.jpg
|
|
.RE
|
|
.PP
|
|
This means that you can use the \f[C]album\f[R] path pretty much like a
|
|
normal filesystem and it is a good target for repeated syncing.
|
|
.PP
|
|
The \f[C]shared\-album\f[R] directory shows albums shared with you or by
|
|
you.
|
|
This is similar to the Sharing tab in the Google Photos web interface.
|
|
.SS Limitations
|
|
.PP
|
|
Only images and videos can be uploaded.
|
|
If you attempt to upload non videos or images or formats that Google
|
|
Photos doesn\[cq]t understand, rclone will upload the file, then Google
|
|
Photos will give an error when it is put turned into a media item.
|
|
.PP
|
|
Note that all media items uploaded to Google Photos through the API are
|
|
stored in full resolution at \[lq]original quality\[rq] and
|
|
\f[B]will\f[R] count towards your storage quota in your Google Account.
|
|
The API does \f[B]not\f[R] offer a way to upload in \[lq]high
|
|
quality\[rq] mode..
|
|
.SS Downloading Images
|
|
.PP
|
|
When Images are downloaded this strips EXIF location (according to the
|
|
docs and my tests).
|
|
This is a limitation of the Google Photos API and is covered by bug
|
|
#112096115 (https://issuetracker.google.com/issues/112096115).
|
|
.PP
|
|
\f[B]The current google API does not allow photos to be downloaded at
|
|
original resolution. This is very important if you are, for example,
|
|
relying on \[lq]Google Photos\[rq] as a backup of your photos. You will
|
|
not be able to use rclone to redownload original images. You could use
|
|
`google takeout' to recover the original photos as a last resort\f[R]
|
|
.SS Downloading Videos
|
|
.PP
|
|
When videos are downloaded they are downloaded in a really compressed
|
|
version of the video compared to downloading it via the Google Photos
|
|
web interface.
|
|
This is covered by bug
|
|
#113672044 (https://issuetracker.google.com/issues/113672044).
|
|
.SS Duplicates
|
|
.PP
|
|
If a file name is duplicated in a directory then rclone will add the
|
|
file ID into its name.
|
|
So two files called \f[C]file.jpg\f[R] would then appear as
|
|
\f[C]file {123456}.jpg\f[R] and \f[C]file {ABCDEF}.jpg\f[R] (the actual
|
|
IDs are a lot longer alas!).
|
|
.PP
|
|
If you upload the same image (with the same binary data) twice then
|
|
Google Photos will deduplicate it.
|
|
However it will retain the filename from the first upload which may
|
|
confuse rclone.
|
|
For example if you uploaded an image to \f[C]upload\f[R] then uploaded
|
|
the same image to \f[C]album/my_album\f[R] the filename of the image in
|
|
\f[C]album/my_album\f[R] will be what it was uploaded with initially,
|
|
not what you uploaded it with to \f[C]album\f[R].
|
|
In practise this shouldn\[cq]t cause too many problems.
|
|
.SS Modified time
|
|
.PP
|
|
The date shown of media in Google Photos is the creation date as
|
|
determined by the EXIF information, or the upload date if that is not
|
|
known.
|
|
.PP
|
|
This is not changeable by rclone and is not the modification date of the
|
|
media on local disk.
|
|
This means that rclone cannot use the dates from Google Photos for
|
|
syncing purposes.
|
|
.SS Size
|
|
.PP
|
|
The Google Photos API does not return the size of media.
|
|
This means that when syncing to Google Photos, rclone can only do a file
|
|
existence check.
|
|
.PP
|
|
It is possible to read the size of the media, but this needs an extra
|
|
HTTP HEAD request per media item so is \f[B]very slow\f[R] and uses up a
|
|
lot of transactions.
|
|
This can be enabled with the \f[C]\-\-gphotos\-read\-size\f[R] option or
|
|
the \f[C]read_size = true\f[R] config parameter.
|
|
.PP
|
|
If you want to use the backend with \f[C]rclone mount\f[R] you may need
|
|
to enable this flag (depending on your OS and application using the
|
|
photos) otherwise you may not be able to read media off the mount.
|
|
You\[cq]ll need to experiment to see if it works for you without the
|
|
flag.
|
|
.SS Albums
|
|
.PP
|
|
Rclone can only upload files to albums it created.
|
|
This is a limitation of the Google Photos
|
|
API (https://developers.google.com/photos/library/guides/manage-albums).
|
|
.PP
|
|
Rclone can remove files it uploaded from albums it created only.
|
|
.SS Deleting files
|
|
.PP
|
|
Rclone can remove files from albums it created, but note that the Google
|
|
Photos API does not allow media to be deleted permanently so this media
|
|
will still remain.
|
|
See bug #109759781 (https://issuetracker.google.com/issues/109759781).
|
|
.PP
|
|
Rclone cannot delete files anywhere except under \f[C]album\f[R].
|
|
.SS Deleting albums
|
|
.PP
|
|
The Google Photos API does not support deleting albums \- see bug
|
|
#135714733 (https://issuetracker.google.com/issues/135714733).
|
|
.SS Standard Options
|
|
.PP
|
|
Here are the standard options specific to google photos (Google Photos).
|
|
.SS \[en]gphotos\-client\-id
|
|
.PP
|
|
Google Application Client Id Leave blank normally.
|
|
.IP \[bu] 2
|
|
Config: client_id
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_GPHOTOS_CLIENT_ID
|
|
.IP \[bu] 2
|
|
Type: string
|
|
.IP \[bu] 2
|
|
Default: \[dq]\[dq]
|
|
.SS \[en]gphotos\-client\-secret
|
|
.PP
|
|
Google Application Client Secret Leave blank normally.
|
|
.IP \[bu] 2
|
|
Config: client_secret
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_GPHOTOS_CLIENT_SECRET
|
|
.IP \[bu] 2
|
|
Type: string
|
|
.IP \[bu] 2
|
|
Default: \[dq]\[dq]
|
|
.SS \[en]gphotos\-read\-only
|
|
.PP
|
|
Set to make the Google Photos backend read only.
|
|
.PP
|
|
If you choose read only then rclone will only request read only access
|
|
to your photos, otherwise rclone will request full access.
|
|
.IP \[bu] 2
|
|
Config: read_only
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_GPHOTOS_READ_ONLY
|
|
.IP \[bu] 2
|
|
Type: bool
|
|
.IP \[bu] 2
|
|
Default: false
|
|
.SS Advanced Options
|
|
.PP
|
|
Here are the advanced options specific to google photos (Google Photos).
|
|
.SS \[en]gphotos\-read\-size
|
|
.PP
|
|
Set to read the size of media items.
|
|
.PP
|
|
Normally rclone does not read the size of media items since this takes
|
|
another transaction.
|
|
This isn\[cq]t necessary for syncing.
|
|
However rclone mount needs to know the size of files in advance of
|
|
reading them, so setting this flag when using rclone mount is
|
|
recommended if you want to read the media.
|
|
.IP \[bu] 2
|
|
Config: read_size
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_GPHOTOS_READ_SIZE
|
|
.IP \[bu] 2
|
|
Type: bool
|
|
.IP \[bu] 2
|
|
Default: false
|
|
.SS \[en]gphotos\-start\-year
|
|
.PP
|
|
Year limits the photos to be downloaded to those which are uploaded
|
|
after the given year
|
|
.IP \[bu] 2
|
|
Config: start_year
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_GPHOTOS_START_YEAR
|
|
.IP \[bu] 2
|
|
Type: int
|
|
.IP \[bu] 2
|
|
Default: 2000
|
|
.SS HTTP
|
|
.PP
|
|
The HTTP remote is a read only remote for reading files of a webserver.
|
|
The webserver should provide file listings which rclone will read and
|
|
turn into a remote.
|
|
This has been tested with common webservers such as Apache/Nginx/Caddy
|
|
and will likely work with file listings from most web servers.
|
|
(If it doesn\[cq]t then please file an issue, or send a pull request!)
|
|
.PP
|
|
Paths are specified as \f[C]remote:\f[R] or
|
|
\f[C]remote:path/to/dir\f[R].
|
|
.PP
|
|
Here is an example of how to make a remote called \f[C]remote\f[R].
|
|
First run:
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone config
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
This will guide you through an interactive setup process:
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
No remotes found \- make a new one
|
|
n) New remote
|
|
s) Set configuration password
|
|
q) Quit config
|
|
n/s/q> n
|
|
name> remote
|
|
Type of storage to configure.
|
|
Choose a number from below, or type in your own value
|
|
[snip]
|
|
XX / http Connection
|
|
\[rs] \[dq]http\[dq]
|
|
[snip]
|
|
Storage> http
|
|
URL of http host to connect to
|
|
Choose a number from below, or type in your own value
|
|
1 / Connect to example.com
|
|
\[rs] \[dq]https://example.com\[dq]
|
|
url> https://beta.rclone.org
|
|
Remote config
|
|
\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-
|
|
[remote]
|
|
url = https://beta.rclone.org
|
|
\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-
|
|
y) Yes this is OK
|
|
e) Edit this remote
|
|
d) Delete this remote
|
|
y/e/d> y
|
|
Current remotes:
|
|
|
|
Name Type
|
|
==== ====
|
|
remote http
|
|
|
|
e) Edit existing remote
|
|
n) New remote
|
|
d) Delete remote
|
|
r) Rename remote
|
|
c) Copy remote
|
|
s) Set configuration password
|
|
q) Quit config
|
|
e/n/d/r/c/s/q> q
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
This remote is called \f[C]remote\f[R] and can now be used like this
|
|
.PP
|
|
See all the top level directories
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone lsd remote:
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
List the contents of a directory
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone ls remote:directory
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
Sync the remote \f[C]directory\f[R] to \f[C]/home/local/directory\f[R],
|
|
deleting any excess files.
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone sync remote:directory /home/local/directory
|
|
\f[R]
|
|
.fi
|
|
.SS Read only
|
|
.PP
|
|
This remote is read only \- you can\[cq]t upload files to an HTTP
|
|
server.
|
|
.SS Modified time
|
|
.PP
|
|
Most HTTP servers store time accurate to 1 second.
|
|
.SS Checksum
|
|
.PP
|
|
No checksums are stored.
|
|
.SS Usage without a config file
|
|
.PP
|
|
Since the http remote only has one config parameter it is easy to use
|
|
without a config file:
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone lsd \-\-http\-url https://beta.rclone.org :http:
|
|
\f[R]
|
|
.fi
|
|
.SS Standard Options
|
|
.PP
|
|
Here are the standard options specific to http (http Connection).
|
|
.SS \[en]http\-url
|
|
.PP
|
|
URL of http host to connect to
|
|
.IP \[bu] 2
|
|
Config: url
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_HTTP_URL
|
|
.IP \[bu] 2
|
|
Type: string
|
|
.IP \[bu] 2
|
|
Default: \[dq]\[dq]
|
|
.IP \[bu] 2
|
|
Examples:
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
\[lq]https://example.com\[rq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Connect to example.com
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[lq]https://user:pass\[at]example.com\[rq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Connect to example.com using a username and password
|
|
.RE
|
|
.RE
|
|
.SS Advanced Options
|
|
.PP
|
|
Here are the advanced options specific to http (http Connection).
|
|
.SS \[en]http\-headers
|
|
.PP
|
|
Set HTTP headers for all transactions
|
|
.PP
|
|
Use this to set additional HTTP headers for all transactions
|
|
.PP
|
|
The input format is comma separated list of key,value pairs.
|
|
Standard CSV encoding (https://godoc.org/encoding/csv) may be used.
|
|
.PP
|
|
For example to set a Cookie use `Cookie,name=value', or
|
|
`\[lq]Cookie\[rq],\[lq]name=value\[rq]'.
|
|
.PP
|
|
You can set multiple headers, eg
|
|
`\[lq]Cookie\[rq],\[lq]name=value\[rq],\[lq]Authorization\[rq],\[lq]xxx\[rq]'.
|
|
.IP \[bu] 2
|
|
Config: headers
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_HTTP_HEADERS
|
|
.IP \[bu] 2
|
|
Type: CommaSepList
|
|
.IP \[bu] 2
|
|
Default:
|
|
.SS \[en]http\-no\-slash
|
|
.PP
|
|
Set this if the site doesn\[cq]t end directories with /
|
|
.PP
|
|
Use this if your target website does not use / on the end of
|
|
directories.
|
|
.PP
|
|
A / on the end of a path is how rclone normally tells the difference
|
|
between files and directories.
|
|
If this flag is set, then rclone will treat all files with
|
|
Content\-Type: text/html as directories and read URLs from them rather
|
|
than downloading them.
|
|
.PP
|
|
Note that this may cause rclone to confuse genuine HTML files with
|
|
directories.
|
|
.IP \[bu] 2
|
|
Config: no_slash
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_HTTP_NO_SLASH
|
|
.IP \[bu] 2
|
|
Type: bool
|
|
.IP \[bu] 2
|
|
Default: false
|
|
.SS \[en]http\-no\-head
|
|
.PP
|
|
Don\[cq]t use HEAD requests to find file sizes in dir listing
|
|
.PP
|
|
If your site is being very slow to load then you can try this option.
|
|
Normally rclone does a HEAD request for each potential file in a
|
|
directory listing to:
|
|
.IP \[bu] 2
|
|
find its size
|
|
.IP \[bu] 2
|
|
check it really exists
|
|
.IP \[bu] 2
|
|
check to see if it is a directory
|
|
.PP
|
|
If you set this option, rclone will not do the HEAD request.
|
|
This will mean
|
|
.IP \[bu] 2
|
|
directory listings are much quicker
|
|
.IP \[bu] 2
|
|
rclone won\[cq]t have the times or sizes of any files
|
|
.IP \[bu] 2
|
|
some files that don\[cq]t exist may be in the listing
|
|
.IP \[bu] 2
|
|
Config: no_head
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_HTTP_NO_HEAD
|
|
.IP \[bu] 2
|
|
Type: bool
|
|
.IP \[bu] 2
|
|
Default: false
|
|
.SS Hubic
|
|
.PP
|
|
Paths are specified as \f[C]remote:path\f[R]
|
|
.PP
|
|
Paths are specified as \f[C]remote:container\f[R] (or \f[C]remote:\f[R]
|
|
for the \f[C]lsd\f[R] command.) You may put subdirectories in too, eg
|
|
\f[C]remote:container/path/to/dir\f[R].
|
|
.PP
|
|
The initial setup for Hubic involves getting a token from Hubic which
|
|
you need to do in your browser.
|
|
\f[C]rclone config\f[R] walks you through it.
|
|
.PP
|
|
Here is an example of how to make a remote called \f[C]remote\f[R].
|
|
First run:
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone config
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
This will guide you through an interactive setup process:
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
n) New remote
|
|
s) Set configuration password
|
|
n/s> n
|
|
name> remote
|
|
Type of storage to configure.
|
|
Choose a number from below, or type in your own value
|
|
[snip]
|
|
XX / Hubic
|
|
\[rs] \[dq]hubic\[dq]
|
|
[snip]
|
|
Storage> hubic
|
|
Hubic Client Id \- leave blank normally.
|
|
client_id>
|
|
Hubic Client Secret \- leave blank normally.
|
|
client_secret>
|
|
Remote config
|
|
Use auto config?
|
|
* Say Y if not sure
|
|
* Say N if you are working on a remote or headless machine
|
|
y) Yes
|
|
n) No
|
|
y/n> y
|
|
If your browser doesn\[aq]t open automatically go to the following link: http://127.0.0.1:53682/auth
|
|
Log in and authorize rclone for access
|
|
Waiting for code...
|
|
Got code
|
|
\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-
|
|
[remote]
|
|
client_id =
|
|
client_secret =
|
|
token = {\[dq]access_token\[dq]:\[dq]XXXXXX\[dq]}
|
|
\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-
|
|
y) Yes this is OK
|
|
e) Edit this remote
|
|
d) Delete this remote
|
|
y/e/d> y
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
See the remote setup docs (https://rclone.org/remote_setup/) for how to
|
|
set it up on a machine with no Internet browser available.
|
|
.PP
|
|
Note that rclone runs a webserver on your local machine to collect the
|
|
token as returned from Hubic.
|
|
This only runs from the moment it opens your browser to the moment you
|
|
get back the verification code.
|
|
This is on \f[C]http://127.0.0.1:53682/\f[R] and this it may require you
|
|
to unblock it temporarily if you are running a host firewall.
|
|
.PP
|
|
Once configured you can then use \f[C]rclone\f[R] like this,
|
|
.PP
|
|
List containers in the top level of your Hubic
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone lsd remote:
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
List all the files in your Hubic
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone ls remote:
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
To copy a local directory to an Hubic directory called backup
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone copy /home/source remote:backup
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
If you want the directory to be visible in the official \f[I]Hubic
|
|
browser\f[R], you need to copy your files to the \f[C]default\f[R]
|
|
directory
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone copy /home/source remote:default/backup
|
|
\f[R]
|
|
.fi
|
|
.SS \[en]fast\-list
|
|
.PP
|
|
This remote supports \f[C]\-\-fast\-list\f[R] which allows you to use
|
|
fewer transactions in exchange for more memory.
|
|
See the rclone docs (https://rclone.org/docs/#fast-list) for more
|
|
details.
|
|
.SS Modified time
|
|
.PP
|
|
The modified time is stored as metadata on the object as
|
|
\f[C]X\-Object\-Meta\-Mtime\f[R] as floating point since the epoch
|
|
accurate to 1 ns.
|
|
.PP
|
|
This is a de facto standard (used in the official python\-swiftclient
|
|
amongst others) for storing the modification time for an object.
|
|
.PP
|
|
Note that Hubic wraps the Swift backend, so most of the properties of
|
|
are the same.
|
|
.SS Standard Options
|
|
.PP
|
|
Here are the standard options specific to hubic (Hubic).
|
|
.SS \[en]hubic\-client\-id
|
|
.PP
|
|
Hubic Client Id Leave blank normally.
|
|
.IP \[bu] 2
|
|
Config: client_id
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_HUBIC_CLIENT_ID
|
|
.IP \[bu] 2
|
|
Type: string
|
|
.IP \[bu] 2
|
|
Default: \[dq]\[dq]
|
|
.SS \[en]hubic\-client\-secret
|
|
.PP
|
|
Hubic Client Secret Leave blank normally.
|
|
.IP \[bu] 2
|
|
Config: client_secret
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_HUBIC_CLIENT_SECRET
|
|
.IP \[bu] 2
|
|
Type: string
|
|
.IP \[bu] 2
|
|
Default: \[dq]\[dq]
|
|
.SS Advanced Options
|
|
.PP
|
|
Here are the advanced options specific to hubic (Hubic).
|
|
.SS \[en]hubic\-chunk\-size
|
|
.PP
|
|
Above this size files will be chunked into a _segments container.
|
|
.PP
|
|
Above this size files will be chunked into a _segments container.
|
|
The default for this is 5GB which is its maximum value.
|
|
.IP \[bu] 2
|
|
Config: chunk_size
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_HUBIC_CHUNK_SIZE
|
|
.IP \[bu] 2
|
|
Type: SizeSuffix
|
|
.IP \[bu] 2
|
|
Default: 5G
|
|
.SS \[en]hubic\-no\-chunk
|
|
.PP
|
|
Don\[cq]t chunk files during streaming upload.
|
|
.PP
|
|
When doing streaming uploads (eg using rcat or mount) setting this flag
|
|
will cause the swift backend to not upload chunked files.
|
|
.PP
|
|
This will limit the maximum upload size to 5GB.
|
|
However non chunked files are easier to deal with and have an MD5SUM.
|
|
.PP
|
|
Rclone will still chunk files bigger than chunk_size when doing normal
|
|
copy operations.
|
|
.IP \[bu] 2
|
|
Config: no_chunk
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_HUBIC_NO_CHUNK
|
|
.IP \[bu] 2
|
|
Type: bool
|
|
.IP \[bu] 2
|
|
Default: false
|
|
.SS \[en]hubic\-encoding
|
|
.PP
|
|
This sets the encoding for the backend.
|
|
.PP
|
|
See: the encoding section in the
|
|
overview (https://rclone.org/overview/#encoding) for more info.
|
|
.IP \[bu] 2
|
|
Config: encoding
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_HUBIC_ENCODING
|
|
.IP \[bu] 2
|
|
Type: MultiEncoder
|
|
.IP \[bu] 2
|
|
Default: Slash,InvalidUtf8
|
|
.SS Limitations
|
|
.PP
|
|
This uses the normal OpenStack Swift mechanism to refresh the Swift API
|
|
credentials and ignores the expires field returned by the Hubic API.
|
|
.PP
|
|
The Swift API doesn\[cq]t return a correct MD5SUM for segmented files
|
|
(Dynamic or Static Large Objects) so rclone won\[cq]t check or use the
|
|
MD5SUM for these.
|
|
.SS Jottacloud
|
|
.PP
|
|
Jottacloud is a cloud storage service provider from a Norwegian company,
|
|
using its own datacenters in Norway.
|
|
.PP
|
|
In addition to the official service at
|
|
jottacloud.com (https://www.jottacloud.com/), there are also several
|
|
whitelabel versions which should work with this backend.
|
|
.PP
|
|
Paths are specified as \f[C]remote:path\f[R]
|
|
.PP
|
|
Paths may be as deep as required, eg
|
|
\f[C]remote:directory/subdirectory\f[R].
|
|
.SS Setup
|
|
.PP
|
|
To configure Jottacloud you will need to generate a personal security
|
|
token in the Jottacloud web interface.
|
|
You will the option to do in your account security
|
|
settings (https://www.jottacloud.com/web/secure) (for whitelabel version
|
|
you need to find this page in its web interface).
|
|
Note that the web interface may refer to this token as a JottaCli token.
|
|
.PP
|
|
Here is an example of how to make a remote called \f[C]remote\f[R].
|
|
First run:
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone config
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
This will guide you through an interactive setup process:
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
No remotes found \- make a new one
|
|
n) New remote
|
|
s) Set configuration password
|
|
q) Quit config
|
|
n/s/q> n
|
|
name> jotta
|
|
Type of storage to configure.
|
|
Enter a string value. Press Enter for the default (\[dq]\[dq]).
|
|
Choose a number from below, or type in your own value
|
|
[snip]
|
|
XX / Jottacloud
|
|
\[rs] \[dq]jottacloud\[dq]
|
|
[snip]
|
|
Storage> jottacloud
|
|
** See help for jottacloud backend at: https://rclone.org/jottacloud/ **
|
|
|
|
Edit advanced config? (y/n)
|
|
y) Yes
|
|
n) No
|
|
y/n> n
|
|
Remote config
|
|
|
|
Generate a personal login token here: https://www.jottacloud.com/web/secure
|
|
Login Token> <your token here>
|
|
|
|
Do you want to use a non standard device/mountpoint e.g. for accessing files uploaded using the official Jottacloud client?
|
|
|
|
y) Yes
|
|
n) No
|
|
y/n> y
|
|
Please select the device to use. Normally this will be Jotta
|
|
Choose a number from below, or type in an existing value
|
|
1 > DESKTOP\-3H31129
|
|
2 > fla1
|
|
3 > Jotta
|
|
Devices> 3
|
|
Please select the mountpoint to user. Normally this will be Archive
|
|
Choose a number from below, or type in an existing value
|
|
1 > Archive
|
|
2 > Links
|
|
3 > Sync
|
|
|
|
Mountpoints> 1
|
|
\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-
|
|
[jotta]
|
|
type = jottacloud
|
|
user = 0xC4KE\[at]gmail.com
|
|
token = {........}
|
|
device = Jotta
|
|
mountpoint = Archive
|
|
configVersion = 1
|
|
\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-
|
|
y) Yes this is OK
|
|
e) Edit this remote
|
|
d) Delete this remote
|
|
y/e/d> y
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
Once configured you can then use \f[C]rclone\f[R] like this,
|
|
.PP
|
|
List directories in top level of your Jottacloud
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone lsd remote:
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
List all the files in your Jottacloud
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone ls remote:
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
To copy a local directory to an Jottacloud directory called backup
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone copy /home/source remote:backup
|
|
\f[R]
|
|
.fi
|
|
.SS Devices and Mountpoints
|
|
.PP
|
|
The official Jottacloud client registers a device for each computer you
|
|
install it on, and then creates a mountpoint for each folder you select
|
|
for Backup.
|
|
The web interface uses a special device called Jotta for the Archive and
|
|
Sync mountpoints.
|
|
In most cases you\[cq]ll want to use the Jotta/Archive
|
|
device/mountpoint, however if you want to access files uploaded by any
|
|
of the official clients rclone provides the option to select other
|
|
devices and mountpoints during config.
|
|
.PP
|
|
The built\-in Jotta device may also contain several other mountpoints,
|
|
such as: Latest, Links, Shared and Trash.
|
|
These are special mountpoints with a different internal representation
|
|
than the \[lq]regular\[rq] mountpoints.
|
|
Rclone will only to a very limited degree support them.
|
|
Generally you should avoid these, unless you know what you are doing.
|
|
.SS \[en]fast\-list
|
|
.PP
|
|
This remote supports \f[C]\-\-fast\-list\f[R] which allows you to use
|
|
fewer transactions in exchange for more memory.
|
|
See the rclone docs (https://rclone.org/docs/#fast-list) for more
|
|
details.
|
|
.PP
|
|
Note that the implementation in Jottacloud always uses only a single API
|
|
request to get the entire list, so for large folders this could lead to
|
|
long wait time before the first results are shown.
|
|
.SS Modified time and hashes
|
|
.PP
|
|
Jottacloud allows modification times to be set on objects accurate to 1
|
|
second.
|
|
These will be used to detect whether objects need syncing or not.
|
|
.PP
|
|
Jottacloud supports MD5 type hashes, so you can use the
|
|
\f[C]\-\-checksum\f[R] flag.
|
|
.PP
|
|
Note that Jottacloud requires the MD5 hash before upload so if the
|
|
source does not have an MD5 checksum then the file will be cached
|
|
temporarily on disk (wherever the \f[C]TMPDIR\f[R] environment variable
|
|
points to) before it is uploaded.
|
|
Small files will be cached in memory \- see the
|
|
\[en]jottacloud\-md5\-memory\-limit flag.
|
|
.SS Restricted filename characters
|
|
.PP
|
|
In addition to the default restricted characters
|
|
set (https://rclone.org/overview/#restricted-characters) the following
|
|
characters are also replaced:
|
|
.PP
|
|
.TS
|
|
tab(@);
|
|
l c c.
|
|
T{
|
|
Character
|
|
T}@T{
|
|
Value
|
|
T}@T{
|
|
Replacement
|
|
T}
|
|
_
|
|
T{
|
|
\[dq]
|
|
T}@T{
|
|
0x22
|
|
T}@T{
|
|
\[uFF02]
|
|
T}
|
|
T{
|
|
*
|
|
T}@T{
|
|
0x2A
|
|
T}@T{
|
|
\[uFF0A]
|
|
T}
|
|
T{
|
|
:
|
|
T}@T{
|
|
0x3A
|
|
T}@T{
|
|
\[uFF1A]
|
|
T}
|
|
T{
|
|
<
|
|
T}@T{
|
|
0x3C
|
|
T}@T{
|
|
\[uFF1C]
|
|
T}
|
|
T{
|
|
>
|
|
T}@T{
|
|
0x3E
|
|
T}@T{
|
|
\[uFF1E]
|
|
T}
|
|
T{
|
|
?
|
|
T}@T{
|
|
0x3F
|
|
T}@T{
|
|
\[uFF1F]
|
|
T}
|
|
T{
|
|
|
|
|
T}@T{
|
|
0x7C
|
|
T}@T{
|
|
\[uFF5C]
|
|
T}
|
|
.TE
|
|
.PP
|
|
Invalid UTF\-8 bytes will also be
|
|
replaced (https://rclone.org/overview/#invalid-utf8), as they can\[cq]t
|
|
be used in XML strings.
|
|
.SS Deleting files
|
|
.PP
|
|
By default rclone will send all files to the trash when deleting files.
|
|
They will be permanently deleted automatically after 30 days.
|
|
You may bypass the trash and permanently delete files immediately by
|
|
using the \[en]jottacloud\-hard\-delete flag, or set the equivalent
|
|
environment variable.
|
|
Emptying the trash is supported by the
|
|
cleanup (https://rclone.org/commands/rclone_cleanup/) command.
|
|
.SS Versions
|
|
.PP
|
|
Jottacloud supports file versioning.
|
|
When rclone uploads a new version of a file it creates a new version of
|
|
it.
|
|
Currently rclone only supports retrieving the current version but older
|
|
versions can be accessed via the Jottacloud Website.
|
|
.SS Quota information
|
|
.PP
|
|
To view your current quota you can use the
|
|
\f[C]rclone about remote:\f[R] command which will display your usage
|
|
limit (unless it is unlimited) and the current usage.
|
|
.SS Advanced Options
|
|
.PP
|
|
Here are the advanced options specific to jottacloud (Jottacloud).
|
|
.SS \[en]jottacloud\-md5\-memory\-limit
|
|
.PP
|
|
Files bigger than this will be cached on disk to calculate the MD5 if
|
|
required.
|
|
.IP \[bu] 2
|
|
Config: md5_memory_limit
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_JOTTACLOUD_MD5_MEMORY_LIMIT
|
|
.IP \[bu] 2
|
|
Type: SizeSuffix
|
|
.IP \[bu] 2
|
|
Default: 10M
|
|
.SS \[en]jottacloud\-trashed\-only
|
|
.PP
|
|
Only show files that are in the trash.
|
|
This will show trashed files in their original directory structure.
|
|
.IP \[bu] 2
|
|
Config: trashed_only
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_JOTTACLOUD_TRASHED_ONLY
|
|
.IP \[bu] 2
|
|
Type: bool
|
|
.IP \[bu] 2
|
|
Default: false
|
|
.SS \[en]jottacloud\-hard\-delete
|
|
.PP
|
|
Delete files permanently rather than putting them into the trash.
|
|
.IP \[bu] 2
|
|
Config: hard_delete
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_JOTTACLOUD_HARD_DELETE
|
|
.IP \[bu] 2
|
|
Type: bool
|
|
.IP \[bu] 2
|
|
Default: false
|
|
.SS \[en]jottacloud\-unlink
|
|
.PP
|
|
Remove existing public link to file/folder with link command rather than
|
|
creating.
|
|
Default is false, meaning link command will create or retrieve public
|
|
link.
|
|
.IP \[bu] 2
|
|
Config: unlink
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_JOTTACLOUD_UNLINK
|
|
.IP \[bu] 2
|
|
Type: bool
|
|
.IP \[bu] 2
|
|
Default: false
|
|
.SS \[en]jottacloud\-upload\-resume\-limit
|
|
.PP
|
|
Files bigger than this can be resumed if the upload fail\[cq]s.
|
|
.IP \[bu] 2
|
|
Config: upload_resume_limit
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_JOTTACLOUD_UPLOAD_RESUME_LIMIT
|
|
.IP \[bu] 2
|
|
Type: SizeSuffix
|
|
.IP \[bu] 2
|
|
Default: 10M
|
|
.SS \[en]jottacloud\-encoding
|
|
.PP
|
|
This sets the encoding for the backend.
|
|
.PP
|
|
See: the encoding section in the
|
|
overview (https://rclone.org/overview/#encoding) for more info.
|
|
.IP \[bu] 2
|
|
Config: encoding
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_JOTTACLOUD_ENCODING
|
|
.IP \[bu] 2
|
|
Type: MultiEncoder
|
|
.IP \[bu] 2
|
|
Default:
|
|
Slash,LtGt,DoubleQuote,Colon,Question,Asterisk,Pipe,Del,Ctl,InvalidUtf8,Dot
|
|
.SS Limitations
|
|
.PP
|
|
Note that Jottacloud is case insensitive so you can\[cq]t have a file
|
|
called \[lq]Hello.doc\[rq] and one called \[lq]hello.doc\[rq].
|
|
.PP
|
|
There are quite a few characters that can\[cq]t be in Jottacloud file
|
|
names.
|
|
Rclone will map these names to and from an identical looking unicode
|
|
equivalent.
|
|
For example if a file has a ?
|
|
in it will be mapped to \[uFF1F] instead.
|
|
.PP
|
|
Jottacloud only supports filenames up to 255 characters in length.
|
|
.SS Troubleshooting
|
|
.PP
|
|
Jottacloud exhibits some inconsistent behaviours regarding deleted files
|
|
and folders which may cause Copy, Move and DirMove operations to
|
|
previously deleted paths to fail.
|
|
Emptying the trash should help in such cases.
|
|
.SS Koofr
|
|
.PP
|
|
Paths are specified as \f[C]remote:path\f[R]
|
|
.PP
|
|
Paths may be as deep as required, eg
|
|
\f[C]remote:directory/subdirectory\f[R].
|
|
.PP
|
|
The initial setup for Koofr involves creating an application password
|
|
for rclone.
|
|
You can do that by opening the Koofr web
|
|
application (https://app.koofr.net/app/admin/preferences/password),
|
|
giving the password a nice name like \f[C]rclone\f[R] and clicking on
|
|
generate.
|
|
.PP
|
|
Here is an example of how to make a remote called \f[C]koofr\f[R].
|
|
First run:
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone config
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
This will guide you through an interactive setup process:
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
No remotes found \- make a new one
|
|
n) New remote
|
|
s) Set configuration password
|
|
q) Quit config
|
|
n/s/q> n
|
|
name> koofr
|
|
Type of storage to configure.
|
|
Enter a string value. Press Enter for the default (\[dq]\[dq]).
|
|
Choose a number from below, or type in your own value
|
|
[snip]
|
|
XX / Koofr
|
|
\[rs] \[dq]koofr\[dq]
|
|
[snip]
|
|
Storage> koofr
|
|
** See help for koofr backend at: https://rclone.org/koofr/ **
|
|
|
|
Your Koofr user name
|
|
Enter a string value. Press Enter for the default (\[dq]\[dq]).
|
|
user> USER\[at]NAME
|
|
Your Koofr password for rclone (generate one at https://app.koofr.net/app/admin/preferences/password)
|
|
y) Yes type in my own password
|
|
g) Generate random password
|
|
y/g> y
|
|
Enter the password:
|
|
password:
|
|
Confirm the password:
|
|
password:
|
|
Edit advanced config? (y/n)
|
|
y) Yes
|
|
n) No
|
|
y/n> n
|
|
Remote config
|
|
\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-
|
|
[koofr]
|
|
type = koofr
|
|
baseurl = https://app.koofr.net
|
|
user = USER\[at]NAME
|
|
password = *** ENCRYPTED ***
|
|
\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-
|
|
y) Yes this is OK
|
|
e) Edit this remote
|
|
d) Delete this remote
|
|
y/e/d> y
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
You can choose to edit advanced config in order to enter your own
|
|
service URL if you use an on\-premise or white label Koofr instance, or
|
|
choose an alternative mount instead of your primary storage.
|
|
.PP
|
|
Once configured you can then use \f[C]rclone\f[R] like this,
|
|
.PP
|
|
List directories in top level of your Koofr
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone lsd koofr:
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
List all the files in your Koofr
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone ls koofr:
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
To copy a local directory to an Koofr directory called backup
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone copy /home/source remote:backup
|
|
\f[R]
|
|
.fi
|
|
.SS Restricted filename characters
|
|
.PP
|
|
In addition to the default restricted characters
|
|
set (https://rclone.org/overview/#restricted-characters) the following
|
|
characters are also replaced:
|
|
.PP
|
|
.TS
|
|
tab(@);
|
|
l c c.
|
|
T{
|
|
Character
|
|
T}@T{
|
|
Value
|
|
T}@T{
|
|
Replacement
|
|
T}
|
|
_
|
|
T{
|
|
\[rs]
|
|
T}@T{
|
|
0x5C
|
|
T}@T{
|
|
\[uFF3C]
|
|
T}
|
|
.TE
|
|
.PP
|
|
Invalid UTF\-8 bytes will also be
|
|
replaced (https://rclone.org/overview/#invalid-utf8), as they can\[cq]t
|
|
be used in XML strings.
|
|
.SS Standard Options
|
|
.PP
|
|
Here are the standard options specific to koofr (Koofr).
|
|
.SS \[en]koofr\-user
|
|
.PP
|
|
Your Koofr user name
|
|
.IP \[bu] 2
|
|
Config: user
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_KOOFR_USER
|
|
.IP \[bu] 2
|
|
Type: string
|
|
.IP \[bu] 2
|
|
Default: \[dq]\[dq]
|
|
.SS \[en]koofr\-password
|
|
.PP
|
|
Your Koofr password for rclone (generate one at
|
|
https://app.koofr.net/app/admin/preferences/password)
|
|
.IP \[bu] 2
|
|
Config: password
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_KOOFR_PASSWORD
|
|
.IP \[bu] 2
|
|
Type: string
|
|
.IP \[bu] 2
|
|
Default: \[dq]\[dq]
|
|
.SS Advanced Options
|
|
.PP
|
|
Here are the advanced options specific to koofr (Koofr).
|
|
.SS \[en]koofr\-endpoint
|
|
.PP
|
|
The Koofr API endpoint to use
|
|
.IP \[bu] 2
|
|
Config: endpoint
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_KOOFR_ENDPOINT
|
|
.IP \[bu] 2
|
|
Type: string
|
|
.IP \[bu] 2
|
|
Default: \[lq]https://app.koofr.net\[rq]
|
|
.SS \[en]koofr\-mountid
|
|
.PP
|
|
Mount ID of the mount to use.
|
|
If omitted, the primary mount is used.
|
|
.IP \[bu] 2
|
|
Config: mountid
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_KOOFR_MOUNTID
|
|
.IP \[bu] 2
|
|
Type: string
|
|
.IP \[bu] 2
|
|
Default: \[dq]\[dq]
|
|
.SS \[en]koofr\-setmtime
|
|
.PP
|
|
Does the backend support setting modification time.
|
|
Set this to false if you use a mount ID that points to a Dropbox or
|
|
Amazon Drive backend.
|
|
.IP \[bu] 2
|
|
Config: setmtime
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_KOOFR_SETMTIME
|
|
.IP \[bu] 2
|
|
Type: bool
|
|
.IP \[bu] 2
|
|
Default: true
|
|
.SS \[en]koofr\-encoding
|
|
.PP
|
|
This sets the encoding for the backend.
|
|
.PP
|
|
See: the encoding section in the
|
|
overview (https://rclone.org/overview/#encoding) for more info.
|
|
.IP \[bu] 2
|
|
Config: encoding
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_KOOFR_ENCODING
|
|
.IP \[bu] 2
|
|
Type: MultiEncoder
|
|
.IP \[bu] 2
|
|
Default: Slash,BackSlash,Del,Ctl,InvalidUtf8,Dot
|
|
.SS Limitations
|
|
.PP
|
|
Note that Koofr is case insensitive so you can\[cq]t have a file called
|
|
\[lq]Hello.doc\[rq] and one called \[lq]hello.doc\[rq].
|
|
.SS Mail.ru Cloud
|
|
.PP
|
|
Mail.ru Cloud (https://cloud.mail.ru/) is a cloud storage provided by a
|
|
Russian internet company Mail.Ru Group (https://mail.ru).
|
|
The official desktop client is Disk\-O: (https://disk-o.cloud/),
|
|
available only on Windows.
|
|
(Please note that official sites are in Russian)
|
|
.PP
|
|
Currently it is recommended to disable 2FA on Mail.ru accounts intended
|
|
for rclone until it gets eventually implemented.
|
|
.SS Features highlights
|
|
.IP \[bu] 2
|
|
Paths may be as deep as required, eg
|
|
\f[C]remote:directory/subdirectory\f[R]
|
|
.IP \[bu] 2
|
|
Files have a \f[C]last modified time\f[R] property, directories
|
|
don\[cq]t
|
|
.IP \[bu] 2
|
|
Deleted files are by default moved to the trash
|
|
.IP \[bu] 2
|
|
Files and directories can be shared via public links
|
|
.IP \[bu] 2
|
|
Partial uploads or streaming are not supported, file size must be known
|
|
before upload
|
|
.IP \[bu] 2
|
|
Maximum file size is limited to 2G for a free account, unlimited for
|
|
paid accounts
|
|
.IP \[bu] 2
|
|
Storage keeps hash for all files and performs transparent deduplication,
|
|
the hash algorithm is a modified SHA1
|
|
.IP \[bu] 2
|
|
If a particular file is already present in storage, one can quickly
|
|
submit file hash instead of long file upload (this optimization is
|
|
supported by rclone)
|
|
.SS Configuration
|
|
.PP
|
|
Here is an example of making a mailru configuration.
|
|
First create a Mail.ru Cloud account and choose a tariff, then run
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone config
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
This will guide you through an interactive setup process:
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
No remotes found \- make a new one
|
|
n) New remote
|
|
s) Set configuration password
|
|
q) Quit config
|
|
n/s/q> n
|
|
name> remote
|
|
Type of storage to configure.
|
|
Type of storage to configure.
|
|
Enter a string value. Press Enter for the default (\[dq]\[dq]).
|
|
Choose a number from below, or type in your own value
|
|
[snip]
|
|
XX / Mail.ru Cloud
|
|
\[rs] \[dq]mailru\[dq]
|
|
[snip]
|
|
Storage> mailru
|
|
User name (usually email)
|
|
Enter a string value. Press Enter for the default (\[dq]\[dq]).
|
|
user> username\[at]mail.ru
|
|
Password
|
|
y) Yes type in my own password
|
|
g) Generate random password
|
|
y/g> y
|
|
Enter the password:
|
|
password:
|
|
Confirm the password:
|
|
password:
|
|
Skip full upload if there is another file with same data hash.
|
|
This feature is called \[dq]speedup\[dq] or \[dq]put by hash\[dq]. It is especially efficient
|
|
in case of generally available files like popular books, video or audio clips
|
|
[snip]
|
|
Enter a boolean value (true or false). Press Enter for the default (\[dq]true\[dq]).
|
|
Choose a number from below, or type in your own value
|
|
1 / Enable
|
|
\[rs] \[dq]true\[dq]
|
|
2 / Disable
|
|
\[rs] \[dq]false\[dq]
|
|
speedup_enable> 1
|
|
Edit advanced config? (y/n)
|
|
y) Yes
|
|
n) No
|
|
y/n> n
|
|
Remote config
|
|
\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-
|
|
[remote]
|
|
type = mailru
|
|
user = username\[at]mail.ru
|
|
pass = *** ENCRYPTED ***
|
|
speedup_enable = true
|
|
\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-
|
|
y) Yes this is OK
|
|
e) Edit this remote
|
|
d) Delete this remote
|
|
y/e/d> y
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
Configuration of this backend does not require a local web browser.
|
|
You can use the configured backend as shown below:
|
|
.PP
|
|
See top level directories
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone lsd remote:
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
Make a new directory
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone mkdir remote:directory
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
List the contents of a directory
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone ls remote:directory
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
Sync \f[C]/home/local/directory\f[R] to the remote path, deleting any
|
|
excess files in the path.
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone sync /home/local/directory remote:directory
|
|
\f[R]
|
|
.fi
|
|
.SS Modified time
|
|
.PP
|
|
Files support a modification time attribute with up to 1 second
|
|
precision.
|
|
Directories do not have a modification time, which is shown as \[lq]Jan
|
|
1 1970\[rq].
|
|
.SS Hash checksums
|
|
.PP
|
|
Hash sums use a custom Mail.ru algorithm based on SHA1.
|
|
If file size is less than or equal to the SHA1 block size (20 bytes),
|
|
its hash is simply its data right\-padded with zero bytes.
|
|
Hash sum of a larger file is computed as a SHA1 sum of the file data
|
|
bytes concatenated with a decimal representation of the data length.
|
|
.SS Emptying Trash
|
|
.PP
|
|
Removing a file or directory actually moves it to the trash, which is
|
|
not visible to rclone but can be seen in a web browser.
|
|
The trashed file still occupies part of total quota.
|
|
If you wish to empty your trash and free some quota, you can use the
|
|
\f[C]rclone cleanup remote:\f[R] command, which will permanently delete
|
|
all your trashed files.
|
|
This command does not take any path arguments.
|
|
.SS Quota information
|
|
.PP
|
|
To view your current quota you can use the
|
|
\f[C]rclone about remote:\f[R] command which will display your usage
|
|
limit (quota) and the current usage.
|
|
.SS Restricted filename characters
|
|
.PP
|
|
In addition to the default restricted characters
|
|
set (https://rclone.org/overview/#restricted-characters) the following
|
|
characters are also replaced:
|
|
.PP
|
|
.TS
|
|
tab(@);
|
|
l c c.
|
|
T{
|
|
Character
|
|
T}@T{
|
|
Value
|
|
T}@T{
|
|
Replacement
|
|
T}
|
|
_
|
|
T{
|
|
\[dq]
|
|
T}@T{
|
|
0x22
|
|
T}@T{
|
|
\[uFF02]
|
|
T}
|
|
T{
|
|
*
|
|
T}@T{
|
|
0x2A
|
|
T}@T{
|
|
\[uFF0A]
|
|
T}
|
|
T{
|
|
:
|
|
T}@T{
|
|
0x3A
|
|
T}@T{
|
|
\[uFF1A]
|
|
T}
|
|
T{
|
|
<
|
|
T}@T{
|
|
0x3C
|
|
T}@T{
|
|
\[uFF1C]
|
|
T}
|
|
T{
|
|
>
|
|
T}@T{
|
|
0x3E
|
|
T}@T{
|
|
\[uFF1E]
|
|
T}
|
|
T{
|
|
?
|
|
T}@T{
|
|
0x3F
|
|
T}@T{
|
|
\[uFF1F]
|
|
T}
|
|
T{
|
|
\[rs]
|
|
T}@T{
|
|
0x5C
|
|
T}@T{
|
|
\[uFF3C]
|
|
T}
|
|
T{
|
|
|
|
|
T}@T{
|
|
0x7C
|
|
T}@T{
|
|
\[uFF5C]
|
|
T}
|
|
.TE
|
|
.PP
|
|
Invalid UTF\-8 bytes will also be
|
|
replaced (https://rclone.org/overview/#invalid-utf8), as they can\[cq]t
|
|
be used in JSON strings.
|
|
.SS Limitations
|
|
.PP
|
|
File size limits depend on your account.
|
|
A single file size is limited by 2G for a free account and unlimited for
|
|
paid tariffs.
|
|
Please refer to the Mail.ru site for the total uploaded size limits.
|
|
.PP
|
|
Note that Mailru is case insensitive so you can\[cq]t have a file called
|
|
\[lq]Hello.doc\[rq] and one called \[lq]hello.doc\[rq].
|
|
.SS Standard Options
|
|
.PP
|
|
Here are the standard options specific to mailru (Mail.ru Cloud).
|
|
.SS \[en]mailru\-user
|
|
.PP
|
|
User name (usually email)
|
|
.IP \[bu] 2
|
|
Config: user
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_MAILRU_USER
|
|
.IP \[bu] 2
|
|
Type: string
|
|
.IP \[bu] 2
|
|
Default: \[dq]\[dq]
|
|
.SS \[en]mailru\-pass
|
|
.PP
|
|
Password
|
|
.IP \[bu] 2
|
|
Config: pass
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_MAILRU_PASS
|
|
.IP \[bu] 2
|
|
Type: string
|
|
.IP \[bu] 2
|
|
Default: \[dq]\[dq]
|
|
.SS \[en]mailru\-speedup\-enable
|
|
.PP
|
|
Skip full upload if there is another file with same data hash.
|
|
This feature is called \[lq]speedup\[rq] or \[lq]put by hash\[rq].
|
|
It is especially efficient in case of generally available files like
|
|
popular books, video or audio clips, because files are searched by hash
|
|
in all accounts of all mailru users.
|
|
Please note that rclone may need local memory and disk space to
|
|
calculate content hash in advance and decide whether full upload is
|
|
required.
|
|
Also, if rclone does not know file size in advance (e.g.\ in case of
|
|
streaming or partial uploads), it will not even try this optimization.
|
|
.IP \[bu] 2
|
|
Config: speedup_enable
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_MAILRU_SPEEDUP_ENABLE
|
|
.IP \[bu] 2
|
|
Type: bool
|
|
.IP \[bu] 2
|
|
Default: true
|
|
.IP \[bu] 2
|
|
Examples:
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
\[lq]true\[rq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Enable
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[lq]false\[rq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Disable
|
|
.RE
|
|
.RE
|
|
.SS Advanced Options
|
|
.PP
|
|
Here are the advanced options specific to mailru (Mail.ru Cloud).
|
|
.SS \[en]mailru\-speedup\-file\-patterns
|
|
.PP
|
|
Comma separated list of file name patterns eligible for speedup (put by
|
|
hash).
|
|
Patterns are case insensitive and can contain \[cq]*\[cq] or `?' meta
|
|
characters.
|
|
.IP \[bu] 2
|
|
Config: speedup_file_patterns
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_MAILRU_SPEEDUP_FILE_PATTERNS
|
|
.IP \[bu] 2
|
|
Type: string
|
|
.IP \[bu] 2
|
|
Default:
|
|
\[lq]\f[I].mkv,\f[R].avi,\f[I].mp4,\f[R].mp3,\f[I].zip,\f[R].gz,\f[I].rar,\f[R].pdf\[rq]
|
|
.IP \[bu] 2
|
|
Examples:
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
\[dq]\[dq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Empty list completely disables speedup (put by hash).
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[dq]*\[dq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
All files will be attempted for speedup.
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[lq]\f[I].mkv,\f[R].avi,\f[I].mp4,\f[R].mp3\[rq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Only common audio/video files will be tried for put by hash.
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[lq]\f[I].zip,\f[R].gz,\f[I].rar,\f[R].pdf\[rq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Only common archives or PDF books will be tried for speedup.
|
|
.RE
|
|
.RE
|
|
.SS \[en]mailru\-speedup\-max\-disk
|
|
.PP
|
|
This option allows you to disable speedup (put by hash) for large files
|
|
(because preliminary hashing can exhaust you RAM or disk space)
|
|
.IP \[bu] 2
|
|
Config: speedup_max_disk
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_MAILRU_SPEEDUP_MAX_DISK
|
|
.IP \[bu] 2
|
|
Type: SizeSuffix
|
|
.IP \[bu] 2
|
|
Default: 3G
|
|
.IP \[bu] 2
|
|
Examples:
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
\[lq]0\[rq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Completely disable speedup (put by hash).
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[lq]1G\[rq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Files larger than 1Gb will be uploaded directly.
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[lq]3G\[rq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Choose this option if you have less than 3Gb free on local disk.
|
|
.RE
|
|
.RE
|
|
.SS \[en]mailru\-speedup\-max\-memory
|
|
.PP
|
|
Files larger than the size given below will always be hashed on disk.
|
|
.IP \[bu] 2
|
|
Config: speedup_max_memory
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_MAILRU_SPEEDUP_MAX_MEMORY
|
|
.IP \[bu] 2
|
|
Type: SizeSuffix
|
|
.IP \[bu] 2
|
|
Default: 32M
|
|
.IP \[bu] 2
|
|
Examples:
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
\[lq]0\[rq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Preliminary hashing will always be done in a temporary disk location.
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[lq]32M\[rq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Do not dedicate more than 32Mb RAM for preliminary hashing.
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[lq]256M\[rq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
You have at most 256Mb RAM free for hash calculations.
|
|
.RE
|
|
.RE
|
|
.SS \[en]mailru\-check\-hash
|
|
.PP
|
|
What should copy do if file checksum is mismatched or invalid
|
|
.IP \[bu] 2
|
|
Config: check_hash
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_MAILRU_CHECK_HASH
|
|
.IP \[bu] 2
|
|
Type: bool
|
|
.IP \[bu] 2
|
|
Default: true
|
|
.IP \[bu] 2
|
|
Examples:
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
\[lq]true\[rq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Fail with error.
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[lq]false\[rq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Ignore and continue.
|
|
.RE
|
|
.RE
|
|
.SS \[en]mailru\-user\-agent
|
|
.PP
|
|
HTTP user agent used internally by client.
|
|
Defaults to \[lq]rclone/VERSION\[rq] or \[lq]\[en]user\-agent\[rq]
|
|
provided on command line.
|
|
.IP \[bu] 2
|
|
Config: user_agent
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_MAILRU_USER_AGENT
|
|
.IP \[bu] 2
|
|
Type: string
|
|
.IP \[bu] 2
|
|
Default: \[dq]\[dq]
|
|
.SS \[en]mailru\-quirks
|
|
.PP
|
|
Comma separated list of internal maintenance flags.
|
|
This option must not be used by an ordinary user.
|
|
It is intended only to facilitate remote troubleshooting of backend
|
|
issues.
|
|
Strict meaning of flags is not documented and not guaranteed to persist
|
|
between releases.
|
|
Quirks will be removed when the backend grows stable.
|
|
Supported quirks: atomicmkdir binlist gzip insecure retry400
|
|
.IP \[bu] 2
|
|
Config: quirks
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_MAILRU_QUIRKS
|
|
.IP \[bu] 2
|
|
Type: string
|
|
.IP \[bu] 2
|
|
Default: \[dq]\[dq]
|
|
.SS \[en]mailru\-encoding
|
|
.PP
|
|
This sets the encoding for the backend.
|
|
.PP
|
|
See: the encoding section in the
|
|
overview (https://rclone.org/overview/#encoding) for more info.
|
|
.IP \[bu] 2
|
|
Config: encoding
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_MAILRU_ENCODING
|
|
.IP \[bu] 2
|
|
Type: MultiEncoder
|
|
.IP \[bu] 2
|
|
Default:
|
|
Slash,LtGt,DoubleQuote,Colon,Question,Asterisk,Pipe,BackSlash,Del,Ctl,InvalidUtf8,Dot
|
|
.SS Mega
|
|
.PP
|
|
Mega (https://mega.nz/) is a cloud storage and file hosting service
|
|
known for its security feature where all files are encrypted locally
|
|
before they are uploaded.
|
|
This prevents anyone (including employees of Mega) from accessing the
|
|
files without knowledge of the key used for encryption.
|
|
.PP
|
|
This is an rclone backend for Mega which supports the file transfer
|
|
features of Mega using the same client side encryption.
|
|
.PP
|
|
Paths are specified as \f[C]remote:path\f[R]
|
|
.PP
|
|
Paths may be as deep as required, eg
|
|
\f[C]remote:directory/subdirectory\f[R].
|
|
.PP
|
|
Here is an example of how to make a remote called \f[C]remote\f[R].
|
|
First run:
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone config
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
This will guide you through an interactive setup process:
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
No remotes found \- make a new one
|
|
n) New remote
|
|
s) Set configuration password
|
|
q) Quit config
|
|
n/s/q> n
|
|
name> remote
|
|
Type of storage to configure.
|
|
Choose a number from below, or type in your own value
|
|
[snip]
|
|
XX / Mega
|
|
\[rs] \[dq]mega\[dq]
|
|
[snip]
|
|
Storage> mega
|
|
User name
|
|
user> you\[at]example.com
|
|
Password.
|
|
y) Yes type in my own password
|
|
g) Generate random password
|
|
n) No leave this optional password blank
|
|
y/g/n> y
|
|
Enter the password:
|
|
password:
|
|
Confirm the password:
|
|
password:
|
|
Remote config
|
|
\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-
|
|
[remote]
|
|
type = mega
|
|
user = you\[at]example.com
|
|
pass = *** ENCRYPTED ***
|
|
\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-
|
|
y) Yes this is OK
|
|
e) Edit this remote
|
|
d) Delete this remote
|
|
y/e/d> y
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
\f[B]NOTE:\f[R] The encryption keys need to have been already generated
|
|
after a regular login via the browser, otherwise attempting to use the
|
|
credentials in \f[C]rclone\f[R] will fail.
|
|
.PP
|
|
Once configured you can then use \f[C]rclone\f[R] like this,
|
|
.PP
|
|
List directories in top level of your Mega
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone lsd remote:
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
List all the files in your Mega
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone ls remote:
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
To copy a local directory to an Mega directory called backup
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone copy /home/source remote:backup
|
|
\f[R]
|
|
.fi
|
|
.SS Modified time and hashes
|
|
.PP
|
|
Mega does not support modification times or hashes yet.
|
|
.SS Restricted filename characters
|
|
.PP
|
|
.TS
|
|
tab(@);
|
|
l c c.
|
|
T{
|
|
Character
|
|
T}@T{
|
|
Value
|
|
T}@T{
|
|
Replacement
|
|
T}
|
|
_
|
|
T{
|
|
NUL
|
|
T}@T{
|
|
0x00
|
|
T}@T{
|
|
\[u2400]
|
|
T}
|
|
T{
|
|
/
|
|
T}@T{
|
|
0x2F
|
|
T}@T{
|
|
\[uFF0F]
|
|
T}
|
|
.TE
|
|
.PP
|
|
Invalid UTF\-8 bytes will also be
|
|
replaced (https://rclone.org/overview/#invalid-utf8), as they can\[cq]t
|
|
be used in JSON strings.
|
|
.SS Duplicated files
|
|
.PP
|
|
Mega can have two files with exactly the same name and path (unlike a
|
|
normal file system).
|
|
.PP
|
|
Duplicated files cause problems with the syncing and you will see
|
|
messages in the log about duplicates.
|
|
.PP
|
|
Use \f[C]rclone dedupe\f[R] to fix duplicated files.
|
|
.SS Failure to log\-in
|
|
.PP
|
|
Mega remotes seem to get blocked (reject logins) under \[lq]heavy
|
|
use\[rq].
|
|
We haven\[cq]t worked out the exact blocking rules but it seems to be
|
|
related to fast paced, successive rclone commands.
|
|
.PP
|
|
For example, executing this command 90 times in a row
|
|
\f[C]rclone link remote:file\f[R] will cause the remote to become
|
|
\[lq]blocked\[rq].
|
|
This is not an abnormal situation, for example if you wish to get the
|
|
public links of a directory with hundred of files\&... After more or
|
|
less a week, the remote will remote accept rclone logins normally again.
|
|
.PP
|
|
You can mitigate this issue by mounting the remote it with
|
|
\f[C]rclone mount\f[R].
|
|
This will log\-in when mounting and a log\-out when unmounting only.
|
|
You can also run \f[C]rclone rcd\f[R] and then use \f[C]rclone rc\f[R]
|
|
to run the commands over the API to avoid logging in each time.
|
|
.PP
|
|
Rclone does not currently close mega sessions (you can see them in the
|
|
web interface), however closing the sessions does not solve the issue.
|
|
.PP
|
|
If you space rclone commands by 3 seconds it will avoid blocking the
|
|
remote.
|
|
We haven\[cq]t identified the exact blocking rules, so perhaps one could
|
|
execute the command 80 times without waiting and avoid blocking by
|
|
waiting 3 seconds, then continuing\&...
|
|
.PP
|
|
Note that this has been observed by trial and error and might not be set
|
|
in stone.
|
|
.PP
|
|
Other tools seem not to produce this blocking effect, as they use a
|
|
different working approach (state\-based, using sessionIDs instead of
|
|
log\-in) which isn\[cq]t compatible with the current stateless rclone
|
|
approach.
|
|
.PP
|
|
Note that once blocked, the use of other tools (such as megacmd) is not
|
|
a sure workaround: following megacmd login times have been observed in
|
|
succession for blocked remote: 7 minutes, 20 min, 30min, 30 min, 30min.
|
|
Web access looks unaffected though.
|
|
.PP
|
|
Investigation is continuing in relation to workarounds based on
|
|
timeouts, pacers, retrials and tpslimits \- if you discover something
|
|
relevant, please post on the forum.
|
|
.PP
|
|
So, if rclone was working nicely and suddenly you are unable to log\-in
|
|
and you are sure the user and the password are correct, likely you have
|
|
got the remote blocked for a while.
|
|
.SS Standard Options
|
|
.PP
|
|
Here are the standard options specific to mega (Mega).
|
|
.SS \[en]mega\-user
|
|
.PP
|
|
User name
|
|
.IP \[bu] 2
|
|
Config: user
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_MEGA_USER
|
|
.IP \[bu] 2
|
|
Type: string
|
|
.IP \[bu] 2
|
|
Default: \[dq]\[dq]
|
|
.SS \[en]mega\-pass
|
|
.PP
|
|
Password.
|
|
.IP \[bu] 2
|
|
Config: pass
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_MEGA_PASS
|
|
.IP \[bu] 2
|
|
Type: string
|
|
.IP \[bu] 2
|
|
Default: \[dq]\[dq]
|
|
.SS Advanced Options
|
|
.PP
|
|
Here are the advanced options specific to mega (Mega).
|
|
.SS \[en]mega\-debug
|
|
.PP
|
|
Output more debug from Mega.
|
|
.PP
|
|
If this flag is set (along with \-vv) it will print further debugging
|
|
information from the mega backend.
|
|
.IP \[bu] 2
|
|
Config: debug
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_MEGA_DEBUG
|
|
.IP \[bu] 2
|
|
Type: bool
|
|
.IP \[bu] 2
|
|
Default: false
|
|
.SS \[en]mega\-hard\-delete
|
|
.PP
|
|
Delete files permanently rather than putting them into the trash.
|
|
.PP
|
|
Normally the mega backend will put all deletions into the trash rather
|
|
than permanently deleting them.
|
|
If you specify this then rclone will permanently delete objects instead.
|
|
.IP \[bu] 2
|
|
Config: hard_delete
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_MEGA_HARD_DELETE
|
|
.IP \[bu] 2
|
|
Type: bool
|
|
.IP \[bu] 2
|
|
Default: false
|
|
.SS \[en]mega\-encoding
|
|
.PP
|
|
This sets the encoding for the backend.
|
|
.PP
|
|
See: the encoding section in the
|
|
overview (https://rclone.org/overview/#encoding) for more info.
|
|
.IP \[bu] 2
|
|
Config: encoding
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_MEGA_ENCODING
|
|
.IP \[bu] 2
|
|
Type: MultiEncoder
|
|
.IP \[bu] 2
|
|
Default: Slash,InvalidUtf8,Dot
|
|
.SS Limitations
|
|
.PP
|
|
This backend uses the go\-mega go
|
|
library (https://github.com/t3rm1n4l/go-mega) which is an opensource go
|
|
library implementing the Mega API.
|
|
There doesn\[cq]t appear to be any documentation for the mega protocol
|
|
beyond the mega C++ SDK (https://github.com/meganz/sdk) source code so
|
|
there are likely quite a few errors still remaining in this library.
|
|
.PP
|
|
Mega allows duplicate files which may confuse rclone.
|
|
.SS Memory
|
|
.PP
|
|
The memory backend is an in RAM backend.
|
|
It does not persist its data \- use the local backend for that.
|
|
.PP
|
|
The memory backend behaves like a bucket based remote (eg like s3).
|
|
Because it has no parameters you can just use it with the
|
|
\f[C]:memory:\f[R] remote name.
|
|
.PP
|
|
You can configure it as a remote like this with \f[C]rclone config\f[R]
|
|
too if you want to:
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
No remotes found \- make a new one
|
|
n) New remote
|
|
s) Set configuration password
|
|
q) Quit config
|
|
n/s/q> n
|
|
name> remote
|
|
Type of storage to configure.
|
|
Enter a string value. Press Enter for the default (\[dq]\[dq]).
|
|
Choose a number from below, or type in your own value
|
|
[snip]
|
|
XX / Memory
|
|
\[rs] \[dq]memory\[dq]
|
|
[snip]
|
|
Storage> memory
|
|
** See help for memory backend at: https://rclone.org/memory/ **
|
|
|
|
Remote config
|
|
|
|
\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-
|
|
[remote]
|
|
type = memory
|
|
\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-
|
|
y) Yes this is OK (default)
|
|
e) Edit this remote
|
|
d) Delete this remote
|
|
y/e/d> y
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
Because the memory backend isn\[cq]t persistent it is most useful for
|
|
testing or with an rclone server or rclone mount, eg
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone mount :memory: /mnt/tmp
|
|
rclone serve webdav :memory:
|
|
rclone serve sftp :memory:
|
|
\f[R]
|
|
.fi
|
|
.SS Modified time and hashes
|
|
.PP
|
|
The memory backend supports MD5 hashes and modification times accurate
|
|
to 1 nS.
|
|
.SS Restricted filename characters
|
|
.PP
|
|
The memory backend replaces the default restricted characters
|
|
set (https://rclone.org/overview/#restricted-characters).
|
|
.SS Microsoft Azure Blob Storage
|
|
.PP
|
|
Paths are specified as \f[C]remote:container\f[R] (or \f[C]remote:\f[R]
|
|
for the \f[C]lsd\f[R] command.) You may put subdirectories in too, eg
|
|
\f[C]remote:container/path/to/dir\f[R].
|
|
.PP
|
|
Here is an example of making a Microsoft Azure Blob Storage
|
|
configuration.
|
|
For a remote called \f[C]remote\f[R].
|
|
First run:
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone config
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
This will guide you through an interactive setup process:
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
No remotes found \- make a new one
|
|
n) New remote
|
|
s) Set configuration password
|
|
q) Quit config
|
|
n/s/q> n
|
|
name> remote
|
|
Type of storage to configure.
|
|
Choose a number from below, or type in your own value
|
|
[snip]
|
|
XX / Microsoft Azure Blob Storage
|
|
\[rs] \[dq]azureblob\[dq]
|
|
[snip]
|
|
Storage> azureblob
|
|
Storage Account Name
|
|
account> account_name
|
|
Storage Account Key
|
|
key> base64encodedkey==
|
|
Endpoint for the service \- leave blank normally.
|
|
endpoint>
|
|
Remote config
|
|
\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-
|
|
[remote]
|
|
account = account_name
|
|
key = base64encodedkey==
|
|
endpoint =
|
|
\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-
|
|
y) Yes this is OK
|
|
e) Edit this remote
|
|
d) Delete this remote
|
|
y/e/d> y
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
See all containers
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone lsd remote:
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
Make a new container
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone mkdir remote:container
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
List the contents of a container
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone ls remote:container
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
Sync \f[C]/home/local/directory\f[R] to the remote container, deleting
|
|
any excess files in the container.
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone sync /home/local/directory remote:container
|
|
\f[R]
|
|
.fi
|
|
.SS \[en]fast\-list
|
|
.PP
|
|
This remote supports \f[C]\-\-fast\-list\f[R] which allows you to use
|
|
fewer transactions in exchange for more memory.
|
|
See the rclone docs (https://rclone.org/docs/#fast-list) for more
|
|
details.
|
|
.SS Modified time
|
|
.PP
|
|
The modified time is stored as metadata on the object with the
|
|
\f[C]mtime\f[R] key.
|
|
It is stored using RFC3339 Format time with nanosecond precision.
|
|
The metadata is supplied during directory listings so there is no
|
|
overhead to using it.
|
|
.SS Restricted filename characters
|
|
.PP
|
|
In addition to the default restricted characters
|
|
set (https://rclone.org/overview/#restricted-characters) the following
|
|
characters are also replaced:
|
|
.PP
|
|
.TS
|
|
tab(@);
|
|
l c c.
|
|
T{
|
|
Character
|
|
T}@T{
|
|
Value
|
|
T}@T{
|
|
Replacement
|
|
T}
|
|
_
|
|
T{
|
|
/
|
|
T}@T{
|
|
0x2F
|
|
T}@T{
|
|
\[uFF0F]
|
|
T}
|
|
T{
|
|
\[rs]
|
|
T}@T{
|
|
0x5C
|
|
T}@T{
|
|
\[uFF3C]
|
|
T}
|
|
.TE
|
|
.PP
|
|
File names can also not end with the following characters.
|
|
These only get replaced if they are the last character in the name:
|
|
.PP
|
|
.TS
|
|
tab(@);
|
|
l c c.
|
|
T{
|
|
Character
|
|
T}@T{
|
|
Value
|
|
T}@T{
|
|
Replacement
|
|
T}
|
|
_
|
|
T{
|
|
\&.
|
|
T}@T{
|
|
0x2E
|
|
T}@T{
|
|
\[uFF0E]
|
|
T}
|
|
.TE
|
|
.PP
|
|
Invalid UTF\-8 bytes will also be
|
|
replaced (https://rclone.org/overview/#invalid-utf8), as they can\[cq]t
|
|
be used in JSON strings.
|
|
.SS Hashes
|
|
.PP
|
|
MD5 hashes are stored with blobs.
|
|
However blobs that were uploaded in chunks only have an MD5 if the
|
|
source remote was capable of MD5 hashes, eg the local disk.
|
|
.SS Authenticating with Azure Blob Storage
|
|
.PP
|
|
Rclone has 3 ways of authenticating with Azure Blob Storage:
|
|
.SS Account and Key
|
|
.PP
|
|
This is the most straight forward and least flexible way.
|
|
Just fill in the \f[C]account\f[R] and \f[C]key\f[R] lines and leave the
|
|
rest blank.
|
|
.SS SAS URL
|
|
.PP
|
|
This can be an account level SAS URL or container level SAS URL.
|
|
.PP
|
|
To use it leave \f[C]account\f[R], \f[C]key\f[R] blank and fill in
|
|
\f[C]sas_url\f[R].
|
|
.PP
|
|
An account level SAS URL or container level SAS URL can be obtained from
|
|
the Azure portal or the Azure Storage Explorer.
|
|
To get a container level SAS URL right click on a container in the Azure
|
|
Blob explorer in the Azure portal.
|
|
.PP
|
|
If you use a container level SAS URL, rclone operations are permitted
|
|
only on a particular container, eg
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone ls azureblob:container
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
You can also list the single container from the root.
|
|
This will only show the container specified by the SAS URL.
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
$ rclone lsd azureblob:
|
|
container/
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
Note that you can\[cq]t see or access any other containers \- this will
|
|
fail
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone ls azureblob:othercontainer
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
Container level SAS URLs are useful for temporarily allowing third
|
|
parties access to a single container or putting credentials into an
|
|
untrusted environment such as a CI build server.
|
|
.SS Multipart uploads
|
|
.PP
|
|
Rclone supports multipart uploads with Azure Blob storage.
|
|
Files bigger than 256MB will be uploaded using chunked upload by
|
|
default.
|
|
.PP
|
|
The files will be uploaded in parallel in 4MB chunks (by default).
|
|
Note that these chunks are buffered in memory and there may be up to
|
|
\f[C]\-\-transfers\f[R] of them being uploaded at once.
|
|
.PP
|
|
Files can\[cq]t be split into more than 50,000 chunks so by default, so
|
|
the largest file that can be uploaded with 4MB chunk size is 195GB.
|
|
Above this rclone will double the chunk size until it creates less than
|
|
50,000 chunks.
|
|
By default this will mean a maximum file size of 3.2TB can be uploaded.
|
|
This can be raised to 5TB using
|
|
\f[C]\-\-azureblob\-chunk\-size 100M\f[R].
|
|
.PP
|
|
Note that rclone doesn\[cq]t commit the block list until the end of the
|
|
upload which means that there is a limit of 9.5TB of multipart uploads
|
|
in progress as Azure won\[cq]t allow more than that amount of
|
|
uncommitted blocks.
|
|
.SS Standard Options
|
|
.PP
|
|
Here are the standard options specific to azureblob (Microsoft Azure
|
|
Blob Storage).
|
|
.SS \[en]azureblob\-account
|
|
.PP
|
|
Storage Account Name (leave blank to use SAS URL or Emulator)
|
|
.IP \[bu] 2
|
|
Config: account
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_AZUREBLOB_ACCOUNT
|
|
.IP \[bu] 2
|
|
Type: string
|
|
.IP \[bu] 2
|
|
Default: \[dq]\[dq]
|
|
.SS \[en]azureblob\-key
|
|
.PP
|
|
Storage Account Key (leave blank to use SAS URL or Emulator)
|
|
.IP \[bu] 2
|
|
Config: key
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_AZUREBLOB_KEY
|
|
.IP \[bu] 2
|
|
Type: string
|
|
.IP \[bu] 2
|
|
Default: \[dq]\[dq]
|
|
.SS \[en]azureblob\-sas\-url
|
|
.PP
|
|
SAS URL for container level access only (leave blank if using
|
|
account/key or Emulator)
|
|
.IP \[bu] 2
|
|
Config: sas_url
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_AZUREBLOB_SAS_URL
|
|
.IP \[bu] 2
|
|
Type: string
|
|
.IP \[bu] 2
|
|
Default: \[dq]\[dq]
|
|
.SS \[en]azureblob\-use\-emulator
|
|
.PP
|
|
Uses local storage emulator if provided as `true' (leave blank if using
|
|
real azure storage endpoint)
|
|
.IP \[bu] 2
|
|
Config: use_emulator
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_AZUREBLOB_USE_EMULATOR
|
|
.IP \[bu] 2
|
|
Type: bool
|
|
.IP \[bu] 2
|
|
Default: false
|
|
.SS Advanced Options
|
|
.PP
|
|
Here are the advanced options specific to azureblob (Microsoft Azure
|
|
Blob Storage).
|
|
.SS \[en]azureblob\-endpoint
|
|
.PP
|
|
Endpoint for the service Leave blank normally.
|
|
.IP \[bu] 2
|
|
Config: endpoint
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_AZUREBLOB_ENDPOINT
|
|
.IP \[bu] 2
|
|
Type: string
|
|
.IP \[bu] 2
|
|
Default: \[dq]\[dq]
|
|
.SS \[en]azureblob\-upload\-cutoff
|
|
.PP
|
|
Cutoff for switching to chunked upload (<= 256MB).
|
|
.IP \[bu] 2
|
|
Config: upload_cutoff
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_AZUREBLOB_UPLOAD_CUTOFF
|
|
.IP \[bu] 2
|
|
Type: SizeSuffix
|
|
.IP \[bu] 2
|
|
Default: 256M
|
|
.SS \[en]azureblob\-chunk\-size
|
|
.PP
|
|
Upload chunk size (<= 100MB).
|
|
.PP
|
|
Note that this is stored in memory and there may be up to
|
|
\[lq]\[en]transfers\[rq] chunks stored at once in memory.
|
|
.IP \[bu] 2
|
|
Config: chunk_size
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_AZUREBLOB_CHUNK_SIZE
|
|
.IP \[bu] 2
|
|
Type: SizeSuffix
|
|
.IP \[bu] 2
|
|
Default: 4M
|
|
.SS \[en]azureblob\-list\-chunk
|
|
.PP
|
|
Size of blob list.
|
|
.PP
|
|
This sets the number of blobs requested in each listing chunk.
|
|
Default is the maximum, 5000.
|
|
\[lq]List blobs\[rq] requests are permitted 2 minutes per megabyte to
|
|
complete.
|
|
If an operation is taking longer than 2 minutes per megabyte on average,
|
|
it will time out (
|
|
source (https://docs.microsoft.com/en-us/rest/api/storageservices/setting-timeouts-for-blob-service-operations#exceptions-to-default-timeout-interval)
|
|
).
|
|
This can be used to limit the number of blobs items to return, to avoid
|
|
the time out.
|
|
.IP \[bu] 2
|
|
Config: list_chunk
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_AZUREBLOB_LIST_CHUNK
|
|
.IP \[bu] 2
|
|
Type: int
|
|
.IP \[bu] 2
|
|
Default: 5000
|
|
.SS \[en]azureblob\-access\-tier
|
|
.PP
|
|
Access tier of blob: hot, cool or archive.
|
|
.PP
|
|
Archived blobs can be restored by setting access tier to hot or cool.
|
|
Leave blank if you intend to use default access tier, which is set at
|
|
account level
|
|
.PP
|
|
If there is no \[lq]access tier\[rq] specified, rclone doesn\[cq]t apply
|
|
any tier.
|
|
rclone performs \[lq]Set Tier\[rq] operation on blobs while uploading,
|
|
if objects are not modified, specifying \[lq]access tier\[rq] to new one
|
|
will have no effect.
|
|
If blobs are in \[lq]archive tier\[rq] at remote, trying to perform data
|
|
transfer operations from remote will not be allowed.
|
|
User should first restore by tiering blob to \[lq]Hot\[rq] or
|
|
\[lq]Cool\[rq].
|
|
.IP \[bu] 2
|
|
Config: access_tier
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_AZUREBLOB_ACCESS_TIER
|
|
.IP \[bu] 2
|
|
Type: string
|
|
.IP \[bu] 2
|
|
Default: \[dq]\[dq]
|
|
.SS \[en]azureblob\-disable\-checksum
|
|
.PP
|
|
Don\[cq]t store MD5 checksum with object metadata.
|
|
.PP
|
|
Normally rclone will calculate the MD5 checksum of the input before
|
|
uploading it so it can add it to metadata on the object.
|
|
This is great for data integrity checking but can cause long delays for
|
|
large files to start uploading.
|
|
.IP \[bu] 2
|
|
Config: disable_checksum
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_AZUREBLOB_DISABLE_CHECKSUM
|
|
.IP \[bu] 2
|
|
Type: bool
|
|
.IP \[bu] 2
|
|
Default: false
|
|
.SS \[en]azureblob\-memory\-pool\-flush\-time
|
|
.PP
|
|
How often internal memory buffer pools will be flushed.
|
|
Uploads which requires additional buffers (f.e multipart) will use
|
|
memory pool for allocations.
|
|
This option controls how often unused buffers will be removed from the
|
|
pool.
|
|
.IP \[bu] 2
|
|
Config: memory_pool_flush_time
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_AZUREBLOB_MEMORY_POOL_FLUSH_TIME
|
|
.IP \[bu] 2
|
|
Type: Duration
|
|
.IP \[bu] 2
|
|
Default: 1m0s
|
|
.SS \[en]azureblob\-memory\-pool\-use\-mmap
|
|
.PP
|
|
Whether to use mmap buffers in internal memory pool.
|
|
.IP \[bu] 2
|
|
Config: memory_pool_use_mmap
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_AZUREBLOB_MEMORY_POOL_USE_MMAP
|
|
.IP \[bu] 2
|
|
Type: bool
|
|
.IP \[bu] 2
|
|
Default: false
|
|
.SS \[en]azureblob\-encoding
|
|
.PP
|
|
This sets the encoding for the backend.
|
|
.PP
|
|
See: the encoding section in the
|
|
overview (https://rclone.org/overview/#encoding) for more info.
|
|
.IP \[bu] 2
|
|
Config: encoding
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_AZUREBLOB_ENCODING
|
|
.IP \[bu] 2
|
|
Type: MultiEncoder
|
|
.IP \[bu] 2
|
|
Default: Slash,BackSlash,Del,Ctl,RightPeriod,InvalidUtf8
|
|
.SS Limitations
|
|
.PP
|
|
MD5 sums are only uploaded with chunked files if the source has an MD5
|
|
sum.
|
|
This will always be the case for a local to azure copy.
|
|
.SS Azure Storage Emulator Support
|
|
.PP
|
|
You can test rclone with storage emulator locally, to do this make sure
|
|
azure storage emulator installed locally and set up a new remote with
|
|
\f[C]rclone config\f[R] follow instructions described in introduction,
|
|
set \f[C]use_emulator\f[R] config as \f[C]true\f[R], you do not need to
|
|
provide default account name or key if using emulator.
|
|
.SS Microsoft OneDrive
|
|
.PP
|
|
Paths are specified as \f[C]remote:path\f[R]
|
|
.PP
|
|
Paths may be as deep as required, eg
|
|
\f[C]remote:directory/subdirectory\f[R].
|
|
.PP
|
|
The initial setup for OneDrive involves getting a token from Microsoft
|
|
which you need to do in your browser.
|
|
\f[C]rclone config\f[R] walks you through it.
|
|
.PP
|
|
Here is an example of how to make a remote called \f[C]remote\f[R].
|
|
First run:
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone config
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
This will guide you through an interactive setup process:
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
e) Edit existing remote
|
|
n) New remote
|
|
d) Delete remote
|
|
r) Rename remote
|
|
c) Copy remote
|
|
s) Set configuration password
|
|
q) Quit config
|
|
e/n/d/r/c/s/q> n
|
|
name> remote
|
|
Type of storage to configure.
|
|
Enter a string value. Press Enter for the default (\[dq]\[dq]).
|
|
Choose a number from below, or type in your own value
|
|
[snip]
|
|
XX / Microsoft OneDrive
|
|
\[rs] \[dq]onedrive\[dq]
|
|
[snip]
|
|
Storage> onedrive
|
|
Microsoft App Client Id
|
|
Leave blank normally.
|
|
Enter a string value. Press Enter for the default (\[dq]\[dq]).
|
|
client_id>
|
|
Microsoft App Client Secret
|
|
Leave blank normally.
|
|
Enter a string value. Press Enter for the default (\[dq]\[dq]).
|
|
client_secret>
|
|
Edit advanced config? (y/n)
|
|
y) Yes
|
|
n) No
|
|
y/n> n
|
|
Remote config
|
|
Use auto config?
|
|
* Say Y if not sure
|
|
* Say N if you are working on a remote or headless machine
|
|
y) Yes
|
|
n) No
|
|
y/n> y
|
|
If your browser doesn\[aq]t open automatically go to the following link: http://127.0.0.1:53682/auth
|
|
Log in and authorize rclone for access
|
|
Waiting for code...
|
|
Got code
|
|
Choose a number from below, or type in an existing value
|
|
1 / OneDrive Personal or Business
|
|
\[rs] \[dq]onedrive\[dq]
|
|
2 / Sharepoint site
|
|
\[rs] \[dq]sharepoint\[dq]
|
|
3 / Type in driveID
|
|
\[rs] \[dq]driveid\[dq]
|
|
4 / Type in SiteID
|
|
\[rs] \[dq]siteid\[dq]
|
|
5 / Search a Sharepoint site
|
|
\[rs] \[dq]search\[dq]
|
|
Your choice> 1
|
|
Found 1 drives, please select the one you want to use:
|
|
0: OneDrive (business) id=b!Eqwertyuiopasdfghjklzxcvbnm\-7mnbvcxzlkjhgfdsapoiuytrewqk
|
|
Chose drive to use:> 0
|
|
Found drive \[aq]root\[aq] of type \[aq]business\[aq], URL: https://org\-my.sharepoint.com/personal/you/Documents
|
|
Is that okay?
|
|
y) Yes
|
|
n) No
|
|
y/n> y
|
|
\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-
|
|
[remote]
|
|
type = onedrive
|
|
token = {\[dq]access_token\[dq]:\[dq]youraccesstoken\[dq],\[dq]token_type\[dq]:\[dq]Bearer\[dq],\[dq]refresh_token\[dq]:\[dq]yourrefreshtoken\[dq],\[dq]expiry\[dq]:\[dq]2018\-08\-26T22:39:52.486512262+08:00\[dq]}
|
|
drive_id = b!Eqwertyuiopasdfghjklzxcvbnm\-7mnbvcxzlkjhgfdsapoiuytrewqk
|
|
drive_type = business
|
|
\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-
|
|
y) Yes this is OK
|
|
e) Edit this remote
|
|
d) Delete this remote
|
|
y/e/d> y
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
See the remote setup docs (https://rclone.org/remote_setup/) for how to
|
|
set it up on a machine with no Internet browser available.
|
|
.PP
|
|
Note that rclone runs a webserver on your local machine to collect the
|
|
token as returned from Microsoft.
|
|
This only runs from the moment it opens your browser to the moment you
|
|
get back the verification code.
|
|
This is on \f[C]http://127.0.0.1:53682/\f[R] and this it may require you
|
|
to unblock it temporarily if you are running a host firewall.
|
|
.PP
|
|
Once configured you can then use \f[C]rclone\f[R] like this,
|
|
.PP
|
|
List directories in top level of your OneDrive
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone lsd remote:
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
List all the files in your OneDrive
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone ls remote:
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
To copy a local directory to an OneDrive directory called backup
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone copy /home/source remote:backup
|
|
\f[R]
|
|
.fi
|
|
.SS Getting your own Client ID and Key
|
|
.PP
|
|
You can use your own Client ID if the default (\f[C]client_id\f[R] left
|
|
blank) one doesn\[cq]t work for you or you see lots of throttling.
|
|
The default Client ID and Key is shared by all rclone users when
|
|
performing requests.
|
|
.PP
|
|
If you are having problems with them (E.g., seeing a lot of throttling),
|
|
you can get your own Client ID and Key by following the steps below:
|
|
.IP "1." 3
|
|
Open
|
|
https://portal.azure.com/#blade/Microsoft_AAD_RegisteredApps/ApplicationsListBlade,
|
|
then click \f[C]New registration\f[R].
|
|
.IP "2." 3
|
|
Enter a name for your app, choose account type
|
|
\f[C]Accounts in any organizational directory (Any Azure AD directory \- Multitenant) and personal Microsoft accounts (e.g. Skype, Xbox)\f[R],
|
|
select \f[C]Web\f[R] in \f[C]Redirect URI\f[R] Enter
|
|
\f[C]http://localhost:53682/\f[R] and click Register.
|
|
Copy and keep the \f[C]Application (client) ID\f[R] under the app name
|
|
for later use.
|
|
.IP "3." 3
|
|
Under \f[C]manage\f[R] select \f[C]Certificates & secrets\f[R], click
|
|
\f[C]New client secret\f[R].
|
|
Copy and keep that secret for later use.
|
|
.IP "4." 3
|
|
Under \f[C]manage\f[R] select \f[C]API permissions\f[R], click
|
|
\f[C]Add a permission\f[R] and select \f[C]Microsoft Graph\f[R] then
|
|
select \f[C]delegated permissions\f[R].
|
|
.IP "5." 3
|
|
Search and select the following permissions: \f[C]Files.Read\f[R],
|
|
\f[C]Files.ReadWrite\f[R], \f[C]Files.Read.All\f[R],
|
|
\f[C]Files.ReadWrite.All\f[R], \f[C]offline_access\f[R],
|
|
\f[C]User.Read\f[R].
|
|
Once selected click \f[C]Add permissions\f[R] at the bottom.
|
|
.PP
|
|
Now the application is complete.
|
|
Run \f[C]rclone config\f[R] to create or edit a OneDrive remote.
|
|
Supply the app ID and password as Client ID and Secret, respectively.
|
|
rclone will walk you through the remaining steps.
|
|
.SS Modification time and hashes
|
|
.PP
|
|
OneDrive allows modification times to be set on objects accurate to 1
|
|
second.
|
|
These will be used to detect whether objects need syncing or not.
|
|
.PP
|
|
OneDrive personal supports SHA1 type hashes.
|
|
OneDrive for business and Sharepoint Server support
|
|
QuickXorHash (https://docs.microsoft.com/en-us/onedrive/developer/code-snippets/quickxorhash).
|
|
.PP
|
|
For all types of OneDrive you can use the \f[C]\-\-checksum\f[R] flag.
|
|
.SS Restricted filename characters
|
|
.PP
|
|
In addition to the default restricted characters
|
|
set (https://rclone.org/overview/#restricted-characters) the following
|
|
characters are also replaced:
|
|
.PP
|
|
.TS
|
|
tab(@);
|
|
l c c.
|
|
T{
|
|
Character
|
|
T}@T{
|
|
Value
|
|
T}@T{
|
|
Replacement
|
|
T}
|
|
_
|
|
T{
|
|
\[dq]
|
|
T}@T{
|
|
0x22
|
|
T}@T{
|
|
\[uFF02]
|
|
T}
|
|
T{
|
|
*
|
|
T}@T{
|
|
0x2A
|
|
T}@T{
|
|
\[uFF0A]
|
|
T}
|
|
T{
|
|
:
|
|
T}@T{
|
|
0x3A
|
|
T}@T{
|
|
\[uFF1A]
|
|
T}
|
|
T{
|
|
<
|
|
T}@T{
|
|
0x3C
|
|
T}@T{
|
|
\[uFF1C]
|
|
T}
|
|
T{
|
|
>
|
|
T}@T{
|
|
0x3E
|
|
T}@T{
|
|
\[uFF1E]
|
|
T}
|
|
T{
|
|
?
|
|
T}@T{
|
|
0x3F
|
|
T}@T{
|
|
\[uFF1F]
|
|
T}
|
|
T{
|
|
\[rs]
|
|
T}@T{
|
|
0x5C
|
|
T}@T{
|
|
\[uFF3C]
|
|
T}
|
|
T{
|
|
|
|
|
T}@T{
|
|
0x7C
|
|
T}@T{
|
|
\[uFF5C]
|
|
T}
|
|
T{
|
|
#
|
|
T}@T{
|
|
0x23
|
|
T}@T{
|
|
\[uFF03]
|
|
T}
|
|
T{
|
|
%
|
|
T}@T{
|
|
0x25
|
|
T}@T{
|
|
\[uFF05]
|
|
T}
|
|
.TE
|
|
.PP
|
|
File names can also not end with the following characters.
|
|
These only get replaced if they are the last character in the name:
|
|
.PP
|
|
.TS
|
|
tab(@);
|
|
l c c.
|
|
T{
|
|
Character
|
|
T}@T{
|
|
Value
|
|
T}@T{
|
|
Replacement
|
|
T}
|
|
_
|
|
T{
|
|
SP
|
|
T}@T{
|
|
0x20
|
|
T}@T{
|
|
\[u2420]
|
|
T}
|
|
T{
|
|
\&.
|
|
T}@T{
|
|
0x2E
|
|
T}@T{
|
|
\[uFF0E]
|
|
T}
|
|
.TE
|
|
.PP
|
|
File names can also not begin with the following characters.
|
|
These only get replaced if they are the first character in the name:
|
|
.PP
|
|
.TS
|
|
tab(@);
|
|
l c c.
|
|
T{
|
|
Character
|
|
T}@T{
|
|
Value
|
|
T}@T{
|
|
Replacement
|
|
T}
|
|
_
|
|
T{
|
|
SP
|
|
T}@T{
|
|
0x20
|
|
T}@T{
|
|
\[u2420]
|
|
T}
|
|
T{
|
|
\[ti]
|
|
T}@T{
|
|
0x7E
|
|
T}@T{
|
|
\[uFF5E]
|
|
T}
|
|
.TE
|
|
.PP
|
|
Invalid UTF\-8 bytes will also be
|
|
replaced (https://rclone.org/overview/#invalid-utf8), as they can\[cq]t
|
|
be used in JSON strings.
|
|
.SS Deleting files
|
|
.PP
|
|
Any files you delete with rclone will end up in the trash.
|
|
Microsoft doesn\[cq]t provide an API to permanently delete files, nor to
|
|
empty the trash, so you will have to do that with one of Microsoft\[cq]s
|
|
apps or via the OneDrive website.
|
|
.SS Standard Options
|
|
.PP
|
|
Here are the standard options specific to onedrive (Microsoft OneDrive).
|
|
.SS \[en]onedrive\-client\-id
|
|
.PP
|
|
Microsoft App Client Id Leave blank normally.
|
|
.IP \[bu] 2
|
|
Config: client_id
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_ONEDRIVE_CLIENT_ID
|
|
.IP \[bu] 2
|
|
Type: string
|
|
.IP \[bu] 2
|
|
Default: \[dq]\[dq]
|
|
.SS \[en]onedrive\-client\-secret
|
|
.PP
|
|
Microsoft App Client Secret Leave blank normally.
|
|
.IP \[bu] 2
|
|
Config: client_secret
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_ONEDRIVE_CLIENT_SECRET
|
|
.IP \[bu] 2
|
|
Type: string
|
|
.IP \[bu] 2
|
|
Default: \[dq]\[dq]
|
|
.SS Advanced Options
|
|
.PP
|
|
Here are the advanced options specific to onedrive (Microsoft OneDrive).
|
|
.SS \[en]onedrive\-chunk\-size
|
|
.PP
|
|
Chunk size to upload files with \- must be multiple of 320k (327,680
|
|
bytes).
|
|
.PP
|
|
Above this size files will be chunked \- must be multiple of 320k
|
|
(327,680 bytes) and should not exceed 250M (262,144,000 bytes) else you
|
|
may encounter
|
|
\[dq]Microsoft.SharePoint.Client.InvalidClientQueryException: The
|
|
request message is too big.\[dq] Note that the chunks will be buffered
|
|
into memory.
|
|
.IP \[bu] 2
|
|
Config: chunk_size
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_ONEDRIVE_CHUNK_SIZE
|
|
.IP \[bu] 2
|
|
Type: SizeSuffix
|
|
.IP \[bu] 2
|
|
Default: 10M
|
|
.SS \[en]onedrive\-drive\-id
|
|
.PP
|
|
The ID of the drive to use
|
|
.IP \[bu] 2
|
|
Config: drive_id
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_ONEDRIVE_DRIVE_ID
|
|
.IP \[bu] 2
|
|
Type: string
|
|
.IP \[bu] 2
|
|
Default: \[dq]\[dq]
|
|
.SS \[en]onedrive\-drive\-type
|
|
.PP
|
|
The type of the drive ( personal | business | documentLibrary )
|
|
.IP \[bu] 2
|
|
Config: drive_type
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_ONEDRIVE_DRIVE_TYPE
|
|
.IP \[bu] 2
|
|
Type: string
|
|
.IP \[bu] 2
|
|
Default: \[dq]\[dq]
|
|
.SS \[en]onedrive\-expose\-onenote\-files
|
|
.PP
|
|
Set to make OneNote files show up in directory listings.
|
|
.PP
|
|
By default rclone will hide OneNote files in directory listings because
|
|
operations like \[lq]Open\[rq] and \[lq]Update\[rq] won\[cq]t work on
|
|
them.
|
|
But this behaviour may also prevent you from deleting them.
|
|
If you want to delete OneNote files or otherwise want them to show up in
|
|
directory listing, set this option.
|
|
.IP \[bu] 2
|
|
Config: expose_onenote_files
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_ONEDRIVE_EXPOSE_ONENOTE_FILES
|
|
.IP \[bu] 2
|
|
Type: bool
|
|
.IP \[bu] 2
|
|
Default: false
|
|
.SS \[en]onedrive\-server\-side\-across\-configs
|
|
.PP
|
|
Allow server side operations (eg copy) to work across different onedrive
|
|
configs.
|
|
.PP
|
|
This can be useful if you wish to do a server side copy between two
|
|
different Onedrives.
|
|
Note that this isn\[cq]t enabled by default because it isn\[cq]t easy to
|
|
tell if it will work between any two configurations.
|
|
.IP \[bu] 2
|
|
Config: server_side_across_configs
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_ONEDRIVE_SERVER_SIDE_ACROSS_CONFIGS
|
|
.IP \[bu] 2
|
|
Type: bool
|
|
.IP \[bu] 2
|
|
Default: false
|
|
.SS \[en]onedrive\-encoding
|
|
.PP
|
|
This sets the encoding for the backend.
|
|
.PP
|
|
See: the encoding section in the
|
|
overview (https://rclone.org/overview/#encoding) for more info.
|
|
.IP \[bu] 2
|
|
Config: encoding
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_ONEDRIVE_ENCODING
|
|
.IP \[bu] 2
|
|
Type: MultiEncoder
|
|
.IP \[bu] 2
|
|
Default:
|
|
Slash,LtGt,DoubleQuote,Colon,Question,Asterisk,Pipe,Hash,Percent,BackSlash,Del,Ctl,LeftSpace,LeftTilde,RightSpace,RightPeriod,InvalidUtf8,Dot
|
|
.SS Limitations
|
|
.SS Naming
|
|
.PP
|
|
Note that OneDrive is case insensitive so you can\[cq]t have a file
|
|
called \[lq]Hello.doc\[rq] and one called \[lq]hello.doc\[rq].
|
|
.PP
|
|
There are quite a few characters that can\[cq]t be in OneDrive file
|
|
names.
|
|
These can\[cq]t occur on Windows platforms, but on non\-Windows
|
|
platforms they are common.
|
|
Rclone will map these names to and from an identical looking unicode
|
|
equivalent.
|
|
For example if a file has a \f[C]?\f[R] in it will be mapped to
|
|
\f[C]\[uFF1F]\f[R] instead.
|
|
.SS File sizes
|
|
.PP
|
|
The largest allowed file sizes are 15GB for OneDrive for Business and
|
|
100GB for OneDrive Personal (Updated 19 May 2020).
|
|
Source:
|
|
https://support.office.com/en\-us/article/upload\-photos\-and\-files\-to\-onedrive\-b00ad3fe\-6643\-4b16\-9212\-de00ef02b586
|
|
.SS Path length
|
|
.PP
|
|
The entire path, including the file name, must contain fewer than 400
|
|
characters for OneDrive, OneDrive for Business and SharePoint Online.
|
|
If you are encrypting file and folder names with rclone, you may want to
|
|
pay attention to this limitation because the encrypted names are
|
|
typically longer than the original ones.
|
|
.SS Number of files
|
|
.PP
|
|
OneDrive seems to be OK with at least 50,000 files in a folder, but at
|
|
100,000 rclone will get errors listing the directory like
|
|
\f[C]couldn\[cq]t list files: UnknownError:\f[R].
|
|
See #2707 (https://github.com/rclone/rclone/issues/2707) for more info.
|
|
.PP
|
|
An official document about the limitations for different types of
|
|
OneDrive can be found
|
|
here (https://support.office.com/en-us/article/invalid-file-names-and-file-types-in-onedrive-onedrive-for-business-and-sharepoint-64883a5d-228e-48f5-b3d2-eb39e07630fa).
|
|
.SS Versioning issue
|
|
.PP
|
|
Every change in OneDrive causes the service to create a new version.
|
|
This counts against a users quota.
|
|
For example changing the modification time of a file creates a second
|
|
version, so the file is using twice the space.
|
|
.PP
|
|
The \f[C]copy\f[R] is the only rclone command affected by this as we
|
|
copy the file and then afterwards set the modification time to match the
|
|
source file.
|
|
.PP
|
|
\f[B]Note\f[R]: Starting October 2018, users will no longer be able to
|
|
disable versioning by default.
|
|
This is because Microsoft has brought an
|
|
update (https://techcommunity.microsoft.com/t5/Microsoft-OneDrive-Blog/New-Updates-to-OneDrive-and-SharePoint-Team-Site-Versioning/ba-p/204390)
|
|
to the mechanism.
|
|
To change this new default setting, a PowerShell command is required to
|
|
be run by a SharePoint admin.
|
|
If you are an admin, you can run these commands in PowerShell to change
|
|
that setting:
|
|
.IP "1." 3
|
|
\f[C]Install\-Module \-Name Microsoft.Online.SharePoint.PowerShell\f[R]
|
|
(in case you haven\[cq]t installed this already)
|
|
.IP "2." 3
|
|
\f[C]Import\-Module Microsoft.Online.SharePoint.PowerShell \-DisableNameChecking\f[R]
|
|
.IP "3." 3
|
|
\f[C]Connect\-SPOService \-Url https://YOURSITE\-admin.sharepoint.com \-Credential YOU\[at]YOURSITE.COM\f[R]
|
|
(replacing \f[C]YOURSITE\f[R], \f[C]YOU\f[R], \f[C]YOURSITE.COM\f[R]
|
|
with the actual values; this will prompt for your credentials)
|
|
.IP "4." 3
|
|
\f[C]Set\-SPOTenant \-EnableMinimumVersionRequirement $False\f[R]
|
|
.IP "5." 3
|
|
\f[C]Disconnect\-SPOService\f[R] (to disconnect from the server)
|
|
.PP
|
|
\f[I]Below are the steps for normal users to disable versioning. If you
|
|
don\[cq]t see the \[lq]No Versioning\[rq] option, make sure the above
|
|
requirements are met.\f[R]
|
|
.PP
|
|
User Weropol (https://github.com/Weropol) has found a method to disable
|
|
versioning on OneDrive
|
|
.IP "1." 3
|
|
Open the settings menu by clicking on the gear symbol at the top of the
|
|
OneDrive Business page.
|
|
.IP "2." 3
|
|
Click Site settings.
|
|
.IP "3." 3
|
|
Once on the Site settings page, navigate to Site Administration > Site
|
|
libraries and lists.
|
|
.IP "4." 3
|
|
Click Customize \[lq]Documents\[rq].
|
|
.IP "5." 3
|
|
Click General Settings > Versioning Settings.
|
|
.IP "6." 3
|
|
Under Document Version History select the option No versioning.
|
|
Note: This will disable the creation of new file versions, but will not
|
|
remove any previous versions.
|
|
Your documents are safe.
|
|
.IP "7." 3
|
|
Apply the changes by clicking OK.
|
|
.IP "8." 3
|
|
Use rclone to upload or modify files.
|
|
(I also use the \[en]no\-update\-modtime flag)
|
|
.IP "9." 3
|
|
Restore the versioning settings after using rclone.
|
|
(Optional)
|
|
.SS Troubleshooting
|
|
.SS Unexpected file size/hash differences on Sharepoint
|
|
.PP
|
|
It is a
|
|
known (https://github.com/OneDrive/onedrive-api-docs/issues/935#issuecomment-441741631)
|
|
issue that Sharepoint (not OneDrive or OneDrive for Business) silently
|
|
modifies uploaded files, mainly Office files (.docx, .xlsx, etc.),
|
|
causing file size and hash checks to fail.
|
|
To use rclone with such affected files on Sharepoint, you may disable
|
|
these checks with the following command line arguments:
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
\-\-ignore\-checksum \-\-ignore\-size
|
|
\f[R]
|
|
.fi
|
|
.SS Replacing/deleting existing files on Sharepoint gets \[lq]item not found\[rq]
|
|
.PP
|
|
It is a
|
|
known (https://github.com/OneDrive/onedrive-api-docs/issues/1068) issue
|
|
that Sharepoint (not OneDrive or OneDrive for Business) may return
|
|
\[lq]item not found\[rq] errors when users try to replace or delete
|
|
uploaded files; this seems to mainly affect Office files (.docx, .xlsx,
|
|
etc.).
|
|
As a workaround, you may use the \f[C]\-\-backup\-dir <BACKUP_DIR>\f[R]
|
|
command line argument so rclone moves the files to be replaced/deleted
|
|
into a given backup directory (instead of directly replacing/deleting
|
|
them).
|
|
For example, to instruct rclone to move the files into the directory
|
|
\f[C]rclone\-backup\-dir\f[R] on backend \f[C]mysharepoint\f[R], you may
|
|
use:
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
\-\-backup\-dir mysharepoint:rclone\-backup\-dir
|
|
\f[R]
|
|
.fi
|
|
.SS access_denied (AADSTS65005)
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
Error: access_denied
|
|
Code: AADSTS65005
|
|
Description: Using application \[aq]rclone\[aq] is currently not supported for your organization [YOUR_ORGANIZATION] because it is in an unmanaged state. An administrator needs to claim ownership of the company by DNS validation of [YOUR_ORGANIZATION] before the application rclone can be provisioned.
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
This means that rclone can\[cq]t use the OneDrive for Business API with
|
|
your account.
|
|
You can\[cq]t do much about it, maybe write an email to your admins.
|
|
.PP
|
|
However, there are other ways to interact with your OneDrive account.
|
|
Have a look at the webdav backend: https://rclone.org/webdav/#sharepoint
|
|
.SS invalid_grant (AADSTS50076)
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
Error: invalid_grant
|
|
Code: AADSTS50076
|
|
Description: Due to a configuration change made by your administrator, or because you moved to a new location, you must use multi\-factor authentication to access \[aq]...\[aq].
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
If you see the error above after enabling multi\-factor authentication
|
|
for your account, you can fix it by refreshing your OAuth refresh token.
|
|
To do that, run \f[C]rclone config\f[R], and choose to edit your
|
|
OneDrive backend.
|
|
Then, you don\[cq]t need to actually make any changes until you reach
|
|
this question: \f[C]Already have a token \- refresh?\f[R].
|
|
For this question, answer \f[C]y\f[R] and go through the process to
|
|
refresh your token, just like the first time the backend is configured.
|
|
After this, rclone should work again for this backend.
|
|
.SS OpenDrive
|
|
.PP
|
|
Paths are specified as \f[C]remote:path\f[R]
|
|
.PP
|
|
Paths may be as deep as required, eg
|
|
\f[C]remote:directory/subdirectory\f[R].
|
|
.PP
|
|
Here is an example of how to make a remote called \f[C]remote\f[R].
|
|
First run:
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone config
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
This will guide you through an interactive setup process:
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
n) New remote
|
|
d) Delete remote
|
|
q) Quit config
|
|
e/n/d/q> n
|
|
name> remote
|
|
Type of storage to configure.
|
|
Choose a number from below, or type in your own value
|
|
[snip]
|
|
XX / OpenDrive
|
|
\[rs] \[dq]opendrive\[dq]
|
|
[snip]
|
|
Storage> opendrive
|
|
Username
|
|
username>
|
|
Password
|
|
y) Yes type in my own password
|
|
g) Generate random password
|
|
y/g> y
|
|
Enter the password:
|
|
password:
|
|
Confirm the password:
|
|
password:
|
|
\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-
|
|
[remote]
|
|
username =
|
|
password = *** ENCRYPTED ***
|
|
\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-
|
|
y) Yes this is OK
|
|
e) Edit this remote
|
|
d) Delete this remote
|
|
y/e/d> y
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
List directories in top level of your OpenDrive
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone lsd remote:
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
List all the files in your OpenDrive
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone ls remote:
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
To copy a local directory to an OpenDrive directory called backup
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone copy /home/source remote:backup
|
|
\f[R]
|
|
.fi
|
|
.SS Modified time and MD5SUMs
|
|
.PP
|
|
OpenDrive allows modification times to be set on objects accurate to 1
|
|
second.
|
|
These will be used to detect whether objects need syncing or not.
|
|
.SS Restricted filename characters
|
|
.PP
|
|
.TS
|
|
tab(@);
|
|
l c c.
|
|
T{
|
|
Character
|
|
T}@T{
|
|
Value
|
|
T}@T{
|
|
Replacement
|
|
T}
|
|
_
|
|
T{
|
|
NUL
|
|
T}@T{
|
|
0x00
|
|
T}@T{
|
|
\[u2400]
|
|
T}
|
|
T{
|
|
/
|
|
T}@T{
|
|
0x2F
|
|
T}@T{
|
|
\[uFF0F]
|
|
T}
|
|
T{
|
|
\[dq]
|
|
T}@T{
|
|
0x22
|
|
T}@T{
|
|
\[uFF02]
|
|
T}
|
|
T{
|
|
*
|
|
T}@T{
|
|
0x2A
|
|
T}@T{
|
|
\[uFF0A]
|
|
T}
|
|
T{
|
|
:
|
|
T}@T{
|
|
0x3A
|
|
T}@T{
|
|
\[uFF1A]
|
|
T}
|
|
T{
|
|
<
|
|
T}@T{
|
|
0x3C
|
|
T}@T{
|
|
\[uFF1C]
|
|
T}
|
|
T{
|
|
>
|
|
T}@T{
|
|
0x3E
|
|
T}@T{
|
|
\[uFF1E]
|
|
T}
|
|
T{
|
|
?
|
|
T}@T{
|
|
0x3F
|
|
T}@T{
|
|
\[uFF1F]
|
|
T}
|
|
T{
|
|
\[rs]
|
|
T}@T{
|
|
0x5C
|
|
T}@T{
|
|
\[uFF3C]
|
|
T}
|
|
T{
|
|
|
|
|
T}@T{
|
|
0x7C
|
|
T}@T{
|
|
\[uFF5C]
|
|
T}
|
|
.TE
|
|
.PP
|
|
File names can also not begin or end with the following characters.
|
|
These only get replaced if they are the first or last character in the
|
|
name:
|
|
.PP
|
|
.TS
|
|
tab(@);
|
|
l c c.
|
|
T{
|
|
Character
|
|
T}@T{
|
|
Value
|
|
T}@T{
|
|
Replacement
|
|
T}
|
|
_
|
|
T{
|
|
SP
|
|
T}@T{
|
|
0x20
|
|
T}@T{
|
|
\[u2420]
|
|
T}
|
|
T{
|
|
HT
|
|
T}@T{
|
|
0x09
|
|
T}@T{
|
|
\[u2409]
|
|
T}
|
|
T{
|
|
LF
|
|
T}@T{
|
|
0x0A
|
|
T}@T{
|
|
\[u240A]
|
|
T}
|
|
T{
|
|
VT
|
|
T}@T{
|
|
0x0B
|
|
T}@T{
|
|
\[u240B]
|
|
T}
|
|
T{
|
|
CR
|
|
T}@T{
|
|
0x0D
|
|
T}@T{
|
|
\[u240D]
|
|
T}
|
|
.TE
|
|
.PP
|
|
Invalid UTF\-8 bytes will also be
|
|
replaced (https://rclone.org/overview/#invalid-utf8), as they can\[cq]t
|
|
be used in JSON strings.
|
|
.SS Standard Options
|
|
.PP
|
|
Here are the standard options specific to opendrive (OpenDrive).
|
|
.SS \[en]opendrive\-username
|
|
.PP
|
|
Username
|
|
.IP \[bu] 2
|
|
Config: username
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_OPENDRIVE_USERNAME
|
|
.IP \[bu] 2
|
|
Type: string
|
|
.IP \[bu] 2
|
|
Default: \[dq]\[dq]
|
|
.SS \[en]opendrive\-password
|
|
.PP
|
|
Password.
|
|
.IP \[bu] 2
|
|
Config: password
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_OPENDRIVE_PASSWORD
|
|
.IP \[bu] 2
|
|
Type: string
|
|
.IP \[bu] 2
|
|
Default: \[dq]\[dq]
|
|
.SS Advanced Options
|
|
.PP
|
|
Here are the advanced options specific to opendrive (OpenDrive).
|
|
.SS \[en]opendrive\-encoding
|
|
.PP
|
|
This sets the encoding for the backend.
|
|
.PP
|
|
See: the encoding section in the
|
|
overview (https://rclone.org/overview/#encoding) for more info.
|
|
.IP \[bu] 2
|
|
Config: encoding
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_OPENDRIVE_ENCODING
|
|
.IP \[bu] 2
|
|
Type: MultiEncoder
|
|
.IP \[bu] 2
|
|
Default:
|
|
Slash,LtGt,DoubleQuote,Colon,Question,Asterisk,Pipe,BackSlash,LeftSpace,LeftCrLfHtVt,RightSpace,RightCrLfHtVt,InvalidUtf8,Dot
|
|
.SS \[en]opendrive\-chunk\-size
|
|
.PP
|
|
Files will be uploaded in chunks this size.
|
|
.PP
|
|
Note that these chunks are buffered in memory so increasing them will
|
|
increase memory use.
|
|
.IP \[bu] 2
|
|
Config: chunk_size
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_OPENDRIVE_CHUNK_SIZE
|
|
.IP \[bu] 2
|
|
Type: SizeSuffix
|
|
.IP \[bu] 2
|
|
Default: 10M
|
|
.SS Limitations
|
|
.PP
|
|
Note that OpenDrive is case insensitive so you can\[cq]t have a file
|
|
called \[lq]Hello.doc\[rq] and one called \[lq]hello.doc\[rq].
|
|
.PP
|
|
There are quite a few characters that can\[cq]t be in OpenDrive file
|
|
names.
|
|
These can\[cq]t occur on Windows platforms, but on non\-Windows
|
|
platforms they are common.
|
|
Rclone will map these names to and from an identical looking unicode
|
|
equivalent.
|
|
For example if a file has a \f[C]?\f[R] in it will be mapped to
|
|
\f[C]\[uFF1F]\f[R] instead.
|
|
.SS QingStor
|
|
.PP
|
|
Paths are specified as \f[C]remote:bucket\f[R] (or \f[C]remote:\f[R] for
|
|
the \f[C]lsd\f[R] command.) You may put subdirectories in too, eg
|
|
\f[C]remote:bucket/path/to/dir\f[R].
|
|
.PP
|
|
Here is an example of making an QingStor configuration.
|
|
First run
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone config
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
This will guide you through an interactive setup process.
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
No remotes found \- make a new one
|
|
n) New remote
|
|
r) Rename remote
|
|
c) Copy remote
|
|
s) Set configuration password
|
|
q) Quit config
|
|
n/r/c/s/q> n
|
|
name> remote
|
|
Type of storage to configure.
|
|
Choose a number from below, or type in your own value
|
|
[snip]
|
|
XX / QingStor Object Storage
|
|
\[rs] \[dq]qingstor\[dq]
|
|
[snip]
|
|
Storage> qingstor
|
|
Get QingStor credentials from runtime. Only applies if access_key_id and secret_access_key is blank.
|
|
Choose a number from below, or type in your own value
|
|
1 / Enter QingStor credentials in the next step
|
|
\[rs] \[dq]false\[dq]
|
|
2 / Get QingStor credentials from the environment (env vars or IAM)
|
|
\[rs] \[dq]true\[dq]
|
|
env_auth> 1
|
|
QingStor Access Key ID \- leave blank for anonymous access or runtime credentials.
|
|
access_key_id> access_key
|
|
QingStor Secret Access Key (password) \- leave blank for anonymous access or runtime credentials.
|
|
secret_access_key> secret_key
|
|
Enter an endpoint URL to connection QingStor API.
|
|
Leave blank will use the default value \[dq]https://qingstor.com:443\[dq]
|
|
endpoint>
|
|
Zone connect to. Default is \[dq]pek3a\[dq].
|
|
Choose a number from below, or type in your own value
|
|
/ The Beijing (China) Three Zone
|
|
1 | Needs location constraint pek3a.
|
|
\[rs] \[dq]pek3a\[dq]
|
|
/ The Shanghai (China) First Zone
|
|
2 | Needs location constraint sh1a.
|
|
\[rs] \[dq]sh1a\[dq]
|
|
zone> 1
|
|
Number of connection retry.
|
|
Leave blank will use the default value \[dq]3\[dq].
|
|
connection_retries>
|
|
Remote config
|
|
\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-
|
|
[remote]
|
|
env_auth = false
|
|
access_key_id = access_key
|
|
secret_access_key = secret_key
|
|
endpoint =
|
|
zone = pek3a
|
|
connection_retries =
|
|
\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-
|
|
y) Yes this is OK
|
|
e) Edit this remote
|
|
d) Delete this remote
|
|
y/e/d> y
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
This remote is called \f[C]remote\f[R] and can now be used like this
|
|
.PP
|
|
See all buckets
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone lsd remote:
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
Make a new bucket
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone mkdir remote:bucket
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
List the contents of a bucket
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone ls remote:bucket
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
Sync \f[C]/home/local/directory\f[R] to the remote bucket, deleting any
|
|
excess files in the bucket.
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone sync /home/local/directory remote:bucket
|
|
\f[R]
|
|
.fi
|
|
.SS \[en]fast\-list
|
|
.PP
|
|
This remote supports \f[C]\-\-fast\-list\f[R] which allows you to use
|
|
fewer transactions in exchange for more memory.
|
|
See the rclone docs (https://rclone.org/docs/#fast-list) for more
|
|
details.
|
|
.SS Multipart uploads
|
|
.PP
|
|
rclone supports multipart uploads with QingStor which means that it can
|
|
upload files bigger than 5GB.
|
|
Note that files uploaded with multipart upload don\[cq]t have an MD5SUM.
|
|
.PP
|
|
Note that incomplete multipart uploads older than 24 hours can be
|
|
removed with \f[C]rclone cleanup remote:bucket\f[R] just for one bucket
|
|
\f[C]rclone cleanup remote:\f[R] for all buckets.
|
|
QingStor does not ever remove incomplete multipart uploads so it may be
|
|
necessary to run this from time to time.
|
|
.SS Buckets and Zone
|
|
.PP
|
|
With QingStor you can list buckets (\f[C]rclone lsd\f[R]) using any
|
|
zone, but you can only access the content of a bucket from the zone it
|
|
was created in.
|
|
If you attempt to access a bucket from the wrong zone, you will get an
|
|
error,
|
|
\f[C]incorrect zone, the bucket is not in \[aq]XXX\[aq] zone\f[R].
|
|
.SS Authentication
|
|
.PP
|
|
There are two ways to supply \f[C]rclone\f[R] with a set of QingStor
|
|
credentials.
|
|
In order of precedence:
|
|
.IP \[bu] 2
|
|
Directly in the rclone configuration file (as configured by
|
|
\f[C]rclone config\f[R])
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
set \f[C]access_key_id\f[R] and \f[C]secret_access_key\f[R]
|
|
.RE
|
|
.IP \[bu] 2
|
|
Runtime configuration:
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
set \f[C]env_auth\f[R] to \f[C]true\f[R] in the config file
|
|
.IP \[bu] 2
|
|
Exporting the following environment variables before running
|
|
\f[C]rclone\f[R]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Access Key ID: \f[C]QS_ACCESS_KEY_ID\f[R] or \f[C]QS_ACCESS_KEY\f[R]
|
|
.IP \[bu] 2
|
|
Secret Access Key: \f[C]QS_SECRET_ACCESS_KEY\f[R] or
|
|
\f[C]QS_SECRET_KEY\f[R]
|
|
.RE
|
|
.RE
|
|
.SS Restricted filename characters
|
|
.PP
|
|
The control characters 0x00\-0x1F and / are replaced as in the default
|
|
restricted characters
|
|
set (https://rclone.org/overview/#restricted-characters).
|
|
Note that 0x7F is not replaced.
|
|
.PP
|
|
Invalid UTF\-8 bytes will also be
|
|
replaced (https://rclone.org/overview/#invalid-utf8), as they can\[cq]t
|
|
be used in JSON strings.
|
|
.SS Standard Options
|
|
.PP
|
|
Here are the standard options specific to qingstor (QingCloud Object
|
|
Storage).
|
|
.SS \[en]qingstor\-env\-auth
|
|
.PP
|
|
Get QingStor credentials from runtime.
|
|
Only applies if access_key_id and secret_access_key is blank.
|
|
.IP \[bu] 2
|
|
Config: env_auth
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_QINGSTOR_ENV_AUTH
|
|
.IP \[bu] 2
|
|
Type: bool
|
|
.IP \[bu] 2
|
|
Default: false
|
|
.IP \[bu] 2
|
|
Examples:
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
\[lq]false\[rq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Enter QingStor credentials in the next step
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[lq]true\[rq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Get QingStor credentials from the environment (env vars or IAM)
|
|
.RE
|
|
.RE
|
|
.SS \[en]qingstor\-access\-key\-id
|
|
.PP
|
|
QingStor Access Key ID Leave blank for anonymous access or runtime
|
|
credentials.
|
|
.IP \[bu] 2
|
|
Config: access_key_id
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_QINGSTOR_ACCESS_KEY_ID
|
|
.IP \[bu] 2
|
|
Type: string
|
|
.IP \[bu] 2
|
|
Default: \[dq]\[dq]
|
|
.SS \[en]qingstor\-secret\-access\-key
|
|
.PP
|
|
QingStor Secret Access Key (password) Leave blank for anonymous access
|
|
or runtime credentials.
|
|
.IP \[bu] 2
|
|
Config: secret_access_key
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_QINGSTOR_SECRET_ACCESS_KEY
|
|
.IP \[bu] 2
|
|
Type: string
|
|
.IP \[bu] 2
|
|
Default: \[dq]\[dq]
|
|
.SS \[en]qingstor\-endpoint
|
|
.PP
|
|
Enter an endpoint URL to connection QingStor API.
|
|
Leave blank will use the default value
|
|
\[lq]https://qingstor.com:443\[rq]
|
|
.IP \[bu] 2
|
|
Config: endpoint
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_QINGSTOR_ENDPOINT
|
|
.IP \[bu] 2
|
|
Type: string
|
|
.IP \[bu] 2
|
|
Default: \[dq]\[dq]
|
|
.SS \[en]qingstor\-zone
|
|
.PP
|
|
Zone to connect to.
|
|
Default is \[lq]pek3a\[rq].
|
|
.IP \[bu] 2
|
|
Config: zone
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_QINGSTOR_ZONE
|
|
.IP \[bu] 2
|
|
Type: string
|
|
.IP \[bu] 2
|
|
Default: \[dq]\[dq]
|
|
.IP \[bu] 2
|
|
Examples:
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
\[lq]pek3a\[rq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
The Beijing (China) Three Zone
|
|
.IP \[bu] 2
|
|
Needs location constraint pek3a.
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[lq]sh1a\[rq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
The Shanghai (China) First Zone
|
|
.IP \[bu] 2
|
|
Needs location constraint sh1a.
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[lq]gd2a\[rq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
The Guangdong (China) Second Zone
|
|
.IP \[bu] 2
|
|
Needs location constraint gd2a.
|
|
.RE
|
|
.RE
|
|
.SS Advanced Options
|
|
.PP
|
|
Here are the advanced options specific to qingstor (QingCloud Object
|
|
Storage).
|
|
.SS \[en]qingstor\-connection\-retries
|
|
.PP
|
|
Number of connection retries.
|
|
.IP \[bu] 2
|
|
Config: connection_retries
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_QINGSTOR_CONNECTION_RETRIES
|
|
.IP \[bu] 2
|
|
Type: int
|
|
.IP \[bu] 2
|
|
Default: 3
|
|
.SS \[en]qingstor\-upload\-cutoff
|
|
.PP
|
|
Cutoff for switching to chunked upload
|
|
.PP
|
|
Any files larger than this will be uploaded in chunks of chunk_size.
|
|
The minimum is 0 and the maximum is 5GB.
|
|
.IP \[bu] 2
|
|
Config: upload_cutoff
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_QINGSTOR_UPLOAD_CUTOFF
|
|
.IP \[bu] 2
|
|
Type: SizeSuffix
|
|
.IP \[bu] 2
|
|
Default: 200M
|
|
.SS \[en]qingstor\-chunk\-size
|
|
.PP
|
|
Chunk size to use for uploading.
|
|
.PP
|
|
When uploading files larger than upload_cutoff they will be uploaded as
|
|
multipart uploads using this chunk size.
|
|
.PP
|
|
Note that \[lq]\[en]qingstor\-upload\-concurrency\[rq] chunks of this
|
|
size are buffered in memory per transfer.
|
|
.PP
|
|
If you are transferring large files over high speed links and you have
|
|
enough memory, then increasing this will speed up the transfers.
|
|
.IP \[bu] 2
|
|
Config: chunk_size
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_QINGSTOR_CHUNK_SIZE
|
|
.IP \[bu] 2
|
|
Type: SizeSuffix
|
|
.IP \[bu] 2
|
|
Default: 4M
|
|
.SS \[en]qingstor\-upload\-concurrency
|
|
.PP
|
|
Concurrency for multipart uploads.
|
|
.PP
|
|
This is the number of chunks of the same file that are uploaded
|
|
concurrently.
|
|
.PP
|
|
NB if you set this to > 1 then the checksums of multpart uploads become
|
|
corrupted (the uploads themselves are not corrupted though).
|
|
.PP
|
|
If you are uploading small numbers of large file over high speed link
|
|
and these uploads do not fully utilize your bandwidth, then increasing
|
|
this may help to speed up the transfers.
|
|
.IP \[bu] 2
|
|
Config: upload_concurrency
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_QINGSTOR_UPLOAD_CONCURRENCY
|
|
.IP \[bu] 2
|
|
Type: int
|
|
.IP \[bu] 2
|
|
Default: 1
|
|
.SS \[en]qingstor\-encoding
|
|
.PP
|
|
This sets the encoding for the backend.
|
|
.PP
|
|
See: the encoding section in the
|
|
overview (https://rclone.org/overview/#encoding) for more info.
|
|
.IP \[bu] 2
|
|
Config: encoding
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_QINGSTOR_ENCODING
|
|
.IP \[bu] 2
|
|
Type: MultiEncoder
|
|
.IP \[bu] 2
|
|
Default: Slash,Ctl,InvalidUtf8
|
|
.SS Swift
|
|
.PP
|
|
Swift refers to OpenStack Object
|
|
Storage (https://docs.openstack.org/swift/latest/).
|
|
Commercial implementations of that being:
|
|
.IP \[bu] 2
|
|
Rackspace Cloud Files (https://www.rackspace.com/cloud/files/)
|
|
.IP \[bu] 2
|
|
Memset Memstore (https://www.memset.com/cloud/storage/)
|
|
.IP \[bu] 2
|
|
OVH Object
|
|
Storage (https://www.ovh.co.uk/public-cloud/storage/object-storage/)
|
|
.IP \[bu] 2
|
|
Oracle Cloud Storage (https://cloud.oracle.com/storage-opc)
|
|
.IP \[bu] 2
|
|
IBM Bluemix Cloud ObjectStorage
|
|
Swift (https://console.bluemix.net/docs/infrastructure/objectstorage-swift/index.html)
|
|
.PP
|
|
Paths are specified as \f[C]remote:container\f[R] (or \f[C]remote:\f[R]
|
|
for the \f[C]lsd\f[R] command.) You may put subdirectories in too, eg
|
|
\f[C]remote:container/path/to/dir\f[R].
|
|
.PP
|
|
Here is an example of making a swift configuration.
|
|
First run
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone config
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
This will guide you through an interactive setup process.
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
No remotes found \- make a new one
|
|
n) New remote
|
|
s) Set configuration password
|
|
q) Quit config
|
|
n/s/q> n
|
|
name> remote
|
|
Type of storage to configure.
|
|
Choose a number from below, or type in your own value
|
|
[snip]
|
|
XX / OpenStack Swift (Rackspace Cloud Files, Memset Memstore, OVH)
|
|
\[rs] \[dq]swift\[dq]
|
|
[snip]
|
|
Storage> swift
|
|
Get swift credentials from environment variables in standard OpenStack form.
|
|
Choose a number from below, or type in your own value
|
|
1 / Enter swift credentials in the next step
|
|
\[rs] \[dq]false\[dq]
|
|
2 / Get swift credentials from environment vars. Leave other fields blank if using this.
|
|
\[rs] \[dq]true\[dq]
|
|
env_auth> true
|
|
User name to log in (OS_USERNAME).
|
|
user>
|
|
API key or password (OS_PASSWORD).
|
|
key>
|
|
Authentication URL for server (OS_AUTH_URL).
|
|
Choose a number from below, or type in your own value
|
|
1 / Rackspace US
|
|
\[rs] \[dq]https://auth.api.rackspacecloud.com/v1.0\[dq]
|
|
2 / Rackspace UK
|
|
\[rs] \[dq]https://lon.auth.api.rackspacecloud.com/v1.0\[dq]
|
|
3 / Rackspace v2
|
|
\[rs] \[dq]https://identity.api.rackspacecloud.com/v2.0\[dq]
|
|
4 / Memset Memstore UK
|
|
\[rs] \[dq]https://auth.storage.memset.com/v1.0\[dq]
|
|
5 / Memset Memstore UK v2
|
|
\[rs] \[dq]https://auth.storage.memset.com/v2.0\[dq]
|
|
6 / OVH
|
|
\[rs] \[dq]https://auth.cloud.ovh.net/v3\[dq]
|
|
auth>
|
|
User ID to log in \- optional \- most swift systems use user and leave this blank (v3 auth) (OS_USER_ID).
|
|
user_id>
|
|
User domain \- optional (v3 auth) (OS_USER_DOMAIN_NAME)
|
|
domain>
|
|
Tenant name \- optional for v1 auth, this or tenant_id required otherwise (OS_TENANT_NAME or OS_PROJECT_NAME)
|
|
tenant>
|
|
Tenant ID \- optional for v1 auth, this or tenant required otherwise (OS_TENANT_ID)
|
|
tenant_id>
|
|
Tenant domain \- optional (v3 auth) (OS_PROJECT_DOMAIN_NAME)
|
|
tenant_domain>
|
|
Region name \- optional (OS_REGION_NAME)
|
|
region>
|
|
Storage URL \- optional (OS_STORAGE_URL)
|
|
storage_url>
|
|
Auth Token from alternate authentication \- optional (OS_AUTH_TOKEN)
|
|
auth_token>
|
|
AuthVersion \- optional \- set to (1,2,3) if your auth URL has no version (ST_AUTH_VERSION)
|
|
auth_version>
|
|
Endpoint type to choose from the service catalogue (OS_ENDPOINT_TYPE)
|
|
Choose a number from below, or type in your own value
|
|
1 / Public (default, choose this if not sure)
|
|
\[rs] \[dq]public\[dq]
|
|
2 / Internal (use internal service net)
|
|
\[rs] \[dq]internal\[dq]
|
|
3 / Admin
|
|
\[rs] \[dq]admin\[dq]
|
|
endpoint_type>
|
|
Remote config
|
|
\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-
|
|
[test]
|
|
env_auth = true
|
|
user =
|
|
key =
|
|
auth =
|
|
user_id =
|
|
domain =
|
|
tenant =
|
|
tenant_id =
|
|
tenant_domain =
|
|
region =
|
|
storage_url =
|
|
auth_token =
|
|
auth_version =
|
|
endpoint_type =
|
|
\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-
|
|
y) Yes this is OK
|
|
e) Edit this remote
|
|
d) Delete this remote
|
|
y/e/d> y
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
This remote is called \f[C]remote\f[R] and can now be used like this
|
|
.PP
|
|
See all containers
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone lsd remote:
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
Make a new container
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone mkdir remote:container
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
List the contents of a container
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone ls remote:container
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
Sync \f[C]/home/local/directory\f[R] to the remote container, deleting
|
|
any excess files in the container.
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone sync /home/local/directory remote:container
|
|
\f[R]
|
|
.fi
|
|
.SS Configuration from an OpenStack credentials file
|
|
.PP
|
|
An OpenStack credentials file typically looks something something like
|
|
this (without the comments)
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
export OS_AUTH_URL=https://a.provider.net/v2.0
|
|
export OS_TENANT_ID=ffffffffffffffffffffffffffffffff
|
|
export OS_TENANT_NAME=\[dq]1234567890123456\[dq]
|
|
export OS_USERNAME=\[dq]123abc567xy\[dq]
|
|
echo \[dq]Please enter your OpenStack Password: \[dq]
|
|
read \-sr OS_PASSWORD_INPUT
|
|
export OS_PASSWORD=$OS_PASSWORD_INPUT
|
|
export OS_REGION_NAME=\[dq]SBG1\[dq]
|
|
if [ \-z \[dq]$OS_REGION_NAME\[dq] ]; then unset OS_REGION_NAME; fi
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
The config file needs to look something like this where
|
|
\f[C]$OS_USERNAME\f[R] represents the value of the \f[C]OS_USERNAME\f[R]
|
|
variable \- \f[C]123abc567xy\f[R] in the example above.
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
[remote]
|
|
type = swift
|
|
user = $OS_USERNAME
|
|
key = $OS_PASSWORD
|
|
auth = $OS_AUTH_URL
|
|
tenant = $OS_TENANT_NAME
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
Note that you may (or may not) need to set \f[C]region\f[R] too \- try
|
|
without first.
|
|
.SS Configuration from the environment
|
|
.PP
|
|
If you prefer you can configure rclone to use swift using a standard set
|
|
of OpenStack environment variables.
|
|
.PP
|
|
When you run through the config, make sure you choose \f[C]true\f[R] for
|
|
\f[C]env_auth\f[R] and leave everything else blank.
|
|
.PP
|
|
rclone will then set any empty config parameters from the environment
|
|
using standard OpenStack environment variables.
|
|
There is a list of the
|
|
variables (https://godoc.org/github.com/ncw/swift#Connection.ApplyEnvironment)
|
|
in the docs for the swift library.
|
|
.SS Using an alternate authentication method
|
|
.PP
|
|
If your OpenStack installation uses a non\-standard authentication
|
|
method that might not be yet supported by rclone or the underlying swift
|
|
library, you can authenticate externally (e.g.\ calling manually the
|
|
\f[C]openstack\f[R] commands to get a token).
|
|
Then, you just need to pass the two configuration variables
|
|
\f[C]auth_token\f[R] and \f[C]storage_url\f[R].
|
|
If they are both provided, the other variables are ignored.
|
|
rclone will not try to authenticate but instead assume it is already
|
|
authenticated and use these two variables to access the OpenStack
|
|
installation.
|
|
.SS Using rclone without a config file
|
|
.PP
|
|
You can use rclone with swift without a config file, if desired, like
|
|
this:
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
source openstack\-credentials\-file
|
|
export RCLONE_CONFIG_MYREMOTE_TYPE=swift
|
|
export RCLONE_CONFIG_MYREMOTE_ENV_AUTH=true
|
|
rclone lsd myremote:
|
|
\f[R]
|
|
.fi
|
|
.SS \[en]fast\-list
|
|
.PP
|
|
This remote supports \f[C]\-\-fast\-list\f[R] which allows you to use
|
|
fewer transactions in exchange for more memory.
|
|
See the rclone docs (https://rclone.org/docs/#fast-list) for more
|
|
details.
|
|
.SS \[en]update and \[en]use\-server\-modtime
|
|
.PP
|
|
As noted below, the modified time is stored on metadata on the object.
|
|
It is used by default for all operations that require checking the time
|
|
a file was last updated.
|
|
It allows rclone to treat the remote more like a true filesystem, but it
|
|
is inefficient because it requires an extra API call to retrieve the
|
|
metadata.
|
|
.PP
|
|
For many operations, the time the object was last uploaded to the remote
|
|
is sufficient to determine if it is \[lq]dirty\[rq].
|
|
By using \f[C]\-\-update\f[R] along with
|
|
\f[C]\-\-use\-server\-modtime\f[R], you can avoid the extra API call and
|
|
simply upload files whose local modtime is newer than the time it was
|
|
last uploaded.
|
|
.SS Standard Options
|
|
.PP
|
|
Here are the standard options specific to swift (OpenStack Swift
|
|
(Rackspace Cloud Files, Memset Memstore, OVH)).
|
|
.SS \[en]swift\-env\-auth
|
|
.PP
|
|
Get swift credentials from environment variables in standard OpenStack
|
|
form.
|
|
.IP \[bu] 2
|
|
Config: env_auth
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_SWIFT_ENV_AUTH
|
|
.IP \[bu] 2
|
|
Type: bool
|
|
.IP \[bu] 2
|
|
Default: false
|
|
.IP \[bu] 2
|
|
Examples:
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
\[lq]false\[rq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Enter swift credentials in the next step
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[lq]true\[rq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Get swift credentials from environment vars.
|
|
Leave other fields blank if using this.
|
|
.RE
|
|
.RE
|
|
.SS \[en]swift\-user
|
|
.PP
|
|
User name to log in (OS_USERNAME).
|
|
.IP \[bu] 2
|
|
Config: user
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_SWIFT_USER
|
|
.IP \[bu] 2
|
|
Type: string
|
|
.IP \[bu] 2
|
|
Default: \[dq]\[dq]
|
|
.SS \[en]swift\-key
|
|
.PP
|
|
API key or password (OS_PASSWORD).
|
|
.IP \[bu] 2
|
|
Config: key
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_SWIFT_KEY
|
|
.IP \[bu] 2
|
|
Type: string
|
|
.IP \[bu] 2
|
|
Default: \[dq]\[dq]
|
|
.SS \[en]swift\-auth
|
|
.PP
|
|
Authentication URL for server (OS_AUTH_URL).
|
|
.IP \[bu] 2
|
|
Config: auth
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_SWIFT_AUTH
|
|
.IP \[bu] 2
|
|
Type: string
|
|
.IP \[bu] 2
|
|
Default: \[dq]\[dq]
|
|
.IP \[bu] 2
|
|
Examples:
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
\[lq]https://auth.api.rackspacecloud.com/v1.0\[rq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Rackspace US
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[lq]https://lon.auth.api.rackspacecloud.com/v1.0\[rq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Rackspace UK
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[lq]https://identity.api.rackspacecloud.com/v2.0\[rq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Rackspace v2
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[lq]https://auth.storage.memset.com/v1.0\[rq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Memset Memstore UK
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[lq]https://auth.storage.memset.com/v2.0\[rq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Memset Memstore UK v2
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[lq]https://auth.cloud.ovh.net/v3\[rq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
OVH
|
|
.RE
|
|
.RE
|
|
.SS \[en]swift\-user\-id
|
|
.PP
|
|
User ID to log in \- optional \- most swift systems use user and leave
|
|
this blank (v3 auth) (OS_USER_ID).
|
|
.IP \[bu] 2
|
|
Config: user_id
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_SWIFT_USER_ID
|
|
.IP \[bu] 2
|
|
Type: string
|
|
.IP \[bu] 2
|
|
Default: \[dq]\[dq]
|
|
.SS \[en]swift\-domain
|
|
.PP
|
|
User domain \- optional (v3 auth) (OS_USER_DOMAIN_NAME)
|
|
.IP \[bu] 2
|
|
Config: domain
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_SWIFT_DOMAIN
|
|
.IP \[bu] 2
|
|
Type: string
|
|
.IP \[bu] 2
|
|
Default: \[dq]\[dq]
|
|
.SS \[en]swift\-tenant
|
|
.PP
|
|
Tenant name \- optional for v1 auth, this or tenant_id required
|
|
otherwise (OS_TENANT_NAME or OS_PROJECT_NAME)
|
|
.IP \[bu] 2
|
|
Config: tenant
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_SWIFT_TENANT
|
|
.IP \[bu] 2
|
|
Type: string
|
|
.IP \[bu] 2
|
|
Default: \[dq]\[dq]
|
|
.SS \[en]swift\-tenant\-id
|
|
.PP
|
|
Tenant ID \- optional for v1 auth, this or tenant required otherwise
|
|
(OS_TENANT_ID)
|
|
.IP \[bu] 2
|
|
Config: tenant_id
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_SWIFT_TENANT_ID
|
|
.IP \[bu] 2
|
|
Type: string
|
|
.IP \[bu] 2
|
|
Default: \[dq]\[dq]
|
|
.SS \[en]swift\-tenant\-domain
|
|
.PP
|
|
Tenant domain \- optional (v3 auth) (OS_PROJECT_DOMAIN_NAME)
|
|
.IP \[bu] 2
|
|
Config: tenant_domain
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_SWIFT_TENANT_DOMAIN
|
|
.IP \[bu] 2
|
|
Type: string
|
|
.IP \[bu] 2
|
|
Default: \[dq]\[dq]
|
|
.SS \[en]swift\-region
|
|
.PP
|
|
Region name \- optional (OS_REGION_NAME)
|
|
.IP \[bu] 2
|
|
Config: region
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_SWIFT_REGION
|
|
.IP \[bu] 2
|
|
Type: string
|
|
.IP \[bu] 2
|
|
Default: \[dq]\[dq]
|
|
.SS \[en]swift\-storage\-url
|
|
.PP
|
|
Storage URL \- optional (OS_STORAGE_URL)
|
|
.IP \[bu] 2
|
|
Config: storage_url
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_SWIFT_STORAGE_URL
|
|
.IP \[bu] 2
|
|
Type: string
|
|
.IP \[bu] 2
|
|
Default: \[dq]\[dq]
|
|
.SS \[en]swift\-auth\-token
|
|
.PP
|
|
Auth Token from alternate authentication \- optional (OS_AUTH_TOKEN)
|
|
.IP \[bu] 2
|
|
Config: auth_token
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_SWIFT_AUTH_TOKEN
|
|
.IP \[bu] 2
|
|
Type: string
|
|
.IP \[bu] 2
|
|
Default: \[dq]\[dq]
|
|
.SS \[en]swift\-application\-credential\-id
|
|
.PP
|
|
Application Credential ID (OS_APPLICATION_CREDENTIAL_ID)
|
|
.IP \[bu] 2
|
|
Config: application_credential_id
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_SWIFT_APPLICATION_CREDENTIAL_ID
|
|
.IP \[bu] 2
|
|
Type: string
|
|
.IP \[bu] 2
|
|
Default: \[dq]\[dq]
|
|
.SS \[en]swift\-application\-credential\-name
|
|
.PP
|
|
Application Credential Name (OS_APPLICATION_CREDENTIAL_NAME)
|
|
.IP \[bu] 2
|
|
Config: application_credential_name
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_SWIFT_APPLICATION_CREDENTIAL_NAME
|
|
.IP \[bu] 2
|
|
Type: string
|
|
.IP \[bu] 2
|
|
Default: \[dq]\[dq]
|
|
.SS \[en]swift\-application\-credential\-secret
|
|
.PP
|
|
Application Credential Secret (OS_APPLICATION_CREDENTIAL_SECRET)
|
|
.IP \[bu] 2
|
|
Config: application_credential_secret
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_SWIFT_APPLICATION_CREDENTIAL_SECRET
|
|
.IP \[bu] 2
|
|
Type: string
|
|
.IP \[bu] 2
|
|
Default: \[dq]\[dq]
|
|
.SS \[en]swift\-auth\-version
|
|
.PP
|
|
AuthVersion \- optional \- set to (1,2,3) if your auth URL has no
|
|
version (ST_AUTH_VERSION)
|
|
.IP \[bu] 2
|
|
Config: auth_version
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_SWIFT_AUTH_VERSION
|
|
.IP \[bu] 2
|
|
Type: int
|
|
.IP \[bu] 2
|
|
Default: 0
|
|
.SS \[en]swift\-endpoint\-type
|
|
.PP
|
|
Endpoint type to choose from the service catalogue (OS_ENDPOINT_TYPE)
|
|
.IP \[bu] 2
|
|
Config: endpoint_type
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_SWIFT_ENDPOINT_TYPE
|
|
.IP \[bu] 2
|
|
Type: string
|
|
.IP \[bu] 2
|
|
Default: \[lq]public\[rq]
|
|
.IP \[bu] 2
|
|
Examples:
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
\[lq]public\[rq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Public (default, choose this if not sure)
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[lq]internal\[rq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Internal (use internal service net)
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[lq]admin\[rq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Admin
|
|
.RE
|
|
.RE
|
|
.SS \[en]swift\-storage\-policy
|
|
.PP
|
|
The storage policy to use when creating a new container
|
|
.PP
|
|
This applies the specified storage policy when creating a new container.
|
|
The policy cannot be changed afterwards.
|
|
The allowed configuration values and their meaning depend on your Swift
|
|
storage provider.
|
|
.IP \[bu] 2
|
|
Config: storage_policy
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_SWIFT_STORAGE_POLICY
|
|
.IP \[bu] 2
|
|
Type: string
|
|
.IP \[bu] 2
|
|
Default: \[dq]\[dq]
|
|
.IP \[bu] 2
|
|
Examples:
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
\[dq]\[dq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Default
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[lq]pcs\[rq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
OVH Public Cloud Storage
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[lq]pca\[rq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
OVH Public Cloud Archive
|
|
.RE
|
|
.RE
|
|
.SS Advanced Options
|
|
.PP
|
|
Here are the advanced options specific to swift (OpenStack Swift
|
|
(Rackspace Cloud Files, Memset Memstore, OVH)).
|
|
.SS \[en]swift\-chunk\-size
|
|
.PP
|
|
Above this size files will be chunked into a _segments container.
|
|
.PP
|
|
Above this size files will be chunked into a _segments container.
|
|
The default for this is 5GB which is its maximum value.
|
|
.IP \[bu] 2
|
|
Config: chunk_size
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_SWIFT_CHUNK_SIZE
|
|
.IP \[bu] 2
|
|
Type: SizeSuffix
|
|
.IP \[bu] 2
|
|
Default: 5G
|
|
.SS \[en]swift\-no\-chunk
|
|
.PP
|
|
Don\[cq]t chunk files during streaming upload.
|
|
.PP
|
|
When doing streaming uploads (eg using rcat or mount) setting this flag
|
|
will cause the swift backend to not upload chunked files.
|
|
.PP
|
|
This will limit the maximum upload size to 5GB.
|
|
However non chunked files are easier to deal with and have an MD5SUM.
|
|
.PP
|
|
Rclone will still chunk files bigger than chunk_size when doing normal
|
|
copy operations.
|
|
.IP \[bu] 2
|
|
Config: no_chunk
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_SWIFT_NO_CHUNK
|
|
.IP \[bu] 2
|
|
Type: bool
|
|
.IP \[bu] 2
|
|
Default: false
|
|
.SS \[en]swift\-encoding
|
|
.PP
|
|
This sets the encoding for the backend.
|
|
.PP
|
|
See: the encoding section in the
|
|
overview (https://rclone.org/overview/#encoding) for more info.
|
|
.IP \[bu] 2
|
|
Config: encoding
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_SWIFT_ENCODING
|
|
.IP \[bu] 2
|
|
Type: MultiEncoder
|
|
.IP \[bu] 2
|
|
Default: Slash,InvalidUtf8
|
|
.SS Modified time
|
|
.PP
|
|
The modified time is stored as metadata on the object as
|
|
\f[C]X\-Object\-Meta\-Mtime\f[R] as floating point since the epoch
|
|
accurate to 1 ns.
|
|
.PP
|
|
This is a de facto standard (used in the official python\-swiftclient
|
|
amongst others) for storing the modification time for an object.
|
|
.SS Restricted filename characters
|
|
.PP
|
|
.TS
|
|
tab(@);
|
|
l c c.
|
|
T{
|
|
Character
|
|
T}@T{
|
|
Value
|
|
T}@T{
|
|
Replacement
|
|
T}
|
|
_
|
|
T{
|
|
NUL
|
|
T}@T{
|
|
0x00
|
|
T}@T{
|
|
\[u2400]
|
|
T}
|
|
T{
|
|
/
|
|
T}@T{
|
|
0x2F
|
|
T}@T{
|
|
\[uFF0F]
|
|
T}
|
|
.TE
|
|
.PP
|
|
Invalid UTF\-8 bytes will also be
|
|
replaced (https://rclone.org/overview/#invalid-utf8), as they can\[cq]t
|
|
be used in JSON strings.
|
|
.SS Limitations
|
|
.PP
|
|
The Swift API doesn\[cq]t return a correct MD5SUM for segmented files
|
|
(Dynamic or Static Large Objects) so rclone won\[cq]t check or use the
|
|
MD5SUM for these.
|
|
.SS Troubleshooting
|
|
.SS Rclone gives Failed to create file system for \[lq]remote:\[rq]: Bad Request
|
|
.PP
|
|
Due to an oddity of the underlying swift library, it gives a \[lq]Bad
|
|
Request\[rq] error rather than a more sensible error when the
|
|
authentication fails for Swift.
|
|
.PP
|
|
So this most likely means your username / password is wrong.
|
|
You can investigate further with the \f[C]\-\-dump\-bodies\f[R] flag.
|
|
.PP
|
|
This may also be caused by specifying the region when you shouldn\[cq]t
|
|
have (eg OVH).
|
|
.SS Rclone gives Failed to create file system: Response didn\[cq]t have storage url and auth token
|
|
.PP
|
|
This is most likely caused by forgetting to specify your tenant when
|
|
setting up a swift remote.
|
|
.SS pCloud
|
|
.PP
|
|
Paths are specified as \f[C]remote:path\f[R]
|
|
.PP
|
|
Paths may be as deep as required, eg
|
|
\f[C]remote:directory/subdirectory\f[R].
|
|
.PP
|
|
The initial setup for pCloud involves getting a token from pCloud which
|
|
you need to do in your browser.
|
|
\f[C]rclone config\f[R] walks you through it.
|
|
.PP
|
|
Here is an example of how to make a remote called \f[C]remote\f[R].
|
|
First run:
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone config
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
This will guide you through an interactive setup process:
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
No remotes found \- make a new one
|
|
n) New remote
|
|
s) Set configuration password
|
|
q) Quit config
|
|
n/s/q> n
|
|
name> remote
|
|
Type of storage to configure.
|
|
Choose a number from below, or type in your own value
|
|
[snip]
|
|
XX / Pcloud
|
|
\[rs] \[dq]pcloud\[dq]
|
|
[snip]
|
|
Storage> pcloud
|
|
Pcloud App Client Id \- leave blank normally.
|
|
client_id>
|
|
Pcloud App Client Secret \- leave blank normally.
|
|
client_secret>
|
|
Remote config
|
|
Use auto config?
|
|
* Say Y if not sure
|
|
* Say N if you are working on a remote or headless machine
|
|
y) Yes
|
|
n) No
|
|
y/n> y
|
|
If your browser doesn\[aq]t open automatically go to the following link: http://127.0.0.1:53682/auth
|
|
Log in and authorize rclone for access
|
|
Waiting for code...
|
|
Got code
|
|
\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-
|
|
[remote]
|
|
client_id =
|
|
client_secret =
|
|
token = {\[dq]access_token\[dq]:\[dq]XXX\[dq],\[dq]token_type\[dq]:\[dq]bearer\[dq],\[dq]expiry\[dq]:\[dq]0001\-01\-01T00:00:00Z\[dq]}
|
|
\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-
|
|
y) Yes this is OK
|
|
e) Edit this remote
|
|
d) Delete this remote
|
|
y/e/d> y
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
See the remote setup docs (https://rclone.org/remote_setup/) for how to
|
|
set it up on a machine with no Internet browser available.
|
|
.PP
|
|
Note that rclone runs a webserver on your local machine to collect the
|
|
token as returned from pCloud.
|
|
This only runs from the moment it opens your browser to the moment you
|
|
get back the verification code.
|
|
This is on \f[C]http://127.0.0.1:53682/\f[R] and this it may require you
|
|
to unblock it temporarily if you are running a host firewall.
|
|
.PP
|
|
Once configured you can then use \f[C]rclone\f[R] like this,
|
|
.PP
|
|
List directories in top level of your pCloud
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone lsd remote:
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
List all the files in your pCloud
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone ls remote:
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
To copy a local directory to an pCloud directory called backup
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone copy /home/source remote:backup
|
|
\f[R]
|
|
.fi
|
|
.SS Modified time and hashes
|
|
.PP
|
|
pCloud allows modification times to be set on objects accurate to 1
|
|
second.
|
|
These will be used to detect whether objects need syncing or not.
|
|
In order to set a Modification time pCloud requires the object be
|
|
re\-uploaded.
|
|
.PP
|
|
pCloud supports MD5 and SHA1 type hashes, so you can use the
|
|
\f[C]\-\-checksum\f[R] flag.
|
|
.SS Restricted filename characters
|
|
.PP
|
|
In addition to the default restricted characters
|
|
set (https://rclone.org/overview/#restricted-characters) the following
|
|
characters are also replaced:
|
|
.PP
|
|
.TS
|
|
tab(@);
|
|
l c c.
|
|
T{
|
|
Character
|
|
T}@T{
|
|
Value
|
|
T}@T{
|
|
Replacement
|
|
T}
|
|
_
|
|
T{
|
|
\[rs]
|
|
T}@T{
|
|
0x5C
|
|
T}@T{
|
|
\[uFF3C]
|
|
T}
|
|
.TE
|
|
.PP
|
|
Invalid UTF\-8 bytes will also be
|
|
replaced (https://rclone.org/overview/#invalid-utf8), as they can\[cq]t
|
|
be used in JSON strings.
|
|
.SS Deleting files
|
|
.PP
|
|
Deleted files will be moved to the trash.
|
|
Your subscription level will determine how long items stay in the trash.
|
|
\f[C]rclone cleanup\f[R] can be used to empty the trash.
|
|
.SS Root folder ID
|
|
.PP
|
|
You can set the \f[C]root_folder_id\f[R] for rclone.
|
|
This is the directory (identified by its \f[C]Folder ID\f[R]) that
|
|
rclone considers to be the root of your pCloud drive.
|
|
.PP
|
|
Normally you will leave this blank and rclone will determine the correct
|
|
root to use itself.
|
|
.PP
|
|
However you can set this to restrict rclone to a specific folder
|
|
hierarchy.
|
|
.PP
|
|
In order to do this you will have to find the \f[C]Folder ID\f[R] of the
|
|
directory you wish rclone to display.
|
|
This will be the \f[C]folder\f[R] field of the URL when you open the
|
|
relevant folder in the pCloud web interface.
|
|
.PP
|
|
So if the folder you want rclone to use has a URL which looks like
|
|
\f[C]https://my.pcloud.com/#page=filemanager&folder=5xxxxxxxx8&tpl=foldergrid\f[R]
|
|
in the browser, then you use \f[C]5xxxxxxxx8\f[R] as the
|
|
\f[C]root_folder_id\f[R] in the config.
|
|
.SS Standard Options
|
|
.PP
|
|
Here are the standard options specific to pcloud (Pcloud).
|
|
.SS \[en]pcloud\-client\-id
|
|
.PP
|
|
Pcloud App Client Id Leave blank normally.
|
|
.IP \[bu] 2
|
|
Config: client_id
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_PCLOUD_CLIENT_ID
|
|
.IP \[bu] 2
|
|
Type: string
|
|
.IP \[bu] 2
|
|
Default: \[dq]\[dq]
|
|
.SS \[en]pcloud\-client\-secret
|
|
.PP
|
|
Pcloud App Client Secret Leave blank normally.
|
|
.IP \[bu] 2
|
|
Config: client_secret
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_PCLOUD_CLIENT_SECRET
|
|
.IP \[bu] 2
|
|
Type: string
|
|
.IP \[bu] 2
|
|
Default: \[dq]\[dq]
|
|
.SS Advanced Options
|
|
.PP
|
|
Here are the advanced options specific to pcloud (Pcloud).
|
|
.SS \[en]pcloud\-encoding
|
|
.PP
|
|
This sets the encoding for the backend.
|
|
.PP
|
|
See: the encoding section in the
|
|
overview (https://rclone.org/overview/#encoding) for more info.
|
|
.IP \[bu] 2
|
|
Config: encoding
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_PCLOUD_ENCODING
|
|
.IP \[bu] 2
|
|
Type: MultiEncoder
|
|
.IP \[bu] 2
|
|
Default: Slash,BackSlash,Del,Ctl,InvalidUtf8,Dot
|
|
.SS \[en]pcloud\-root\-folder\-id
|
|
.PP
|
|
Fill in for rclone to use a non root folder as its starting point.
|
|
.IP \[bu] 2
|
|
Config: root_folder_id
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_PCLOUD_ROOT_FOLDER_ID
|
|
.IP \[bu] 2
|
|
Type: string
|
|
.IP \[bu] 2
|
|
Default: \[lq]d0\[rq]
|
|
.SS premiumize.me
|
|
.PP
|
|
Paths are specified as \f[C]remote:path\f[R]
|
|
.PP
|
|
Paths may be as deep as required, eg
|
|
\f[C]remote:directory/subdirectory\f[R].
|
|
.PP
|
|
The initial setup for premiumize.me (https://premiumize.me/) involves
|
|
getting a token from premiumize.me which you need to do in your browser.
|
|
\f[C]rclone config\f[R] walks you through it.
|
|
.PP
|
|
Here is an example of how to make a remote called \f[C]remote\f[R].
|
|
First run:
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone config
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
This will guide you through an interactive setup process:
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
No remotes found \- make a new one
|
|
n) New remote
|
|
s) Set configuration password
|
|
q) Quit config
|
|
n/s/q> n
|
|
name> remote
|
|
Type of storage to configure.
|
|
Enter a string value. Press Enter for the default (\[dq]\[dq]).
|
|
Choose a number from below, or type in your own value
|
|
[snip]
|
|
XX / premiumize.me
|
|
\[rs] \[dq]premiumizeme\[dq]
|
|
[snip]
|
|
Storage> premiumizeme
|
|
** See help for premiumizeme backend at: https://rclone.org/premiumizeme/ **
|
|
|
|
Remote config
|
|
Use auto config?
|
|
* Say Y if not sure
|
|
* Say N if you are working on a remote or headless machine
|
|
y) Yes
|
|
n) No
|
|
y/n> y
|
|
If your browser doesn\[aq]t open automatically go to the following link: http://127.0.0.1:53682/auth
|
|
Log in and authorize rclone for access
|
|
Waiting for code...
|
|
Got code
|
|
\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-
|
|
[remote]
|
|
type = premiumizeme
|
|
token = {\[dq]access_token\[dq]:\[dq]XXX\[dq],\[dq]token_type\[dq]:\[dq]Bearer\[dq],\[dq]refresh_token\[dq]:\[dq]XXX\[dq],\[dq]expiry\[dq]:\[dq]2029\-08\-07T18:44:15.548915378+01:00\[dq]}
|
|
\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-
|
|
y) Yes this is OK
|
|
e) Edit this remote
|
|
d) Delete this remote
|
|
y/e/d>
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
See the remote setup docs (https://rclone.org/remote_setup/) for how to
|
|
set it up on a machine with no Internet browser available.
|
|
.PP
|
|
Note that rclone runs a webserver on your local machine to collect the
|
|
token as returned from premiumize.me.
|
|
This only runs from the moment it opens your browser to the moment you
|
|
get back the verification code.
|
|
This is on \f[C]http://127.0.0.1:53682/\f[R] and this it may require you
|
|
to unblock it temporarily if you are running a host firewall.
|
|
.PP
|
|
Once configured you can then use \f[C]rclone\f[R] like this,
|
|
.PP
|
|
List directories in top level of your premiumize.me
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone lsd remote:
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
List all the files in your premiumize.me
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone ls remote:
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
To copy a local directory to an premiumize.me directory called backup
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone copy /home/source remote:backup
|
|
\f[R]
|
|
.fi
|
|
.SS Modified time and hashes
|
|
.PP
|
|
premiumize.me does not support modification times or hashes, therefore
|
|
syncing will default to \f[C]\-\-size\-only\f[R] checking.
|
|
Note that using \f[C]\-\-update\f[R] will work.
|
|
.SS Restricted filename characters
|
|
.PP
|
|
In addition to the default restricted characters
|
|
set (https://rclone.org/overview/#restricted-characters) the following
|
|
characters are also replaced:
|
|
.PP
|
|
.TS
|
|
tab(@);
|
|
l c c.
|
|
T{
|
|
Character
|
|
T}@T{
|
|
Value
|
|
T}@T{
|
|
Replacement
|
|
T}
|
|
_
|
|
T{
|
|
\[rs]
|
|
T}@T{
|
|
0x5C
|
|
T}@T{
|
|
\[uFF3C]
|
|
T}
|
|
T{
|
|
\[dq]
|
|
T}@T{
|
|
0x22
|
|
T}@T{
|
|
\[uFF02]
|
|
T}
|
|
.TE
|
|
.PP
|
|
Invalid UTF\-8 bytes will also be
|
|
replaced (https://rclone.org/overview/#invalid-utf8), as they can\[cq]t
|
|
be used in JSON strings.
|
|
.SS Standard Options
|
|
.PP
|
|
Here are the standard options specific to premiumizeme (premiumize.me).
|
|
.SS \[en]premiumizeme\-api\-key
|
|
.PP
|
|
API Key.
|
|
.PP
|
|
This is not normally used \- use oauth instead.
|
|
.IP \[bu] 2
|
|
Config: api_key
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_PREMIUMIZEME_API_KEY
|
|
.IP \[bu] 2
|
|
Type: string
|
|
.IP \[bu] 2
|
|
Default: \[dq]\[dq]
|
|
.SS Advanced Options
|
|
.PP
|
|
Here are the advanced options specific to premiumizeme (premiumize.me).
|
|
.SS \[en]premiumizeme\-encoding
|
|
.PP
|
|
This sets the encoding for the backend.
|
|
.PP
|
|
See: the encoding section in the
|
|
overview (https://rclone.org/overview/#encoding) for more info.
|
|
.IP \[bu] 2
|
|
Config: encoding
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_PREMIUMIZEME_ENCODING
|
|
.IP \[bu] 2
|
|
Type: MultiEncoder
|
|
.IP \[bu] 2
|
|
Default: Slash,DoubleQuote,BackSlash,Del,Ctl,InvalidUtf8,Dot
|
|
.SS Limitations
|
|
.PP
|
|
Note that premiumize.me is case insensitive so you can\[cq]t have a file
|
|
called \[lq]Hello.doc\[rq] and one called \[lq]hello.doc\[rq].
|
|
.PP
|
|
premiumize.me file names can\[cq]t have the \f[C]\[rs]\f[R] or
|
|
\f[C]\[dq]\f[R] characters in.
|
|
rclone maps these to and from an identical looking unicode equivalents
|
|
\f[C]\[uFF3C]\f[R] and \f[C]\[uFF02]\f[R]
|
|
.PP
|
|
premiumize.me only supports filenames up to 255 characters in length.
|
|
.SS put.io
|
|
.PP
|
|
Paths are specified as \f[C]remote:path\f[R]
|
|
.PP
|
|
put.io paths may be as deep as required, eg
|
|
\f[C]remote:directory/subdirectory\f[R].
|
|
.PP
|
|
The initial setup for put.io involves getting a token from put.io which
|
|
you need to do in your browser.
|
|
\f[C]rclone config\f[R] walks you through it.
|
|
.PP
|
|
Here is an example of how to make a remote called \f[C]remote\f[R].
|
|
First run:
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone config
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
This will guide you through an interactive setup process:
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
No remotes found \- make a new one
|
|
n) New remote
|
|
s) Set configuration password
|
|
q) Quit config
|
|
n/s/q> n
|
|
name> putio
|
|
Type of storage to configure.
|
|
Enter a string value. Press Enter for the default (\[dq]\[dq]).
|
|
Choose a number from below, or type in your own value
|
|
[snip]
|
|
XX / Put.io
|
|
\[rs] \[dq]putio\[dq]
|
|
[snip]
|
|
Storage> putio
|
|
** See help for putio backend at: https://rclone.org/putio/ **
|
|
|
|
Remote config
|
|
Use auto config?
|
|
* Say Y if not sure
|
|
* Say N if you are working on a remote or headless machine
|
|
y) Yes
|
|
n) No
|
|
y/n> y
|
|
If your browser doesn\[aq]t open automatically go to the following link: http://127.0.0.1:53682/auth
|
|
Log in and authorize rclone for access
|
|
Waiting for code...
|
|
Got code
|
|
\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-
|
|
[putio]
|
|
type = putio
|
|
token = {\[dq]access_token\[dq]:\[dq]XXXXXXXX\[dq],\[dq]expiry\[dq]:\[dq]0001\-01\-01T00:00:00Z\[dq]}
|
|
\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-
|
|
y) Yes this is OK
|
|
e) Edit this remote
|
|
d) Delete this remote
|
|
y/e/d> y
|
|
Current remotes:
|
|
|
|
Name Type
|
|
==== ====
|
|
putio putio
|
|
|
|
e) Edit existing remote
|
|
n) New remote
|
|
d) Delete remote
|
|
r) Rename remote
|
|
c) Copy remote
|
|
s) Set configuration password
|
|
q) Quit config
|
|
e/n/d/r/c/s/q> q
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
Note that rclone runs a webserver on your local machine to collect the
|
|
token as returned from Google if you use auto config mode.
|
|
This only runs from the moment it opens your browser to the moment you
|
|
get back the verification code.
|
|
This is on \f[C]http://127.0.0.1:53682/\f[R] and this it may require you
|
|
to unblock it temporarily if you are running a host firewall, or use
|
|
manual mode.
|
|
.PP
|
|
You can then use it like this,
|
|
.PP
|
|
List directories in top level of your put.io
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone lsd remote:
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
List all the files in your put.io
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone ls remote:
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
To copy a local directory to a put.io directory called backup
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone copy /home/source remote:backup
|
|
\f[R]
|
|
.fi
|
|
.SS Restricted filename characters
|
|
.PP
|
|
In addition to the default restricted characters
|
|
set (https://rclone.org/overview/#restricted-characters) the following
|
|
characters are also replaced:
|
|
.PP
|
|
.TS
|
|
tab(@);
|
|
l c c.
|
|
T{
|
|
Character
|
|
T}@T{
|
|
Value
|
|
T}@T{
|
|
Replacement
|
|
T}
|
|
_
|
|
T{
|
|
\[rs]
|
|
T}@T{
|
|
0x5C
|
|
T}@T{
|
|
\[uFF3C]
|
|
T}
|
|
.TE
|
|
.PP
|
|
Invalid UTF\-8 bytes will also be
|
|
replaced (https://rclone.org/overview/#invalid-utf8), as they can\[cq]t
|
|
be used in JSON strings.
|
|
.SS Advanced Options
|
|
.PP
|
|
Here are the advanced options specific to putio (Put.io).
|
|
.SS \[en]putio\-encoding
|
|
.PP
|
|
This sets the encoding for the backend.
|
|
.PP
|
|
See: the encoding section in the
|
|
overview (https://rclone.org/overview/#encoding) for more info.
|
|
.IP \[bu] 2
|
|
Config: encoding
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_PUTIO_ENCODING
|
|
.IP \[bu] 2
|
|
Type: MultiEncoder
|
|
.IP \[bu] 2
|
|
Default: Slash,BackSlash,Del,Ctl,InvalidUtf8,Dot
|
|
.SS Seafile
|
|
.PP
|
|
This is a backend for the Seafile (https://www.seafile.com/) storage
|
|
service: \- It works with both the free community edition or the
|
|
professional edition.
|
|
\- Seafile versions 6.x and 7.x are all supported.
|
|
\- Encrypted libraries are also supported.
|
|
\- It supports 2FA enabled users
|
|
.SS Root mode vs Library mode
|
|
.PP
|
|
There are two distinct modes you can setup your remote: \- you point
|
|
your remote to the \f[B]root of the server\f[R], meaning you don\[cq]t
|
|
specify a library during the configuration: Paths are specified as
|
|
\f[C]remote:library\f[R].
|
|
You may put subdirectories in too, eg
|
|
\f[C]remote:library/path/to/dir\f[R].
|
|
\- you point your remote to a specific library during the configuration:
|
|
Paths are specified as \f[C]remote:path/to/dir\f[R].
|
|
\f[B]This is the recommended mode when using encrypted libraries\f[R].
|
|
(\f[I]This mode is possibly slightly faster than the root mode\f[R])
|
|
.SS Configuration in root mode
|
|
.PP
|
|
Here is an example of making a seafile configuration for a user with
|
|
\f[B]no\f[R] two\-factor authentication.
|
|
First run
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone config
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
This will guide you through an interactive setup process.
|
|
To authenticate you will need the URL of your server, your email (or
|
|
username) and your password.
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
No remotes found \- make a new one
|
|
n) New remote
|
|
s) Set configuration password
|
|
q) Quit config
|
|
n/s/q> n
|
|
name> seafile
|
|
Type of storage to configure.
|
|
Enter a string value. Press Enter for the default (\[dq]\[dq]).
|
|
Choose a number from below, or type in your own value
|
|
[snip]
|
|
XX / Seafile
|
|
\[rs] \[dq]seafile\[dq]
|
|
[snip]
|
|
Storage> seafile
|
|
** See help for seafile backend at: https://rclone.org/seafile/ **
|
|
|
|
URL of seafile host to connect to
|
|
Enter a string value. Press Enter for the default (\[dq]\[dq]).
|
|
Choose a number from below, or type in your own value
|
|
1 / Connect to cloud.seafile.com
|
|
\[rs] \[dq]https://cloud.seafile.com/\[dq]
|
|
url> http://my.seafile.server/
|
|
User name (usually email address)
|
|
Enter a string value. Press Enter for the default (\[dq]\[dq]).
|
|
user> me\[at]example.com
|
|
Password
|
|
y) Yes type in my own password
|
|
g) Generate random password
|
|
n) No leave this optional password blank (default)
|
|
y/g> y
|
|
Enter the password:
|
|
password:
|
|
Confirm the password:
|
|
password:
|
|
Two\-factor authentication (\[aq]true\[aq] if the account has 2FA enabled)
|
|
Enter a boolean value (true or false). Press Enter for the default (\[dq]false\[dq]).
|
|
2fa> false
|
|
Name of the library. Leave blank to access all non\-encrypted libraries.
|
|
Enter a string value. Press Enter for the default (\[dq]\[dq]).
|
|
library>
|
|
Library password (for encrypted libraries only). Leave blank if you pass it through the command line.
|
|
y) Yes type in my own password
|
|
g) Generate random password
|
|
n) No leave this optional password blank (default)
|
|
y/g/n> n
|
|
Edit advanced config? (y/n)
|
|
y) Yes
|
|
n) No (default)
|
|
y/n> n
|
|
Remote config
|
|
Two\-factor authentication is not enabled on this account.
|
|
\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-
|
|
[seafile]
|
|
type = seafile
|
|
url = http://my.seafile.server/
|
|
user = me\[at]example.com
|
|
pass = *** ENCRYPTED ***
|
|
2fa = false
|
|
\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-
|
|
y) Yes this is OK (default)
|
|
e) Edit this remote
|
|
d) Delete this remote
|
|
y/e/d> y
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
This remote is called \f[C]seafile\f[R].
|
|
It\[cq]s pointing to the root of your seafile server and can now be used
|
|
like this:
|
|
.PP
|
|
See all libraries
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone lsd seafile:
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
Create a new library
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone mkdir seafile:library
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
List the contents of a library
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone ls seafile:library
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
Sync \f[C]/home/local/directory\f[R] to the remote library, deleting any
|
|
excess files in the library.
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone sync /home/local/directory seafile:library
|
|
\f[R]
|
|
.fi
|
|
.SS Configuration in library mode
|
|
.PP
|
|
Here\[cq]s an example of a configuration in library mode with a user
|
|
that has the two\-factor authentication enabled.
|
|
Your 2FA code will be asked at the end of the configuration, and will
|
|
attempt to authenticate you:
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
No remotes found \- make a new one
|
|
n) New remote
|
|
s) Set configuration password
|
|
q) Quit config
|
|
n/s/q> n
|
|
name> seafile
|
|
Type of storage to configure.
|
|
Enter a string value. Press Enter for the default (\[dq]\[dq]).
|
|
Choose a number from below, or type in your own value
|
|
[snip]
|
|
XX / Seafile
|
|
\[rs] \[dq]seafile\[dq]
|
|
[snip]
|
|
Storage> seafile
|
|
** See help for seafile backend at: https://rclone.org/seafile/ **
|
|
|
|
URL of seafile host to connect to
|
|
Enter a string value. Press Enter for the default (\[dq]\[dq]).
|
|
Choose a number from below, or type in your own value
|
|
1 / Connect to cloud.seafile.com
|
|
\[rs] \[dq]https://cloud.seafile.com/\[dq]
|
|
url> http://my.seafile.server/
|
|
User name (usually email address)
|
|
Enter a string value. Press Enter for the default (\[dq]\[dq]).
|
|
user> me\[at]example.com
|
|
Password
|
|
y) Yes type in my own password
|
|
g) Generate random password
|
|
n) No leave this optional password blank (default)
|
|
y/g> y
|
|
Enter the password:
|
|
password:
|
|
Confirm the password:
|
|
password:
|
|
Two\-factor authentication (\[aq]true\[aq] if the account has 2FA enabled)
|
|
Enter a boolean value (true or false). Press Enter for the default (\[dq]false\[dq]).
|
|
2fa> true
|
|
Name of the library. Leave blank to access all non\-encrypted libraries.
|
|
Enter a string value. Press Enter for the default (\[dq]\[dq]).
|
|
library> My Library
|
|
Library password (for encrypted libraries only). Leave blank if you pass it through the command line.
|
|
y) Yes type in my own password
|
|
g) Generate random password
|
|
n) No leave this optional password blank (default)
|
|
y/g/n> n
|
|
Edit advanced config? (y/n)
|
|
y) Yes
|
|
n) No (default)
|
|
y/n> n
|
|
Remote config
|
|
Two\-factor authentication: please enter your 2FA code
|
|
2fa code> 123456
|
|
Authenticating...
|
|
Success!
|
|
\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-
|
|
[seafile]
|
|
type = seafile
|
|
url = http://my.seafile.server/
|
|
user = me\[at]example.com
|
|
pass =
|
|
2fa = true
|
|
library = My Library
|
|
\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-
|
|
y) Yes this is OK (default)
|
|
e) Edit this remote
|
|
d) Delete this remote
|
|
y/e/d> y
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
You\[cq]ll notice your password is blank in the configuration.
|
|
It\[cq]s because we only need the password to authenticate you once.
|
|
.PP
|
|
You specified \f[C]My Library\f[R] during the configuration.
|
|
The root of the remote is pointing at the root of the library
|
|
\f[C]My Library\f[R]:
|
|
.PP
|
|
See all files in the library:
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone lsd seafile:
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
Create a new directory inside the library
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone mkdir seafile:directory
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
List the contents of a directory
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone ls seafile:directory
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
Sync \f[C]/home/local/directory\f[R] to the remote library, deleting any
|
|
excess files in the library.
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone sync /home/local/directory seafile:
|
|
\f[R]
|
|
.fi
|
|
.SS \[en]fast\-list
|
|
.PP
|
|
Seafile version 7+ supports \f[C]\-\-fast\-list\f[R] which allows you to
|
|
use fewer transactions in exchange for more memory.
|
|
See the rclone docs (https://rclone.org/docs/#fast-list) for more
|
|
details.
|
|
Please note this is not supported on seafile server version 6.x
|
|
.SS Restricted filename characters
|
|
.PP
|
|
In addition to the default restricted characters
|
|
set (https://rclone.org/overview/#restricted-characters) the following
|
|
characters are also replaced:
|
|
.PP
|
|
.TS
|
|
tab(@);
|
|
l c c.
|
|
T{
|
|
Character
|
|
T}@T{
|
|
Value
|
|
T}@T{
|
|
Replacement
|
|
T}
|
|
_
|
|
T{
|
|
/
|
|
T}@T{
|
|
0x2F
|
|
T}@T{
|
|
\[uFF0F]
|
|
T}
|
|
T{
|
|
\[dq]
|
|
T}@T{
|
|
0x22
|
|
T}@T{
|
|
\[uFF02]
|
|
T}
|
|
T{
|
|
\[rs]
|
|
T}@T{
|
|
0x5C
|
|
T}@T{
|
|
\[uFF3C]
|
|
T}
|
|
.TE
|
|
.PP
|
|
Invalid UTF\-8 bytes will also be
|
|
replaced (https://rclone.org/overview/#invalid-utf8), as they can\[cq]t
|
|
be used in JSON strings.
|
|
.SS Seafile and rclone link
|
|
.PP
|
|
Rclone supports generating share links for non\-encrypted libraries
|
|
only.
|
|
They can either be for a file or a directory:
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone link seafile:seafile\-tutorial.doc
|
|
http://my.seafile.server/f/fdcd8a2f93f84b8b90f4/
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
or if run on a directory you will get:
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone link seafile:dir
|
|
http://my.seafile.server/d/9ea2455f6f55478bbb0d/
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
Please note a share link is unique for each file or directory.
|
|
If you run a link command on a file/dir that has already been shared,
|
|
you will get the exact same link.
|
|
.SS Compatibility
|
|
.PP
|
|
It has been actively tested using the seafile docker
|
|
image (https://github.com/haiwen/seafile-docker) of these versions: \-
|
|
6.3.4 community edition \- 7.0.5 community edition \- 7.1.3 community
|
|
edition
|
|
.PP
|
|
Versions below 6.0 are not supported.
|
|
Versions between 6.0 and 6.3 haven\[cq]t been tested and might not work
|
|
properly.
|
|
.SS Standard Options
|
|
.PP
|
|
Here are the standard options specific to seafile (seafile).
|
|
.SS \[en]seafile\-url
|
|
.PP
|
|
URL of seafile host to connect to
|
|
.IP \[bu] 2
|
|
Config: url
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_SEAFILE_URL
|
|
.IP \[bu] 2
|
|
Type: string
|
|
.IP \[bu] 2
|
|
Default: \[dq]\[dq]
|
|
.IP \[bu] 2
|
|
Examples:
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
\[lq]https://cloud.seafile.com/\[rq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Connect to cloud.seafile.com
|
|
.RE
|
|
.RE
|
|
.SS \[en]seafile\-user
|
|
.PP
|
|
User name (usually email address)
|
|
.IP \[bu] 2
|
|
Config: user
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_SEAFILE_USER
|
|
.IP \[bu] 2
|
|
Type: string
|
|
.IP \[bu] 2
|
|
Default: \[dq]\[dq]
|
|
.SS \[en]seafile\-pass
|
|
.PP
|
|
Password
|
|
.IP \[bu] 2
|
|
Config: pass
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_SEAFILE_PASS
|
|
.IP \[bu] 2
|
|
Type: string
|
|
.IP \[bu] 2
|
|
Default: \[dq]\[dq]
|
|
.SS \[en]seafile\-2fa
|
|
.PP
|
|
Two\-factor authentication (`true' if the account has 2FA enabled)
|
|
.IP \[bu] 2
|
|
Config: 2fa
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_SEAFILE_2FA
|
|
.IP \[bu] 2
|
|
Type: bool
|
|
.IP \[bu] 2
|
|
Default: false
|
|
.SS \[en]seafile\-library
|
|
.PP
|
|
Name of the library.
|
|
Leave blank to access all non\-encrypted libraries.
|
|
.IP \[bu] 2
|
|
Config: library
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_SEAFILE_LIBRARY
|
|
.IP \[bu] 2
|
|
Type: string
|
|
.IP \[bu] 2
|
|
Default: \[dq]\[dq]
|
|
.SS \[en]seafile\-library\-key
|
|
.PP
|
|
Library password (for encrypted libraries only).
|
|
Leave blank if you pass it through the command line.
|
|
.IP \[bu] 2
|
|
Config: library_key
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_SEAFILE_LIBRARY_KEY
|
|
.IP \[bu] 2
|
|
Type: string
|
|
.IP \[bu] 2
|
|
Default: \[dq]\[dq]
|
|
.SS \[en]seafile\-auth\-token
|
|
.PP
|
|
Authentication token
|
|
.IP \[bu] 2
|
|
Config: auth_token
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_SEAFILE_AUTH_TOKEN
|
|
.IP \[bu] 2
|
|
Type: string
|
|
.IP \[bu] 2
|
|
Default: \[dq]\[dq]
|
|
.SS Advanced Options
|
|
.PP
|
|
Here are the advanced options specific to seafile (seafile).
|
|
.SS \[en]seafile\-create\-library
|
|
.PP
|
|
Should rclone create a library if it doesn\[cq]t exist
|
|
.IP \[bu] 2
|
|
Config: create_library
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_SEAFILE_CREATE_LIBRARY
|
|
.IP \[bu] 2
|
|
Type: bool
|
|
.IP \[bu] 2
|
|
Default: false
|
|
.SS \[en]seafile\-encoding
|
|
.PP
|
|
This sets the encoding for the backend.
|
|
.PP
|
|
See: the encoding section in the
|
|
overview (https://rclone.org/overview/#encoding) for more info.
|
|
.IP \[bu] 2
|
|
Config: encoding
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_SEAFILE_ENCODING
|
|
.IP \[bu] 2
|
|
Type: MultiEncoder
|
|
.IP \[bu] 2
|
|
Default: Slash,DoubleQuote,BackSlash,Ctl,InvalidUtf8
|
|
.SS SFTP
|
|
.PP
|
|
SFTP is the Secure (or SSH) File Transfer
|
|
Protocol (https://en.wikipedia.org/wiki/SSH_File_Transfer_Protocol).
|
|
.PP
|
|
The SFTP backend can be used with a number of different providers:
|
|
.IP \[bu] 2
|
|
C14
|
|
.IP \[bu] 2
|
|
rsync.net
|
|
.PP
|
|
SFTP runs over SSH v2 and is installed as standard with most modern SSH
|
|
installations.
|
|
.PP
|
|
Paths are specified as \f[C]remote:path\f[R].
|
|
If the path does not begin with a \f[C]/\f[R] it is relative to the home
|
|
directory of the user.
|
|
An empty path \f[C]remote:\f[R] refers to the user\[cq]s home directory.
|
|
.PP
|
|
\[dq]Note that some SFTP servers will need the leading / \- Synology is
|
|
a good example of this.
|
|
rsync.net, on the other hand, requires users to OMIT the leading /.
|
|
.PP
|
|
Here is an example of making an SFTP configuration.
|
|
First run
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone config
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
This will guide you through an interactive setup process.
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
No remotes found \- make a new one
|
|
n) New remote
|
|
s) Set configuration password
|
|
q) Quit config
|
|
n/s/q> n
|
|
name> remote
|
|
Type of storage to configure.
|
|
Choose a number from below, or type in your own value
|
|
[snip]
|
|
XX / SSH/SFTP Connection
|
|
\[rs] \[dq]sftp\[dq]
|
|
[snip]
|
|
Storage> sftp
|
|
SSH host to connect to
|
|
Choose a number from below, or type in your own value
|
|
1 / Connect to example.com
|
|
\[rs] \[dq]example.com\[dq]
|
|
host> example.com
|
|
SSH username, leave blank for current username, ncw
|
|
user> sftpuser
|
|
SSH port, leave blank to use default (22)
|
|
port>
|
|
SSH password, leave blank to use ssh\-agent.
|
|
y) Yes type in my own password
|
|
g) Generate random password
|
|
n) No leave this optional password blank
|
|
y/g/n> n
|
|
Path to unencrypted PEM\-encoded private key file, leave blank to use ssh\-agent.
|
|
key_file>
|
|
Remote config
|
|
\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-
|
|
[remote]
|
|
host = example.com
|
|
user = sftpuser
|
|
port =
|
|
pass =
|
|
key_file =
|
|
\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-
|
|
y) Yes this is OK
|
|
e) Edit this remote
|
|
d) Delete this remote
|
|
y/e/d> y
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
This remote is called \f[C]remote\f[R] and can now be used like this:
|
|
.PP
|
|
See all directories in the home directory
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone lsd remote:
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
Make a new directory
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone mkdir remote:path/to/directory
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
List the contents of a directory
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone ls remote:path/to/directory
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
Sync \f[C]/home/local/directory\f[R] to the remote directory, deleting
|
|
any excess files in the directory.
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone sync /home/local/directory remote:directory
|
|
\f[R]
|
|
.fi
|
|
.SS SSH Authentication
|
|
.PP
|
|
The SFTP remote supports three authentication methods:
|
|
.IP \[bu] 2
|
|
Password
|
|
.IP \[bu] 2
|
|
Key file
|
|
.IP \[bu] 2
|
|
ssh\-agent
|
|
.PP
|
|
Key files should be PEM\-encoded private key files.
|
|
For instance \f[C]/home/$USER/.ssh/id_rsa\f[R].
|
|
Only unencrypted OpenSSH or PEM encrypted files are supported.
|
|
.PP
|
|
The key file can be specified in either an external file (key_file) or
|
|
contained within the rclone config file (key_pem).
|
|
If using key_pem in the config file, the entry should be on a single
|
|
line with new line (`' or `') separating lines.
|
|
i.e.\
|
|
.PP
|
|
key_pem = \[em]\[en]BEGIN RSA PRIVATE
|
|
KEY\[em]\[en]0gAMbMbaSsd\[em]\[en]END RSA PRIVATE KEY\[em]\[en]
|
|
.PP
|
|
This will generate it correctly for key_pem for use in the config:
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
awk \[aq]{printf \[dq]%s\[rs]\[rs]n\[dq], $0}\[aq] < \[ti]/.ssh/id_rsa
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
If you don\[cq]t specify \f[C]pass\f[R], \f[C]key_file\f[R], or
|
|
\f[C]key_pem\f[R] then rclone will attempt to contact an ssh\-agent.
|
|
.PP
|
|
You can also specify \f[C]key_use_agent\f[R] to force the usage of an
|
|
ssh\-agent.
|
|
In this case \f[C]key_file\f[R] or \f[C]key_pem\f[R] can also be
|
|
specified to force the usage of a specific key in the ssh\-agent.
|
|
.PP
|
|
Using an ssh\-agent is the only way to load encrypted OpenSSH keys at
|
|
the moment.
|
|
.PP
|
|
If you set the \f[C]\-\-sftp\-ask\-password\f[R] option, rclone will
|
|
prompt for a password when needed and no password has been configured.
|
|
.SS ssh\-agent on macOS
|
|
.PP
|
|
Note that there seem to be various problems with using an ssh\-agent on
|
|
macOS due to recent changes in the OS.
|
|
The most effective work\-around seems to be to start an ssh\-agent in
|
|
each session, eg
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
eval \[ga]ssh\-agent \-s\[ga] && ssh\-add \-A
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
And then at the end of the session
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
eval \[ga]ssh\-agent \-k\[ga]
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
These commands can be used in scripts of course.
|
|
.SS Modified time
|
|
.PP
|
|
Modified times are stored on the server to 1 second precision.
|
|
.PP
|
|
Modified times are used in syncing and are fully supported.
|
|
.PP
|
|
Some SFTP servers disable setting/modifying the file modification time
|
|
after upload (for example, certain configurations of ProFTPd with
|
|
mod_sftp).
|
|
If you are using one of these servers, you can set the option
|
|
\f[C]set_modtime = false\f[R] in your RClone backend configuration to
|
|
disable this behaviour.
|
|
.SS Standard Options
|
|
.PP
|
|
Here are the standard options specific to sftp (SSH/SFTP Connection).
|
|
.SS \[en]sftp\-host
|
|
.PP
|
|
SSH host to connect to
|
|
.IP \[bu] 2
|
|
Config: host
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_SFTP_HOST
|
|
.IP \[bu] 2
|
|
Type: string
|
|
.IP \[bu] 2
|
|
Default: \[dq]\[dq]
|
|
.IP \[bu] 2
|
|
Examples:
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
\[lq]example.com\[rq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Connect to example.com
|
|
.RE
|
|
.RE
|
|
.SS \[en]sftp\-user
|
|
.PP
|
|
SSH username, leave blank for current username, ncw
|
|
.IP \[bu] 2
|
|
Config: user
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_SFTP_USER
|
|
.IP \[bu] 2
|
|
Type: string
|
|
.IP \[bu] 2
|
|
Default: \[dq]\[dq]
|
|
.SS \[en]sftp\-port
|
|
.PP
|
|
SSH port, leave blank to use default (22)
|
|
.IP \[bu] 2
|
|
Config: port
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_SFTP_PORT
|
|
.IP \[bu] 2
|
|
Type: string
|
|
.IP \[bu] 2
|
|
Default: \[dq]\[dq]
|
|
.SS \[en]sftp\-pass
|
|
.PP
|
|
SSH password, leave blank to use ssh\-agent.
|
|
.IP \[bu] 2
|
|
Config: pass
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_SFTP_PASS
|
|
.IP \[bu] 2
|
|
Type: string
|
|
.IP \[bu] 2
|
|
Default: \[dq]\[dq]
|
|
.SS \[en]sftp\-key\-pem
|
|
.PP
|
|
Raw PEM\-encoded private key, If specified, will override key_file
|
|
parameter.
|
|
.IP \[bu] 2
|
|
Config: key_pem
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_SFTP_KEY_PEM
|
|
.IP \[bu] 2
|
|
Type: string
|
|
.IP \[bu] 2
|
|
Default: \[dq]\[dq]
|
|
.SS \[en]sftp\-key\-file
|
|
.PP
|
|
Path to PEM\-encoded private key file, leave blank or set
|
|
key\-use\-agent to use ssh\-agent.
|
|
.IP \[bu] 2
|
|
Config: key_file
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_SFTP_KEY_FILE
|
|
.IP \[bu] 2
|
|
Type: string
|
|
.IP \[bu] 2
|
|
Default: \[dq]\[dq]
|
|
.SS \[en]sftp\-key\-file\-pass
|
|
.PP
|
|
The passphrase to decrypt the PEM\-encoded private key file.
|
|
.PP
|
|
Only PEM encrypted key files (old OpenSSH format) are supported.
|
|
Encrypted keys in the new OpenSSH format can\[cq]t be used.
|
|
.IP \[bu] 2
|
|
Config: key_file_pass
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_SFTP_KEY_FILE_PASS
|
|
.IP \[bu] 2
|
|
Type: string
|
|
.IP \[bu] 2
|
|
Default: \[dq]\[dq]
|
|
.SS \[en]sftp\-key\-use\-agent
|
|
.PP
|
|
When set forces the usage of the ssh\-agent.
|
|
.PP
|
|
When key\-file is also set, the \[lq].pub\[rq] file of the specified
|
|
key\-file is read and only the associated key is requested from the
|
|
ssh\-agent.
|
|
This allows to avoid
|
|
\f[C]Too many authentication failures for *username*\f[R] errors when
|
|
the ssh\-agent contains many keys.
|
|
.IP \[bu] 2
|
|
Config: key_use_agent
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_SFTP_KEY_USE_AGENT
|
|
.IP \[bu] 2
|
|
Type: bool
|
|
.IP \[bu] 2
|
|
Default: false
|
|
.SS \[en]sftp\-use\-insecure\-cipher
|
|
.PP
|
|
Enable the use of insecure ciphers and key exchange methods.
|
|
.PP
|
|
This enables the use of the following insecure ciphers and key exchange
|
|
methods:
|
|
.IP \[bu] 2
|
|
aes128\-cbc
|
|
.IP \[bu] 2
|
|
aes192\-cbc
|
|
.IP \[bu] 2
|
|
aes256\-cbc
|
|
.IP \[bu] 2
|
|
3des\-cbc
|
|
.IP \[bu] 2
|
|
diffie\-hellman\-group\-exchange\-sha256
|
|
.IP \[bu] 2
|
|
diffie\-hellman\-group\-exchange\-sha1
|
|
.PP
|
|
Those algorithms are insecure and may allow plaintext data to be
|
|
recovered by an attacker.
|
|
.IP \[bu] 2
|
|
Config: use_insecure_cipher
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_SFTP_USE_INSECURE_CIPHER
|
|
.IP \[bu] 2
|
|
Type: bool
|
|
.IP \[bu] 2
|
|
Default: false
|
|
.IP \[bu] 2
|
|
Examples:
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
\[lq]false\[rq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Use default Cipher list.
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[lq]true\[rq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Enables the use of the aes128\-cbc cipher and
|
|
diffie\-hellman\-group\-exchange\-sha256,
|
|
diffie\-hellman\-group\-exchange\-sha1 key exchange.
|
|
.RE
|
|
.RE
|
|
.SS \[en]sftp\-disable\-hashcheck
|
|
.PP
|
|
Disable the execution of SSH commands to determine if remote file
|
|
hashing is available.
|
|
Leave blank or set to false to enable hashing (recommended), set to true
|
|
to disable hashing.
|
|
.IP \[bu] 2
|
|
Config: disable_hashcheck
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_SFTP_DISABLE_HASHCHECK
|
|
.IP \[bu] 2
|
|
Type: bool
|
|
.IP \[bu] 2
|
|
Default: false
|
|
.SS Advanced Options
|
|
.PP
|
|
Here are the advanced options specific to sftp (SSH/SFTP Connection).
|
|
.SS \[en]sftp\-ask\-password
|
|
.PP
|
|
Allow asking for SFTP password when needed.
|
|
.PP
|
|
If this is set and no password is supplied then rclone will: \- ask for
|
|
a password \- not contact the ssh agent
|
|
.IP \[bu] 2
|
|
Config: ask_password
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_SFTP_ASK_PASSWORD
|
|
.IP \[bu] 2
|
|
Type: bool
|
|
.IP \[bu] 2
|
|
Default: false
|
|
.SS \[en]sftp\-path\-override
|
|
.PP
|
|
Override path used by SSH connection.
|
|
.PP
|
|
This allows checksum calculation when SFTP and SSH paths are different.
|
|
This issue affects among others Synology NAS boxes.
|
|
.PP
|
|
Shared folders can be found in directories representing volumes
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone sync /home/local/directory remote:/directory \-\-ssh\-path\-override /volume2/directory
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
Home directory can be found in a shared folder called \[lq]home\[rq]
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone sync /home/local/directory remote:/home/directory \-\-ssh\-path\-override /volume1/homes/USER/directory
|
|
\f[R]
|
|
.fi
|
|
.IP \[bu] 2
|
|
Config: path_override
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_SFTP_PATH_OVERRIDE
|
|
.IP \[bu] 2
|
|
Type: string
|
|
.IP \[bu] 2
|
|
Default: \[dq]\[dq]
|
|
.SS \[en]sftp\-set\-modtime
|
|
.PP
|
|
Set the modified time on the remote if set.
|
|
.IP \[bu] 2
|
|
Config: set_modtime
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_SFTP_SET_MODTIME
|
|
.IP \[bu] 2
|
|
Type: bool
|
|
.IP \[bu] 2
|
|
Default: true
|
|
.SS \[en]sftp\-md5sum\-command
|
|
.PP
|
|
The command used to read md5 hashes.
|
|
Leave blank for autodetect.
|
|
.IP \[bu] 2
|
|
Config: md5sum_command
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_SFTP_MD5SUM_COMMAND
|
|
.IP \[bu] 2
|
|
Type: string
|
|
.IP \[bu] 2
|
|
Default: \[dq]\[dq]
|
|
.SS \[en]sftp\-sha1sum\-command
|
|
.PP
|
|
The command used to read sha1 hashes.
|
|
Leave blank for autodetect.
|
|
.IP \[bu] 2
|
|
Config: sha1sum_command
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_SFTP_SHA1SUM_COMMAND
|
|
.IP \[bu] 2
|
|
Type: string
|
|
.IP \[bu] 2
|
|
Default: \[dq]\[dq]
|
|
.SS \[en]sftp\-skip\-links
|
|
.PP
|
|
Set to skip any symlinks and any other non regular files.
|
|
.IP \[bu] 2
|
|
Config: skip_links
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_SFTP_SKIP_LINKS
|
|
.IP \[bu] 2
|
|
Type: bool
|
|
.IP \[bu] 2
|
|
Default: false
|
|
.SS Limitations
|
|
.PP
|
|
SFTP supports checksums if the same login has shell access and
|
|
\f[C]md5sum\f[R] or \f[C]sha1sum\f[R] as well as \f[C]echo\f[R] are in
|
|
the remote\[cq]s PATH.
|
|
This remote checksumming (file hashing) is recommended and enabled by
|
|
default.
|
|
Disabling the checksumming may be required if you are connecting to SFTP
|
|
servers which are not under your control, and to which the execution of
|
|
remote commands is prohibited.
|
|
Set the configuration option \f[C]disable_hashcheck\f[R] to
|
|
\f[C]true\f[R] to disable checksumming.
|
|
.PP
|
|
SFTP also supports \f[C]about\f[R] if the same login has shell access
|
|
and \f[C]df\f[R] are in the remote\[cq]s PATH.
|
|
\f[C]about\f[R] will return the total space, free space, and used space
|
|
on the remote for the disk of the specified path on the remote or, if
|
|
not set, the disk of the root on the remote.
|
|
\f[C]about\f[R] will fail if it does not have shell access or if
|
|
\f[C]df\f[R] is not in the remote\[cq]s PATH.
|
|
.PP
|
|
Note that some SFTP servers (eg Synology) the paths are different for
|
|
SSH and SFTP so the hashes can\[cq]t be calculated properly.
|
|
For them using \f[C]disable_hashcheck\f[R] is a good idea.
|
|
.PP
|
|
The only ssh agent supported under Windows is Putty\[cq]s pageant.
|
|
.PP
|
|
The Go SSH library disables the use of the aes128\-cbc cipher by
|
|
default, due to security concerns.
|
|
This can be re\-enabled on a per\-connection basis by setting the
|
|
\f[C]use_insecure_cipher\f[R] setting in the configuration file to
|
|
\f[C]true\f[R].
|
|
Further details on the insecurity of this cipher can be found [in this
|
|
paper] (http://www.isg.rhul.ac.uk/\[ti]kp/SandPfinal.pdf).
|
|
.PP
|
|
SFTP isn\[cq]t supported under plan9 until this
|
|
issue (https://github.com/pkg/sftp/issues/156) is fixed.
|
|
.PP
|
|
Note that since SFTP isn\[cq]t HTTP based the following flags don\[cq]t
|
|
work with it: \f[C]\-\-dump\-headers\f[R], \f[C]\-\-dump\-bodies\f[R],
|
|
\f[C]\-\-dump\-auth\f[R]
|
|
.PP
|
|
Note that \f[C]\-\-timeout\f[R] isn\[cq]t supported (but
|
|
\f[C]\-\-contimeout\f[R] is).
|
|
.SS C14
|
|
.PP
|
|
C14 is supported through the SFTP backend.
|
|
.PP
|
|
See C14\[cq]s
|
|
documentation (https://www.online.net/en/storage/c14-cold-storage)
|
|
.SS rsync.net
|
|
.PP
|
|
rsync.net is supported through the SFTP backend.
|
|
.PP
|
|
See rsync.net\[cq]s documentation of rclone
|
|
examples (https://www.rsync.net/products/rclone.html).
|
|
.SS SugarSync
|
|
.PP
|
|
SugarSync (https://sugarsync.com) is a cloud service that enables active
|
|
synchronization of files across computers and other devices for file
|
|
backup, access, syncing, and sharing.
|
|
.PP
|
|
The initial setup for SugarSync involves getting a token from SugarSync
|
|
which you can do with rclone.
|
|
\f[C]rclone config\f[R] walks you through it.
|
|
.PP
|
|
Here is an example of how to make a remote called \f[C]remote\f[R].
|
|
First run:
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone config
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
This will guide you through an interactive setup process:
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
No remotes found \- make a new one
|
|
n) New remote
|
|
s) Set configuration password
|
|
q) Quit config
|
|
n/s/q> n
|
|
name> remote
|
|
Type of storage to configure.
|
|
Enter a string value. Press Enter for the default (\[dq]\[dq]).
|
|
Choose a number from below, or type in your own value
|
|
[snip]
|
|
XX / Sugarsync
|
|
\[rs] \[dq]sugarsync\[dq]
|
|
[snip]
|
|
Storage> sugarsync
|
|
** See help for sugarsync backend at: https://rclone.org/sugarsync/ **
|
|
|
|
Sugarsync App ID.
|
|
Leave blank to use rclone\[aq]s.
|
|
Enter a string value. Press Enter for the default (\[dq]\[dq]).
|
|
app_id>
|
|
Sugarsync Access Key ID.
|
|
Leave blank to use rclone\[aq]s.
|
|
Enter a string value. Press Enter for the default (\[dq]\[dq]).
|
|
access_key_id>
|
|
Sugarsync Private Access Key
|
|
Leave blank to use rclone\[aq]s.
|
|
Enter a string value. Press Enter for the default (\[dq]\[dq]).
|
|
private_access_key>
|
|
Permanently delete files if true
|
|
otherwise put them in the deleted files.
|
|
Enter a boolean value (true or false). Press Enter for the default (\[dq]false\[dq]).
|
|
hard_delete>
|
|
Edit advanced config? (y/n)
|
|
y) Yes
|
|
n) No (default)
|
|
y/n> n
|
|
Remote config
|
|
Username (email address)> nick\[at]craig\-wood.com
|
|
Your Sugarsync password is only required during setup and will not be stored.
|
|
password:
|
|
\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-
|
|
[remote]
|
|
type = sugarsync
|
|
refresh_token = https://api.sugarsync.com/app\-authorization/XXXXXXXXXXXXXXXXXX
|
|
\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-
|
|
y) Yes this is OK (default)
|
|
e) Edit this remote
|
|
d) Delete this remote
|
|
y/e/d> y
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
Note that the config asks for your email and password but doesn\[cq]t
|
|
store them, it only uses them to get the initial token.
|
|
.PP
|
|
Once configured you can then use \f[C]rclone\f[R] like this,
|
|
.PP
|
|
List directories (sync folders) in top level of your SugarSync
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone lsd remote:
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
List all the files in your SugarSync folder \[lq]Test\[rq]
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone ls remote:Test
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
To copy a local directory to an SugarSync folder called backup
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone copy /home/source remote:backup
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
Paths are specified as \f[C]remote:path\f[R]
|
|
.PP
|
|
Paths may be as deep as required, eg
|
|
\f[C]remote:directory/subdirectory\f[R].
|
|
.PP
|
|
\f[B]NB\f[R] you can\[cq]t create files in the top level folder you have
|
|
to create a folder, which rclone will create as a \[lq]Sync Folder\[rq]
|
|
with SugarSync.
|
|
.SS Modified time and hashes
|
|
.PP
|
|
SugarSync does not support modification times or hashes, therefore
|
|
syncing will default to \f[C]\-\-size\-only\f[R] checking.
|
|
Note that using \f[C]\-\-update\f[R] will work as rclone can read the
|
|
time files were uploaded.
|
|
.SS Restricted filename characters
|
|
.PP
|
|
SugarSync replaces the default restricted characters
|
|
set (https://rclone.org/overview/#restricted-characters) except for DEL.
|
|
.PP
|
|
Invalid UTF\-8 bytes will also be
|
|
replaced (https://rclone.org/overview/#invalid-utf8), as they can\[cq]t
|
|
be used in XML strings.
|
|
.SS Deleting files
|
|
.PP
|
|
Deleted files will be moved to the \[lq]Deleted items\[rq] folder by
|
|
default.
|
|
.PP
|
|
However you can supply the flag \f[C]\-\-sugarsync\-hard\-delete\f[R] or
|
|
set the config parameter \f[C]hard_delete = true\f[R] if you would like
|
|
files to be deleted straight away.
|
|
.SS Standard Options
|
|
.PP
|
|
Here are the standard options specific to sugarsync (Sugarsync).
|
|
.SS \[en]sugarsync\-app\-id
|
|
.PP
|
|
Sugarsync App ID.
|
|
.PP
|
|
Leave blank to use rclone\[cq]s.
|
|
.IP \[bu] 2
|
|
Config: app_id
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_SUGARSYNC_APP_ID
|
|
.IP \[bu] 2
|
|
Type: string
|
|
.IP \[bu] 2
|
|
Default: \[dq]\[dq]
|
|
.SS \[en]sugarsync\-access\-key\-id
|
|
.PP
|
|
Sugarsync Access Key ID.
|
|
.PP
|
|
Leave blank to use rclone\[cq]s.
|
|
.IP \[bu] 2
|
|
Config: access_key_id
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_SUGARSYNC_ACCESS_KEY_ID
|
|
.IP \[bu] 2
|
|
Type: string
|
|
.IP \[bu] 2
|
|
Default: \[dq]\[dq]
|
|
.SS \[en]sugarsync\-private\-access\-key
|
|
.PP
|
|
Sugarsync Private Access Key
|
|
.PP
|
|
Leave blank to use rclone\[cq]s.
|
|
.IP \[bu] 2
|
|
Config: private_access_key
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_SUGARSYNC_PRIVATE_ACCESS_KEY
|
|
.IP \[bu] 2
|
|
Type: string
|
|
.IP \[bu] 2
|
|
Default: \[dq]\[dq]
|
|
.SS \[en]sugarsync\-hard\-delete
|
|
.PP
|
|
Permanently delete files if true otherwise put them in the deleted
|
|
files.
|
|
.IP \[bu] 2
|
|
Config: hard_delete
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_SUGARSYNC_HARD_DELETE
|
|
.IP \[bu] 2
|
|
Type: bool
|
|
.IP \[bu] 2
|
|
Default: false
|
|
.SS Advanced Options
|
|
.PP
|
|
Here are the advanced options specific to sugarsync (Sugarsync).
|
|
.SS \[en]sugarsync\-refresh\-token
|
|
.PP
|
|
Sugarsync refresh token
|
|
.PP
|
|
Leave blank normally, will be auto configured by rclone.
|
|
.IP \[bu] 2
|
|
Config: refresh_token
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_SUGARSYNC_REFRESH_TOKEN
|
|
.IP \[bu] 2
|
|
Type: string
|
|
.IP \[bu] 2
|
|
Default: \[dq]\[dq]
|
|
.SS \[en]sugarsync\-authorization
|
|
.PP
|
|
Sugarsync authorization
|
|
.PP
|
|
Leave blank normally, will be auto configured by rclone.
|
|
.IP \[bu] 2
|
|
Config: authorization
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_SUGARSYNC_AUTHORIZATION
|
|
.IP \[bu] 2
|
|
Type: string
|
|
.IP \[bu] 2
|
|
Default: \[dq]\[dq]
|
|
.SS \[en]sugarsync\-authorization\-expiry
|
|
.PP
|
|
Sugarsync authorization expiry
|
|
.PP
|
|
Leave blank normally, will be auto configured by rclone.
|
|
.IP \[bu] 2
|
|
Config: authorization_expiry
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_SUGARSYNC_AUTHORIZATION_EXPIRY
|
|
.IP \[bu] 2
|
|
Type: string
|
|
.IP \[bu] 2
|
|
Default: \[dq]\[dq]
|
|
.SS \[en]sugarsync\-user
|
|
.PP
|
|
Sugarsync user
|
|
.PP
|
|
Leave blank normally, will be auto configured by rclone.
|
|
.IP \[bu] 2
|
|
Config: user
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_SUGARSYNC_USER
|
|
.IP \[bu] 2
|
|
Type: string
|
|
.IP \[bu] 2
|
|
Default: \[dq]\[dq]
|
|
.SS \[en]sugarsync\-root\-id
|
|
.PP
|
|
Sugarsync root id
|
|
.PP
|
|
Leave blank normally, will be auto configured by rclone.
|
|
.IP \[bu] 2
|
|
Config: root_id
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_SUGARSYNC_ROOT_ID
|
|
.IP \[bu] 2
|
|
Type: string
|
|
.IP \[bu] 2
|
|
Default: \[dq]\[dq]
|
|
.SS \[en]sugarsync\-deleted\-id
|
|
.PP
|
|
Sugarsync deleted folder id
|
|
.PP
|
|
Leave blank normally, will be auto configured by rclone.
|
|
.IP \[bu] 2
|
|
Config: deleted_id
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_SUGARSYNC_DELETED_ID
|
|
.IP \[bu] 2
|
|
Type: string
|
|
.IP \[bu] 2
|
|
Default: \[dq]\[dq]
|
|
.SS \[en]sugarsync\-encoding
|
|
.PP
|
|
This sets the encoding for the backend.
|
|
.PP
|
|
See: the encoding section in the
|
|
overview (https://rclone.org/overview/#encoding) for more info.
|
|
.IP \[bu] 2
|
|
Config: encoding
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_SUGARSYNC_ENCODING
|
|
.IP \[bu] 2
|
|
Type: MultiEncoder
|
|
.IP \[bu] 2
|
|
Default: Slash,Ctl,InvalidUtf8,Dot
|
|
.SS Tardigrade
|
|
.PP
|
|
Tardigrade (https://tardigrade.io) is an encrypted, secure, and
|
|
cost\-effective object storage service that enables you to store, back
|
|
up, and archive large amounts of data in a decentralized manner.
|
|
.SS Setup
|
|
.PP
|
|
To make a new Tardigrade configuration you need one of the following: *
|
|
Access Grant that someone else shared with you.
|
|
* API
|
|
Key (https://documentation.tardigrade.io/getting-started/uploading-your-first-object/create-an-api-key)
|
|
of a Tardigrade project you are a member of.
|
|
.PP
|
|
Here is an example of how to make a remote called \f[C]remote\f[R].
|
|
First run:
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone config
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
This will guide you through an interactive setup process:
|
|
.SS Setup with access grant
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
No remotes found \- make a new one
|
|
n) New remote
|
|
s) Set configuration password
|
|
q) Quit config
|
|
n/s/q> n
|
|
name> remote
|
|
Type of storage to configure.
|
|
Enter a string value. Press Enter for the default (\[dq]\[dq]).
|
|
Choose a number from below, or type in your own value
|
|
[snip]
|
|
XX / Tardigrade Decentralized Cloud Storage
|
|
\[rs] \[dq]tardigrade\[dq]
|
|
[snip]
|
|
Storage> tardigrade
|
|
** See help for tardigrade backend at: https://rclone.org/tardigrade/ **
|
|
|
|
Choose an authentication method.
|
|
Enter a string value. Press Enter for the default (\[dq]existing\[dq]).
|
|
Choose a number from below, or type in your own value
|
|
1 / Use an existing access grant.
|
|
\[rs] \[dq]existing\[dq]
|
|
2 / Create a new access grant from satellite address, API key, and passphrase.
|
|
\[rs] \[dq]new\[dq]
|
|
provider> existing
|
|
Access Grant.
|
|
Enter a string value. Press Enter for the default (\[dq]\[dq]).
|
|
access_grant> your\-access\-grant\-received\-by\-someone\-else
|
|
Remote config
|
|
\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-
|
|
[remote]
|
|
type = tardigrade
|
|
access_grant = your\-access\-grant\-received\-by\-someone\-else
|
|
\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-
|
|
y) Yes this is OK (default)
|
|
e) Edit this remote
|
|
d) Delete this remote
|
|
y/e/d> y
|
|
\f[R]
|
|
.fi
|
|
.SS Setup with API key and passhprase
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
No remotes found \- make a new one
|
|
n) New remote
|
|
s) Set configuration password
|
|
q) Quit config
|
|
n/s/q> n
|
|
name> remote
|
|
Type of storage to configure.
|
|
Enter a string value. Press Enter for the default (\[dq]\[dq]).
|
|
Choose a number from below, or type in your own value
|
|
[snip]
|
|
XX / Tardigrade Decentralized Cloud Storage
|
|
\[rs] \[dq]tardigrade\[dq]
|
|
[snip]
|
|
Storage> tardigrade
|
|
** See help for tardigrade backend at: https://rclone.org/tardigrade/ **
|
|
|
|
Choose an authentication method.
|
|
Enter a string value. Press Enter for the default (\[dq]existing\[dq]).
|
|
Choose a number from below, or type in your own value
|
|
1 / Use an existing access grant.
|
|
\[rs] \[dq]existing\[dq]
|
|
2 / Create a new access grant from satellite address, API key, and passphrase.
|
|
\[rs] \[dq]new\[dq]
|
|
provider> new
|
|
Satellite Address. Custom satellite address should match the format: \[ga]<nodeid>\[at]<address>:<port>\[ga].
|
|
Enter a string value. Press Enter for the default (\[dq]us\-central\-1.tardigrade.io\[dq]).
|
|
Choose a number from below, or type in your own value
|
|
1 / US Central 1
|
|
\[rs] \[dq]us\-central\-1.tardigrade.io\[dq]
|
|
2 / Europe West 1
|
|
\[rs] \[dq]europe\-west\-1.tardigrade.io\[dq]
|
|
3 / Asia East 1
|
|
\[rs] \[dq]asia\-east\-1.tardigrade.io\[dq]
|
|
satellite_address> 1
|
|
API Key.
|
|
Enter a string value. Press Enter for the default (\[dq]\[dq]).
|
|
api_key> your\-api\-key\-for\-your\-tardigrade\-project
|
|
Encryption Passphrase. To access existing objects enter passphrase used for uploading.
|
|
Enter a string value. Press Enter for the default (\[dq]\[dq]).
|
|
passphrase> your\-human\-readable\-encryption\-passphrase
|
|
Remote config
|
|
\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-
|
|
[remote]
|
|
type = tardigrade
|
|
satellite_address = 12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S\[at]us\-central\-1.tardigrade.io:7777
|
|
api_key = your\-api\-key\-for\-your\-tardigrade\-project
|
|
passphrase = your\-human\-readable\-encryption\-passphrase
|
|
access_grant = the\-access\-grant\-generated\-from\-the\-api\-key\-and\-passphrase
|
|
\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-
|
|
y) Yes this is OK (default)
|
|
e) Edit this remote
|
|
d) Delete this remote
|
|
y/e/d> y
|
|
\f[R]
|
|
.fi
|
|
.SS Usage
|
|
.PP
|
|
Paths are specified as \f[C]remote:bucket\f[R] (or \f[C]remote:\f[R] for
|
|
the \f[C]lsf\f[R] command.) You may put subdirectories in too, eg
|
|
\f[C]remote:bucket/path/to/dir\f[R].
|
|
.PP
|
|
Once configured you can then use \f[C]rclone\f[R] like this.
|
|
.SS Create a new bucket
|
|
.PP
|
|
Use the \f[C]mkdir\f[R] command to create new bucket,
|
|
e.g.\ \f[C]bucket\f[R].
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone mkdir remote:bucket
|
|
\f[R]
|
|
.fi
|
|
.SS List all buckets
|
|
.PP
|
|
Use the \f[C]lsf\f[R] command to list all buckets.
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone lsf remote:
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
Note the colon (\f[C]:\f[R]) character at the end of the command line.
|
|
.SS Delete a bucket
|
|
.PP
|
|
Use the \f[C]rmdir\f[R] command to delete an empty bucket.
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone rmdir remote:bucket
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
Use the \f[C]purge\f[R] command to delete a non\-empty bucket with all
|
|
its content.
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone purge remote:bucket
|
|
\f[R]
|
|
.fi
|
|
.SS Upload objects
|
|
.PP
|
|
Use the \f[C]copy\f[R] command to upload an object.
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone copy \-\-progress /home/local/directory/file.ext remote:bucket/path/to/dir/
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
The \f[C]\-\-progress\f[R] flag is for displaying progress information.
|
|
Remove it if you don\[cq]t need this information.
|
|
.PP
|
|
Use a folder in the local path to upload all its objects.
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone copy \-\-progress /home/local/directory/ remote:bucket/path/to/dir/
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
Only modified files will be copied.
|
|
.SS List objects
|
|
.PP
|
|
Use the \f[C]ls\f[R] command to list recursively all objects in a
|
|
bucket.
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone ls remote:bucket
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
Add the folder to the remote path to list recursively all objects in
|
|
this folder.
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone ls remote:bucket/path/to/dir/
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
Use the \f[C]lsf\f[R] command to list non\-recursively all objects in a
|
|
bucket or a folder.
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone lsf remote:bucket/path/to/dir/
|
|
\f[R]
|
|
.fi
|
|
.SS Download objects
|
|
.PP
|
|
Use the \f[C]copy\f[R] command to download an object.
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone copy \-\-progress remote:bucket/path/to/dir/file.ext /home/local/directory/
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
The \f[C]\-\-progress\f[R] flag is for displaying progress information.
|
|
Remove it if you don\[cq]t need this information.
|
|
.PP
|
|
Use a folder in the remote path to download all its objects.
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone copy \-\-progress remote:bucket/path/to/dir/ /home/local/directory/
|
|
\f[R]
|
|
.fi
|
|
.SS Delete objects
|
|
.PP
|
|
Use the \f[C]deletefile\f[R] command to delete a single object.
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone deletefile remote:bucket/path/to/dir/file.ext
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
Use the \f[C]delete\f[R] command to delete all object in a folder.
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone delete remote:bucket/path/to/dir/
|
|
\f[R]
|
|
.fi
|
|
.SS Print the total size of objects
|
|
.PP
|
|
Use the \f[C]size\f[R] command to print the total size of objects in a
|
|
bucket or a folder.
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone size remote:bucket/path/to/dir/
|
|
\f[R]
|
|
.fi
|
|
.SS Sync two Locations
|
|
.PP
|
|
Use the \f[C]sync\f[R] command to sync the source to the destination,
|
|
changing the destination only, deleting any excess files.
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone sync \-\-progress /home/local/directory/ remote:bucket/path/to/dir/
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
The \f[C]\-\-progress\f[R] flag is for displaying progress information.
|
|
Remove it if you don\[cq]t need this information.
|
|
.PP
|
|
Since this can cause data loss, test first with the
|
|
\f[C]\-\-dry\-run\f[R] flag to see exactly what would be copied and
|
|
deleted.
|
|
.PP
|
|
The sync can be done also from Tardigrade to the local file system.
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone sync \-\-progress remote:bucket/path/to/dir/ /home/local/directory/
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
Or between two Tardigrade buckets.
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone sync \-\-progress remote\-us:bucket/path/to/dir/ remote\-europe:bucket/path/to/dir/
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
Or even between another cloud storage and Tardigrade.
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone sync \-\-progress s3:bucket/path/to/dir/ tardigrade:bucket/path/to/dir/
|
|
\f[R]
|
|
.fi
|
|
.SS Standard Options
|
|
.PP
|
|
Here are the standard options specific to tardigrade (Tardigrade
|
|
Decentralized Cloud Storage).
|
|
.SS \[en]tardigrade\-provider
|
|
.PP
|
|
Choose an authentication method.
|
|
.IP \[bu] 2
|
|
Config: provider
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_TARDIGRADE_PROVIDER
|
|
.IP \[bu] 2
|
|
Type: string
|
|
.IP \[bu] 2
|
|
Default: \[lq]existing\[rq]
|
|
.IP \[bu] 2
|
|
Examples:
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
\[lq]existing\[rq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Use an existing access grant.
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[lq]new\[rq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Create a new access grant from satellite address, API key, and
|
|
passphrase.
|
|
.RE
|
|
.RE
|
|
.SS \[en]tardigrade\-access\-grant
|
|
.PP
|
|
Access Grant.
|
|
.IP \[bu] 2
|
|
Config: access_grant
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_TARDIGRADE_ACCESS_GRANT
|
|
.IP \[bu] 2
|
|
Type: string
|
|
.IP \[bu] 2
|
|
Default: \[dq]\[dq]
|
|
.SS \[en]tardigrade\-satellite\-address
|
|
.PP
|
|
Satellite Address.
|
|
Custom satellite address should match the format:
|
|
\f[C]<nodeid>\[at]<address>:<port>\f[R].
|
|
.IP \[bu] 2
|
|
Config: satellite_address
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_TARDIGRADE_SATELLITE_ADDRESS
|
|
.IP \[bu] 2
|
|
Type: string
|
|
.IP \[bu] 2
|
|
Default: \[lq]us\-central\-1.tardigrade.io\[rq]
|
|
.IP \[bu] 2
|
|
Examples:
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
\[lq]us\-central\-1.tardigrade.io\[rq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
US Central 1
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[lq]europe\-west\-1.tardigrade.io\[rq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Europe West 1
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[lq]asia\-east\-1.tardigrade.io\[rq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Asia East 1
|
|
.RE
|
|
.RE
|
|
.SS \[en]tardigrade\-api\-key
|
|
.PP
|
|
API Key.
|
|
.IP \[bu] 2
|
|
Config: api_key
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_TARDIGRADE_API_KEY
|
|
.IP \[bu] 2
|
|
Type: string
|
|
.IP \[bu] 2
|
|
Default: \[dq]\[dq]
|
|
.SS \[en]tardigrade\-passphrase
|
|
.PP
|
|
Encryption Passphrase.
|
|
To access existing objects enter passphrase used for uploading.
|
|
.IP \[bu] 2
|
|
Config: passphrase
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_TARDIGRADE_PASSPHRASE
|
|
.IP \[bu] 2
|
|
Type: string
|
|
.IP \[bu] 2
|
|
Default: \[dq]\[dq]
|
|
.SS Union
|
|
.PP
|
|
The \f[C]union\f[R] remote provides a unification similar to UnionFS
|
|
using other remotes.
|
|
.PP
|
|
Paths may be as deep as required or a local path, eg
|
|
\f[C]remote:directory/subdirectory\f[R] or
|
|
\f[C]/directory/subdirectory\f[R].
|
|
.PP
|
|
During the initial setup with \f[C]rclone config\f[R] you will specify
|
|
the upstream remotes as a space separated list.
|
|
The upstream remotes can either be a local paths or other remotes.
|
|
.PP
|
|
Attribute \f[C]:ro\f[R] and \f[C]:nc\f[R] can be attach to the end of
|
|
path to tag the remote as \f[B]read only\f[R] or \f[B]no create\f[R], eg
|
|
\f[C]remote:directory/subdirectory:ro\f[R] or
|
|
\f[C]remote:directory/subdirectory:nc\f[R].
|
|
.PP
|
|
Subfolders can be used in upstream remotes.
|
|
Assume a union remote named \f[C]backup\f[R] with the remotes
|
|
\f[C]mydrive:private/backup\f[R].
|
|
Invoking \f[C]rclone mkdir backup:desktop\f[R] is exactly the same as
|
|
invoking \f[C]rclone mkdir mydrive2:/backup/desktop\f[R].
|
|
.PP
|
|
There will be no special handling of paths containing \f[C]..\f[R]
|
|
segments.
|
|
Invoking \f[C]rclone mkdir backup:../desktop\f[R] is exactly the same as
|
|
invoking \f[C]rclone mkdir mydrive2:/backup/../desktop\f[R].
|
|
.SS Behavior / Policies
|
|
.PP
|
|
The behavior of union backend is inspired by
|
|
trapexit/mergerfs (https://github.com/trapexit/mergerfs).
|
|
All functions are grouped into 3 categories: \f[B]action\f[R],
|
|
\f[B]create\f[R] and \f[B]search\f[R].
|
|
These functions and categories can be assigned a policy which dictates
|
|
what file or directory is chosen when performing that behavior.
|
|
Any policy can be assigned to a function or category though some may not
|
|
be very useful in practice.
|
|
For instance: \f[B]rand\f[R] (random) may be useful for file creation
|
|
(create) but could lead to very odd behavior if used for
|
|
\f[C]delete\f[R] if there were more than one copy of the file.
|
|
.SS Function / Category classifications
|
|
.PP
|
|
.TS
|
|
tab(@);
|
|
lw(5.8n) lw(15.0n) lw(49.2n).
|
|
T{
|
|
Category
|
|
T}@T{
|
|
Description
|
|
T}@T{
|
|
Functions
|
|
T}
|
|
_
|
|
T{
|
|
action
|
|
T}@T{
|
|
Writing Existing file
|
|
T}@T{
|
|
move, rmdir, rmdirs, delete, purge and copy, sync (as destination when
|
|
file exist)
|
|
T}
|
|
T{
|
|
create
|
|
T}@T{
|
|
Create non\-existing file
|
|
T}@T{
|
|
copy, sync (as destination when file not exist)
|
|
T}
|
|
T{
|
|
search
|
|
T}@T{
|
|
Reading and listing file
|
|
T}@T{
|
|
ls, lsd, lsl, cat, md5sum, sha1sum and copy, sync (as source)
|
|
T}
|
|
T{
|
|
N/A
|
|
T}@T{
|
|
T}@T{
|
|
size, about
|
|
T}
|
|
.TE
|
|
.SS Path Preservation
|
|
.PP
|
|
Policies, as described below, are of two basic types.
|
|
\f[C]path preserving\f[R] and \f[C]non\-path preserving\f[R].
|
|
.PP
|
|
All policies which start with \f[C]ep\f[R] (\f[B]epff\f[R],
|
|
\f[B]eplfs\f[R], \f[B]eplus\f[R], \f[B]epmfs\f[R], \f[B]eprand\f[R]) are
|
|
\f[C]path preserving\f[R].
|
|
\f[C]ep\f[R] stands for \f[C]existing path\f[R].
|
|
.PP
|
|
A path preserving policy will only consider upstreams where the relative
|
|
path being accessed already exists.
|
|
.PP
|
|
When using non\-path preserving policies paths will be created in target
|
|
upstreams as necessary.
|
|
.SS Quota Relevant Policies
|
|
.PP
|
|
Some policies rely on quota information.
|
|
These policies should be used only if your upstreams support the
|
|
respective quota fields.
|
|
.PP
|
|
.TS
|
|
tab(@);
|
|
l l.
|
|
T{
|
|
Policy
|
|
T}@T{
|
|
Required Field
|
|
T}
|
|
_
|
|
T{
|
|
lfs, eplfs
|
|
T}@T{
|
|
Free
|
|
T}
|
|
T{
|
|
mfs, epmfs
|
|
T}@T{
|
|
Free
|
|
T}
|
|
T{
|
|
lus, eplus
|
|
T}@T{
|
|
Used
|
|
T}
|
|
T{
|
|
lno, eplno
|
|
T}@T{
|
|
Objects
|
|
T}
|
|
.TE
|
|
.PP
|
|
To check if your upstream supports the field, run
|
|
\f[C]rclone about remote: [flags]\f[R] and see if the required field
|
|
exists.
|
|
.SS Filters
|
|
.PP
|
|
Policies basically search upstream remotes and create a list of files /
|
|
paths for functions to work on.
|
|
The policy is responsible for filtering and sorting.
|
|
The policy type defines the sorting but filtering is mostly uniform as
|
|
described below.
|
|
.IP \[bu] 2
|
|
No \f[B]search\f[R] policies filter.
|
|
.IP \[bu] 2
|
|
All \f[B]action\f[R] policies will filter out remotes which are tagged
|
|
as \f[B]read\-only\f[R].
|
|
.IP \[bu] 2
|
|
All \f[B]create\f[R] policies will filter out remotes which are tagged
|
|
\f[B]read\-only\f[R] or \f[B]no\-create\f[R].
|
|
.PP
|
|
If all remotes are filtered an error will be returned.
|
|
.SS Policy descriptions
|
|
.PP
|
|
The policies definition are inspired by
|
|
trapexit/mergerfs (https://github.com/trapexit/mergerfs) but not exactly
|
|
the same.
|
|
Some policy definition could be different due to the much larger latency
|
|
of remote file systems.
|
|
.PP
|
|
.TS
|
|
tab(@);
|
|
lw(16.2n) lw(53.8n).
|
|
T{
|
|
Policy
|
|
T}@T{
|
|
Description
|
|
T}
|
|
_
|
|
T{
|
|
all
|
|
T}@T{
|
|
Search category: same as \f[B]epall\f[R].
|
|
Action category: same as \f[B]epall\f[R].
|
|
Create category: act on all upstreams.
|
|
T}
|
|
T{
|
|
epall (existing path, all)
|
|
T}@T{
|
|
Search category: Given this order configured, act on the first one found
|
|
where the relative path exists.
|
|
Action category: apply to all found.
|
|
Create category: act on all upstreams where the relative path exists.
|
|
T}
|
|
T{
|
|
epff (existing path, first found)
|
|
T}@T{
|
|
Act on the first one found, by the time upstreams reply, where the
|
|
relative path exists.
|
|
T}
|
|
T{
|
|
eplfs (existing path, least free space)
|
|
T}@T{
|
|
Of all the upstreams on which the relative path exists choose the one
|
|
with the least free space.
|
|
T}
|
|
T{
|
|
eplus (existing path, least used space)
|
|
T}@T{
|
|
Of all the upstreams on which the relative path exists choose the one
|
|
with the least used space.
|
|
T}
|
|
T{
|
|
eplno (existing path, least number of objects)
|
|
T}@T{
|
|
Of all the upstreams on which the relative path exists choose the one
|
|
with the least number of objects.
|
|
T}
|
|
T{
|
|
epmfs (existing path, most free space)
|
|
T}@T{
|
|
Of all the upstreams on which the relative path exists choose the one
|
|
with the most free space.
|
|
T}
|
|
T{
|
|
eprand (existing path, random)
|
|
T}@T{
|
|
Calls \f[B]epall\f[R] and then randomizes.
|
|
Returns only one upstream.
|
|
T}
|
|
T{
|
|
ff (first found)
|
|
T}@T{
|
|
Search category: same as \f[B]epff\f[R].
|
|
Action category: same as \f[B]epff\f[R].
|
|
Create category: Act on the first one found by the time upstreams reply.
|
|
T}
|
|
T{
|
|
lfs (least free space)
|
|
T}@T{
|
|
Search category: same as \f[B]eplfs\f[R].
|
|
Action category: same as \f[B]eplfs\f[R].
|
|
Create category: Pick the upstream with the least available free space.
|
|
T}
|
|
T{
|
|
lus (least used space)
|
|
T}@T{
|
|
Search category: same as \f[B]eplus\f[R].
|
|
Action category: same as \f[B]eplus\f[R].
|
|
Create category: Pick the upstream with the least used space.
|
|
T}
|
|
T{
|
|
lno (least number of objects)
|
|
T}@T{
|
|
Search category: same as \f[B]eplno\f[R].
|
|
Action category: same as \f[B]eplno\f[R].
|
|
Create category: Pick the upstream with the least number of objects.
|
|
T}
|
|
T{
|
|
mfs (most free space)
|
|
T}@T{
|
|
Search category: same as \f[B]epmfs\f[R].
|
|
Action category: same as \f[B]epmfs\f[R].
|
|
Create category: Pick the upstream with the most available free space.
|
|
T}
|
|
T{
|
|
newest
|
|
T}@T{
|
|
Pick the file / directory with the largest mtime.
|
|
T}
|
|
T{
|
|
rand (random)
|
|
T}@T{
|
|
Calls \f[B]all\f[R] and then randomizes.
|
|
Returns only one upstream.
|
|
T}
|
|
.TE
|
|
.SS Setup
|
|
.PP
|
|
Here is an example of how to make a union called \f[C]remote\f[R] for
|
|
local folders.
|
|
First run:
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone config
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
This will guide you through an interactive setup process:
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
No remotes found \- make a new one
|
|
n) New remote
|
|
s) Set configuration password
|
|
q) Quit config
|
|
n/s/q> n
|
|
name> remote
|
|
Type of storage to configure.
|
|
Choose a number from below, or type in your own value
|
|
[snip]
|
|
XX / Union merges the contents of several remotes
|
|
\[rs] \[dq]union\[dq]
|
|
[snip]
|
|
Storage> union
|
|
List of space separated upstreams.
|
|
Can be \[aq]upstreama:test/dir upstreamb:\[aq], \[aq]\[rs]\[dq]upstreama:test/space:ro dir\[rs]\[dq] upstreamb:\[aq], etc.
|
|
Enter a string value. Press Enter for the default (\[dq]\[dq]).
|
|
upstreams>
|
|
Policy to choose upstream on ACTION class.
|
|
Enter a string value. Press Enter for the default (\[dq]epall\[dq]).
|
|
action_policy>
|
|
Policy to choose upstream on CREATE class.
|
|
Enter a string value. Press Enter for the default (\[dq]epmfs\[dq]).
|
|
create_policy>
|
|
Policy to choose upstream on SEARCH class.
|
|
Enter a string value. Press Enter for the default (\[dq]ff\[dq]).
|
|
search_policy>
|
|
Cache time of usage and free space (in seconds). This option is only useful when a path preserving policy is used.
|
|
Enter a signed integer. Press Enter for the default (\[dq]120\[dq]).
|
|
cache_time>
|
|
Remote config
|
|
\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-
|
|
[remote]
|
|
type = union
|
|
upstreams = C:\[rs]dir1 C:\[rs]dir2 C:\[rs]dir3
|
|
\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-
|
|
y) Yes this is OK
|
|
e) Edit this remote
|
|
d) Delete this remote
|
|
y/e/d> y
|
|
Current remotes:
|
|
|
|
Name Type
|
|
==== ====
|
|
remote union
|
|
|
|
e) Edit existing remote
|
|
n) New remote
|
|
d) Delete remote
|
|
r) Rename remote
|
|
c) Copy remote
|
|
s) Set configuration password
|
|
q) Quit config
|
|
e/n/d/r/c/s/q> q
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
Once configured you can then use \f[C]rclone\f[R] like this,
|
|
.PP
|
|
List directories in top level in \f[C]C:\[rs]dir1\f[R],
|
|
\f[C]C:\[rs]dir2\f[R] and \f[C]C:\[rs]dir3\f[R]
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone lsd remote:
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
List all the files in \f[C]C:\[rs]dir1\f[R], \f[C]C:\[rs]dir2\f[R] and
|
|
\f[C]C:\[rs]dir3\f[R]
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone ls remote:
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
Copy another local directory to the union directory called source, which
|
|
will be placed into \f[C]C:\[rs]dir3\f[R]
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone copy C:\[rs]source remote:source
|
|
\f[R]
|
|
.fi
|
|
.SS Standard Options
|
|
.PP
|
|
Here are the standard options specific to union (Union merges the
|
|
contents of several upstream fs).
|
|
.SS \[en]union\-upstreams
|
|
.PP
|
|
List of space separated upstreams.
|
|
Can be `upstreama:test/dir upstreamb:', `\[lq]upstreama:test/space:ro
|
|
dir\[rq] upstreamb:', etc.
|
|
.IP \[bu] 2
|
|
Config: upstreams
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_UNION_UPSTREAMS
|
|
.IP \[bu] 2
|
|
Type: string
|
|
.IP \[bu] 2
|
|
Default: \[dq]\[dq]
|
|
.SS \[en]union\-action\-policy
|
|
.PP
|
|
Policy to choose upstream on ACTION category.
|
|
.IP \[bu] 2
|
|
Config: action_policy
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_UNION_ACTION_POLICY
|
|
.IP \[bu] 2
|
|
Type: string
|
|
.IP \[bu] 2
|
|
Default: \[lq]epall\[rq]
|
|
.SS \[en]union\-create\-policy
|
|
.PP
|
|
Policy to choose upstream on CREATE category.
|
|
.IP \[bu] 2
|
|
Config: create_policy
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_UNION_CREATE_POLICY
|
|
.IP \[bu] 2
|
|
Type: string
|
|
.IP \[bu] 2
|
|
Default: \[lq]epmfs\[rq]
|
|
.SS \[en]union\-search\-policy
|
|
.PP
|
|
Policy to choose upstream on SEARCH category.
|
|
.IP \[bu] 2
|
|
Config: search_policy
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_UNION_SEARCH_POLICY
|
|
.IP \[bu] 2
|
|
Type: string
|
|
.IP \[bu] 2
|
|
Default: \[lq]ff\[rq]
|
|
.SS \[en]union\-cache\-time
|
|
.PP
|
|
Cache time of usage and free space (in seconds).
|
|
This option is only useful when a path preserving policy is used.
|
|
.IP \[bu] 2
|
|
Config: cache_time
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_UNION_CACHE_TIME
|
|
.IP \[bu] 2
|
|
Type: int
|
|
.IP \[bu] 2
|
|
Default: 120
|
|
.SS WebDAV
|
|
.PP
|
|
Paths are specified as \f[C]remote:path\f[R]
|
|
.PP
|
|
Paths may be as deep as required, eg
|
|
\f[C]remote:directory/subdirectory\f[R].
|
|
.PP
|
|
To configure the WebDAV remote you will need to have a URL for it, and a
|
|
username and password.
|
|
If you know what kind of system you are connecting to then rclone can
|
|
enable extra features.
|
|
.PP
|
|
Here is an example of how to make a remote called \f[C]remote\f[R].
|
|
First run:
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone config
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
This will guide you through an interactive setup process:
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
No remotes found \- make a new one
|
|
n) New remote
|
|
s) Set configuration password
|
|
q) Quit config
|
|
n/s/q> n
|
|
name> remote
|
|
Type of storage to configure.
|
|
Choose a number from below, or type in your own value
|
|
[snip]
|
|
XX / Webdav
|
|
\[rs] \[dq]webdav\[dq]
|
|
[snip]
|
|
Storage> webdav
|
|
URL of http host to connect to
|
|
Choose a number from below, or type in your own value
|
|
1 / Connect to example.com
|
|
\[rs] \[dq]https://example.com\[dq]
|
|
url> https://example.com/remote.php/webdav/
|
|
Name of the Webdav site/service/software you are using
|
|
Choose a number from below, or type in your own value
|
|
1 / Nextcloud
|
|
\[rs] \[dq]nextcloud\[dq]
|
|
2 / Owncloud
|
|
\[rs] \[dq]owncloud\[dq]
|
|
3 / Sharepoint
|
|
\[rs] \[dq]sharepoint\[dq]
|
|
4 / Other site/service or software
|
|
\[rs] \[dq]other\[dq]
|
|
vendor> 1
|
|
User name
|
|
user> user
|
|
Password.
|
|
y) Yes type in my own password
|
|
g) Generate random password
|
|
n) No leave this optional password blank
|
|
y/g/n> y
|
|
Enter the password:
|
|
password:
|
|
Confirm the password:
|
|
password:
|
|
Bearer token instead of user/pass (eg a Macaroon)
|
|
bearer_token>
|
|
Remote config
|
|
\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-
|
|
[remote]
|
|
type = webdav
|
|
url = https://example.com/remote.php/webdav/
|
|
vendor = nextcloud
|
|
user = user
|
|
pass = *** ENCRYPTED ***
|
|
bearer_token =
|
|
\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-
|
|
y) Yes this is OK
|
|
e) Edit this remote
|
|
d) Delete this remote
|
|
y/e/d> y
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
Once configured you can then use \f[C]rclone\f[R] like this,
|
|
.PP
|
|
List directories in top level of your WebDAV
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone lsd remote:
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
List all the files in your WebDAV
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone ls remote:
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
To copy a local directory to an WebDAV directory called backup
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone copy /home/source remote:backup
|
|
\f[R]
|
|
.fi
|
|
.SS Modified time and hashes
|
|
.PP
|
|
Plain WebDAV does not support modified times.
|
|
However when used with Owncloud or Nextcloud rclone will support
|
|
modified times.
|
|
.PP
|
|
Likewise plain WebDAV does not support hashes, however when used with
|
|
Owncloud or Nextcloud rclone will support SHA1 and MD5 hashes.
|
|
Depending on the exact version of Owncloud or Nextcloud hashes may
|
|
appear on all objects, or only on objects which had a hash uploaded with
|
|
them.
|
|
.SS Standard Options
|
|
.PP
|
|
Here are the standard options specific to webdav (Webdav).
|
|
.SS \[en]webdav\-url
|
|
.PP
|
|
URL of http host to connect to
|
|
.IP \[bu] 2
|
|
Config: url
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_WEBDAV_URL
|
|
.IP \[bu] 2
|
|
Type: string
|
|
.IP \[bu] 2
|
|
Default: \[dq]\[dq]
|
|
.IP \[bu] 2
|
|
Examples:
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
\[lq]https://example.com\[rq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Connect to example.com
|
|
.RE
|
|
.RE
|
|
.SS \[en]webdav\-vendor
|
|
.PP
|
|
Name of the Webdav site/service/software you are using
|
|
.IP \[bu] 2
|
|
Config: vendor
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_WEBDAV_VENDOR
|
|
.IP \[bu] 2
|
|
Type: string
|
|
.IP \[bu] 2
|
|
Default: \[dq]\[dq]
|
|
.IP \[bu] 2
|
|
Examples:
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
\[lq]nextcloud\[rq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Nextcloud
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[lq]owncloud\[rq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Owncloud
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[lq]sharepoint\[rq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Sharepoint
|
|
.RE
|
|
.IP \[bu] 2
|
|
\[lq]other\[rq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Other site/service or software
|
|
.RE
|
|
.RE
|
|
.SS \[en]webdav\-user
|
|
.PP
|
|
User name
|
|
.IP \[bu] 2
|
|
Config: user
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_WEBDAV_USER
|
|
.IP \[bu] 2
|
|
Type: string
|
|
.IP \[bu] 2
|
|
Default: \[dq]\[dq]
|
|
.SS \[en]webdav\-pass
|
|
.PP
|
|
Password.
|
|
.IP \[bu] 2
|
|
Config: pass
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_WEBDAV_PASS
|
|
.IP \[bu] 2
|
|
Type: string
|
|
.IP \[bu] 2
|
|
Default: \[dq]\[dq]
|
|
.SS \[en]webdav\-bearer\-token
|
|
.PP
|
|
Bearer token instead of user/pass (eg a Macaroon)
|
|
.IP \[bu] 2
|
|
Config: bearer_token
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_WEBDAV_BEARER_TOKEN
|
|
.IP \[bu] 2
|
|
Type: string
|
|
.IP \[bu] 2
|
|
Default: \[dq]\[dq]
|
|
.SS Advanced Options
|
|
.PP
|
|
Here are the advanced options specific to webdav (Webdav).
|
|
.SS \[en]webdav\-bearer\-token\-command
|
|
.PP
|
|
Command to run to get a bearer token
|
|
.IP \[bu] 2
|
|
Config: bearer_token_command
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_WEBDAV_BEARER_TOKEN_COMMAND
|
|
.IP \[bu] 2
|
|
Type: string
|
|
.IP \[bu] 2
|
|
Default: \[dq]\[dq]
|
|
.SS Provider notes
|
|
.PP
|
|
See below for notes on specific providers.
|
|
.SS Owncloud
|
|
.PP
|
|
Click on the settings cog in the bottom right of the page and this will
|
|
show the WebDAV URL that rclone needs in the config step.
|
|
It will look something like
|
|
\f[C]https://example.com/remote.php/webdav/\f[R].
|
|
.PP
|
|
Owncloud supports modified times using the \f[C]X\-OC\-Mtime\f[R]
|
|
header.
|
|
.SS Nextcloud
|
|
.PP
|
|
This is configured in an identical way to Owncloud.
|
|
Note that Nextcloud does not support streaming of files (\f[C]rcat\f[R])
|
|
whereas Owncloud does.
|
|
This may be
|
|
fixed (https://github.com/nextcloud/nextcloud-snap/issues/365) in the
|
|
future.
|
|
.SS Sharepoint
|
|
.PP
|
|
Rclone can be used with Sharepoint provided by OneDrive for Business or
|
|
Office365 Education Accounts.
|
|
This feature is only needed for a few of these Accounts, mostly
|
|
Office365 Education ones.
|
|
These accounts are sometimes not verified by the domain owner
|
|
github#1975 (https://github.com/rclone/rclone/issues/1975)
|
|
.PP
|
|
This means that these accounts can\[cq]t be added using the official API
|
|
(other Accounts should work with the \[lq]onedrive\[rq] option).
|
|
However, it is possible to access them using webdav.
|
|
.PP
|
|
To use a sharepoint remote with rclone, add it like this: First, you
|
|
need to get your remote\[cq]s URL:
|
|
.IP \[bu] 2
|
|
Go here (https://onedrive.live.com/about/en-us/signin/) to open your
|
|
OneDrive or to sign in
|
|
.IP \[bu] 2
|
|
Now take a look at your address bar, the URL should look like this:
|
|
\f[C]https://[YOUR\-DOMAIN]\-my.sharepoint.com/personal/[YOUR\-EMAIL]/_layouts/15/onedrive.aspx\f[R]
|
|
.PP
|
|
You\[cq]ll only need this URL up to the email address.
|
|
After that, you\[cq]ll most likely want to add \[lq]/Documents\[rq].
|
|
That subdirectory contains the actual data stored on your OneDrive.
|
|
.PP
|
|
Add the remote to rclone like this: Configure the \f[C]url\f[R] as
|
|
\f[C]https://[YOUR\-DOMAIN]\-my.sharepoint.com/personal/[YOUR\-EMAIL]/Documents\f[R]
|
|
and use your normal account email and password for \f[C]user\f[R] and
|
|
\f[C]pass\f[R].
|
|
If you have 2FA enabled, you have to generate an app password.
|
|
Set the \f[C]vendor\f[R] to \f[C]sharepoint\f[R].
|
|
.PP
|
|
Your config file should look like this:
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
[sharepoint]
|
|
type = webdav
|
|
url = https://[YOUR\-DOMAIN]\-my.sharepoint.com/personal/[YOUR\-EMAIL]/Documents
|
|
vendor = other
|
|
user = YourEmailAddress
|
|
pass = encryptedpassword
|
|
\f[R]
|
|
.fi
|
|
.SS Required Flags for SharePoint
|
|
.PP
|
|
As SharePoint does some special things with uploaded documents, you
|
|
won\[cq]t be able to use the documents size or the documents hash to
|
|
compare if a file has been changed since the upload / which file is
|
|
newer.
|
|
.PP
|
|
For Rclone calls copying files (especially Office files such as .docx,
|
|
\&.xlsx, etc.) from/to SharePoint (like copy, sync, etc.), you should
|
|
append these flags to ensure Rclone uses the \[lq]Last Modified\[rq]
|
|
datetime property to compare your documents:
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
\-\-ignore\-size \-\-ignore\-checksum \-\-update
|
|
\f[R]
|
|
.fi
|
|
.SS dCache
|
|
.PP
|
|
dCache is a storage system that supports many protocols and
|
|
authentication/authorisation schemes.
|
|
For WebDAV clients, it allows users to authenticate with username and
|
|
password (BASIC), X.509, Kerberos, and various bearer tokens, including
|
|
Macaroons (https://www.dcache.org/manuals/workshop-2017-05-29-Umea/000-Final/anupam_macaroons_v02.pdf)
|
|
and OpenID\-Connect (https://en.wikipedia.org/wiki/OpenID_Connect)
|
|
access tokens.
|
|
.PP
|
|
Configure as normal using the \f[C]other\f[R] type.
|
|
Don\[cq]t enter a username or password, instead enter your Macaroon as
|
|
the \f[C]bearer_token\f[R].
|
|
.PP
|
|
The config will end up looking something like this.
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
[dcache]
|
|
type = webdav
|
|
url = https://dcache...
|
|
vendor = other
|
|
user =
|
|
pass =
|
|
bearer_token = your\-macaroon
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
There is a
|
|
script (https://github.com/sara-nl/GridScripts/blob/master/get-macaroon)
|
|
that obtains a Macaroon from a dCache WebDAV endpoint, and creates an
|
|
rclone config file.
|
|
.PP
|
|
Macaroons may also be obtained from the dCacheView
|
|
web\-browser/JavaScript client that comes with dCache.
|
|
.SS OpenID\-Connect
|
|
.PP
|
|
dCache also supports authenticating with OpenID\-Connect access tokens.
|
|
OpenID\-Connect is a protocol (based on OAuth 2.0) that allows services
|
|
to identify users who have authenticated with some central service.
|
|
.PP
|
|
Support for OpenID\-Connect in rclone is currently achieved using
|
|
another software package called
|
|
oidc\-agent (https://github.com/indigo-dc/oidc-agent).
|
|
This is a command\-line tool that facilitates obtaining an access token.
|
|
Once installed and configured, an access token is obtained by running
|
|
the \f[C]oidc\-token\f[R] command.
|
|
The following example shows a (shortened) access token obtained from the
|
|
\f[I]XDC\f[R] OIDC Provider.
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
paul\[at]celebrimbor:\[ti]$ oidc\-token XDC
|
|
eyJraWQ[...]QFXDt0
|
|
paul\[at]celebrimbor:\[ti]$
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
\f[B]Note\f[R] Before the \f[C]oidc\-token\f[R] command will work, the
|
|
refresh token must be loaded into the oidc agent.
|
|
This is done with the \f[C]oidc\-add\f[R] command (e.g.,
|
|
\f[C]oidc\-add XDC\f[R]).
|
|
This is typically done once per login session.
|
|
Full details on this and how to register oidc\-agent with your OIDC
|
|
Provider are provided in the oidc\-agent
|
|
documentation (https://indigo-dc.gitbooks.io/oidc-agent/).
|
|
.PP
|
|
The rclone \f[C]bearer_token_command\f[R] configuration option is used
|
|
to fetch the access token from oidc\-agent.
|
|
.PP
|
|
Configure as a normal WebDAV endpoint, using the `other' vendor, leaving
|
|
the username and password empty.
|
|
When prompted, choose to edit the advanced config and enter the command
|
|
to get a bearer token (e.g., \f[C]oidc\-agent XDC\f[R]).
|
|
.PP
|
|
The following example config shows a WebDAV endpoint that uses
|
|
oidc\-agent to supply an access token from the \f[I]XDC\f[R] OIDC
|
|
Provider.
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
[dcache]
|
|
type = webdav
|
|
url = https://dcache.example.org/
|
|
vendor = other
|
|
bearer_token_command = oidc\-token XDC
|
|
\f[R]
|
|
.fi
|
|
.SS Yandex Disk
|
|
.PP
|
|
Yandex Disk (https://disk.yandex.com) is a cloud storage solution
|
|
created by Yandex (https://yandex.com).
|
|
.PP
|
|
Here is an example of making a yandex configuration.
|
|
First run
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone config
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
This will guide you through an interactive setup process:
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
No remotes found \- make a new one
|
|
n) New remote
|
|
s) Set configuration password
|
|
n/s> n
|
|
name> remote
|
|
Type of storage to configure.
|
|
Choose a number from below, or type in your own value
|
|
[snip]
|
|
XX / Yandex Disk
|
|
\[rs] \[dq]yandex\[dq]
|
|
[snip]
|
|
Storage> yandex
|
|
Yandex Client Id \- leave blank normally.
|
|
client_id>
|
|
Yandex Client Secret \- leave blank normally.
|
|
client_secret>
|
|
Remote config
|
|
Use auto config?
|
|
* Say Y if not sure
|
|
* Say N if you are working on a remote or headless machine
|
|
y) Yes
|
|
n) No
|
|
y/n> y
|
|
If your browser doesn\[aq]t open automatically go to the following link: http://127.0.0.1:53682/auth
|
|
Log in and authorize rclone for access
|
|
Waiting for code...
|
|
Got code
|
|
\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-
|
|
[remote]
|
|
client_id =
|
|
client_secret =
|
|
token = {\[dq]access_token\[dq]:\[dq]xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx\[dq],\[dq]token_type\[dq]:\[dq]bearer\[dq],\[dq]expiry\[dq]:\[dq]2016\-12\-29T12:27:11.362788025Z\[dq]}
|
|
\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-
|
|
y) Yes this is OK
|
|
e) Edit this remote
|
|
d) Delete this remote
|
|
y/e/d> y
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
See the remote setup docs (https://rclone.org/remote_setup/) for how to
|
|
set it up on a machine with no Internet browser available.
|
|
.PP
|
|
Note that rclone runs a webserver on your local machine to collect the
|
|
token as returned from Yandex Disk.
|
|
This only runs from the moment it opens your browser to the moment you
|
|
get back the verification code.
|
|
This is on \f[C]http://127.0.0.1:53682/\f[R] and this it may require you
|
|
to unblock it temporarily if you are running a host firewall.
|
|
.PP
|
|
Once configured you can then use \f[C]rclone\f[R] like this,
|
|
.PP
|
|
See top level directories
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone lsd remote:
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
Make a new directory
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone mkdir remote:directory
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
List the contents of a directory
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone ls remote:directory
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
Sync \f[C]/home/local/directory\f[R] to the remote path, deleting any
|
|
excess files in the path.
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone sync /home/local/directory remote:directory
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
Yandex paths may be as deep as required, eg
|
|
\f[C]remote:directory/subdirectory\f[R].
|
|
.SS Modified time
|
|
.PP
|
|
Modified times are supported and are stored accurate to 1 ns in custom
|
|
metadata called \f[C]rclone_modified\f[R] in RFC3339 with nanoseconds
|
|
format.
|
|
.SS MD5 checksums
|
|
.PP
|
|
MD5 checksums are natively supported by Yandex Disk.
|
|
.SS Emptying Trash
|
|
.PP
|
|
If you wish to empty your trash you can use the
|
|
\f[C]rclone cleanup remote:\f[R] command which will permanently delete
|
|
all your trashed files.
|
|
This command does not take any path arguments.
|
|
.SS Quota information
|
|
.PP
|
|
To view your current quota you can use the
|
|
\f[C]rclone about remote:\f[R] command which will display your usage
|
|
limit (quota) and the current usage.
|
|
.SS Restricted filename characters
|
|
.PP
|
|
The default restricted characters
|
|
set (https://rclone.org/overview/#restricted-characters) are replaced.
|
|
.PP
|
|
Invalid UTF\-8 bytes will also be
|
|
replaced (https://rclone.org/overview/#invalid-utf8), as they can\[cq]t
|
|
be used in JSON strings.
|
|
.SS Limitations
|
|
.PP
|
|
When uploading very large files (bigger than about 5GB) you will need to
|
|
increase the \f[C]\-\-timeout\f[R] parameter.
|
|
This is because Yandex pauses (perhaps to calculate the MD5SUM for the
|
|
entire file) before returning confirmation that the file has been
|
|
uploaded.
|
|
The default handling of timeouts in rclone is to assume a 5 minute pause
|
|
is an error and close the connection \- you\[cq]ll see
|
|
\f[C]net/http: timeout awaiting response headers\f[R] errors in the logs
|
|
if this is happening.
|
|
Setting the timeout to twice the max size of file in GB should be
|
|
enough, so if you want to upload a 30GB file set a timeout of
|
|
\f[C]2 * 30 = 60m\f[R], that is \f[C]\-\-timeout 60m\f[R].
|
|
.SS Standard Options
|
|
.PP
|
|
Here are the standard options specific to yandex (Yandex Disk).
|
|
.SS \[en]yandex\-client\-id
|
|
.PP
|
|
Yandex Client Id Leave blank normally.
|
|
.IP \[bu] 2
|
|
Config: client_id
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_YANDEX_CLIENT_ID
|
|
.IP \[bu] 2
|
|
Type: string
|
|
.IP \[bu] 2
|
|
Default: \[dq]\[dq]
|
|
.SS \[en]yandex\-client\-secret
|
|
.PP
|
|
Yandex Client Secret Leave blank normally.
|
|
.IP \[bu] 2
|
|
Config: client_secret
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_YANDEX_CLIENT_SECRET
|
|
.IP \[bu] 2
|
|
Type: string
|
|
.IP \[bu] 2
|
|
Default: \[dq]\[dq]
|
|
.SS Advanced Options
|
|
.PP
|
|
Here are the advanced options specific to yandex (Yandex Disk).
|
|
.SS \[en]yandex\-unlink
|
|
.PP
|
|
Remove existing public link to file/folder with link command rather than
|
|
creating.
|
|
Default is false, meaning link command will create or retrieve public
|
|
link.
|
|
.IP \[bu] 2
|
|
Config: unlink
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_YANDEX_UNLINK
|
|
.IP \[bu] 2
|
|
Type: bool
|
|
.IP \[bu] 2
|
|
Default: false
|
|
.SS \[en]yandex\-encoding
|
|
.PP
|
|
This sets the encoding for the backend.
|
|
.PP
|
|
See: the encoding section in the
|
|
overview (https://rclone.org/overview/#encoding) for more info.
|
|
.IP \[bu] 2
|
|
Config: encoding
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_YANDEX_ENCODING
|
|
.IP \[bu] 2
|
|
Type: MultiEncoder
|
|
.IP \[bu] 2
|
|
Default: Slash,Del,Ctl,InvalidUtf8,Dot
|
|
.SS Local Filesystem
|
|
.PP
|
|
Local paths are specified as normal filesystem paths, eg
|
|
\f[C]/path/to/wherever\f[R], so
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone sync /home/source /tmp/destination
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
Will sync \f[C]/home/source\f[R] to \f[C]/tmp/destination\f[R]
|
|
.PP
|
|
These can be configured into the config file for consistencies sake, but
|
|
it is probably easier not to.
|
|
.SS Modified time
|
|
.PP
|
|
Rclone reads and writes the modified time using an accuracy determined
|
|
by the OS.
|
|
Typically this is 1ns on Linux, 10 ns on Windows and 1 Second on OS X.
|
|
.SS Filenames
|
|
.PP
|
|
Filenames should be encoded in UTF\-8 on disk.
|
|
This is the normal case for Windows and OS X.
|
|
.PP
|
|
There is a bit more uncertainty in the Linux world, but new
|
|
distributions will have UTF\-8 encoded files names.
|
|
If you are using an old Linux filesystem with non UTF\-8 file names (eg
|
|
latin1) then you can use the \f[C]convmv\f[R] tool to convert the
|
|
filesystem to UTF\-8.
|
|
This tool is available in most distributions\[cq] package managers.
|
|
.PP
|
|
If an invalid (non\-UTF8) filename is read, the invalid characters will
|
|
be replaced with a quoted representation of the invalid bytes.
|
|
The name \f[C]gro\[rs]xdf\f[R] will be transferred as
|
|
\f[C]gro\[u201B]DF\f[R].
|
|
\f[C]rclone\f[R] will emit a debug message in this case (use
|
|
\f[C]\-v\f[R] to see), eg
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
Local file system at .: Replacing invalid UTF\-8 characters in \[dq]gro\[rs]xdf\[dq]
|
|
\f[R]
|
|
.fi
|
|
.SS Restricted characters
|
|
.PP
|
|
On non Windows platforms the following characters are replaced when
|
|
handling file names.
|
|
.PP
|
|
.TS
|
|
tab(@);
|
|
l c c.
|
|
T{
|
|
Character
|
|
T}@T{
|
|
Value
|
|
T}@T{
|
|
Replacement
|
|
T}
|
|
_
|
|
T{
|
|
NUL
|
|
T}@T{
|
|
0x00
|
|
T}@T{
|
|
\[u2400]
|
|
T}
|
|
T{
|
|
/
|
|
T}@T{
|
|
0x2F
|
|
T}@T{
|
|
\[uFF0F]
|
|
T}
|
|
.TE
|
|
.PP
|
|
When running on Windows the following characters are replaced.
|
|
This list is based on the Windows file naming
|
|
conventions (https://docs.microsoft.com/de-de/windows/desktop/FileIO/naming-a-file#naming-conventions).
|
|
.PP
|
|
.TS
|
|
tab(@);
|
|
l c c.
|
|
T{
|
|
Character
|
|
T}@T{
|
|
Value
|
|
T}@T{
|
|
Replacement
|
|
T}
|
|
_
|
|
T{
|
|
NUL
|
|
T}@T{
|
|
0x00
|
|
T}@T{
|
|
\[u2400]
|
|
T}
|
|
T{
|
|
SOH
|
|
T}@T{
|
|
0x01
|
|
T}@T{
|
|
\[u2401]
|
|
T}
|
|
T{
|
|
STX
|
|
T}@T{
|
|
0x02
|
|
T}@T{
|
|
\[u2402]
|
|
T}
|
|
T{
|
|
ETX
|
|
T}@T{
|
|
0x03
|
|
T}@T{
|
|
\[u2403]
|
|
T}
|
|
T{
|
|
EOT
|
|
T}@T{
|
|
0x04
|
|
T}@T{
|
|
\[u2404]
|
|
T}
|
|
T{
|
|
ENQ
|
|
T}@T{
|
|
0x05
|
|
T}@T{
|
|
\[u2405]
|
|
T}
|
|
T{
|
|
ACK
|
|
T}@T{
|
|
0x06
|
|
T}@T{
|
|
\[u2406]
|
|
T}
|
|
T{
|
|
BEL
|
|
T}@T{
|
|
0x07
|
|
T}@T{
|
|
\[u2407]
|
|
T}
|
|
T{
|
|
BS
|
|
T}@T{
|
|
0x08
|
|
T}@T{
|
|
\[u2408]
|
|
T}
|
|
T{
|
|
HT
|
|
T}@T{
|
|
0x09
|
|
T}@T{
|
|
\[u2409]
|
|
T}
|
|
T{
|
|
LF
|
|
T}@T{
|
|
0x0A
|
|
T}@T{
|
|
\[u240A]
|
|
T}
|
|
T{
|
|
VT
|
|
T}@T{
|
|
0x0B
|
|
T}@T{
|
|
\[u240B]
|
|
T}
|
|
T{
|
|
FF
|
|
T}@T{
|
|
0x0C
|
|
T}@T{
|
|
\[u240C]
|
|
T}
|
|
T{
|
|
CR
|
|
T}@T{
|
|
0x0D
|
|
T}@T{
|
|
\[u240D]
|
|
T}
|
|
T{
|
|
SO
|
|
T}@T{
|
|
0x0E
|
|
T}@T{
|
|
\[u240E]
|
|
T}
|
|
T{
|
|
SI
|
|
T}@T{
|
|
0x0F
|
|
T}@T{
|
|
\[u240F]
|
|
T}
|
|
T{
|
|
DLE
|
|
T}@T{
|
|
0x10
|
|
T}@T{
|
|
\[u2410]
|
|
T}
|
|
T{
|
|
DC1
|
|
T}@T{
|
|
0x11
|
|
T}@T{
|
|
\[u2411]
|
|
T}
|
|
T{
|
|
DC2
|
|
T}@T{
|
|
0x12
|
|
T}@T{
|
|
\[u2412]
|
|
T}
|
|
T{
|
|
DC3
|
|
T}@T{
|
|
0x13
|
|
T}@T{
|
|
\[u2413]
|
|
T}
|
|
T{
|
|
DC4
|
|
T}@T{
|
|
0x14
|
|
T}@T{
|
|
\[u2414]
|
|
T}
|
|
T{
|
|
NAK
|
|
T}@T{
|
|
0x15
|
|
T}@T{
|
|
\[u2415]
|
|
T}
|
|
T{
|
|
SYN
|
|
T}@T{
|
|
0x16
|
|
T}@T{
|
|
\[u2416]
|
|
T}
|
|
T{
|
|
ETB
|
|
T}@T{
|
|
0x17
|
|
T}@T{
|
|
\[u2417]
|
|
T}
|
|
T{
|
|
CAN
|
|
T}@T{
|
|
0x18
|
|
T}@T{
|
|
\[u2418]
|
|
T}
|
|
T{
|
|
EM
|
|
T}@T{
|
|
0x19
|
|
T}@T{
|
|
\[u2419]
|
|
T}
|
|
T{
|
|
SUB
|
|
T}@T{
|
|
0x1A
|
|
T}@T{
|
|
\[u241A]
|
|
T}
|
|
T{
|
|
ESC
|
|
T}@T{
|
|
0x1B
|
|
T}@T{
|
|
\[u241B]
|
|
T}
|
|
T{
|
|
FS
|
|
T}@T{
|
|
0x1C
|
|
T}@T{
|
|
\[u241C]
|
|
T}
|
|
T{
|
|
GS
|
|
T}@T{
|
|
0x1D
|
|
T}@T{
|
|
\[u241D]
|
|
T}
|
|
T{
|
|
RS
|
|
T}@T{
|
|
0x1E
|
|
T}@T{
|
|
\[u241E]
|
|
T}
|
|
T{
|
|
US
|
|
T}@T{
|
|
0x1F
|
|
T}@T{
|
|
\[u241F]
|
|
T}
|
|
T{
|
|
/
|
|
T}@T{
|
|
0x2F
|
|
T}@T{
|
|
\[uFF0F]
|
|
T}
|
|
T{
|
|
\[dq]
|
|
T}@T{
|
|
0x22
|
|
T}@T{
|
|
\[uFF02]
|
|
T}
|
|
T{
|
|
*
|
|
T}@T{
|
|
0x2A
|
|
T}@T{
|
|
\[uFF0A]
|
|
T}
|
|
T{
|
|
:
|
|
T}@T{
|
|
0x3A
|
|
T}@T{
|
|
\[uFF1A]
|
|
T}
|
|
T{
|
|
<
|
|
T}@T{
|
|
0x3C
|
|
T}@T{
|
|
\[uFF1C]
|
|
T}
|
|
T{
|
|
>
|
|
T}@T{
|
|
0x3E
|
|
T}@T{
|
|
\[uFF1E]
|
|
T}
|
|
T{
|
|
?
|
|
T}@T{
|
|
0x3F
|
|
T}@T{
|
|
\[uFF1F]
|
|
T}
|
|
T{
|
|
\[rs]
|
|
T}@T{
|
|
0x5C
|
|
T}@T{
|
|
\[uFF3C]
|
|
T}
|
|
T{
|
|
|
|
|
T}@T{
|
|
0x7C
|
|
T}@T{
|
|
\[uFF5C]
|
|
T}
|
|
.TE
|
|
.PP
|
|
File names on Windows can also not end with the following characters.
|
|
These only get replaced if they are the last character in the name:
|
|
.PP
|
|
.TS
|
|
tab(@);
|
|
l c c.
|
|
T{
|
|
Character
|
|
T}@T{
|
|
Value
|
|
T}@T{
|
|
Replacement
|
|
T}
|
|
_
|
|
T{
|
|
SP
|
|
T}@T{
|
|
0x20
|
|
T}@T{
|
|
\[u2420]
|
|
T}
|
|
T{
|
|
\&.
|
|
T}@T{
|
|
0x2E
|
|
T}@T{
|
|
\[uFF0E]
|
|
T}
|
|
.TE
|
|
.PP
|
|
Invalid UTF\-8 bytes will also be
|
|
replaced (https://rclone.org/overview/#invalid-utf8), as they can\[cq]t
|
|
be converted to UTF\-16.
|
|
.SS Long paths on Windows
|
|
.PP
|
|
Rclone handles long paths automatically, by converting all paths to long
|
|
UNC
|
|
paths (https://msdn.microsoft.com/en-us/library/windows/desktop/aa365247(v=vs.85).aspx#maxpath)
|
|
which allows paths up to 32,767 characters.
|
|
.PP
|
|
This is why you will see that your paths, for instance
|
|
\f[C]c:\[rs]files\f[R] is converted to the UNC path
|
|
\f[C]\[rs]\[rs]?\[rs]c:\[rs]files\f[R] in the output, and
|
|
\f[C]\[rs]\[rs]server\[rs]share\f[R] is converted to
|
|
\f[C]\[rs]\[rs]?\[rs]UNC\[rs]server\[rs]share\f[R].
|
|
.PP
|
|
However, in rare cases this may cause problems with buggy file system
|
|
drivers like EncFS (https://github.com/rclone/rclone/issues/261).
|
|
To disable UNC conversion globally, add this to your
|
|
\f[C].rclone.conf\f[R] file:
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
[local]
|
|
nounc = true
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
If you want to selectively disable UNC, you can add it to a separate
|
|
entry like this:
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
[nounc]
|
|
type = local
|
|
nounc = true
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
And use rclone like this:
|
|
.PP
|
|
\f[C]rclone copy c:\[rs]src nounc:z:\[rs]dst\f[R]
|
|
.PP
|
|
This will use UNC paths on \f[C]c:\[rs]src\f[R] but not on
|
|
\f[C]z:\[rs]dst\f[R].
|
|
Of course this will cause problems if the absolute path length of a file
|
|
exceeds 258 characters on z, so only use this option if you have to.
|
|
.SS Symlinks / Junction points
|
|
.PP
|
|
Normally rclone will ignore symlinks or junction points (which behave
|
|
like symlinks under Windows).
|
|
.PP
|
|
If you supply \f[C]\-\-copy\-links\f[R] or \f[C]\-L\f[R] then rclone
|
|
will follow the symlink and copy the pointed to file or directory.
|
|
Note that this flag is incompatible with \f[C]\-links\f[R] /
|
|
\f[C]\-l\f[R].
|
|
.PP
|
|
This flag applies to all commands.
|
|
.PP
|
|
For example, supposing you have a directory structure like this
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
$ tree /tmp/a
|
|
/tmp/a
|
|
\[u251C]\[u2500]\[u2500] b \-> ../b
|
|
\[u251C]\[u2500]\[u2500] expected \-> ../expected
|
|
\[u251C]\[u2500]\[u2500] one
|
|
\[u2514]\[u2500]\[u2500] two
|
|
\[u2514]\[u2500]\[u2500] three
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
Then you can see the difference with and without the flag like this
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
$ rclone ls /tmp/a
|
|
6 one
|
|
6 two/three
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
and
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
$ rclone \-L ls /tmp/a
|
|
4174 expected
|
|
6 one
|
|
6 two/three
|
|
6 b/two
|
|
6 b/one
|
|
\f[R]
|
|
.fi
|
|
.SS \[en]links, \-l
|
|
.PP
|
|
Normally rclone will ignore symlinks or junction points (which behave
|
|
like symlinks under Windows).
|
|
.PP
|
|
If you supply this flag then rclone will copy symbolic links from the
|
|
local storage, and store them as text files, with a `.rclonelink' suffix
|
|
in the remote storage.
|
|
.PP
|
|
The text file will contain the target of the symbolic link (see
|
|
example).
|
|
.PP
|
|
This flag applies to all commands.
|
|
.PP
|
|
For example, supposing you have a directory structure like this
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
$ tree /tmp/a
|
|
/tmp/a
|
|
\[u251C]\[u2500]\[u2500] file1 \-> ./file4
|
|
\[u2514]\[u2500]\[u2500] file2 \-> /home/user/file3
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
Copying the entire directory with `\-l'
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
$ rclone copyto \-l /tmp/a/file1 remote:/tmp/a/
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
The remote files are created with a `.rclonelink' suffix
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
$ rclone ls remote:/tmp/a
|
|
5 file1.rclonelink
|
|
14 file2.rclonelink
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
The remote files will contain the target of the symbolic links
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
$ rclone cat remote:/tmp/a/file1.rclonelink
|
|
\&./file4
|
|
|
|
$ rclone cat remote:/tmp/a/file2.rclonelink
|
|
/home/user/file3
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
Copying them back with `\-l'
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
$ rclone copyto \-l remote:/tmp/a/ /tmp/b/
|
|
|
|
$ tree /tmp/b
|
|
/tmp/b
|
|
\[u251C]\[u2500]\[u2500] file1 \-> ./file4
|
|
\[u2514]\[u2500]\[u2500] file2 \-> /home/user/file3
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
However, if copied back without `\-l'
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
$ rclone copyto remote:/tmp/a/ /tmp/b/
|
|
|
|
$ tree /tmp/b
|
|
/tmp/b
|
|
\[u251C]\[u2500]\[u2500] file1.rclonelink
|
|
\[u2514]\[u2500]\[u2500] file2.rclonelink
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
Note that this flag is incompatible with \f[C]\-copy\-links\f[R] /
|
|
\f[C]\-L\f[R].
|
|
.SS Restricting filesystems with \[en]one\-file\-system
|
|
.PP
|
|
Normally rclone will recurse through filesystems as mounted.
|
|
.PP
|
|
However if you set \f[C]\-\-one\-file\-system\f[R] or \f[C]\-x\f[R] this
|
|
tells rclone to stay in the filesystem specified by the root and not to
|
|
recurse into different file systems.
|
|
.PP
|
|
For example if you have a directory hierarchy like this
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
root
|
|
\[u251C]\[u2500]\[u2500] disk1 \- disk1 mounted on the root
|
|
\[br]\ \ \[u2514]\[u2500]\[u2500] file3 \- stored on disk1
|
|
\[u251C]\[u2500]\[u2500] disk2 \- disk2 mounted on the root
|
|
\[br]\ \ \[u2514]\[u2500]\[u2500] file4 \- stored on disk12
|
|
\[u251C]\[u2500]\[u2500] file1 \- stored on the root disk
|
|
\[u2514]\[u2500]\[u2500] file2 \- stored on the root disk
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
Using \f[C]rclone \-\-one\-file\-system copy root remote:\f[R] will only
|
|
copy \f[C]file1\f[R] and \f[C]file2\f[R].
|
|
Eg
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
$ rclone \-q \-\-one\-file\-system ls root
|
|
0 file1
|
|
0 file2
|
|
\f[R]
|
|
.fi
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
$ rclone \-q ls root
|
|
0 disk1/file3
|
|
0 disk2/file4
|
|
0 file1
|
|
0 file2
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
\f[B]NB\f[R] Rclone (like most unix tools such as \f[C]du\f[R],
|
|
\f[C]rsync\f[R] and \f[C]tar\f[R]) treats a bind mount to the same
|
|
device as being on the same filesystem.
|
|
.PP
|
|
\f[B]NB\f[R] This flag is only available on Unix based systems.
|
|
On systems where it isn\[cq]t supported (eg Windows) it will be ignored.
|
|
.SS Standard Options
|
|
.PP
|
|
Here are the standard options specific to local (Local Disk).
|
|
.SS \[en]local\-nounc
|
|
.PP
|
|
Disable UNC (long path names) conversion on Windows
|
|
.IP \[bu] 2
|
|
Config: nounc
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_LOCAL_NOUNC
|
|
.IP \[bu] 2
|
|
Type: string
|
|
.IP \[bu] 2
|
|
Default: \[dq]\[dq]
|
|
.IP \[bu] 2
|
|
Examples:
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
\[lq]true\[rq]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Disables long file names
|
|
.RE
|
|
.RE
|
|
.SS Advanced Options
|
|
.PP
|
|
Here are the advanced options specific to local (Local Disk).
|
|
.SS \[en]copy\-links / \-L
|
|
.PP
|
|
Follow symlinks and copy the pointed to item.
|
|
.IP \[bu] 2
|
|
Config: copy_links
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_LOCAL_COPY_LINKS
|
|
.IP \[bu] 2
|
|
Type: bool
|
|
.IP \[bu] 2
|
|
Default: false
|
|
.SS \[en]links / \-l
|
|
.PP
|
|
Translate symlinks to/from regular files with a `.rclonelink' extension
|
|
.IP \[bu] 2
|
|
Config: links
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_LOCAL_LINKS
|
|
.IP \[bu] 2
|
|
Type: bool
|
|
.IP \[bu] 2
|
|
Default: false
|
|
.SS \[en]skip\-links
|
|
.PP
|
|
Don\[cq]t warn about skipped symlinks.
|
|
This flag disables warning messages on skipped symlinks or junction
|
|
points, as you explicitly acknowledge that they should be skipped.
|
|
.IP \[bu] 2
|
|
Config: skip_links
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_LOCAL_SKIP_LINKS
|
|
.IP \[bu] 2
|
|
Type: bool
|
|
.IP \[bu] 2
|
|
Default: false
|
|
.SS \[en]local\-no\-unicode\-normalization
|
|
.PP
|
|
Don\[cq]t apply unicode normalization to paths and filenames
|
|
(Deprecated)
|
|
.PP
|
|
This flag is deprecated now.
|
|
Rclone no longer normalizes unicode file names, but it compares them
|
|
with unicode normalization in the sync routine instead.
|
|
.IP \[bu] 2
|
|
Config: no_unicode_normalization
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_LOCAL_NO_UNICODE_NORMALIZATION
|
|
.IP \[bu] 2
|
|
Type: bool
|
|
.IP \[bu] 2
|
|
Default: false
|
|
.SS \[en]local\-no\-check\-updated
|
|
.PP
|
|
Don\[cq]t check to see if the files change during upload
|
|
.PP
|
|
Normally rclone checks the size and modification time of files as they
|
|
are being uploaded and aborts with a message which starts \[lq]can\[cq]t
|
|
copy \- source file is being updated\[rq] if the file changes during
|
|
upload.
|
|
.PP
|
|
However on some file systems this modification time check may fail (eg
|
|
Glusterfs #2206 (https://github.com/rclone/rclone/issues/2206)) so this
|
|
check can be disabled with this flag.
|
|
.IP \[bu] 2
|
|
Config: no_check_updated
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_LOCAL_NO_CHECK_UPDATED
|
|
.IP \[bu] 2
|
|
Type: bool
|
|
.IP \[bu] 2
|
|
Default: false
|
|
.SS \[en]one\-file\-system / \-x
|
|
.PP
|
|
Don\[cq]t cross filesystem boundaries (unix/macOS only).
|
|
.IP \[bu] 2
|
|
Config: one_file_system
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_LOCAL_ONE_FILE_SYSTEM
|
|
.IP \[bu] 2
|
|
Type: bool
|
|
.IP \[bu] 2
|
|
Default: false
|
|
.SS \[en]local\-case\-sensitive
|
|
.PP
|
|
Force the filesystem to report itself as case sensitive.
|
|
.PP
|
|
Normally the local backend declares itself as case insensitive on
|
|
Windows/macOS and case sensitive for everything else.
|
|
Use this flag to override the default choice.
|
|
.IP \[bu] 2
|
|
Config: case_sensitive
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_LOCAL_CASE_SENSITIVE
|
|
.IP \[bu] 2
|
|
Type: bool
|
|
.IP \[bu] 2
|
|
Default: false
|
|
.SS \[en]local\-case\-insensitive
|
|
.PP
|
|
Force the filesystem to report itself as case insensitive
|
|
.PP
|
|
Normally the local backend declares itself as case insensitive on
|
|
Windows/macOS and case sensitive for everything else.
|
|
Use this flag to override the default choice.
|
|
.IP \[bu] 2
|
|
Config: case_insensitive
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_LOCAL_CASE_INSENSITIVE
|
|
.IP \[bu] 2
|
|
Type: bool
|
|
.IP \[bu] 2
|
|
Default: false
|
|
.SS \[en]local\-no\-sparse
|
|
.PP
|
|
Disable sparse files for multi\-thread downloads
|
|
.PP
|
|
On Windows platforms rclone will make sparse files when doing
|
|
multi\-thread downloads.
|
|
This avoids long pauses on large files where the OS zeros the file.
|
|
However sparse files may be undesirable as they cause disk fragmentation
|
|
and can be slow to work with.
|
|
.IP \[bu] 2
|
|
Config: no_sparse
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_LOCAL_NO_SPARSE
|
|
.IP \[bu] 2
|
|
Type: bool
|
|
.IP \[bu] 2
|
|
Default: false
|
|
.SS \[en]local\-encoding
|
|
.PP
|
|
This sets the encoding for the backend.
|
|
.PP
|
|
See: the encoding section in the
|
|
overview (https://rclone.org/overview/#encoding) for more info.
|
|
.IP \[bu] 2
|
|
Config: encoding
|
|
.IP \[bu] 2
|
|
Env Var: RCLONE_LOCAL_ENCODING
|
|
.IP \[bu] 2
|
|
Type: MultiEncoder
|
|
.IP \[bu] 2
|
|
Default: Slash,Dot
|
|
.SS Backend commands
|
|
.PP
|
|
Here are the commands specific to the local backend.
|
|
.PP
|
|
Run them with
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone backend COMMAND remote:
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
The help below will explain what arguments each command takes.
|
|
.PP
|
|
See the \[lq]rclone backend\[rq]
|
|
command (https://rclone.org/commands/rclone_backend/) for more info on
|
|
how to pass options and arguments.
|
|
.PP
|
|
These can be run on a running backend using the rc command
|
|
backend/command (https://rclone.org/rc/#backend/command).
|
|
.SS noop
|
|
.PP
|
|
A null operation for testing backend commands
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone backend noop remote: [options] [<arguments>+]
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
This is a test command which has some options you can try to change the
|
|
output.
|
|
.PP
|
|
Options:
|
|
.IP \[bu] 2
|
|
\[lq]echo\[rq]: echo the input arguments
|
|
.IP \[bu] 2
|
|
\[lq]error\[rq]: return an error based on option value
|
|
.SH Changelog
|
|
.SS v1.52.0 \- 2020\-05\-27
|
|
.PP
|
|
Special thanks to Martin Michlmayr for proof reading and correcting all
|
|
the docs and Edward Barker for helping re\-write the front page.
|
|
.PP
|
|
See commits (https://github.com/rclone/rclone/compare/v1.51.0...v1.52.0)
|
|
.IP \[bu] 2
|
|
New backends
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Tardigrade (https://rclone.org/tardigrade/) backend for use with
|
|
storj.io (Caleb Case)
|
|
.IP \[bu] 2
|
|
Union (https://rclone.org/union/) re\-write to have multiple writable
|
|
remotes (Max Sum)
|
|
.IP \[bu] 2
|
|
Seafile (/seafile) for Seafile server (Fred \[at]creativeprojects)
|
|
.RE
|
|
.IP \[bu] 2
|
|
New commands
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
backend: command for backend specific commands (see backends) (Nick
|
|
Craig\-Wood)
|
|
.IP \[bu] 2
|
|
cachestats: Deprecate in favour of \f[C]rclone backend stats cache:\f[R]
|
|
(Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
dbhashsum: Deprecate in favour of \f[C]rclone hashsum DropboxHash\f[R]
|
|
(Nick Craig\-Wood)
|
|
.RE
|
|
.IP \[bu] 2
|
|
New Features
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Add \f[C]\-\-header\-download\f[R] and \f[C]\-\-header\-upload\f[R]
|
|
flags for setting HTTP headers when uploading/downloading (Tim Gallant)
|
|
.IP \[bu] 2
|
|
Add \f[C]\-\-header\f[R] flag to add HTTP headers to every HTTP
|
|
transaction (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Add \f[C]\-\-check\-first\f[R] to do all checking before starting
|
|
transfers (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Add \f[C]\-\-track\-renames\-strategy\f[R] for configurable matching
|
|
criteria for \f[C]\-\-track\-renames\f[R] (Bernd Schoolmann)
|
|
.IP \[bu] 2
|
|
Add \f[C]\-\-cutoff\-mode\f[R] hard,soft,catious (Shing Kit Chan &
|
|
Franklyn Tackitt)
|
|
.IP \[bu] 2
|
|
Filter flags (eg \f[C]\-\-files\-from \-\f[R]) can read from stdin
|
|
(fishbullet)
|
|
.IP \[bu] 2
|
|
Add \f[C]\-\-error\-on\-no\-transfer\f[R] option (Jon Fautley)
|
|
.IP \[bu] 2
|
|
Implement \f[C]\-\-order\-by xxx,mixed\f[R] for copying some small and
|
|
some big files (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Allow \f[C]\-\-max\-backlog\f[R] to be negative meaning as large as
|
|
possible (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Added \f[C]\-\-no\-unicode\-normalization\f[R] flag to allow Unicode
|
|
filenames to remain unique (Ben Zenker)
|
|
.IP \[bu] 2
|
|
Allow \f[C]\-\-min\-age\f[R]/\f[C]\-\-max\-age\f[R] to take a date as
|
|
well as a duration (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Add rename statistics for file and directory renames (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Add statistics output to JSON log (reddi)
|
|
.IP \[bu] 2
|
|
Make stats be printed on non\-zero exit code (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
When running \f[C]\-\-password\-command\f[R] allow use of stdin
|
|
(S\['e]bastien Gross)
|
|
.IP \[bu] 2
|
|
Stop empty strings being a valid remote path (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
accounting: support WriterTo for less memory copying (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
build
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Update to use go1.14 for the build (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Add \f[C]\-trimpath\f[R] to release build for reproduceable builds (Nick
|
|
Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Remove GOOS and GOARCH from Dockerfile (Brandon Philips)
|
|
.RE
|
|
.IP \[bu] 2
|
|
config
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Fsync the config file after writing to save more reliably (Nick
|
|
Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Add \f[C]\-\-obscure\f[R] and \f[C]\-\-no\-obscure\f[R] flags to
|
|
\f[C]config create\f[R]/\f[C]update\f[R] (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Make \f[C]config show\f[R] take \f[C]remote:\f[R] as well as
|
|
\f[C]remote\f[R] (Nick Craig\-Wood)
|
|
.RE
|
|
.IP \[bu] 2
|
|
copyurl: Add \f[C]\-\-no\-clobber\f[R] flag (Denis)
|
|
.IP \[bu] 2
|
|
delete: Added \f[C]\-\-rmdirs\f[R] flag to delete directories as well
|
|
(Kush)
|
|
.IP \[bu] 2
|
|
filter: Added \f[C]\-\-files\-from\-raw\f[R] flag (Ankur Gupta)
|
|
.IP \[bu] 2
|
|
genautocomplete: Add support for fish shell (Matan Rosenberg)
|
|
.IP \[bu] 2
|
|
log: Add support for syslog LOCAL facilities (Patryk Jakuszew)
|
|
.IP \[bu] 2
|
|
lsjson: Add \f[C]\-\-hash\-type\f[R] parameter and use it in lsf to
|
|
speed up hashing (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
rc
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Add \f[C]\-o\f[R]/\f[C]\-\-opt\f[R] and \f[C]\-a\f[R]/\f[C]\-\-arg\f[R]
|
|
for more structured input (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Implement \f[C]backend/command\f[R] for running backend specific
|
|
commands remotely (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Add \f[C]mount/mount\f[R] command for starting \f[C]rclone mount\f[R]
|
|
via the API (Chaitanya)
|
|
.RE
|
|
.IP \[bu] 2
|
|
rcd: Add Prometheus metrics support (Gary Kim)
|
|
.IP \[bu] 2
|
|
serve http
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Added a \f[C]\-\-template\f[R] flag for user defined markup (calistri)
|
|
.IP \[bu] 2
|
|
Add Last\-Modified headers to files and directories (Nick Craig\-Wood)
|
|
.RE
|
|
.IP \[bu] 2
|
|
serve sftp: Add support for multiple host keys by repeating
|
|
\f[C]\-\-key\f[R] flag (Maxime Suret)
|
|
.IP \[bu] 2
|
|
touch: Add \f[C]\-\-localtime\f[R] flag to make \f[C]\-\-timestamp\f[R]
|
|
localtime not UTC (Nick Craig\-Wood)
|
|
.RE
|
|
.IP \[bu] 2
|
|
Bug Fixes
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
accounting
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Restore \[lq]Max number of stats groups reached\[rq] log line
|
|
(Micha\[/l] Matczuk)
|
|
.IP \[bu] 2
|
|
Correct exitcode on Transfer Limit Exceeded flag.
|
|
(Anuar Serdaliyev)
|
|
.IP \[bu] 2
|
|
Reset bytes read during copy retry (Ankur Gupta)
|
|
.IP \[bu] 2
|
|
Fix race clearing stats (Nick Craig\-Wood)
|
|
.RE
|
|
.IP \[bu] 2
|
|
copy: Only create empty directories when they don\[cq]t exist on the
|
|
remote (Ishuah Kariuki)
|
|
.IP \[bu] 2
|
|
dedupe: Stop dedupe deleting files with identical IDs (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
oauth
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Use custom http client so that \f[C]\-\-no\-check\-certificate\f[R] is
|
|
honored by oauth token fetch (Mark Spieth)
|
|
.IP \[bu] 2
|
|
Replace deprecated oauth2.NoContext (Lars Lehtonen)
|
|
.RE
|
|
.IP \[bu] 2
|
|
operations
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Fix setting the timestamp on Windows for multithread copy (Nick
|
|
Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Make rcat obey \f[C]\-\-ignore\-checksum\f[R] (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Make \f[C]\-\-max\-transfer\f[R] more accurate (Nick Craig\-Wood)
|
|
.RE
|
|
.IP \[bu] 2
|
|
rc
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Fix dropped error (Lars Lehtonen)
|
|
.IP \[bu] 2
|
|
Fix misplaced http server config (Xiaoxing Ye)
|
|
.IP \[bu] 2
|
|
Disable duplicate log (ElonH)
|
|
.RE
|
|
.IP \[bu] 2
|
|
serve dlna
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Cds: don\[cq]t specify childCount at all when unknown (Dan Walters)
|
|
.IP \[bu] 2
|
|
Cds: use modification time as date in dlna metadata (Dan Walters)
|
|
.RE
|
|
.IP \[bu] 2
|
|
serve restic: Fix tests after restic project removed vendoring (Nick
|
|
Craig\-Wood)
|
|
.IP \[bu] 2
|
|
sync
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Fix incorrect \[lq]nothing to transfer\[rq] message using
|
|
\f[C]\-\-delete\-before\f[R] (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Only create empty directories when they don\[cq]t exist on the remote
|
|
(Ishuah Kariuki)
|
|
.RE
|
|
.RE
|
|
.IP \[bu] 2
|
|
Mount
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Add \f[C]\-\-async\-read\f[R] flag to disable asynchronous reads (Nick
|
|
Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Ignore \f[C]\-\-allow\-root\f[R] flag with a warning as it has been
|
|
removed upstream (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Warn if \f[C]\-\-allow\-non\-empty\f[R] used on Windows and clarify docs
|
|
(Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Constrain to go1.13 or above otherwise bazil.org/fuse fails to compile
|
|
(Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Fix fail because of too long volume name (evileye)
|
|
.IP \[bu] 2
|
|
Report 1PB free for unknown disk sizes (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Map more rclone errors into file systems errors (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Fix disappearing cwd problem (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Use ReaddirPlus on Windows to improve directory listing performance
|
|
(Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Send a hint as to whether the filesystem is case insensitive or not
|
|
(Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Add rc command \f[C]mount/types\f[R] (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Change maximum leaf name length to 1024 bytes (Nick Craig\-Wood)
|
|
.RE
|
|
.IP \[bu] 2
|
|
VFS
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Add \f[C]\-\-vfs\-read\-wait\f[R] and \f[C]\-\-vfs\-write\-wait\f[R]
|
|
flags to control time waiting for a sequential read/write (Nick
|
|
Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Change default \f[C]\-\-vfs\-read\-wait\f[R] to 20ms (it was 5ms and not
|
|
configurable) (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Make \f[C]df\f[R] output more consistent on a rclone mount.
|
|
(Yves G)
|
|
.IP \[bu] 2
|
|
Report 1PB free for unknown disk sizes (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Fix race condition caused by unlocked reading of Dir.path (Nick
|
|
Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Make File lock and Dir lock not overlap to avoid deadlock (Nick
|
|
Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Implement lock ordering between File and Dir to eliminate deadlocks
|
|
(Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Factor the vfs cache into its own package (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Pin the Fs in use in the Fs cache (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Add SetSys() methods to Node to allow caching stuff on a node (Nick
|
|
Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Ignore file not found errors from Hash in Read.Release (Nick
|
|
Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Fix hang in read wait code (Nick Craig\-Wood)
|
|
.RE
|
|
.IP \[bu] 2
|
|
Local
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Speed up multi thread downloads by using sparse files on Windows (Nick
|
|
Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Implement \f[C]\-\-local\-no\-sparse\f[R] flag for disabling sparse
|
|
files (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Implement \f[C]rclone backend noop\f[R] for testing purposes (Nick
|
|
Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Fix \[lq]file not found\[rq] errors on post transfer Hash calculation
|
|
(Nick Craig\-Wood)
|
|
.RE
|
|
.IP \[bu] 2
|
|
Cache
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Implement \f[C]rclone backend stats\f[R] command (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Fix Server Side Copy with Temp Upload (Brandon McNama)
|
|
.IP \[bu] 2
|
|
Remove Unused Functions (Lars Lehtonen)
|
|
.IP \[bu] 2
|
|
Disable race tests until bbolt is fixed (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Move methods used for testing into test file (greatroar)
|
|
.IP \[bu] 2
|
|
Add Pin and Unpin and canonicalised lookup (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Use proper import path go.etcd.io/bbolt (Robert\-Andr\['e] Mauchin)
|
|
.RE
|
|
.IP \[bu] 2
|
|
Crypt
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Calculate hashes for uploads from local disk (Nick Craig\-Wood)
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
This allows crypted Jottacloud uploads without using local disk
|
|
.IP \[bu] 2
|
|
This means crypted s3/b2 uploads will now have hashes
|
|
.RE
|
|
.IP \[bu] 2
|
|
Added \f[C]rclone backend decode\f[R]/\f[C]encode\f[R] commands to
|
|
replicate functionality of \f[C]cryptdecode\f[R] (Anagh Kumar Baranwal)
|
|
.IP \[bu] 2
|
|
Get rid of the unused Cipher interface as it obfuscated the code (Nick
|
|
Craig\-Wood)
|
|
.RE
|
|
.IP \[bu] 2
|
|
Azure Blob
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Implement streaming of unknown sized files so \f[C]rcat\f[R] is now
|
|
supported (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Implement memory pooling to control memory use (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Add \f[C]\-\-azureblob\-disable\-checksum\f[R] flag (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Retry \f[C]InvalidBlobOrBlock\f[R] error as it may indicate block
|
|
concurrency problems (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Remove unused \f[C]Object.parseTimeString()\f[R] (Lars Lehtonen)
|
|
.IP \[bu] 2
|
|
Fix permission error on SAS URL limited to container (Nick Craig\-Wood)
|
|
.RE
|
|
.IP \[bu] 2
|
|
B2
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Add support for \f[C]\-\-header\-upload\f[R] and
|
|
\f[C]\-\-header\-download\f[R] (Tim Gallant)
|
|
.IP \[bu] 2
|
|
Ignore directory markers at the root also (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Force the case of the SHA1 to lowercase (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Remove unused \f[C]largeUpload.clearUploadURL()\f[R] (Lars Lehtonen)
|
|
.RE
|
|
.IP \[bu] 2
|
|
Box
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Add support for \f[C]\-\-header\-upload\f[R] and
|
|
\f[C]\-\-header\-download\f[R] (Tim Gallant)
|
|
.IP \[bu] 2
|
|
Implement About to read size used (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Add token renew function for jwt auth (David Bramwell)
|
|
.IP \[bu] 2
|
|
Added support for interchangeable root folder for Box backend (Sunil
|
|
Patra)
|
|
.IP \[bu] 2
|
|
Remove unnecessary iat from jws claims (David)
|
|
.RE
|
|
.IP \[bu] 2
|
|
Drive
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Follow shortcuts by default, skip with
|
|
\f[C]\-\-drive\-skip\-shortcuts\f[R] (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Implement \f[C]rclone backend shortcut\f[R] command for creating
|
|
shortcuts (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Added \f[C]rclone backend\f[R] command to change
|
|
\f[C]service_account_file\f[R] and \f[C]chunk_size\f[R] (Anagh Kumar
|
|
Baranwal)
|
|
.IP \[bu] 2
|
|
Fix missing files when using \f[C]\-\-fast\-list\f[R] and
|
|
\f[C]\-\-drive\-shared\-with\-me\f[R] (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Fix duplicate items when using \f[C]\-\-drive\-shared\-with\-me\f[R]
|
|
(Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Extend \f[C]\-\-drive\-stop\-on\-upload\-limit\f[R] to respond to
|
|
\f[C]teamDriveFileLimitExceeded\f[R].
|
|
(harry)
|
|
.IP \[bu] 2
|
|
Don\[cq]t delete files with multiple parents to avoid data loss (Nick
|
|
Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Server side copy docs use default description if empty (Nick
|
|
Craig\-Wood)
|
|
.RE
|
|
.IP \[bu] 2
|
|
Dropbox
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Make error insufficient space to be fatal (harry)
|
|
.IP \[bu] 2
|
|
Add info about required redirect url (Elan Ruusam\[:a]e)
|
|
.RE
|
|
.IP \[bu] 2
|
|
Fichier
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Add support for \f[C]\-\-header\-upload\f[R] and
|
|
\f[C]\-\-header\-download\f[R] (Tim Gallant)
|
|
.IP \[bu] 2
|
|
Implement custom pacer to deal with the new rate limiting (buengese)
|
|
.RE
|
|
.IP \[bu] 2
|
|
FTP
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Fix lockup when using concurrency limit on failed connections (Nick
|
|
Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Fix lockup on failed upload when using concurrency limit (Nick
|
|
Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Fix lockup on Close failures when using concurrency limit (Nick
|
|
Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Work around pureftp sending spurious 150 messages (Nick Craig\-Wood)
|
|
.RE
|
|
.IP \[bu] 2
|
|
Google Cloud Storage
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Add support for \f[C]\-\-header\-upload\f[R] and
|
|
\f[C]\-\-header\-download\f[R] (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Add \f[C]ARCHIVE\f[R] storage class to help (Adam Stroud)
|
|
.IP \[bu] 2
|
|
Ignore directory markers at the root (Nick Craig\-Wood)
|
|
.RE
|
|
.IP \[bu] 2
|
|
Googlephotos
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Make the start year configurable (Daven)
|
|
.IP \[bu] 2
|
|
Add support for \f[C]\-\-header\-upload\f[R] and
|
|
\f[C]\-\-header\-download\f[R] (Tim Gallant)
|
|
.IP \[bu] 2
|
|
Create feature/favorites directory (Brandon Philips)
|
|
.IP \[bu] 2
|
|
Fix \[lq]concurrent map write\[rq] error (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Don\[cq]t put an image in error message (Nick Craig\-Wood)
|
|
.RE
|
|
.IP \[bu] 2
|
|
HTTP
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Improved directory listing with new template from Caddy project
|
|
(calisro)
|
|
.RE
|
|
.IP \[bu] 2
|
|
Jottacloud
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Implement \f[C]\-\-jottacloud\-trashed\-only\f[R] (buengese)
|
|
.IP \[bu] 2
|
|
Add support for \f[C]\-\-header\-upload\f[R] and
|
|
\f[C]\-\-header\-download\f[R] (Tim Gallant)
|
|
.IP \[bu] 2
|
|
Use \f[C]RawURLEncoding\f[R] when decoding base64 encoded login token
|
|
(buengese)
|
|
.IP \[bu] 2
|
|
Implement cleanup (buengese)
|
|
.IP \[bu] 2
|
|
Update docs regarding cleanup, removed remains from old auth, and added
|
|
warning about special mountpoints.
|
|
(albertony)
|
|
.RE
|
|
.IP \[bu] 2
|
|
Mailru
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Describe 2FA requirements (valery1707)
|
|
.RE
|
|
.IP \[bu] 2
|
|
Onedrive
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Implement \f[C]\-\-onedrive\-server\-side\-across\-configs\f[R] (Nick
|
|
Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Add support for \f[C]\-\-header\-upload\f[R] and
|
|
\f[C]\-\-header\-download\f[R] (Tim Gallant)
|
|
.IP \[bu] 2
|
|
Fix occasional 416 errors on multipart uploads (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Added maximum chunk size limit warning in the docs (Harry)
|
|
.IP \[bu] 2
|
|
Fix missing drive on config (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Make error \f[C]quotaLimitReached\f[R] to be fatal (harry)
|
|
.RE
|
|
.IP \[bu] 2
|
|
Opendrive
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Add support for \f[C]\-\-header\-upload\f[R] and
|
|
\f[C]\-\-header\-download\f[R] (Tim Gallant)
|
|
.RE
|
|
.IP \[bu] 2
|
|
Pcloud
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Added support for interchangeable root folder for pCloud backend (Sunil
|
|
Patra)
|
|
.IP \[bu] 2
|
|
Add support for \f[C]\-\-header\-upload\f[R] and
|
|
\f[C]\-\-header\-download\f[R] (Tim Gallant)
|
|
.IP \[bu] 2
|
|
Fix initial config \[lq]Auth state doesn\[cq]t match\[rq] message (Nick
|
|
Craig\-Wood)
|
|
.RE
|
|
.IP \[bu] 2
|
|
Premiumizeme
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Add support for \f[C]\-\-header\-upload\f[R] and
|
|
\f[C]\-\-header\-download\f[R] (Tim Gallant)
|
|
.IP \[bu] 2
|
|
Prune unused functions (Lars Lehtonen)
|
|
.RE
|
|
.IP \[bu] 2
|
|
Putio
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Add support for \f[C]\-\-header\-upload\f[R] and
|
|
\f[C]\-\-header\-download\f[R] (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Make downloading files use the rclone http Client (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Fix parsing of remotes with leading and trailing / (Nick Craig\-Wood)
|
|
.RE
|
|
.IP \[bu] 2
|
|
Qingstor
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Make \f[C]rclone cleanup\f[R] remove pending multipart uploads older
|
|
than 24h (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Try harder to cancel failed multipart uploads (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Prune \f[C]multiUploader.list()\f[R] (Lars Lehtonen)
|
|
.IP \[bu] 2
|
|
Lint fix (Lars Lehtonen)
|
|
.RE
|
|
.IP \[bu] 2
|
|
S3
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Add support for \f[C]\-\-header\-upload\f[R] and
|
|
\f[C]\-\-header\-download\f[R] (Tim Gallant)
|
|
.IP \[bu] 2
|
|
Use memory pool for buffer allocations (Maciej Zimnoch)
|
|
.IP \[bu] 2
|
|
Add SSE\-C support for AWS, Ceph, and MinIO (Jack Anderson)
|
|
.IP \[bu] 2
|
|
Fail fast multipart upload (Micha\[/l] Matczuk)
|
|
.IP \[bu] 2
|
|
Report errors on bucket creation (mkdir) correctly (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Specify that Minio supports URL encoding in listings (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Added 500 as retryErrorCode (Micha\[/l] Matczuk)
|
|
.IP \[bu] 2
|
|
Use \f[C]\-\-low\-level\-retries\f[R] as the number of SDK retries
|
|
(Aleksandar Jankovi\['c])
|
|
.IP \[bu] 2
|
|
Fix multipart abort context (Aleksandar Jankovic)
|
|
.IP \[bu] 2
|
|
Replace deprecated \f[C]session.New()\f[R] with
|
|
\f[C]session.NewSession()\f[R] (Lars Lehtonen)
|
|
.IP \[bu] 2
|
|
Use the provided size parameter when allocating a new memory pool
|
|
(Joachim Brandon LeBlanc)
|
|
.IP \[bu] 2
|
|
Use rclone\[cq]s low level retries instead of AWS SDK to fix listing
|
|
retries (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Ignore directory markers at the root also (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Use single memory pool (Micha\[/l] Matczuk)
|
|
.IP \[bu] 2
|
|
Do not resize buf on put to memBuf (Micha\[/l] Matczuk)
|
|
.IP \[bu] 2
|
|
Improve docs for \f[C]\-\-s3\-disable\-checksum\f[R] (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Don\[cq]t leak memory or tokens in edge cases for multipart upload (Nick
|
|
Craig\-Wood)
|
|
.RE
|
|
.IP \[bu] 2
|
|
Seafile
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Implement 2FA (Fred)
|
|
.RE
|
|
.IP \[bu] 2
|
|
SFTP
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Added \f[C]\-\-sftp\-pem\-key\f[R] to support inline key files (calisro)
|
|
.IP \[bu] 2
|
|
Fix post transfer copies failing with 0 size when using
|
|
\f[C]set_modtime=false\f[R] (Nick Craig\-Wood)
|
|
.RE
|
|
.IP \[bu] 2
|
|
Sharefile
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Add support for \f[C]\-\-header\-upload\f[R] and
|
|
\f[C]\-\-header\-download\f[R] (Tim Gallant)
|
|
.RE
|
|
.IP \[bu] 2
|
|
Sugarsync
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Add support for \f[C]\-\-header\-upload\f[R] and
|
|
\f[C]\-\-header\-download\f[R] (Tim Gallant)
|
|
.RE
|
|
.IP \[bu] 2
|
|
Swift
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Add support for \f[C]\-\-header\-upload\f[R] and
|
|
\f[C]\-\-header\-download\f[R] (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Fix cosmetic issue in error message (Martin Michlmayr)
|
|
.RE
|
|
.IP \[bu] 2
|
|
Union
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Implement multiple writable remotes (Max Sum)
|
|
.IP \[bu] 2
|
|
Fix server\-side copy (Max Sum)
|
|
.IP \[bu] 2
|
|
Implement ListR (Max Sum)
|
|
.IP \[bu] 2
|
|
Enable ListR when upstreams contain local (Max Sum)
|
|
.RE
|
|
.IP \[bu] 2
|
|
WebDAV
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Add support for \f[C]\-\-header\-upload\f[R] and
|
|
\f[C]\-\-header\-download\f[R] (Tim Gallant)
|
|
.IP \[bu] 2
|
|
Fix \f[C]X\-OC\-Mtime\f[R] header for Transip compatibility (Nick
|
|
Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Report full and consistent usage with \f[C]about\f[R] (Yves G)
|
|
.RE
|
|
.IP \[bu] 2
|
|
Yandex
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Add support for \f[C]\-\-header\-upload\f[R] and
|
|
\f[C]\-\-header\-download\f[R] (Tim Gallant)
|
|
.RE
|
|
.SS v1.51.0 \- 2020\-02\-01
|
|
.IP \[bu] 2
|
|
New backends
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Memory (https://rclone.org/memory/) (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Sugarsync (https://rclone.org/sugarsync/) (Nick Craig\-Wood)
|
|
.RE
|
|
.IP \[bu] 2
|
|
New Features
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Adjust all backends to have \f[C]\-\-backend\-encoding\f[R] parameter
|
|
(Nick Craig\-Wood)
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
this enables the encoding for special characters to be adjusted or
|
|
disabled
|
|
.RE
|
|
.IP \[bu] 2
|
|
Add \f[C]\-\-max\-duration\f[R] flag to control the maximum duration of
|
|
a transfer session (boosh)
|
|
.IP \[bu] 2
|
|
Add \f[C]\-\-expect\-continue\-timeout\f[R] flag, default 1s (Nick
|
|
Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Add \f[C]\-\-no\-check\-dest\f[R] flag for copying without testing the
|
|
destination (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Implement \f[C]\-\-order\-by\f[R] flag to order transfers (Nick
|
|
Craig\-Wood)
|
|
.IP \[bu] 2
|
|
accounting
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Don\[cq]t show entries in both transferring and checking (Nick
|
|
Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Add option to delete stats (Aleksandar Jankovic)
|
|
.RE
|
|
.IP \[bu] 2
|
|
build
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Compress the test builds with gzip (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Implement a framework for starting test servers during tests (Nick
|
|
Craig\-Wood)
|
|
.RE
|
|
.IP \[bu] 2
|
|
cmd: Always print elapsed time to tenth place seconds in progress (Gary
|
|
Kim)
|
|
.IP \[bu] 2
|
|
config
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Add \f[C]\-\-password\-command\f[R] to allow dynamic config password
|
|
(Damon Permezel)
|
|
.IP \[bu] 2
|
|
Give config questions default values (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Check a remote exists when creating a new one (Nick Craig\-Wood)
|
|
.RE
|
|
.IP \[bu] 2
|
|
copyurl: Add \f[C]\-\-stdout\f[R] flag to write to stdout (Nick
|
|
Craig\-Wood)
|
|
.IP \[bu] 2
|
|
dedupe: Implement keep smallest too (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
hashsum: Add flag \f[C]\-\-base64\f[R] flag (landall)
|
|
.IP \[bu] 2
|
|
lsf: Speed up on s3/swift/etc by not reading mimetype by default (Nick
|
|
Craig\-Wood)
|
|
.IP \[bu] 2
|
|
lsjson: Add \f[C]\-\-no\-mimetype\f[R] flag (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
rc: Add methods to turn on blocking and mutex profiling (Nick
|
|
Craig\-Wood)
|
|
.IP \[bu] 2
|
|
rcd
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Adding group parameter to stats (Chaitanya)
|
|
.IP \[bu] 2
|
|
Move webgui apart; option to disable browser (Xiaoxing Ye)
|
|
.RE
|
|
.IP \[bu] 2
|
|
serve sftp: Add support for public key with auth proxy (Paul Tinsley)
|
|
.IP \[bu] 2
|
|
stats: Show deletes in stats and hide zero stats (anuar45)
|
|
.RE
|
|
.IP \[bu] 2
|
|
Bug Fixes
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
accounting
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Fix error counter counting multiple times (Ankur Gupta)
|
|
.IP \[bu] 2
|
|
Fix error count shown as checks (Cnly)
|
|
.IP \[bu] 2
|
|
Clear finished transfer in stats\-reset (Maciej Zimnoch)
|
|
.IP \[bu] 2
|
|
Added StatsInfo locking in statsGroups sum function (Micha\[/l] Matczuk)
|
|
.RE
|
|
.IP \[bu] 2
|
|
asyncreader: Fix EOF error (buengese)
|
|
.IP \[bu] 2
|
|
check: Fix \f[C]\-\-one\-way\f[R] recursing more directories than it
|
|
needs to (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
chunkedreader: Disable hash calculation for first segment (Nick
|
|
Craig\-Wood)
|
|
.IP \[bu] 2
|
|
config
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Do not open browser on headless on drive/gcs/google photos (Xiaoxing Ye)
|
|
.IP \[bu] 2
|
|
SetValueAndSave ignore error if config section does not exist yet
|
|
(buengese)
|
|
.RE
|
|
.IP \[bu] 2
|
|
cmd: Fix completion with an encrypted config (Danil Semelenov)
|
|
.IP \[bu] 2
|
|
dbhashsum: Stop it returning UNSUPPORTED on dropbox (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
dedupe: Add missing modes to help string (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
operations
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Fix dedupe continuing on errors like insufficientFilePermisson
|
|
(SezalAgrawal)
|
|
.IP \[bu] 2
|
|
Clear accounting before low level retry (Maciej Zimnoch)
|
|
.IP \[bu] 2
|
|
Write debug message when hashes could not be checked (Ole Sch\[:u]tt)
|
|
.IP \[bu] 2
|
|
Move interface assertion to tests to remove pflag dependency (Nick
|
|
Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Make NewOverrideObjectInfo public and factor uses (Nick Craig\-Wood)
|
|
.RE
|
|
.IP \[bu] 2
|
|
proxy: Replace use of bcrypt with sha256 (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
vendor
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Update bazil.org/fuse to fix FreeBSD 12.1 (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Update github.com/t3rm1n4l/go\-mega to fix mega \[lq]illegal base64 data
|
|
at input byte 22\[rq] (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Update termbox\-go to fix ncdu command on FreeBSD (Kuang\-che Wu)
|
|
.IP \[bu] 2
|
|
Update t3rm1n4l/go\-mega \- fixes mega: couldn\[cq]t login: crypto/aes:
|
|
invalid key size 0 (Nick Craig\-Wood)
|
|
.RE
|
|
.RE
|
|
.IP \[bu] 2
|
|
Mount
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Enable async reads for a 20% speedup (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Replace use of WriteAt with Write for cache mode >= writes and O_APPEND
|
|
(Brett Dutro)
|
|
.IP \[bu] 2
|
|
Make sure we call unmount when exiting (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Don\[cq]t build on go1.10 as bazil/fuse no longer supports it (Nick
|
|
Craig\-Wood)
|
|
.IP \[bu] 2
|
|
When setting dates discard out of range dates (Nick Craig\-Wood)
|
|
.RE
|
|
.IP \[bu] 2
|
|
VFS
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Add a newly created file straight into the directory (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Only calculate one hash for reads for a speedup (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Make ReadAt for non cached files work better with non\-sequential reads
|
|
(Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Fix edge cases when reading ModTime from file (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Make sure existing files opened for write show correct size (Nick
|
|
Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Don\[cq]t cache the path in RW file objects to fix renaming (Nick
|
|
Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Fix rename of open files when using the VFS cache (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
When renaming files in the cache, rename the cache item in memory too
|
|
(Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Fix open file renaming on drive when using
|
|
\f[C]\-\-vfs\-cache\-mode writes\f[R] (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Fix incorrect modtime for mv into mount with
|
|
\f[C]\-\-vfs\-cache\-modes writes\f[R] (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
On rename, rename in cache too if the file exists (Anagh Kumar Baranwal)
|
|
.RE
|
|
.IP \[bu] 2
|
|
Local
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Make source file being updated errors be NoLowLevelRetry errors (Nick
|
|
Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Fix update of hidden files on Windows (Nick Craig\-Wood)
|
|
.RE
|
|
.IP \[bu] 2
|
|
Cache
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Follow move of upstream library github.com/coreos/bbolt
|
|
github.com/etcd\-io/bbolt (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Fix \f[C]fatal error: concurrent map writes\f[R] (Nick Craig\-Wood)
|
|
.RE
|
|
.IP \[bu] 2
|
|
Crypt
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Reorder the filename encryption options (Thomas Eales)
|
|
.IP \[bu] 2
|
|
Correctly handle trailing dot (buengese)
|
|
.RE
|
|
.IP \[bu] 2
|
|
Chunker
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Reduce length of temporary suffix (Ivan Andreev)
|
|
.RE
|
|
.IP \[bu] 2
|
|
Drive
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Add \f[C]\-\-drive\-stop\-on\-upload\-limit\f[R] flag to stop syncs when
|
|
upload limit reached (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Add \f[C]\-\-drive\-use\-shared\-date\f[R] to use date file was shared
|
|
instead of modified date (Garry McNulty)
|
|
.IP \[bu] 2
|
|
Make sure invalid auth for teamdrives always reports an error (Nick
|
|
Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Fix \f[C]\-\-fast\-list\f[R] when using appDataFolder (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Use multipart resumable uploads for streaming and uploads in mount (Nick
|
|
Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Log an ERROR if an incomplete search is returned (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Hide dangerous config from the configurator (Nick Craig\-Wood)
|
|
.RE
|
|
.IP \[bu] 2
|
|
Dropbox
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Treat \f[C]insufficient_space\f[R] errors as non retriable errors (Nick
|
|
Craig\-Wood)
|
|
.RE
|
|
.IP \[bu] 2
|
|
Jottacloud
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Use new auth method used by official client (buengese)
|
|
.IP \[bu] 2
|
|
Add URL to generate Login Token to config wizard (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Add support whitelabel versions (buengese)
|
|
.RE
|
|
.IP \[bu] 2
|
|
Koofr
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Use rclone HTTP client.
|
|
(jaKa)
|
|
.RE
|
|
.IP \[bu] 2
|
|
Onedrive
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Add Sites.Read.All permission (Benjamin Richter)
|
|
.IP \[bu] 2
|
|
Add support \[lq]Retry\-After\[rq] header (Motonori IWAMURO)
|
|
.RE
|
|
.IP \[bu] 2
|
|
Opendrive
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Implement \f[C]\-\-opendrive\-chunk\-size\f[R] (Nick Craig\-Wood)
|
|
.RE
|
|
.IP \[bu] 2
|
|
S3
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Re\-implement multipart upload to fix memory issues (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Add \f[C]\-\-s3\-copy\-cutoff\f[R] for size to switch to multipart copy
|
|
(Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Add new region Asia Patific (Hong Kong) (Outvi V)
|
|
.IP \[bu] 2
|
|
Reduce memory usage streaming files by reducing max stream upload size
|
|
(Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Add \f[C]\-\-s3\-list\-chunk\f[R] option for bucket listing (Thomas
|
|
Kriechbaumer)
|
|
.IP \[bu] 2
|
|
Force path style bucket access to off for AWS deprecation (Nick
|
|
Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Use AWS web identity role provider if available (Tennix)
|
|
.IP \[bu] 2
|
|
Add StackPath Object Storage Support (Dave Koston)
|
|
.IP \[bu] 2
|
|
Fix ExpiryWindow value (Aleksandar Jankovic)
|
|
.IP \[bu] 2
|
|
Fix DisableChecksum condition (Aleksandar Jankovi\['c])
|
|
.IP \[bu] 2
|
|
Fix URL decoding of NextMarker (Nick Craig\-Wood)
|
|
.RE
|
|
.IP \[bu] 2
|
|
SFTP
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Add \f[C]\-\-sftp\-skip\-links\f[R] to skip symlinks and non regular
|
|
files (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Retry Creation of Connection (Sebastian Brandt)
|
|
.IP \[bu] 2
|
|
Fix \[lq]failed to parse private key file: ssh: not an encrypted
|
|
key\[rq] error (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Open files for update write only to fix AWS SFTP interop (Nick
|
|
Craig\-Wood)
|
|
.RE
|
|
.IP \[bu] 2
|
|
Swift
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Reserve segments of dynamic large object when delete objects in
|
|
container what was enabled versioning.
|
|
(Nguy\[u1EC5]n H\[u1EEF]u Lu\[^a]n)
|
|
.IP \[bu] 2
|
|
Fix parsing of X\-Object\-Manifest (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Update OVH API endpoint (unbelauscht)
|
|
.RE
|
|
.IP \[bu] 2
|
|
WebDAV
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Make nextcloud only upload SHA1 checksums (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Fix case of \[lq]Bearer\[rq] in Authorization: header to agree with RFC
|
|
(Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Add Referer header to fix problems with WAFs (Nick Craig\-Wood)
|
|
.RE
|
|
.SS v1.50.2 \- 2019\-11\-19
|
|
.IP \[bu] 2
|
|
Bug Fixes
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
accounting: Fix memory leak on retries operations (Nick Craig\-Wood)
|
|
.RE
|
|
.IP \[bu] 2
|
|
Drive
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Fix listing of the root directory with drive.files scope (Nick
|
|
Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Fix \[en]drive\-root\-folder\-id with team/shared drives (Nick
|
|
Craig\-Wood)
|
|
.RE
|
|
.SS v1.50.1 \- 2019\-11\-02
|
|
.IP \[bu] 2
|
|
Bug Fixes
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
hash: Fix accidentally changed hash names for \f[C]DropboxHash\f[R] and
|
|
\f[C]CRC\-32\f[R] (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
fshttp: Fix error reporting on tpslimit token bucket errors (Nick
|
|
Craig\-Wood)
|
|
.IP \[bu] 2
|
|
fshttp: Don\[cq]t print token bucket errors on context cancelled (Nick
|
|
Craig\-Wood)
|
|
.RE
|
|
.IP \[bu] 2
|
|
Local
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Fix listings of .
|
|
on Windows (Nick Craig\-Wood)
|
|
.RE
|
|
.IP \[bu] 2
|
|
Onedrive
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Fix DirMove/Move after Onedrive change (Xiaoxing Ye)
|
|
.RE
|
|
.SS v1.50.0 \- 2019\-10\-26
|
|
.IP \[bu] 2
|
|
New backends
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Citrix Sharefile (https://rclone.org/sharefile/) (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Chunker (https://rclone.org/chunker/) \- an overlay backend to split
|
|
files into smaller parts (Ivan Andreev)
|
|
.IP \[bu] 2
|
|
Mail.ru Cloud (https://rclone.org/mailru/) (Ivan Andreev)
|
|
.RE
|
|
.IP \[bu] 2
|
|
New Features
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
encodings (Fabian M\[:o]ller & Nick Craig\-Wood)
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
All backends now use file name encoding to ensure any file name can be
|
|
written to any backend.
|
|
.IP \[bu] 2
|
|
See the restricted file name
|
|
docs (https://rclone.org/overview/#restricted-filenames) for more info
|
|
and the local backend docs (/local/#filenames).
|
|
.IP \[bu] 2
|
|
Some file names may look different in rclone if you are using any
|
|
control characters in names or unicode FULLWIDTH
|
|
symbols (https://en.wikipedia.org/wiki/Halfwidth_and_Fullwidth_Forms_(Unicode_block)).
|
|
.RE
|
|
.IP \[bu] 2
|
|
build
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Update to use go1.13 for the build (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Drop support for go1.9 (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Build rclone with GitHub actions (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Convert python scripts to python3 (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Swap Azure/go\-ansiterm for mattn/go\-colorable (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Dockerfile fixes (Matei David)
|
|
.IP \[bu] 2
|
|
Add plugin
|
|
support (https://github.com/rclone/rclone/blob/master/CONTRIBUTING.md#writing-a-plugin)
|
|
for backends and commands (Richard Patel)
|
|
.RE
|
|
.IP \[bu] 2
|
|
config
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Use alternating Red/Green in config to make more obvious (Nick
|
|
Craig\-Wood)
|
|
.RE
|
|
.IP \[bu] 2
|
|
contrib
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Add sample DLNA server Docker Compose manifest.
|
|
(pataquets)
|
|
.IP \[bu] 2
|
|
Add sample WebDAV server Docker Compose manifest.
|
|
(pataquets)
|
|
.RE
|
|
.IP \[bu] 2
|
|
copyurl
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Add \f[C]\-\-auto\-filename\f[R] flag for using file name from URL in
|
|
destination path (Denis)
|
|
.RE
|
|
.IP \[bu] 2
|
|
serve dlna:
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Many compatibility improvements (Dan Walters)
|
|
.IP \[bu] 2
|
|
Support for external srt subtitles (Dan Walters)
|
|
.RE
|
|
.IP \[bu] 2
|
|
rc
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Added command core/quit (Saksham Khanna)
|
|
.RE
|
|
.RE
|
|
.IP \[bu] 2
|
|
Bug Fixes
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
sync
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Make \f[C]\-\-update\f[R]/\f[C]\-u\f[R] not transfer files that
|
|
haven\[cq]t changed (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Free objects after they come out of the transfer pipe to save memory
|
|
(Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Fix \f[C]\-\-files\-from without \-\-no\-traverse\f[R] doing a recursive
|
|
scan (Nick Craig\-Wood)
|
|
.RE
|
|
.IP \[bu] 2
|
|
operations
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Fix accounting for server side copies (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Display `All duplicates removed' only if dedupe successful (Sezal
|
|
Agrawal)
|
|
.IP \[bu] 2
|
|
Display `Deleted X extra copies' only if dedupe successful (Sezal
|
|
Agrawal)
|
|
.RE
|
|
.IP \[bu] 2
|
|
accounting
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Only allow up to 100 completed transfers in the accounting list to save
|
|
memory (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Cull the old time ranges when possible to save memory (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Fix panic due to server\-side copy fallback (Ivan Andreev)
|
|
.IP \[bu] 2
|
|
Fix memory leak noticeable for transfers of large numbers of objects
|
|
(Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Fix total duration calculation (Nick Craig\-Wood)
|
|
.RE
|
|
.IP \[bu] 2
|
|
cmd
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Fix environment variables not setting command line flags (Nick
|
|
Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Make autocomplete compatible with bash\[cq]s posix mode for macOS (Danil
|
|
Semelenov)
|
|
.IP \[bu] 2
|
|
Make \f[C]\-\-progress\f[R] work in git bash on Windows (Nick
|
|
Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Fix `compopt: command not found' on autocomplete on macOS (Danil
|
|
Semelenov)
|
|
.RE
|
|
.IP \[bu] 2
|
|
config
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Fix setting of non top level flags from environment variables (Nick
|
|
Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Check config names more carefully and report errors (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Remove error: can\[cq]t use \f[C]\-\-size\-only\f[R] and
|
|
\f[C]\-\-ignore\-size\f[R] together.
|
|
(Nick Craig\-Wood)
|
|
.RE
|
|
.IP \[bu] 2
|
|
filter: Prevent mixing options when \f[C]\-\-files\-from\f[R] is in use
|
|
(Michele Caci)
|
|
.IP \[bu] 2
|
|
serve sftp: Fix crash on unsupported operations (eg Readlink) (Nick
|
|
Craig\-Wood)
|
|
.RE
|
|
.IP \[bu] 2
|
|
Mount
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Allow files of unknown size to be read properly (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Skip tests on <= 2 CPUs to avoid lockup (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Fix panic on File.Open (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Fix \[lq]mount_fusefs: \-o timeout=: option not supported\[rq] on
|
|
FreeBSD (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Don\[cq]t pass huge filenames (>4k) to FUSE as it can\[cq]t cope (Nick
|
|
Craig\-Wood)
|
|
.RE
|
|
.IP \[bu] 2
|
|
VFS
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Add flag \f[C]\-\-vfs\-case\-insensitive\f[R] for windows/macOS mounts
|
|
(Ivan Andreev)
|
|
.IP \[bu] 2
|
|
Make objects of unknown size readable through the VFS (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Move writeback of dirty data out of close() method into its own method
|
|
(FlushWrites) and remove close() call from Flush() (Brett Dutro)
|
|
.IP \[bu] 2
|
|
Stop empty dirs disappearing when renamed on bucket based remotes (Nick
|
|
Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Stop change notify polling clearing so much of the directory cache (Nick
|
|
Craig\-Wood)
|
|
.RE
|
|
.IP \[bu] 2
|
|
Azure Blob
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Disable logging to the Windows event log (Nick Craig\-Wood)
|
|
.RE
|
|
.IP \[bu] 2
|
|
B2
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Remove \f[C]unverified:\f[R] prefix on sha1 to improve interop (eg with
|
|
CyberDuck) (Nick Craig\-Wood)
|
|
.RE
|
|
.IP \[bu] 2
|
|
Box
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Add options to get access token via JWT auth (David)
|
|
.RE
|
|
.IP \[bu] 2
|
|
Drive
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Disable HTTP/2 by default to work around INTERNAL_ERROR problems (Nick
|
|
Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Make sure that drive root ID is always canonical (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Fix \f[C]\-\-drive\-shared\-with\-me\f[R] from the root with lsand
|
|
\f[C]\-\-fast\-list\f[R] (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Fix ChangeNotify polling for shared drives (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Fix change notify polling when using appDataFolder (Nick Craig\-Wood)
|
|
.RE
|
|
.IP \[bu] 2
|
|
Dropbox
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Make disallowed filenames errors not retry (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Fix nil pointer exception on restricted files (Nick Craig\-Wood)
|
|
.RE
|
|
.IP \[bu] 2
|
|
Fichier
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Fix accessing files > 2GB on 32 bit systems (Nick Craig\-Wood)
|
|
.RE
|
|
.IP \[bu] 2
|
|
FTP
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Allow disabling EPSV mode (Jon Fautley)
|
|
.RE
|
|
.IP \[bu] 2
|
|
HTTP
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
HEAD directory entries in parallel to speedup (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Add \f[C]\-\-http\-no\-head\f[R] to stop rclone doing HEAD in listings
|
|
(Nick Craig\-Wood)
|
|
.RE
|
|
.IP \[bu] 2
|
|
Putio
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Add ability to resume uploads (Cenk Alti)
|
|
.RE
|
|
.IP \[bu] 2
|
|
S3
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Fix signature v2_auth headers (Anthony Rusdi)
|
|
.IP \[bu] 2
|
|
Fix encoding for control characters (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Only ask for URL encoded directory listings if we need them on Ceph
|
|
(Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Add option for multipart failure behaviour (Aleksandar Jankovic)
|
|
.IP \[bu] 2
|
|
Support for multipart copy (\[u5E84]\[u5929]\[u7FFC])
|
|
.IP \[bu] 2
|
|
Fix nil pointer reference if no metadata returned for object (Nick
|
|
Craig\-Wood)
|
|
.RE
|
|
.IP \[bu] 2
|
|
SFTP
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Fix \f[C]\-\-sftp\-ask\-password\f[R] trying to contact the ssh agent
|
|
(Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Fix hashes of files with backslashes (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Include more ciphers with \f[C]\-\-sftp\-use\-insecure\-cipher\f[R]
|
|
(Carlos Ferreyra)
|
|
.RE
|
|
.IP \[bu] 2
|
|
WebDAV
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Parse and return Sharepoint error response (Henning Surmeier)
|
|
.RE
|
|
.SS v1.49.5 \- 2019\-10\-05
|
|
.IP \[bu] 2
|
|
Bug Fixes
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Revert back to go1.12.x for the v1.49.x builds as go1.13.x was causing
|
|
issues (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Fix rpm packages by using master builds of nfpm (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Fix macOS build after brew changes (Nick Craig\-Wood)
|
|
.RE
|
|
.SS v1.49.4 \- 2019\-09\-29
|
|
.IP \[bu] 2
|
|
Bug Fixes
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
cmd/rcd: Address ZipSlip vulnerability (Richard Patel)
|
|
.IP \[bu] 2
|
|
accounting: Fix file handle leak on errors (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
oauthutil: Fix security problem when running with two users on the same
|
|
machine (Nick Craig\-Wood)
|
|
.RE
|
|
.IP \[bu] 2
|
|
FTP
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Fix listing of an empty root returning: error dir not found (Nick
|
|
Craig\-Wood)
|
|
.RE
|
|
.IP \[bu] 2
|
|
S3
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Fix SetModTime on GLACIER/ARCHIVE objects and implement set/get tier
|
|
(Nick Craig\-Wood)
|
|
.RE
|
|
.SS v1.49.3 \- 2019\-09\-15
|
|
.IP \[bu] 2
|
|
Bug Fixes
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
accounting
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Fix total duration calculation (Aleksandar Jankovic)
|
|
.IP \[bu] 2
|
|
Fix \[lq]file already closed\[rq] on transfer retries (Nick Craig\-Wood)
|
|
.RE
|
|
.RE
|
|
.SS v1.49.2 \- 2019\-09\-08
|
|
.IP \[bu] 2
|
|
New Features
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
build: Add Docker workflow support (Alfonso Montero)
|
|
.RE
|
|
.IP \[bu] 2
|
|
Bug Fixes
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
accounting: Fix locking in Transfer to avoid deadlock with
|
|
\f[C]\-\-progress\f[R] (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
docs: Fix template argument for mktemp in install.sh (Cnly)
|
|
.IP \[bu] 2
|
|
operations: Fix \f[C]\-u\f[R]/\f[C]\-\-update\f[R] with google photos /
|
|
files of unknown size (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
rc: Fix docs for config/create /update /password (Nick Craig\-Wood)
|
|
.RE
|
|
.IP \[bu] 2
|
|
Google Cloud Storage
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Fix need for elevated permissions on SetModTime (Nick Craig\-Wood)
|
|
.RE
|
|
.SS v1.49.1 \- 2019\-08\-28
|
|
.IP \[bu] 2
|
|
Bug Fixes
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
config: Fix generated passwords being stored as empty password (Nick
|
|
Craig\-Wood)
|
|
.IP \[bu] 2
|
|
rcd: Added missing parameter for web\-gui info logs.
|
|
(Chaitanya)
|
|
.RE
|
|
.IP \[bu] 2
|
|
Googlephotos
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Fix crash on error response (Nick Craig\-Wood)
|
|
.RE
|
|
.IP \[bu] 2
|
|
Onedrive
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Fix crash on error response (Nick Craig\-Wood)
|
|
.RE
|
|
.SS v1.49.0 \- 2019\-08\-26
|
|
.IP \[bu] 2
|
|
New backends
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
1fichier (https://rclone.org/fichier/) (Laura Hausmann)
|
|
.IP \[bu] 2
|
|
Google Photos (https://rclone.org/googlephotos/) (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Putio (https://rclone.org/putio/) (Cenk Alti)
|
|
.IP \[bu] 2
|
|
premiumize.me (https://rclone.org/premiumizeme/) (Nick Craig\-Wood)
|
|
.RE
|
|
.IP \[bu] 2
|
|
New Features
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Experimental web GUI (https://rclone.org/gui/) (Chaitanya Bankanhal)
|
|
.IP \[bu] 2
|
|
Implement \f[C]\-\-compare\-dest\f[R] & \f[C]\-\-copy\-dest\f[R]
|
|
(yparitcher)
|
|
.IP \[bu] 2
|
|
Implement \f[C]\-\-suffix\f[R] without \f[C]\-\-backup\-dir\f[R] for
|
|
backup to current dir (yparitcher)
|
|
.IP \[bu] 2
|
|
\f[C]config reconnect\f[R] to re\-login (re\-run the oauth login) for
|
|
the backend.
|
|
(Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
\f[C]config userinfo\f[R] to discover which user you are logged in as.
|
|
(Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
\f[C]config disconnect\f[R] to disconnect you (log out) from the
|
|
backend.
|
|
(Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Add \f[C]\-\-use\-json\-log\f[R] for JSON logging (justinalin)
|
|
.IP \[bu] 2
|
|
Add context propagation to rclone (Aleksandar Jankovic)
|
|
.IP \[bu] 2
|
|
Reworking internal statistics interfaces so they work with rc jobs
|
|
(Aleksandar Jankovic)
|
|
.IP \[bu] 2
|
|
Add Higher units for ETA (AbelThar)
|
|
.IP \[bu] 2
|
|
Update rclone logos to new design (Andreas Chlupka)
|
|
.IP \[bu] 2
|
|
hash: Add CRC\-32 support (Cenk Alti)
|
|
.IP \[bu] 2
|
|
help showbackend: Fixed advanced option category when there are no
|
|
standard options (buengese)
|
|
.IP \[bu] 2
|
|
ncdu: Display/Copy to Clipboard Current Path (Gary Kim)
|
|
.IP \[bu] 2
|
|
operations:
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Run hashing operations in parallel (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Don\[cq]t calculate checksums when using \f[C]\-\-ignore\-checksum\f[R]
|
|
(Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Check transfer hashes when using \f[C]\-\-size\-only\f[R] mode (Nick
|
|
Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Disable multi thread copy for local to local copies (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Debug successful hashes as well as failures (Nick Craig\-Wood)
|
|
.RE
|
|
.IP \[bu] 2
|
|
rc
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Add ability to stop async jobs (Aleksandar Jankovic)
|
|
.IP \[bu] 2
|
|
Return current settings if core/bwlimit called without parameters (Nick
|
|
Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Rclone\-WebUI integration with rclone (Chaitanya Bankanhal)
|
|
.IP \[bu] 2
|
|
Added command line parameter to control the cross origin resource
|
|
sharing (CORS) in the rcd.
|
|
(Security Improvement) (Chaitanya Bankanhal)
|
|
.IP \[bu] 2
|
|
Add anchor tags to the docs so links are consistent (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Remove _async key from input parameters after parsing so later
|
|
operations won\[cq]t get confused (buengese)
|
|
.IP \[bu] 2
|
|
Add call to clear stats (Aleksandar Jankovic)
|
|
.RE
|
|
.IP \[bu] 2
|
|
rcd
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Auto\-login for web\-gui (Chaitanya Bankanhal)
|
|
.IP \[bu] 2
|
|
Implement \f[C]\-\-baseurl\f[R] for rcd and web\-gui (Chaitanya
|
|
Bankanhal)
|
|
.RE
|
|
.IP \[bu] 2
|
|
serve dlna
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Only select interfaces which can multicast for SSDP (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Add more builtin mime types to cover standard audio/video (Nick
|
|
Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Fix missing mime types on Android causing missing videos (Nick
|
|
Craig\-Wood)
|
|
.RE
|
|
.IP \[bu] 2
|
|
serve ftp
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Refactor to bring into line with other serve commands (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Implement \f[C]\-\-auth\-proxy\f[R] (Nick Craig\-Wood)
|
|
.RE
|
|
.IP \[bu] 2
|
|
serve http: Implement \f[C]\-\-baseurl\f[R] (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
serve restic: Implement \f[C]\-\-baseurl\f[R] (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
serve sftp
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Implement auth proxy (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Fix detection of whether server is authorized (Nick Craig\-Wood)
|
|
.RE
|
|
.IP \[bu] 2
|
|
serve webdav
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Implement \f[C]\-\-baseurl\f[R] (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Support \f[C]\-\-auth\-proxy\f[R] (Nick Craig\-Wood)
|
|
.RE
|
|
.RE
|
|
.IP \[bu] 2
|
|
Bug Fixes
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Make \[lq]bad record MAC\[rq] a retriable error (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
copyurl: Fix copying files that return HTTP errors (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
march: Fix checking sub\-directories when using
|
|
\f[C]\-\-no\-traverse\f[R] (buengese)
|
|
.IP \[bu] 2
|
|
rc
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Fix unmarshalable http.AuthFn in options and put in test for
|
|
marshalability (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Move job expire flags to rc to fix initialization problem (Nick
|
|
Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Fix \f[C]\-\-loopback\f[R] with rc/list and others (Nick Craig\-Wood)
|
|
.RE
|
|
.IP \[bu] 2
|
|
rcat: Fix slowdown on systems with multiple hashes (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
rcd: Fix permissions problems on cache directory with web gui download
|
|
(Nick Craig\-Wood)
|
|
.RE
|
|
.IP \[bu] 2
|
|
Mount
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Default \f[C]\-\-deamon\-timout\f[R] to 15 minutes on macOS and FreeBSD
|
|
(Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Update docs to show mounting from root OK for bucket based (Nick
|
|
Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Remove nonseekable flag from write files (Nick Craig\-Wood)
|
|
.RE
|
|
.IP \[bu] 2
|
|
VFS
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Make write without cache more efficient (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Fix \f[C]\-\-vfs\-cache\-mode minimal\f[R] and \f[C]writes\f[R] ignoring
|
|
cached files (Nick Craig\-Wood)
|
|
.RE
|
|
.IP \[bu] 2
|
|
Local
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Add \f[C]\-\-local\-case\-sensitive\f[R] and
|
|
\f[C]\-\-local\-case\-insensitive\f[R] (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Avoid polluting page cache when uploading local files to remote backends
|
|
(Micha\[/l] Matczuk)
|
|
.IP \[bu] 2
|
|
Don\[cq]t calculate any hashes by default (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Fadvise run syscall on a dedicated go routine (Micha\[/l] Matczuk)
|
|
.RE
|
|
.IP \[bu] 2
|
|
Azure Blob
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Azure Storage Emulator support (Sandeep)
|
|
.IP \[bu] 2
|
|
Updated config help details to remove connection string references
|
|
(Sandeep)
|
|
.IP \[bu] 2
|
|
Make all operations work from the root (Nick Craig\-Wood)
|
|
.RE
|
|
.IP \[bu] 2
|
|
B2
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Implement link sharing (yparitcher)
|
|
.IP \[bu] 2
|
|
Enable server side copy to copy between buckets (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Make all operations work from the root (Nick Craig\-Wood)
|
|
.RE
|
|
.IP \[bu] 2
|
|
Drive
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Fix server side copy of big files (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Update API for teamdrive use (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Add error for purge with \f[C]\-\-drive\-trashed\-only\f[R] (ginvine)
|
|
.RE
|
|
.IP \[bu] 2
|
|
Fichier
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Make FolderID int and adjust related code (buengese)
|
|
.RE
|
|
.IP \[bu] 2
|
|
Google Cloud Storage
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Reduce oauth scope requested as suggested by Google (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Make all operations work from the root (Nick Craig\-Wood)
|
|
.RE
|
|
.IP \[bu] 2
|
|
HTTP
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Add \f[C]\-\-http\-headers\f[R] flag for setting arbitrary headers (Nick
|
|
Craig\-Wood)
|
|
.RE
|
|
.IP \[bu] 2
|
|
Jottacloud
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Use new api for retrieving internal username (buengese)
|
|
.IP \[bu] 2
|
|
Refactor configuration and minor cleanup (buengese)
|
|
.RE
|
|
.IP \[bu] 2
|
|
Koofr
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Support setting modification times on Koofr backend.
|
|
(jaKa)
|
|
.RE
|
|
.IP \[bu] 2
|
|
Opendrive
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Refactor to use existing lib/rest facilities for uploads (Nick
|
|
Craig\-Wood)
|
|
.RE
|
|
.IP \[bu] 2
|
|
Qingstor
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Upgrade to v3 SDK and fix listing loop (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Make all operations work from the root (Nick Craig\-Wood)
|
|
.RE
|
|
.IP \[bu] 2
|
|
S3
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Add INTELLIGENT_TIERING storage class (Matti Niemenmaa)
|
|
.IP \[bu] 2
|
|
Make all operations work from the root (Nick Craig\-Wood)
|
|
.RE
|
|
.IP \[bu] 2
|
|
SFTP
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Add missing interface check and fix About (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Completely ignore all modtime checks if SetModTime=false (Jon Fautley)
|
|
.IP \[bu] 2
|
|
Support md5/sha1 with rsync.net (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Save the md5/sha1 command in use to the config file for efficiency (Nick
|
|
Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Opt\-in support for diffie\-hellman\-group\-exchange\-sha256
|
|
diffie\-hellman\-group\-exchange\-sha1 (Yi FU)
|
|
.RE
|
|
.IP \[bu] 2
|
|
Swift
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Use FixRangeOption to fix 0 length files via the VFS (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Fix upload when using no_chunk to return the correct size (Nick
|
|
Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Make all operations work from the root (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Fix segments leak during failed large file uploads.
|
|
(nguyenhuuluan434)
|
|
.RE
|
|
.IP \[bu] 2
|
|
WebDAV
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Add \f[C]\-\-webdav\-bearer\-token\-command\f[R] (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Refresh token when it expires with
|
|
\f[C]\-\-webdav\-bearer\-token\-command\f[R] (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Add docs for using bearer_token_command with oidc\-agent (Paul Millar)
|
|
.RE
|
|
.SS v1.48.0 \- 2019\-06\-15
|
|
.IP \[bu] 2
|
|
New commands
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
serve sftp: Serve an rclone remote over SFTP (Nick Craig\-Wood)
|
|
.RE
|
|
.IP \[bu] 2
|
|
New Features
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Multi threaded downloads to local storage (Nick Craig\-Wood)
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
controlled with \f[C]\-\-multi\-thread\-cutoff\f[R] and
|
|
\f[C]\-\-multi\-thread\-streams\f[R]
|
|
.RE
|
|
.IP \[bu] 2
|
|
Use rclone.conf from rclone executable directory to enable portable use
|
|
(albertony)
|
|
.IP \[bu] 2
|
|
Allow sync of a file and a directory with the same name (forgems)
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
this is common on bucket based remotes, eg s3, gcs
|
|
.RE
|
|
.IP \[bu] 2
|
|
Add \f[C]\-\-ignore\-case\-sync\f[R] for forced case insensitivity
|
|
(garry415)
|
|
.IP \[bu] 2
|
|
Implement \f[C]\-\-stats\-one\-line\-date\f[R] and
|
|
\f[C]\-\-stats\-one\-line\-date\-format\f[R] (Peter Berbec)
|
|
.IP \[bu] 2
|
|
Log an ERROR for all commands which exit with non\-zero status (Nick
|
|
Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Use go\-homedir to read the home directory more reliably (Nick
|
|
Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Enable creating encrypted config through external script invocation
|
|
(Wojciech Smigielski)
|
|
.IP \[bu] 2
|
|
build: Drop support for go1.8 (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
config: Make config create/update encrypt passwords where necessary
|
|
(Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
copyurl: Honor \f[C]\-\-no\-check\-certificate\f[R] (Stefan Breunig)
|
|
.IP \[bu] 2
|
|
install: Linux skip man pages if no mandb (didil)
|
|
.IP \[bu] 2
|
|
lsf: Support showing the Tier of the object (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
lsjson
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Added EncryptedPath to output (calisro)
|
|
.IP \[bu] 2
|
|
Support showing the Tier of the object (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Add IsBucket field for bucket based remote listing of the root (Nick
|
|
Craig\-Wood)
|
|
.RE
|
|
.IP \[bu] 2
|
|
rc
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Add \f[C]\-\-loopback\f[R] flag to run commands directly without a
|
|
server (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Add operations/fsinfo: Return information about the remote (Nick
|
|
Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Skip auth for OPTIONS request (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
cmd/providers: Add DefaultStr, ValueStr and Type fields (Nick
|
|
Craig\-Wood)
|
|
.IP \[bu] 2
|
|
jobs: Make job expiry timeouts configurable (Aleksandar Jankovic)
|
|
.RE
|
|
.IP \[bu] 2
|
|
serve dlna reworked and improved (Dan Walters)
|
|
.IP \[bu] 2
|
|
serve ftp: add \f[C]\-\-ftp\-public\-ip\f[R] flag to specify public IP
|
|
(calistri)
|
|
.IP \[bu] 2
|
|
serve restic: Add support for \f[C]\-\-private\-repos\f[R] in
|
|
\f[C]serve restic\f[R] (Florian Apolloner)
|
|
.IP \[bu] 2
|
|
serve webdav: Combine serve webdav and serve http (Gary Kim)
|
|
.IP \[bu] 2
|
|
size: Ignore negative sizes when calculating total (Garry McNulty)
|
|
.RE
|
|
.IP \[bu] 2
|
|
Bug Fixes
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Make move and copy individual files obey \f[C]\-\-backup\-dir\f[R] (Nick
|
|
Craig\-Wood)
|
|
.IP \[bu] 2
|
|
If \f[C]\-\-ignore\-checksum\f[R] is in effect, don\[cq]t calculate
|
|
checksum (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
moveto: Fix case\-insensitive same remote move (Gary Kim)
|
|
.IP \[bu] 2
|
|
rc: Fix serving bucket based objects with \f[C]\-\-rc\-serve\f[R] (Nick
|
|
Craig\-Wood)
|
|
.IP \[bu] 2
|
|
serve webdav: Fix serveDir not being updated with changes from webdav
|
|
(Gary Kim)
|
|
.RE
|
|
.IP \[bu] 2
|
|
Mount
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Fix poll interval documentation (Animosity022)
|
|
.RE
|
|
.IP \[bu] 2
|
|
VFS
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Make WriteAt for non cached files work with non\-sequential writes (Nick
|
|
Craig\-Wood)
|
|
.RE
|
|
.IP \[bu] 2
|
|
Local
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Only calculate the required hashes for big speedup (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Log errors when listing instead of returning an error (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Fix preallocate warning on Linux with ZFS (Nick Craig\-Wood)
|
|
.RE
|
|
.IP \[bu] 2
|
|
Crypt
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Make rclone dedupe work through crypt (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Fix wrapping of ChangeNotify to decrypt directories properly (Nick
|
|
Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Support PublicLink (rclone link) of underlying backend (Nick
|
|
Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Implement Optional methods SetTier, GetTier (Nick Craig\-Wood)
|
|
.RE
|
|
.IP \[bu] 2
|
|
B2
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Implement server side copy (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Implement SetModTime (Nick Craig\-Wood)
|
|
.RE
|
|
.IP \[bu] 2
|
|
Drive
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Fix move and copy from TeamDrive to GDrive (Fionera)
|
|
.IP \[bu] 2
|
|
Add notes that cleanup works in the background on drive (Nick
|
|
Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Add \f[C]\-\-drive\-server\-side\-across\-configs\f[R] to default back
|
|
to old server side copy semantics by default (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Add \f[C]\-\-drive\-size\-as\-quota\f[R] to show storage quota usage for
|
|
file size (Garry McNulty)
|
|
.RE
|
|
.IP \[bu] 2
|
|
FTP
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Add FTP List timeout (Jeff Quinn)
|
|
.IP \[bu] 2
|
|
Add FTP over TLS support (Gary Kim)
|
|
.IP \[bu] 2
|
|
Add \f[C]\-\-ftp\-no\-check\-certificate\f[R] option for FTPS (Gary Kim)
|
|
.RE
|
|
.IP \[bu] 2
|
|
Google Cloud Storage
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Fix upload errors when uploading pre 1970 files (Nick Craig\-Wood)
|
|
.RE
|
|
.IP \[bu] 2
|
|
Jottacloud
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Add support for selecting device and mountpoint.
|
|
(buengese)
|
|
.RE
|
|
.IP \[bu] 2
|
|
Mega
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Add cleanup support (Gary Kim)
|
|
.RE
|
|
.IP \[bu] 2
|
|
Onedrive
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
More accurately check if root is found (Cnly)
|
|
.RE
|
|
.IP \[bu] 2
|
|
S3
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Support S3 Accelerated endpoints with
|
|
\f[C]\-\-s3\-use\-accelerate\-endpoint\f[R] (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Add config info for Wasabi\[cq]s EU Central endpoint (Robert Marko)
|
|
.IP \[bu] 2
|
|
Make SetModTime work for GLACIER while syncing (Philip Harvey)
|
|
.RE
|
|
.IP \[bu] 2
|
|
SFTP
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Add About support (Gary Kim)
|
|
.IP \[bu] 2
|
|
Fix about parsing of \f[C]df\f[R] results so it can cope with \-ve
|
|
results (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Send custom client version and debug server version (Nick Craig\-Wood)
|
|
.RE
|
|
.IP \[bu] 2
|
|
WebDAV
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Retry on 423 Locked errors (Nick Craig\-Wood)
|
|
.RE
|
|
.SS v1.47.0 \- 2019\-04\-13
|
|
.IP \[bu] 2
|
|
New backends
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Backend for Koofr cloud storage service.
|
|
(jaKa)
|
|
.RE
|
|
.IP \[bu] 2
|
|
New Features
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Resume downloads if the reader fails in copy (Nick Craig\-Wood)
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
this means rclone will restart transfers if the source has an error
|
|
.IP \[bu] 2
|
|
this is most useful for downloads or cloud to cloud copies
|
|
.RE
|
|
.IP \[bu] 2
|
|
Use \f[C]\-\-fast\-list\f[R] for listing operations where it won\[cq]t
|
|
use more memory (Nick Craig\-Wood)
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
this should speed up the following operations on remotes which support
|
|
\f[C]ListR\f[R]
|
|
.IP \[bu] 2
|
|
\f[C]dedupe\f[R], \f[C]serve restic\f[R] \f[C]lsf\f[R], \f[C]ls\f[R],
|
|
\f[C]lsl\f[R], \f[C]lsjson\f[R], \f[C]lsd\f[R], \f[C]md5sum\f[R],
|
|
\f[C]sha1sum\f[R], \f[C]hashsum\f[R], \f[C]size\f[R], \f[C]delete\f[R],
|
|
\f[C]cat\f[R], \f[C]settier\f[R]
|
|
.IP \[bu] 2
|
|
use \f[C]\-\-disable ListR\f[R] to get old behaviour if required
|
|
.RE
|
|
.IP \[bu] 2
|
|
Make \f[C]\-\-files\-from\f[R] traverse the destination unless
|
|
\f[C]\-\-no\-traverse\f[R] is set (Nick Craig\-Wood)
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
this fixes \f[C]\-\-files\-from\f[R] with Google drive and excessive API
|
|
use in general.
|
|
.RE
|
|
.IP \[bu] 2
|
|
Make server side copy account bytes and obey \f[C]\-\-max\-transfer\f[R]
|
|
(Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Add \f[C]\-\-create\-empty\-src\-dirs\f[R] flag and default to not
|
|
creating empty dirs (ishuah)
|
|
.IP \[bu] 2
|
|
Add client side TLS/SSL flags
|
|
\f[C]\-\-ca\-cert\f[R]/\f[C]\-\-client\-cert\f[R]/\f[C]\-\-client\-key\f[R]
|
|
(Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Implement \f[C]\-\-suffix\-keep\-extension\f[R] for use with
|
|
\f[C]\-\-suffix\f[R] (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
build:
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Switch to semvar compliant version tags to be go modules compliant (Nick
|
|
Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Update to use go1.12.x for the build (Nick Craig\-Wood)
|
|
.RE
|
|
.IP \[bu] 2
|
|
serve dlna: Add connection manager service description to improve
|
|
compatibility (Dan Walters)
|
|
.IP \[bu] 2
|
|
lsf: Add `e' format to show encrypted names and `o' for original IDs
|
|
(Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
lsjson: Added \f[C]\-\-files\-only\f[R] and \f[C]\-\-dirs\-only\f[R]
|
|
flags (calistri)
|
|
.IP \[bu] 2
|
|
rc: Implement operations/publiclink the equivalent of
|
|
\f[C]rclone link\f[R] (Nick Craig\-Wood)
|
|
.RE
|
|
.IP \[bu] 2
|
|
Bug Fixes
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
accounting: Fix total ETA when \f[C]\-\-stats\-unit bits\f[R] is in
|
|
effect (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Bash TAB completion
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Use private custom func to fix clash between rclone and kubectl (Nick
|
|
Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Fix for remotes with underscores in their names (Six)
|
|
.IP \[bu] 2
|
|
Fix completion of remotes (Florian Gamb\[:o]ck)
|
|
.IP \[bu] 2
|
|
Fix autocompletion of remote paths with spaces (Danil Semelenov)
|
|
.RE
|
|
.IP \[bu] 2
|
|
serve dlna: Fix root XML service descriptor (Dan Walters)
|
|
.IP \[bu] 2
|
|
ncdu: Fix display corruption with Chinese characters (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Add SIGTERM to signals which run the exit handlers on unix (Nick
|
|
Craig\-Wood)
|
|
.IP \[bu] 2
|
|
rc: Reload filter when the options are set via the rc (Nick Craig\-Wood)
|
|
.RE
|
|
.IP \[bu] 2
|
|
VFS / Mount
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Fix FreeBSD: Ignore Truncate if called with no readers and already the
|
|
correct size (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Read directory and check for a file before mkdir (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Shorten the locking window for vfs/refresh (Nick Craig\-Wood)
|
|
.RE
|
|
.IP \[bu] 2
|
|
Azure Blob
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Enable MD5 checksums when uploading files bigger than the
|
|
\[lq]Cutoff\[rq] (Dr.Rx)
|
|
.IP \[bu] 2
|
|
Fix SAS URL support (Nick Craig\-Wood)
|
|
.RE
|
|
.IP \[bu] 2
|
|
B2
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Allow manual configuration of backblaze downloadUrl (Vince)
|
|
.IP \[bu] 2
|
|
Ignore already_hidden error on remove (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Ignore malformed \f[C]src_last_modified_millis\f[R] (Nick Craig\-Wood)
|
|
.RE
|
|
.IP \[bu] 2
|
|
Drive
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Add \f[C]\-\-skip\-checksum\-gphotos\f[R] to ignore incorrect checksums
|
|
on Google Photos (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Allow server side move/copy between different remotes.
|
|
(Fionera)
|
|
.IP \[bu] 2
|
|
Add docs on team drives and \f[C]\-\-fast\-list\f[R] eventual
|
|
consistency (Nestar47)
|
|
.IP \[bu] 2
|
|
Fix imports of text files (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Fix range requests on 0 length files (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Fix creation of duplicates with server side copy (Nick Craig\-Wood)
|
|
.RE
|
|
.IP \[bu] 2
|
|
Dropbox
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Retry blank errors to fix long listings (Nick Craig\-Wood)
|
|
.RE
|
|
.IP \[bu] 2
|
|
FTP
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Add \f[C]\-\-ftp\-concurrency\f[R] to limit maximum number of
|
|
connections (Nick Craig\-Wood)
|
|
.RE
|
|
.IP \[bu] 2
|
|
Google Cloud Storage
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Fall back to default application credentials (marcintustin)
|
|
.IP \[bu] 2
|
|
Allow bucket policy only buckets (Nick Craig\-Wood)
|
|
.RE
|
|
.IP \[bu] 2
|
|
HTTP
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Add \f[C]\-\-http\-no\-slash\f[R] for websites with directories with no
|
|
slashes (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Remove duplicates from listings (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Fix socket leak on 404 errors (Nick Craig\-Wood)
|
|
.RE
|
|
.IP \[bu] 2
|
|
Jottacloud
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Fix token refresh (Sebastian B\[:u]nger)
|
|
.IP \[bu] 2
|
|
Add device registration (Oliver Heyme)
|
|
.RE
|
|
.IP \[bu] 2
|
|
Onedrive
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Implement graceful cancel of multipart uploads if rclone is interrupted
|
|
(Cnly)
|
|
.IP \[bu] 2
|
|
Always add trailing colon to path when addressing items, (Cnly)
|
|
.IP \[bu] 2
|
|
Return errors instead of panic for invalid uploads (Fabian M\[:o]ller)
|
|
.RE
|
|
.IP \[bu] 2
|
|
S3
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Add support for \[lq]Glacier Deep Archive\[rq] storage class (Manu)
|
|
.IP \[bu] 2
|
|
Update Dreamhost endpoint (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Note incompatibility with CEPH Jewel (Nick Craig\-Wood)
|
|
.RE
|
|
.IP \[bu] 2
|
|
SFTP
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Allow custom ssh client config (Alexandru Bumbacea)
|
|
.RE
|
|
.IP \[bu] 2
|
|
Swift
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Obey Retry\-After to enable OVH restore from cold storage (Nick
|
|
Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Work around token expiry on CEPH (Nick Craig\-Wood)
|
|
.RE
|
|
.IP \[bu] 2
|
|
WebDAV
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Allow IsCollection property to be integer or boolean (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Fix race when creating directories (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Fix About/df when reading the available/total returns 0 (Nick
|
|
Craig\-Wood)
|
|
.RE
|
|
.SS v1.46 \- 2019\-02\-09
|
|
.IP \[bu] 2
|
|
New backends
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Support Alibaba Cloud (Aliyun) OSS via the s3 backend (Nick Craig\-Wood)
|
|
.RE
|
|
.IP \[bu] 2
|
|
New commands
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
serve dlna: serves a remove via DLNA for the local network (nicolov)
|
|
.RE
|
|
.IP \[bu] 2
|
|
New Features
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
copy, move: Restore deprecated \f[C]\-\-no\-traverse\f[R] flag (Nick
|
|
Craig\-Wood)
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
This is useful for when transferring a small number of files into a
|
|
large destination
|
|
.RE
|
|
.IP \[bu] 2
|
|
genautocomplete: Add remote path completion for bash completion
|
|
(Christopher Peterson & Danil Semelenov)
|
|
.IP \[bu] 2
|
|
Buffer memory handling reworked to return memory to the OS better (Nick
|
|
Craig\-Wood)
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Buffer recycling library to replace sync.Pool
|
|
.IP \[bu] 2
|
|
Optionally use memory mapped memory for better memory shrinking
|
|
.IP \[bu] 2
|
|
Enable with \f[C]\-\-use\-mmap\f[R] if having memory problems \- not
|
|
default yet
|
|
.RE
|
|
.IP \[bu] 2
|
|
Parallelise reading of files specified by \f[C]\-\-files\-from\f[R]
|
|
(Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
check: Add stats showing total files matched.
|
|
(Dario Guzik)
|
|
.IP \[bu] 2
|
|
Allow rename/delete open files under Windows (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
lsjson: Use exactly the correct number of decimal places in the seconds
|
|
(Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Add cookie support with cmdline switch \f[C]\-\-use\-cookies\f[R] for
|
|
all HTTP based remotes (qip)
|
|
.IP \[bu] 2
|
|
Warn if \f[C]\-\-checksum\f[R] is set but there are no hashes available
|
|
(Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Rework rate limiting (pacer) to be more accurate and allow bursting
|
|
(Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Improve error reporting for too many/few arguments in commands (Nick
|
|
Craig\-Wood)
|
|
.IP \[bu] 2
|
|
listremotes: Remove \f[C]\-l\f[R] short flag as it conflicts with the
|
|
new global flag (weetmuts)
|
|
.IP \[bu] 2
|
|
Make http serving with auth generate INFO messages on auth fail (Nick
|
|
Craig\-Wood)
|
|
.RE
|
|
.IP \[bu] 2
|
|
Bug Fixes
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Fix layout of stats (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Fix \f[C]\-\-progress\f[R] crash under Windows Jenkins (Nick
|
|
Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Fix transfer of google/onedrive docs by calling Rcat in Copy when size
|
|
is \-1 (Cnly)
|
|
.IP \[bu] 2
|
|
copyurl: Fix checking of \f[C]\-\-dry\-run\f[R] (Denis Skovpen)
|
|
.RE
|
|
.IP \[bu] 2
|
|
Mount
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Check that mountpoint and local directory to mount don\[cq]t overlap
|
|
(Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Fix mount size under 32 bit Windows (Nick Craig\-Wood)
|
|
.RE
|
|
.IP \[bu] 2
|
|
VFS
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Implement renaming of directories for backends without DirMove (Nick
|
|
Craig\-Wood)
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
now all backends except b2 support renaming directories
|
|
.RE
|
|
.IP \[bu] 2
|
|
Implement \f[C]\-\-vfs\-cache\-max\-size\f[R] to limit the total size of
|
|
the cache (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Add \f[C]\-\-dir\-perms\f[R] and \f[C]\-\-file\-perms\f[R] flags to set
|
|
default permissions (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Fix deadlock on concurrent operations on a directory (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Fix deadlock between RWFileHandle.close and File.Remove (Nick
|
|
Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Fix renaming/deleting open files with cache mode \[lq]writes\[rq] under
|
|
Windows (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Fix panic on rename with \f[C]\-\-dry\-run\f[R] set (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Fix vfs/refresh with recurse=true needing the \f[C]\-\-fast\-list\f[R]
|
|
flag
|
|
.RE
|
|
.IP \[bu] 2
|
|
Local
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Add support for \f[C]\-l\f[R]/\f[C]\-\-links\f[R] (symbolic link
|
|
translation) (yair\[at]unicorn)
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
this works by showing links as \f[C]link.rclonelink\f[R] \- see local
|
|
backend docs for more info
|
|
.IP \[bu] 2
|
|
this errors if used with \f[C]\-L\f[R]/\f[C]\-\-copy\-links\f[R]
|
|
.RE
|
|
.IP \[bu] 2
|
|
Fix renaming/deleting open files on Windows (Nick Craig\-Wood)
|
|
.RE
|
|
.IP \[bu] 2
|
|
Crypt
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Check for maximum length before decrypting filename to fix panic (Garry
|
|
McNulty)
|
|
.RE
|
|
.IP \[bu] 2
|
|
Azure Blob
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Allow building azureblob backend on *BSD (themylogin)
|
|
.IP \[bu] 2
|
|
Use the rclone HTTP client to support \f[C]\-\-dump headers\f[R],
|
|
\f[C]\-\-tpslimit\f[R] etc (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Use the s3 pacer for 0 delay in non error conditions (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Ignore directory markers (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Stop Mkdir attempting to create existing containers (Nick Craig\-Wood)
|
|
.RE
|
|
.IP \[bu] 2
|
|
B2
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
cleanup: will remove unfinished large files >24hrs old (Garry McNulty)
|
|
.IP \[bu] 2
|
|
For a bucket limited application key check the bucket name (Nick
|
|
Craig\-Wood)
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
before this, rclone would use the authorised bucket regardless of what
|
|
you put on the command line
|
|
.RE
|
|
.IP \[bu] 2
|
|
Added \f[C]\-\-b2\-disable\-checksum\f[R] flag (Wojciech Smigielski)
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
this enables large files to be uploaded without a SHA\-1 hash for speed
|
|
reasons
|
|
.RE
|
|
.RE
|
|
.IP \[bu] 2
|
|
Drive
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Set default pacer to 100ms for 10 tps (Nick Craig\-Wood)
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
This fits the Google defaults much better and reduces the 403 errors
|
|
massively
|
|
.IP \[bu] 2
|
|
Add \f[C]\-\-drive\-pacer\-min\-sleep\f[R] and
|
|
\f[C]\-\-drive\-pacer\-burst\f[R] to control the pacer
|
|
.RE
|
|
.IP \[bu] 2
|
|
Improve ChangeNotify support for items with multiple parents (Fabian
|
|
M\[:o]ller)
|
|
.IP \[bu] 2
|
|
Fix ListR for items with multiple parents \- this fixes oddities with
|
|
\f[C]vfs/refresh\f[R] (Fabian M\[:o]ller)
|
|
.IP \[bu] 2
|
|
Fix using \f[C]\-\-drive\-impersonate\f[R] and appfolders (Nick
|
|
Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Fix google docs in rclone mount for some (not all) applications (Nick
|
|
Craig\-Wood)
|
|
.RE
|
|
.IP \[bu] 2
|
|
Dropbox
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Retry\-After support for Dropbox backend (Mathieu Carbou)
|
|
.RE
|
|
.IP \[bu] 2
|
|
FTP
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Wait for 60 seconds for a connection to Close then declare it dead (Nick
|
|
Craig\-Wood)
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
helps with indefinite hangs on some FTP servers
|
|
.RE
|
|
.RE
|
|
.IP \[bu] 2
|
|
Google Cloud Storage
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Update google cloud storage endpoints (weetmuts)
|
|
.RE
|
|
.IP \[bu] 2
|
|
HTTP
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Add an example with username and password which is supported but
|
|
wasn\[cq]t documented (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Fix backend with \f[C]\-\-files\-from\f[R] and non\-existent files (Nick
|
|
Craig\-Wood)
|
|
.RE
|
|
.IP \[bu] 2
|
|
Hubic
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Make error message more informative if authentication fails (Nick
|
|
Craig\-Wood)
|
|
.RE
|
|
.IP \[bu] 2
|
|
Jottacloud
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Resume and deduplication support (Oliver Heyme)
|
|
.IP \[bu] 2
|
|
Use token auth for all API requests Don\[cq]t store password anymore
|
|
(Sebastian B\[:u]nger)
|
|
.IP \[bu] 2
|
|
Add support for 2\-factor authentication (Sebastian B\[:u]nger)
|
|
.RE
|
|
.IP \[bu] 2
|
|
Mega
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Implement v2 account login which fixes logins for newer Mega accounts
|
|
(Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Return error if an unknown length file is attempted to be uploaded (Nick
|
|
Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Add new error codes for better error reporting (Nick Craig\-Wood)
|
|
.RE
|
|
.IP \[bu] 2
|
|
Onedrive
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Fix broken support for \[lq]shared with me\[rq] folders (Alex Chen)
|
|
.IP \[bu] 2
|
|
Fix root ID not normalised (Cnly)
|
|
.IP \[bu] 2
|
|
Return err instead of panic on unknown\-sized uploads (Cnly)
|
|
.RE
|
|
.IP \[bu] 2
|
|
Qingstor
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Fix go routine leak on multipart upload errors (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Add upload chunk size/concurrency/cutoff control (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Default \f[C]\-\-qingstor\-upload\-concurrency\f[R] to 1 to work around
|
|
bug (Nick Craig\-Wood)
|
|
.RE
|
|
.IP \[bu] 2
|
|
S3
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Implement \f[C]\-\-s3\-upload\-cutoff\f[R] for single part uploads below
|
|
this (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Change \f[C]\-\-s3\-upload\-concurrency\f[R] default to 4 to increase
|
|
performance (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Add \f[C]\-\-s3\-bucket\-acl\f[R] to control bucket ACL (Nick
|
|
Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Auto detect region for buckets on operation failure (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Add GLACIER storage class (William Cocker)
|
|
.IP \[bu] 2
|
|
Add Scaleway to s3 documentation (R\['e]my L\['e]one)
|
|
.IP \[bu] 2
|
|
Add AWS endpoint eu\-north\-1 (weetmuts)
|
|
.RE
|
|
.IP \[bu] 2
|
|
SFTP
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Add support for PEM encrypted private keys (Fabian M\[:o]ller)
|
|
.IP \[bu] 2
|
|
Add option to force the usage of an ssh\-agent (Fabian M\[:o]ller)
|
|
.IP \[bu] 2
|
|
Perform environment variable expansion on key\-file (Fabian M\[:o]ller)
|
|
.IP \[bu] 2
|
|
Fix rmdir on Windows based servers (eg CrushFTP) (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Fix rmdir deleting directory contents on some SFTP servers (Nick
|
|
Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Fix error on dangling symlinks (Nick Craig\-Wood)
|
|
.RE
|
|
.IP \[bu] 2
|
|
Swift
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Add \f[C]\-\-swift\-no\-chunk\f[R] to disable segmented uploads in
|
|
rcat/mount (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Introduce application credential auth support (kayrus)
|
|
.IP \[bu] 2
|
|
Fix memory usage by slimming Object (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Fix extra requests on upload (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Fix reauth on big files (Nick Craig\-Wood)
|
|
.RE
|
|
.IP \[bu] 2
|
|
Union
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Fix poll\-interval not working (Nick Craig\-Wood)
|
|
.RE
|
|
.IP \[bu] 2
|
|
WebDAV
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Support About which means rclone mount will show the correct disk size
|
|
(Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Support MD5 and SHA1 hashes with Owncloud and Nextcloud (Nick
|
|
Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Fail soft on time parsing errors (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Fix infinite loop on failed directory creation (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Fix identification of directories for Bitrix Site Manager (Nick
|
|
Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Fix upload of 0 length files on some servers (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Fix if MKCOL fails with 423 Locked assume the directory exists (Nick
|
|
Craig\-Wood)
|
|
.RE
|
|
.SS v1.45 \- 2018\-11\-24
|
|
.IP \[bu] 2
|
|
New backends
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
The Yandex backend was re\-written \- see below for details (Sebastian
|
|
B\[:u]nger)
|
|
.RE
|
|
.IP \[bu] 2
|
|
New commands
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
rcd: New command just to serve the remote control API (Nick Craig\-Wood)
|
|
.RE
|
|
.IP \[bu] 2
|
|
New Features
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
The remote control API (rc) was greatly expanded to allow full control
|
|
over rclone (Nick Craig\-Wood)
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
sensitive operations require authorization or the
|
|
\f[C]\-\-rc\-no\-auth\f[R] flag
|
|
.IP \[bu] 2
|
|
config/* operations to configure rclone
|
|
.IP \[bu] 2
|
|
options/* for reading/setting command line flags
|
|
.IP \[bu] 2
|
|
operations/* for all low level operations, eg copy file, list directory
|
|
.IP \[bu] 2
|
|
sync/* for sync, copy and move
|
|
.IP \[bu] 2
|
|
\f[C]\-\-rc\-files\f[R] flag to serve files on the rc http server
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
this is for building web native GUIs for rclone
|
|
.RE
|
|
.IP \[bu] 2
|
|
Optionally serving objects on the rc http server
|
|
.IP \[bu] 2
|
|
Ensure rclone fails to start up if the \f[C]\-\-rc\f[R] port is in use
|
|
already
|
|
.IP \[bu] 2
|
|
See the rc docs (https://rclone.org/rc/) for more info
|
|
.RE
|
|
.IP \[bu] 2
|
|
sync/copy/move
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Make \f[C]\-\-files\-from\f[R] only read the objects specified and
|
|
don\[cq]t scan directories (Nick Craig\-Wood)
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
This is a huge speed improvement for destinations with lots of files
|
|
.RE
|
|
.RE
|
|
.IP \[bu] 2
|
|
filter: Add \f[C]\-\-ignore\-case\f[R] flag (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
ncdu: Add remove function (`d' key) (Henning Surmeier)
|
|
.IP \[bu] 2
|
|
rc command
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Add \f[C]\-\-json\f[R] flag for structured JSON input (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Add \f[C]\-\-user\f[R] and \f[C]\-\-pass\f[R] flags and interpret
|
|
\f[C]\-\-rc\-user\f[R], \f[C]\-\-rc\-pass\f[R], \f[C]\-\-rc\-addr\f[R]
|
|
(Nick Craig\-Wood)
|
|
.RE
|
|
.IP \[bu] 2
|
|
build
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Require go1.8 or later for compilation (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Enable softfloat on MIPS arch (Scott Edlund)
|
|
.IP \[bu] 2
|
|
Integration test framework revamped with a better report and better
|
|
retries (Nick Craig\-Wood)
|
|
.RE
|
|
.RE
|
|
.IP \[bu] 2
|
|
Bug Fixes
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
cmd: Make \f[C]\-\-progress\f[R] update the stats correctly at the end
|
|
(Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
config: Create config directory on save if it is missing (Nick
|
|
Craig\-Wood)
|
|
.IP \[bu] 2
|
|
dedupe: Check for existing filename before renaming a dupe file (ssaqua)
|
|
.IP \[bu] 2
|
|
move: Don\[cq]t create directories with \f[C]\-\-dry\-run\f[R] (Nick
|
|
Craig\-Wood)
|
|
.IP \[bu] 2
|
|
operations: Fix Purge and Rmdirs when dir is not the root (Nick
|
|
Craig\-Wood)
|
|
.IP \[bu] 2
|
|
serve http/webdav/restic: Ensure rclone exits if the port is in use
|
|
(Nick Craig\-Wood)
|
|
.RE
|
|
.IP \[bu] 2
|
|
Mount
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Make \f[C]\-\-volname\f[R] work for Windows and macOS (Nick Craig\-Wood)
|
|
.RE
|
|
.IP \[bu] 2
|
|
Azure Blob
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Avoid context deadline exceeded error by setting a large TryTimeout
|
|
value (brused27)
|
|
.IP \[bu] 2
|
|
Fix erroneous Rmdir error \[lq]directory not empty\[rq] (Nick
|
|
Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Wait for up to 60s to create a just deleted container (Nick Craig\-Wood)
|
|
.RE
|
|
.IP \[bu] 2
|
|
Dropbox
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Add dropbox impersonate support (Jake Coggiano)
|
|
.RE
|
|
.IP \[bu] 2
|
|
Jottacloud
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Fix bug in \f[C]\-\-fast\-list\f[R] handing of empty folders (albertony)
|
|
.RE
|
|
.IP \[bu] 2
|
|
Opendrive
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Fix transfer of files with \f[C]+\f[R] and \f[C]&\f[R] in (Nick
|
|
Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Fix retries of upload chunks (Nick Craig\-Wood)
|
|
.RE
|
|
.IP \[bu] 2
|
|
S3
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Set ACL for server side copies to that provided by the user (Nick
|
|
Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Fix role_arn, credential_source, \&... (Erik Swanson)
|
|
.IP \[bu] 2
|
|
Add config info for Wasabi\[cq]s US\-West endpoint (Henry Ptasinski)
|
|
.RE
|
|
.IP \[bu] 2
|
|
SFTP
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Ensure file hash checking is really disabled (Jon Fautley)
|
|
.RE
|
|
.IP \[bu] 2
|
|
Swift
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Add pacer for retries to make swift more reliable (Nick Craig\-Wood)
|
|
.RE
|
|
.IP \[bu] 2
|
|
WebDAV
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Add Content\-Type to PUT requests (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Fix config parsing so \f[C]\-\-webdav\-user\f[R] and
|
|
\f[C]\-\-webdav\-pass\f[R] flags work (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Add RFC3339 date format (Ralf Hemberger)
|
|
.RE
|
|
.IP \[bu] 2
|
|
Yandex
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
The yandex backend was re\-written (Sebastian B\[:u]nger)
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
This implements low level retries (Sebastian B\[:u]nger)
|
|
.IP \[bu] 2
|
|
Copy, Move, DirMove, PublicLink and About optional interfaces (Sebastian
|
|
B\[:u]nger)
|
|
.IP \[bu] 2
|
|
Improved general error handling (Sebastian B\[:u]nger)
|
|
.IP \[bu] 2
|
|
Removed ListR for now due to inconsistent behaviour (Sebastian
|
|
B\[:u]nger)
|
|
.RE
|
|
.RE
|
|
.SS v1.44 \- 2018\-10\-15
|
|
.IP \[bu] 2
|
|
New commands
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
serve ftp: Add ftp server (Antoine GIRARD)
|
|
.IP \[bu] 2
|
|
settier: perform storage tier changes on supported remotes (sandeepkru)
|
|
.RE
|
|
.IP \[bu] 2
|
|
New Features
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Reworked command line help
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Make default help less verbose (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Split flags up into global and backend flags (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Implement specialised help for flags and backends (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Show URL of backend help page when starting config (Nick Craig\-Wood)
|
|
.RE
|
|
.IP \[bu] 2
|
|
stats: Long names now split in center (Joanna Marek)
|
|
.IP \[bu] 2
|
|
Add \f[C]\-\-log\-format\f[R] flag for more control over log output
|
|
(dcpu)
|
|
.IP \[bu] 2
|
|
rc: Add support for OPTIONS and basic CORS (frenos)
|
|
.IP \[bu] 2
|
|
stats: show FatalErrors and NoRetryErrors in stats (C\['e]dric Connes)
|
|
.RE
|
|
.IP \[bu] 2
|
|
Bug Fixes
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Fix \-P not ending with a new line (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
config: don\[cq]t create default config dir when user supplies
|
|
\f[C]\-\-config\f[R] (albertony)
|
|
.IP \[bu] 2
|
|
Don\[cq]t print non\-ASCII characters with \f[C]\-\-progress\f[R] on
|
|
windows (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Correct logs for excluded items (ssaqua)
|
|
.RE
|
|
.IP \[bu] 2
|
|
Mount
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Remove EXPERIMENTAL tags (Nick Craig\-Wood)
|
|
.RE
|
|
.IP \[bu] 2
|
|
VFS
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Fix race condition detected by serve ftp tests (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Add vfs/poll\-interval rc command (Fabian M\[:o]ller)
|
|
.IP \[bu] 2
|
|
Enable rename for nearly all remotes using server side Move or Copy
|
|
(Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Reduce directory cache cleared by poll\-interval (Fabian M\[:o]ller)
|
|
.IP \[bu] 2
|
|
Remove EXPERIMENTAL tags (Nick Craig\-Wood)
|
|
.RE
|
|
.IP \[bu] 2
|
|
Local
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Skip bad symlinks in dir listing with \-L enabled (C\['e]dric Connes)
|
|
.IP \[bu] 2
|
|
Preallocate files on Windows to reduce fragmentation (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Preallocate files on linux with fallocate(2) (Nick Craig\-Wood)
|
|
.RE
|
|
.IP \[bu] 2
|
|
Cache
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Add cache/fetch rc function (Fabian M\[:o]ller)
|
|
.IP \[bu] 2
|
|
Fix worker scale down (Fabian M\[:o]ller)
|
|
.IP \[bu] 2
|
|
Improve performance by not sending info requests for cached chunks
|
|
(dcpu)
|
|
.IP \[bu] 2
|
|
Fix error return value of cache/fetch rc method (Fabian M\[:o]ller)
|
|
.IP \[bu] 2
|
|
Documentation fix for cache\-chunk\-total\-size (Anagh Kumar Baranwal)
|
|
.IP \[bu] 2
|
|
Preserve leading / in wrapped remote path (Fabian M\[:o]ller)
|
|
.IP \[bu] 2
|
|
Add plex_insecure option to skip certificate validation (Fabian
|
|
M\[:o]ller)
|
|
.IP \[bu] 2
|
|
Remove entries that no longer exist in the source (dcpu)
|
|
.RE
|
|
.IP \[bu] 2
|
|
Crypt
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Preserve leading / in wrapped remote path (Fabian M\[:o]ller)
|
|
.RE
|
|
.IP \[bu] 2
|
|
Alias
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Fix handling of Windows network paths (Nick Craig\-Wood)
|
|
.RE
|
|
.IP \[bu] 2
|
|
Azure Blob
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Add \f[C]\-\-azureblob\-list\-chunk\f[R] parameter (Santiago
|
|
Rodr\['i]guez)
|
|
.IP \[bu] 2
|
|
Implemented settier command support on azureblob remote.
|
|
(sandeepkru)
|
|
.IP \[bu] 2
|
|
Work around SDK bug which causes errors for chunk\-sized files (Nick
|
|
Craig\-Wood)
|
|
.RE
|
|
.IP \[bu] 2
|
|
Box
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Implement link sharing.
|
|
(Sebastian B\[:u]nger)
|
|
.RE
|
|
.IP \[bu] 2
|
|
Drive
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Add \f[C]\-\-drive\-import\-formats\f[R] \- google docs can now be
|
|
imported (Fabian M\[:o]ller)
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Rewrite mime type and extension handling (Fabian M\[:o]ller)
|
|
.IP \[bu] 2
|
|
Add document links (Fabian M\[:o]ller)
|
|
.IP \[bu] 2
|
|
Add support for multipart document extensions (Fabian M\[:o]ller)
|
|
.IP \[bu] 2
|
|
Add support for apps\-script to json export (Fabian M\[:o]ller)
|
|
.IP \[bu] 2
|
|
Fix escaped chars in documents during list (Fabian M\[:o]ller)
|
|
.RE
|
|
.IP \[bu] 2
|
|
Add \f[C]\-\-drive\-v2\-download\-min\-size\f[R] a workaround for slow
|
|
downloads (Fabian M\[:o]ller)
|
|
.IP \[bu] 2
|
|
Improve directory notifications in ChangeNotify (Fabian M\[:o]ller)
|
|
.IP \[bu] 2
|
|
When listing team drives in config, continue on failure (Nick
|
|
Craig\-Wood)
|
|
.RE
|
|
.IP \[bu] 2
|
|
FTP
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Add a small pause after failed upload before deleting file (Nick
|
|
Craig\-Wood)
|
|
.RE
|
|
.IP \[bu] 2
|
|
Google Cloud Storage
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Fix service_account_file being ignored (Fabian M\[:o]ller)
|
|
.RE
|
|
.IP \[bu] 2
|
|
Jottacloud
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Minor improvement in quota info (omit if unlimited) (albertony)
|
|
.IP \[bu] 2
|
|
Add \f[C]\-\-fast\-list\f[R] support (albertony)
|
|
.IP \[bu] 2
|
|
Add permanent delete support: \f[C]\-\-jottacloud\-hard\-delete\f[R]
|
|
(albertony)
|
|
.IP \[bu] 2
|
|
Add link sharing support (albertony)
|
|
.IP \[bu] 2
|
|
Fix handling of reserved characters.
|
|
(Sebastian B\[:u]nger)
|
|
.IP \[bu] 2
|
|
Fix socket leak on Object.Remove (Nick Craig\-Wood)
|
|
.RE
|
|
.IP \[bu] 2
|
|
Onedrive
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Rework to support Microsoft Graph (Cnly)
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
\f[B]NB\f[R] this will require re\-authenticating the remote
|
|
.RE
|
|
.IP \[bu] 2
|
|
Removed upload cutoff and always do session uploads (Oliver Heyme)
|
|
.IP \[bu] 2
|
|
Use single\-part upload for empty files (Cnly)
|
|
.IP \[bu] 2
|
|
Fix new fields not saved when editing old config (Alex Chen)
|
|
.IP \[bu] 2
|
|
Fix sometimes special chars in filenames not replaced (Alex Chen)
|
|
.IP \[bu] 2
|
|
Ignore OneNote files by default (Alex Chen)
|
|
.IP \[bu] 2
|
|
Add link sharing support (jackyzy823)
|
|
.RE
|
|
.IP \[bu] 2
|
|
S3
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Use custom pacer, to retry operations when reasonable (Craig Miskell)
|
|
.IP \[bu] 2
|
|
Use configured server\-side\-encryption and storage class options when
|
|
calling CopyObject() (Paul Kohout)
|
|
.IP \[bu] 2
|
|
Make \f[C]\-\-s3\-v2\-auth\f[R] flag (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Fix v2 auth on files with spaces (Nick Craig\-Wood)
|
|
.RE
|
|
.IP \[bu] 2
|
|
Union
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Implement union backend which reads from multiple backends (Felix
|
|
Brucker)
|
|
.IP \[bu] 2
|
|
Implement optional interfaces (Move, DirMove, Copy etc) (Nick
|
|
Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Fix ChangeNotify to support multiple remotes (Fabian M\[:o]ller)
|
|
.IP \[bu] 2
|
|
Fix \f[C]\-\-backup\-dir\f[R] on union backend (Nick Craig\-Wood)
|
|
.RE
|
|
.IP \[bu] 2
|
|
WebDAV
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Add another time format (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Add a small pause after failed upload before deleting file (Nick
|
|
Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Add workaround for missing mtime (buergi)
|
|
.IP \[bu] 2
|
|
Sharepoint: Renew cookies after 12hrs (Henning Surmeier)
|
|
.RE
|
|
.IP \[bu] 2
|
|
Yandex
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Remove redundant nil checks (teresy)
|
|
.RE
|
|
.SS v1.43.1 \- 2018\-09\-07
|
|
.PP
|
|
Point release to fix hubic and azureblob backends.
|
|
.IP \[bu] 2
|
|
Bug Fixes
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
ncdu: Return error instead of log.Fatal in Show (Fabian M\[:o]ller)
|
|
.IP \[bu] 2
|
|
cmd: Fix crash with \f[C]\-\-progress\f[R] and \f[C]\-\-stats 0\f[R]
|
|
(Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
docs: Tidy website display (Anagh Kumar Baranwal)
|
|
.RE
|
|
.IP \[bu] 2
|
|
Azure Blob:
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Fix multi\-part uploads.
|
|
(sandeepkru)
|
|
.RE
|
|
.IP \[bu] 2
|
|
Hubic
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Fix uploads (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Retry auth fetching if it fails to make hubic more reliable (Nick
|
|
Craig\-Wood)
|
|
.RE
|
|
.SS v1.43 \- 2018\-09\-01
|
|
.IP \[bu] 2
|
|
New backends
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Jottacloud (Sebastian B\[:u]nger)
|
|
.RE
|
|
.IP \[bu] 2
|
|
New commands
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
copyurl: copies a URL to a remote (Denis)
|
|
.RE
|
|
.IP \[bu] 2
|
|
New Features
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Reworked config for backends (Nick Craig\-Wood)
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
All backend config can now be supplied by command line, env var or
|
|
config file
|
|
.IP \[bu] 2
|
|
Advanced section in the config wizard for the optional items
|
|
.IP \[bu] 2
|
|
A large step towards rclone backends being usable in other go software
|
|
.IP \[bu] 2
|
|
Allow on the fly remotes with :backend: syntax
|
|
.RE
|
|
.IP \[bu] 2
|
|
Stats revamp
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Add \f[C]\-\-progress\f[R]/\f[C]\-P\f[R] flag to show interactive
|
|
progress (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Show the total progress of the sync in the stats (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Add \f[C]\-\-stats\-one\-line\f[R] flag for single line stats (Nick
|
|
Craig\-Wood)
|
|
.RE
|
|
.IP \[bu] 2
|
|
Added weekday schedule into \f[C]\-\-bwlimit\f[R] (Mateusz)
|
|
.IP \[bu] 2
|
|
lsjson: Add option to show the original object IDs (Fabian M\[:o]ller)
|
|
.IP \[bu] 2
|
|
serve webdav: Make Content\-Type without reading the file and add
|
|
\f[C]\-\-etag\-hash\f[R] (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
build
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Build macOS with native compiler (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Update to use go1.11 for the build (Nick Craig\-Wood)
|
|
.RE
|
|
.IP \[bu] 2
|
|
rc
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Added core/stats to return the stats (reddi1)
|
|
.RE
|
|
.IP \[bu] 2
|
|
\f[C]version \-\-check\f[R]: Prints the current release and beta
|
|
versions (Nick Craig\-Wood)
|
|
.RE
|
|
.IP \[bu] 2
|
|
Bug Fixes
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
accounting
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Fix time to completion estimates (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Fix moving average speed for file stats (Nick Craig\-Wood)
|
|
.RE
|
|
.IP \[bu] 2
|
|
config: Fix error reading password from piped input (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
move: Fix \f[C]\-\-delete\-empty\-src\-dirs\f[R] flag to delete all
|
|
empty dirs on move (ishuah)
|
|
.RE
|
|
.IP \[bu] 2
|
|
Mount
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Implement \f[C]\-\-daemon\-timeout\f[R] flag for OSXFUSE (Nick
|
|
Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Fix mount \f[C]\-\-daemon\f[R] not working with encrypted config (Alex
|
|
Chen)
|
|
.IP \[bu] 2
|
|
Clip the number of blocks to 2\[ha]32\-1 on macOS \- fixes borg backup
|
|
(Nick Craig\-Wood)
|
|
.RE
|
|
.IP \[bu] 2
|
|
VFS
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Enable vfs\-read\-chunk\-size by default (Fabian M\[:o]ller)
|
|
.IP \[bu] 2
|
|
Add the vfs/refresh rc command (Fabian M\[:o]ller)
|
|
.IP \[bu] 2
|
|
Add non recursive mode to vfs/refresh rc command (Fabian M\[:o]ller)
|
|
.IP \[bu] 2
|
|
Try to seek buffer on read only files (Fabian M\[:o]ller)
|
|
.RE
|
|
.IP \[bu] 2
|
|
Local
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Fix crash when deprecated
|
|
\f[C]\-\-local\-no\-unicode\-normalization\f[R] is supplied (Nick
|
|
Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Fix mkdir error when trying to copy files to the root of a drive on
|
|
windows (Nick Craig\-Wood)
|
|
.RE
|
|
.IP \[bu] 2
|
|
Cache
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Fix nil pointer deref when using lsjson on cached directory (Nick
|
|
Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Fix nil pointer deref for occasional crash on playback (Nick
|
|
Craig\-Wood)
|
|
.RE
|
|
.IP \[bu] 2
|
|
Crypt
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Fix accounting when checking hashes on upload (Nick Craig\-Wood)
|
|
.RE
|
|
.IP \[bu] 2
|
|
Amazon Cloud Drive
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Make very clear in the docs that rclone has no ACD keys (Nick
|
|
Craig\-Wood)
|
|
.RE
|
|
.IP \[bu] 2
|
|
Azure Blob
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Add connection string and SAS URL auth (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
List the container to see if it exists (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Port new Azure Blob Storage SDK (sandeepkru)
|
|
.IP \[bu] 2
|
|
Added blob tier, tier between Hot, Cool and Archive.
|
|
(sandeepkru)
|
|
.IP \[bu] 2
|
|
Remove leading / from paths (Nick Craig\-Wood)
|
|
.RE
|
|
.IP \[bu] 2
|
|
B2
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Support Application Keys (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Remove leading / from paths (Nick Craig\-Wood)
|
|
.RE
|
|
.IP \[bu] 2
|
|
Box
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Fix upload of > 2GB files on 32 bit platforms (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Make \f[C]\-\-box\-commit\-retries\f[R] flag defaulting to 100 to fix
|
|
large uploads (Nick Craig\-Wood)
|
|
.RE
|
|
.IP \[bu] 2
|
|
Drive
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Add \f[C]\-\-drive\-keep\-revision\-forever\f[R] flag (lewapm)
|
|
.IP \[bu] 2
|
|
Handle gdocs when filtering file names in list (Fabian M\[:o]ller)
|
|
.IP \[bu] 2
|
|
Support using \f[C]\-\-fast\-list\f[R] for large speedups (Fabian
|
|
M\[:o]ller)
|
|
.RE
|
|
.IP \[bu] 2
|
|
FTP
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Fix Put mkParentDir failed: 521 for BunnyCDN (Nick Craig\-Wood)
|
|
.RE
|
|
.IP \[bu] 2
|
|
Google Cloud Storage
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Fix index out of range error with \f[C]\-\-fast\-list\f[R] (Nick
|
|
Craig\-Wood)
|
|
.RE
|
|
.IP \[bu] 2
|
|
Jottacloud
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Fix MD5 error check (Oliver Heyme)
|
|
.IP \[bu] 2
|
|
Handle empty time values (Martin Polden)
|
|
.IP \[bu] 2
|
|
Calculate missing MD5s (Oliver Heyme)
|
|
.IP \[bu] 2
|
|
Docs, fixes and tests for MD5 calculation (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Add optional MimeTyper interface.
|
|
(Sebastian B\[:u]nger)
|
|
.IP \[bu] 2
|
|
Implement optional About interface (for \f[C]df\f[R] support).
|
|
(Sebastian B\[:u]nger)
|
|
.RE
|
|
.IP \[bu] 2
|
|
Mega
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Wait for events instead of arbitrary sleeping (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Add \f[C]\-\-mega\-hard\-delete\f[R] flag (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Fix failed logins with upper case chars in email (Nick Craig\-Wood)
|
|
.RE
|
|
.IP \[bu] 2
|
|
Onedrive
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Shared folder support (Yoni Jah)
|
|
.IP \[bu] 2
|
|
Implement DirMove (Cnly)
|
|
.IP \[bu] 2
|
|
Fix rmdir sometimes deleting directories with contents (Nick
|
|
Craig\-Wood)
|
|
.RE
|
|
.IP \[bu] 2
|
|
Pcloud
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Delete half uploaded files on upload error (Nick Craig\-Wood)
|
|
.RE
|
|
.IP \[bu] 2
|
|
Qingstor
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Remove leading / from paths (Nick Craig\-Wood)
|
|
.RE
|
|
.IP \[bu] 2
|
|
S3
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Fix index out of range error with \f[C]\-\-fast\-list\f[R] (Nick
|
|
Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Add \f[C]\-\-s3\-force\-path\-style\f[R] (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Add support for KMS Key ID (bsteiss)
|
|
.IP \[bu] 2
|
|
Remove leading / from paths (Nick Craig\-Wood)
|
|
.RE
|
|
.IP \[bu] 2
|
|
Swift
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Add \f[C]storage_policy\f[R] (Ruben Vandamme)
|
|
.IP \[bu] 2
|
|
Make it so just \f[C]storage_url\f[R] or \f[C]auth_token\f[R] can be
|
|
overridden (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Fix server side copy bug for unusual file names (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Remove leading / from paths (Nick Craig\-Wood)
|
|
.RE
|
|
.IP \[bu] 2
|
|
WebDAV
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Ensure we call MKCOL with a URL with a trailing / for QNAP interop (Nick
|
|
Craig\-Wood)
|
|
.IP \[bu] 2
|
|
If root ends with / then don\[cq]t check if it is a file (Nick
|
|
Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Don\[cq]t accept redirects when reading metadata (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Add bearer token (Macaroon) support for dCache (Nick Craig\-Wood)
|
|
.IP \[bu] 2
|
|
Document dCache and Macaroons (Onno Zweers)
|
|
.IP \[bu] 2
|
|
Sharepoint recursion with different depth (Henning)
|
|
.IP \[bu] 2
|
|
Attempt to remove failed uploads (Nick Craig\-Wood)
|
|
.RE
|
|
.IP \[bu] 2
|
|
Yandex
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Fix listing/deleting files in the root (Nick Craig\-Wood)
|
|
.RE
|
|
.SS v1.42 \- 2018\-06\-16
|
|
.IP \[bu] 2
|
|
New backends
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
OpenDrive (Oliver Heyme, Jakub Karlicek, ncw)
|
|
.RE
|
|
.IP \[bu] 2
|
|
New commands
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
deletefile command (Filip Bartodziej)
|
|
.RE
|
|
.IP \[bu] 2
|
|
New Features
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
copy, move: Copy single files directly, don\[cq]t use
|
|
\f[C]\-\-files\-from\f[R] work\-around
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
this makes them much more efficient
|
|
.RE
|
|
.IP \[bu] 2
|
|
Implement \f[C]\-\-max\-transfer\f[R] flag to quit transferring at a
|
|
limit
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
make exit code 8 for \f[C]\-\-max\-transfer\f[R] exceeded
|
|
.RE
|
|
.IP \[bu] 2
|
|
copy: copy empty source directories to destination (Ishuah Kariuki)
|
|
.IP \[bu] 2
|
|
check: Add \f[C]\-\-one\-way\f[R] flag (Kasper Byrdal Nielsen)
|
|
.IP \[bu] 2
|
|
Add siginfo handler for macOS for ctrl\-T stats (kubatasiemski)
|
|
.IP \[bu] 2
|
|
rc
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
add core/gc to run a garbage collection on demand
|
|
.IP \[bu] 2
|
|
enable go profiling by default on the \f[C]\-\-rc\f[R] port
|
|
.IP \[bu] 2
|
|
return error from remote on failure
|
|
.RE
|
|
.IP \[bu] 2
|
|
lsf
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Add \f[C]\-\-absolute\f[R] flag to add a leading / onto path names
|
|
.IP \[bu] 2
|
|
Add \f[C]\-\-csv\f[R] flag for compliant CSV output
|
|
.IP \[bu] 2
|
|
Add `m' format specifier to show the MimeType
|
|
.IP \[bu] 2
|
|
Implement `i' format for showing object ID
|
|
.RE
|
|
.IP \[bu] 2
|
|
lsjson
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Add MimeType to the output
|
|
.IP \[bu] 2
|
|
Add ID field to output to show Object ID
|
|
.RE
|
|
.IP \[bu] 2
|
|
Add \f[C]\-\-retries\-sleep\f[R] flag (Benjamin Joseph Dag)
|
|
.IP \[bu] 2
|
|
Oauth tidy up web page and error handling (Henning Surmeier)
|
|
.RE
|
|
.IP \[bu] 2
|
|
Bug Fixes
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Password prompt output with \f[C]\-\-log\-file\f[R] fixed for unix
|
|
(Filip Bartodziej)
|
|
.IP \[bu] 2
|
|
Calculate ModifyWindow each time on the fly to fix various problems
|
|
(Stefan Breunig)
|
|
.RE
|
|
.IP \[bu] 2
|
|
Mount
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Only print \[lq]File.rename error\[rq] if there actually is an error
|
|
(Stefan Breunig)
|
|
.IP \[bu] 2
|
|
Delay rename if file has open writers instead of failing outright
|
|
(Stefan Breunig)
|
|
.IP \[bu] 2
|
|
Ensure atexit gets run on interrupt
|
|
.IP \[bu] 2
|
|
macOS enhancements
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Make \f[C]\-\-noappledouble\f[R] \f[C]\-\-noapplexattr\f[R]
|
|
.IP \[bu] 2
|
|
Add \f[C]\-\-volname\f[R] flag and remove special chars from it
|
|
.IP \[bu] 2
|
|
Make Get/List/Set/Remove xattr return ENOSYS for efficiency
|
|
.IP \[bu] 2
|
|
Make \f[C]\-\-daemon\f[R] work for macOS without CGO
|
|
.RE
|
|
.RE
|
|
.IP \[bu] 2
|
|
VFS
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Add \f[C]\-\-vfs\-read\-chunk\-size\f[R] and
|
|
\f[C]\-\-vfs\-read\-chunk\-size\-limit\f[R] (Fabian M\[:o]ller)
|
|
.IP \[bu] 2
|
|
Fix ChangeNotify for new or changed folders (Fabian M\[:o]ller)
|
|
.RE
|
|
.IP \[bu] 2
|
|
Local
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Fix symlink/junction point directory handling under Windows
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
\f[B]NB\f[R] you will need to add \f[C]\-L\f[R] to your command line to
|
|
copy files with reparse points
|
|
.RE
|
|
.RE
|
|
.IP \[bu] 2
|
|
Cache
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Add non cached dirs on notifications (Remus Bunduc)
|
|
.IP \[bu] 2
|
|
Allow root to be expired from rc (Remus Bunduc)
|
|
.IP \[bu] 2
|
|
Clean remaining empty folders from temp upload path (Remus Bunduc)
|
|
.IP \[bu] 2
|
|
Cache lists using batch writes (Remus Bunduc)
|
|
.IP \[bu] 2
|
|
Use secure websockets for HTTPS Plex addresses (John Clayton)
|
|
.IP \[bu] 2
|
|
Reconnect plex websocket on failures (Remus Bunduc)
|
|
.IP \[bu] 2
|
|
Fix panic when running without plex configs (Remus Bunduc)
|
|
.IP \[bu] 2
|
|
Fix root folder caching (Remus Bunduc)
|
|
.RE
|
|
.IP \[bu] 2
|
|
Crypt
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Check the crypted hash of files when uploading for extra data security
|
|
.RE
|
|
.IP \[bu] 2
|
|
Dropbox
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Make Dropbox for business folders accessible using an initial
|
|
\f[C]/\f[R] in the path
|
|
.RE
|
|
.IP \[bu] 2
|
|
Google Cloud Storage
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Low level retry all operations if necessary
|
|
.RE
|
|
.IP \[bu] 2
|
|
Google Drive
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Add \f[C]\-\-drive\-acknowledge\-abuse\f[R] to download flagged files
|
|
.IP \[bu] 2
|
|
Add \f[C]\-\-drive\-alternate\-export\f[R] to fix large doc export
|
|
.IP \[bu] 2
|
|
Don\[cq]t attempt to choose Team Drives when using rclone config create
|
|
.IP \[bu] 2
|
|
Fix change list polling with team drives
|
|
.IP \[bu] 2
|
|
Fix ChangeNotify for folders (Fabian M\[:o]ller)
|
|
.IP \[bu] 2
|
|
Fix about (and df on a mount) for team drives
|
|
.RE
|
|
.IP \[bu] 2
|
|
Onedrive
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Errorhandler for onedrive for business requests (Henning Surmeier)
|
|
.RE
|
|
.IP \[bu] 2
|
|
S3
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Adjust upload concurrency with \f[C]\-\-s3\-upload\-concurrency\f[R]
|
|
(themylogin)
|
|
.IP \[bu] 2
|
|
Fix \f[C]\-\-s3\-chunk\-size\f[R] which was always using the minimum
|
|
.RE
|
|
.IP \[bu] 2
|
|
SFTP
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Add \f[C]\-\-ssh\-path\-override\f[R] flag (Piotr Oleszczyk)
|
|
.IP \[bu] 2
|
|
Fix slow downloads for long latency connections
|
|
.RE
|
|
.IP \[bu] 2
|
|
Webdav
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Add workarounds for biz.mail.ru
|
|
.IP \[bu] 2
|
|
Ignore Reason\-Phrase in status line to fix 4shared (Rodrigo)
|
|
.IP \[bu] 2
|
|
Better error message generation
|
|
.RE
|
|
.SS v1.41 \- 2018\-04\-28
|
|
.IP \[bu] 2
|
|
New backends
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Mega support added
|
|
.IP \[bu] 2
|
|
Webdav now supports SharePoint cookie authentication (hensur)
|
|
.RE
|
|
.IP \[bu] 2
|
|
New commands
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
link: create public link to files and folders (Stefan Breunig)
|
|
.IP \[bu] 2
|
|
about: gets quota info from a remote (a\-roussos, ncw)
|
|
.IP \[bu] 2
|
|
hashsum: a generic tool for any hash to produce md5sum like output
|
|
.RE
|
|
.IP \[bu] 2
|
|
New Features
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
lsd: Add \-R flag and fix and update docs for all ls commands
|
|
.IP \[bu] 2
|
|
ncdu: added a \[lq]refresh\[rq] key \- CTRL\-L (Keith Goldfarb)
|
|
.IP \[bu] 2
|
|
serve restic: Add append\-only mode (Steve Kriss)
|
|
.IP \[bu] 2
|
|
serve restic: Disallow overwriting files in append\-only mode (Alexander
|
|
Neumann)
|
|
.IP \[bu] 2
|
|
serve restic: Print actual listener address (Matt Holt)
|
|
.IP \[bu] 2
|
|
size: Add \[en]json flag (Matthew Holt)
|
|
.IP \[bu] 2
|
|
sync: implement \[en]ignore\-errors (Mateusz Pabian)
|
|
.IP \[bu] 2
|
|
dedupe: Add dedupe largest functionality (Richard Yang)
|
|
.IP \[bu] 2
|
|
fs: Extend SizeSuffix to include TB and PB for rclone about
|
|
.IP \[bu] 2
|
|
fs: add \[en]dump goroutines and \[en]dump openfiles for debugging
|
|
.IP \[bu] 2
|
|
rc: implement core/memstats to print internal memory usage info
|
|
.IP \[bu] 2
|
|
rc: new call rc/pid (Michael P.
|
|
Dubner)
|
|
.RE
|
|
.IP \[bu] 2
|
|
Compile
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Drop support for go1.6
|
|
.RE
|
|
.IP \[bu] 2
|
|
Release
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Fix \f[C]make tarball\f[R] (Chih\-Hsuan Yen)
|
|
.RE
|
|
.IP \[bu] 2
|
|
Bug Fixes
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
filter: fix \[en]min\-age and \[en]max\-age together check
|
|
.IP \[bu] 2
|
|
fs: limit MaxIdleConns and MaxIdleConnsPerHost in transport
|
|
.IP \[bu] 2
|
|
lsd,lsf: make sure all times we output are in local time
|
|
.IP \[bu] 2
|
|
rc: fix setting bwlimit to unlimited
|
|
.IP \[bu] 2
|
|
rc: take note of the \[en]rc\-addr flag too as per the docs
|
|
.RE
|
|
.IP \[bu] 2
|
|
Mount
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Use About to return the correct disk total/used/free (eg in
|
|
\f[C]df\f[R])
|
|
.IP \[bu] 2
|
|
Set \f[C]\-\-attr\-timeout default\f[R] to \f[C]1s\f[R] \- fixes:
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
rclone using too much memory
|
|
.IP \[bu] 2
|
|
rclone not serving files to samba
|
|
.IP \[bu] 2
|
|
excessive time listing directories
|
|
.RE
|
|
.IP \[bu] 2
|
|
Fix \f[C]df \-i\f[R] (upstream fix)
|
|
.RE
|
|
.IP \[bu] 2
|
|
VFS
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Filter files \f[C].\f[R] and \f[C]..\f[R] from directory listing
|
|
.IP \[bu] 2
|
|
Only make the VFS cache if \[en]vfs\-cache\-mode > Off
|
|
.RE
|
|
.IP \[bu] 2
|
|
Local
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Add \[en]local\-no\-check\-updated to disable updated file checks
|
|
.IP \[bu] 2
|
|
Retry remove on Windows sharing violation error
|
|
.RE
|
|
.IP \[bu] 2
|
|
Cache
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Flush the memory cache after close
|
|
.IP \[bu] 2
|
|
Purge file data on notification
|
|
.IP \[bu] 2
|
|
Always forget parent dir for notifications
|
|
.IP \[bu] 2
|
|
Integrate with Plex websocket
|
|
.IP \[bu] 2
|
|
Add rc cache/stats (seuffert)
|
|
.IP \[bu] 2
|
|
Add info log on notification
|
|
.RE
|
|
.IP \[bu] 2
|
|
Box
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Fix failure reading large directories \- parse file/directory size as
|
|
float
|
|
.RE
|
|
.IP \[bu] 2
|
|
Dropbox
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Fix crypt+obfuscate on dropbox
|
|
.IP \[bu] 2
|
|
Fix repeatedly uploading the same files
|
|
.RE
|
|
.IP \[bu] 2
|
|
FTP
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Work around strange response from box FTP server
|
|
.IP \[bu] 2
|
|
More workarounds for FTP servers to fix mkParentDir error
|
|
.IP \[bu] 2
|
|
Fix no error on listing non\-existent directory
|
|
.RE
|
|
.IP \[bu] 2
|
|
Google Cloud Storage
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Add service_account_credentials (Matt Holt)
|
|
.IP \[bu] 2
|
|
Detect bucket presence by listing it \- minimises permissions needed
|
|
.IP \[bu] 2
|
|
Ignore zero length directory markers
|
|
.RE
|
|
.IP \[bu] 2
|
|
Google Drive
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Add service_account_credentials (Matt Holt)
|
|
.IP \[bu] 2
|
|
Fix directory move leaving a hardlinked directory behind
|
|
.IP \[bu] 2
|
|
Return proper google errors when Opening files
|
|
.IP \[bu] 2
|
|
When initialized with a filepath, optional features used incorrect root
|
|
path (Stefan Breunig)
|
|
.RE
|
|
.IP \[bu] 2
|
|
HTTP
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Fix sync for servers which don\[cq]t return Content\-Length in HEAD
|
|
.RE
|
|
.IP \[bu] 2
|
|
Onedrive
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Add QuickXorHash support for OneDrive for business
|
|
.IP \[bu] 2
|
|
Fix socket leak in multipart session upload
|
|
.RE
|
|
.IP \[bu] 2
|
|
S3
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Look in S3 named profile files for credentials
|
|
.IP \[bu] 2
|
|
Add \f[C]\-\-s3\-disable\-checksum\f[R] to disable checksum uploading
|
|
(Chris Redekop)
|
|
.IP \[bu] 2
|
|
Hierarchical configuration support (Giri Badanahatti)
|
|
.IP \[bu] 2
|
|
Add in config for all the supported S3 providers
|
|
.IP \[bu] 2
|
|
Add One Zone Infrequent Access storage class (Craig Rachel)
|
|
.IP \[bu] 2
|
|
Add \[en]use\-server\-modtime support (Peter Baumgartner)
|
|
.IP \[bu] 2
|
|
Add \[en]s3\-chunk\-size option to control multipart uploads
|
|
.IP \[bu] 2
|
|
Ignore zero length directory markers
|
|
.RE
|
|
.IP \[bu] 2
|
|
SFTP
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Update docs to match code, fix typos and clarify disable_hashcheck
|
|
prompt (Michael G.
|
|
Noll)
|
|
.IP \[bu] 2
|
|
Update docs with Synology quirks
|
|
.IP \[bu] 2
|
|
Fail soft with a debug on hash failure
|
|
.RE
|
|
.IP \[bu] 2
|
|
Swift
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Add \[en]use\-server\-modtime support (Peter Baumgartner)
|
|
.RE
|
|
.IP \[bu] 2
|
|
Webdav
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Support SharePoint cookie authentication (hensur)
|
|
.IP \[bu] 2
|
|
Strip leading and trailing / off root
|
|
.RE
|
|
.SS v1.40 \- 2018\-03\-19
|
|
.IP \[bu] 2
|
|
New backends
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Alias backend to create aliases for existing remote names (Fabian
|
|
M\[:o]ller)
|
|
.RE
|
|
.IP \[bu] 2
|
|
New commands
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
\f[C]lsf\f[R]: list for parsing purposes (Jakub Tasiemski)
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
by default this is a simple non recursive list of files and directories
|
|
.IP \[bu] 2
|
|
it can be configured to add more info in an easy to parse way
|
|
.RE
|
|
.IP \[bu] 2
|
|
\f[C]serve restic\f[R]: for serving a remote as a Restic REST endpoint
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
This enables restic to use any backends that rclone can access
|
|
.IP \[bu] 2
|
|
Thanks Alexander Neumann for help, patches and review
|
|
.RE
|
|
.IP \[bu] 2
|
|
\f[C]rc\f[R]: enable the remote control of a running rclone
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
The running rclone must be started with \[en]rc and related flags.
|
|
.IP \[bu] 2
|
|
Currently there is support for bwlimit, and flushing for mount and
|
|
cache.
|
|
.RE
|
|
.RE
|
|
.IP \[bu] 2
|
|
New Features
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
\f[C]\-\-max\-delete\f[R] flag to add a delete threshold (Bj\[/o]rn Erik
|
|
Pedersen)
|
|
.IP \[bu] 2
|
|
All backends now support RangeOption for ranged Open
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
\f[C]cat\f[R]: Use RangeOption for limited fetches to make more
|
|
efficient
|
|
.IP \[bu] 2
|
|
\f[C]cryptcheck\f[R]: make reading of nonce more efficient with
|
|
RangeOption
|
|
.RE
|
|
.IP \[bu] 2
|
|
serve http/webdav/restic
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
support SSL/TLS
|
|
.IP \[bu] 2
|
|
add \f[C]\-\-user\f[R] \f[C]\-\-pass\f[R] and \f[C]\-\-htpasswd\f[R] for
|
|
authentication
|
|
.RE
|
|
.IP \[bu] 2
|
|
\f[C]copy\f[R]/\f[C]move\f[R]: detect file size change during copy/move
|
|
and abort transfer (ishuah)
|
|
.IP \[bu] 2
|
|
\f[C]cryptdecode\f[R]: added option to return encrypted file names.
|
|
(ishuah)
|
|
.IP \[bu] 2
|
|
\f[C]lsjson\f[R]: add \f[C]\-\-encrypted\f[R] to show encrypted name
|
|
(Jakub Tasiemski)
|
|
.IP \[bu] 2
|
|
Add \f[C]\-\-stats\-file\-name\-length\f[R] to specify the printed file
|
|
name length for stats (Will Gunn)
|
|
.RE
|
|
.IP \[bu] 2
|
|
Compile
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Code base was shuffled and factored
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
backends moved into a backend directory
|
|
.IP \[bu] 2
|
|
large packages split up
|
|
.IP \[bu] 2
|
|
See the CONTRIBUTING.md doc for info as to what lives where now
|
|
.RE
|
|
.IP \[bu] 2
|
|
Update to using go1.10 as the default go version
|
|
.IP \[bu] 2
|
|
Implement daily full integration
|
|
tests (https://pub.rclone.org/integration-tests/)
|
|
.RE
|
|
.IP \[bu] 2
|
|
Release
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Include a source tarball and sign it and the binaries
|
|
.IP \[bu] 2
|
|
Sign the git tags as part of the release process
|
|
.IP \[bu] 2
|
|
Add .deb and .rpm packages as part of the build
|
|
.IP \[bu] 2
|
|
Make a beta release for all branches on the main repo (but not pull
|
|
requests)
|
|
.RE
|
|
.IP \[bu] 2
|
|
Bug Fixes
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
config: fixes errors on non existing config by loading config file only
|
|
on first access
|
|
.IP \[bu] 2
|
|
config: retry saving the config after failure (Mateusz)
|
|
.IP \[bu] 2
|
|
sync: when using \f[C]\-\-backup\-dir\f[R] don\[cq]t delete files if we
|
|
can\[cq]t set their modtime
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
this fixes odd behaviour with Dropbox and \f[C]\-\-backup\-dir\f[R]
|
|
.RE
|
|
.IP \[bu] 2
|
|
fshttp: fix idle timeouts for HTTP connections
|
|
.IP \[bu] 2
|
|
\f[C]serve http\f[R]: fix serving files with : in \- fixes
|
|
.IP \[bu] 2
|
|
Fix \f[C]\-\-exclude\-if\-present\f[R] to ignore directories which it
|
|
doesn\[cq]t have permission for (Iakov Davydov)
|
|
.IP \[bu] 2
|
|
Make accounting work properly with crypt and b2
|
|
.IP \[bu] 2
|
|
remove \f[C]\-\-no\-traverse\f[R] flag because it is obsolete
|
|
.RE
|
|
.IP \[bu] 2
|
|
Mount
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Add \f[C]\-\-attr\-timeout\f[R] flag to control attribute caching in
|
|
kernel
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
this now defaults to 0 which is correct but less efficient
|
|
.IP \[bu] 2
|
|
see the mount
|
|
docs (https://rclone.org/commands/rclone_mount/#attribute-caching) for
|
|
more info
|
|
.RE
|
|
.IP \[bu] 2
|
|
Add \f[C]\-\-daemon\f[R] flag to allow mount to run in the background
|
|
(ishuah)
|
|
.IP \[bu] 2
|
|
Fix: Return ENOSYS rather than EIO on attempted link
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
This fixes FileZilla accessing an rclone mount served over sftp.
|
|
.RE
|
|
.IP \[bu] 2
|
|
Fix setting modtime twice
|
|
.IP \[bu] 2
|
|
Mount tests now run on CI for Linux (mount & cmount)/Mac/Windows
|
|
.IP \[bu] 2
|
|
Many bugs fixed in the VFS layer \- see below
|
|
.RE
|
|
.IP \[bu] 2
|
|
VFS
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Many fixes for \f[C]\-\-vfs\-cache\-mode\f[R] writes and above
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Update cached copy if we know it has changed (fixes stale data)
|
|
.IP \[bu] 2
|
|
Clean path names before using them in the cache
|
|
.IP \[bu] 2
|
|
Disable cache cleaner if \f[C]\-\-vfs\-cache\-poll\-interval=0\f[R]
|
|
.IP \[bu] 2
|
|
Fill and clean the cache immediately on startup
|
|
.RE
|
|
.IP \[bu] 2
|
|
Fix Windows opening every file when it stats the file
|
|
.IP \[bu] 2
|
|
Fix applying modtime for an open Write Handle
|
|
.IP \[bu] 2
|
|
Fix creation of files when truncating
|
|
.IP \[bu] 2
|
|
Write 0 bytes when flushing unwritten handles to avoid race conditions
|
|
in FUSE
|
|
.IP \[bu] 2
|
|
Downgrade \[lq]poll\-interval is not supported\[rq] message to Info
|
|
.IP \[bu] 2
|
|
Make OpenFile and friends return EINVAL if O_RDONLY and O_TRUNC
|
|
.RE
|
|
.IP \[bu] 2
|
|
Local
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Downgrade \[lq]invalid cross\-device link: trying copy\[rq] to debug
|
|
.IP \[bu] 2
|
|
Make DirMove return fs.ErrorCantDirMove to allow fallback to Copy for
|
|
cross device
|
|
.IP \[bu] 2
|
|
Fix race conditions updating the hashes
|
|
.RE
|
|
.IP \[bu] 2
|
|
Cache
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Add support for polling \- cache will update when remote changes on
|
|
supported backends
|
|
.IP \[bu] 2
|
|
Reduce log level for Plex api
|
|
.IP \[bu] 2
|
|
Fix dir cache issue
|
|
.IP \[bu] 2
|
|
Implement \f[C]\-\-cache\-db\-wait\-time\f[R] flag
|
|
.IP \[bu] 2
|
|
Improve efficiency with RangeOption and RangeSeek
|
|
.IP \[bu] 2
|
|
Fix dirmove with temp fs enabled
|
|
.IP \[bu] 2
|
|
Notify vfs when using temp fs
|
|
.IP \[bu] 2
|
|
Offline uploading
|
|
.IP \[bu] 2
|
|
Remote control support for path flushing
|
|
.RE
|
|
.IP \[bu] 2
|
|
Amazon cloud drive
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Rclone no longer has any working keys \- disable integration tests
|
|
.IP \[bu] 2
|
|
Implement DirChangeNotify to notify cache/vfs/mount of changes
|
|
.RE
|
|
.IP \[bu] 2
|
|
Azureblob
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Don\[cq]t check for bucket/container presence if listing was OK
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
this makes rclone do one less request per invocation
|
|
.RE
|
|
.IP \[bu] 2
|
|
Improve accounting for chunked uploads
|
|
.RE
|
|
.IP \[bu] 2
|
|
Backblaze B2
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Don\[cq]t check for bucket/container presence if listing was OK
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
this makes rclone do one less request per invocation
|
|
.RE
|
|
.RE
|
|
.IP \[bu] 2
|
|
Box
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Improve accounting for chunked uploads
|
|
.RE
|
|
.IP \[bu] 2
|
|
Dropbox
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Fix custom oauth client parameters
|
|
.RE
|
|
.IP \[bu] 2
|
|
Google Cloud Storage
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Don\[cq]t check for bucket/container presence if listing was OK
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
this makes rclone do one less request per invocation
|
|
.RE
|
|
.RE
|
|
.IP \[bu] 2
|
|
Google Drive
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Migrate to api v3 (Fabian M\[:o]ller)
|
|
.IP \[bu] 2
|
|
Add scope configuration and root folder selection
|
|
.IP \[bu] 2
|
|
Add \f[C]\-\-drive\-impersonate\f[R] for service accounts
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
thanks to everyone who tested, explored and contributed docs
|
|
.RE
|
|
.IP \[bu] 2
|
|
Add \f[C]\-\-drive\-use\-created\-date\f[R] to use created date as
|
|
modified date (nbuchanan)
|
|
.IP \[bu] 2
|
|
Request the export formats only when required
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
This makes rclone quicker when there are no google docs
|
|
.RE
|
|
.IP \[bu] 2
|
|
Fix finding paths with latin1 chars (a workaround for a drive bug)
|
|
.IP \[bu] 2
|
|
Fix copying of a single Google doc file
|
|
.IP \[bu] 2
|
|
Fix \f[C]\-\-drive\-auth\-owner\-only\f[R] to look in all directories
|
|
.RE
|
|
.IP \[bu] 2
|
|
HTTP
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Fix handling of directories with & in
|
|
.RE
|
|
.IP \[bu] 2
|
|
Onedrive
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Removed upload cutoff and always do session uploads
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
this stops the creation of multiple versions on business onedrive
|
|
.RE
|
|
.IP \[bu] 2
|
|
Overwrite object size value with real size when reading file.
|
|
(Victor)
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
this fixes oddities when onedrive misreports the size of images
|
|
.RE
|
|
.RE
|
|
.IP \[bu] 2
|
|
Pcloud
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Remove unused chunked upload flag and code
|
|
.RE
|
|
.IP \[bu] 2
|
|
Qingstor
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Don\[cq]t check for bucket/container presence if listing was OK
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
this makes rclone do one less request per invocation
|
|
.RE
|
|
.RE
|
|
.IP \[bu] 2
|
|
S3
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Support hashes for multipart files (Chris Redekop)
|
|
.IP \[bu] 2
|
|
Initial support for IBM COS (S3) (Giri Badanahatti)
|
|
.IP \[bu] 2
|
|
Update docs to discourage use of v2 auth with CEPH and others
|
|
.IP \[bu] 2
|
|
Don\[cq]t check for bucket/container presence if listing was OK
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
this makes rclone do one less request per invocation
|
|
.RE
|
|
.IP \[bu] 2
|
|
Fix server side copy and set modtime on files with + in
|
|
.RE
|
|
.IP \[bu] 2
|
|
SFTP
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Add option to disable remote hash check command execution (Jon Fautley)
|
|
.IP \[bu] 2
|
|
Add \f[C]\-\-sftp\-ask\-password\f[R] flag to prompt for password when
|
|
needed (Leo R.
|
|
Lundgren)
|
|
.IP \[bu] 2
|
|
Add \f[C]set_modtime\f[R] configuration option
|
|
.IP \[bu] 2
|
|
Fix following of symlinks
|
|
.IP \[bu] 2
|
|
Fix reading config file outside of Fs setup
|
|
.IP \[bu] 2
|
|
Fix reading $USER in username fallback not $HOME
|
|
.IP \[bu] 2
|
|
Fix running under crontab \- Use correct OS way of reading username
|
|
.RE
|
|
.IP \[bu] 2
|
|
Swift
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Fix refresh of authentication token
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
in v1.39 a bug was introduced which ignored new tokens \- this fixes it
|
|
.RE
|
|
.IP \[bu] 2
|
|
Fix extra HEAD transaction when uploading a new file
|
|
.IP \[bu] 2
|
|
Don\[cq]t check for bucket/container presence if listing was OK
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
this makes rclone do one less request per invocation
|
|
.RE
|
|
.RE
|
|
.IP \[bu] 2
|
|
Webdav
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Add new time formats to support mydrive.ch and others
|
|
.RE
|
|
.SS v1.39 \- 2017\-12\-23
|
|
.IP \[bu] 2
|
|
New backends
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
WebDAV
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
tested with nextcloud, owncloud, put.io and others!
|
|
.RE
|
|
.IP \[bu] 2
|
|
Pcloud
|
|
.IP \[bu] 2
|
|
cache \- wraps a cache around other backends (Remus Bunduc)
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
useful in combination with mount
|
|
.IP \[bu] 2
|
|
NB this feature is in beta so use with care
|
|
.RE
|
|
.RE
|
|
.IP \[bu] 2
|
|
New commands
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
serve command with subcommands:
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
serve webdav: this implements a webdav server for any rclone remote.
|
|
.IP \[bu] 2
|
|
serve http: command to serve a remote over HTTP
|
|
.RE
|
|
.IP \[bu] 2
|
|
config: add sub commands for full config file management
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
create/delete/dump/edit/file/password/providers/show/update
|
|
.RE
|
|
.IP \[bu] 2
|
|
touch: to create or update the timestamp of a file (Jakub Tasiemski)
|
|
.RE
|
|
.IP \[bu] 2
|
|
New Features
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
curl install for rclone (Filip Bartodziej)
|
|
.IP \[bu] 2
|
|
\[en]stats now shows percentage, size, rate and ETA in condensed form
|
|
(Ishuah Kariuki)
|
|
.IP \[bu] 2
|
|
\[en]exclude\-if\-present to exclude a directory if a file is present
|
|
(Iakov Davydov)
|
|
.IP \[bu] 2
|
|
rmdirs: add \[en]leave\-root flag (lewapm)
|
|
.IP \[bu] 2
|
|
move: add \[en]delete\-empty\-src\-dirs flag to remove dirs after move
|
|
(Ishuah Kariuki)
|
|
.IP \[bu] 2
|
|
Add \[en]dump flag, introduce \[en]dump requests, responses and remove
|
|
\[en]dump\-auth, \[en]dump\-filters
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Obscure X\-Auth\-Token: from headers when dumping too
|
|
.RE
|
|
.IP \[bu] 2
|
|
Document and implement exit codes for different failure modes (Ishuah
|
|
Kariuki)
|
|
.RE
|
|
.IP \[bu] 2
|
|
Compile
|
|
.IP \[bu] 2
|
|
Bug Fixes
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Retry lots more different types of errors to make multipart transfers
|
|
more reliable
|
|
.IP \[bu] 2
|
|
Save the config before asking for a token, fixes disappearing oauth
|
|
config
|
|
.IP \[bu] 2
|
|
Warn the user if \[en]include and \[en]exclude are used together (Ernest
|
|
Borowski)
|
|
.IP \[bu] 2
|
|
Fix duplicate files (eg on Google drive) causing spurious copies
|
|
.IP \[bu] 2
|
|
Allow trailing and leading whitespace for passwords (Jason Rose)
|
|
.IP \[bu] 2
|
|
ncdu: fix crashes on empty directories
|
|
.IP \[bu] 2
|
|
rcat: fix goroutine leak
|
|
.IP \[bu] 2
|
|
moveto/copyto: Fix to allow copying to the same name
|
|
.RE
|
|
.IP \[bu] 2
|
|
Mount
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
\[en]vfs\-cache mode to make writes into mounts more reliable.
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
this requires caching files on the disk (see \[en]cache\-dir)
|
|
.IP \[bu] 2
|
|
As this is a new feature, use with care
|
|
.RE
|
|
.IP \[bu] 2
|
|
Use sdnotify to signal systemd the mount is ready (Fabian M\[:o]ller)
|
|
.IP \[bu] 2
|
|
Check if directory is not empty before mounting (Ernest Borowski)
|
|
.RE
|
|
.IP \[bu] 2
|
|
Local
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Add error message for cross file system moves
|
|
.IP \[bu] 2
|
|
Fix equality check for times
|
|
.RE
|
|
.IP \[bu] 2
|
|
Dropbox
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Rework multipart upload
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
buffer the chunks when uploading large files so they can be retried
|
|
.IP \[bu] 2
|
|
change default chunk size to 48MB now we are buffering them in memory
|
|
.IP \[bu] 2
|
|
retry every error after the first chunk is done successfully
|
|
.RE
|
|
.IP \[bu] 2
|
|
Fix error when renaming directories
|
|
.RE
|
|
.IP \[bu] 2
|
|
Swift
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Fix crash on bad authentication
|
|
.RE
|
|
.IP \[bu] 2
|
|
Google Drive
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Add service account support (Tim Cooijmans)
|
|
.RE
|
|
.IP \[bu] 2
|
|
S3
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Make it work properly with Digital Ocean Spaces (Andrew
|
|
Starr\-Bochicchio)
|
|
.IP \[bu] 2
|
|
Fix crash if a bad listing is received
|
|
.IP \[bu] 2
|
|
Add support for ECS task IAM roles (David Minor)
|
|
.RE
|
|
.IP \[bu] 2
|
|
Backblaze B2
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Fix multipart upload retries
|
|
.IP \[bu] 2
|
|
Fix \[en]hard\-delete to make it work 100% of the time
|
|
.RE
|
|
.IP \[bu] 2
|
|
Swift
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Allow authentication with storage URL and auth key (Giovanni Pizzi)
|
|
.IP \[bu] 2
|
|
Add new fields for swift configuration to support IBM Bluemix Swift
|
|
(Pierre Carlson)
|
|
.IP \[bu] 2
|
|
Add OS_TENANT_ID and OS_USER_ID to config
|
|
.IP \[bu] 2
|
|
Allow configs with user id instead of user name
|
|
.IP \[bu] 2
|
|
Check if swift segments container exists before creating (John Leach)
|
|
.IP \[bu] 2
|
|
Fix memory leak in swift transfers (upstream fix)
|
|
.RE
|
|
.IP \[bu] 2
|
|
SFTP
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Add option to enable the use of aes128\-cbc cipher (Jon Fautley)
|
|
.RE
|
|
.IP \[bu] 2
|
|
Amazon cloud drive
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Fix download of large files failing with \[lq]Only one auth mechanism
|
|
allowed\[rq]
|
|
.RE
|
|
.IP \[bu] 2
|
|
crypt
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Option to encrypt directory names or leave them intact
|
|
.IP \[bu] 2
|
|
Implement DirChangeNotify (Fabian M\[:o]ller)
|
|
.RE
|
|
.IP \[bu] 2
|
|
onedrive
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Add option to choose resourceURL during setup of OneDrive Business
|
|
account if more than one is available for user
|
|
.RE
|
|
.SS v1.38 \- 2017\-09\-30
|
|
.IP \[bu] 2
|
|
New backends
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Azure Blob Storage (thanks Andrei Dragomir)
|
|
.IP \[bu] 2
|
|
Box
|
|
.IP \[bu] 2
|
|
Onedrive for Business (thanks Oliver Heyme)
|
|
.IP \[bu] 2
|
|
QingStor from QingCloud (thanks wuyu)
|
|
.RE
|
|
.IP \[bu] 2
|
|
New commands
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
\f[C]rcat\f[R] \- read from standard input and stream upload
|
|
.IP \[bu] 2
|
|
\f[C]tree\f[R] \- shows a nicely formatted recursive listing
|
|
.IP \[bu] 2
|
|
\f[C]cryptdecode\f[R] \- decode crypted file names (thanks ishuah)
|
|
.IP \[bu] 2
|
|
\f[C]config show\f[R] \- print the config file
|
|
.IP \[bu] 2
|
|
\f[C]config file\f[R] \- print the config file location
|
|
.RE
|
|
.IP \[bu] 2
|
|
New Features
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Empty directories are deleted on \f[C]sync\f[R]
|
|
.IP \[bu] 2
|
|
\f[C]dedupe\f[R] \- implement merging of duplicate directories
|
|
.IP \[bu] 2
|
|
\f[C]check\f[R] and \f[C]cryptcheck\f[R] made more consistent and use
|
|
less memory
|
|
.IP \[bu] 2
|
|
\f[C]cleanup\f[R] for remaining remotes (thanks ishuah)
|
|
.IP \[bu] 2
|
|
\f[C]\-\-immutable\f[R] for ensuring that files don\[cq]t change (thanks
|
|
Jacob McNamee)
|
|
.IP \[bu] 2
|
|
\f[C]\-\-user\-agent\f[R] option (thanks Alex McGrath Kraak)
|
|
.IP \[bu] 2
|
|
\f[C]\-\-disable\f[R] flag to disable optional features
|
|
.IP \[bu] 2
|
|
\f[C]\-\-bind\f[R] flag for choosing the local addr on outgoing
|
|
connections
|
|
.IP \[bu] 2
|
|
Support for zsh auto\-completion (thanks bpicode)
|
|
.IP \[bu] 2
|
|
Stop normalizing file names but do a normalized compare in
|
|
\f[C]sync\f[R]
|
|
.RE
|
|
.IP \[bu] 2
|
|
Compile
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Update to using go1.9 as the default go version
|
|
.IP \[bu] 2
|
|
Remove snapd build due to maintenance problems
|
|
.RE
|
|
.IP \[bu] 2
|
|
Bug Fixes
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Improve retriable error detection which makes multipart uploads better
|
|
.IP \[bu] 2
|
|
Make \f[C]check\f[R] obey \f[C]\-\-ignore\-size\f[R]
|
|
.IP \[bu] 2
|
|
Fix bwlimit toggle in conjunction with schedules (thanks cbruegg)
|
|
.IP \[bu] 2
|
|
\f[C]config\f[R] ensures newly written config is on the same mount
|
|
.RE
|
|
.IP \[bu] 2
|
|
Local
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Revert to copy when moving file across file system boundaries
|
|
.IP \[bu] 2
|
|
\f[C]\-\-skip\-links\f[R] to suppress symlink warnings (thanks Zhiming
|
|
Wang)
|
|
.RE
|
|
.IP \[bu] 2
|
|
Mount
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Re\-use \f[C]rcat\f[R] internals to support uploads from all remotes
|
|
.RE
|
|
.IP \[bu] 2
|
|
Dropbox
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Fix \[lq]entry doesn\[cq]t belong in directory\[rq] error
|
|
.IP \[bu] 2
|
|
Stop using deprecated API methods
|
|
.RE
|
|
.IP \[bu] 2
|
|
Swift
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Fix server side copy to empty container with \f[C]\-\-fast\-list\f[R]
|
|
.RE
|
|
.IP \[bu] 2
|
|
Google Drive
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Change the default for \f[C]\-\-drive\-use\-trash\f[R] to \f[C]true\f[R]
|
|
.RE
|
|
.IP \[bu] 2
|
|
S3
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Set session token when using STS (thanks Girish Ramakrishnan)
|
|
.IP \[bu] 2
|
|
Glacier docs and error messages (thanks Jan Varho)
|
|
.IP \[bu] 2
|
|
Read 1000 (not 1024) items in dir listings to fix Wasabi
|
|
.RE
|
|
.IP \[bu] 2
|
|
Backblaze B2
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Fix SHA1 mismatch when downloading files with no SHA1
|
|
.IP \[bu] 2
|
|
Calculate missing hashes on the fly instead of spooling
|
|
.IP \[bu] 2
|
|
\f[C]\-\-b2\-hard\-delete\f[R] to permanently delete (not hide) files
|
|
(thanks John Papandriopoulos)
|
|
.RE
|
|
.IP \[bu] 2
|
|
Hubic
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Fix creating containers \- no longer have to use the \f[C]default\f[R]
|
|
container
|
|
.RE
|
|
.IP \[bu] 2
|
|
Swift
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Optionally configure from a standard set of OpenStack environment vars
|
|
.IP \[bu] 2
|
|
Add \f[C]endpoint_type\f[R] config
|
|
.RE
|
|
.IP \[bu] 2
|
|
Google Cloud Storage
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Fix bucket creation to work with limited permission users
|
|
.RE
|
|
.IP \[bu] 2
|
|
SFTP
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Implement connection pooling for multiple ssh connections
|
|
.IP \[bu] 2
|
|
Limit new connections per second
|
|
.IP \[bu] 2
|
|
Add support for MD5 and SHA1 hashes where available (thanks Christian
|
|
Br\[:u]ggemann)
|
|
.RE
|
|
.IP \[bu] 2
|
|
HTTP
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Fix URL encoding issues
|
|
.IP \[bu] 2
|
|
Fix directories with \f[C]:\f[R] in
|
|
.IP \[bu] 2
|
|
Fix panic with URL encoded content
|
|
.RE
|
|
.SS v1.37 \- 2017\-07\-22
|
|
.IP \[bu] 2
|
|
New backends
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
FTP \- thanks to Antonio Messina
|
|
.IP \[bu] 2
|
|
HTTP \- thanks to Vasiliy Tolstov
|
|
.RE
|
|
.IP \[bu] 2
|
|
New commands
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
rclone ncdu \- for exploring a remote with a text based user interface.
|
|
.IP \[bu] 2
|
|
rclone lsjson \- for listing with a machine readable output
|
|
.IP \[bu] 2
|
|
rclone dbhashsum \- to show Dropbox style hashes of files (local or
|
|
Dropbox)
|
|
.RE
|
|
.IP \[bu] 2
|
|
New Features
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Implement \[en]fast\-list flag
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
This allows remotes to list recursively if they can
|
|
.IP \[bu] 2
|
|
This uses less transactions (important if you pay for them)
|
|
.IP \[bu] 2
|
|
This may or may not be quicker
|
|
.IP \[bu] 2
|
|
This will use more memory as it has to hold the listing in memory
|
|
.IP \[bu] 2
|
|
\[en]old\-sync\-method deprecated \- the remaining uses are covered by
|
|
\[en]fast\-list
|
|
.IP \[bu] 2
|
|
This involved a major re\-write of all the listing code
|
|
.RE
|
|
.IP \[bu] 2
|
|
Add \[en]tpslimit and \[en]tpslimit\-burst to limit transactions per
|
|
second
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
this is useful in conjunction with \f[C]rclone mount\f[R] to limit
|
|
external apps
|
|
.RE
|
|
.IP \[bu] 2
|
|
Add \[en]stats\-log\-level so can see \[en]stats without \-v
|
|
.IP \[bu] 2
|
|
Print password prompts to stderr \- Hraban Luyat
|
|
.IP \[bu] 2
|
|
Warn about duplicate files when syncing
|
|
.IP \[bu] 2
|
|
Oauth improvements
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
allow auth_url and token_url to be set in the config file
|
|
.IP \[bu] 2
|
|
Print redirection URI if using own credentials.
|
|
.RE
|
|
.IP \[bu] 2
|
|
Don\[cq]t Mkdir at the start of sync to save transactions
|
|
.RE
|
|
.IP \[bu] 2
|
|
Compile
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Update build to go1.8.3
|
|
.IP \[bu] 2
|
|
Require go1.6 for building rclone
|
|
.IP \[bu] 2
|
|
Compile 386 builds with \[lq]GO386=387\[rq] for maximum compatibility
|
|
.RE
|
|
.IP \[bu] 2
|
|
Bug Fixes
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Fix menu selection when no remotes
|
|
.IP \[bu] 2
|
|
Config saving reworked to not kill the file if disk gets full
|
|
.IP \[bu] 2
|
|
Don\[cq]t delete remote if name does not change while renaming
|
|
.IP \[bu] 2
|
|
moveto, copyto: report transfers and checks as per move and copy
|
|
.RE
|
|
.IP \[bu] 2
|
|
Local
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Add \[en]local\-no\-unicode\-normalization flag \- Bob Potter
|
|
.RE
|
|
.IP \[bu] 2
|
|
Mount
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Now supported on Windows using cgofuse and WinFsp \- thanks to Bill
|
|
Zissimopoulos for much help
|
|
.IP \[bu] 2
|
|
Compare checksums on upload/download via FUSE
|
|
.IP \[bu] 2
|
|
Unmount when program ends with SIGINT (Ctrl+C) or SIGTERM \-
|
|
J\['e]r\[^o]me Vizcaino
|
|
.IP \[bu] 2
|
|
On read only open of file, make open pending until first read
|
|
.IP \[bu] 2
|
|
Make \[en]read\-only reject modify operations
|
|
.IP \[bu] 2
|
|
Implement ModTime via FUSE for remotes that support it
|
|
.IP \[bu] 2
|
|
Allow modTime to be changed even before all writers are closed
|
|
.IP \[bu] 2
|
|
Fix panic on renames
|
|
.IP \[bu] 2
|
|
Fix hang on errored upload
|
|
.RE
|
|
.IP \[bu] 2
|
|
Crypt
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Report the name:root as specified by the user
|
|
.IP \[bu] 2
|
|
Add an \[lq]obfuscate\[rq] option for filename encryption \- Stephen
|
|
Harris
|
|
.RE
|
|
.IP \[bu] 2
|
|
Amazon Drive
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Fix initialization order for token renewer
|
|
.IP \[bu] 2
|
|
Remove revoked credentials, allow oauth proxy config and update docs
|
|
.RE
|
|
.IP \[bu] 2
|
|
B2
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Reduce minimum chunk size to 5MB
|
|
.RE
|
|
.IP \[bu] 2
|
|
Drive
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Add team drive support
|
|
.IP \[bu] 2
|
|
Reduce bandwidth by adding fields for partial responses \- Martin
|
|
Kristensen
|
|
.IP \[bu] 2
|
|
Implement \[en]drive\-shared\-with\-me flag to view shared with me files
|
|
\- Danny Tsai
|
|
.IP \[bu] 2
|
|
Add \[en]drive\-trashed\-only to read only the files in the trash
|
|
.IP \[bu] 2
|
|
Remove obsolete \[en]drive\-full\-list
|
|
.IP \[bu] 2
|
|
Add missing seek to start on retries of chunked uploads
|
|
.IP \[bu] 2
|
|
Fix stats accounting for upload
|
|
.IP \[bu] 2
|
|
Convert / in names to a unicode equivalent (\[uFF0F])
|
|
.IP \[bu] 2
|
|
Poll for Google Drive changes when mounted
|
|
.RE
|
|
.IP \[bu] 2
|
|
OneDrive
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Fix the uploading of files with spaces
|
|
.IP \[bu] 2
|
|
Fix initialization order for token renewer
|
|
.IP \[bu] 2
|
|
Display speeds accurately when uploading \- Yoni Jah
|
|
.IP \[bu] 2
|
|
Swap to using http://localhost:53682/ as redirect URL \- Michael Ledin
|
|
.IP \[bu] 2
|
|
Retry on token expired error, reset upload body on retry \- Yoni Jah
|
|
.RE
|
|
.IP \[bu] 2
|
|
Google Cloud Storage
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Add ability to specify location and storage class via config and command
|
|
line \- thanks gdm85
|
|
.IP \[bu] 2
|
|
Create container if necessary on server side copy
|
|
.IP \[bu] 2
|
|
Increase directory listing chunk to 1000 to increase performance
|
|
.IP \[bu] 2
|
|
Obtain a refresh token for GCS \- Steven Lu
|
|
.RE
|
|
.IP \[bu] 2
|
|
Yandex
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Fix the name reported in log messages (was empty)
|
|
.IP \[bu] 2
|
|
Correct error return for listing empty directory
|
|
.RE
|
|
.IP \[bu] 2
|
|
Dropbox
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Rewritten to use the v2 API
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Now supports ModTime
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Can only set by uploading the file again
|
|
.IP \[bu] 2
|
|
If you uploaded with an old rclone, rclone may upload everything again
|
|
.IP \[bu] 2
|
|
Use \f[C]\-\-size\-only\f[R] or \f[C]\-\-checksum\f[R] to avoid this
|
|
.RE
|
|
.IP \[bu] 2
|
|
Now supports the Dropbox content hashing scheme
|
|
.IP \[bu] 2
|
|
Now supports low level retries
|
|
.RE
|
|
.RE
|
|
.IP \[bu] 2
|
|
S3
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Work around eventual consistency in bucket creation
|
|
.IP \[bu] 2
|
|
Create container if necessary on server side copy
|
|
.IP \[bu] 2
|
|
Add us\-east\-2 (Ohio) and eu\-west\-2 (London) S3 regions \- Zahiar
|
|
Ahmed
|
|
.RE
|
|
.IP \[bu] 2
|
|
Swift, Hubic
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Fix zero length directory markers showing in the subdirectory listing
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
this caused lots of duplicate transfers
|
|
.RE
|
|
.IP \[bu] 2
|
|
Fix paged directory listings
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
this caused duplicate directory errors
|
|
.RE
|
|
.IP \[bu] 2
|
|
Create container if necessary on server side copy
|
|
.IP \[bu] 2
|
|
Increase directory listing chunk to 1000 to increase performance
|
|
.IP \[bu] 2
|
|
Make sensible error if the user forgets the container
|
|
.RE
|
|
.IP \[bu] 2
|
|
SFTP
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Add support for using ssh key files
|
|
.IP \[bu] 2
|
|
Fix under Windows
|
|
.IP \[bu] 2
|
|
Fix ssh agent on Windows
|
|
.IP \[bu] 2
|
|
Adapt to latest version of library \- Igor Kharin
|
|
.RE
|
|
.SS v1.36 \- 2017\-03\-18
|
|
.IP \[bu] 2
|
|
New Features
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
SFTP remote (Jack Schmidt)
|
|
.IP \[bu] 2
|
|
Re\-implement sync routine to work a directory at a time reducing memory
|
|
usage
|
|
.IP \[bu] 2
|
|
Logging revamped to be more inline with rsync \- now much quieter * \-v
|
|
only shows transfers * \-vv is for full debug * \[en]syslog to log to
|
|
syslog on capable platforms
|
|
.IP \[bu] 2
|
|
Implement \[en]backup\-dir and \[en]suffix
|
|
.IP \[bu] 2
|
|
Implement \[en]track\-renames (initial implementation by Bj\[/o]rn Erik
|
|
Pedersen)
|
|
.IP \[bu] 2
|
|
Add time\-based bandwidth limits (Lukas Loesche)
|
|
.IP \[bu] 2
|
|
rclone cryptcheck: checks integrity of crypt remotes
|
|
.IP \[bu] 2
|
|
Allow all config file variables and options to be set from environment
|
|
variables
|
|
.IP \[bu] 2
|
|
Add \[en]buffer\-size parameter to control buffer size for copy
|
|
.IP \[bu] 2
|
|
Make \[en]delete\-after the default
|
|
.IP \[bu] 2
|
|
Add \[en]ignore\-checksum flag (fixed by Hisham Zarka)
|
|
.IP \[bu] 2
|
|
rclone check: Add \[en]download flag to check all the data, not just
|
|
hashes
|
|
.IP \[bu] 2
|
|
rclone cat: add \[en]head, \[en]tail, \[en]offset, \[en]count and
|
|
\[en]discard
|
|
.IP \[bu] 2
|
|
rclone config: when choosing from a list, allow the value to be entered
|
|
too
|
|
.IP \[bu] 2
|
|
rclone config: allow rename and copy of remotes
|
|
.IP \[bu] 2
|
|
rclone obscure: for generating encrypted passwords for rclone\[cq]s
|
|
config (T.C.
|
|
Ferguson)
|
|
.IP \[bu] 2
|
|
Comply with XDG Base Directory specification (Dario Giovannetti)
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
this moves the default location of the config file in a backwards
|
|
compatible way
|
|
.RE
|
|
.IP \[bu] 2
|
|
Release changes
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Ubuntu snap support (Dedsec1)
|
|
.IP \[bu] 2
|
|
Compile with go 1.8
|
|
.IP \[bu] 2
|
|
MIPS/Linux big and little endian support
|
|
.RE
|
|
.RE
|
|
.IP \[bu] 2
|
|
Bug Fixes
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Fix copyto copying things to the wrong place if the destination dir
|
|
didn\[cq]t exist
|
|
.IP \[bu] 2
|
|
Fix parsing of remotes in moveto and copyto
|
|
.IP \[bu] 2
|
|
Fix \[en]delete\-before deleting files on copy
|
|
.IP \[bu] 2
|
|
Fix \[en]files\-from with an empty file copying everything
|
|
.IP \[bu] 2
|
|
Fix sync: don\[cq]t update mod times if \[en]dry\-run set
|
|
.IP \[bu] 2
|
|
Fix MimeType propagation
|
|
.IP \[bu] 2
|
|
Fix filters to add ** rules to directory rules
|
|
.RE
|
|
.IP \[bu] 2
|
|
Local
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Implement \-L, \[en]copy\-links flag to allow rclone to follow symlinks
|
|
.IP \[bu] 2
|
|
Open files in write only mode so rclone can write to an rclone mount
|
|
.IP \[bu] 2
|
|
Fix unnormalised unicode causing problems reading directories
|
|
.IP \[bu] 2
|
|
Fix interaction between \-x flag and \[en]max\-depth
|
|
.RE
|
|
.IP \[bu] 2
|
|
Mount
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Implement proper directory handling (mkdir, rmdir, renaming)
|
|
.IP \[bu] 2
|
|
Make include and exclude filters apply to mount
|
|
.IP \[bu] 2
|
|
Implement read and write async buffers \- control with \[en]buffer\-size
|
|
.IP \[bu] 2
|
|
Fix fsync on for directories
|
|
.IP \[bu] 2
|
|
Fix retry on network failure when reading off crypt
|
|
.RE
|
|
.IP \[bu] 2
|
|
Crypt
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Add \[en]crypt\-show\-mapping to show encrypted file mapping
|
|
.IP \[bu] 2
|
|
Fix crypt writer getting stuck in a loop
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
\f[B]IMPORTANT\f[R] this bug had the potential to cause data corruption
|
|
when
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
reading data from a network based remote and
|
|
.IP \[bu] 2
|
|
writing to a crypt on Google Drive
|
|
.RE
|
|
.IP \[bu] 2
|
|
Use the cryptcheck command to validate your data if you are concerned
|
|
.IP \[bu] 2
|
|
If syncing two crypt remotes, sync the unencrypted remote
|
|
.RE
|
|
.RE
|
|
.IP \[bu] 2
|
|
Amazon Drive
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Fix panics on Move (rename)
|
|
.IP \[bu] 2
|
|
Fix panic on token expiry
|
|
.RE
|
|
.IP \[bu] 2
|
|
B2
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Fix inconsistent listings and rclone check
|
|
.IP \[bu] 2
|
|
Fix uploading empty files with go1.8
|
|
.IP \[bu] 2
|
|
Constrain memory usage when doing multipart uploads
|
|
.IP \[bu] 2
|
|
Fix upload url not being refreshed properly
|
|
.RE
|
|
.IP \[bu] 2
|
|
Drive
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Fix Rmdir on directories with trashed files
|
|
.IP \[bu] 2
|
|
Fix \[lq]Ignoring unknown object\[rq] when downloading
|
|
.IP \[bu] 2
|
|
Add \[en]drive\-list\-chunk
|
|
.IP \[bu] 2
|
|
Add \[en]drive\-skip\-gdocs (K\['a]roly Ol\['a]h)
|
|
.RE
|
|
.IP \[bu] 2
|
|
OneDrive
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Implement Move
|
|
.IP \[bu] 2
|
|
Fix Copy
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Fix overwrite detection in Copy
|
|
.IP \[bu] 2
|
|
Fix waitForJob to parse errors correctly
|
|
.RE
|
|
.IP \[bu] 2
|
|
Use token renewer to stop auth errors on long uploads
|
|
.IP \[bu] 2
|
|
Fix uploading empty files with go1.8
|
|
.RE
|
|
.IP \[bu] 2
|
|
Google Cloud Storage
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Fix depth 1 directory listings
|
|
.RE
|
|
.IP \[bu] 2
|
|
Yandex
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Fix single level directory listing
|
|
.RE
|
|
.IP \[bu] 2
|
|
Dropbox
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Normalise the case for single level directory listings
|
|
.IP \[bu] 2
|
|
Fix depth 1 listing
|
|
.RE
|
|
.IP \[bu] 2
|
|
S3
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Added ca\-central\-1 region (Jon Yergatian)
|
|
.RE
|
|
.SS v1.35 \- 2017\-01\-02
|
|
.IP \[bu] 2
|
|
New Features
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
moveto and copyto commands for choosing a destination name on copy/move
|
|
.IP \[bu] 2
|
|
rmdirs command to recursively delete empty directories
|
|
.IP \[bu] 2
|
|
Allow repeated \[en]include/\[en]exclude/\[en]filter options
|
|
.IP \[bu] 2
|
|
Only show transfer stats on commands which transfer stuff
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
show stats on any command using the \f[C]\-\-stats\f[R] flag
|
|
.RE
|
|
.IP \[bu] 2
|
|
Allow overlapping directories in move when server side dir move is
|
|
supported
|
|
.IP \[bu] 2
|
|
Add \[en]stats\-unit option \- thanks Scott McGillivray
|
|
.RE
|
|
.IP \[bu] 2
|
|
Bug Fixes
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Fix the config file being overwritten when two rclone instances are
|
|
running
|
|
.IP \[bu] 2
|
|
Make rclone lsd obey the filters properly
|
|
.IP \[bu] 2
|
|
Fix compilation on mips
|
|
.IP \[bu] 2
|
|
Fix not transferring files that don\[cq]t differ in size
|
|
.IP \[bu] 2
|
|
Fix panic on nil retry/fatal error
|
|
.RE
|
|
.IP \[bu] 2
|
|
Mount
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Retry reads on error \- should help with reliability a lot
|
|
.IP \[bu] 2
|
|
Report the modification times for directories from the remote
|
|
.IP \[bu] 2
|
|
Add bandwidth accounting and limiting (fixes \[en]bwlimit)
|
|
.IP \[bu] 2
|
|
If \[en]stats provided will show stats and which files are transferring
|
|
.IP \[bu] 2
|
|
Support R/W files if truncate is set.
|
|
.IP \[bu] 2
|
|
Implement statfs interface so df works
|
|
.IP \[bu] 2
|
|
Note that write is now supported on Amazon Drive
|
|
.IP \[bu] 2
|
|
Report number of blocks in a file \- thanks Stefan Breunig
|
|
.RE
|
|
.IP \[bu] 2
|
|
Crypt
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Prevent the user pointing crypt at itself
|
|
.IP \[bu] 2
|
|
Fix failed to authenticate decrypted block errors
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
these will now return the underlying unexpected EOF instead
|
|
.RE
|
|
.RE
|
|
.IP \[bu] 2
|
|
Amazon Drive
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Add support for server side move and directory move \- thanks Stefan
|
|
Breunig
|
|
.IP \[bu] 2
|
|
Fix nil pointer deref on size attribute
|
|
.RE
|
|
.IP \[bu] 2
|
|
B2
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Use new prefix and delimiter parameters in directory listings
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
This makes \[en]max\-depth 1 dir listings as used in mount much faster
|
|
.RE
|
|
.IP \[bu] 2
|
|
Reauth the account while doing uploads too \- should help with token
|
|
expiry
|
|
.RE
|
|
.IP \[bu] 2
|
|
Drive
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Make DirMove more efficient and complain about moving the root
|
|
.IP \[bu] 2
|
|
Create destination directory on Move()
|
|
.RE
|
|
.SS v1.34 \- 2016\-11\-06
|
|
.IP \[bu] 2
|
|
New Features
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Stop single file and \f[C]\-\-files\-from\f[R] operations iterating
|
|
through the source bucket.
|
|
.IP \[bu] 2
|
|
Stop removing failed upload to cloud storage remotes
|
|
.IP \[bu] 2
|
|
Make ContentType be preserved for cloud to cloud copies
|
|
.IP \[bu] 2
|
|
Add support to toggle bandwidth limits via SIGUSR2 \- thanks Marco
|
|
Paganini
|
|
.IP \[bu] 2
|
|
\f[C]rclone check\f[R] shows count of hashes that couldn\[cq]t be
|
|
checked
|
|
.IP \[bu] 2
|
|
\f[C]rclone listremotes\f[R] command
|
|
.IP \[bu] 2
|
|
Support linux/arm64 build \- thanks Fredrik Fornwall
|
|
.IP \[bu] 2
|
|
Remove \f[C]Authorization:\f[R] lines from \f[C]\-\-dump\-headers\f[R]
|
|
output
|
|
.RE
|
|
.IP \[bu] 2
|
|
Bug Fixes
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Ignore files with control characters in the names
|
|
.IP \[bu] 2
|
|
Fix \f[C]rclone move\f[R] command
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Delete src files which already existed in dst
|
|
.IP \[bu] 2
|
|
Fix deletion of src file when dst file older
|
|
.RE
|
|
.IP \[bu] 2
|
|
Fix \f[C]rclone check\f[R] on crypted file systems
|
|
.IP \[bu] 2
|
|
Make failed uploads not count as \[lq]Transferred\[rq]
|
|
.IP \[bu] 2
|
|
Make sure high level retries show with \f[C]\-q\f[R]
|
|
.IP \[bu] 2
|
|
Use a vendor directory with godep for repeatable builds
|
|
.RE
|
|
.IP \[bu] 2
|
|
\f[C]rclone mount\f[R] \- FUSE
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Implement FUSE mount options
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
\f[C]\-\-no\-modtime\f[R], \f[C]\-\-debug\-fuse\f[R],
|
|
\f[C]\-\-read\-only\f[R], \f[C]\-\-allow\-non\-empty\f[R],
|
|
\f[C]\-\-allow\-root\f[R], \f[C]\-\-allow\-other\f[R]
|
|
.IP \[bu] 2
|
|
\f[C]\-\-default\-permissions\f[R], \f[C]\-\-write\-back\-cache\f[R],
|
|
\f[C]\-\-max\-read\-ahead\f[R], \f[C]\-\-umask\f[R], \f[C]\-\-uid\f[R],
|
|
\f[C]\-\-gid\f[R]
|
|
.RE
|
|
.IP \[bu] 2
|
|
Add \f[C]\-\-dir\-cache\-time\f[R] to control caching of directory
|
|
entries
|
|
.IP \[bu] 2
|
|
Implement seek for files opened for read (useful for video players)
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
with \f[C]\-no\-seek\f[R] flag to disable
|
|
.RE
|
|
.IP \[bu] 2
|
|
Fix crash on 32 bit ARM (alignment of 64 bit counter)
|
|
.IP \[bu] 2
|
|
\&...and many more internal fixes and improvements!
|
|
.RE
|
|
.IP \[bu] 2
|
|
Crypt
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Don\[cq]t show encrypted password in configurator to stop confusion
|
|
.RE
|
|
.IP \[bu] 2
|
|
Amazon Drive
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
New wait for upload option \f[C]\-\-acd\-upload\-wait\-per\-gb\f[R]
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
upload timeouts scale by file size and can be disabled
|
|
.RE
|
|
.IP \[bu] 2
|
|
Add 502 Bad Gateway to list of errors we retry
|
|
.IP \[bu] 2
|
|
Fix overwriting a file with a zero length file
|
|
.IP \[bu] 2
|
|
Fix ACD file size warning limit \- thanks Felix B\[:u]nemann
|
|
.RE
|
|
.IP \[bu] 2
|
|
Local
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Unix: implement \f[C]\-x\f[R]/\f[C]\-\-one\-file\-system\f[R] to stay on
|
|
a single file system
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
thanks Durval Menezes and Luiz Carlos Rumbelsperger Viana
|
|
.RE
|
|
.IP \[bu] 2
|
|
Windows: ignore the symlink bit on files
|
|
.IP \[bu] 2
|
|
Windows: Ignore directory based junction points
|
|
.RE
|
|
.IP \[bu] 2
|
|
B2
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Make sure each upload has at least one upload slot \- fixes strange
|
|
upload stats
|
|
.IP \[bu] 2
|
|
Fix uploads when using crypt
|
|
.IP \[bu] 2
|
|
Fix download of large files (sha1 mismatch)
|
|
.IP \[bu] 2
|
|
Return error when we try to create a bucket which someone else owns
|
|
.IP \[bu] 2
|
|
Update B2 docs with Data usage, and Crypt section \- thanks Tomasz Mazur
|
|
.RE
|
|
.IP \[bu] 2
|
|
S3
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Command line and config file support for
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Setting/overriding ACL \- thanks Radek Senfeld
|
|
.IP \[bu] 2
|
|
Setting storage class \- thanks Asko Tamm
|
|
.RE
|
|
.RE
|
|
.IP \[bu] 2
|
|
Drive
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Make exponential backoff work exactly as per Google specification
|
|
.IP \[bu] 2
|
|
add \f[C].epub\f[R], \f[C].odp\f[R] and \f[C].tsv\f[R] as export
|
|
formats.
|
|
.RE
|
|
.IP \[bu] 2
|
|
Swift
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Don\[cq]t read metadata for directory marker objects
|
|
.RE
|
|
.SS v1.33 \- 2016\-08\-24
|
|
.IP \[bu] 2
|
|
New Features
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Implement encryption
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
data encrypted in NACL secretbox format
|
|
.IP \[bu] 2
|
|
with optional file name encryption
|
|
.RE
|
|
.IP \[bu] 2
|
|
New commands
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
rclone mount \- implements FUSE mounting of remotes (EXPERIMENTAL)
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
works on Linux, FreeBSD and OS X (need testers for the last 2!)
|
|
.RE
|
|
.IP \[bu] 2
|
|
rclone cat \- outputs remote file or files to the terminal
|
|
.IP \[bu] 2
|
|
rclone genautocomplete \- command to make a bash completion script for
|
|
rclone
|
|
.RE
|
|
.IP \[bu] 2
|
|
Editing a remote using \f[C]rclone config\f[R] now goes through the
|
|
wizard
|
|
.IP \[bu] 2
|
|
Compile with go 1.7 \- this fixes rclone on macOS Sierra and on 386
|
|
processors
|
|
.IP \[bu] 2
|
|
Use cobra for sub commands and docs generation
|
|
.RE
|
|
.IP \[bu] 2
|
|
drive
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Document how to make your own client_id
|
|
.RE
|
|
.IP \[bu] 2
|
|
s3
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
User\-configurable Amazon S3 ACL (thanks Radek \[vS]enfeld)
|
|
.RE
|
|
.IP \[bu] 2
|
|
b2
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Fix stats accounting for upload \- no more jumping to 100% done
|
|
.IP \[bu] 2
|
|
On cleanup delete hide marker if it is the current file
|
|
.IP \[bu] 2
|
|
New B2 API endpoint (thanks Per Cederberg)
|
|
.IP \[bu] 2
|
|
Set maximum backoff to 5 Minutes
|
|
.RE
|
|
.IP \[bu] 2
|
|
onedrive
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Fix URL escaping in file names \- eg uploading files with \f[C]+\f[R] in
|
|
them.
|
|
.RE
|
|
.IP \[bu] 2
|
|
amazon cloud drive
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Fix token expiry during large uploads
|
|
.IP \[bu] 2
|
|
Work around 408 REQUEST_TIMEOUT and 504 GATEWAY_TIMEOUT errors
|
|
.RE
|
|
.IP \[bu] 2
|
|
local
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Fix filenames with invalid UTF\-8 not being uploaded
|
|
.IP \[bu] 2
|
|
Fix problem with some UTF\-8 characters on OS X
|
|
.RE
|
|
.SS v1.32 \- 2016\-07\-13
|
|
.IP \[bu] 2
|
|
Backblaze B2
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Fix upload of files large files not in root
|
|
.RE
|
|
.SS v1.31 \- 2016\-07\-13
|
|
.IP \[bu] 2
|
|
New Features
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Reduce memory on sync by about 50%
|
|
.IP \[bu] 2
|
|
Implement \[en]no\-traverse flag to stop copy traversing the destination
|
|
remote.
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
This can be used to reduce memory usage down to the smallest possible.
|
|
.IP \[bu] 2
|
|
Useful to copy a small number of files into a large destination folder.
|
|
.RE
|
|
.IP \[bu] 2
|
|
Implement cleanup command for emptying trash / removing old versions of
|
|
files
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Currently B2 only
|
|
.RE
|
|
.IP \[bu] 2
|
|
Single file handling improved
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Now copied with \[en]files\-from
|
|
.IP \[bu] 2
|
|
Automatically sets \[en]no\-traverse when copying a single file
|
|
.RE
|
|
.IP \[bu] 2
|
|
Info on using installing with ansible \- thanks Stefan Weichinger
|
|
.IP \[bu] 2
|
|
Implement \[en]no\-update\-modtime flag to stop rclone fixing the remote
|
|
modified times.
|
|
.RE
|
|
.IP \[bu] 2
|
|
Bug Fixes
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Fix move command \- stop it running for overlapping Fses \- this was
|
|
causing data loss.
|
|
.RE
|
|
.IP \[bu] 2
|
|
Local
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Fix incomplete hashes \- this was causing problems for B2.
|
|
.RE
|
|
.IP \[bu] 2
|
|
Amazon Drive
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Rename Amazon Cloud Drive to Amazon Drive \- no changes to config file
|
|
needed.
|
|
.RE
|
|
.IP \[bu] 2
|
|
Swift
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Add support for non\-default project domain \- thanks Antonio Messina.
|
|
.RE
|
|
.IP \[bu] 2
|
|
S3
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Add instructions on how to use rclone with minio.
|
|
.IP \[bu] 2
|
|
Add ap\-northeast\-2 (Seoul) and ap\-south\-1 (Mumbai) regions.
|
|
.IP \[bu] 2
|
|
Skip setting the modified time for objects > 5GB as it isn\[cq]t
|
|
possible.
|
|
.RE
|
|
.IP \[bu] 2
|
|
Backblaze B2
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Add \[en]b2\-versions flag so old versions can be listed and retrieved.
|
|
.IP \[bu] 2
|
|
Treat 403 errors (eg cap exceeded) as fatal.
|
|
.IP \[bu] 2
|
|
Implement cleanup command for deleting old file versions.
|
|
.IP \[bu] 2
|
|
Make error handling compliant with B2 integrations notes.
|
|
.IP \[bu] 2
|
|
Fix handling of token expiry.
|
|
.IP \[bu] 2
|
|
Implement \[en]b2\-test\-mode to set \f[C]X\-Bz\-Test\-Mode\f[R] header.
|
|
.IP \[bu] 2
|
|
Set cutoff for chunked upload to 200MB as per B2 guidelines.
|
|
.IP \[bu] 2
|
|
Make upload multi\-threaded.
|
|
.RE
|
|
.IP \[bu] 2
|
|
Dropbox
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Don\[cq]t retry 461 errors.
|
|
.RE
|
|
.SS v1.30 \- 2016\-06\-18
|
|
.IP \[bu] 2
|
|
New Features
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Directory listing code reworked for more features and better error
|
|
reporting (thanks to Klaus Post for help).
|
|
This enables
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Directory include filtering for efficiency
|
|
.IP \[bu] 2
|
|
\[en]max\-depth parameter
|
|
.IP \[bu] 2
|
|
Better error reporting
|
|
.IP \[bu] 2
|
|
More to come
|
|
.RE
|
|
.IP \[bu] 2
|
|
Retry more errors
|
|
.IP \[bu] 2
|
|
Add \[en]ignore\-size flag \- for uploading images to onedrive
|
|
.IP \[bu] 2
|
|
Log \-v output to stdout by default
|
|
.IP \[bu] 2
|
|
Display the transfer stats in more human readable form
|
|
.IP \[bu] 2
|
|
Make 0 size files specifiable with \f[C]\-\-max\-size 0b\f[R]
|
|
.IP \[bu] 2
|
|
Add \f[C]b\f[R] suffix so we can specify bytes in \[en]bwlimit,
|
|
\[en]min\-size etc
|
|
.IP \[bu] 2
|
|
Use \[lq]password:\[rq] instead of \[lq]password>\[rq] prompt \- thanks
|
|
Klaus Post and Leigh Klotz
|
|
.RE
|
|
.IP \[bu] 2
|
|
Bug Fixes
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Fix retry doing one too many retries
|
|
.RE
|
|
.IP \[bu] 2
|
|
Local
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Fix problems with OS X and UTF\-8 characters
|
|
.RE
|
|
.IP \[bu] 2
|
|
Amazon Drive
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Check a file exists before uploading to help with 408 Conflict errors
|
|
.IP \[bu] 2
|
|
Reauth on 401 errors \- this has been causing a lot of problems
|
|
.IP \[bu] 2
|
|
Work around spurious 403 errors
|
|
.IP \[bu] 2
|
|
Restart directory listings on error
|
|
.RE
|
|
.IP \[bu] 2
|
|
Google Drive
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Check a file exists before uploading to help with duplicates
|
|
.IP \[bu] 2
|
|
Fix retry of multipart uploads
|
|
.RE
|
|
.IP \[bu] 2
|
|
Backblaze B2
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Implement large file uploading
|
|
.RE
|
|
.IP \[bu] 2
|
|
S3
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Add AES256 server\-side encryption for \- thanks Justin R.
|
|
Wilson
|
|
.RE
|
|
.IP \[bu] 2
|
|
Google Cloud Storage
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Make sure we don\[cq]t use conflicting content types on upload
|
|
.IP \[bu] 2
|
|
Add service account support \- thanks Michal Witkowski
|
|
.RE
|
|
.IP \[bu] 2
|
|
Swift
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Add auth version parameter
|
|
.IP \[bu] 2
|
|
Add domain option for openstack (v3 auth) \- thanks Fabian Ruff
|
|
.RE
|
|
.SS v1.29 \- 2016\-04\-18
|
|
.IP \[bu] 2
|
|
New Features
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Implement \f[C]\-I, \-\-ignore\-times\f[R] for unconditional upload
|
|
.IP \[bu] 2
|
|
Improve \f[C]dedupe\f[R]command
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Now removes identical copies without asking
|
|
.IP \[bu] 2
|
|
Now obeys \f[C]\-\-dry\-run\f[R]
|
|
.IP \[bu] 2
|
|
Implement \f[C]\-\-dedupe\-mode\f[R] for non interactive running
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
\f[C]\-\-dedupe\-mode interactive\f[R] \- interactive the default.
|
|
.IP \[bu] 2
|
|
\f[C]\-\-dedupe\-mode skip\f[R] \- removes identical files then skips
|
|
anything left.
|
|
.IP \[bu] 2
|
|
\f[C]\-\-dedupe\-mode first\f[R] \- removes identical files then keeps
|
|
the first one.
|
|
.IP \[bu] 2
|
|
\f[C]\-\-dedupe\-mode newest\f[R] \- removes identical files then keeps
|
|
the newest one.
|
|
.IP \[bu] 2
|
|
\f[C]\-\-dedupe\-mode oldest\f[R] \- removes identical files then keeps
|
|
the oldest one.
|
|
.IP \[bu] 2
|
|
\f[C]\-\-dedupe\-mode rename\f[R] \- removes identical files then
|
|
renames the rest to be different.
|
|
.RE
|
|
.RE
|
|
.RE
|
|
.IP \[bu] 2
|
|
Bug fixes
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Make rclone check obey the \f[C]\-\-size\-only\f[R] flag.
|
|
.IP \[bu] 2
|
|
Use \[lq]application/octet\-stream\[rq] if discovered mime type is
|
|
invalid.
|
|
.IP \[bu] 2
|
|
Fix missing \[lq]quit\[rq] option when there are no remotes.
|
|
.RE
|
|
.IP \[bu] 2
|
|
Google Drive
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Increase default chunk size to 8 MB \- increases upload speed of big
|
|
files
|
|
.IP \[bu] 2
|
|
Speed up directory listings and make more reliable
|
|
.IP \[bu] 2
|
|
Add missing retries for Move and DirMove \- increases reliability
|
|
.IP \[bu] 2
|
|
Preserve mime type on file update
|
|
.RE
|
|
.IP \[bu] 2
|
|
Backblaze B2
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Enable mod time syncing
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
This means that B2 will now check modification times
|
|
.IP \[bu] 2
|
|
It will upload new files to update the modification times
|
|
.IP \[bu] 2
|
|
(there isn\[cq]t an API to just set the mod time.)
|
|
.IP \[bu] 2
|
|
If you want the old behaviour use \f[C]\-\-size\-only\f[R].
|
|
.RE
|
|
.IP \[bu] 2
|
|
Update API to new version
|
|
.IP \[bu] 2
|
|
Fix parsing of mod time when not in metadata
|
|
.RE
|
|
.IP \[bu] 2
|
|
Swift/Hubic
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Don\[cq]t return an MD5SUM for static large objects
|
|
.RE
|
|
.IP \[bu] 2
|
|
S3
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Fix uploading files bigger than 50GB
|
|
.RE
|
|
.SS v1.28 \- 2016\-03\-01
|
|
.IP \[bu] 2
|
|
New Features
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Configuration file encryption \- thanks Klaus Post
|
|
.IP \[bu] 2
|
|
Improve \f[C]rclone config\f[R] adding more help and making it easier to
|
|
understand
|
|
.IP \[bu] 2
|
|
Implement \f[C]\-u\f[R]/\f[C]\-\-update\f[R] so creation times can be
|
|
used on all remotes
|
|
.IP \[bu] 2
|
|
Implement \f[C]\-\-low\-level\-retries\f[R] flag
|
|
.IP \[bu] 2
|
|
Optionally disable gzip compression on downloads with
|
|
\f[C]\-\-no\-gzip\-encoding\f[R]
|
|
.RE
|
|
.IP \[bu] 2
|
|
Bug fixes
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Don\[cq]t make directories if \f[C]\-\-dry\-run\f[R] set
|
|
.IP \[bu] 2
|
|
Fix and document the \f[C]move\f[R] command
|
|
.IP \[bu] 2
|
|
Fix redirecting stderr on unix\-like OSes when using
|
|
\f[C]\-\-log\-file\f[R]
|
|
.IP \[bu] 2
|
|
Fix \f[C]delete\f[R] command to wait until all finished \- fixes missing
|
|
deletes.
|
|
.RE
|
|
.IP \[bu] 2
|
|
Backblaze B2
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Use one upload URL per go routine fixes
|
|
\f[C]more than one upload using auth token\f[R]
|
|
.IP \[bu] 2
|
|
Add pacing, retries and reauthentication \- fixes token expiry problems
|
|
.IP \[bu] 2
|
|
Upload without using a temporary file from local (and remotes which
|
|
support SHA1)
|
|
.IP \[bu] 2
|
|
Fix reading metadata for all files when it shouldn\[cq]t have been
|
|
.RE
|
|
.IP \[bu] 2
|
|
Drive
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Fix listing drive documents at root
|
|
.IP \[bu] 2
|
|
Disable copy and move for Google docs
|
|
.RE
|
|
.IP \[bu] 2
|
|
Swift
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Fix uploading of chunked files with non ASCII characters
|
|
.IP \[bu] 2
|
|
Allow setting of \f[C]storage_url\f[R] in the config \- thanks Xavier
|
|
Lucas
|
|
.RE
|
|
.IP \[bu] 2
|
|
S3
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Allow IAM role and credentials from environment variables \- thanks
|
|
Brian Stengaard
|
|
.IP \[bu] 2
|
|
Allow low privilege users to use S3 (check if directory exists during
|
|
Mkdir) \- thanks Jakub Gedeon
|
|
.RE
|
|
.IP \[bu] 2
|
|
Amazon Drive
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Retry on more things to make directory listings more reliable
|
|
.RE
|
|
.SS v1.27 \- 2016\-01\-31
|
|
.IP \[bu] 2
|
|
New Features
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Easier headless configuration with \f[C]rclone authorize\f[R]
|
|
.IP \[bu] 2
|
|
Add support for multiple hash types \- we now check SHA1 as well as MD5
|
|
hashes.
|
|
.IP \[bu] 2
|
|
\f[C]delete\f[R] command which does obey the filters (unlike
|
|
\f[C]purge\f[R])
|
|
.IP \[bu] 2
|
|
\f[C]dedupe\f[R] command to deduplicate a remote.
|
|
Useful with Google Drive.
|
|
.IP \[bu] 2
|
|
Add \f[C]\-\-ignore\-existing\f[R] flag to skip all files that exist on
|
|
destination.
|
|
.IP \[bu] 2
|
|
Add \f[C]\-\-delete\-before\f[R], \f[C]\-\-delete\-during\f[R],
|
|
\f[C]\-\-delete\-after\f[R] flags.
|
|
.IP \[bu] 2
|
|
Add \f[C]\-\-memprofile\f[R] flag to debug memory use.
|
|
.IP \[bu] 2
|
|
Warn the user about files with same name but different case
|
|
.IP \[bu] 2
|
|
Make \f[C]\-\-include\f[R] rules add their implicit exclude * at the end
|
|
of the filter list
|
|
.IP \[bu] 2
|
|
Deprecate compiling with go1.3
|
|
.RE
|
|
.IP \[bu] 2
|
|
Amazon Drive
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Fix download of files > 10 GB
|
|
.IP \[bu] 2
|
|
Fix directory traversal (\[lq]Next token is expired\[rq]) for large
|
|
directory listings
|
|
.IP \[bu] 2
|
|
Remove 409 conflict from error codes we will retry \- stops very long
|
|
pauses
|
|
.RE
|
|
.IP \[bu] 2
|
|
Backblaze B2
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
SHA1 hashes now checked by rclone core
|
|
.RE
|
|
.IP \[bu] 2
|
|
Drive
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Add \f[C]\-\-drive\-auth\-owner\-only\f[R] to only consider files owned
|
|
by the user \- thanks Bj\[:o]rn Harrtell
|
|
.IP \[bu] 2
|
|
Export Google documents
|
|
.RE
|
|
.IP \[bu] 2
|
|
Dropbox
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Make file exclusion error controllable with \-q
|
|
.RE
|
|
.IP \[bu] 2
|
|
Swift
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Fix upload from unprivileged user.
|
|
.RE
|
|
.IP \[bu] 2
|
|
S3
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Fix updating of mod times of files with \f[C]+\f[R] in.
|
|
.RE
|
|
.IP \[bu] 2
|
|
Local
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Add local file system option to disable UNC on Windows.
|
|
.RE
|
|
.SS v1.26 \- 2016\-01\-02
|
|
.IP \[bu] 2
|
|
New Features
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Yandex storage backend \- thank you Dmitry Burdeev (\[lq]dibu\[rq])
|
|
.IP \[bu] 2
|
|
Implement Backblaze B2 storage backend
|
|
.IP \[bu] 2
|
|
Add \[en]min\-age and \[en]max\-age flags \- thank you Adriano
|
|
Aur\['e]lio Meirelles
|
|
.IP \[bu] 2
|
|
Make ls/lsl/md5sum/size/check obey includes and excludes
|
|
.RE
|
|
.IP \[bu] 2
|
|
Fixes
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Fix crash in http logging
|
|
.IP \[bu] 2
|
|
Upload releases to github too
|
|
.RE
|
|
.IP \[bu] 2
|
|
Swift
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Fix sync for chunked files
|
|
.RE
|
|
.IP \[bu] 2
|
|
OneDrive
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Re\-enable server side copy
|
|
.IP \[bu] 2
|
|
Don\[cq]t mask HTTP error codes with JSON decode error
|
|
.RE
|
|
.IP \[bu] 2
|
|
S3
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Fix corrupting Content\-Type on mod time update (thanks Joseph Spurrier)
|
|
.RE
|
|
.SS v1.25 \- 2015\-11\-14
|
|
.IP \[bu] 2
|
|
New features
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Implement Hubic storage system
|
|
.RE
|
|
.IP \[bu] 2
|
|
Fixes
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Fix deletion of some excluded files without \[en]delete\-excluded
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
This could have deleted files unexpectedly on sync
|
|
.IP \[bu] 2
|
|
Always check first with \f[C]\-\-dry\-run\f[R]!
|
|
.RE
|
|
.RE
|
|
.IP \[bu] 2
|
|
Swift
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Stop SetModTime losing metadata (eg X\-Object\-Manifest)
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
This could have caused data loss for files > 5GB in size
|
|
.RE
|
|
.IP \[bu] 2
|
|
Use ContentType from Object to avoid lookups in listings
|
|
.RE
|
|
.IP \[bu] 2
|
|
OneDrive
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
disable server side copy as it seems to be broken at Microsoft
|
|
.RE
|
|
.SS v1.24 \- 2015\-11\-07
|
|
.IP \[bu] 2
|
|
New features
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Add support for Microsoft OneDrive
|
|
.IP \[bu] 2
|
|
Add \f[C]\-\-no\-check\-certificate\f[R] option to disable server
|
|
certificate verification
|
|
.IP \[bu] 2
|
|
Add async readahead buffer for faster transfer of big files
|
|
.RE
|
|
.IP \[bu] 2
|
|
Fixes
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Allow spaces in remotes and check remote names for validity at creation
|
|
time
|
|
.IP \[bu] 2
|
|
Allow `&' and disallow `:' in Windows filenames.
|
|
.RE
|
|
.IP \[bu] 2
|
|
Swift
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Ignore directory marker objects where appropriate \- allows working with
|
|
Hubic
|
|
.IP \[bu] 2
|
|
Don\[cq]t delete the container if fs wasn\[cq]t at root
|
|
.RE
|
|
.IP \[bu] 2
|
|
S3
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Don\[cq]t delete the bucket if fs wasn\[cq]t at root
|
|
.RE
|
|
.IP \[bu] 2
|
|
Google Cloud Storage
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Don\[cq]t delete the bucket if fs wasn\[cq]t at root
|
|
.RE
|
|
.SS v1.23 \- 2015\-10\-03
|
|
.IP \[bu] 2
|
|
New features
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Implement \f[C]rclone size\f[R] for measuring remotes
|
|
.RE
|
|
.IP \[bu] 2
|
|
Fixes
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Fix headless config for drive and gcs
|
|
.IP \[bu] 2
|
|
Tell the user they should try again if the webserver method failed
|
|
.IP \[bu] 2
|
|
Improve output of \f[C]\-\-dump\-headers\f[R]
|
|
.RE
|
|
.IP \[bu] 2
|
|
S3
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Allow anonymous access to public buckets
|
|
.RE
|
|
.IP \[bu] 2
|
|
Swift
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Stop chunked operations logging \[lq]Failed to read info: Object Not
|
|
Found\[rq]
|
|
.IP \[bu] 2
|
|
Use Content\-Length on uploads for extra reliability
|
|
.RE
|
|
.SS v1.22 \- 2015\-09\-28
|
|
.IP \[bu] 2
|
|
Implement rsync like include and exclude flags
|
|
.IP \[bu] 2
|
|
swift
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Support files > 5GB \- thanks Sergey Tolmachev
|
|
.RE
|
|
.SS v1.21 \- 2015\-09\-22
|
|
.IP \[bu] 2
|
|
New features
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Display individual transfer progress
|
|
.IP \[bu] 2
|
|
Make lsl output times in localtime
|
|
.RE
|
|
.IP \[bu] 2
|
|
Fixes
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Fix allowing user to override credentials again in Drive, GCS and ACD
|
|
.RE
|
|
.IP \[bu] 2
|
|
Amazon Drive
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Implement compliant pacing scheme
|
|
.RE
|
|
.IP \[bu] 2
|
|
Google Drive
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Make directory reads concurrent for increased speed.
|
|
.RE
|
|
.SS v1.20 \- 2015\-09\-15
|
|
.IP \[bu] 2
|
|
New features
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Amazon Drive support
|
|
.IP \[bu] 2
|
|
Oauth support redone \- fix many bugs and improve usability
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Use \[lq]golang.org/x/oauth2\[rq] as oauth library of choice
|
|
.IP \[bu] 2
|
|
Improve oauth usability for smoother initial signup
|
|
.IP \[bu] 2
|
|
drive, googlecloudstorage: optionally use auto config for the oauth
|
|
token
|
|
.RE
|
|
.IP \[bu] 2
|
|
Implement \[en]dump\-headers and \[en]dump\-bodies debug flags
|
|
.IP \[bu] 2
|
|
Show multiple matched commands if abbreviation too short
|
|
.IP \[bu] 2
|
|
Implement server side move where possible
|
|
.RE
|
|
.IP \[bu] 2
|
|
local
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Always use UNC paths internally on Windows \- fixes a lot of bugs
|
|
.RE
|
|
.IP \[bu] 2
|
|
dropbox
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
force use of our custom transport which makes timeouts work
|
|
.RE
|
|
.IP \[bu] 2
|
|
Thanks to Klaus Post for lots of help with this release
|
|
.SS v1.19 \- 2015\-08\-28
|
|
.IP \[bu] 2
|
|
New features
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Server side copies for s3/swift/drive/dropbox/gcs
|
|
.IP \[bu] 2
|
|
Move command \- uses server side copies if it can
|
|
.IP \[bu] 2
|
|
Implement \[en]retries flag \- tries 3 times by default
|
|
.IP \[bu] 2
|
|
Build for plan9/amd64 and solaris/amd64 too
|
|
.RE
|
|
.IP \[bu] 2
|
|
Fixes
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Make a current version download with a fixed URL for scripting
|
|
.IP \[bu] 2
|
|
Ignore rmdir in limited fs rather than throwing error
|
|
.RE
|
|
.IP \[bu] 2
|
|
dropbox
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Increase chunk size to improve upload speeds massively
|
|
.IP \[bu] 2
|
|
Issue an error message when trying to upload bad file name
|
|
.RE
|
|
.SS v1.18 \- 2015\-08\-17
|
|
.IP \[bu] 2
|
|
drive
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Add \f[C]\-\-drive\-use\-trash\f[R] flag so rclone trashes instead of
|
|
deletes
|
|
.IP \[bu] 2
|
|
Add \[lq]Forbidden to download\[rq] message for files with no
|
|
downloadURL
|
|
.RE
|
|
.IP \[bu] 2
|
|
dropbox
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Remove datastore
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
This was deprecated and it caused a lot of problems
|
|
.IP \[bu] 2
|
|
Modification times and MD5SUMs no longer stored
|
|
.RE
|
|
.IP \[bu] 2
|
|
Fix uploading files > 2GB
|
|
.RE
|
|
.IP \[bu] 2
|
|
s3
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
use official AWS SDK from github.com/aws/aws\-sdk\-go
|
|
.IP \[bu] 2
|
|
\f[B]NB\f[R] will most likely require you to delete and recreate remote
|
|
.IP \[bu] 2
|
|
enable multipart upload which enables files > 5GB
|
|
.IP \[bu] 2
|
|
tested with Ceph / RadosGW / S3 emulation
|
|
.IP \[bu] 2
|
|
many thanks to Sam Liston and Brian Haymore at the Utah Center for High
|
|
Performance Computing (https://www.chpc.utah.edu/) for a Ceph test
|
|
account
|
|
.RE
|
|
.IP \[bu] 2
|
|
misc
|
|
.RS 2
|
|
.IP \[bu] 2
|
|
Show errors when reading the config file
|
|
.IP \[bu] 2
|
|
Do not print stats in quiet mode \- thanks Leonid Shalupov
|
|
.IP \[bu] 2
|
|
Add FAQ
|
|
.IP \[bu] 2
|
|
Fix created directories not obeying umask
|
|
.IP \[bu] 2
|
|
Linux installation instructions \- thanks Shimon Doodkin
|
|
.RE
|
|
.SS v1.17 \- 2015\-06\-14
|
|
.IP \[bu] 2
|
|
dropbox: fix case insensitivity issues \- thanks Leonid Shalupov
|
|
.SS v1.16 \- 2015\-06\-09
|
|
.IP \[bu] 2
|
|
Fix uploading big files which was causing timeouts or panics
|
|
.IP \[bu] 2
|
|
Don\[cq]t check md5sum after download with \[en]size\-only
|
|
.SS v1.15 \- 2015\-06\-06
|
|
.IP \[bu] 2
|
|
Add \[en]checksum flag to only discard transfers by MD5SUM \- thanks
|
|
Alex Couper
|
|
.IP \[bu] 2
|
|
Implement \[en]size\-only flag to sync on size not checksum & modtime
|
|
.IP \[bu] 2
|
|
Expand docs and remove duplicated information
|
|
.IP \[bu] 2
|
|
Document rclone\[cq]s limitations with directories
|
|
.IP \[bu] 2
|
|
dropbox: update docs about case insensitivity
|
|
.SS v1.14 \- 2015\-05\-21
|
|
.IP \[bu] 2
|
|
local: fix encoding of non utf\-8 file names \- fixes a duplicate file
|
|
problem
|
|
.IP \[bu] 2
|
|
drive: docs about rate limiting
|
|
.IP \[bu] 2
|
|
google cloud storage: Fix compile after API change in
|
|
\[lq]google.golang.org/api/storage/v1\[rq]
|
|
.SS v1.13 \- 2015\-05\-10
|
|
.IP \[bu] 2
|
|
Revise documentation (especially sync)
|
|
.IP \[bu] 2
|
|
Implement \[en]timeout and \[en]conntimeout
|
|
.IP \[bu] 2
|
|
s3: ignore etags from multipart uploads which aren\[cq]t md5sums
|
|
.SS v1.12 \- 2015\-03\-15
|
|
.IP \[bu] 2
|
|
drive: Use chunked upload for files above a certain size
|
|
.IP \[bu] 2
|
|
drive: add \[en]drive\-chunk\-size and \[en]drive\-upload\-cutoff
|
|
parameters
|
|
.IP \[bu] 2
|
|
drive: switch to insert from update when a failed copy deletes the
|
|
upload
|
|
.IP \[bu] 2
|
|
core: Log duplicate files if they are detected
|
|
.SS v1.11 \- 2015\-03\-04
|
|
.IP \[bu] 2
|
|
swift: add region parameter
|
|
.IP \[bu] 2
|
|
drive: fix crash on failed to update remote mtime
|
|
.IP \[bu] 2
|
|
In remote paths, change native directory separators to /
|
|
.IP \[bu] 2
|
|
Add synchronization to ls/lsl/lsd output to stop corruptions
|
|
.IP \[bu] 2
|
|
Ensure all stats/log messages to go stderr
|
|
.IP \[bu] 2
|
|
Add \[en]log\-file flag to log everything (including panics) to file
|
|
.IP \[bu] 2
|
|
Make it possible to disable stats printing with \[en]stats=0
|
|
.IP \[bu] 2
|
|
Implement \[en]bwlimit to limit data transfer bandwidth
|
|
.SS v1.10 \- 2015\-02\-12
|
|
.IP \[bu] 2
|
|
s3: list an unlimited number of items
|
|
.IP \[bu] 2
|
|
Fix getting stuck in the configurator
|
|
.SS v1.09 \- 2015\-02\-07
|
|
.IP \[bu] 2
|
|
windows: Stop drive letters (eg C:) getting mixed up with remotes (eg
|
|
drive:)
|
|
.IP \[bu] 2
|
|
local: Fix directory separators on Windows
|
|
.IP \[bu] 2
|
|
drive: fix rate limit exceeded errors
|
|
.SS v1.08 \- 2015\-02\-04
|
|
.IP \[bu] 2
|
|
drive: fix subdirectory listing to not list entire drive
|
|
.IP \[bu] 2
|
|
drive: Fix SetModTime
|
|
.IP \[bu] 2
|
|
dropbox: adapt code to recent library changes
|
|
.SS v1.07 \- 2014\-12\-23
|
|
.IP \[bu] 2
|
|
google cloud storage: fix memory leak
|
|
.SS v1.06 \- 2014\-12\-12
|
|
.IP \[bu] 2
|
|
Fix \[lq]Couldn\[cq]t find home directory\[rq] on OSX
|
|
.IP \[bu] 2
|
|
swift: Add tenant parameter
|
|
.IP \[bu] 2
|
|
Use new location of Google API packages
|
|
.SS v1.05 \- 2014\-08\-09
|
|
.IP \[bu] 2
|
|
Improved tests and consequently lots of minor fixes
|
|
.IP \[bu] 2
|
|
core: Fix race detected by go race detector
|
|
.IP \[bu] 2
|
|
core: Fixes after running errcheck
|
|
.IP \[bu] 2
|
|
drive: reset root directory on Rmdir and Purge
|
|
.IP \[bu] 2
|
|
fs: Document that Purger returns error on empty directory, test and fix
|
|
.IP \[bu] 2
|
|
google cloud storage: fix ListDir on subdirectory
|
|
.IP \[bu] 2
|
|
google cloud storage: re\-read metadata in SetModTime
|
|
.IP \[bu] 2
|
|
s3: make reading metadata more reliable to work around eventual
|
|
consistency problems
|
|
.IP \[bu] 2
|
|
s3: strip trailing / from ListDir()
|
|
.IP \[bu] 2
|
|
swift: return directories without / in ListDir
|
|
.SS v1.04 \- 2014\-07\-21
|
|
.IP \[bu] 2
|
|
google cloud storage: Fix crash on Update
|
|
.SS v1.03 \- 2014\-07\-20
|
|
.IP \[bu] 2
|
|
swift, s3, dropbox: fix updated files being marked as corrupted
|
|
.IP \[bu] 2
|
|
Make compile with go 1.1 again
|
|
.SS v1.02 \- 2014\-07\-19
|
|
.IP \[bu] 2
|
|
Implement Dropbox remote
|
|
.IP \[bu] 2
|
|
Implement Google Cloud Storage remote
|
|
.IP \[bu] 2
|
|
Verify Md5sums and Sizes after copies
|
|
.IP \[bu] 2
|
|
Remove times from \[lq]ls\[rq] command \- lists sizes only
|
|
.IP \[bu] 2
|
|
Add add \[lq]lsl\[rq] \- lists times and sizes
|
|
.IP \[bu] 2
|
|
Add \[lq]md5sum\[rq] command
|
|
.SS v1.01 \- 2014\-07\-04
|
|
.IP \[bu] 2
|
|
drive: fix transfer of big files using up lots of memory
|
|
.SS v1.00 \- 2014\-07\-03
|
|
.IP \[bu] 2
|
|
drive: fix whole second dates
|
|
.SS v0.99 \- 2014\-06\-26
|
|
.IP \[bu] 2
|
|
Fix \[en]dry\-run not working
|
|
.IP \[bu] 2
|
|
Make compatible with go 1.1
|
|
.SS v0.98 \- 2014\-05\-30
|
|
.IP \[bu] 2
|
|
s3: Treat missing Content\-Length as 0 for some ceph installations
|
|
.IP \[bu] 2
|
|
rclonetest: add file with a space in
|
|
.SS v0.97 \- 2014\-05\-05
|
|
.IP \[bu] 2
|
|
Implement copying of single files
|
|
.IP \[bu] 2
|
|
s3 & swift: support paths inside containers/buckets
|
|
.SS v0.96 \- 2014\-04\-24
|
|
.IP \[bu] 2
|
|
drive: Fix multiple files of same name being created
|
|
.IP \[bu] 2
|
|
drive: Use o.Update and fs.Put to optimise transfers
|
|
.IP \[bu] 2
|
|
Add version number, \-V and \[en]version
|
|
.SS v0.95 \- 2014\-03\-28
|
|
.IP \[bu] 2
|
|
rclone.org: website, docs and graphics
|
|
.IP \[bu] 2
|
|
drive: fix path parsing
|
|
.SS v0.94 \- 2014\-03\-27
|
|
.IP \[bu] 2
|
|
Change remote format one last time
|
|
.IP \[bu] 2
|
|
GNU style flags
|
|
.SS v0.93 \- 2014\-03\-16
|
|
.IP \[bu] 2
|
|
drive: store token in config file
|
|
.IP \[bu] 2
|
|
cross compile other versions
|
|
.IP \[bu] 2
|
|
set strict permissions on config file
|
|
.SS v0.92 \- 2014\-03\-15
|
|
.IP \[bu] 2
|
|
Config fixes and \[en]config option
|
|
.SS v0.91 \- 2014\-03\-15
|
|
.IP \[bu] 2
|
|
Make config file
|
|
.SS v0.90 \- 2013\-06\-27
|
|
.IP \[bu] 2
|
|
Project named rclone
|
|
.SS v0.00 \- 2012\-11\-18
|
|
.IP \[bu] 2
|
|
Project started
|
|
.SH Bugs and Limitations
|
|
.SS Limitations
|
|
.SS Directory timestamps aren\[cq]t preserved
|
|
.PP
|
|
Rclone doesn\[cq]t currently preserve the timestamps of directories.
|
|
This is because rclone only really considers objects when syncing.
|
|
.SS Rclone struggles with millions of files in a directory
|
|
.PP
|
|
Currently rclone loads each directory entirely into memory before using
|
|
it.
|
|
Since each Rclone object takes 0.5k\-1k of memory this can take a very
|
|
long time and use an extremely large amount of memory.
|
|
.PP
|
|
Millions of files in a directory tend caused by software writing cloud
|
|
storage (eg S3 buckets).
|
|
.SS Bucket based remotes and folders
|
|
.PP
|
|
Bucket based remotes (eg S3/GCS/Swift/B2) do not have a concept of
|
|
directories.
|
|
Rclone therefore cannot create directories in them which means that
|
|
empty directories on a bucket based remote will tend to disappear.
|
|
.PP
|
|
Some software creates empty keys ending in \f[C]/\f[R] as directory
|
|
markers.
|
|
Rclone doesn\[cq]t do this as it potentially creates more objects and
|
|
costs more.
|
|
It may do in future (probably with a flag).
|
|
.SS Bugs
|
|
.PP
|
|
Bugs are stored in rclone\[cq]s GitHub project:
|
|
.IP \[bu] 2
|
|
Reported
|
|
bugs (https://github.com/rclone/rclone/issues?q=is%3Aopen+is%3Aissue+label%3Abug)
|
|
.IP \[bu] 2
|
|
Known
|
|
issues (https://github.com/rclone/rclone/issues?q=is%3Aopen+is%3Aissue+milestone%3A%22Known+Problem%22)
|
|
.SS Frequently Asked Questions
|
|
.SS Do all cloud storage systems support all rclone commands
|
|
.PP
|
|
Yes they do.
|
|
All the rclone commands (eg \f[C]sync\f[R], \f[C]copy\f[R] etc) will
|
|
work on all the remote storage systems.
|
|
.SS Can I copy the config from one machine to another
|
|
.PP
|
|
Sure! Rclone stores all of its config in a single file.
|
|
If you want to find this file, run \f[C]rclone config file\f[R] which
|
|
will tell you where it is.
|
|
.PP
|
|
See the remote setup docs (https://rclone.org/remote_setup/) for more
|
|
info.
|
|
.SS How do I configure rclone on a remote / headless box with no browser?
|
|
.PP
|
|
This has now been documented in its own remote setup
|
|
page (https://rclone.org/remote_setup/).
|
|
.SS Can rclone sync directly from drive to s3
|
|
.PP
|
|
Rclone can sync between two remote cloud storage systems just fine.
|
|
.PP
|
|
Note that it effectively downloads the file and uploads it again, so the
|
|
node running rclone would need to have lots of bandwidth.
|
|
.PP
|
|
The syncs would be incremental (on a file by file basis).
|
|
.PP
|
|
Eg
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
rclone sync drive:Folder s3:bucket
|
|
\f[R]
|
|
.fi
|
|
.SS Using rclone from multiple locations at the same time
|
|
.PP
|
|
You can use rclone from multiple places at the same time if you choose
|
|
different subdirectory for the output, eg
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
Server A> rclone sync /tmp/whatever remote:ServerA
|
|
Server B> rclone sync /tmp/whatever remote:ServerB
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
If you sync to the same directory then you should use rclone copy
|
|
otherwise the two instances of rclone may delete each other\[cq]s files,
|
|
eg
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
Server A> rclone copy /tmp/whatever remote:Backup
|
|
Server B> rclone copy /tmp/whatever remote:Backup
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
The file names you upload from Server A and Server B should be different
|
|
in this case, otherwise some file systems (eg Drive) may make
|
|
duplicates.
|
|
.SS Why doesn\[cq]t rclone support partial transfers / binary diffs like rsync?
|
|
.PP
|
|
Rclone stores each file you transfer as a native object on the remote
|
|
cloud storage system.
|
|
This means that you can see the files you upload as expected using
|
|
alternative access methods (eg using the Google Drive web interface).
|
|
There is a 1:1 mapping between files on your hard disk and objects
|
|
created in the cloud storage system.
|
|
.PP
|
|
Cloud storage systems (at least none I\[cq]ve come across yet) don\[cq]t
|
|
support partially uploading an object.
|
|
You can\[cq]t take an existing object, and change some bytes in the
|
|
middle of it.
|
|
.PP
|
|
It would be possible to make a sync system which stored binary diffs
|
|
instead of whole objects like rclone does, but that would break the 1:1
|
|
mapping of files on your hard disk to objects in the remote cloud
|
|
storage system.
|
|
.PP
|
|
All the cloud storage systems support partial downloads of content, so
|
|
it would be possible to make partial downloads work.
|
|
However to make this work efficiently this would require storing a
|
|
significant amount of metadata, which breaks the desired 1:1 mapping of
|
|
files to objects.
|
|
.SS Can rclone do bi\-directional sync?
|
|
.PP
|
|
No, not at present.
|
|
rclone only does uni\-directional sync from A \-> B.
|
|
It may do in the future though since it has all the primitives \- it
|
|
just requires writing the algorithm to do it.
|
|
.SS Can I use rclone with an HTTP proxy?
|
|
.PP
|
|
Yes.
|
|
rclone will follow the standard environment variables for proxies,
|
|
similar to cURL and other programs.
|
|
.PP
|
|
In general the variables are called \f[C]http_proxy\f[R] (for services
|
|
reached over \f[C]http\f[R]) and \f[C]https_proxy\f[R] (for services
|
|
reached over \f[C]https\f[R]).
|
|
Most public services will be using \f[C]https\f[R], but you may wish to
|
|
set both.
|
|
.PP
|
|
The content of the variable is \f[C]protocol://server:port\f[R].
|
|
The protocol value is the one used to talk to the proxy server, itself,
|
|
and is commonly either \f[C]http\f[R] or \f[C]socks5\f[R].
|
|
.PP
|
|
Slightly annoyingly, there is no \f[I]standard\f[R] for the name; some
|
|
applications may use \f[C]http_proxy\f[R] but another one
|
|
\f[C]HTTP_PROXY\f[R].
|
|
The \f[C]Go\f[R] libraries used by \f[C]rclone\f[R] will try both
|
|
variations, but you may wish to set all possibilities.
|
|
So, on Linux, you may end up with code similar to
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
export http_proxy=http://proxyserver:12345
|
|
export https_proxy=$http_proxy
|
|
export HTTP_PROXY=$http_proxy
|
|
export HTTPS_PROXY=$http_proxy
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
The \f[C]NO_PROXY\f[R] allows you to disable the proxy for specific
|
|
hosts.
|
|
Hosts must be comma separated, and can contain domains or parts.
|
|
For instance \[lq]foo.com\[rq] also matches \[lq]bar.foo.com\[rq].
|
|
.PP
|
|
e.g.
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
export no_proxy=localhost,127.0.0.0/8,my.host.name
|
|
export NO_PROXY=$no_proxy
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
Note that the ftp backend does not support \f[C]ftp_proxy\f[R] yet.
|
|
.SS Rclone gives x509: failed to load system roots and no roots provided error
|
|
.PP
|
|
This means that \f[C]rclone\f[R] can\[cq]t file the SSL root
|
|
certificates.
|
|
Likely you are running \f[C]rclone\f[R] on a NAS with a cut\-down Linux
|
|
OS, or possibly on Solaris.
|
|
.PP
|
|
Rclone (via the Go runtime) tries to load the root certificates from
|
|
these places on Linux.
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
\[dq]/etc/ssl/certs/ca\-certificates.crt\[dq], // Debian/Ubuntu/Gentoo etc.
|
|
\[dq]/etc/pki/tls/certs/ca\-bundle.crt\[dq], // Fedora/RHEL
|
|
\[dq]/etc/ssl/ca\-bundle.pem\[dq], // OpenSUSE
|
|
\[dq]/etc/pki/tls/cacert.pem\[dq], // OpenELEC
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
So doing something like this should fix the problem.
|
|
It also sets the time which is important for SSL to work properly.
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
mkdir \-p /etc/ssl/certs/
|
|
curl \-o /etc/ssl/certs/ca\-certificates.crt https://raw.githubusercontent.com/bagder/ca\-bundle/master/ca\-bundle.crt
|
|
ntpclient \-s \-h pool.ntp.org
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
The two environment variables \f[C]SSL_CERT_FILE\f[R] and
|
|
\f[C]SSL_CERT_DIR\f[R], mentioned in the x509
|
|
package (https://godoc.org/crypto/x509), provide an additional way to
|
|
provide the SSL root certificates.
|
|
.PP
|
|
Note that you may need to add the \f[C]\-\-insecure\f[R] option to the
|
|
\f[C]curl\f[R] command line if it doesn\[cq]t work without.
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
curl \-\-insecure \-o /etc/ssl/certs/ca\-certificates.crt https://raw.githubusercontent.com/bagder/ca\-bundle/master/ca\-bundle.crt
|
|
\f[R]
|
|
.fi
|
|
.SS Rclone gives Failed to load config file: function not implemented error
|
|
.PP
|
|
Likely this means that you are running rclone on Linux version not
|
|
supported by the go runtime, ie earlier than version 2.6.23.
|
|
.PP
|
|
See the system requirements section in the go install
|
|
docs (https://golang.org/doc/install) for full details.
|
|
.SS All my uploaded docx/xlsx/pptx files appear as archive/zip
|
|
.PP
|
|
This is caused by uploading these files from a Windows computer which
|
|
hasn\[cq]t got the Microsoft Office suite installed.
|
|
The easiest way to fix is to install the Word viewer and the Microsoft
|
|
Office Compatibility Pack for Word, Excel, and PowerPoint 2007 and later
|
|
versions\[cq] file formats
|
|
.SS tcp lookup some.domain.com no such host
|
|
.PP
|
|
This happens when rclone cannot resolve a domain.
|
|
Please check that your DNS setup is generally working, e.g.
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
# both should print a long list of possible IP addresses
|
|
dig www.googleapis.com # resolve using your default DNS
|
|
dig www.googleapis.com \[at]8.8.8.8 # resolve with Google\[aq]s DNS server
|
|
\f[R]
|
|
.fi
|
|
.PP
|
|
If you are using \f[C]systemd\-resolved\f[R] (default on Arch Linux),
|
|
ensure it is at version 233 or higher.
|
|
Previous releases contain a bug which causes not all domains to be
|
|
resolved properly.
|
|
.PP
|
|
Additionally with the \f[C]GODEBUG=netdns=\f[R] environment variable the
|
|
Go resolver decision can be influenced.
|
|
This also allows to resolve certain issues with DNS resolution.
|
|
See the name resolution section in the go
|
|
docs (https://golang.org/pkg/net/#hdr-Name_Resolution).
|
|
.SS The total size reported in the stats for a sync is wrong and keeps changing
|
|
.PP
|
|
It is likely you have more than 10,000 files that need to be synced.
|
|
By default rclone only gets 10,000 files ahead in a sync so as not to
|
|
use up too much memory.
|
|
You can change this default with the
|
|
\[en]max\-backlog (https://rclone.org/docs/#max-backlog-n) flag.
|
|
.SS Rclone is using too much memory or appears to have a memory leak
|
|
.PP
|
|
Rclone is written in Go which uses a garbage collector.
|
|
The default settings for the garbage collector mean that it runs when
|
|
the heap size has doubled.
|
|
.PP
|
|
However it is possible to tune the garbage collector to use less memory
|
|
by setting GOGC (https://dave.cheney.net/tag/gogc) to a lower value, say
|
|
\f[C]export GOGC=20\f[R].
|
|
This will make the garbage collector work harder, reducing memory size
|
|
at the expense of CPU usage.
|
|
.PP
|
|
The most common cause of rclone using lots of memory is a single
|
|
directory with thousands or millions of files in.
|
|
Rclone has to load this entirely into memory as rclone objects.
|
|
Each rclone object takes 0.5k\-1k of memory.
|
|
.SS License
|
|
.PP
|
|
This is free software under the terms of MIT the license (check the
|
|
COPYING file included with the source code).
|
|
.IP
|
|
.nf
|
|
\f[C]
|
|
Copyright (C) 2019 by Nick Craig\-Wood https://www.craig\-wood.com/nick/
|
|
|
|
Permission is hereby granted, free of charge, to any person obtaining a copy
|
|
of this software and associated documentation files (the \[dq]Software\[dq]), to deal
|
|
in the Software without restriction, including without limitation the rights
|
|
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
|
|
copies of the Software, and to permit persons to whom the Software is
|
|
furnished to do so, subject to the following conditions:
|
|
|
|
The above copyright notice and this permission notice shall be included in
|
|
all copies or substantial portions of the Software.
|
|
|
|
THE SOFTWARE IS PROVIDED \[dq]AS IS\[dq], WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
|
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
|
|
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
|
|
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
|
|
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
|
|
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
|
|
THE SOFTWARE.
|
|
\f[R]
|
|
.fi
|
|
.SS Authors
|
|
.IP \[bu] 2
|
|
Nick Craig\-Wood <nick@craig-wood.com>
|
|
.SS Contributors
|
|
.PP
|
|
{{< rem
|
|
\f[C]email addresses removed from here need to be addeed to bin/.ignore\-emails to make sure update\-authors.py doesn\[aq]t immediately put them back in again.\f[R]
|
|
>}}
|
|
.IP \[bu] 2
|
|
Alex Couper <amcouper@gmail.com>
|
|
.IP \[bu] 2
|
|
Leonid Shalupov <leonid@shalupov.com> <shalupov@diverse.org.ru>
|
|
.IP \[bu] 2
|
|
Shimon Doodkin <helpmepro1@gmail.com>
|
|
.IP \[bu] 2
|
|
Colin Nicholson <colin@colinn.com>
|
|
.IP \[bu] 2
|
|
Klaus Post <klauspost@gmail.com>
|
|
.IP \[bu] 2
|
|
Sergey Tolmachev <tolsi.ru@gmail.com>
|
|
.IP \[bu] 2
|
|
Adriano Aur\['e]lio Meirelles <adriano@atinge.com>
|
|
.IP \[bu] 2
|
|
C.
|
|
Bess <cbess@users.noreply.github.com>
|
|
.IP \[bu] 2
|
|
Dmitry Burdeev <dibu28@gmail.com>
|
|
.IP \[bu] 2
|
|
Joseph Spurrier <github@josephspurrier.com>
|
|
.IP \[bu] 2
|
|
Bj\[:o]rn Harrtell <bjorn@wololo.org>
|
|
.IP \[bu] 2
|
|
Xavier Lucas <xavier.lucas@corp.ovh.com>
|
|
.IP \[bu] 2
|
|
Werner Beroux <werner@beroux.com>
|
|
.IP \[bu] 2
|
|
Brian Stengaard <brian@stengaard.eu>
|
|
.IP \[bu] 2
|
|
Jakub Gedeon <jgedeon@sofi.com>
|
|
.IP \[bu] 2
|
|
Jim Tittsler <jwt@onjapan.net>
|
|
.IP \[bu] 2
|
|
Michal Witkowski <michal@improbable.io>
|
|
.IP \[bu] 2
|
|
Fabian Ruff <fabian.ruff@sap.com>
|
|
.IP \[bu] 2
|
|
Leigh Klotz <klotz@quixey.com>
|
|
.IP \[bu] 2
|
|
Romain Lapray <lapray.romain@gmail.com>
|
|
.IP \[bu] 2
|
|
Justin R.
|
|
Wilson <jrw972@gmail.com>
|
|
.IP \[bu] 2
|
|
Antonio Messina <antonio.s.messina@gmail.com>
|
|
.IP \[bu] 2
|
|
Stefan G.
|
|
Weichinger <office@oops.co.at>
|
|
.IP \[bu] 2
|
|
Per Cederberg <cederberg@gmail.com>
|
|
.IP \[bu] 2
|
|
Radek \[vS]enfeld <rush@logic.cz>
|
|
.IP \[bu] 2
|
|
Fredrik Fornwall <fredrik@fornwall.net>
|
|
.IP \[bu] 2
|
|
Asko Tamm <asko@deekit.net>
|
|
.IP \[bu] 2
|
|
xor\-zz <xor@gstocco.com>
|
|
.IP \[bu] 2
|
|
Tomasz Mazur <tmazur90@gmail.com>
|
|
.IP \[bu] 2
|
|
Marco Paganini <paganini@paganini.net>
|
|
.IP \[bu] 2
|
|
Felix B\[:u]nemann <buenemann@louis.info>
|
|
.IP \[bu] 2
|
|
Durval Menezes <jmrclone@durval.com>
|
|
.IP \[bu] 2
|
|
Luiz Carlos Rumbelsperger Viana <maxd13_luiz_carlos@hotmail.com>
|
|
.IP \[bu] 2
|
|
Stefan Breunig <stefan-github@yrden.de>
|
|
.IP \[bu] 2
|
|
Alishan Ladhani <ali-l@users.noreply.github.com>
|
|
.IP \[bu] 2
|
|
0xJAKE <0xJAKE@users.noreply.github.com>
|
|
.IP \[bu] 2
|
|
Thibault Molleman <thibaultmol@users.noreply.github.com>
|
|
.IP \[bu] 2
|
|
Scott McGillivray <scott.mcgillivray@gmail.com>
|
|
.IP \[bu] 2
|
|
Bj\[/o]rn Erik Pedersen <bjorn.erik.pedersen@gmail.com>
|
|
.IP \[bu] 2
|
|
Lukas Loesche <lukas@mesosphere.io>
|
|
.IP \[bu] 2
|
|
emyarod <allllaboutyou@gmail.com>
|
|
.IP \[bu] 2
|
|
T.C.
|
|
Ferguson <tcf909@gmail.com>
|
|
.IP \[bu] 2
|
|
Brandur <brandur@mutelight.org>
|
|
.IP \[bu] 2
|
|
Dario Giovannetti <dev@dariogiovannetti.net>
|
|
.IP \[bu] 2
|
|
K\['a]roly Ol\['a]h <okaresz@aol.com>
|
|
.IP \[bu] 2
|
|
Jon Yergatian <jon@macfanatic.ca>
|
|
.IP \[bu] 2
|
|
Jack Schmidt <github@mowsey.org>
|
|
.IP \[bu] 2
|
|
Dedsec1 <Dedsec1@users.noreply.github.com>
|
|
.IP \[bu] 2
|
|
Hisham Zarka <hzarka@gmail.com>
|
|
.IP \[bu] 2
|
|
J\['e]r\[^o]me Vizcaino <jerome.vizcaino@gmail.com>
|
|
.IP \[bu] 2
|
|
Mike Tesch <mjt6129@rit.edu>
|
|
.IP \[bu] 2
|
|
Marvin Watson <marvwatson@users.noreply.github.com>
|
|
.IP \[bu] 2
|
|
Danny Tsai <danny8376@gmail.com>
|
|
.IP \[bu] 2
|
|
Yoni Jah <yonjah+git@gmail.com> <yonjah+github@gmail.com>
|
|
.IP \[bu] 2
|
|
Stephen Harris <github@spuddy.org> <sweharris@users.noreply.github.com>
|
|
.IP \[bu] 2
|
|
Ihor Dvoretskyi <ihor.dvoretskyi@gmail.com>
|
|
.IP \[bu] 2
|
|
Jon Craton <jncraton@gmail.com>
|
|
.IP \[bu] 2
|
|
Hraban Luyat <hraban@0brg.net>
|
|
.IP \[bu] 2
|
|
Michael Ledin <mledin89@gmail.com>
|
|
.IP \[bu] 2
|
|
Martin Kristensen <me@azgul.com>
|
|
.IP \[bu] 2
|
|
Too Much IO <toomuchio@users.noreply.github.com>
|
|
.IP \[bu] 2
|
|
Anisse Astier <anisse@astier.eu>
|
|
.IP \[bu] 2
|
|
Zahiar Ahmed <zahiar@live.com>
|
|
.IP \[bu] 2
|
|
Igor Kharin <igorkharin@gmail.com>
|
|
.IP \[bu] 2
|
|
Bill Zissimopoulos <billziss@navimatics.com>
|
|
.IP \[bu] 2
|
|
Bob Potter <bobby.potter@gmail.com>
|
|
.IP \[bu] 2
|
|
Steven Lu <tacticalazn@gmail.com>
|
|
.IP \[bu] 2
|
|
Sjur Fredriksen <sjurtf@ifi.uio.no>
|
|
.IP \[bu] 2
|
|
Ruwbin <hubus12345@gmail.com>
|
|
.IP \[bu] 2
|
|
Fabian M\[:o]ller <fabianm88@gmail.com> <f.moeller@nynex.de>
|
|
.IP \[bu] 2
|
|
Edward Q.
|
|
Bridges <github@eqbridges.com>
|
|
.IP \[bu] 2
|
|
Vasiliy Tolstov <v.tolstov@selfip.ru>
|
|
.IP \[bu] 2
|
|
Harshavardhana <harsha@minio.io>
|
|
.IP \[bu] 2
|
|
sainaen <sainaen@gmail.com>
|
|
.IP \[bu] 2
|
|
gdm85 <gdm85@users.noreply.github.com>
|
|
.IP \[bu] 2
|
|
Yaroslav Halchenko <debian@onerussian.com>
|
|
.IP \[bu] 2
|
|
John Papandriopoulos <jpap@users.noreply.github.com>
|
|
.IP \[bu] 2
|
|
Zhiming Wang <zmwangx@gmail.com>
|
|
.IP \[bu] 2
|
|
Andy Pilate <cubox@cubox.me>
|
|
.IP \[bu] 2
|
|
Oliver Heyme <olihey@googlemail.com> <olihey@users.noreply.github.com>
|
|
<de8olihe@lego.com>
|
|
.IP \[bu] 2
|
|
wuyu <wuyu@yunify.com>
|
|
.IP \[bu] 2
|
|
Andrei Dragomir <adragomi@adobe.com>
|
|
.IP \[bu] 2
|
|
Christian Br\[:u]ggemann <mail@cbruegg.com>
|
|
.IP \[bu] 2
|
|
Alex McGrath Kraak <amkdude@gmail.com>
|
|
.IP \[bu] 2
|
|
bpicode <bjoern.pirnay@googlemail.com>
|
|
.IP \[bu] 2
|
|
Daniel Jagszent <daniel@jagszent.de>
|
|
.IP \[bu] 2
|
|
Josiah White <thegenius2009@gmail.com>
|
|
.IP \[bu] 2
|
|
Ishuah Kariuki <kariuki@ishuah.com> <ishuah91@gmail.com>
|
|
.IP \[bu] 2
|
|
Jan Varho <jan@varho.org>
|
|
.IP \[bu] 2
|
|
Girish Ramakrishnan <girish@cloudron.io>
|
|
.IP \[bu] 2
|
|
LingMan <LingMan@users.noreply.github.com>
|
|
.IP \[bu] 2
|
|
Jacob McNamee <jacobmcnamee@gmail.com>
|
|
.IP \[bu] 2
|
|
jersou <jertux@gmail.com>
|
|
.IP \[bu] 2
|
|
thierry <thierry@substantiel.fr>
|
|
.IP \[bu] 2
|
|
Simon Leinen <simon.leinen@gmail.com> <ubuntu@s3-test.novalocal>
|
|
.IP \[bu] 2
|
|
Dan Dascalescu <ddascalescu+github@gmail.com>
|
|
.IP \[bu] 2
|
|
Jason Rose <jason@jro.io>
|
|
.IP \[bu] 2
|
|
Andrew Starr\-Bochicchio <a.starr.b@gmail.com>
|
|
.IP \[bu] 2
|
|
John Leach <john@johnleach.co.uk>
|
|
.IP \[bu] 2
|
|
Corban Raun <craun@instructure.com>
|
|
.IP \[bu] 2
|
|
Pierre Carlson <mpcarl@us.ibm.com>
|
|
.IP \[bu] 2
|
|
Ernest Borowski <er.borowski@gmail.com>
|
|
.IP \[bu] 2
|
|
Remus Bunduc <remus.bunduc@gmail.com>
|
|
.IP \[bu] 2
|
|
Iakov Davydov <iakov.davydov@unil.ch> <dav05.gith@myths.ru>
|
|
.IP \[bu] 2
|
|
Jakub Tasiemski <tasiemski@gmail.com>
|
|
.IP \[bu] 2
|
|
David Minor <dminor@saymedia.com>
|
|
.IP \[bu] 2
|
|
Tim Cooijmans <cooijmans.tim@gmail.com>
|
|
.IP \[bu] 2
|
|
Laurence <liuxy6@gmail.com>
|
|
.IP \[bu] 2
|
|
Giovanni Pizzi <gio.piz@gmail.com>
|
|
.IP \[bu] 2
|
|
Filip Bartodziej <filipbartodziej@gmail.com>
|
|
.IP \[bu] 2
|
|
Jon Fautley <jon@dead.li>
|
|
.IP \[bu] 2
|
|
lewapm <32110057+lewapm@users.noreply.github.com>
|
|
.IP \[bu] 2
|
|
Yassine Imounachen <yassine256@gmail.com>
|
|
.IP \[bu] 2
|
|
Chris Redekop <chris-redekop@users.noreply.github.com>
|
|
<chris.redekop@gmail.com>
|
|
.IP \[bu] 2
|
|
Jon Fautley <jon@adenoid.appstal.co.uk>
|
|
.IP \[bu] 2
|
|
Will Gunn <WillGunn@users.noreply.github.com>
|
|
.IP \[bu] 2
|
|
Lucas Bremgartner <lucas@bremis.ch>
|
|
.IP \[bu] 2
|
|
Jody Frankowski <jody.frankowski@gmail.com>
|
|
.IP \[bu] 2
|
|
Andreas Roussos <arouss1980@gmail.com>
|
|
.IP \[bu] 2
|
|
nbuchanan <nbuchanan@utah.gov>
|
|
.IP \[bu] 2
|
|
Durval Menezes <rclone@durval.com>
|
|
.IP \[bu] 2
|
|
Victor <vb-github@viblo.se>
|
|
.IP \[bu] 2
|
|
Mateusz <pabian.mateusz@gmail.com>
|
|
.IP \[bu] 2
|
|
Daniel Loader <spicypixel@gmail.com>
|
|
.IP \[bu] 2
|
|
David0rk <davidork@gmail.com>
|
|
.IP \[bu] 2
|
|
Alexander Neumann <alexander@bumpern.de>
|
|
.IP \[bu] 2
|
|
Giri Badanahatti <gbadanahatti@us.ibm.com@Giris-MacBook-Pro.local>
|
|
.IP \[bu] 2
|
|
Leo R.
|
|
Lundgren <leo@finalresort.org>
|
|
.IP \[bu] 2
|
|
wolfv <wolfv6@users.noreply.github.com>
|
|
.IP \[bu] 2
|
|
Dave Pedu <dave@davepedu.com>
|
|
.IP \[bu] 2
|
|
Stefan Lindblom <lindblom@spotify.com>
|
|
.IP \[bu] 2
|
|
seuffert <oliver@seuffert.biz>
|
|
.IP \[bu] 2
|
|
gbadanahatti <37121690+gbadanahatti@users.noreply.github.com>
|
|
.IP \[bu] 2
|
|
Keith Goldfarb <barkofdelight@gmail.com>
|
|
.IP \[bu] 2
|
|
Steve Kriss <steve@heptio.com>
|
|
.IP \[bu] 2
|
|
Chih\-Hsuan Yen <yan12125@gmail.com>
|
|
.IP \[bu] 2
|
|
Alexander Neumann <fd0@users.noreply.github.com>
|
|
.IP \[bu] 2
|
|
Matt Holt <mholt@users.noreply.github.com>
|
|
.IP \[bu] 2
|
|
Eri Bastos <bastos.eri@gmail.com>
|
|
.IP \[bu] 2
|
|
Michael P.
|
|
Dubner <pywebmail@list.ru>
|
|
.IP \[bu] 2
|
|
Antoine GIRARD <sapk@users.noreply.github.com>
|
|
.IP \[bu] 2
|
|
Mateusz Piotrowski <mpp302@gmail.com>
|
|
.IP \[bu] 2
|
|
Animosity022 <animosity22@users.noreply.github.com>
|
|
<earl.texter@gmail.com>
|
|
.IP \[bu] 2
|
|
Peter Baumgartner <pete@lincolnloop.com>
|
|
.IP \[bu] 2
|
|
Craig Rachel <craig@craigrachel.com>
|
|
.IP \[bu] 2
|
|
Michael G.
|
|
Noll <miguno@users.noreply.github.com>
|
|
.IP \[bu] 2
|
|
hensur <me@hensur.de>
|
|
.IP \[bu] 2
|
|
Oliver Heyme <de8olihe@lego.com>
|
|
.IP \[bu] 2
|
|
Richard Yang <richard@yenforyang.com>
|
|
.IP \[bu] 2
|
|
Piotr Oleszczyk <piotr.oleszczyk@gmail.com>
|
|
.IP \[bu] 2
|
|
Rodrigo <rodarima@gmail.com>
|
|
.IP \[bu] 2
|
|
NoLooseEnds <NoLooseEnds@users.noreply.github.com>
|
|
.IP \[bu] 2
|
|
Jakub Karlicek <jakub@karlicek.me>
|
|
.IP \[bu] 2
|
|
John Clayton <john@codemonkeylabs.com>
|
|
.IP \[bu] 2
|
|
Kasper Byrdal Nielsen <byrdal76@gmail.com>
|
|
.IP \[bu] 2
|
|
Benjamin Joseph Dag <bjdag1234@users.noreply.github.com>
|
|
.IP \[bu] 2
|
|
themylogin <themylogin@gmail.com>
|
|
.IP \[bu] 2
|
|
Onno Zweers <onno.zweers@surfsara.nl>
|
|
.IP \[bu] 2
|
|
Jasper Lievisse Adriaanse <jasper@humppa.nl>
|
|
.IP \[bu] 2
|
|
sandeepkru <sandeep.ummadi@gmail.com>
|
|
<sandeepkru@users.noreply.github.com>
|
|
.IP \[bu] 2
|
|
HerrH <atomtigerzoo@users.noreply.github.com>
|
|
.IP \[bu] 2
|
|
Andrew <4030760+sparkyman215@users.noreply.github.com>
|
|
.IP \[bu] 2
|
|
dan smith <XX1011@gmail.com>
|
|
.IP \[bu] 2
|
|
Oleg Kovalov <iamolegkovalov@gmail.com>
|
|
.IP \[bu] 2
|
|
Ruben Vandamme <github-com-00ff86@vandamme.email>
|
|
.IP \[bu] 2
|
|
Cnly <minecnly@gmail.com>
|
|
.IP \[bu] 2
|
|
Andres Alvarez <1671935+kir4h@users.noreply.github.com>
|
|
.IP \[bu] 2
|
|
reddi1 <xreddi@gmail.com>
|
|
.IP \[bu] 2
|
|
Matt Tucker <matthewtckr@gmail.com>
|
|
.IP \[bu] 2
|
|
Sebastian B\[:u]nger <buengese@gmail.com>
|
|
.IP \[bu] 2
|
|
Martin Polden <mpolden@mpolden.no>
|
|
.IP \[bu] 2
|
|
Alex Chen <Cnly@users.noreply.github.com>
|
|
.IP \[bu] 2
|
|
Denis <deniskovpen@gmail.com>
|
|
.IP \[bu] 2
|
|
bsteiss <35940619+bsteiss@users.noreply.github.com>
|
|
.IP \[bu] 2
|
|
C\['e]dric Connes <cedric.connes@gmail.com>
|
|
.IP \[bu] 2
|
|
Dr.\ Tobias Quathamer <toddy15@users.noreply.github.com>
|
|
.IP \[bu] 2
|
|
dcpu <42736967+dcpu@users.noreply.github.com>
|
|
.IP \[bu] 2
|
|
Sheldon Rupp <me@shel.io>
|
|
.IP \[bu] 2
|
|
albertony <12441419+albertony@users.noreply.github.com>
|
|
.IP \[bu] 2
|
|
cron410 <cron410@gmail.com>
|
|
.IP \[bu] 2
|
|
Anagh Kumar Baranwal <6824881+darthShadow@users.noreply.github.com>
|
|
.IP \[bu] 2
|
|
Felix Brucker <felix@felixbrucker.com>
|
|
.IP \[bu] 2
|
|
Santiago Rodr\['i]guez <scollazo@users.noreply.github.com>
|
|
.IP \[bu] 2
|
|
Craig Miskell <craig.miskell@fluxfederation.com>
|
|
.IP \[bu] 2
|
|
Antoine GIRARD <sapk@sapk.fr>
|
|
.IP \[bu] 2
|
|
Joanna Marek <joanna.marek@u2i.com>
|
|
.IP \[bu] 2
|
|
frenos <frenos@users.noreply.github.com>
|
|
.IP \[bu] 2
|
|
ssaqua <ssaqua@users.noreply.github.com>
|
|
.IP \[bu] 2
|
|
xnaas <me@xnaas.info>
|
|
.IP \[bu] 2
|
|
Frantisek Fuka <fuka@fuxoft.cz>
|
|
.IP \[bu] 2
|
|
Paul Kohout <pauljkohout@yahoo.com>
|
|
.IP \[bu] 2
|
|
dcpu <43330287+dcpu@users.noreply.github.com>
|
|
.IP \[bu] 2
|
|
jackyzy823 <jackyzy823@gmail.com>
|
|
.IP \[bu] 2
|
|
David Haguenauer <ml@kurokatta.org>
|
|
.IP \[bu] 2
|
|
teresy <hi.teresy@gmail.com>
|
|
.IP \[bu] 2
|
|
buergi <patbuergi@gmx.de>
|
|
.IP \[bu] 2
|
|
Florian Gamboeck <mail@floga.de>
|
|
.IP \[bu] 2
|
|
Ralf Hemberger <10364191+rhemberger@users.noreply.github.com>
|
|
.IP \[bu] 2
|
|
Scott Edlund <sedlund@users.noreply.github.com>
|
|
.IP \[bu] 2
|
|
Erik Swanson <erik@retailnext.net>
|
|
.IP \[bu] 2
|
|
Jake Coggiano <jake@stripe.com>
|
|
.IP \[bu] 2
|
|
brused27 <brused27@noemailaddress>
|
|
.IP \[bu] 2
|
|
Peter Kaminski <kaminski@istori.com>
|
|
.IP \[bu] 2
|
|
Henry Ptasinski <henry@logout.com>
|
|
.IP \[bu] 2
|
|
Alexander <kharkovalexander@gmail.com>
|
|
.IP \[bu] 2
|
|
Garry McNulty <garrmcnu@gmail.com>
|
|
.IP \[bu] 2
|
|
Mathieu Carbou <mathieu.carbou@gmail.com>
|
|
.IP \[bu] 2
|
|
Mark Otway <mark@otway.com>
|
|
.IP \[bu] 2
|
|
William Cocker <37018962+WilliamCocker@users.noreply.github.com>
|
|
.IP \[bu] 2
|
|
Fran\[,c]ois Leurent <131.js@cloudyks.org>
|
|
.IP \[bu] 2
|
|
Arkadius Stefanski <arkste@gmail.com>
|
|
.IP \[bu] 2
|
|
Jay <dev@jaygoel.com>
|
|
.IP \[bu] 2
|
|
andrea rota <a@xelera.eu>
|
|
.IP \[bu] 2
|
|
nicolov <nicolov@users.noreply.github.com>
|
|
.IP \[bu] 2
|
|
Dario Guzik <dario@guzik.com.ar>
|
|
.IP \[bu] 2
|
|
qip <qip@users.noreply.github.com>
|
|
.IP \[bu] 2
|
|
yair\[at]unicorn <yair@unicorn>
|
|
.IP \[bu] 2
|
|
Matt Robinson <brimstone@the.narro.ws>
|
|
.IP \[bu] 2
|
|
kayrus <kay.diam@gmail.com>
|
|
.IP \[bu] 2
|
|
R\['e]my L\['e]one <remy.leone@gmail.com>
|
|
.IP \[bu] 2
|
|
Wojciech Smigielski <wojciech.hieronim.smigielski@gmail.com>
|
|
.IP \[bu] 2
|
|
weetmuts <oehrstroem@gmail.com>
|
|
.IP \[bu] 2
|
|
Jonathan <vanillajonathan@users.noreply.github.com>
|
|
.IP \[bu] 2
|
|
James Carpenter <orbsmiv@users.noreply.github.com>
|
|
.IP \[bu] 2
|
|
Vince <vince0villamora@gmail.com>
|
|
.IP \[bu] 2
|
|
Nestar47 <47841759+Nestar47@users.noreply.github.com>
|
|
.IP \[bu] 2
|
|
Six <brbsix@gmail.com>
|
|
.IP \[bu] 2
|
|
Alexandru Bumbacea <alexandru.bumbacea@booking.com>
|
|
.IP \[bu] 2
|
|
calisro <robert.calistri@gmail.com>
|
|
.IP \[bu] 2
|
|
Dr.Rx <david.rey@nventive.com>
|
|
.IP \[bu] 2
|
|
marcintustin <marcintustin@users.noreply.github.com>
|
|
.IP \[bu] 2
|
|
jaKa Mo\[u010D]nik <jaka@koofr.net>
|
|
.IP \[bu] 2
|
|
Fionera <fionera@fionera.de>
|
|
.IP \[bu] 2
|
|
Dan Walters <dan@walters.io>
|
|
.IP \[bu] 2
|
|
Danil Semelenov <sgtpep@users.noreply.github.com>
|
|
.IP \[bu] 2
|
|
xopez <28950736+xopez@users.noreply.github.com>
|
|
.IP \[bu] 2
|
|
Ben Boeckel <mathstuf@gmail.com>
|
|
.IP \[bu] 2
|
|
Manu <manu@snapdragon.cc>
|
|
.IP \[bu] 2
|
|
Kyle E.
|
|
Mitchell <kyle@kemitchell.com>
|
|
.IP \[bu] 2
|
|
Gary Kim <gary@garykim.dev>
|
|
.IP \[bu] 2
|
|
Jon <jonathn@github.com>
|
|
.IP \[bu] 2
|
|
Jeff Quinn <jeffrey.quinn@bluevoyant.com>
|
|
.IP \[bu] 2
|
|
Peter Berbec <peter@berbec.com>
|
|
.IP \[bu] 2
|
|
didil <1284255+didil@users.noreply.github.com>
|
|
.IP \[bu] 2
|
|
id01 <gaviniboom@gmail.com>
|
|
.IP \[bu] 2
|
|
Robert Marko <robimarko@gmail.com>
|
|
.IP \[bu] 2
|
|
Philip Harvey <32467456+pharveybattelle@users.noreply.github.com>
|
|
.IP \[bu] 2
|
|
JorisE <JorisE@users.noreply.github.com>
|
|
.IP \[bu] 2
|
|
garry415 <garry.415@gmail.com>
|
|
.IP \[bu] 2
|
|
forgems <forgems@gmail.com>
|
|
.IP \[bu] 2
|
|
Florian Apolloner <florian@apolloner.eu>
|
|
.IP \[bu] 2
|
|
Aleksandar Jankovi\['c] <office@ajankovic.com>
|
|
<ajankovic@users.noreply.github.com>
|
|
.IP \[bu] 2
|
|
Maran <maran@protonmail.com>
|
|
.IP \[bu] 2
|
|
nguyenhuuluan434 <nguyenhuuluan434@gmail.com>
|
|
.IP \[bu] 2
|
|
Laura Hausmann <zotan@zotan.pw> <laura@hausmann.dev>
|
|
.IP \[bu] 2
|
|
yparitcher <y@paritcher.com>
|
|
.IP \[bu] 2
|
|
AbelThar <abela.tharen@gmail.com>
|
|
.IP \[bu] 2
|
|
Matti Niemenmaa <matti.niemenmaa+git@iki.fi>
|
|
.IP \[bu] 2
|
|
Russell Davis <russelldavis@users.noreply.github.com>
|
|
.IP \[bu] 2
|
|
Yi FU <yi.fu@tink.se>
|
|
.IP \[bu] 2
|
|
Paul Millar <paul.millar@desy.de>
|
|
.IP \[bu] 2
|
|
justinalin <justinalin@qnap.com>
|
|
.IP \[bu] 2
|
|
EliEron <subanimehd@gmail.com>
|
|
.IP \[bu] 2
|
|
justina777 <chiahuei.lin@gmail.com>
|
|
.IP \[bu] 2
|
|
Chaitanya Bankanhal <bchaitanya15@gmail.com>
|
|
.IP \[bu] 2
|
|
Micha\[/l] Matczuk <michal@scylladb.com>
|
|
.IP \[bu] 2
|
|
Macavirus <macavirus@zoho.com>
|
|
.IP \[bu] 2
|
|
Abhinav Sharma <abhi18av@users.noreply.github.com>
|
|
.IP \[bu] 2
|
|
ginvine <34869051+ginvine@users.noreply.github.com>
|
|
.IP \[bu] 2
|
|
Patrick Wang <mail6543210@yahoo.com.tw>
|
|
.IP \[bu] 2
|
|
Cenk Alti <cenkalti@gmail.com>
|
|
.IP \[bu] 2
|
|
Andreas Chlupka <andy@chlupka.com>
|
|
.IP \[bu] 2
|
|
Alfonso Montero <amontero@tinet.org>
|
|
.IP \[bu] 2
|
|
Ivan Andreev <ivandeex@gmail.com>
|
|
.IP \[bu] 2
|
|
David Baumgold <david@davidbaumgold.com>
|
|
.IP \[bu] 2
|
|
Lars Lehtonen <lars.lehtonen@gmail.com>
|
|
.IP \[bu] 2
|
|
Matei David <matei.david@gmail.com>
|
|
.IP \[bu] 2
|
|
David <david.bramwell@endemolshine.com>
|
|
.IP \[bu] 2
|
|
Anthony Rusdi <33247310+antrusd@users.noreply.github.com>
|
|
.IP \[bu] 2
|
|
Richard Patel <me@terorie.dev>
|
|
.IP \[bu] 2
|
|
\[u5E84]\[u5929]\[u7FFC] <zty0826@gmail.com>
|
|
.IP \[bu] 2
|
|
SwitchJS <dev@switchjs.com>
|
|
.IP \[bu] 2
|
|
Raphael <PowershellNinja@users.noreply.github.com>
|
|
.IP \[bu] 2
|
|
Sezal Agrawal <sezalagrawal@gmail.com>
|
|
.IP \[bu] 2
|
|
Tyler <TylerNakamura@users.noreply.github.com>
|
|
.IP \[bu] 2
|
|
Brett Dutro <brett.dutro@gmail.com>
|
|
.IP \[bu] 2
|
|
Vighnesh SK <booterror99@gmail.com>
|
|
.IP \[bu] 2
|
|
Arijit Biswas <dibbyo456@gmail.com>
|
|
.IP \[bu] 2
|
|
Michele Caci <michele.caci@gmail.com>
|
|
.IP \[bu] 2
|
|
AlexandrBoltris <ua2fgb@gmail.com>
|
|
.IP \[bu] 2
|
|
Bryce Larson <blarson@saltstack.com>
|
|
.IP \[bu] 2
|
|
Carlos Ferreyra <crypticmind@gmail.com>
|
|
.IP \[bu] 2
|
|
Saksham Khanna <sakshamkhanna@outlook.com>
|
|
.IP \[bu] 2
|
|
dausruddin <5763466+dausruddin@users.noreply.github.com>
|
|
.IP \[bu] 2
|
|
zero\-24 <zero-24@users.noreply.github.com>
|
|
.IP \[bu] 2
|
|
Xiaoxing Ye <ye@xiaoxing.us>
|
|
.IP \[bu] 2
|
|
Barry Muldrey <barry@muldrey.net>
|
|
.IP \[bu] 2
|
|
Sebastian Brandt <sebastian.brandt@friday.de>
|
|
.IP \[bu] 2
|
|
Marco Molteni <marco.molteni@mailbox.org>
|
|
.IP \[bu] 2
|
|
Ankur Gupta <ankur0493@gmail.com>
|
|
<7876747+ankur0493@users.noreply.github.com>
|
|
.IP \[bu] 2
|
|
Maciej Zimnoch <maciej@scylladb.com>
|
|
.IP \[bu] 2
|
|
anuar45 <serdaliyev.anuar@gmail.com>
|
|
.IP \[bu] 2
|
|
Fernando <ferferga@users.noreply.github.com>
|
|
.IP \[bu] 2
|
|
David Cole <david.cole@sohonet.com>
|
|
.IP \[bu] 2
|
|
Wei He <git@weispot.com>
|
|
.IP \[bu] 2
|
|
Outvi V <19144373+outloudvi@users.noreply.github.com>
|
|
.IP \[bu] 2
|
|
Thomas Kriechbaumer <thomas@kriechbaumer.name>
|
|
.IP \[bu] 2
|
|
Tennix <tennix@users.noreply.github.com>
|
|
.IP \[bu] 2
|
|
Ole Sch\[:u]tt <ole@schuett.name>
|
|
.IP \[bu] 2
|
|
Kuang\-che Wu <kcwu@csie.org>
|
|
.IP \[bu] 2
|
|
Thomas Eales <wingsuit@users.noreply.github.com>
|
|
.IP \[bu] 2
|
|
Paul Tinsley <paul.tinsley@vitalsource.com>
|
|
.IP \[bu] 2
|
|
Felix Hungenberg <git@shiftgeist.com>
|
|
.IP \[bu] 2
|
|
Benjamin Richter <github@dev.telepath.de>
|
|
.IP \[bu] 2
|
|
landall <cst_zf@qq.com>
|
|
.IP \[bu] 2
|
|
thestigma <thestigma@gmail.com>
|
|
.IP \[bu] 2
|
|
jtagcat <38327267+jtagcat@users.noreply.github.com>
|
|
.IP \[bu] 2
|
|
Damon Permezel <permezel@me.com>
|
|
.IP \[bu] 2
|
|
boosh <boosh@users.noreply.github.com>
|
|
.IP \[bu] 2
|
|
unbelauscht <58393353+unbelauscht@users.noreply.github.com>
|
|
.IP \[bu] 2
|
|
Motonori IWAMURO <vmi@nifty.com>
|
|
.IP \[bu] 2
|
|
Benjapol Worakan <benwrk@live.com>
|
|
.IP \[bu] 2
|
|
Dave Koston <dave.koston@stackpath.com>
|
|
.IP \[bu] 2
|
|
Durval Menezes <DurvalMenezes@users.noreply.github.com>
|
|
.IP \[bu] 2
|
|
Tim Gallant <me@timgallant.us>
|
|
.IP \[bu] 2
|
|
Frederick Zhang <frederick888@tsundere.moe>
|
|
.IP \[bu] 2
|
|
valery1707 <valery1707@gmail.com>
|
|
.IP \[bu] 2
|
|
Yves G <theYinYeti@yalis.fr>
|
|
.IP \[bu] 2
|
|
Shing Kit Chan <chanshingkit@gmail.com>
|
|
.IP \[bu] 2
|
|
Franklyn Tackitt <franklyn@tackitt.net>
|
|
.IP \[bu] 2
|
|
Robert\-Andr\['e] Mauchin <zebob.m@gmail.com>
|
|
.IP \[bu] 2
|
|
evileye <48332831+ibiruai@users.noreply.github.com>
|
|
.IP \[bu] 2
|
|
Joachim Brandon LeBlanc <brandon@leblanc.codes>
|
|
.IP \[bu] 2
|
|
Patryk Jakuszew <patryk.jakuszew@gmail.com>
|
|
.IP \[bu] 2
|
|
fishbullet <shindu666@gmail.com>
|
|
.IP \[bu] 2
|
|
greatroar <\[at]>
|
|
.IP \[bu] 2
|
|
Bernd Schoolmann <mail@quexten.com>
|
|
.IP \[bu] 2
|
|
Elan Ruusam\[:a]e <glen@pld-linux.org>
|
|
.IP \[bu] 2
|
|
Max Sum <max@lolyculture.com>
|
|
.IP \[bu] 2
|
|
Mark Spieth <mspieth@users.noreply.github.com>
|
|
.IP \[bu] 2
|
|
harry <me@harry.plus>
|
|
.IP \[bu] 2
|
|
Samantha McVey <samantham@posteo.net>
|
|
.IP \[bu] 2
|
|
Jack Anderson <jack.anderson@metaswitch.com>
|
|
.IP \[bu] 2
|
|
Michael G <draget@speciesm.net>
|
|
.IP \[bu] 2
|
|
Brandon Philips <brandon@ifup.org>
|
|
.IP \[bu] 2
|
|
Daven <dooven@users.noreply.github.com>
|
|
.IP \[bu] 2
|
|
Martin Stone <martin@d7415.co.uk>
|
|
.IP \[bu] 2
|
|
David Bramwell <13053834+dbramwell@users.noreply.github.com>
|
|
.IP \[bu] 2
|
|
Sunil Patra <snl_su@live.com>
|
|
.IP \[bu] 2
|
|
Adam Stroud <adam.stroud@gmail.com>
|
|
.IP \[bu] 2
|
|
Kush <kushsharma@users.noreply.github.com>
|
|
.IP \[bu] 2
|
|
Matan Rosenberg <matan129@gmail.com>
|
|
.IP \[bu] 2
|
|
gitch1 <63495046+gitch1@users.noreply.github.com>
|
|
.IP \[bu] 2
|
|
ElonH <elonhhuang@gmail.com>
|
|
.IP \[bu] 2
|
|
Fred <fred@creativeprojects.tech>
|
|
.IP \[bu] 2
|
|
S\['e]bastien Gross <renard@users.noreply.github.com>
|
|
.IP \[bu] 2
|
|
Maxime Suret <11944422+msuret@users.noreply.github.com>
|
|
.IP \[bu] 2
|
|
Caleb Case <caleb@storj.io>
|
|
.IP \[bu] 2
|
|
Ben Zenker <imbenzenker@gmail.com>
|
|
.IP \[bu] 2
|
|
Martin Michlmayr <tbm@cyrius.com>
|
|
.IP \[bu] 2
|
|
Brandon McNama <bmcnama@pagerduty.com>
|
|
.IP \[bu] 2
|
|
Daniel Slyman <github@skylayer.eu>
|
|
.SH Contact the rclone project
|
|
.SS Forum
|
|
.PP
|
|
Forum for questions and general discussion:
|
|
.IP \[bu] 2
|
|
https://forum.rclone.org
|
|
.SS GitHub repository
|
|
.PP
|
|
The project\[cq]s repository is located at:
|
|
.IP \[bu] 2
|
|
https://github.com/rclone/rclone
|
|
.PP
|
|
There you can file bug reports or contribute with pull requests.
|
|
.SS Twitter
|
|
.PP
|
|
You can also follow me on twitter for rclone announcements:
|
|
.IP \[bu] 2
|
|
[\[at]njcw](https://twitter.com/njcw)
|
|
.SS Email
|
|
.PP
|
|
Or if all else fails or you want to ask something private or
|
|
confidential email Nick Craig\-Wood (mailto:nick@craig-wood.com).
|
|
Please don\[cq]t email me requests for help \- those are better directed
|
|
to the forum.
|
|
Thanks!
|
|
.SH AUTHORS
|
|
Nick Craig\-Wood.
|