mirror of
https://github.com/rclone/rclone
synced 2024-12-04 14:35:20 +01:00
13020 lines
347 KiB
Groff
13020 lines
347 KiB
Groff
.\"t
|
||
.\" Automatically generated by Pandoc 1.17.2
|
||
.\"
|
||
.TH "rclone" "1" "Dec 23, 2017" "User Manual" ""
|
||
.hy
|
||
.SH Rclone
|
||
.PP
|
||
[IMAGE: Logo (https://rclone.org/img/rclone-120x120.png)] (https://rclone.org/)
|
||
.PP
|
||
Rclone is a command line program to sync files and directories to and
|
||
from:
|
||
.IP \[bu] 2
|
||
Amazon Drive
|
||
.IP \[bu] 2
|
||
Amazon S3
|
||
.IP \[bu] 2
|
||
Backblaze B2
|
||
.IP \[bu] 2
|
||
Box
|
||
.IP \[bu] 2
|
||
Ceph
|
||
.IP \[bu] 2
|
||
DigitalOcean Spaces
|
||
.IP \[bu] 2
|
||
Dreamhost
|
||
.IP \[bu] 2
|
||
Dropbox
|
||
.IP \[bu] 2
|
||
FTP
|
||
.IP \[bu] 2
|
||
Google Cloud Storage
|
||
.IP \[bu] 2
|
||
Google Drive
|
||
.IP \[bu] 2
|
||
HTTP
|
||
.IP \[bu] 2
|
||
Hubic
|
||
.IP \[bu] 2
|
||
Memset Memstore
|
||
.IP \[bu] 2
|
||
Microsoft Azure Blob Storage
|
||
.IP \[bu] 2
|
||
Microsoft OneDrive
|
||
.IP \[bu] 2
|
||
Minio
|
||
.IP \[bu] 2
|
||
Nextloud
|
||
.IP \[bu] 2
|
||
OVH
|
||
.IP \[bu] 2
|
||
Openstack Swift
|
||
.IP \[bu] 2
|
||
Oracle Cloud Storage
|
||
.IP \[bu] 2
|
||
Ownloud
|
||
.IP \[bu] 2
|
||
pCloud
|
||
.IP \[bu] 2
|
||
put.io
|
||
.IP \[bu] 2
|
||
QingStor
|
||
.IP \[bu] 2
|
||
Rackspace Cloud Files
|
||
.IP \[bu] 2
|
||
SFTP
|
||
.IP \[bu] 2
|
||
Wasabi
|
||
.IP \[bu] 2
|
||
WebDAV
|
||
.IP \[bu] 2
|
||
Yandex Disk
|
||
.IP \[bu] 2
|
||
The local filesystem
|
||
.PP
|
||
Features
|
||
.IP \[bu] 2
|
||
MD5/SHA1 hashes checked at all times for file integrity
|
||
.IP \[bu] 2
|
||
Timestamps preserved on files
|
||
.IP \[bu] 2
|
||
Partial syncs supported on a whole file basis
|
||
.IP \[bu] 2
|
||
Copy (https://rclone.org/commands/rclone_copy/) mode to just copy
|
||
new/changed files
|
||
.IP \[bu] 2
|
||
Sync (https://rclone.org/commands/rclone_sync/) (one way) mode to make a
|
||
directory identical
|
||
.IP \[bu] 2
|
||
Check (https://rclone.org/commands/rclone_check/) mode to check for file
|
||
hash equality
|
||
.IP \[bu] 2
|
||
Can sync to and from network, eg two different cloud accounts
|
||
.IP \[bu] 2
|
||
Optional encryption (Crypt (https://rclone.org/crypt/))
|
||
.IP \[bu] 2
|
||
Optional cache (Cache (https://rclone.org/cache/))
|
||
.IP \[bu] 2
|
||
Optional FUSE mount (rclone
|
||
mount (https://rclone.org/commands/rclone_mount/))
|
||
.PP
|
||
Links
|
||
.IP \[bu] 2
|
||
Home page (https://rclone.org/)
|
||
.IP \[bu] 2
|
||
Github project page for source and bug
|
||
tracker (https://github.com/ncw/rclone)
|
||
.IP \[bu] 2
|
||
Rclone Forum (https://forum.rclone.org)
|
||
.IP \[bu] 2
|
||
Google+ page
|
||
.IP \[bu] 2
|
||
Downloads (https://rclone.org/downloads/)
|
||
.SH Install
|
||
.PP
|
||
Rclone is a Go program and comes as a single binary file.
|
||
.SS Quickstart
|
||
.IP \[bu] 2
|
||
Download (https://rclone.org/downloads/) the relevant binary.
|
||
.IP \[bu] 2
|
||
Unpack and the \f[C]rclone\f[] binary.
|
||
.IP \[bu] 2
|
||
Run \f[C]rclone\ config\f[] to setup.
|
||
See rclone config docs (https://rclone.org/docs/) for more details.
|
||
.PP
|
||
See below for some expanded Linux / macOS instructions.
|
||
.PP
|
||
See the Usage section (https://rclone.org/docs/) of the docs for how to
|
||
use rclone, or run \f[C]rclone\ \-h\f[].
|
||
.SS Script installation
|
||
.PP
|
||
To install rclone on Linux/MacOs/BSD systems, run:
|
||
.IP
|
||
.nf
|
||
\f[C]
|
||
curl\ https://rclone.org/install.sh\ |\ sudo\ bash
|
||
\f[]
|
||
.fi
|
||
.PP
|
||
For beta installation, run:
|
||
.IP
|
||
.nf
|
||
\f[C]
|
||
curl\ https://rclone.org/install.sh\ |\ sudo\ bash\ \-s\ beta
|
||
\f[]
|
||
.fi
|
||
.PP
|
||
Note that this script checks the version of rclone installed first and
|
||
won\[aq]t re\-download if not needed.
|
||
.SS Linux installation from precompiled binary
|
||
.PP
|
||
Fetch and unpack
|
||
.IP
|
||
.nf
|
||
\f[C]
|
||
curl\ \-O\ https://downloads.rclone.org/rclone\-current\-linux\-amd64.zip
|
||
unzip\ rclone\-current\-linux\-amd64.zip
|
||
cd\ rclone\-*\-linux\-amd64
|
||
\f[]
|
||
.fi
|
||
.PP
|
||
Copy binary file
|
||
.IP
|
||
.nf
|
||
\f[C]
|
||
sudo\ cp\ rclone\ /usr/bin/
|
||
sudo\ chown\ root:root\ /usr/bin/rclone
|
||
sudo\ chmod\ 755\ /usr/bin/rclone
|
||
\f[]
|
||
.fi
|
||
.PP
|
||
Install manpage
|
||
.IP
|
||
.nf
|
||
\f[C]
|
||
sudo\ mkdir\ \-p\ /usr/local/share/man/man1
|
||
sudo\ cp\ rclone.1\ /usr/local/share/man/man1/
|
||
sudo\ mandb\
|
||
\f[]
|
||
.fi
|
||
.PP
|
||
Run \f[C]rclone\ config\f[] to setup.
|
||
See rclone config docs (https://rclone.org/docs/) for more details.
|
||
.IP
|
||
.nf
|
||
\f[C]
|
||
rclone\ config
|
||
\f[]
|
||
.fi
|
||
.SS macOS installation from precompiled binary
|
||
.PP
|
||
Download the latest version of rclone.
|
||
.IP
|
||
.nf
|
||
\f[C]
|
||
cd\ &&\ curl\ \-O\ https://downloads.rclone.org/rclone\-current\-osx\-amd64.zip
|
||
\f[]
|
||
.fi
|
||
.PP
|
||
Unzip the download and cd to the extracted folder.
|
||
.IP
|
||
.nf
|
||
\f[C]
|
||
unzip\ \-a\ rclone\-current\-osx\-amd64.zip\ &&\ cd\ rclone\-*\-osx\-amd64
|
||
\f[]
|
||
.fi
|
||
.PP
|
||
Move rclone to your $PATH.
|
||
You will be prompted for your password.
|
||
.IP
|
||
.nf
|
||
\f[C]
|
||
sudo\ mkdir\ \-p\ /usr/local/bin
|
||
sudo\ mv\ rclone\ /usr/local/bin/
|
||
\f[]
|
||
.fi
|
||
.PP
|
||
(the \f[C]mkdir\f[] command is safe to run, even if the directory
|
||
already exists).
|
||
.PP
|
||
Remove the leftover files.
|
||
.IP
|
||
.nf
|
||
\f[C]
|
||
cd\ ..\ &&\ rm\ \-rf\ rclone\-*\-osx\-amd64\ rclone\-current\-osx\-amd64.zip
|
||
\f[]
|
||
.fi
|
||
.PP
|
||
Run \f[C]rclone\ config\f[] to setup.
|
||
See rclone config docs (https://rclone.org/docs/) for more details.
|
||
.IP
|
||
.nf
|
||
\f[C]
|
||
rclone\ config
|
||
\f[]
|
||
.fi
|
||
.SS Install from source
|
||
.PP
|
||
Make sure you have at least Go (https://golang.org/) 1.6 installed.
|
||
Make sure your \f[C]GOPATH\f[] is set, then:
|
||
.IP
|
||
.nf
|
||
\f[C]
|
||
go\ get\ \-u\ \-v\ github.com/ncw/rclone
|
||
\f[]
|
||
.fi
|
||
.PP
|
||
and this will build the binary in \f[C]$GOPATH/bin\f[].
|
||
If you have built rclone before then you will want to update its
|
||
dependencies first with this
|
||
.IP
|
||
.nf
|
||
\f[C]
|
||
go\ get\ \-u\ \-v\ github.com/ncw/rclone/...
|
||
\f[]
|
||
.fi
|
||
.SS Installation with Ansible
|
||
.PP
|
||
This can be done with Stefan Weichinger\[aq]s ansible
|
||
role (https://github.com/stefangweichinger/ansible-rclone).
|
||
.PP
|
||
Instructions
|
||
.IP "1." 3
|
||
\f[C]git\ clone\ https://github.com/stefangweichinger/ansible\-rclone.git\f[]
|
||
into your local roles\-directory
|
||
.IP "2." 3
|
||
add the role to the hosts you want rclone installed to:
|
||
.IP
|
||
.nf
|
||
\f[C]
|
||
\ \ \ \ \-\ hosts:\ rclone\-hosts
|
||
\ \ \ \ \ \ roles:
|
||
\ \ \ \ \ \ \ \ \ \ \-\ rclone
|
||
\f[]
|
||
.fi
|
||
.SS Configure
|
||
.PP
|
||
First, you\[aq]ll need to configure rclone.
|
||
As the object storage systems have quite complicated authentication
|
||
these are kept in a config file.
|
||
(See the \f[C]\-\-config\f[] entry for how to find the config file and
|
||
choose its location.)
|
||
.PP
|
||
The easiest way to make the config is to run rclone with the config
|
||
option:
|
||
.IP
|
||
.nf
|
||
\f[C]
|
||
rclone\ config
|
||
\f[]
|
||
.fi
|
||
.PP
|
||
See the following for detailed instructions for
|
||
.IP \[bu] 2
|
||
Amazon Drive (https://rclone.org/amazonclouddrive/)
|
||
.IP \[bu] 2
|
||
Amazon S3 (https://rclone.org/s3/)
|
||
.IP \[bu] 2
|
||
Backblaze B2 (https://rclone.org/b2/)
|
||
.IP \[bu] 2
|
||
Box (https://rclone.org/box/)
|
||
.IP \[bu] 2
|
||
Cache (https://rclone.org/cache/)
|
||
.IP \[bu] 2
|
||
Crypt (https://rclone.org/crypt/) \- to encrypt other remotes
|
||
.IP \[bu] 2
|
||
DigitalOcean Spaces (/s3/#digitalocean-spaces)
|
||
.IP \[bu] 2
|
||
Dropbox (https://rclone.org/dropbox/)
|
||
.IP \[bu] 2
|
||
FTP (https://rclone.org/ftp/)
|
||
.IP \[bu] 2
|
||
Google Cloud Storage (https://rclone.org/googlecloudstorage/)
|
||
.IP \[bu] 2
|
||
Google Drive (https://rclone.org/drive/)
|
||
.IP \[bu] 2
|
||
HTTP (https://rclone.org/http/)
|
||
.IP \[bu] 2
|
||
Hubic (https://rclone.org/hubic/)
|
||
.IP \[bu] 2
|
||
Microsoft Azure Blob Storage (https://rclone.org/azureblob/)
|
||
.IP \[bu] 2
|
||
Microsoft OneDrive (https://rclone.org/onedrive/)
|
||
.IP \[bu] 2
|
||
Openstack Swift / Rackspace Cloudfiles / Memset
|
||
Memstore (https://rclone.org/swift/)
|
||
.IP \[bu] 2
|
||
Pcloud (https://rclone.org/pcloud/)
|
||
.IP \[bu] 2
|
||
QingStor (https://rclone.org/qingstor/)
|
||
.IP \[bu] 2
|
||
SFTP (https://rclone.org/sftp/)
|
||
.IP \[bu] 2
|
||
WebDAV (https://rclone.org/webdav/)
|
||
.IP \[bu] 2
|
||
Yandex Disk (https://rclone.org/yandex/)
|
||
.IP \[bu] 2
|
||
The local filesystem (https://rclone.org/local/)
|
||
.SS Usage
|
||
.PP
|
||
Rclone syncs a directory tree from one storage system to another.
|
||
.PP
|
||
Its syntax is like this
|
||
.IP
|
||
.nf
|
||
\f[C]
|
||
Syntax:\ [options]\ subcommand\ <parameters>\ <parameters...>
|
||
\f[]
|
||
.fi
|
||
.PP
|
||
Source and destination paths are specified by the name you gave the
|
||
storage system in the config file then the sub path, eg "drive:myfolder"
|
||
to look at "myfolder" in Google drive.
|
||
.PP
|
||
You can define as many storage paths as you like in the config file.
|
||
.SS Subcommands
|
||
.PP
|
||
rclone uses a system of subcommands.
|
||
For example
|
||
.IP
|
||
.nf
|
||
\f[C]
|
||
rclone\ ls\ remote:path\ #\ lists\ a\ re
|
||
rclone\ copy\ /local/path\ remote:path\ #\ copies\ /local/path\ to\ the\ remote
|
||
rclone\ sync\ /local/path\ remote:path\ #\ syncs\ /local/path\ to\ the\ remote
|
||
\f[]
|
||
.fi
|
||
.SS rclone config
|
||
.PP
|
||
Enter an interactive configuration session.
|
||
.SS Synopsis
|
||
.PP
|
||
Enter an interactive configuration session where you can setup new
|
||
remotes and manage existing ones.
|
||
You may also set or remove a password to protect your configuration.
|
||
.IP
|
||
.nf
|
||
\f[C]
|
||
rclone\ config\ [flags]
|
||
\f[]
|
||
.fi
|
||
.SS Options
|
||
.IP
|
||
.nf
|
||
\f[C]
|
||
\ \ \-h,\ \-\-help\ \ \ help\ for\ config
|
||
\f[]
|
||
.fi
|
||
.SS rclone copy
|
||
.PP
|
||
Copy files from source to dest, skipping already copied
|
||
.SS Synopsis
|
||
.PP
|
||
Copy the source to the destination.
|
||
Doesn\[aq]t transfer unchanged files, testing by size and modification
|
||
time or MD5SUM.
|
||
Doesn\[aq]t delete files from the destination.
|
||
.PP
|
||
Note that it is always the contents of the directory that is synced, not
|
||
the directory so when source:path is a directory, it\[aq]s the contents
|
||
of source:path that are copied, not the directory name and contents.
|
||
.PP
|
||
If dest:path doesn\[aq]t exist, it is created and the source:path
|
||
contents go there.
|
||
.PP
|
||
For example
|
||
.IP
|
||
.nf
|
||
\f[C]
|
||
rclone\ copy\ source:sourcepath\ dest:destpath
|
||
\f[]
|
||
.fi
|
||
.PP
|
||
Let\[aq]s say there are two files in sourcepath
|
||
.IP
|
||
.nf
|
||
\f[C]
|
||
sourcepath/one.txt
|
||
sourcepath/two.txt
|
||
\f[]
|
||
.fi
|
||
.PP
|
||
This copies them to
|
||
.IP
|
||
.nf
|
||
\f[C]
|
||
destpath/one.txt
|
||
destpath/two.txt
|
||
\f[]
|
||
.fi
|
||
.PP
|
||
Not to
|
||
.IP
|
||
.nf
|
||
\f[C]
|
||
destpath/sourcepath/one.txt
|
||
destpath/sourcepath/two.txt
|
||
\f[]
|
||
.fi
|
||
.PP
|
||
If you are familiar with \f[C]rsync\f[], rclone always works as if you
|
||
had written a trailing / \- meaning "copy the contents of this
|
||
directory".
|
||
This applies to all commands and whether you are talking about the
|
||
source or destination.
|
||
.PP
|
||
See the \f[C]\-\-no\-traverse\f[] option for controlling whether rclone
|
||
lists the destination directory or not.
|
||
.IP
|
||
.nf
|
||
\f[C]
|
||
rclone\ copy\ source:path\ dest:path\ [flags]
|
||
\f[]
|
||
.fi
|
||
.SS Options
|
||
.IP
|
||
.nf
|
||
\f[C]
|
||
\ \ \-h,\ \-\-help\ \ \ help\ for\ copy
|
||
\f[]
|
||
.fi
|
||
.SS rclone sync
|
||
.PP
|
||
Make source and dest identical, modifying destination only.
|
||
.SS Synopsis
|
||
.PP
|
||
Sync the source to the destination, changing the destination only.
|
||
Doesn\[aq]t transfer unchanged files, testing by size and modification
|
||
time or MD5SUM.
|
||
Destination is updated to match source, including deleting files if
|
||
necessary.
|
||
.PP
|
||
\f[B]Important\f[]: Since this can cause data loss, test first with the
|
||
\f[C]\-\-dry\-run\f[] flag to see exactly what would be copied and
|
||
deleted.
|
||
.PP
|
||
Note that files in the destination won\[aq]t be deleted if there were
|
||
any errors at any point.
|
||
.PP
|
||
It is always the contents of the directory that is synced, not the
|
||
directory so when source:path is a directory, it\[aq]s the contents of
|
||
source:path that are copied, not the directory name and contents.
|
||
See extended explanation in the \f[C]copy\f[] command above if unsure.
|
||
.PP
|
||
If dest:path doesn\[aq]t exist, it is created and the source:path
|
||
contents go there.
|
||
.IP
|
||
.nf
|
||
\f[C]
|
||
rclone\ sync\ source:path\ dest:path\ [flags]
|
||
\f[]
|
||
.fi
|
||
.SS Options
|
||
.IP
|
||
.nf
|
||
\f[C]
|
||
\ \ \-h,\ \-\-help\ \ \ help\ for\ sync
|
||
\f[]
|
||
.fi
|
||
.SS rclone move
|
||
.PP
|
||
Move files from source to dest.
|
||
.SS Synopsis
|
||
.PP
|
||
Moves the contents of the source directory to the destination directory.
|
||
Rclone will error if the source and destination overlap and the remote
|
||
does not support a server side directory move operation.
|
||
.PP
|
||
If no filters are in use and if possible this will server side move
|
||
\f[C]source:path\f[] into \f[C]dest:path\f[].
|
||
After this \f[C]source:path\f[] will no longer longer exist.
|
||
.PP
|
||
Otherwise for each file in \f[C]source:path\f[] selected by the filters
|
||
(if any) this will move it into \f[C]dest:path\f[].
|
||
If possible a server side move will be used, otherwise it will copy it
|
||
(server side if possible) into \f[C]dest:path\f[] then delete the
|
||
original (if no errors on copy) in \f[C]source:path\f[].
|
||
.PP
|
||
If you want to delete empty source directories after move, use the
|
||
\-\-delete\-empty\-src\-dirs flag.
|
||
.PP
|
||
\f[B]Important\f[]: Since this can cause data loss, test first with the
|
||
\-\-dry\-run flag.
|
||
.IP
|
||
.nf
|
||
\f[C]
|
||
rclone\ move\ source:path\ dest:path\ [flags]
|
||
\f[]
|
||
.fi
|
||
.SS Options
|
||
.IP
|
||
.nf
|
||
\f[C]
|
||
\ \ \ \ \ \ \-\-delete\-empty\-src\-dirs\ \ \ Delete\ empty\ source\ dirs\ after\ move
|
||
\ \ \-h,\ \-\-help\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ help\ for\ move
|
||
\f[]
|
||
.fi
|
||
.SS rclone delete
|
||
.PP
|
||
Remove the contents of path.
|
||
.SS Synopsis
|
||
.PP
|
||
Remove the contents of path.
|
||
Unlike \f[C]purge\f[] it obeys include/exclude filters so can be used to
|
||
selectively delete files.
|
||
.PP
|
||
Eg delete all files bigger than 100MBytes
|
||
.PP
|
||
Check what would be deleted first (use either)
|
||
.IP
|
||
.nf
|
||
\f[C]
|
||
rclone\ \-\-min\-size\ 100M\ lsl\ remote:path
|
||
rclone\ \-\-dry\-run\ \-\-min\-size\ 100M\ delete\ remote:path
|
||
\f[]
|
||
.fi
|
||
.PP
|
||
Then delete
|
||
.IP
|
||
.nf
|
||
\f[C]
|
||
rclone\ \-\-min\-size\ 100M\ delete\ remote:path
|
||
\f[]
|
||
.fi
|
||
.PP
|
||
That reads "delete everything with a minimum size of 100 MB", hence
|
||
delete all files bigger than 100MBytes.
|
||
.IP
|
||
.nf
|
||
\f[C]
|
||
rclone\ delete\ remote:path\ [flags]
|
||
\f[]
|
||
.fi
|
||
.SS Options
|
||
.IP
|
||
.nf
|
||
\f[C]
|
||
\ \ \-h,\ \-\-help\ \ \ help\ for\ delete
|
||
\f[]
|
||
.fi
|
||
.SS rclone purge
|
||
.PP
|
||
Remove the path and all of its contents.
|
||
.SS Synopsis
|
||
.PP
|
||
Remove the path and all of its contents.
|
||
Note that this does not obey include/exclude filters \- everything will
|
||
be removed.
|
||
Use \f[C]delete\f[] if you want to selectively delete files.
|
||
.IP
|
||
.nf
|
||
\f[C]
|
||
rclone\ purge\ remote:path\ [flags]
|
||
\f[]
|
||
.fi
|
||
.SS Options
|
||
.IP
|
||
.nf
|
||
\f[C]
|
||
\ \ \-h,\ \-\-help\ \ \ help\ for\ purge
|
||
\f[]
|
||
.fi
|
||
.SS rclone mkdir
|
||
.PP
|
||
Make the path if it doesn\[aq]t already exist.
|
||
.SS Synopsis
|
||
.PP
|
||
Make the path if it doesn\[aq]t already exist.
|
||
.IP
|
||
.nf
|
||
\f[C]
|
||
rclone\ mkdir\ remote:path\ [flags]
|
||
\f[]
|
||
.fi
|
||
.SS Options
|
||
.IP
|
||
.nf
|
||
\f[C]
|
||
\ \ \-h,\ \-\-help\ \ \ help\ for\ mkdir
|
||
\f[]
|
||
.fi
|
||
.SS rclone rmdir
|
||
.PP
|
||
Remove the path if empty.
|
||
.SS Synopsis
|
||
.PP
|
||
Remove the path.
|
||
Note that you can\[aq]t remove a path with objects in it, use purge for
|
||
that.
|
||
.IP
|
||
.nf
|
||
\f[C]
|
||
rclone\ rmdir\ remote:path\ [flags]
|
||
\f[]
|
||
.fi
|
||
.SS Options
|
||
.IP
|
||
.nf
|
||
\f[C]
|
||
\ \ \-h,\ \-\-help\ \ \ help\ for\ rmdir
|
||
\f[]
|
||
.fi
|
||
.SS rclone check
|
||
.PP
|
||
Checks the files in the source and destination match.
|
||
.SS Synopsis
|
||
.PP
|
||
Checks the files in the source and destination match.
|
||
It compares sizes and hashes (MD5 or SHA1) and logs a report of files
|
||
which don\[aq]t match.
|
||
It doesn\[aq]t alter the source or destination.
|
||
.PP
|
||
If you supply the \-\-size\-only flag, it will only compare the sizes
|
||
not the hashes as well.
|
||
Use this for a quick check.
|
||
.PP
|
||
If you supply the \-\-download flag, it will download the data from both
|
||
remotes and check them against each other on the fly.
|
||
This can be useful for remotes that don\[aq]t support hashes or if you
|
||
really want to check all the data.
|
||
.IP
|
||
.nf
|
||
\f[C]
|
||
rclone\ check\ source:path\ dest:path\ [flags]
|
||
\f[]
|
||
.fi
|
||
.SS Options
|
||
.IP
|
||
.nf
|
||
\f[C]
|
||
\ \ \ \ \ \ \-\-download\ \ \ Check\ by\ downloading\ rather\ than\ with\ hash.
|
||
\ \ \-h,\ \-\-help\ \ \ \ \ \ \ help\ for\ check
|
||
\f[]
|
||
.fi
|
||
.SS rclone ls
|
||
.PP
|
||
List all the objects in the path with size and path.
|
||
.SS Synopsis
|
||
.PP
|
||
List all the objects in the path with size and path.
|
||
.IP
|
||
.nf
|
||
\f[C]
|
||
rclone\ ls\ remote:path\ [flags]
|
||
\f[]
|
||
.fi
|
||
.SS Options
|
||
.IP
|
||
.nf
|
||
\f[C]
|
||
\ \ \-h,\ \-\-help\ \ \ help\ for\ ls
|
||
\f[]
|
||
.fi
|
||
.SS rclone lsd
|
||
.PP
|
||
List all directories/containers/buckets in the path.
|
||
.SS Synopsis
|
||
.PP
|
||
List all directories/containers/buckets in the path.
|
||
.IP
|
||
.nf
|
||
\f[C]
|
||
rclone\ lsd\ remote:path\ [flags]
|
||
\f[]
|
||
.fi
|
||
.SS Options
|
||
.IP
|
||
.nf
|
||
\f[C]
|
||
\ \ \-h,\ \-\-help\ \ \ help\ for\ lsd
|
||
\f[]
|
||
.fi
|
||
.SS rclone lsl
|
||
.PP
|
||
List all the objects path with modification time, size and path.
|
||
.SS Synopsis
|
||
.PP
|
||
List all the objects path with modification time, size and path.
|
||
.IP
|
||
.nf
|
||
\f[C]
|
||
rclone\ lsl\ remote:path\ [flags]
|
||
\f[]
|
||
.fi
|
||
.SS Options
|
||
.IP
|
||
.nf
|
||
\f[C]
|
||
\ \ \-h,\ \-\-help\ \ \ help\ for\ lsl
|
||
\f[]
|
||
.fi
|
||
.SS rclone md5sum
|
||
.PP
|
||
Produces an md5sum file for all the objects in the path.
|
||
.SS Synopsis
|
||
.PP
|
||
Produces an md5sum file for all the objects in the path.
|
||
This is in the same format as the standard md5sum tool produces.
|
||
.IP
|
||
.nf
|
||
\f[C]
|
||
rclone\ md5sum\ remote:path\ [flags]
|
||
\f[]
|
||
.fi
|
||
.SS Options
|
||
.IP
|
||
.nf
|
||
\f[C]
|
||
\ \ \-h,\ \-\-help\ \ \ help\ for\ md5sum
|
||
\f[]
|
||
.fi
|
||
.SS rclone sha1sum
|
||
.PP
|
||
Produces an sha1sum file for all the objects in the path.
|
||
.SS Synopsis
|
||
.PP
|
||
Produces an sha1sum file for all the objects in the path.
|
||
This is in the same format as the standard sha1sum tool produces.
|
||
.IP
|
||
.nf
|
||
\f[C]
|
||
rclone\ sha1sum\ remote:path\ [flags]
|
||
\f[]
|
||
.fi
|
||
.SS Options
|
||
.IP
|
||
.nf
|
||
\f[C]
|
||
\ \ \-h,\ \-\-help\ \ \ help\ for\ sha1sum
|
||
\f[]
|
||
.fi
|
||
.SS rclone size
|
||
.PP
|
||
Prints the total size and number of objects in remote:path.
|
||
.SS Synopsis
|
||
.PP
|
||
Prints the total size and number of objects in remote:path.
|
||
.IP
|
||
.nf
|
||
\f[C]
|
||
rclone\ size\ remote:path\ [flags]
|
||
\f[]
|
||
.fi
|
||
.SS Options
|
||
.IP
|
||
.nf
|
||
\f[C]
|
||
\ \ \-h,\ \-\-help\ \ \ help\ for\ size
|
||
\f[]
|
||
.fi
|
||
.SS rclone version
|
||
.PP
|
||
Show the version number.
|
||
.SS Synopsis
|
||
.PP
|
||
Show the version number.
|
||
.IP
|
||
.nf
|
||
\f[C]
|
||
rclone\ version\ [flags]
|
||
\f[]
|
||
.fi
|
||
.SS Options
|
||
.IP
|
||
.nf
|
||
\f[C]
|
||
\ \ \-h,\ \-\-help\ \ \ help\ for\ version
|
||
\f[]
|
||
.fi
|
||
.SS rclone cleanup
|
||
.PP
|
||
Clean up the remote if possible
|
||
.SS Synopsis
|
||
.PP
|
||
Clean up the remote if possible.
|
||
Empty the trash or delete old file versions.
|
||
Not supported by all remotes.
|
||
.IP
|
||
.nf
|
||
\f[C]
|
||
rclone\ cleanup\ remote:path\ [flags]
|
||
\f[]
|
||
.fi
|
||
.SS Options
|
||
.IP
|
||
.nf
|
||
\f[C]
|
||
\ \ \-h,\ \-\-help\ \ \ help\ for\ cleanup
|
||
\f[]
|
||
.fi
|
||
.SS rclone dedupe
|
||
.PP
|
||
Interactively find duplicate files and delete/rename them.
|
||
.SS Synopsis
|
||
.PP
|
||
By default \f[C]dedupe\f[] interactively finds duplicate files and
|
||
offers to delete all but one or rename them to be different.
|
||
Only useful with Google Drive which can have duplicate file names.
|
||
.PP
|
||
In the first pass it will merge directories with the same name.
|
||
It will do this iteratively until all the identical directories have
|
||
been merged.
|
||
.PP
|
||
The \f[C]dedupe\f[] command will delete all but one of any identical
|
||
(same md5sum) files it finds without confirmation.
|
||
This means that for most duplicated files the \f[C]dedupe\f[] command
|
||
will not be interactive.
|
||
You can use \f[C]\-\-dry\-run\f[] to see what would happen without doing
|
||
anything.
|
||
.PP
|
||
Here is an example run.
|
||
.PP
|
||
Before \- with duplicates
|
||
.IP
|
||
.nf
|
||
\f[C]
|
||
$\ rclone\ lsl\ drive:dupes
|
||
\ \ 6048320\ 2016\-03\-05\ 16:23:16.798000000\ one.txt
|
||
\ \ 6048320\ 2016\-03\-05\ 16:23:11.775000000\ one.txt
|
||
\ \ \ 564374\ 2016\-03\-05\ 16:23:06.731000000\ one.txt
|
||
\ \ 6048320\ 2016\-03\-05\ 16:18:26.092000000\ one.txt
|
||
\ \ 6048320\ 2016\-03\-05\ 16:22:46.185000000\ two.txt
|
||
\ \ 1744073\ 2016\-03\-05\ 16:22:38.104000000\ two.txt
|
||
\ \ \ 564374\ 2016\-03\-05\ 16:22:52.118000000\ two.txt
|
||
\f[]
|
||
.fi
|
||
.PP
|
||
Now the \f[C]dedupe\f[] session
|
||
.IP
|
||
.nf
|
||
\f[C]
|
||
$\ rclone\ dedupe\ drive:dupes
|
||
2016/03/05\ 16:24:37\ Google\ drive\ root\ \[aq]dupes\[aq]:\ Looking\ for\ duplicates\ using\ interactive\ mode.
|
||
one.txt:\ Found\ 4\ duplicates\ \-\ deleting\ identical\ copies
|
||
one.txt:\ Deleting\ 2/3\ identical\ duplicates\ (md5sum\ "1eedaa9fe86fd4b8632e2ac549403b36")
|
||
one.txt:\ 2\ duplicates\ remain
|
||
\ \ 1:\ \ \ \ \ \ 6048320\ bytes,\ 2016\-03\-05\ 16:23:16.798000000,\ md5sum\ 1eedaa9fe86fd4b8632e2ac549403b36
|
||
\ \ 2:\ \ \ \ \ \ \ 564374\ bytes,\ 2016\-03\-05\ 16:23:06.731000000,\ md5sum\ 7594e7dc9fc28f727c42ee3e0749de81
|
||
s)\ Skip\ and\ do\ nothing
|
||
k)\ Keep\ just\ one\ (choose\ which\ in\ next\ step)
|
||
r)\ Rename\ all\ to\ be\ different\ (by\ changing\ file.jpg\ to\ file\-1.jpg)
|
||
s/k/r>\ k
|
||
Enter\ the\ number\ of\ the\ file\ to\ keep>\ 1
|
||
one.txt:\ Deleted\ 1\ extra\ copies
|
||
two.txt:\ Found\ 3\ duplicates\ \-\ deleting\ identical\ copies
|
||
two.txt:\ 3\ duplicates\ remain
|
||
\ \ 1:\ \ \ \ \ \ \ 564374\ bytes,\ 2016\-03\-05\ 16:22:52.118000000,\ md5sum\ 7594e7dc9fc28f727c42ee3e0749de81
|
||
\ \ 2:\ \ \ \ \ \ 6048320\ bytes,\ 2016\-03\-05\ 16:22:46.185000000,\ md5sum\ 1eedaa9fe86fd4b8632e2ac549403b36
|
||
\ \ 3:\ \ \ \ \ \ 1744073\ bytes,\ 2016\-03\-05\ 16:22:38.104000000,\ md5sum\ 851957f7fb6f0bc4ce76be966d336802
|
||
s)\ Skip\ and\ do\ nothing
|
||
k)\ Keep\ just\ one\ (choose\ which\ in\ next\ step)
|
||
r)\ Rename\ all\ to\ be\ different\ (by\ changing\ file.jpg\ to\ file\-1.jpg)
|
||
s/k/r>\ r
|
||
two\-1.txt:\ renamed\ from:\ two.txt
|
||
two\-2.txt:\ renamed\ from:\ two.txt
|
||
two\-3.txt:\ renamed\ from:\ two.txt
|
||
\f[]
|
||
.fi
|
||
.PP
|
||
The result being
|
||
.IP
|
||
.nf
|
||
\f[C]
|
||
$\ rclone\ lsl\ drive:dupes
|
||
\ \ 6048320\ 2016\-03\-05\ 16:23:16.798000000\ one.txt
|
||
\ \ \ 564374\ 2016\-03\-05\ 16:22:52.118000000\ two\-1.txt
|
||
\ \ 6048320\ 2016\-03\-05\ 16:22:46.185000000\ two\-2.txt
|
||
\ \ 1744073\ 2016\-03\-05\ 16:22:38.104000000\ two\-3.txt
|
||
\f[]
|
||
.fi
|
||
.PP
|
||
Dedupe can be run non interactively using the \f[C]\-\-dedupe\-mode\f[]
|
||
flag or by using an extra parameter with the same value
|
||
.IP \[bu] 2
|
||
\f[C]\-\-dedupe\-mode\ interactive\f[] \- interactive as above.
|
||
.IP \[bu] 2
|
||
\f[C]\-\-dedupe\-mode\ skip\f[] \- removes identical files then skips
|
||
anything left.
|
||
.IP \[bu] 2
|
||
\f[C]\-\-dedupe\-mode\ first\f[] \- removes identical files then keeps
|
||
the first one.
|
||
.IP \[bu] 2
|
||
\f[C]\-\-dedupe\-mode\ newest\f[] \- removes identical files then keeps
|
||
the newest one.
|
||
.IP \[bu] 2
|
||
\f[C]\-\-dedupe\-mode\ oldest\f[] \- removes identical files then keeps
|
||
the oldest one.
|
||
.IP \[bu] 2
|
||
\f[C]\-\-dedupe\-mode\ rename\f[] \- removes identical files then
|
||
renames the rest to be different.
|
||
.PP
|
||
For example to rename all the identically named photos in your Google
|
||
Photos directory, do
|
||
.IP
|
||
.nf
|
||
\f[C]
|
||
rclone\ dedupe\ \-\-dedupe\-mode\ rename\ "drive:Google\ Photos"
|
||
\f[]
|
||
.fi
|
||
.PP
|
||
Or
|
||
.IP
|
||
.nf
|
||
\f[C]
|
||
rclone\ dedupe\ rename\ "drive:Google\ Photos"
|
||
\f[]
|
||
.fi
|
||
.IP
|
||
.nf
|
||
\f[C]
|
||
rclone\ dedupe\ [mode]\ remote:path\ [flags]
|
||
\f[]
|
||
.fi
|
||
.SS Options
|
||
.IP
|
||
.nf
|
||
\f[C]
|
||
\ \ \ \ \ \ \-\-dedupe\-mode\ string\ \ \ Dedupe\ mode\ interactive|skip|first|newest|oldest|rename.\ (default\ "interactive")
|
||
\ \ \-h,\ \-\-help\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ help\ for\ dedupe
|
||
\f[]
|
||
.fi
|
||
.SS rclone authorize
|
||
.PP
|
||
Remote authorization.
|
||
.SS Synopsis
|
||
.PP
|
||
Remote authorization.
|
||
Used to authorize a remote or headless rclone from a machine with a
|
||
browser \- use as instructed by rclone config.
|
||
.IP
|
||
.nf
|
||
\f[C]
|
||
rclone\ authorize\ [flags]
|
||
\f[]
|
||
.fi
|
||
.SS Options
|
||
.IP
|
||
.nf
|
||
\f[C]
|
||
\ \ \-h,\ \-\-help\ \ \ help\ for\ authorize
|
||
\f[]
|
||
.fi
|
||
.SS rclone cachestats
|
||
.PP
|
||
Print cache stats for a remote
|
||
.SS Synopsis
|
||
.PP
|
||
Print cache stats for a remote in JSON format
|
||
.IP
|
||
.nf
|
||
\f[C]
|
||
rclone\ cachestats\ source:\ [flags]
|
||
\f[]
|
||
.fi
|
||
.SS Options
|
||
.IP
|
||
.nf
|
||
\f[C]
|
||
\ \ \-h,\ \-\-help\ \ \ help\ for\ cachestats
|
||
\f[]
|
||
.fi
|
||
.SS rclone cat
|
||
.PP
|
||
Concatenates any files and sends them to stdout.
|
||
.SS Synopsis
|
||
.PP
|
||
rclone cat sends any files to standard output.
|
||
.PP
|
||
You can use it like this to output a single file
|
||
.IP
|
||
.nf
|
||
\f[C]
|
||
rclone\ cat\ remote:path/to/file
|
||
\f[]
|
||
.fi
|
||
.PP
|
||
Or like this to output any file in dir or subdirectories.
|
||
.IP
|
||
.nf
|
||
\f[C]
|
||
rclone\ cat\ remote:path/to/dir
|
||
\f[]
|
||
.fi
|
||
.PP
|
||
Or like this to output any .txt files in dir or subdirectories.
|
||
.IP
|
||
.nf
|
||
\f[C]
|
||
rclone\ \-\-include\ "*.txt"\ cat\ remote:path/to/dir
|
||
\f[]
|
||
.fi
|
||
.PP
|
||
Use the \-\-head flag to print characters only at the start, \-\-tail
|
||
for the end and \-\-offset and \-\-count to print a section in the
|
||
middle.
|
||
Note that if offset is negative it will count from the end, so
|
||
\-\-offset \-1 \-\-count 1 is equivalent to \-\-tail 1.
|
||
.IP
|
||
.nf
|
||
\f[C]
|
||
rclone\ cat\ remote:path\ [flags]
|
||
\f[]
|
||
.fi
|
||
.SS Options
|
||
.IP
|
||
.nf
|
||
\f[C]
|
||
\ \ \ \ \ \ \-\-count\ int\ \ \ \ Only\ print\ N\ characters.\ (default\ \-1)
|
||
\ \ \ \ \ \ \-\-discard\ \ \ \ \ \ Discard\ the\ output\ instead\ of\ printing.
|
||
\ \ \ \ \ \ \-\-head\ int\ \ \ \ \ Only\ print\ the\ first\ N\ characters.
|
||
\ \ \-h,\ \-\-help\ \ \ \ \ \ \ \ \ help\ for\ cat
|
||
\ \ \ \ \ \ \-\-offset\ int\ \ \ Start\ printing\ at\ offset\ N\ (or\ from\ end\ if\ \-ve).
|
||
\ \ \ \ \ \ \-\-tail\ int\ \ \ \ \ Only\ print\ the\ last\ N\ characters.
|
||
\f[]
|
||
.fi
|
||
.SS rclone config create
|
||
.PP
|
||
Create a new remote with name, type and options.
|
||
.SS Synopsis
|
||
.PP
|
||
Create a new remote of with and options.
|
||
The options should be passed in in pairs of .
|
||
.PP
|
||
For example to make a swift remote of name myremote using auto config
|
||
you would do:
|
||
.IP
|
||
.nf
|
||
\f[C]
|
||
rclone\ config\ create\ myremote\ swift\ env_auth\ true
|
||
\f[]
|
||
.fi
|
||
.IP
|
||
.nf
|
||
\f[C]
|
||
rclone\ config\ create\ <name>\ <type>\ [<key>\ <value>]*\ [flags]
|
||
\f[]
|
||
.fi
|
||
.SS Options
|
||
.IP
|
||
.nf
|
||
\f[C]
|
||
\ \ \-h,\ \-\-help\ \ \ help\ for\ create
|
||
\f[]
|
||
.fi
|
||
.SS rclone config delete
|
||
.PP
|
||
Delete an existing remote .
|
||
.SS Synopsis
|
||
.PP
|
||
Delete an existing remote .
|
||
.IP
|
||
.nf
|
||
\f[C]
|
||
rclone\ config\ delete\ <name>\ [flags]
|
||
\f[]
|
||
.fi
|
||
.SS Options
|
||
.IP
|
||
.nf
|
||
\f[C]
|
||
\ \ \-h,\ \-\-help\ \ \ help\ for\ delete
|
||
\f[]
|
||
.fi
|
||
.SS rclone config dump
|
||
.PP
|
||
Dump the config file as JSON.
|
||
.SS Synopsis
|
||
.PP
|
||
Dump the config file as JSON.
|
||
.IP
|
||
.nf
|
||
\f[C]
|
||
rclone\ config\ dump\ [flags]
|
||
\f[]
|
||
.fi
|
||
.SS Options
|
||
.IP
|
||
.nf
|
||
\f[C]
|
||
\ \ \-h,\ \-\-help\ \ \ help\ for\ dump
|
||
\f[]
|
||
.fi
|
||
.SS rclone config edit
|
||
.PP
|
||
Enter an interactive configuration session.
|
||
.SS Synopsis
|
||
.PP
|
||
Enter an interactive configuration session where you can setup new
|
||
remotes and manage existing ones.
|
||
You may also set or remove a password to protect your configuration.
|
||
.IP
|
||
.nf
|
||
\f[C]
|
||
rclone\ config\ edit\ [flags]
|
||
\f[]
|
||
.fi
|
||
.SS Options
|
||
.IP
|
||
.nf
|
||
\f[C]
|
||
\ \ \-h,\ \-\-help\ \ \ help\ for\ edit
|
||
\f[]
|
||
.fi
|
||
.SS rclone config file
|
||
.PP
|
||
Show path of configuration file in use.
|
||
.SS Synopsis
|
||
.PP
|
||
Show path of configuration file in use.
|
||
.IP
|
||
.nf
|
||
\f[C]
|
||
rclone\ config\ file\ [flags]
|
||
\f[]
|
||
.fi
|
||
.SS Options
|
||
.IP
|
||
.nf
|
||
\f[C]
|
||
\ \ \-h,\ \-\-help\ \ \ help\ for\ file
|
||
\f[]
|
||
.fi
|
||
.SS rclone config password
|
||
.PP
|
||
Update password in an existing remote.
|
||
.SS Synopsis
|
||
.PP
|
||
Update an existing remote\[aq]s password.
|
||
The password should be passed in in pairs of .
|
||
.PP
|
||
For example to set password of a remote of name myremote you would do:
|
||
.IP
|
||
.nf
|
||
\f[C]
|
||
rclone\ config\ password\ myremote\ fieldname\ mypassword
|
||
\f[]
|
||
.fi
|
||
.IP
|
||
.nf
|
||
\f[C]
|
||
rclone\ config\ password\ <name>\ [<key>\ <value>]+\ [flags]
|
||
\f[]
|
||
.fi
|
||
.SS Options
|
||
.IP
|
||
.nf
|
||
\f[C]
|
||
\ \ \-h,\ \-\-help\ \ \ help\ for\ password
|
||
\f[]
|
||
.fi
|
||
.SS rclone config providers
|
||
.PP
|
||
List in JSON format all the providers and options.
|
||
.SS Synopsis
|
||
.PP
|
||
List in JSON format all the providers and options.
|
||
.IP
|
||
.nf
|
||
\f[C]
|
||
rclone\ config\ providers\ [flags]
|
||
\f[]
|
||
.fi
|
||
.SS Options
|
||
.IP
|
||
.nf
|
||
\f[C]
|
||
\ \ \-h,\ \-\-help\ \ \ help\ for\ providers
|
||
\f[]
|
||
.fi
|
||
.SS rclone config show
|
||
.PP
|
||
Print (decrypted) config file, or the config for a single remote.
|
||
.SS Synopsis
|
||
.PP
|
||
Print (decrypted) config file, or the config for a single remote.
|
||
.IP
|
||
.nf
|
||
\f[C]
|
||
rclone\ config\ show\ [<remote>]\ [flags]
|
||
\f[]
|
||
.fi
|
||
.SS Options
|
||
.IP
|
||
.nf
|
||
\f[C]
|
||
\ \ \-h,\ \-\-help\ \ \ help\ for\ show
|
||
\f[]
|
||
.fi
|
||
.SS rclone config update
|
||
.PP
|
||
Update options in an existing remote.
|
||
.SS Synopsis
|
||
.PP
|
||
Update an existing remote\[aq]s options.
|
||
The options should be passed in in pairs of .
|
||
.PP
|
||
For example to update the env_auth field of a remote of name myremote
|
||
you would do:
|
||
.IP
|
||
.nf
|
||
\f[C]
|
||
rclone\ config\ update\ myremote\ swift\ env_auth\ true
|
||
\f[]
|
||
.fi
|
||
.IP
|
||
.nf
|
||
\f[C]
|
||
rclone\ config\ update\ <name>\ [<key>\ <value>]+\ [flags]
|
||
\f[]
|
||
.fi
|
||
.SS Options
|
||
.IP
|
||
.nf
|
||
\f[C]
|
||
\ \ \-h,\ \-\-help\ \ \ help\ for\ update
|
||
\f[]
|
||
.fi
|
||
.SS rclone copyto
|
||
.PP
|
||
Copy files from source to dest, skipping already copied
|
||
.SS Synopsis
|
||
.PP
|
||
If source:path is a file or directory then it copies it to a file or
|
||
directory named dest:path.
|
||
.PP
|
||
This can be used to upload single files to other than their current
|
||
name.
|
||
If the source is a directory then it acts exactly like the copy command.
|
||
.PP
|
||
So
|
||
.IP
|
||
.nf
|
||
\f[C]
|
||
rclone\ copyto\ src\ dst
|
||
\f[]
|
||
.fi
|
||
.PP
|
||
where src and dst are rclone paths, either remote:path or /path/to/local
|
||
or C:.
|
||
.PP
|
||
This will:
|
||
.IP
|
||
.nf
|
||
\f[C]
|
||
if\ src\ is\ file
|
||
\ \ \ \ copy\ it\ to\ dst,\ overwriting\ an\ existing\ file\ if\ it\ exists
|
||
if\ src\ is\ directory
|
||
\ \ \ \ copy\ it\ to\ dst,\ overwriting\ existing\ files\ if\ they\ exist
|
||
\ \ \ \ see\ copy\ command\ for\ full\ details
|
||
\f[]
|
||
.fi
|
||
.PP
|
||
This doesn\[aq]t transfer unchanged files, testing by size and
|
||
modification time or MD5SUM.
|
||
It doesn\[aq]t delete files from the destination.
|
||
.IP
|
||
.nf
|
||
\f[C]
|
||
rclone\ copyto\ source:path\ dest:path\ [flags]
|
||
\f[]
|
||
.fi
|
||
.SS Options
|
||
.IP
|
||
.nf
|
||
\f[C]
|
||
\ \ \-h,\ \-\-help\ \ \ help\ for\ copyto
|
||
\f[]
|
||
.fi
|
||
.SS rclone cryptcheck
|
||
.PP
|
||
Cryptcheck checks the integrity of a crypted remote.
|
||
.SS Synopsis
|
||
.PP
|
||
rclone cryptcheck checks a remote against a crypted remote.
|
||
This is the equivalent of running rclone check, but able to check the
|
||
checksums of the crypted remote.
|
||
.PP
|
||
For it to work the underlying remote of the cryptedremote must support
|
||
some kind of checksum.
|
||
.PP
|
||
It works by reading the nonce from each file on the cryptedremote: and
|
||
using that to encrypt each file on the remote:.
|
||
It then checks the checksum of the underlying file on the cryptedremote:
|
||
against the checksum of the file it has just encrypted.
|
||
.PP
|
||
Use it like this
|
||
.IP
|
||
.nf
|
||
\f[C]
|
||
rclone\ cryptcheck\ /path/to/files\ encryptedremote:path
|
||
\f[]
|
||
.fi
|
||
.PP
|
||
You can use it like this also, but that will involve downloading all the
|
||
files in remote:path.
|
||
.IP
|
||
.nf
|
||
\f[C]
|
||
rclone\ cryptcheck\ remote:path\ encryptedremote:path
|
||
\f[]
|
||
.fi
|
||
.PP
|
||
After it has run it will log the status of the encryptedremote:.
|
||
.IP
|
||
.nf
|
||
\f[C]
|
||
rclone\ cryptcheck\ remote:path\ cryptedremote:path\ [flags]
|
||
\f[]
|
||
.fi
|
||
.SS Options
|
||
.IP
|
||
.nf
|
||
\f[C]
|
||
\ \ \-h,\ \-\-help\ \ \ help\ for\ cryptcheck
|
||
\f[]
|
||
.fi
|
||
.SS rclone cryptdecode
|
||
.PP
|
||
Cryptdecode returns unencrypted file names.
|
||
.SS Synopsis
|
||
.PP
|
||
rclone cryptdecode returns unencrypted file names when provided with a
|
||
list of encrypted file names.
|
||
List limit is 10 items.
|
||
.PP
|
||
use it like this
|
||
.IP
|
||
.nf
|
||
\f[C]
|
||
rclone\ cryptdecode\ encryptedremote:\ encryptedfilename1\ encryptedfilename2
|
||
\f[]
|
||
.fi
|
||
.IP
|
||
.nf
|
||
\f[C]
|
||
rclone\ cryptdecode\ encryptedremote:\ encryptedfilename\ [flags]
|
||
\f[]
|
||
.fi
|
||
.SS Options
|
||
.IP
|
||
.nf
|
||
\f[C]
|
||
\ \ \-h,\ \-\-help\ \ \ help\ for\ cryptdecode
|
||
\f[]
|
||
.fi
|
||
.SS rclone dbhashsum
|
||
.PP
|
||
Produces a Dropbox hash file for all the objects in the path.
|
||
.SS Synopsis
|
||
.PP
|
||
Produces a Dropbox hash file for all the objects in the path.
|
||
The hashes are calculated according to Dropbox content hash
|
||
rules (https://www.dropbox.com/developers/reference/content-hash).
|
||
The output is in the same format as md5sum and sha1sum.
|
||
.IP
|
||
.nf
|
||
\f[C]
|
||
rclone\ dbhashsum\ remote:path\ [flags]
|
||
\f[]
|
||
.fi
|
||
.SS Options
|
||
.IP
|
||
.nf
|
||
\f[C]
|
||
\ \ \-h,\ \-\-help\ \ \ help\ for\ dbhashsum
|
||
\f[]
|
||
.fi
|
||
.SS rclone genautocomplete
|
||
.PP
|
||
Output completion script for a given shell.
|
||
.SS Synopsis
|
||
.PP
|
||
Generates a shell completion script for rclone.
|
||
Run with \-\-help to list the supported shells.
|
||
.SS Options
|
||
.IP
|
||
.nf
|
||
\f[C]
|
||
\ \ \-h,\ \-\-help\ \ \ help\ for\ genautocomplete
|
||
\f[]
|
||
.fi
|
||
.SS rclone genautocomplete bash
|
||
.PP
|
||
Output bash completion script for rclone.
|
||
.SS Synopsis
|
||
.PP
|
||
Generates a bash shell autocompletion script for rclone.
|
||
.PP
|
||
This writes to /etc/bash_completion.d/rclone by default so will probably
|
||
need to be run with sudo or as root, eg
|
||
.IP
|
||
.nf
|
||
\f[C]
|
||
sudo\ rclone\ genautocomplete\ bash
|
||
\f[]
|
||
.fi
|
||
.PP
|
||
Logout and login again to use the autocompletion scripts, or source them
|
||
directly
|
||
.IP
|
||
.nf
|
||
\f[C]
|
||
\&.\ /etc/bash_completion
|
||
\f[]
|
||
.fi
|
||
.PP
|
||
If you supply a command line argument the script will be written there.
|
||
.IP
|
||
.nf
|
||
\f[C]
|
||
rclone\ genautocomplete\ bash\ [output_file]\ [flags]
|
||
\f[]
|
||
.fi
|
||
.SS Options
|
||
.IP
|
||
.nf
|
||
\f[C]
|
||
\ \ \-h,\ \-\-help\ \ \ help\ for\ bash
|
||
\f[]
|
||
.fi
|
||
.SS rclone genautocomplete zsh
|
||
.PP
|
||
Output zsh completion script for rclone.
|
||
.SS Synopsis
|
||
.PP
|
||
Generates a zsh autocompletion script for rclone.
|
||
.PP
|
||
This writes to /usr/share/zsh/vendor\-completions/_rclone by default so
|
||
will probably need to be run with sudo or as root, eg
|
||
.IP
|
||
.nf
|
||
\f[C]
|
||
sudo\ rclone\ genautocomplete\ zsh
|
||
\f[]
|
||
.fi
|
||
.PP
|
||
Logout and login again to use the autocompletion scripts, or source them
|
||
directly
|
||
.IP
|
||
.nf
|
||
\f[C]
|
||
autoload\ \-U\ compinit\ &&\ compinit
|
||
\f[]
|
||
.fi
|
||
.PP
|
||
If you supply a command line argument the script will be written there.
|
||
.IP
|
||
.nf
|
||
\f[C]
|
||
rclone\ genautocomplete\ zsh\ [output_file]\ [flags]
|
||
\f[]
|
||
.fi
|
||
.SS Options
|
||
.IP
|
||
.nf
|
||
\f[C]
|
||
\ \ \-h,\ \-\-help\ \ \ help\ for\ zsh
|
||
\f[]
|
||
.fi
|
||
.SS rclone gendocs
|
||
.PP
|
||
Output markdown docs for rclone to the directory supplied.
|
||
.SS Synopsis
|
||
.PP
|
||
This produces markdown docs for the rclone commands to the directory
|
||
supplied.
|
||
These are in a format suitable for hugo to render into the rclone.org
|
||
website.
|
||
.IP
|
||
.nf
|
||
\f[C]
|
||
rclone\ gendocs\ output_directory\ [flags]
|
||
\f[]
|
||
.fi
|
||
.SS Options
|
||
.IP
|
||
.nf
|
||
\f[C]
|
||
\ \ \-h,\ \-\-help\ \ \ help\ for\ gendocs
|
||
\f[]
|
||
.fi
|
||
.SS rclone listremotes
|
||
.PP
|
||
List all the remotes in the config file.
|
||
.SS Synopsis
|
||
.PP
|
||
rclone listremotes lists all the available remotes from the config file.
|
||
.PP
|
||
When uses with the \-l flag it lists the types too.
|
||
.IP
|
||
.nf
|
||
\f[C]
|
||
rclone\ listremotes\ [flags]
|
||
\f[]
|
||
.fi
|
||
.SS Options
|
||
.IP
|
||
.nf
|
||
\f[C]
|
||
\ \ \-h,\ \-\-help\ \ \ help\ for\ listremotes
|
||
\ \ \-l,\ \-\-long\ \ \ Show\ the\ type\ as\ well\ as\ names.
|
||
\f[]
|
||
.fi
|
||
.SS rclone lsjson
|
||
.PP
|
||
List directories and objects in the path in JSON format.
|
||
.SS Synopsis
|
||
.PP
|
||
List directories and objects in the path in JSON format.
|
||
.PP
|
||
The output is an array of Items, where each Item looks like this
|
||
.PP
|
||
{ "Hashes" : { "SHA\-1" : "f572d396fae9206628714fb2ce00f72e94f2258f",
|
||
"MD5" : "b1946ac92492d2347c6235b4d2611184", "DropboxHash" :
|
||
"ecb65bb98f9d905b70458986c39fcbad7715e5f2fcc3b1f07767d7c83e2438cc" },
|
||
"IsDir" : false, "ModTime" : "2017\-05\-31T16:15:57.034468261+01:00",
|
||
"Name" : "file.txt", "Path" : "full/path/goes/here/file.txt", "Size" : 6
|
||
}
|
||
.PP
|
||
If \-\-hash is not specified the the Hashes property won\[aq]t be
|
||
emitted.
|
||
.PP
|
||
If \-\-no\-modtime is specified then ModTime will be blank.
|
||
.PP
|
||
The time is in RFC3339 format with nanosecond precision.
|
||
.PP
|
||
The whole output can be processed as a JSON blob, or alternatively it
|
||
can be processed line by line as each item is written one to a line.
|
||
.IP
|
||
.nf
|
||
\f[C]
|
||
rclone\ lsjson\ remote:path\ [flags]
|
||
\f[]
|
||
.fi
|
||
.SS Options
|
||
.IP
|
||
.nf
|
||
\f[C]
|
||
\ \ \ \ \ \ \-\-hash\ \ \ \ \ \ \ \ \ Include\ hashes\ in\ the\ output\ (may\ take\ longer).
|
||
\ \ \-h,\ \-\-help\ \ \ \ \ \ \ \ \ help\ for\ lsjson
|
||
\ \ \ \ \ \ \-\-no\-modtime\ \ \ Don\[aq]t\ read\ the\ modification\ time\ (can\ speed\ things\ up).
|
||
\ \ \-R,\ \-\-recursive\ \ \ \ Recurse\ into\ the\ listing.
|
||
\f[]
|
||
.fi
|
||
.SS rclone mount
|
||
.PP
|
||
Mount the remote as a mountpoint.
|
||
\f[B]EXPERIMENTAL\f[]
|
||
.SS Synopsis
|
||
.PP
|
||
rclone mount allows Linux, FreeBSD, macOS and Windows to mount any of
|
||
Rclone\[aq]s cloud storage systems as a file system with FUSE.
|
||
.PP
|
||
This is \f[B]EXPERIMENTAL\f[] \- use with care.
|
||
.PP
|
||
First set up your remote using \f[C]rclone\ config\f[].
|
||
Check it works with \f[C]rclone\ ls\f[] etc.
|
||
.PP
|
||
Start the mount like this
|
||
.IP
|
||
.nf
|
||
\f[C]
|
||
rclone\ mount\ remote:path/to/files\ /path/to/local/mount
|
||
\f[]
|
||
.fi
|
||
.PP
|
||
Or on Windows like this where X: is an unused drive letter
|
||
.IP
|
||
.nf
|
||
\f[C]
|
||
rclone\ mount\ remote:path/to/files\ X:
|
||
\f[]
|
||
.fi
|
||
.PP
|
||
When the program ends, either via Ctrl+C or receiving a SIGINT or
|
||
SIGTERM signal, the mount is automatically stopped.
|
||
.PP
|
||
The umount operation can fail, for example when the mountpoint is busy.
|
||
When that happens, it is the user\[aq]s responsibility to stop the mount
|
||
manually with
|
||
.IP
|
||
.nf
|
||
\f[C]
|
||
#\ Linux
|
||
fusermount\ \-u\ /path/to/local/mount
|
||
#\ OS\ X
|
||
umount\ /path/to/local/mount
|
||
\f[]
|
||
.fi
|
||
.SS Installing on Windows
|
||
.PP
|
||
To run rclone mount on Windows, you will need to download and install
|
||
WinFsp (http://www.secfs.net/winfsp/).
|
||
.PP
|
||
WinFsp is an open source (https://github.com/billziss-gh/winfsp) Windows
|
||
File System Proxy which makes it easy to write user space file systems
|
||
for Windows.
|
||
It provides a FUSE emulation layer which rclone uses combination with
|
||
cgofuse (https://github.com/billziss-gh/cgofuse).
|
||
Both of these packages are by Bill Zissimopoulos who was very helpful
|
||
during the implementation of rclone mount for Windows.
|
||
.SS Windows caveats
|
||
.PP
|
||
Note that drives created as Administrator are not visible by other
|
||
accounts (including the account that was elevated as Administrator).
|
||
So if you start a Windows drive from an Administrative Command Prompt
|
||
and then try to access the same drive from Explorer (which does not run
|
||
as Administrator), you will not be able to see the new drive.
|
||
.PP
|
||
The easiest way around this is to start the drive from a normal command
|
||
prompt.
|
||
It is also possible to start a drive from the SYSTEM account (using the
|
||
WinFsp.Launcher
|
||
infrastructure (https://github.com/billziss-gh/winfsp/wiki/WinFsp-Service-Architecture))
|
||
which creates drives accessible for everyone on the system.
|
||
.SS Limitations
|
||
.PP
|
||
This can only write files seqentially, it can only seek when reading.
|
||
This means that many applications won\[aq]t work with their files on an
|
||
rclone mount.
|
||
.PP
|
||
The bucket based remotes (eg Swift, S3, Google Compute Storage, B2,
|
||
Hubic) won\[aq]t work from the root \- you will need to specify a
|
||
bucket, or a path within the bucket.
|
||
So \f[C]swift:\f[] won\[aq]t work whereas \f[C]swift:bucket\f[] will as
|
||
will \f[C]swift:bucket/path\f[].
|
||
None of these support the concept of directories, so empty directories
|
||
will have a tendency to disappear once they fall out of the directory
|
||
cache.
|
||
.PP
|
||
Only supported on Linux, FreeBSD, OS X and Windows at the moment.
|
||
.SS rclone mount vs rclone sync/copy
|
||
.PP
|
||
File systems expect things to be 100% reliable, whereas cloud storage
|
||
systems are a long way from 100% reliable.
|
||
The rclone sync/copy commands cope with this with lots of retries.
|
||
However rclone mount can\[aq]t use retries in the same way without
|
||
making local copies of the uploads.
|
||
This might happen in the future, but for the moment rclone mount
|
||
won\[aq]t do that, so will be less reliable than the rclone command.
|
||
.SS Filters
|
||
.PP
|
||
Note that all the rclone filters can be used to select a subset of the
|
||
files to be visible in the mount.
|
||
.SS systemd
|
||
.PP
|
||
When running rclone mount as a systemd service, it is possible to use
|
||
Type=notify.
|
||
In this case the service will enter the started state after the
|
||
mountpoint has been successfully set up.
|
||
Units having the rclone mount service specified as a requirement will
|
||
see all files and folders immediately in this mode.
|
||
.SS Directory Cache
|
||
.PP
|
||
Using the \f[C]\-\-dir\-cache\-time\f[] flag, you can set how long a
|
||
directory should be considered up to date and not refreshed from the
|
||
backend.
|
||
Changes made locally in the mount may appear immediately or invalidate
|
||
the cache.
|
||
However, changes done on the remote will only be picked up once the
|
||
cache expires.
|
||
.PP
|
||
Alternatively, you can send a \f[C]SIGHUP\f[] signal to rclone for it to
|
||
flush all directory caches, regardless of how old they are.
|
||
Assuming only one rclone instance is running, you can reset the cache
|
||
like this:
|
||
.IP
|
||
.nf
|
||
\f[C]
|
||
kill\ \-SIGHUP\ $(pidof\ rclone)
|
||
\f[]
|
||
.fi
|
||
.SS File Caching
|
||
.PP
|
||
\f[B]NB\f[] File caching is \f[B]EXPERIMENTAL\f[] \- use with care!
|
||
.PP
|
||
These flags control the VFS file caching options.
|
||
The VFS layer is used by rclone mount to make a cloud storage systm work
|
||
more like a normal file system.
|
||
.PP
|
||
You\[aq]ll need to enable VFS caching if you want, for example, to read
|
||
and write simultaneously to a file.
|
||
See below for more details.
|
||
.PP
|
||
Note that the VFS cache works in addition to the cache backend and you
|
||
may find that you need one or the other or both.
|
||
.IP
|
||
.nf
|
||
\f[C]
|
||
\-\-vfs\-cache\-dir\ string\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ Directory\ rclone\ will\ use\ for\ caching.
|
||
\-\-vfs\-cache\-max\-age\ duration\ \ \ \ \ \ \ \ \ Max\ age\ of\ objects\ in\ the\ cache.\ (default\ 1h0m0s)
|
||
\-\-vfs\-cache\-mode\ string\ \ \ \ \ \ \ \ \ \ \ \ \ \ Cache\ mode\ off|minimal|writes|full\ (default\ "off")
|
||
\-\-vfs\-cache\-poll\-interval\ duration\ \ \ Interval\ to\ poll\ the\ cache\ for\ stale\ objects.\ (default\ 1m0s)
|
||
\f[]
|
||
.fi
|
||
.PP
|
||
If run with \f[C]\-vv\f[] rclone will print the location of the file
|
||
cache.
|
||
The files are stored in the user cache file area which is OS dependent
|
||
but can be controlled with \f[C]\-\-cache\-dir\f[] or setting the
|
||
appropriate environment variable.
|
||
.PP
|
||
The cache has 4 different modes selected by
|
||
\f[C]\-\-vfs\-cache\-mode\f[].
|
||
The higher the cache mode the more compatible rclone becomes at the cost
|
||
of using disk space.
|
||
.PP
|
||
Note that files are written back to the remote only when they are closed
|
||
so if rclone is quit or dies with open files then these won\[aq]t get
|
||
written back to the remote.
|
||
However they will still be in the on disk cache.
|
||
.SS \-\-vfs\-cache\-mode off
|
||
.PP
|
||
In this mode the cache will read directly from the remote and write
|
||
directly to the remote without caching anything on disk.
|
||
.PP
|
||
This will mean some operations are not possible
|
||
.IP \[bu] 2
|
||
Files can\[aq]t be opened for both read AND write
|
||
.IP \[bu] 2
|
||
Files opened for write can\[aq]t be seeked
|
||
.IP \[bu] 2
|
||
Existing files opened for write must have O_TRUNC set
|
||
.IP \[bu] 2
|
||
Files open for read with O_TRUNC will be opened write only
|
||
.IP \[bu] 2
|
||
Files open for write only will behave as if O_TRUNC was supplied
|
||
.IP \[bu] 2
|
||
Open modes O_APPEND, O_TRUNC are ignored
|
||
.IP \[bu] 2
|
||
If an upload fails it can\[aq]t be retried
|
||
.SS \-\-vfs\-cache\-mode minimal
|
||
.PP
|
||
This is very similar to "off" except that files opened for read AND
|
||
write will be buffered to disks.
|
||
This means that files opened for write will be a lot more compatible,
|
||
but uses the minimal disk space.
|
||
.PP
|
||
These operations are not possible
|
||
.IP \[bu] 2
|
||
Files opened for write only can\[aq]t be seeked
|
||
.IP \[bu] 2
|
||
Existing files opened for write must have O_TRUNC set
|
||
.IP \[bu] 2
|
||
Files opened for write only will ignore O_APPEND, O_TRUNC
|
||
.IP \[bu] 2
|
||
If an upload fails it can\[aq]t be retried
|
||
.SS \-\-vfs\-cache\-mode writes
|
||
.PP
|
||
In this mode files opened for read only are still read directly from the
|
||
remote, write only and read/write files are buffered to disk first.
|
||
.PP
|
||
This mode should support all normal file system operations.
|
||
.PP
|
||
If an upload fails it will be retried up to \-\-low\-level\-retries
|
||
times.
|
||
.SS \-\-vfs\-cache\-mode full
|
||
.PP
|
||
In this mode all reads and writes are buffered to and from disk.
|
||
When a file is opened for read it will be downloaded in its entirety
|
||
first.
|
||
.PP
|
||
This may be appropriate for your needs, or you may prefer to look at the
|
||
cache backend which does a much more sophisticated job of caching,
|
||
including caching directory heirachies and chunks of files.q
|
||
.PP
|
||
In this mode, unlike the others, when a file is written to the disk, it
|
||
will be kept on the disk after it is written to the remote.
|
||
It will be purged on a schedule according to
|
||
\f[C]\-\-vfs\-cache\-max\-age\f[].
|
||
.PP
|
||
This mode should support all normal file system operations.
|
||
.PP
|
||
If an upload or download fails it will be retried up to
|
||
\-\-low\-level\-retries times.
|
||
.IP
|
||
.nf
|
||
\f[C]
|
||
rclone\ mount\ remote:path\ /path/to/mountpoint\ [flags]
|
||
\f[]
|
||
.fi
|
||
.SS Options
|
||
.IP
|
||
.nf
|
||
\f[C]
|
||
\ \ \ \ \ \ \-\-allow\-non\-empty\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ Allow\ mounting\ over\ a\ non\-empty\ directory.
|
||
\ \ \ \ \ \ \-\-allow\-other\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ Allow\ access\ to\ other\ users.
|
||
\ \ \ \ \ \ \-\-allow\-root\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ Allow\ access\ to\ root\ user.
|
||
\ \ \ \ \ \ \-\-debug\-fuse\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ Debug\ the\ FUSE\ internals\ \-\ needs\ \-v.
|
||
\ \ \ \ \ \ \-\-default\-permissions\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ Makes\ kernel\ enforce\ access\ control\ based\ on\ the\ file\ mode.
|
||
\ \ \ \ \ \ \-\-dir\-cache\-time\ duration\ \ \ \ \ \ \ \ \ \ \ \ Time\ to\ cache\ directory\ entries\ for.\ (default\ 5m0s)
|
||
\ \ \ \ \ \ \-\-fuse\-flag\ stringArray\ \ \ \ \ \ \ \ \ \ \ \ \ \ Flags\ or\ arguments\ to\ be\ passed\ direct\ to\ libfuse/WinFsp.\ Repeat\ if\ required.
|
||
\ \ \ \ \ \ \-\-gid\ uint32\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ Override\ the\ gid\ field\ set\ by\ the\ filesystem.\ (default\ 502)
|
||
\ \ \-h,\ \-\-help\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ help\ for\ mount
|
||
\ \ \ \ \ \ \-\-max\-read\-ahead\ int\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ The\ number\ of\ bytes\ that\ can\ be\ prefetched\ for\ sequential\ reads.\ (default\ 128k)
|
||
\ \ \ \ \ \ \-\-no\-checksum\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ Don\[aq]t\ compare\ checksums\ on\ up/download.
|
||
\ \ \ \ \ \ \-\-no\-modtime\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ Don\[aq]t\ read/write\ the\ modification\ time\ (can\ speed\ things\ up).
|
||
\ \ \ \ \ \ \-\-no\-seek\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ Don\[aq]t\ allow\ seeking\ in\ files.
|
||
\ \ \-o,\ \-\-option\ stringArray\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ Option\ for\ libfuse/WinFsp.\ Repeat\ if\ required.
|
||
\ \ \ \ \ \ \-\-poll\-interval\ duration\ \ \ \ \ \ \ \ \ \ \ \ \ Time\ to\ wait\ between\ polling\ for\ changes.\ Must\ be\ smaller\ than\ dir\-cache\-time.\ Only\ on\ supported\ remotes.\ Set\ to\ 0\ to\ disable.\ (default\ 1m0s)
|
||
\ \ \ \ \ \ \-\-read\-only\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ Mount\ read\-only.
|
||
\ \ \ \ \ \ \-\-uid\ uint32\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ Override\ the\ uid\ field\ set\ by\ the\ filesystem.\ (default\ 502)
|
||
\ \ \ \ \ \ \-\-umask\ int\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ Override\ the\ permission\ bits\ set\ by\ the\ filesystem.
|
||
\ \ \ \ \ \ \-\-vfs\-cache\-max\-age\ duration\ \ \ \ \ \ \ \ \ Max\ age\ of\ objects\ in\ the\ cache.\ (default\ 1h0m0s)
|
||
\ \ \ \ \ \ \-\-vfs\-cache\-mode\ string\ \ \ \ \ \ \ \ \ \ \ \ \ \ Cache\ mode\ off|minimal|writes|full\ (default\ "off")
|
||
\ \ \ \ \ \ \-\-vfs\-cache\-poll\-interval\ duration\ \ \ Interval\ to\ poll\ the\ cache\ for\ stale\ objects.\ (default\ 1m0s)
|
||
\ \ \ \ \ \ \-\-write\-back\-cache\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ Makes\ kernel\ buffer\ writes\ before\ sending\ them\ to\ rclone.\ Without\ this,\ writethrough\ caching\ is\ used.
|
||
\f[]
|
||
.fi
|
||
.SS rclone moveto
|
||
.PP
|
||
Move file or directory from source to dest.
|
||
.SS Synopsis
|
||
.PP
|
||
If source:path is a file or directory then it moves it to a file or
|
||
directory named dest:path.
|
||
.PP
|
||
This can be used to rename files or upload single files to other than
|
||
their existing name.
|
||
If the source is a directory then it acts exacty like the move command.
|
||
.PP
|
||
So
|
||
.IP
|
||
.nf
|
||
\f[C]
|
||
rclone\ moveto\ src\ dst
|
||
\f[]
|
||
.fi
|
||
.PP
|
||
where src and dst are rclone paths, either remote:path or /path/to/local
|
||
or C:.
|
||
.PP
|
||
This will:
|
||
.IP
|
||
.nf
|
||
\f[C]
|
||
if\ src\ is\ file
|
||
\ \ \ \ move\ it\ to\ dst,\ overwriting\ an\ existing\ file\ if\ it\ exists
|
||
if\ src\ is\ directory
|
||
\ \ \ \ move\ it\ to\ dst,\ overwriting\ existing\ files\ if\ they\ exist
|
||
\ \ \ \ see\ move\ command\ for\ full\ details
|
||
\f[]
|
||
.fi
|
||
.PP
|
||
This doesn\[aq]t transfer unchanged files, testing by size and
|
||
modification time or MD5SUM.
|
||
src will be deleted on successful transfer.
|
||
.PP
|
||
\f[B]Important\f[]: Since this can cause data loss, test first with the
|
||
\-\-dry\-run flag.
|
||
.IP
|
||
.nf
|
||
\f[C]
|
||
rclone\ moveto\ source:path\ dest:path\ [flags]
|
||
\f[]
|
||
.fi
|
||
.SS Options
|
||
.IP
|
||
.nf
|
||
\f[C]
|
||
\ \ \-h,\ \-\-help\ \ \ help\ for\ moveto
|
||
\f[]
|
||
.fi
|
||
.SS rclone ncdu
|
||
.PP
|
||
Explore a remote with a text based user interface.
|
||
.SS Synopsis
|
||
.PP
|
||
This displays a text based user interface allowing the navigation of a
|
||
remote.
|
||
It is most useful for answering the question \- "What is using all my
|
||
disk space?".
|
||
.PP
|
||
To make the user interface it first scans the entire remote given and
|
||
builds an in memory representation.
|
||
rclone ncdu can be used during this scanning phase and you will see it
|
||
building up the directory structure as it goes along.
|
||
.PP
|
||
Here are the keys \- press \[aq]?\[aq] to toggle the help on and off
|
||
.IP
|
||
.nf
|
||
\f[C]
|
||
\ ↑,↓\ or\ k,j\ to\ Move
|
||
\ →,l\ to\ enter
|
||
\ ←,h\ to\ return
|
||
\ c\ toggle\ counts
|
||
\ g\ toggle\ graph
|
||
\ n,s,C\ sort\ by\ name,size,count
|
||
\ ?\ to\ toggle\ help\ on\ and\ off
|
||
\ q/ESC/c\-C\ to\ quit
|
||
\f[]
|
||
.fi
|
||
.PP
|
||
This an homage to the ncdu tool (https://dev.yorhel.nl/ncdu) but for
|
||
rclone remotes.
|
||
It is missing lots of features at the moment, most importantly deleting
|
||
files, but is useful as it stands.
|
||
.IP
|
||
.nf
|
||
\f[C]
|
||
rclone\ ncdu\ remote:path\ [flags]
|
||
\f[]
|
||
.fi
|
||
.SS Options
|
||
.IP
|
||
.nf
|
||
\f[C]
|
||
\ \ \-h,\ \-\-help\ \ \ help\ for\ ncdu
|
||
\f[]
|
||
.fi
|
||
.SS rclone obscure
|
||
.PP
|
||
Obscure password for use in the rclone.conf
|
||
.SS Synopsis
|
||
.PP
|
||
Obscure password for use in the rclone.conf
|
||
.IP
|
||
.nf
|
||
\f[C]
|
||
rclone\ obscure\ password\ [flags]
|
||
\f[]
|
||
.fi
|
||
.SS Options
|
||
.IP
|
||
.nf
|
||
\f[C]
|
||
\ \ \-h,\ \-\-help\ \ \ help\ for\ obscure
|
||
\f[]
|
||
.fi
|
||
.SS rclone rcat
|
||
.PP
|
||
Copies standard input to file on remote.
|
||
.SS Synopsis
|
||
.PP
|
||
rclone rcat reads from standard input (stdin) and copies it to a single
|
||
remote file.
|
||
.IP
|
||
.nf
|
||
\f[C]
|
||
echo\ "hello\ world"\ |\ rclone\ rcat\ remote:path/to/file
|
||
ffmpeg\ \-\ |\ rclone\ rcat\ \-\-checksum\ remote:path/to/file
|
||
\f[]
|
||
.fi
|
||
.PP
|
||
If the remote file already exists, it will be overwritten.
|
||
.PP
|
||
rcat will try to upload small files in a single request, which is
|
||
usually more efficient than the streaming/chunked upload endpoints,
|
||
which use multiple requests.
|
||
Exact behaviour depends on the remote.
|
||
What is considered a small file may be set through
|
||
\f[C]\-\-streaming\-upload\-cutoff\f[].
|
||
Uploading only starts after the cutoff is reached or if the file ends
|
||
before that.
|
||
The data must fit into RAM.
|
||
The cutoff needs to be small enough to adhere the limits of your remote,
|
||
please see there.
|
||
Generally speaking, setting this cutoff too high will decrease your
|
||
performance.
|
||
.PP
|
||
Note that the upload can also not be retried because the data is not
|
||
kept around until the upload succeeds.
|
||
If you need to transfer a lot of data, you\[aq]re better off caching
|
||
locally and then \f[C]rclone\ move\f[] it to the destination.
|
||
.IP
|
||
.nf
|
||
\f[C]
|
||
rclone\ rcat\ remote:path\ [flags]
|
||
\f[]
|
||
.fi
|
||
.SS Options
|
||
.IP
|
||
.nf
|
||
\f[C]
|
||
\ \ \-h,\ \-\-help\ \ \ help\ for\ rcat
|
||
\f[]
|
||
.fi
|
||
.SS rclone rmdirs
|
||
.PP
|
||
Remove empty directories under the path.
|
||
.SS Synopsis
|
||
.PP
|
||
This removes any empty directories (or directories that only contain
|
||
empty directories) under the path that it finds, including the path if
|
||
it has nothing in.
|
||
.PP
|
||
If you supply the \-\-leave\-root flag, it will not remove the root
|
||
directory.
|
||
.PP
|
||
This is useful for tidying up remotes that rclone has left a lot of
|
||
empty directories in.
|
||
.IP
|
||
.nf
|
||
\f[C]
|
||
rclone\ rmdirs\ remote:path\ [flags]
|
||
\f[]
|
||
.fi
|
||
.SS Options
|
||
.IP
|
||
.nf
|
||
\f[C]
|
||
\ \ \-h,\ \-\-help\ \ \ \ \ \ \ \ \ help\ for\ rmdirs
|
||
\ \ \ \ \ \ \-\-leave\-root\ \ \ Do\ not\ remove\ root\ directory\ if\ empty
|
||
\f[]
|
||
.fi
|
||
.SS rclone serve
|
||
.PP
|
||
Serve a remote over a protocol.
|
||
.SS Synopsis
|
||
.PP
|
||
rclone serve is used to serve a remote over a given protocol.
|
||
This command requires the use of a subcommand to specify the protocol,
|
||
eg
|
||
.IP
|
||
.nf
|
||
\f[C]
|
||
rclone\ serve\ http\ remote:
|
||
\f[]
|
||
.fi
|
||
.PP
|
||
Each subcommand has its own options which you can see in their help.
|
||
.IP
|
||
.nf
|
||
\f[C]
|
||
rclone\ serve\ <protocol>\ [opts]\ <remote>\ [flags]
|
||
\f[]
|
||
.fi
|
||
.SS Options
|
||
.IP
|
||
.nf
|
||
\f[C]
|
||
\ \ \-h,\ \-\-help\ \ \ help\ for\ serve
|
||
\f[]
|
||
.fi
|
||
.SS rclone serve http
|
||
.PP
|
||
Serve the remote over HTTP.
|
||
.SS Synopsis
|
||
.PP
|
||
rclone serve http implements a basic web server to serve the remote over
|
||
HTTP.
|
||
This can be viewed in a web browser or you can make a remote of type
|
||
http read from it.
|
||
.PP
|
||
Use \-\-addr to specify which IP address and port the server should
|
||
listen on, eg \-\-addr 1.2.3.4:8000 or \-\-addr :8080 to listen to all
|
||
IPs.
|
||
By default it only listens on localhost.
|
||
.PP
|
||
You can use the filter flags (eg \-\-include, \-\-exclude) to control
|
||
what is served.
|
||
.PP
|
||
The server will log errors.
|
||
Use \-v to see access logs.
|
||
.PP
|
||
\-\-bwlimit will be respected for file transfers.
|
||
Use \-\-stats to control the stats printing.
|
||
.SS Directory Cache
|
||
.PP
|
||
Using the \f[C]\-\-dir\-cache\-time\f[] flag, you can set how long a
|
||
directory should be considered up to date and not refreshed from the
|
||
backend.
|
||
Changes made locally in the mount may appear immediately or invalidate
|
||
the cache.
|
||
However, changes done on the remote will only be picked up once the
|
||
cache expires.
|
||
.PP
|
||
Alternatively, you can send a \f[C]SIGHUP\f[] signal to rclone for it to
|
||
flush all directory caches, regardless of how old they are.
|
||
Assuming only one rclone instance is running, you can reset the cache
|
||
like this:
|
||
.IP
|
||
.nf
|
||
\f[C]
|
||
kill\ \-SIGHUP\ $(pidof\ rclone)
|
||
\f[]
|
||
.fi
|
||
.SS File Caching
|
||
.PP
|
||
\f[B]NB\f[] File caching is \f[B]EXPERIMENTAL\f[] \- use with care!
|
||
.PP
|
||
These flags control the VFS file caching options.
|
||
The VFS layer is used by rclone mount to make a cloud storage systm work
|
||
more like a normal file system.
|
||
.PP
|
||
You\[aq]ll need to enable VFS caching if you want, for example, to read
|
||
and write simultaneously to a file.
|
||
See below for more details.
|
||
.PP
|
||
Note that the VFS cache works in addition to the cache backend and you
|
||
may find that you need one or the other or both.
|
||
.IP
|
||
.nf
|
||
\f[C]
|
||
\-\-vfs\-cache\-dir\ string\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ Directory\ rclone\ will\ use\ for\ caching.
|
||
\-\-vfs\-cache\-max\-age\ duration\ \ \ \ \ \ \ \ \ Max\ age\ of\ objects\ in\ the\ cache.\ (default\ 1h0m0s)
|
||
\-\-vfs\-cache\-mode\ string\ \ \ \ \ \ \ \ \ \ \ \ \ \ Cache\ mode\ off|minimal|writes|full\ (default\ "off")
|
||
\-\-vfs\-cache\-poll\-interval\ duration\ \ \ Interval\ to\ poll\ the\ cache\ for\ stale\ objects.\ (default\ 1m0s)
|
||
\f[]
|
||
.fi
|
||
.PP
|
||
If run with \f[C]\-vv\f[] rclone will print the location of the file
|
||
cache.
|
||
The files are stored in the user cache file area which is OS dependent
|
||
but can be controlled with \f[C]\-\-cache\-dir\f[] or setting the
|
||
appropriate environment variable.
|
||
.PP
|
||
The cache has 4 different modes selected by
|
||
\f[C]\-\-vfs\-cache\-mode\f[].
|
||
The higher the cache mode the more compatible rclone becomes at the cost
|
||
of using disk space.
|
||
.PP
|
||
Note that files are written back to the remote only when they are closed
|
||
so if rclone is quit or dies with open files then these won\[aq]t get
|
||
written back to the remote.
|
||
However they will still be in the on disk cache.
|
||
.SS \-\-vfs\-cache\-mode off
|
||
.PP
|
||
In this mode the cache will read directly from the remote and write
|
||
directly to the remote without caching anything on disk.
|
||
.PP
|
||
This will mean some operations are not possible
|
||
.IP \[bu] 2
|
||
Files can\[aq]t be opened for both read AND write
|
||
.IP \[bu] 2
|
||
Files opened for write can\[aq]t be seeked
|
||
.IP \[bu] 2
|
||
Existing files opened for write must have O_TRUNC set
|
||
.IP \[bu] 2
|
||
Files open for read with O_TRUNC will be opened write only
|
||
.IP \[bu] 2
|
||
Files open for write only will behave as if O_TRUNC was supplied
|
||
.IP \[bu] 2
|
||
Open modes O_APPEND, O_TRUNC are ignored
|
||
.IP \[bu] 2
|
||
If an upload fails it can\[aq]t be retried
|
||
.SS \-\-vfs\-cache\-mode minimal
|
||
.PP
|
||
This is very similar to "off" except that files opened for read AND
|
||
write will be buffered to disks.
|
||
This means that files opened for write will be a lot more compatible,
|
||
but uses the minimal disk space.
|
||
.PP
|
||
These operations are not possible
|
||
.IP \[bu] 2
|
||
Files opened for write only can\[aq]t be seeked
|
||
.IP \[bu] 2
|
||
Existing files opened for write must have O_TRUNC set
|
||
.IP \[bu] 2
|
||
Files opened for write only will ignore O_APPEND, O_TRUNC
|
||
.IP \[bu] 2
|
||
If an upload fails it can\[aq]t be retried
|
||
.SS \-\-vfs\-cache\-mode writes
|
||
.PP
|
||
In this mode files opened for read only are still read directly from the
|
||
remote, write only and read/write files are buffered to disk first.
|
||
.PP
|
||
This mode should support all normal file system operations.
|
||
.PP
|
||
If an upload fails it will be retried up to \-\-low\-level\-retries
|
||
times.
|
||
.SS \-\-vfs\-cache\-mode full
|
||
.PP
|
||
In this mode all reads and writes are buffered to and from disk.
|
||
When a file is opened for read it will be downloaded in its entirety
|
||
first.
|
||
.PP
|
||
This may be appropriate for your needs, or you may prefer to look at the
|
||
cache backend which does a much more sophisticated job of caching,
|
||
including caching directory heirachies and chunks of files.q
|
||
.PP
|
||
In this mode, unlike the others, when a file is written to the disk, it
|
||
will be kept on the disk after it is written to the remote.
|
||
It will be purged on a schedule according to
|
||
\f[C]\-\-vfs\-cache\-max\-age\f[].
|
||
.PP
|
||
This mode should support all normal file system operations.
|
||
.PP
|
||
If an upload or download fails it will be retried up to
|
||
\-\-low\-level\-retries times.
|
||
.IP
|
||
.nf
|
||
\f[C]
|
||
rclone\ serve\ http\ remote:path\ [flags]
|
||
\f[]
|
||
.fi
|
||
.SS Options
|
||
.IP
|
||
.nf
|
||
\f[C]
|
||
\ \ \ \ \ \ \-\-addr\ string\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ IPaddress:Port\ to\ bind\ server\ to.\ (default\ "localhost:8080")
|
||
\ \ \ \ \ \ \-\-dir\-cache\-time\ duration\ \ \ \ \ \ \ \ \ \ \ \ Time\ to\ cache\ directory\ entries\ for.\ (default\ 5m0s)
|
||
\ \ \ \ \ \ \-\-gid\ uint32\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ Override\ the\ gid\ field\ set\ by\ the\ filesystem.\ (default\ 502)
|
||
\ \ \-h,\ \-\-help\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ help\ for\ http
|
||
\ \ \ \ \ \ \-\-no\-checksum\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ Don\[aq]t\ compare\ checksums\ on\ up/download.
|
||
\ \ \ \ \ \ \-\-no\-modtime\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ Don\[aq]t\ read/write\ the\ modification\ time\ (can\ speed\ things\ up).
|
||
\ \ \ \ \ \ \-\-no\-seek\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ Don\[aq]t\ allow\ seeking\ in\ files.
|
||
\ \ \ \ \ \ \-\-poll\-interval\ duration\ \ \ \ \ \ \ \ \ \ \ \ \ Time\ to\ wait\ between\ polling\ for\ changes.\ Must\ be\ smaller\ than\ dir\-cache\-time.\ Only\ on\ supported\ remotes.\ Set\ to\ 0\ to\ disable.\ (default\ 1m0s)
|
||
\ \ \ \ \ \ \-\-read\-only\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ Mount\ read\-only.
|
||
\ \ \ \ \ \ \-\-uid\ uint32\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ Override\ the\ uid\ field\ set\ by\ the\ filesystem.\ (default\ 502)
|
||
\ \ \ \ \ \ \-\-umask\ int\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ Override\ the\ permission\ bits\ set\ by\ the\ filesystem.\ (default\ 2)
|
||
\ \ \ \ \ \ \-\-vfs\-cache\-max\-age\ duration\ \ \ \ \ \ \ \ \ Max\ age\ of\ objects\ in\ the\ cache.\ (default\ 1h0m0s)
|
||
\ \ \ \ \ \ \-\-vfs\-cache\-mode\ string\ \ \ \ \ \ \ \ \ \ \ \ \ \ Cache\ mode\ off|minimal|writes|full\ (default\ "off")
|
||
\ \ \ \ \ \ \-\-vfs\-cache\-poll\-interval\ duration\ \ \ Interval\ to\ poll\ the\ cache\ for\ stale\ objects.\ (default\ 1m0s)
|
||
\f[]
|
||
.fi
|
||
.SS rclone serve webdav
|
||
.PP
|
||
Serve remote:path over webdav.
|
||
.SS Synopsis
|
||
.PP
|
||
rclone serve webdav implements a basic webdav server to serve the remote
|
||
over HTTP via the webdav protocol.
|
||
This can be viewed with a webdav client or you can make a remote of type
|
||
webdav to read and write it.
|
||
.PP
|
||
NB at the moment each directory listing reads the start of each file
|
||
which is undesirable: see https://github.com/golang/go/issues/22577
|
||
.SS Directory Cache
|
||
.PP
|
||
Using the \f[C]\-\-dir\-cache\-time\f[] flag, you can set how long a
|
||
directory should be considered up to date and not refreshed from the
|
||
backend.
|
||
Changes made locally in the mount may appear immediately or invalidate
|
||
the cache.
|
||
However, changes done on the remote will only be picked up once the
|
||
cache expires.
|
||
.PP
|
||
Alternatively, you can send a \f[C]SIGHUP\f[] signal to rclone for it to
|
||
flush all directory caches, regardless of how old they are.
|
||
Assuming only one rclone instance is running, you can reset the cache
|
||
like this:
|
||
.IP
|
||
.nf
|
||
\f[C]
|
||
kill\ \-SIGHUP\ $(pidof\ rclone)
|
||
\f[]
|
||
.fi
|
||
.SS File Caching
|
||
.PP
|
||
\f[B]NB\f[] File caching is \f[B]EXPERIMENTAL\f[] \- use with care!
|
||
.PP
|
||
These flags control the VFS file caching options.
|
||
The VFS layer is used by rclone mount to make a cloud storage systm work
|
||
more like a normal file system.
|
||
.PP
|
||
You\[aq]ll need to enable VFS caching if you want, for example, to read
|
||
and write simultaneously to a file.
|
||
See below for more details.
|
||
.PP
|
||
Note that the VFS cache works in addition to the cache backend and you
|
||
may find that you need one or the other or both.
|
||
.IP
|
||
.nf
|
||
\f[C]
|
||
\-\-vfs\-cache\-dir\ string\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ Directory\ rclone\ will\ use\ for\ caching.
|
||
\-\-vfs\-cache\-max\-age\ duration\ \ \ \ \ \ \ \ \ Max\ age\ of\ objects\ in\ the\ cache.\ (default\ 1h0m0s)
|
||
\-\-vfs\-cache\-mode\ string\ \ \ \ \ \ \ \ \ \ \ \ \ \ Cache\ mode\ off|minimal|writes|full\ (default\ "off")
|
||
\-\-vfs\-cache\-poll\-interval\ duration\ \ \ Interval\ to\ poll\ the\ cache\ for\ stale\ objects.\ (default\ 1m0s)
|
||
\f[]
|
||
.fi
|
||
.PP
|
||
If run with \f[C]\-vv\f[] rclone will print the location of the file
|
||
cache.
|
||
The files are stored in the user cache file area which is OS dependent
|
||
but can be controlled with \f[C]\-\-cache\-dir\f[] or setting the
|
||
appropriate environment variable.
|
||
.PP
|
||
The cache has 4 different modes selected by
|
||
\f[C]\-\-vfs\-cache\-mode\f[].
|
||
The higher the cache mode the more compatible rclone becomes at the cost
|
||
of using disk space.
|
||
.PP
|
||
Note that files are written back to the remote only when they are closed
|
||
so if rclone is quit or dies with open files then these won\[aq]t get
|
||
written back to the remote.
|
||
However they will still be in the on disk cache.
|
||
.SS \-\-vfs\-cache\-mode off
|
||
.PP
|
||
In this mode the cache will read directly from the remote and write
|
||
directly to the remote without caching anything on disk.
|
||
.PP
|
||
This will mean some operations are not possible
|
||
.IP \[bu] 2
|
||
Files can\[aq]t be opened for both read AND write
|
||
.IP \[bu] 2
|
||
Files opened for write can\[aq]t be seeked
|
||
.IP \[bu] 2
|
||
Existing files opened for write must have O_TRUNC set
|
||
.IP \[bu] 2
|
||
Files open for read with O_TRUNC will be opened write only
|
||
.IP \[bu] 2
|
||
Files open for write only will behave as if O_TRUNC was supplied
|
||
.IP \[bu] 2
|
||
Open modes O_APPEND, O_TRUNC are ignored
|
||
.IP \[bu] 2
|
||
If an upload fails it can\[aq]t be retried
|
||
.SS \-\-vfs\-cache\-mode minimal
|
||
.PP
|
||
This is very similar to "off" except that files opened for read AND
|
||
write will be buffered to disks.
|
||
This means that files opened for write will be a lot more compatible,
|
||
but uses the minimal disk space.
|
||
.PP
|
||
These operations are not possible
|
||
.IP \[bu] 2
|
||
Files opened for write only can\[aq]t be seeked
|
||
.IP \[bu] 2
|
||
Existing files opened for write must have O_TRUNC set
|
||
.IP \[bu] 2
|
||
Files opened for write only will ignore O_APPEND, O_TRUNC
|
||
.IP \[bu] 2
|
||
If an upload fails it can\[aq]t be retried
|
||
.SS \-\-vfs\-cache\-mode writes
|
||
.PP
|
||
In this mode files opened for read only are still read directly from the
|
||
remote, write only and read/write files are buffered to disk first.
|
||
.PP
|
||
This mode should support all normal file system operations.
|
||
.PP
|
||
If an upload fails it will be retried up to \-\-low\-level\-retries
|
||
times.
|
||
.SS \-\-vfs\-cache\-mode full
|
||
.PP
|
||
In this mode all reads and writes are buffered to and from disk.
|
||
When a file is opened for read it will be downloaded in its entirety
|
||
first.
|
||
.PP
|
||
This may be appropriate for your needs, or you may prefer to look at the
|
||
cache backend which does a much more sophisticated job of caching,
|
||
including caching directory heirachies and chunks of files.q
|
||
.PP
|
||
In this mode, unlike the others, when a file is written to the disk, it
|
||
will be kept on the disk after it is written to the remote.
|
||
It will be purged on a schedule according to
|
||
\f[C]\-\-vfs\-cache\-max\-age\f[].
|
||
.PP
|
||
This mode should support all normal file system operations.
|
||
.PP
|
||
If an upload or download fails it will be retried up to
|
||
\-\-low\-level\-retries times.
|
||
.IP
|
||
.nf
|
||
\f[C]
|
||
rclone\ serve\ webdav\ remote:path\ [flags]
|
||
\f[]
|
||
.fi
|
||
.SS Options
|
||
.IP
|
||
.nf
|
||
\f[C]
|
||
\ \ \ \ \ \ \-\-addr\ string\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ IPaddress:Port\ to\ bind\ server\ to.\ (default\ "localhost:8081")
|
||
\ \ \ \ \ \ \-\-dir\-cache\-time\ duration\ \ \ \ \ \ \ \ \ \ \ \ Time\ to\ cache\ directory\ entries\ for.\ (default\ 5m0s)
|
||
\ \ \ \ \ \ \-\-gid\ uint32\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ Override\ the\ gid\ field\ set\ by\ the\ filesystem.\ (default\ 502)
|
||
\ \ \-h,\ \-\-help\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ help\ for\ webdav
|
||
\ \ \ \ \ \ \-\-no\-checksum\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ Don\[aq]t\ compare\ checksums\ on\ up/download.
|
||
\ \ \ \ \ \ \-\-no\-modtime\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ Don\[aq]t\ read/write\ the\ modification\ time\ (can\ speed\ things\ up).
|
||
\ \ \ \ \ \ \-\-no\-seek\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ Don\[aq]t\ allow\ seeking\ in\ files.
|
||
\ \ \ \ \ \ \-\-poll\-interval\ duration\ \ \ \ \ \ \ \ \ \ \ \ \ Time\ to\ wait\ between\ polling\ for\ changes.\ Must\ be\ smaller\ than\ dir\-cache\-time.\ Only\ on\ supported\ remotes.\ Set\ to\ 0\ to\ disable.\ (default\ 1m0s)
|
||
\ \ \ \ \ \ \-\-read\-only\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ Mount\ read\-only.
|
||
\ \ \ \ \ \ \-\-uid\ uint32\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ Override\ the\ uid\ field\ set\ by\ the\ filesystem.\ (default\ 502)
|
||
\ \ \ \ \ \ \-\-umask\ int\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ Override\ the\ permission\ bits\ set\ by\ the\ filesystem.\ (default\ 2)
|
||
\ \ \ \ \ \ \-\-vfs\-cache\-max\-age\ duration\ \ \ \ \ \ \ \ \ Max\ age\ of\ objects\ in\ the\ cache.\ (default\ 1h0m0s)
|
||
\ \ \ \ \ \ \-\-vfs\-cache\-mode\ string\ \ \ \ \ \ \ \ \ \ \ \ \ \ Cache\ mode\ off|minimal|writes|full\ (default\ "off")
|
||
\ \ \ \ \ \ \-\-vfs\-cache\-poll\-interval\ duration\ \ \ Interval\ to\ poll\ the\ cache\ for\ stale\ objects.\ (default\ 1m0s)
|
||
\f[]
|
||
.fi
|
||
.SS rclone touch
|
||
.PP
|
||
Create new file or change file modification time.
|
||
.SS Synopsis
|
||
.PP
|
||
Create new file or change file modification time.
|
||
.IP
|
||
.nf
|
||
\f[C]
|
||
rclone\ touch\ remote:path\ [flags]
|
||
\f[]
|
||
.fi
|
||
.SS Options
|
||
.IP
|
||
.nf
|
||
\f[C]
|
||
\ \ \-h,\ \-\-help\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ help\ for\ touch
|
||
\ \ \-C,\ \-\-no\-create\ \ \ \ \ \ \ \ \ \ Do\ not\ create\ the\ file\ if\ it\ does\ not\ exist.
|
||
\ \ \-t,\ \-\-timestamp\ string\ \ \ Change\ the\ modification\ times\ to\ the\ specified\ time\ instead\ of\ the\ current\ time\ of\ day.\ The\ argument\ is\ of\ the\ form\ \[aq]YYMMDD\[aq]\ (ex.\ 17.10.30)\ or\ \[aq]YYYY\-MM\-DDTHH:MM:SS\[aq]\ (ex.\ 2006\-01\-02T15:04:05)
|
||
\f[]
|
||
.fi
|
||
.SS rclone tree
|
||
.PP
|
||
List the contents of the remote in a tree like fashion.
|
||
.SS Synopsis
|
||
.PP
|
||
rclone tree lists the contents of a remote in a similar way to the unix
|
||
tree command.
|
||
.PP
|
||
For example
|
||
.IP
|
||
.nf
|
||
\f[C]
|
||
$\ rclone\ tree\ remote:path
|
||
/
|
||
├──\ file1
|
||
├──\ file2
|
||
├──\ file3
|
||
└──\ subdir
|
||
\ \ \ \ ├──\ file4
|
||
\ \ \ \ └──\ file5
|
||
|
||
1\ directories,\ 5\ files
|
||
\f[]
|
||
.fi
|
||
.PP
|
||
You can use any of the filtering options with the tree command (eg
|
||
\-\-include and \-\-exclude).
|
||
You can also use \-\-fast\-list.
|
||
.PP
|
||
The tree command has many options for controlling the listing which are
|
||
compatible with the tree command.
|
||
Note that not all of them have short options as they conflict with
|
||
rclone\[aq]s short options.
|
||
.IP
|
||
.nf
|
||
\f[C]
|
||
rclone\ tree\ remote:path\ [flags]
|
||
\f[]
|
||
.fi
|
||
.SS Options
|
||
.IP
|
||
.nf
|
||
\f[C]
|
||
\ \ \-a,\ \-\-all\ \ \ \ \ \ \ \ \ \ \ \ \ All\ files\ are\ listed\ (list\ .\ files\ too).
|
||
\ \ \-C,\ \-\-color\ \ \ \ \ \ \ \ \ \ \ Turn\ colorization\ on\ always.
|
||
\ \ \-d,\ \-\-dirs\-only\ \ \ \ \ \ \ List\ directories\ only.
|
||
\ \ \ \ \ \ \-\-dirsfirst\ \ \ \ \ \ \ List\ directories\ before\ files\ (\-U\ disables).
|
||
\ \ \ \ \ \ \-\-full\-path\ \ \ \ \ \ \ Print\ the\ full\ path\ prefix\ for\ each\ file.
|
||
\ \ \-h,\ \-\-help\ \ \ \ \ \ \ \ \ \ \ \ help\ for\ tree
|
||
\ \ \ \ \ \ \-\-human\ \ \ \ \ \ \ \ \ \ \ Print\ the\ size\ in\ a\ more\ human\ readable\ way.
|
||
\ \ \ \ \ \ \-\-level\ int\ \ \ \ \ \ \ Descend\ only\ level\ directories\ deep.
|
||
\ \ \-D,\ \-\-modtime\ \ \ \ \ \ \ \ \ Print\ the\ date\ of\ last\ modification.
|
||
\ \ \-i,\ \-\-noindent\ \ \ \ \ \ \ \ Don\[aq]t\ print\ indentation\ lines.
|
||
\ \ \ \ \ \ \-\-noreport\ \ \ \ \ \ \ \ Turn\ off\ file/directory\ count\ at\ end\ of\ tree\ listing.
|
||
\ \ \-o,\ \-\-output\ string\ \ \ Output\ to\ file\ instead\ of\ stdout.
|
||
\ \ \-p,\ \-\-protections\ \ \ \ \ Print\ the\ protections\ for\ each\ file.
|
||
\ \ \-Q,\ \-\-quote\ \ \ \ \ \ \ \ \ \ \ Quote\ filenames\ with\ double\ quotes.
|
||
\ \ \-s,\ \-\-size\ \ \ \ \ \ \ \ \ \ \ \ Print\ the\ size\ in\ bytes\ of\ each\ file.
|
||
\ \ \ \ \ \ \-\-sort\ string\ \ \ \ \ Select\ sort:\ name,version,size,mtime,ctime.
|
||
\ \ \ \ \ \ \-\-sort\-ctime\ \ \ \ \ \ Sort\ files\ by\ last\ status\ change\ time.
|
||
\ \ \-t,\ \-\-sort\-modtime\ \ \ \ Sort\ files\ by\ last\ modification\ time.
|
||
\ \ \-r,\ \-\-sort\-reverse\ \ \ \ Reverse\ the\ order\ of\ the\ sort.
|
||
\ \ \-U,\ \-\-unsorted\ \ \ \ \ \ \ \ Leave\ files\ unsorted.
|
||
\ \ \ \ \ \ \-\-version\ \ \ \ \ \ \ \ \ Sort\ files\ alphanumerically\ by\ version.
|
||
\f[]
|
||
.fi
|
||
.SS Copying single files
|
||
.PP
|
||
rclone normally syncs or copies directories.
|
||
However, if the source remote points to a file, rclone will just copy
|
||
that file.
|
||
The destination remote must point to a directory \- rclone will give the
|
||
error
|
||
\f[C]Failed\ to\ create\ file\ system\ for\ "remote:file":\ is\ a\ file\ not\ a\ directory\f[]
|
||
if it isn\[aq]t.
|
||
.PP
|
||
For example, suppose you have a remote with a file in called
|
||
\f[C]test.jpg\f[], then you could copy just that file like this
|
||
.IP
|
||
.nf
|
||
\f[C]
|
||
rclone\ copy\ remote:test.jpg\ /tmp/download
|
||
\f[]
|
||
.fi
|
||
.PP
|
||
The file \f[C]test.jpg\f[] will be placed inside \f[C]/tmp/download\f[].
|
||
.PP
|
||
This is equivalent to specifying
|
||
.IP
|
||
.nf
|
||
\f[C]
|
||
rclone\ copy\ \-\-no\-traverse\ \-\-files\-from\ /tmp/files\ remote:\ /tmp/download
|
||
\f[]
|
||
.fi
|
||
.PP
|
||
Where \f[C]/tmp/files\f[] contains the single line
|
||
.IP
|
||
.nf
|
||
\f[C]
|
||
test.jpg
|
||
\f[]
|
||
.fi
|
||
.PP
|
||
It is recommended to use \f[C]copy\f[] when copying individual files,
|
||
not \f[C]sync\f[].
|
||
They have pretty much the same effect but \f[C]copy\f[] will use a lot
|
||
less memory.
|
||
.SS Quoting and the shell
|
||
.PP
|
||
When you are typing commands to your computer you are using something
|
||
called the command line shell.
|
||
This interprets various characters in an OS specific way.
|
||
.PP
|
||
Here are some gotchas which may help users unfamiliar with the shell
|
||
rules
|
||
.SS Linux / OSX
|
||
.PP
|
||
If your names have spaces or shell metacharacters (eg \f[C]*\f[],
|
||
\f[C]?\f[], \f[C]$\f[], \f[C]\[aq]\f[], \f[C]"\f[] etc) then you must
|
||
quote them.
|
||
Use single quotes \f[C]\[aq]\f[] by default.
|
||
.IP
|
||
.nf
|
||
\f[C]
|
||
rclone\ copy\ \[aq]Important\ files?\[aq]\ remote:backup
|
||
\f[]
|
||
.fi
|
||
.PP
|
||
If you want to send a \f[C]\[aq]\f[] you will need to use \f[C]"\f[], eg
|
||
.IP
|
||
.nf
|
||
\f[C]
|
||
rclone\ copy\ "O\[aq]Reilly\ Reviews"\ remote:backup
|
||
\f[]
|
||
.fi
|
||
.PP
|
||
The rules for quoting metacharacters are complicated and if you want the
|
||
full details you\[aq]ll have to consult the manual page for your shell.
|
||
.SS Windows
|
||
.PP
|
||
If your names have spaces in you need to put them in \f[C]"\f[], eg
|
||
.IP
|
||
.nf
|
||
\f[C]
|
||
rclone\ copy\ "E:\\folder\ name\\folder\ name\\folder\ name"\ remote:backup
|
||
\f[]
|
||
.fi
|
||
.PP
|
||
If you are using the root directory on its own then don\[aq]t quote it
|
||
(see #464 (https://github.com/ncw/rclone/issues/464) for why), eg
|
||
.IP
|
||
.nf
|
||
\f[C]
|
||
rclone\ copy\ E:\\\ remote:backup
|
||
\f[]
|
||
.fi
|
||
.SS Copying files or directories with \f[C]:\f[] in the names
|
||
.PP
|
||
rclone uses \f[C]:\f[] to mark a remote name.
|
||
This is, however, a valid filename component in non\-Windows OSes.
|
||
The remote name parser will only search for a \f[C]:\f[] up to the first
|
||
\f[C]/\f[] so if you need to act on a file or directory like this then
|
||
use the full path starting with a \f[C]/\f[], or use \f[C]\&./\f[] as a
|
||
current directory prefix.
|
||
.PP
|
||
So to sync a directory called \f[C]sync:me\f[] to a remote called
|
||
\f[C]remote:\f[] use
|
||
.IP
|
||
.nf
|
||
\f[C]
|
||
rclone\ sync\ ./sync:me\ remote:path
|
||
\f[]
|
||
.fi
|
||
.PP
|
||
or
|
||
.IP
|
||
.nf
|
||
\f[C]
|
||
rclone\ sync\ /full/path/to/sync:me\ remote:path
|
||
\f[]
|
||
.fi
|
||
.SS Server Side Copy
|
||
.PP
|
||
Most remotes (but not all \- see the
|
||
overview (/overview/#optional-features)) support server side copy.
|
||
.PP
|
||
This means if you want to copy one folder to another then rclone
|
||
won\[aq]t download all the files and re\-upload them; it will instruct
|
||
the server to copy them in place.
|
||
.PP
|
||
Eg
|
||
.IP
|
||
.nf
|
||
\f[C]
|
||
rclone\ copy\ s3:oldbucket\ s3:newbucket
|
||
\f[]
|
||
.fi
|
||
.PP
|
||
Will copy the contents of \f[C]oldbucket\f[] to \f[C]newbucket\f[]
|
||
without downloading and re\-uploading.
|
||
.PP
|
||
Remotes which don\[aq]t support server side copy \f[B]will\f[] download
|
||
and re\-upload in this case.
|
||
.PP
|
||
Server side copies are used with \f[C]sync\f[] and \f[C]copy\f[] and
|
||
will be identified in the log when using the \f[C]\-v\f[] flag.
|
||
The \f[C]move\f[] command may also use them if remote doesn\[aq]t
|
||
support server side move directly.
|
||
This is done by issuing a server side copy then a delete which is much
|
||
quicker than a download and re\-upload.
|
||
.PP
|
||
Server side copies will only be attempted if the remote names are the
|
||
same.
|
||
.PP
|
||
This can be used when scripting to make aged backups efficiently, eg
|
||
.IP
|
||
.nf
|
||
\f[C]
|
||
rclone\ sync\ remote:current\-backup\ remote:previous\-backup
|
||
rclone\ sync\ /path/to/files\ remote:current\-backup
|
||
\f[]
|
||
.fi
|
||
.SS Options
|
||
.PP
|
||
Rclone has a number of options to control its behaviour.
|
||
.PP
|
||
Options which use TIME use the go time parser.
|
||
A duration string is a possibly signed sequence of decimal numbers, each
|
||
with optional fraction and a unit suffix, such as "300ms", "\-1.5h" or
|
||
"2h45m".
|
||
Valid time units are "ns", "us" (or "µs"), "ms", "s", "m", "h".
|
||
.PP
|
||
Options which use SIZE use kByte by default.
|
||
However, a suffix of \f[C]b\f[] for bytes, \f[C]k\f[] for kBytes,
|
||
\f[C]M\f[] for MBytes and \f[C]G\f[] for GBytes may be used.
|
||
These are the binary units, eg 1, 2**10, 2**20, 2**30 respectively.
|
||
.SS \-\-backup\-dir=DIR
|
||
.PP
|
||
When using \f[C]sync\f[], \f[C]copy\f[] or \f[C]move\f[] any files which
|
||
would have been overwritten or deleted are moved in their original
|
||
hierarchy into this directory.
|
||
.PP
|
||
If \f[C]\-\-suffix\f[] is set, then the moved files will have the suffix
|
||
added to them.
|
||
If there is a file with the same path (after the suffix has been added)
|
||
in DIR, then it will be overwritten.
|
||
.PP
|
||
The remote in use must support server side move or copy and you must use
|
||
the same remote as the destination of the sync.
|
||
The backup directory must not overlap the destination directory.
|
||
.PP
|
||
For example
|
||
.IP
|
||
.nf
|
||
\f[C]
|
||
rclone\ sync\ /path/to/local\ remote:current\ \-\-backup\-dir\ remote:old
|
||
\f[]
|
||
.fi
|
||
.PP
|
||
will sync \f[C]/path/to/local\f[] to \f[C]remote:current\f[], but for
|
||
any files which would have been updated or deleted will be stored in
|
||
\f[C]remote:old\f[].
|
||
.PP
|
||
If running rclone from a script you might want to use today\[aq]s date
|
||
as the directory name passed to \f[C]\-\-backup\-dir\f[] to store the
|
||
old files, or you might want to pass \f[C]\-\-suffix\f[] with
|
||
today\[aq]s date.
|
||
.SS \-\-bind string
|
||
.PP
|
||
Local address to bind to for outgoing connections.
|
||
This can be an IPv4 address (1.2.3.4), an IPv6 address (1234::789A) or
|
||
host name.
|
||
If the host name doesn\[aq]t resolve or resolves to more than one IP
|
||
address it will give an error.
|
||
.SS \-\-bwlimit=BANDWIDTH_SPEC
|
||
.PP
|
||
This option controls the bandwidth limit.
|
||
Limits can be specified in two ways: As a single limit, or as a
|
||
timetable.
|
||
.PP
|
||
Single limits last for the duration of the session.
|
||
To use a single limit, specify the desired bandwidth in kBytes/s, or use
|
||
a suffix b|k|M|G.
|
||
The default is \f[C]0\f[] which means to not limit bandwidth.
|
||
.PP
|
||
For example, to limit bandwidth usage to 10 MBytes/s use
|
||
\f[C]\-\-bwlimit\ 10M\f[]
|
||
.PP
|
||
It is also possible to specify a "timetable" of limits, which will cause
|
||
certain limits to be applied at certain times.
|
||
To specify a timetable, format your entries as "HH:MM,BANDWIDTH
|
||
HH:MM,BANDWIDTH...".
|
||
.PP
|
||
An example of a typical timetable to avoid link saturation during
|
||
daytime working hours could be:
|
||
.PP
|
||
\f[C]\-\-bwlimit\ "08:00,512\ 12:00,10M\ 13:00,512\ 18:00,30M\ 23:00,off"\f[]
|
||
.PP
|
||
In this example, the transfer bandwidth will be set to 512kBytes/sec at
|
||
8am.
|
||
At noon, it will raise to 10Mbytes/s, and drop back to 512kBytes/sec at
|
||
1pm.
|
||
At 6pm, the bandwidth limit will be set to 30MBytes/s, and at 11pm it
|
||
will be completely disabled (full speed).
|
||
Anything between 11pm and 8am will remain unlimited.
|
||
.PP
|
||
Bandwidth limits only apply to the data transfer.
|
||
They don\[aq]t apply to the bandwidth of the directory listings etc.
|
||
.PP
|
||
Note that the units are Bytes/s, not Bits/s.
|
||
Typically connections are measured in Bits/s \- to convert divide by 8.
|
||
For example, let\[aq]s say you have a 10 Mbit/s connection and you wish
|
||
rclone to use half of it \- 5 Mbit/s.
|
||
This is 5/8 = 0.625MByte/s so you would use a
|
||
\f[C]\-\-bwlimit\ 0.625M\f[] parameter for rclone.
|
||
.PP
|
||
On Unix systems (Linux, MacOS, \&...) the bandwidth limiter can be
|
||
toggled by sending a \f[C]SIGUSR2\f[] signal to rclone.
|
||
This allows to remove the limitations of a long running rclone transfer
|
||
and to restore it back to the value specified with \f[C]\-\-bwlimit\f[]
|
||
quickly when needed.
|
||
Assuming there is only one rclone instance running, you can toggle the
|
||
limiter like this:
|
||
.IP
|
||
.nf
|
||
\f[C]
|
||
kill\ \-SIGUSR2\ $(pidof\ rclone)
|
||
\f[]
|
||
.fi
|
||
.SS \-\-buffer\-size=SIZE
|
||
.PP
|
||
Use this sized buffer to speed up file transfers.
|
||
Each \f[C]\-\-transfer\f[] will use this much memory for buffering.
|
||
.PP
|
||
Set to 0 to disable the buffering for the minimum memory usage.
|
||
.SS \-\-checkers=N
|
||
.PP
|
||
The number of checkers to run in parallel.
|
||
Checkers do the equality checking of files during a sync.
|
||
For some storage systems (eg S3, Swift, Dropbox) this can take a
|
||
significant amount of time so they are run in parallel.
|
||
.PP
|
||
The default is to run 8 checkers in parallel.
|
||
.SS \-c, \-\-checksum
|
||
.PP
|
||
Normally rclone will look at modification time and size of files to see
|
||
if they are equal.
|
||
If you set this flag then rclone will check the file hash and size to
|
||
determine if files are equal.
|
||
.PP
|
||
This is useful when the remote doesn\[aq]t support setting modified time
|
||
and a more accurate sync is desired than just checking the file size.
|
||
.PP
|
||
This is very useful when transferring between remotes which store the
|
||
same hash type on the object, eg Drive and Swift.
|
||
For details of which remotes support which hash type see the table in
|
||
the overview section (https://rclone.org/overview/).
|
||
.PP
|
||
Eg \f[C]rclone\ \-\-checksum\ sync\ s3:/bucket\ swift:/bucket\f[] would
|
||
run much quicker than without the \f[C]\-\-checksum\f[] flag.
|
||
.PP
|
||
When using this flag, rclone won\[aq]t update mtimes of remote files if
|
||
they are incorrect as it would normally.
|
||
.SS \-\-config=CONFIG_FILE
|
||
.PP
|
||
Specify the location of the rclone config file.
|
||
.PP
|
||
Normally the config file is in your home directory as a file called
|
||
\f[C]\&.config/rclone/rclone.conf\f[] (or \f[C]\&.rclone.conf\f[] if
|
||
created with an older version).
|
||
If \f[C]$XDG_CONFIG_HOME\f[] is set it will be at
|
||
\f[C]$XDG_CONFIG_HOME/rclone/rclone.conf\f[]
|
||
.PP
|
||
If you run \f[C]rclone\ \-h\f[] and look at the help for the
|
||
\f[C]\-\-config\f[] option you will see where the default location is
|
||
for you.
|
||
.PP
|
||
Use this flag to override the config location, eg
|
||
\f[C]rclone\ \-\-config=".myconfig"\ .config\f[].
|
||
.SS \-\-contimeout=TIME
|
||
.PP
|
||
Set the connection timeout.
|
||
This should be in go time format which looks like \f[C]5s\f[] for 5
|
||
seconds, \f[C]10m\f[] for 10 minutes, or \f[C]3h30m\f[].
|
||
.PP
|
||
The connection timeout is the amount of time rclone will wait for a
|
||
connection to go through to a remote object storage system.
|
||
It is \f[C]1m\f[] by default.
|
||
.SS \-\-dedupe\-mode MODE
|
||
.PP
|
||
Mode to run dedupe command in.
|
||
One of \f[C]interactive\f[], \f[C]skip\f[], \f[C]first\f[],
|
||
\f[C]newest\f[], \f[C]oldest\f[], \f[C]rename\f[].
|
||
The default is \f[C]interactive\f[].
|
||
See the dedupe command for more information as to what these options
|
||
mean.
|
||
.SS \-\-disable FEATURE,FEATURE,...
|
||
.PP
|
||
This disables a comma separated list of optional features.
|
||
For example to disable server side move and server side copy use:
|
||
.IP
|
||
.nf
|
||
\f[C]
|
||
\-\-disable\ move,copy
|
||
\f[]
|
||
.fi
|
||
.PP
|
||
The features can be put in in any case.
|
||
.PP
|
||
To see a list of which features can be disabled use:
|
||
.IP
|
||
.nf
|
||
\f[C]
|
||
\-\-disable\ help
|
||
\f[]
|
||
.fi
|
||
.PP
|
||
See the overview features (/overview/#features) and optional
|
||
features (/overview/#optional-features) to get an idea of which feature
|
||
does what.
|
||
.PP
|
||
This flag can be useful for debugging and in exceptional circumstances
|
||
(eg Google Drive limiting the total volume of Server Side Copies to
|
||
100GB/day).
|
||
.SS \-n, \-\-dry\-run
|
||
.PP
|
||
Do a trial run with no permanent changes.
|
||
Use this to see what rclone would do without actually doing it.
|
||
Useful when setting up the \f[C]sync\f[] command which deletes files in
|
||
the destination.
|
||
.SS \-\-ignore\-checksum
|
||
.PP
|
||
Normally rclone will check that the checksums of transferred files
|
||
match, and give an error "corrupted on transfer" if they don\[aq]t.
|
||
.PP
|
||
You can use this option to skip that check.
|
||
You should only use it if you have had the "corrupted on transfer" error
|
||
message and you are sure you might want to transfer potentially
|
||
corrupted data.
|
||
.SS \-\-ignore\-existing
|
||
.PP
|
||
Using this option will make rclone unconditionally skip all files that
|
||
exist on the destination, no matter the content of these files.
|
||
.PP
|
||
While this isn\[aq]t a generally recommended option, it can be useful in
|
||
cases where your files change due to encryption.
|
||
However, it cannot correct partial transfers in case a transfer was
|
||
interrupted.
|
||
.SS \-\-ignore\-size
|
||
.PP
|
||
Normally rclone will look at modification time and size of files to see
|
||
if they are equal.
|
||
If you set this flag then rclone will check only the modification time.
|
||
If \f[C]\-\-checksum\f[] is set then it only checks the checksum.
|
||
.PP
|
||
It will also cause rclone to skip verifying the sizes are the same after
|
||
transfer.
|
||
.PP
|
||
This can be useful for transferring files to and from OneDrive which
|
||
occasionally misreports the size of image files (see
|
||
#399 (https://github.com/ncw/rclone/issues/399) for more info).
|
||
.SS \-I, \-\-ignore\-times
|
||
.PP
|
||
Using this option will cause rclone to unconditionally upload all files
|
||
regardless of the state of files on the destination.
|
||
.PP
|
||
Normally rclone would skip any files that have the same modification
|
||
time and are the same size (or have the same checksum if using
|
||
\f[C]\-\-checksum\f[]).
|
||
.SS \-\-immutable
|
||
.PP
|
||
Treat source and destination files as immutable and disallow
|
||
modification.
|
||
.PP
|
||
With this option set, files will be created and deleted as requested,
|
||
but existing files will never be updated.
|
||
If an existing file does not match between the source and destination,
|
||
rclone will give the error
|
||
\f[C]Source\ and\ destination\ exist\ but\ do\ not\ match:\ immutable\ file\ modified\f[].
|
||
.PP
|
||
Note that only commands which transfer files (e.g.
|
||
\f[C]sync\f[], \f[C]copy\f[], \f[C]move\f[]) are affected by this
|
||
behavior, and only modification is disallowed.
|
||
Files may still be deleted explicitly (e.g.
|
||
\f[C]delete\f[], \f[C]purge\f[]) or implicitly (e.g.
|
||
\f[C]sync\f[], \f[C]move\f[]).
|
||
Use \f[C]copy\ \-\-immutable\f[] if it is desired to avoid deletion as
|
||
well as modification.
|
||
.PP
|
||
This can be useful as an additional layer of protection for immutable or
|
||
append\-only data sets (notably backup archives), where modification
|
||
implies corruption and should not be propagated.
|
||
.SS \-\-leave\-root
|
||
.PP
|
||
During rmdirs it will not remove root directory, even if it\[aq]s empty.
|
||
.SS \-\-log\-file=FILE
|
||
.PP
|
||
Log all of rclone\[aq]s output to FILE.
|
||
This is not active by default.
|
||
This can be useful for tracking down problems with syncs in combination
|
||
with the \f[C]\-v\f[] flag.
|
||
See the Logging section (#logging) for more info.
|
||
.SS \-\-log\-level LEVEL
|
||
.PP
|
||
This sets the log level for rclone.
|
||
The default log level is \f[C]NOTICE\f[].
|
||
.PP
|
||
\f[C]DEBUG\f[] is equivalent to \f[C]\-vv\f[].
|
||
It outputs lots of debug info \- useful for bug reports and really
|
||
finding out what rclone is doing.
|
||
.PP
|
||
\f[C]INFO\f[] is equivalent to \f[C]\-v\f[].
|
||
It outputs information about each transfer and prints stats once a
|
||
minute by default.
|
||
.PP
|
||
\f[C]NOTICE\f[] is the default log level if no logging flags are
|
||
supplied.
|
||
It outputs very little when things are working normally.
|
||
It outputs warnings and significant events.
|
||
.PP
|
||
\f[C]ERROR\f[] is equivalent to \f[C]\-q\f[].
|
||
It only outputs error messages.
|
||
.SS \-\-low\-level\-retries NUMBER
|
||
.PP
|
||
This controls the number of low level retries rclone does.
|
||
.PP
|
||
A low level retry is used to retry a failing operation \- typically one
|
||
HTTP request.
|
||
This might be uploading a chunk of a big file for example.
|
||
You will see low level retries in the log with the \f[C]\-v\f[] flag.
|
||
.PP
|
||
This shouldn\[aq]t need to be changed from the default in normal
|
||
operations.
|
||
However, if you get a lot of low level retries you may wish to reduce
|
||
the value so rclone moves on to a high level retry (see the
|
||
\f[C]\-\-retries\f[] flag) quicker.
|
||
.PP
|
||
Disable low level retries with \f[C]\-\-low\-level\-retries\ 1\f[].
|
||
.SS \-\-max\-depth=N
|
||
.PP
|
||
This modifies the recursion depth for all the commands except purge.
|
||
.PP
|
||
So if you do \f[C]rclone\ \-\-max\-depth\ 1\ ls\ remote:path\f[] you
|
||
will see only the files in the top level directory.
|
||
Using \f[C]\-\-max\-depth\ 2\f[] means you will see all the files in
|
||
first two directory levels and so on.
|
||
.PP
|
||
For historical reasons the \f[C]lsd\f[] command defaults to using a
|
||
\f[C]\-\-max\-depth\f[] of 1 \- you can override this with the command
|
||
line flag.
|
||
.PP
|
||
You can use this command to disable recursion (with
|
||
\f[C]\-\-max\-depth\ 1\f[]).
|
||
.PP
|
||
Note that if you use this with \f[C]sync\f[] and
|
||
\f[C]\-\-delete\-excluded\f[] the files not recursed through are
|
||
considered excluded and will be deleted on the destination.
|
||
Test first with \f[C]\-\-dry\-run\f[] if you are not sure what will
|
||
happen.
|
||
.SS \-\-modify\-window=TIME
|
||
.PP
|
||
When checking whether a file has been modified, this is the maximum
|
||
allowed time difference that a file can have and still be considered
|
||
equivalent.
|
||
.PP
|
||
The default is \f[C]1ns\f[] unless this is overridden by a remote.
|
||
For example OS X only stores modification times to the nearest second so
|
||
if you are reading and writing to an OS X filing system this will be
|
||
\f[C]1s\f[] by default.
|
||
.PP
|
||
This command line flag allows you to override that computed default.
|
||
.SS \-\-no\-gzip\-encoding
|
||
.PP
|
||
Don\[aq]t set \f[C]Accept\-Encoding:\ gzip\f[].
|
||
This means that rclone won\[aq]t ask the server for compressed files
|
||
automatically.
|
||
Useful if you\[aq]ve set the server to return files with
|
||
\f[C]Content\-Encoding:\ gzip\f[] but you uploaded compressed files.
|
||
.PP
|
||
There is no need to set this in normal operation, and doing so will
|
||
decrease the network transfer efficiency of rclone.
|
||
.SS \-\-no\-update\-modtime
|
||
.PP
|
||
When using this flag, rclone won\[aq]t update modification times of
|
||
remote files if they are incorrect as it would normally.
|
||
.PP
|
||
This can be used if the remote is being synced with another tool also
|
||
(eg the Google Drive client).
|
||
.SS \-q, \-\-quiet
|
||
.PP
|
||
Normally rclone outputs stats and a completion message.
|
||
If you set this flag it will make as little output as possible.
|
||
.SS \-\-retries int
|
||
.PP
|
||
Retry the entire sync if it fails this many times it fails (default 3).
|
||
.PP
|
||
Some remotes can be unreliable and a few retries help pick up the files
|
||
which didn\[aq]t get transferred because of errors.
|
||
.PP
|
||
Disable retries with \f[C]\-\-retries\ 1\f[].
|
||
.SS \-\-size\-only
|
||
.PP
|
||
Normally rclone will look at modification time and size of files to see
|
||
if they are equal.
|
||
If you set this flag then rclone will check only the size.
|
||
.PP
|
||
This can be useful transferring files from Dropbox which have been
|
||
modified by the desktop sync client which doesn\[aq]t set checksums of
|
||
modification times in the same way as rclone.
|
||
.SS \-\-stats=TIME
|
||
.PP
|
||
Commands which transfer data (\f[C]sync\f[], \f[C]copy\f[],
|
||
\f[C]copyto\f[], \f[C]move\f[], \f[C]moveto\f[]) will print data
|
||
transfer stats at regular intervals to show their progress.
|
||
.PP
|
||
This sets the interval.
|
||
.PP
|
||
The default is \f[C]1m\f[].
|
||
Use 0 to disable.
|
||
.PP
|
||
If you set the stats interval then all commands can show stats.
|
||
This can be useful when running other commands, \f[C]check\f[] or
|
||
\f[C]mount\f[] for example.
|
||
.PP
|
||
Stats are logged at \f[C]INFO\f[] level by default which means they
|
||
won\[aq]t show at default log level \f[C]NOTICE\f[].
|
||
Use \f[C]\-\-stats\-log\-level\ NOTICE\f[] or \f[C]\-v\f[] to make them
|
||
show.
|
||
See the Logging section (#logging) for more info on log levels.
|
||
.SS \-\-stats\-log\-level string
|
||
.PP
|
||
Log level to show \f[C]\-\-stats\f[] output at.
|
||
This can be \f[C]DEBUG\f[], \f[C]INFO\f[], \f[C]NOTICE\f[], or
|
||
\f[C]ERROR\f[].
|
||
The default is \f[C]INFO\f[].
|
||
This means at the default level of logging which is \f[C]NOTICE\f[] the
|
||
stats won\[aq]t show \- if you want them to then use
|
||
\f[C]\-stats\-log\-level\ NOTICE\f[].
|
||
See the Logging section (#logging) for more info on log levels.
|
||
.SS \-\-stats\-unit=bits|bytes
|
||
.PP
|
||
By default, data transfer rates will be printed in bytes/second.
|
||
.PP
|
||
This option allows the data rate to be printed in bits/second.
|
||
.PP
|
||
Data transfer volume will still be reported in bytes.
|
||
.PP
|
||
The rate is reported as a binary unit, not SI unit.
|
||
So 1 Mbit/s equals 1,048,576 bits/s and not 1,000,000 bits/s.
|
||
.PP
|
||
The default is \f[C]bytes\f[].
|
||
.SS \-\-suffix=SUFFIX
|
||
.PP
|
||
This is for use with \f[C]\-\-backup\-dir\f[] only.
|
||
If this isn\[aq]t set then \f[C]\-\-backup\-dir\f[] will move files with
|
||
their original name.
|
||
If it is set then the files will have SUFFIX added on to them.
|
||
.PP
|
||
See \f[C]\-\-backup\-dir\f[] for more info.
|
||
.SS \-\-syslog
|
||
.PP
|
||
On capable OSes (not Windows or Plan9) send all log output to syslog.
|
||
.PP
|
||
This can be useful for running rclone in a script or
|
||
\f[C]rclone\ mount\f[].
|
||
.SS \-\-syslog\-facility string
|
||
.PP
|
||
If using \f[C]\-\-syslog\f[] this sets the syslog facility (eg
|
||
\f[C]KERN\f[], \f[C]USER\f[]).
|
||
See \f[C]man\ syslog\f[] for a list of possible facilities.
|
||
The default facility is \f[C]DAEMON\f[].
|
||
.SS \-\-tpslimit float
|
||
.PP
|
||
Limit HTTP transactions per second to this.
|
||
Default is 0 which is used to mean unlimited transactions per second.
|
||
.PP
|
||
For example to limit rclone to 10 HTTP transactions per second use
|
||
\f[C]\-\-tpslimit\ 10\f[], or to 1 transaction every 2 seconds use
|
||
\f[C]\-\-tpslimit\ 0.5\f[].
|
||
.PP
|
||
Use this when the number of transactions per second from rclone is
|
||
causing a problem with the cloud storage provider (eg getting you banned
|
||
or rate limited).
|
||
.PP
|
||
This can be very useful for \f[C]rclone\ mount\f[] to control the
|
||
behaviour of applications using it.
|
||
.PP
|
||
See also \f[C]\-\-tpslimit\-burst\f[].
|
||
.SS \-\-tpslimit\-burst int
|
||
.PP
|
||
Max burst of transactions for \f[C]\-\-tpslimit\f[].
|
||
(default 1)
|
||
.PP
|
||
Normally \f[C]\-\-tpslimit\f[] will do exactly the number of transaction
|
||
per second specified.
|
||
However if you supply \f[C]\-\-tps\-burst\f[] then rclone can save up
|
||
some transactions from when it was idle giving a burst of up to the
|
||
parameter supplied.
|
||
.PP
|
||
For example if you provide \f[C]\-\-tpslimit\-burst\ 10\f[] then if
|
||
rclone has been idle for more than 10*\f[C]\-\-tpslimit\f[] then it can
|
||
do 10 transactions very quickly before they are limited again.
|
||
.PP
|
||
This may be used to increase performance of \f[C]\-\-tpslimit\f[]
|
||
without changing the long term average number of transactions per
|
||
second.
|
||
.SS \-\-track\-renames
|
||
.PP
|
||
By default, rclone doesn\[aq]t keep track of renamed files, so if you
|
||
rename a file locally then sync it to a remote, rclone will delete the
|
||
old file on the remote and upload a new copy.
|
||
.PP
|
||
If you use this flag, and the remote supports server side copy or server
|
||
side move, and the source and destination have a compatible hash, then
|
||
this will track renames during \f[C]sync\f[], \f[C]copy\f[], and
|
||
\f[C]move\f[] operations and perform renaming server\-side.
|
||
.PP
|
||
Files will be matched by size and hash \- if both match then a rename
|
||
will be considered.
|
||
.PP
|
||
If the destination does not support server\-side copy or move, rclone
|
||
will fall back to the default behaviour and log an error level message
|
||
to the console.
|
||
.PP
|
||
Note that \f[C]\-\-track\-renames\f[] is incompatible with
|
||
\f[C]\-\-no\-traverse\f[] and that it uses extra memory to keep track of
|
||
all the rename candidates.
|
||
.PP
|
||
Note also that \f[C]\-\-track\-renames\f[] is incompatible with
|
||
\f[C]\-\-delete\-before\f[] and will select \f[C]\-\-delete\-after\f[]
|
||
instead of \f[C]\-\-delete\-during\f[].
|
||
.SS \-\-delete\-(before,during,after)
|
||
.PP
|
||
This option allows you to specify when files on your destination are
|
||
deleted when you sync folders.
|
||
.PP
|
||
Specifying the value \f[C]\-\-delete\-before\f[] will delete all files
|
||
present on the destination, but not on the source \f[I]before\f[]
|
||
starting the transfer of any new or updated files.
|
||
This uses two passes through the file systems, one for the deletions and
|
||
one for the copies.
|
||
.PP
|
||
Specifying \f[C]\-\-delete\-during\f[] will delete files while checking
|
||
and uploading files.
|
||
This is the fastest option and uses the least memory.
|
||
.PP
|
||
Specifying \f[C]\-\-delete\-after\f[] (the default value) will delay
|
||
deletion of files until all new/updated files have been successfully
|
||
transferred.
|
||
The files to be deleted are collected in the copy pass then deleted
|
||
after the copy pass has completed successfully.
|
||
The files to be deleted are held in memory so this mode may use more
|
||
memory.
|
||
This is the safest mode as it will only delete files if there have been
|
||
no errors subsequent to that.
|
||
If there have been errors before the deletions start then you will get
|
||
the message \f[C]not\ deleting\ files\ as\ there\ were\ IO\ errors\f[].
|
||
.SS \-\-fast\-list
|
||
.PP
|
||
When doing anything which involves a directory listing (eg
|
||
\f[C]sync\f[], \f[C]copy\f[], \f[C]ls\f[] \- in fact nearly every
|
||
command), rclone normally lists a directory and processes it before
|
||
using more directory lists to process any subdirectories.
|
||
This can be parallelised and works very quickly using the least amount
|
||
of memory.
|
||
.PP
|
||
However, some remotes have a way of listing all files beneath a
|
||
directory in one (or a small number) of transactions.
|
||
These tend to be the bucket based remotes (eg S3, B2, GCS, Swift,
|
||
Hubic).
|
||
.PP
|
||
If you use the \f[C]\-\-fast\-list\f[] flag then rclone will use this
|
||
method for listing directories.
|
||
This will have the following consequences for the listing:
|
||
.IP \[bu] 2
|
||
It \f[B]will\f[] use fewer transactions (important if you pay for them)
|
||
.IP \[bu] 2
|
||
It \f[B]will\f[] use more memory.
|
||
Rclone has to load the whole listing into memory.
|
||
.IP \[bu] 2
|
||
It \f[I]may\f[] be faster because it uses fewer transactions
|
||
.IP \[bu] 2
|
||
It \f[I]may\f[] be slower because it can\[aq]t be parallelized
|
||
.PP
|
||
rclone should always give identical results with and without
|
||
\f[C]\-\-fast\-list\f[].
|
||
.PP
|
||
If you pay for transactions and can fit your entire sync listing into
|
||
memory then \f[C]\-\-fast\-list\f[] is recommended.
|
||
If you have a very big sync to do then don\[aq]t use
|
||
\f[C]\-\-fast\-list\f[] otherwise you will run out of memory.
|
||
.PP
|
||
If you use \f[C]\-\-fast\-list\f[] on a remote which doesn\[aq]t support
|
||
it, then rclone will just ignore it.
|
||
.SS \-\-timeout=TIME
|
||
.PP
|
||
This sets the IO idle timeout.
|
||
If a transfer has started but then becomes idle for this long it is
|
||
considered broken and disconnected.
|
||
.PP
|
||
The default is \f[C]5m\f[].
|
||
Set to 0 to disable.
|
||
.SS \-\-transfers=N
|
||
.PP
|
||
The number of file transfers to run in parallel.
|
||
It can sometimes be useful to set this to a smaller number if the remote
|
||
is giving a lot of timeouts or bigger if you have lots of bandwidth and
|
||
a fast remote.
|
||
.PP
|
||
The default is to run 4 file transfers in parallel.
|
||
.SS \-u, \-\-update
|
||
.PP
|
||
This forces rclone to skip any files which exist on the destination and
|
||
have a modified time that is newer than the source file.
|
||
.PP
|
||
If an existing destination file has a modification time equal (within
|
||
the computed modify window precision) to the source file\[aq]s, it will
|
||
be updated if the sizes are different.
|
||
.PP
|
||
On remotes which don\[aq]t support mod time directly the time checked
|
||
will be the uploaded time.
|
||
This means that if uploading to one of these remotes, rclone will skip
|
||
any files which exist on the destination and have an uploaded time that
|
||
is newer than the modification time of the source file.
|
||
.PP
|
||
This can be useful when transferring to a remote which doesn\[aq]t
|
||
support mod times directly as it is more accurate than a
|
||
\f[C]\-\-size\-only\f[] check and faster than using
|
||
\f[C]\-\-checksum\f[].
|
||
.SS \-v, \-vv, \-\-verbose
|
||
.PP
|
||
With \f[C]\-v\f[] rclone will tell you about each file that is
|
||
transferred and a small number of significant events.
|
||
.PP
|
||
With \f[C]\-vv\f[] rclone will become very verbose telling you about
|
||
every file it considers and transfers.
|
||
Please send bug reports with a log with this setting.
|
||
.SS \-V, \-\-version
|
||
.PP
|
||
Prints the version number
|
||
.SS Configuration Encryption
|
||
.PP
|
||
Your configuration file contains information for logging in to your
|
||
cloud services.
|
||
This means that you should keep your \f[C]\&.rclone.conf\f[] file in a
|
||
secure location.
|
||
.PP
|
||
If you are in an environment where that isn\[aq]t possible, you can add
|
||
a password to your configuration.
|
||
This means that you will have to enter the password every time you start
|
||
rclone.
|
||
.PP
|
||
To add a password to your rclone configuration, execute
|
||
\f[C]rclone\ config\f[].
|
||
.IP
|
||
.nf
|
||
\f[C]
|
||
>rclone\ config
|
||
Current\ remotes:
|
||
|
||
e)\ Edit\ existing\ remote
|
||
n)\ New\ remote
|
||
d)\ Delete\ remote
|
||
s)\ Set\ configuration\ password
|
||
q)\ Quit\ config
|
||
e/n/d/s/q>
|
||
\f[]
|
||
.fi
|
||
.PP
|
||
Go into \f[C]s\f[], Set configuration password:
|
||
.IP
|
||
.nf
|
||
\f[C]
|
||
e/n/d/s/q>\ s
|
||
Your\ configuration\ is\ not\ encrypted.
|
||
If\ you\ add\ a\ password,\ you\ will\ protect\ your\ login\ information\ to\ cloud\ services.
|
||
a)\ Add\ Password
|
||
q)\ Quit\ to\ main\ menu
|
||
a/q>\ a
|
||
Enter\ NEW\ configuration\ password:
|
||
password:
|
||
Confirm\ NEW\ password:
|
||
password:
|
||
Password\ set
|
||
Your\ configuration\ is\ encrypted.
|
||
c)\ Change\ Password
|
||
u)\ Unencrypt\ configuration
|
||
q)\ Quit\ to\ main\ menu
|
||
c/u/q>
|
||
\f[]
|
||
.fi
|
||
.PP
|
||
Your configuration is now encrypted, and every time you start rclone you
|
||
will now be asked for the password.
|
||
In the same menu, you can change the password or completely remove
|
||
encryption from your configuration.
|
||
.PP
|
||
There is no way to recover the configuration if you lose your password.
|
||
.PP
|
||
rclone uses nacl
|
||
secretbox (https://godoc.org/golang.org/x/crypto/nacl/secretbox) which
|
||
in turn uses XSalsa20 and Poly1305 to encrypt and authenticate your
|
||
configuration with secret\-key cryptography.
|
||
The password is SHA\-256 hashed, which produces the key for secretbox.
|
||
The hashed password is not stored.
|
||
.PP
|
||
While this provides very good security, we do not recommend storing your
|
||
encrypted rclone configuration in public if it contains sensitive
|
||
information, maybe except if you use a very strong password.
|
||
.PP
|
||
If it is safe in your environment, you can set the
|
||
\f[C]RCLONE_CONFIG_PASS\f[] environment variable to contain your
|
||
password, in which case it will be used for decrypting the
|
||
configuration.
|
||
.PP
|
||
You can set this for a session from a script.
|
||
For unix like systems save this to a file called
|
||
\f[C]set\-rclone\-password\f[]:
|
||
.IP
|
||
.nf
|
||
\f[C]
|
||
#!/bin/echo\ Source\ this\ file\ don\[aq]t\ run\ it
|
||
|
||
read\ \-s\ RCLONE_CONFIG_PASS
|
||
export\ RCLONE_CONFIG_PASS
|
||
\f[]
|
||
.fi
|
||
.PP
|
||
Then source the file when you want to use it.
|
||
From the shell you would do \f[C]source\ set\-rclone\-password\f[].
|
||
It will then ask you for the password and set it in the environment
|
||
variable.
|
||
.PP
|
||
If you are running rclone inside a script, you might want to disable
|
||
password prompts.
|
||
To do that, pass the parameter \f[C]\-\-ask\-password=false\f[] to
|
||
rclone.
|
||
This will make rclone fail instead of asking for a password if
|
||
\f[C]RCLONE_CONFIG_PASS\f[] doesn\[aq]t contain a valid password.
|
||
.SS Developer options
|
||
.PP
|
||
These options are useful when developing or debugging rclone.
|
||
There are also some more remote specific options which aren\[aq]t
|
||
documented here which are used for testing.
|
||
These start with remote name eg \f[C]\-\-drive\-test\-option\f[] \- see
|
||
the docs for the remote in question.
|
||
.SS \-\-cpuprofile=FILE
|
||
.PP
|
||
Write CPU profile to file.
|
||
This can be analysed with \f[C]go\ tool\ pprof\f[].
|
||
.SS \-\-dump flag,flag,flag
|
||
.PP
|
||
The \f[C]\-\-dump\f[] flag takes a comma separated list of flags to dump
|
||
info about.
|
||
These are:
|
||
.SS \-\-dump headers
|
||
.PP
|
||
Dump HTTP headers with \f[C]Authorization:\f[] lines removed.
|
||
May still contain sensitive info.
|
||
Can be very verbose.
|
||
Useful for debugging only.
|
||
.PP
|
||
Use \f[C]\-\-dump\ auth\f[] if you do want the \f[C]Authorization:\f[]
|
||
headers.
|
||
.SS \-\-dump bodies
|
||
.PP
|
||
Dump HTTP headers and bodies \- may contain sensitive info.
|
||
Can be very verbose.
|
||
Useful for debugging only.
|
||
.PP
|
||
Note that the bodies are buffered in memory so don\[aq]t use this for
|
||
enormous files.
|
||
.SS \-\-dump requests
|
||
.PP
|
||
Like \f[C]\-\-dump\ bodies\f[] but dumps the request bodies and the
|
||
response headers.
|
||
Useful for debugging download problems.
|
||
.SS \-\-dump responses
|
||
.PP
|
||
Like \f[C]\-\-dump\ bodies\f[] but dumps the response bodies and the
|
||
request headers.
|
||
Useful for debugging upload problems.
|
||
.SS \-\-dump auth
|
||
.PP
|
||
Dump HTTP headers \- will contain sensitive info such as
|
||
\f[C]Authorization:\f[] headers \- use \f[C]\-\-dump\ headers\f[] to
|
||
dump without \f[C]Authorization:\f[] headers.
|
||
Can be very verbose.
|
||
Useful for debugging only.
|
||
.SS \-\-dump filters
|
||
.PP
|
||
Dump the filters to the output.
|
||
Useful to see exactly what include and exclude options are filtering on.
|
||
.SS \-\-memprofile=FILE
|
||
.PP
|
||
Write memory profile to file.
|
||
This can be analysed with \f[C]go\ tool\ pprof\f[].
|
||
.SS \-\-no\-check\-certificate=true/false
|
||
.PP
|
||
\f[C]\-\-no\-check\-certificate\f[] controls whether a client verifies
|
||
the server\[aq]s certificate chain and host name.
|
||
If \f[C]\-\-no\-check\-certificate\f[] is true, TLS accepts any
|
||
certificate presented by the server and any host name in that
|
||
certificate.
|
||
In this mode, TLS is susceptible to man\-in\-the\-middle attacks.
|
||
.PP
|
||
This option defaults to \f[C]false\f[].
|
||
.PP
|
||
\f[B]This should be used only for testing.\f[]
|
||
.SS \-\-no\-traverse
|
||
.PP
|
||
The \f[C]\-\-no\-traverse\f[] flag controls whether the destination file
|
||
system is traversed when using the \f[C]copy\f[] or \f[C]move\f[]
|
||
commands.
|
||
\f[C]\-\-no\-traverse\f[] is not compatible with \f[C]sync\f[] and will
|
||
be ignored if you supply it with \f[C]sync\f[].
|
||
.PP
|
||
If you are only copying a small number of files and/or have a large
|
||
number of files on the destination then \f[C]\-\-no\-traverse\f[] will
|
||
stop rclone listing the destination and save time.
|
||
.PP
|
||
However, if you are copying a large number of files, especially if you
|
||
are doing a copy where lots of the files haven\[aq]t changed and
|
||
won\[aq]t need copying then you shouldn\[aq]t use
|
||
\f[C]\-\-no\-traverse\f[].
|
||
.PP
|
||
It can also be used to reduce the memory usage of rclone when copying \-
|
||
\f[C]rclone\ \-\-no\-traverse\ copy\ src\ dst\f[] won\[aq]t load either
|
||
the source or destination listings into memory so will use the minimum
|
||
amount of memory.
|
||
.SS Filtering
|
||
.PP
|
||
For the filtering options
|
||
.IP \[bu] 2
|
||
\f[C]\-\-delete\-excluded\f[]
|
||
.IP \[bu] 2
|
||
\f[C]\-\-filter\f[]
|
||
.IP \[bu] 2
|
||
\f[C]\-\-filter\-from\f[]
|
||
.IP \[bu] 2
|
||
\f[C]\-\-exclude\f[]
|
||
.IP \[bu] 2
|
||
\f[C]\-\-exclude\-from\f[]
|
||
.IP \[bu] 2
|
||
\f[C]\-\-include\f[]
|
||
.IP \[bu] 2
|
||
\f[C]\-\-include\-from\f[]
|
||
.IP \[bu] 2
|
||
\f[C]\-\-files\-from\f[]
|
||
.IP \[bu] 2
|
||
\f[C]\-\-min\-size\f[]
|
||
.IP \[bu] 2
|
||
\f[C]\-\-max\-size\f[]
|
||
.IP \[bu] 2
|
||
\f[C]\-\-min\-age\f[]
|
||
.IP \[bu] 2
|
||
\f[C]\-\-max\-age\f[]
|
||
.IP \[bu] 2
|
||
\f[C]\-\-dump\ filters\f[]
|
||
.PP
|
||
See the filtering section (https://rclone.org/filtering/).
|
||
.SS Logging
|
||
.PP
|
||
rclone has 4 levels of logging, \f[C]Error\f[], \f[C]Notice\f[],
|
||
\f[C]Info\f[] and \f[C]Debug\f[].
|
||
.PP
|
||
By default, rclone logs to standard error.
|
||
This means you can redirect standard error and still see the normal
|
||
output of rclone commands (eg \f[C]rclone\ ls\f[]).
|
||
.PP
|
||
By default, rclone will produce \f[C]Error\f[] and \f[C]Notice\f[] level
|
||
messages.
|
||
.PP
|
||
If you use the \f[C]\-q\f[] flag, rclone will only produce
|
||
\f[C]Error\f[] messages.
|
||
.PP
|
||
If you use the \f[C]\-v\f[] flag, rclone will produce \f[C]Error\f[],
|
||
\f[C]Notice\f[] and \f[C]Info\f[] messages.
|
||
.PP
|
||
If you use the \f[C]\-vv\f[] flag, rclone will produce \f[C]Error\f[],
|
||
\f[C]Notice\f[], \f[C]Info\f[] and \f[C]Debug\f[] messages.
|
||
.PP
|
||
You can also control the log levels with the \f[C]\-\-log\-level\f[]
|
||
flag.
|
||
.PP
|
||
If you use the \f[C]\-\-log\-file=FILE\f[] option, rclone will redirect
|
||
\f[C]Error\f[], \f[C]Info\f[] and \f[C]Debug\f[] messages along with
|
||
standard error to FILE.
|
||
.PP
|
||
If you use the \f[C]\-\-syslog\f[] flag then rclone will log to syslog
|
||
and the \f[C]\-\-syslog\-facility\f[] control which facility it uses.
|
||
.PP
|
||
Rclone prefixes all log messages with their level in capitals, eg INFO
|
||
which makes it easy to grep the log file for different kinds of
|
||
information.
|
||
.SS Exit Code
|
||
.PP
|
||
If any errors occur during the command execution, rclone will exit with
|
||
a non\-zero exit code.
|
||
This allows scripts to detect when rclone operations have failed.
|
||
.PP
|
||
During the startup phase, rclone will exit immediately if an error is
|
||
detected in the configuration.
|
||
There will always be a log message immediately before exiting.
|
||
.PP
|
||
When rclone is running it will accumulate errors as it goes along, and
|
||
only exit with a non\-zero exit code if (after retries) there were still
|
||
failed transfers.
|
||
For every error counted there will be a high priority log message
|
||
(visible with \f[C]\-q\f[]) showing the message and which file caused
|
||
the problem.
|
||
A high priority message is also shown when starting a retry so the user
|
||
can see that any previous error messages may not be valid after the
|
||
retry.
|
||
If rclone has done a retry it will log a high priority message if the
|
||
retry was successful.
|
||
.SS List of exit codes
|
||
.IP \[bu] 2
|
||
\f[C]0\f[] \- success
|
||
.IP \[bu] 2
|
||
\f[C]1\f[] \- Syntax or usage error
|
||
.IP \[bu] 2
|
||
\f[C]2\f[] \- Error not otherwise categorised
|
||
.IP \[bu] 2
|
||
\f[C]3\f[] \- Directory not found
|
||
.IP \[bu] 2
|
||
\f[C]4\f[] \- File not found
|
||
.IP \[bu] 2
|
||
\f[C]5\f[] \- Temporary error (one that more retries might fix) (Retry
|
||
errors)
|
||
.IP \[bu] 2
|
||
\f[C]6\f[] \- Less serious errors (like 461 errors from dropbox)
|
||
(NoRetry errors)
|
||
.IP \[bu] 2
|
||
\f[C]7\f[] \- Fatal error (one that more retries won\[aq]t fix, like
|
||
account suspended) (Fatal errors)
|
||
.SS Environment Variables
|
||
.PP
|
||
Rclone can be configured entirely using environment variables.
|
||
These can be used to set defaults for options or config file entries.
|
||
.SS Options
|
||
.PP
|
||
Every option in rclone can have its default set by environment variable.
|
||
.PP
|
||
To find the name of the environment variable, first, take the long
|
||
option name, strip the leading \f[C]\-\-\f[], change \f[C]\-\f[] to
|
||
\f[C]_\f[], make upper case and prepend \f[C]RCLONE_\f[].
|
||
.PP
|
||
For example, to always set \f[C]\-\-stats\ 5s\f[], set the environment
|
||
variable \f[C]RCLONE_STATS=5s\f[].
|
||
If you set stats on the command line this will override the environment
|
||
variable setting.
|
||
.PP
|
||
Or to always use the trash in drive \f[C]\-\-drive\-use\-trash\f[], set
|
||
\f[C]RCLONE_DRIVE_USE_TRASH=true\f[].
|
||
.PP
|
||
The same parser is used for the options and the environment variables so
|
||
they take exactly the same form.
|
||
.SS Config file
|
||
.PP
|
||
You can set defaults for values in the config file on an individual
|
||
remote basis.
|
||
If you want to use this feature, you will need to discover the name of
|
||
the config items that you want.
|
||
The easiest way is to run through \f[C]rclone\ config\f[] by hand, then
|
||
look in the config file to see what the values are (the config file can
|
||
be found by looking at the help for \f[C]\-\-config\f[] in
|
||
\f[C]rclone\ help\f[]).
|
||
.PP
|
||
To find the name of the environment variable, you need to set, take
|
||
\f[C]RCLONE_CONFIG_\f[] + name of remote + \f[C]_\f[] + name of config
|
||
file option and make it all uppercase.
|
||
.PP
|
||
For example, to configure an S3 remote named \f[C]mys3:\f[] without a
|
||
config file (using unix ways of setting environment variables):
|
||
.IP
|
||
.nf
|
||
\f[C]
|
||
$\ export\ RCLONE_CONFIG_MYS3_TYPE=s3
|
||
$\ export\ RCLONE_CONFIG_MYS3_ACCESS_KEY_ID=XXX
|
||
$\ export\ RCLONE_CONFIG_MYS3_SECRET_ACCESS_KEY=XXX
|
||
$\ rclone\ lsd\ MYS3:
|
||
\ \ \ \ \ \ \ \ \ \ \-1\ 2016\-09\-21\ 12:54:21\ \ \ \ \ \ \ \ \-1\ my\-bucket
|
||
$\ rclone\ listremotes\ |\ grep\ mys3
|
||
mys3:
|
||
\f[]
|
||
.fi
|
||
.PP
|
||
Note that if you want to create a remote using environment variables you
|
||
must create the \f[C]\&..._TYPE\f[] variable as above.
|
||
.SS Other environment variables
|
||
.IP \[bu] 2
|
||
RCLONE_CONFIG_PASS` set to contain your config file password (see
|
||
Configuration Encryption (#configuration-encryption) section)
|
||
.IP \[bu] 2
|
||
HTTP_PROXY, HTTPS_PROXY and NO_PROXY (or the lowercase versions
|
||
thereof).
|
||
.RS 2
|
||
.IP \[bu] 2
|
||
HTTPS_PROXY takes precedence over HTTP_PROXY for https requests.
|
||
.IP \[bu] 2
|
||
The environment values may be either a complete URL or a "host[:port]"
|
||
for, in which case the "http" scheme is assumed.
|
||
.RE
|
||
.SH Configuring rclone on a remote / headless machine
|
||
.PP
|
||
Some of the configurations (those involving oauth2) require an Internet
|
||
connected web browser.
|
||
.PP
|
||
If you are trying to set rclone up on a remote or headless box with no
|
||
browser available on it (eg a NAS or a server in a datacenter) then you
|
||
will need to use an alternative means of configuration.
|
||
There are two ways of doing it, described below.
|
||
.SS Configuring using rclone authorize
|
||
.PP
|
||
On the headless box
|
||
.IP
|
||
.nf
|
||
\f[C]
|
||
\&...
|
||
Remote\ config
|
||
Use\ auto\ config?
|
||
\ *\ Say\ Y\ if\ not\ sure
|
||
\ *\ Say\ N\ if\ you\ are\ working\ on\ a\ remote\ or\ headless\ machine
|
||
y)\ Yes
|
||
n)\ No
|
||
y/n>\ n
|
||
For\ this\ to\ work,\ you\ will\ need\ rclone\ available\ on\ a\ machine\ that\ has\ a\ web\ browser\ available.
|
||
Execute\ the\ following\ on\ your\ machine:
|
||
\ \ \ \ rclone\ authorize\ "amazon\ cloud\ drive"
|
||
Then\ paste\ the\ result\ below:
|
||
result>
|
||
\f[]
|
||
.fi
|
||
.PP
|
||
Then on your main desktop machine
|
||
.IP
|
||
.nf
|
||
\f[C]
|
||
rclone\ authorize\ "amazon\ cloud\ drive"
|
||
If\ your\ browser\ doesn\[aq]t\ open\ automatically\ go\ to\ the\ following\ link:\ http://127.0.0.1:53682/auth
|
||
Log\ in\ and\ authorize\ rclone\ for\ access
|
||
Waiting\ for\ code...
|
||
Got\ code
|
||
Paste\ the\ following\ into\ your\ remote\ machine\ \-\-\->
|
||
SECRET_TOKEN
|
||
<\-\-\-End\ paste
|
||
\f[]
|
||
.fi
|
||
.PP
|
||
Then back to the headless box, paste in the code
|
||
.IP
|
||
.nf
|
||
\f[C]
|
||
result>\ SECRET_TOKEN
|
||
\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-
|
||
[acd12]
|
||
client_id\ =\
|
||
client_secret\ =\
|
||
token\ =\ SECRET_TOKEN
|
||
\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-
|
||
y)\ Yes\ this\ is\ OK
|
||
e)\ Edit\ this\ remote
|
||
d)\ Delete\ this\ remote
|
||
y/e/d>
|
||
\f[]
|
||
.fi
|
||
.SS Configuring by copying the config file
|
||
.PP
|
||
Rclone stores all of its config in a single configuration file.
|
||
This can easily be copied to configure a remote rclone.
|
||
.PP
|
||
So first configure rclone on your desktop machine
|
||
.IP
|
||
.nf
|
||
\f[C]
|
||
rclone\ config
|
||
\f[]
|
||
.fi
|
||
.PP
|
||
to set up the config file.
|
||
.PP
|
||
Find the config file by running \f[C]rclone\ \-h\f[] and looking for the
|
||
help for the \f[C]\-\-config\f[] option
|
||
.IP
|
||
.nf
|
||
\f[C]
|
||
$\ rclone\ \-h
|
||
[snip]
|
||
\ \ \ \ \ \ \-\-config="/home/user/.rclone.conf":\ Config\ file.
|
||
[snip]
|
||
\f[]
|
||
.fi
|
||
.PP
|
||
Now transfer it to the remote box (scp, cut paste, ftp, sftp etc) and
|
||
place it in the correct place (use \f[C]rclone\ \-h\f[] on the remote
|
||
box to find out where).
|
||
.SH Filtering, includes and excludes
|
||
.PP
|
||
Rclone has a sophisticated set of include and exclude rules.
|
||
Some of these are based on patterns and some on other things like file
|
||
size.
|
||
.PP
|
||
The filters are applied for the \f[C]copy\f[], \f[C]sync\f[],
|
||
\f[C]move\f[], \f[C]ls\f[], \f[C]lsl\f[], \f[C]md5sum\f[],
|
||
\f[C]sha1sum\f[], \f[C]size\f[], \f[C]delete\f[] and \f[C]check\f[]
|
||
operations.
|
||
Note that \f[C]purge\f[] does not obey the filters.
|
||
.PP
|
||
Each path as it passes through rclone is matched against the include and
|
||
exclude rules like \f[C]\-\-include\f[], \f[C]\-\-exclude\f[],
|
||
\f[C]\-\-include\-from\f[], \f[C]\-\-exclude\-from\f[],
|
||
\f[C]\-\-filter\f[], or \f[C]\-\-filter\-from\f[].
|
||
The simplest way to try them out is using the \f[C]ls\f[] command, or
|
||
\f[C]\-\-dry\-run\f[] together with \f[C]\-v\f[].
|
||
.SS Patterns
|
||
.PP
|
||
The patterns used to match files for inclusion or exclusion are based on
|
||
"file globs" as used by the unix shell.
|
||
.PP
|
||
If the pattern starts with a \f[C]/\f[] then it only matches at the top
|
||
level of the directory tree, \f[B]relative to the root of the remote\f[]
|
||
(not necessarily the root of the local drive).
|
||
If it doesn\[aq]t start with \f[C]/\f[] then it is matched starting at
|
||
the \f[B]end of the path\f[], but it will only match a complete path
|
||
element:
|
||
.IP
|
||
.nf
|
||
\f[C]
|
||
file.jpg\ \ \-\ matches\ "file.jpg"
|
||
\ \ \ \ \ \ \ \ \ \ \-\ matches\ "directory/file.jpg"
|
||
\ \ \ \ \ \ \ \ \ \ \-\ doesn\[aq]t\ match\ "afile.jpg"
|
||
\ \ \ \ \ \ \ \ \ \ \-\ doesn\[aq]t\ match\ "directory/afile.jpg"
|
||
/file.jpg\ \-\ matches\ "file.jpg"\ in\ the\ root\ directory\ of\ the\ remote
|
||
\ \ \ \ \ \ \ \ \ \ \-\ doesn\[aq]t\ match\ "afile.jpg"
|
||
\ \ \ \ \ \ \ \ \ \ \-\ doesn\[aq]t\ match\ "directory/file.jpg"
|
||
\f[]
|
||
.fi
|
||
.PP
|
||
\f[B]Important\f[] Note that you must use \f[C]/\f[] in patterns and not
|
||
\f[C]\\\f[] even if running on Windows.
|
||
.PP
|
||
A \f[C]*\f[] matches anything but not a \f[C]/\f[].
|
||
.IP
|
||
.nf
|
||
\f[C]
|
||
*.jpg\ \ \-\ matches\ "file.jpg"
|
||
\ \ \ \ \ \ \ \-\ matches\ "directory/file.jpg"
|
||
\ \ \ \ \ \ \ \-\ doesn\[aq]t\ match\ "file.jpg/something"
|
||
\f[]
|
||
.fi
|
||
.PP
|
||
Use \f[C]**\f[] to match anything, including slashes (\f[C]/\f[]).
|
||
.IP
|
||
.nf
|
||
\f[C]
|
||
dir/**\ \-\ matches\ "dir/file.jpg"
|
||
\ \ \ \ \ \ \ \-\ matches\ "dir/dir1/dir2/file.jpg"
|
||
\ \ \ \ \ \ \ \-\ doesn\[aq]t\ match\ "directory/file.jpg"
|
||
\ \ \ \ \ \ \ \-\ doesn\[aq]t\ match\ "adir/file.jpg"
|
||
\f[]
|
||
.fi
|
||
.PP
|
||
A \f[C]?\f[] matches any character except a slash \f[C]/\f[].
|
||
.IP
|
||
.nf
|
||
\f[C]
|
||
l?ss\ \ \-\ matches\ "less"
|
||
\ \ \ \ \ \ \-\ matches\ "lass"
|
||
\ \ \ \ \ \ \-\ doesn\[aq]t\ match\ "floss"
|
||
\f[]
|
||
.fi
|
||
.PP
|
||
A \f[C][\f[] and \f[C]]\f[] together make a a character class, such as
|
||
\f[C][a\-z]\f[] or \f[C][aeiou]\f[] or \f[C][[:alpha:]]\f[].
|
||
See the go regexp docs (https://golang.org/pkg/regexp/syntax/) for more
|
||
info on these.
|
||
.IP
|
||
.nf
|
||
\f[C]
|
||
h[ae]llo\ \-\ matches\ "hello"
|
||
\ \ \ \ \ \ \ \ \ \-\ matches\ "hallo"
|
||
\ \ \ \ \ \ \ \ \ \-\ doesn\[aq]t\ match\ "hullo"
|
||
\f[]
|
||
.fi
|
||
.PP
|
||
A \f[C]{\f[] and \f[C]}\f[] define a choice between elements.
|
||
It should contain a comma separated list of patterns, any of which might
|
||
match.
|
||
These patterns can contain wildcards.
|
||
.IP
|
||
.nf
|
||
\f[C]
|
||
{one,two}_potato\ \-\ matches\ "one_potato"
|
||
\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \-\ matches\ "two_potato"
|
||
\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \-\ doesn\[aq]t\ match\ "three_potato"
|
||
\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \-\ doesn\[aq]t\ match\ "_potato"
|
||
\f[]
|
||
.fi
|
||
.PP
|
||
Special characters can be escaped with a \f[C]\\\f[] before them.
|
||
.IP
|
||
.nf
|
||
\f[C]
|
||
\\*.jpg\ \ \ \ \ \ \ \-\ matches\ "*.jpg"
|
||
\\\\.jpg\ \ \ \ \ \ \ \-\ matches\ "\\.jpg"
|
||
\\[one\\].jpg\ \ \-\ matches\ "[one].jpg"
|
||
\f[]
|
||
.fi
|
||
.PP
|
||
Note also that rclone filter globs can only be used in one of the filter
|
||
command line flags, not in the specification of the remote, so
|
||
\f[C]rclone\ copy\ "remote:dir*.jpg"\ /path/to/dir\f[] won\[aq]t work \-
|
||
what is required is
|
||
\f[C]rclone\ \-\-include\ "*.jpg"\ copy\ remote:dir\ /path/to/dir\f[]
|
||
.SS Directories
|
||
.PP
|
||
Rclone keeps track of directories that could match any file patterns.
|
||
.PP
|
||
Eg if you add the include rule
|
||
.IP
|
||
.nf
|
||
\f[C]
|
||
/a/*.jpg
|
||
\f[]
|
||
.fi
|
||
.PP
|
||
Rclone will synthesize the directory include rule
|
||
.IP
|
||
.nf
|
||
\f[C]
|
||
/a/
|
||
\f[]
|
||
.fi
|
||
.PP
|
||
If you put any rules which end in \f[C]/\f[] then it will only match
|
||
directories.
|
||
.PP
|
||
Directory matches are \f[B]only\f[] used to optimise directory access
|
||
patterns \- you must still match the files that you want to match.
|
||
Directory matches won\[aq]t optimise anything on bucket based remotes
|
||
(eg s3, swift, google compute storage, b2) which don\[aq]t have a
|
||
concept of directory.
|
||
.SS Differences between rsync and rclone patterns
|
||
.PP
|
||
Rclone implements bash style \f[C]{a,b,c}\f[] glob matching which rsync
|
||
doesn\[aq]t.
|
||
.PP
|
||
Rclone always does a wildcard match so \f[C]\\\f[] must always escape a
|
||
\f[C]\\\f[].
|
||
.SS How the rules are used
|
||
.PP
|
||
Rclone maintains a combined list of include rules and exclude rules.
|
||
.PP
|
||
Each file is matched in order, starting from the top, against the rule
|
||
in the list until it finds a match.
|
||
The file is then included or excluded according to the rule type.
|
||
.PP
|
||
If the matcher fails to find a match after testing against all the
|
||
entries in the list then the path is included.
|
||
.PP
|
||
For example given the following rules, \f[C]+\f[] being include,
|
||
\f[C]\-\f[] being exclude,
|
||
.IP
|
||
.nf
|
||
\f[C]
|
||
\-\ secret*.jpg
|
||
+\ *.jpg
|
||
+\ *.png
|
||
+\ file2.avi
|
||
\-\ *
|
||
\f[]
|
||
.fi
|
||
.PP
|
||
This would include
|
||
.IP \[bu] 2
|
||
\f[C]file1.jpg\f[]
|
||
.IP \[bu] 2
|
||
\f[C]file3.png\f[]
|
||
.IP \[bu] 2
|
||
\f[C]file2.avi\f[]
|
||
.PP
|
||
This would exclude
|
||
.IP \[bu] 2
|
||
\f[C]secret17.jpg\f[]
|
||
.IP \[bu] 2
|
||
non \f[C]*.jpg\f[] and \f[C]*.png\f[]
|
||
.PP
|
||
A similar process is done on directory entries before recursing into
|
||
them.
|
||
This only works on remotes which have a concept of directory (Eg local,
|
||
google drive, onedrive, amazon drive) and not on bucket based remotes
|
||
(eg s3, swift, google compute storage, b2).
|
||
.SS Adding filtering rules
|
||
.PP
|
||
Filtering rules are added with the following command line flags.
|
||
.SS Repeating options
|
||
.PP
|
||
You can repeat the following options to add more than one rule of that
|
||
type.
|
||
.IP \[bu] 2
|
||
\f[C]\-\-include\f[]
|
||
.IP \[bu] 2
|
||
\f[C]\-\-include\-from\f[]
|
||
.IP \[bu] 2
|
||
\f[C]\-\-exclude\f[]
|
||
.IP \[bu] 2
|
||
\f[C]\-\-exclude\-from\f[]
|
||
.IP \[bu] 2
|
||
\f[C]\-\-filter\f[]
|
||
.IP \[bu] 2
|
||
\f[C]\-\-filter\-from\f[]
|
||
.PP
|
||
\f[B]Important\f[] You should not use \f[C]\-\-include*\f[] together
|
||
with \f[C]\-\-exclude*\f[].
|
||
It may produce different results than you expected.
|
||
In that case try to use: \f[C]\-\-filter*\f[].
|
||
.PP
|
||
Note that all the options of the same type are processed together in the
|
||
order above, regardless of what order they were placed on the command
|
||
line.
|
||
.PP
|
||
So all \f[C]\-\-include\f[] options are processed first in the order
|
||
they appeared on the command line, then all \f[C]\-\-include\-from\f[]
|
||
options etc.
|
||
.PP
|
||
To mix up the order includes and excludes, the \f[C]\-\-filter\f[] flag
|
||
can be used.
|
||
.SS \f[C]\-\-exclude\f[] \- Exclude files matching pattern
|
||
.PP
|
||
Add a single exclude rule with \f[C]\-\-exclude\f[].
|
||
.PP
|
||
This flag can be repeated.
|
||
See above for the order the flags are processed in.
|
||
.PP
|
||
Eg \f[C]\-\-exclude\ *.bak\f[] to exclude all bak files from the sync.
|
||
.SS \f[C]\-\-exclude\-from\f[] \- Read exclude patterns from file
|
||
.PP
|
||
Add exclude rules from a file.
|
||
.PP
|
||
This flag can be repeated.
|
||
See above for the order the flags are processed in.
|
||
.PP
|
||
Prepare a file like this \f[C]exclude\-file.txt\f[]
|
||
.IP
|
||
.nf
|
||
\f[C]
|
||
#\ a\ sample\ exclude\ rule\ file
|
||
*.bak
|
||
file2.jpg
|
||
\f[]
|
||
.fi
|
||
.PP
|
||
Then use as \f[C]\-\-exclude\-from\ exclude\-file.txt\f[].
|
||
This will sync all files except those ending in \f[C]bak\f[] and
|
||
\f[C]file2.jpg\f[].
|
||
.PP
|
||
This is useful if you have a lot of rules.
|
||
.SS \f[C]\-\-include\f[] \- Include files matching pattern
|
||
.PP
|
||
Add a single include rule with \f[C]\-\-include\f[].
|
||
.PP
|
||
This flag can be repeated.
|
||
See above for the order the flags are processed in.
|
||
.PP
|
||
Eg \f[C]\-\-include\ *.{png,jpg}\f[] to include all \f[C]png\f[] and
|
||
\f[C]jpg\f[] files in the backup and no others.
|
||
.PP
|
||
This adds an implicit \f[C]\-\-exclude\ *\f[] at the very end of the
|
||
filter list.
|
||
This means you can mix \f[C]\-\-include\f[] and
|
||
\f[C]\-\-include\-from\f[] with the other filters (eg
|
||
\f[C]\-\-exclude\f[]) but you must include all the files you want in the
|
||
include statement.
|
||
If this doesn\[aq]t provide enough flexibility then you must use
|
||
\f[C]\-\-filter\-from\f[].
|
||
.SS \f[C]\-\-include\-from\f[] \- Read include patterns from file
|
||
.PP
|
||
Add include rules from a file.
|
||
.PP
|
||
This flag can be repeated.
|
||
See above for the order the flags are processed in.
|
||
.PP
|
||
Prepare a file like this \f[C]include\-file.txt\f[]
|
||
.IP
|
||
.nf
|
||
\f[C]
|
||
#\ a\ sample\ include\ rule\ file
|
||
*.jpg
|
||
*.png
|
||
file2.avi
|
||
\f[]
|
||
.fi
|
||
.PP
|
||
Then use as \f[C]\-\-include\-from\ include\-file.txt\f[].
|
||
This will sync all \f[C]jpg\f[], \f[C]png\f[] files and
|
||
\f[C]file2.avi\f[].
|
||
.PP
|
||
This is useful if you have a lot of rules.
|
||
.PP
|
||
This adds an implicit \f[C]\-\-exclude\ *\f[] at the very end of the
|
||
filter list.
|
||
This means you can mix \f[C]\-\-include\f[] and
|
||
\f[C]\-\-include\-from\f[] with the other filters (eg
|
||
\f[C]\-\-exclude\f[]) but you must include all the files you want in the
|
||
include statement.
|
||
If this doesn\[aq]t provide enough flexibility then you must use
|
||
\f[C]\-\-filter\-from\f[].
|
||
.SS \f[C]\-\-filter\f[] \- Add a file\-filtering rule
|
||
.PP
|
||
This can be used to add a single include or exclude rule.
|
||
Include rules start with \f[C]+\f[] and exclude rules start with
|
||
\f[C]\-\f[].
|
||
A special rule called \f[C]!\f[] can be used to clear the existing
|
||
rules.
|
||
.PP
|
||
This flag can be repeated.
|
||
See above for the order the flags are processed in.
|
||
.PP
|
||
Eg \f[C]\-\-filter\ "\-\ *.bak"\f[] to exclude all bak files from the
|
||
sync.
|
||
.SS \f[C]\-\-filter\-from\f[] \- Read filtering patterns from a file
|
||
.PP
|
||
Add include/exclude rules from a file.
|
||
.PP
|
||
This flag can be repeated.
|
||
See above for the order the flags are processed in.
|
||
.PP
|
||
Prepare a file like this \f[C]filter\-file.txt\f[]
|
||
.IP
|
||
.nf
|
||
\f[C]
|
||
#\ a\ sample\ filter\ rule\ file
|
||
\-\ secret*.jpg
|
||
+\ *.jpg
|
||
+\ *.png
|
||
+\ file2.avi
|
||
\-\ /dir/Trash/**
|
||
+\ /dir/**
|
||
#\ exclude\ everything\ else
|
||
\-\ *
|
||
\f[]
|
||
.fi
|
||
.PP
|
||
Then use as \f[C]\-\-filter\-from\ filter\-file.txt\f[].
|
||
The rules are processed in the order that they are defined.
|
||
.PP
|
||
This example will include all \f[C]jpg\f[] and \f[C]png\f[] files,
|
||
exclude any files matching \f[C]secret*.jpg\f[] and include
|
||
\f[C]file2.avi\f[].
|
||
It will also include everything in the directory \f[C]dir\f[] at the
|
||
root of the sync, except \f[C]dir/Trash\f[] which it will exclude.
|
||
Everything else will be excluded from the sync.
|
||
.SS \f[C]\-\-files\-from\f[] \- Read list of source\-file names
|
||
.PP
|
||
This reads a list of file names from the file passed in and
|
||
\f[B]only\f[] these files are transferred.
|
||
The filtering rules are ignored completely if you use this option.
|
||
.PP
|
||
This option can be repeated to read from more than one file.
|
||
These are read in the order that they are placed on the command line.
|
||
.PP
|
||
Prepare a file like this \f[C]files\-from.txt\f[]
|
||
.IP
|
||
.nf
|
||
\f[C]
|
||
#\ comment
|
||
file1.jpg
|
||
file2.jpg
|
||
\f[]
|
||
.fi
|
||
.PP
|
||
Then use as \f[C]\-\-files\-from\ files\-from.txt\f[].
|
||
This will only transfer \f[C]file1.jpg\f[] and \f[C]file2.jpg\f[]
|
||
providing they exist.
|
||
.PP
|
||
For example, let\[aq]s say you had a few files you want to back up
|
||
regularly with these absolute paths:
|
||
.IP
|
||
.nf
|
||
\f[C]
|
||
/home/user1/important
|
||
/home/user1/dir/file
|
||
/home/user2/stuff
|
||
\f[]
|
||
.fi
|
||
.PP
|
||
To copy these you\[aq]d find a common subdirectory \- in this case
|
||
\f[C]/home\f[] and put the remaining files in \f[C]files\-from.txt\f[]
|
||
with or without leading \f[C]/\f[], eg
|
||
.IP
|
||
.nf
|
||
\f[C]
|
||
user1/important
|
||
user1/dir/file
|
||
user2/stuff
|
||
\f[]
|
||
.fi
|
||
.PP
|
||
You could then copy these to a remote like this
|
||
.IP
|
||
.nf
|
||
\f[C]
|
||
rclone\ copy\ \-\-files\-from\ files\-from.txt\ /home\ remote:backup
|
||
\f[]
|
||
.fi
|
||
.PP
|
||
The 3 files will arrive in \f[C]remote:backup\f[] with the paths as in
|
||
the \f[C]files\-from.txt\f[].
|
||
.PP
|
||
You could of course choose \f[C]/\f[] as the root too in which case your
|
||
\f[C]files\-from.txt\f[] might look like this.
|
||
.IP
|
||
.nf
|
||
\f[C]
|
||
/home/user1/important
|
||
/home/user1/dir/file
|
||
/home/user2/stuff
|
||
\f[]
|
||
.fi
|
||
.PP
|
||
And you would transfer it like this
|
||
.IP
|
||
.nf
|
||
\f[C]
|
||
rclone\ copy\ \-\-files\-from\ files\-from.txt\ /\ remote:backup
|
||
\f[]
|
||
.fi
|
||
.PP
|
||
In this case there will be an extra \f[C]home\f[] directory on the
|
||
remote.
|
||
.SS \f[C]\-\-min\-size\f[] \- Don\[aq]t transfer any file smaller than
|
||
this
|
||
.PP
|
||
This option controls the minimum size file which will be transferred.
|
||
This defaults to \f[C]kBytes\f[] but a suffix of \f[C]k\f[], \f[C]M\f[],
|
||
or \f[C]G\f[] can be used.
|
||
.PP
|
||
For example \f[C]\-\-min\-size\ 50k\f[] means no files smaller than
|
||
50kByte will be transferred.
|
||
.SS \f[C]\-\-max\-size\f[] \- Don\[aq]t transfer any file larger than
|
||
this
|
||
.PP
|
||
This option controls the maximum size file which will be transferred.
|
||
This defaults to \f[C]kBytes\f[] but a suffix of \f[C]k\f[], \f[C]M\f[],
|
||
or \f[C]G\f[] can be used.
|
||
.PP
|
||
For example \f[C]\-\-max\-size\ 1G\f[] means no files larger than 1GByte
|
||
will be transferred.
|
||
.SS \f[C]\-\-max\-age\f[] \- Don\[aq]t transfer any file older than this
|
||
.PP
|
||
This option controls the maximum age of files to transfer.
|
||
Give in seconds or with a suffix of:
|
||
.IP \[bu] 2
|
||
\f[C]ms\f[] \- Milliseconds
|
||
.IP \[bu] 2
|
||
\f[C]s\f[] \- Seconds
|
||
.IP \[bu] 2
|
||
\f[C]m\f[] \- Minutes
|
||
.IP \[bu] 2
|
||
\f[C]h\f[] \- Hours
|
||
.IP \[bu] 2
|
||
\f[C]d\f[] \- Days
|
||
.IP \[bu] 2
|
||
\f[C]w\f[] \- Weeks
|
||
.IP \[bu] 2
|
||
\f[C]M\f[] \- Months
|
||
.IP \[bu] 2
|
||
\f[C]y\f[] \- Years
|
||
.PP
|
||
For example \f[C]\-\-max\-age\ 2d\f[] means no files older than 2 days
|
||
will be transferred.
|
||
.SS \f[C]\-\-min\-age\f[] \- Don\[aq]t transfer any file younger than
|
||
this
|
||
.PP
|
||
This option controls the minimum age of files to transfer.
|
||
Give in seconds or with a suffix (see \f[C]\-\-max\-age\f[] for list of
|
||
suffixes)
|
||
.PP
|
||
For example \f[C]\-\-min\-age\ 2d\f[] means no files younger than 2 days
|
||
will be transferred.
|
||
.SS \f[C]\-\-delete\-excluded\f[] \- Delete files on dest excluded from
|
||
sync
|
||
.PP
|
||
\f[B]Important\f[] this flag is dangerous \- use with
|
||
\f[C]\-\-dry\-run\f[] and \f[C]\-v\f[] first.
|
||
.PP
|
||
When doing \f[C]rclone\ sync\f[] this will delete any files which are
|
||
excluded from the sync on the destination.
|
||
.PP
|
||
If for example you did a sync from \f[C]A\f[] to \f[C]B\f[] without the
|
||
\f[C]\-\-min\-size\ 50k\f[] flag
|
||
.IP
|
||
.nf
|
||
\f[C]
|
||
rclone\ sync\ A:\ B:
|
||
\f[]
|
||
.fi
|
||
.PP
|
||
Then you repeated it like this with the \f[C]\-\-delete\-excluded\f[]
|
||
.IP
|
||
.nf
|
||
\f[C]
|
||
rclone\ \-\-min\-size\ 50k\ \-\-delete\-excluded\ sync\ A:\ B:
|
||
\f[]
|
||
.fi
|
||
.PP
|
||
This would delete all files on \f[C]B\f[] which are less than 50 kBytes
|
||
as these are now excluded from the sync.
|
||
.PP
|
||
Always test first with \f[C]\-\-dry\-run\f[] and \f[C]\-v\f[] before
|
||
using this flag.
|
||
.SS \f[C]\-\-dump\ filters\f[] \- dump the filters to the output
|
||
.PP
|
||
This dumps the defined filters to the output as regular expressions.
|
||
.PP
|
||
Useful for debugging.
|
||
.SS Quoting shell metacharacters
|
||
.PP
|
||
The examples above may not work verbatim in your shell as they have
|
||
shell metacharacters in them (eg \f[C]*\f[]), and may require quoting.
|
||
.PP
|
||
Eg linux, OSX
|
||
.IP \[bu] 2
|
||
\f[C]\-\-include\ \\*.jpg\f[]
|
||
.IP \[bu] 2
|
||
\f[C]\-\-include\ \[aq]*.jpg\[aq]\f[]
|
||
.IP \[bu] 2
|
||
\f[C]\-\-include=\[aq]*.jpg\[aq]\f[]
|
||
.PP
|
||
In Windows the expansion is done by the command not the shell so this
|
||
should work fine
|
||
.IP \[bu] 2
|
||
\f[C]\-\-include\ *.jpg\f[]
|
||
.SS Exclude directory based on a file
|
||
.PP
|
||
It is possible to exclude a directory based on a file, which is present
|
||
in this directory.
|
||
Filename should be specified using the \f[C]\-\-exclude\-if\-present\f[]
|
||
flag.
|
||
This flag has a priority over the other filtering flags.
|
||
.PP
|
||
Imagine, you have the following directory structure:
|
||
.IP
|
||
.nf
|
||
\f[C]
|
||
dir1/file1
|
||
dir1/dir2/file2
|
||
dir1/dir2/dir3/file3
|
||
dir1/dir2/dir3/.ignore
|
||
\f[]
|
||
.fi
|
||
.PP
|
||
You can exclude \f[C]dir3\f[] from sync by running the following
|
||
command:
|
||
.IP
|
||
.nf
|
||
\f[C]
|
||
rclone\ sync\ \-\-exclude\-if\-present\ .ignore\ dir1\ remote:backup
|
||
\f[]
|
||
.fi
|
||
.PP
|
||
Currently only one filename is supported, i.e.
|
||
\f[C]\-\-exclude\-if\-present\f[] should not be used multiple times.
|
||
.SH Overview of cloud storage systems
|
||
.PP
|
||
Each cloud storage system is slightly different.
|
||
Rclone attempts to provide a unified interface to them, but some
|
||
underlying differences show through.
|
||
.SS Features
|
||
.PP
|
||
Here is an overview of the major features of each cloud storage system.
|
||
.PP
|
||
.TS
|
||
tab(@);
|
||
l c c c c c.
|
||
T{
|
||
Name
|
||
T}@T{
|
||
Hash
|
||
T}@T{
|
||
ModTime
|
||
T}@T{
|
||
Case Insensitive
|
||
T}@T{
|
||
Duplicate Files
|
||
T}@T{
|
||
MIME Type
|
||
T}
|
||
_
|
||
T{
|
||
Amazon Drive
|
||
T}@T{
|
||
MD5
|
||
T}@T{
|
||
No
|
||
T}@T{
|
||
Yes
|
||
T}@T{
|
||
No
|
||
T}@T{
|
||
R
|
||
T}
|
||
T{
|
||
Amazon S3
|
||
T}@T{
|
||
MD5
|
||
T}@T{
|
||
Yes
|
||
T}@T{
|
||
No
|
||
T}@T{
|
||
No
|
||
T}@T{
|
||
R/W
|
||
T}
|
||
T{
|
||
Backblaze B2
|
||
T}@T{
|
||
SHA1
|
||
T}@T{
|
||
Yes
|
||
T}@T{
|
||
No
|
||
T}@T{
|
||
No
|
||
T}@T{
|
||
R/W
|
||
T}
|
||
T{
|
||
Box
|
||
T}@T{
|
||
SHA1
|
||
T}@T{
|
||
Yes
|
||
T}@T{
|
||
Yes
|
||
T}@T{
|
||
No
|
||
T}@T{
|
||
\-
|
||
T}
|
||
T{
|
||
Dropbox
|
||
T}@T{
|
||
DBHASH †
|
||
T}@T{
|
||
Yes
|
||
T}@T{
|
||
Yes
|
||
T}@T{
|
||
No
|
||
T}@T{
|
||
\-
|
||
T}
|
||
T{
|
||
FTP
|
||
T}@T{
|
||
\-
|
||
T}@T{
|
||
No
|
||
T}@T{
|
||
No
|
||
T}@T{
|
||
No
|
||
T}@T{
|
||
\-
|
||
T}
|
||
T{
|
||
Google Cloud Storage
|
||
T}@T{
|
||
MD5
|
||
T}@T{
|
||
Yes
|
||
T}@T{
|
||
No
|
||
T}@T{
|
||
No
|
||
T}@T{
|
||
R/W
|
||
T}
|
||
T{
|
||
Google Drive
|
||
T}@T{
|
||
MD5
|
||
T}@T{
|
||
Yes
|
||
T}@T{
|
||
No
|
||
T}@T{
|
||
Yes
|
||
T}@T{
|
||
R/W
|
||
T}
|
||
T{
|
||
HTTP
|
||
T}@T{
|
||
\-
|
||
T}@T{
|
||
No
|
||
T}@T{
|
||
No
|
||
T}@T{
|
||
No
|
||
T}@T{
|
||
R
|
||
T}
|
||
T{
|
||
Hubic
|
||
T}@T{
|
||
MD5
|
||
T}@T{
|
||
Yes
|
||
T}@T{
|
||
No
|
||
T}@T{
|
||
No
|
||
T}@T{
|
||
R/W
|
||
T}
|
||
T{
|
||
Microsoft Azure Blob Storage
|
||
T}@T{
|
||
MD5
|
||
T}@T{
|
||
Yes
|
||
T}@T{
|
||
No
|
||
T}@T{
|
||
No
|
||
T}@T{
|
||
R/W
|
||
T}
|
||
T{
|
||
Microsoft OneDrive
|
||
T}@T{
|
||
SHA1
|
||
T}@T{
|
||
Yes
|
||
T}@T{
|
||
Yes
|
||
T}@T{
|
||
No
|
||
T}@T{
|
||
R
|
||
T}
|
||
T{
|
||
Openstack Swift
|
||
T}@T{
|
||
MD5
|
||
T}@T{
|
||
Yes
|
||
T}@T{
|
||
No
|
||
T}@T{
|
||
No
|
||
T}@T{
|
||
R/W
|
||
T}
|
||
T{
|
||
pCloud
|
||
T}@T{
|
||
MD5, SHA1
|
||
T}@T{
|
||
Yes
|
||
T}@T{
|
||
No
|
||
T}@T{
|
||
No
|
||
T}@T{
|
||
W
|
||
T}
|
||
T{
|
||
QingStor
|
||
T}@T{
|
||
MD5
|
||
T}@T{
|
||
No
|
||
T}@T{
|
||
No
|
||
T}@T{
|
||
No
|
||
T}@T{
|
||
R/W
|
||
T}
|
||
T{
|
||
SFTP
|
||
T}@T{
|
||
MD5, SHA1 ‡
|
||
T}@T{
|
||
Yes
|
||
T}@T{
|
||
Depends
|
||
T}@T{
|
||
No
|
||
T}@T{
|
||
\-
|
||
T}
|
||
T{
|
||
WebDAV
|
||
T}@T{
|
||
\-
|
||
T}@T{
|
||
Yes ††
|
||
T}@T{
|
||
Depends
|
||
T}@T{
|
||
No
|
||
T}@T{
|
||
\-
|
||
T}
|
||
T{
|
||
Yandex Disk
|
||
T}@T{
|
||
MD5
|
||
T}@T{
|
||
Yes
|
||
T}@T{
|
||
No
|
||
T}@T{
|
||
No
|
||
T}@T{
|
||
R/W
|
||
T}
|
||
T{
|
||
The local filesystem
|
||
T}@T{
|
||
All
|
||
T}@T{
|
||
Yes
|
||
T}@T{
|
||
Depends
|
||
T}@T{
|
||
No
|
||
T}@T{
|
||
\-
|
||
T}
|
||
.TE
|
||
.SS Hash
|
||
.PP
|
||
The cloud storage system supports various hash types of the objects.
|
||
The hashes are used when transferring data as an integrity check and can
|
||
be specifically used with the \f[C]\-\-checksum\f[] flag in syncs and in
|
||
the \f[C]check\f[] command.
|
||
.PP
|
||
To use the verify checksums when transferring between cloud storage
|
||
systems they must support a common hash type.
|
||
.PP
|
||
† Note that Dropbox supports its own custom
|
||
hash (https://www.dropbox.com/developers/reference/content-hash).
|
||
This is an SHA256 sum of all the 4MB block SHA256s.
|
||
.PP
|
||
‡ SFTP supports checksums if the same login has shell access and
|
||
\f[C]md5sum\f[] or \f[C]sha1sum\f[] as well as \f[C]echo\f[] are in the
|
||
remote\[aq]s PATH.
|
||
.PP
|
||
†† WebDAV supports modtimes when used with Owncloud and Nextcloud only.
|
||
.SS ModTime
|
||
.PP
|
||
The cloud storage system supports setting modification times on objects.
|
||
If it does then this enables a using the modification times as part of
|
||
the sync.
|
||
If not then only the size will be checked by default, though the MD5SUM
|
||
can be checked with the \f[C]\-\-checksum\f[] flag.
|
||
.PP
|
||
All cloud storage systems support some kind of date on the object and
|
||
these will be set when transferring from the cloud storage system.
|
||
.SS Case Insensitive
|
||
.PP
|
||
If a cloud storage systems is case sensitive then it is possible to have
|
||
two files which differ only in case, eg \f[C]file.txt\f[] and
|
||
\f[C]FILE.txt\f[].
|
||
If a cloud storage system is case insensitive then that isn\[aq]t
|
||
possible.
|
||
.PP
|
||
This can cause problems when syncing between a case insensitive system
|
||
and a case sensitive system.
|
||
The symptom of this is that no matter how many times you run the sync it
|
||
never completes fully.
|
||
.PP
|
||
The local filesystem and SFTP may or may not be case sensitive depending
|
||
on OS.
|
||
.IP \[bu] 2
|
||
Windows \- usually case insensitive, though case is preserved
|
||
.IP \[bu] 2
|
||
OSX \- usually case insensitive, though it is possible to format case
|
||
sensitive
|
||
.IP \[bu] 2
|
||
Linux \- usually case sensitive, but there are case insensitive file
|
||
systems (eg FAT formatted USB keys)
|
||
.PP
|
||
Most of the time this doesn\[aq]t cause any problems as people tend to
|
||
avoid files whose name differs only by case even on case sensitive
|
||
systems.
|
||
.SS Duplicate files
|
||
.PP
|
||
If a cloud storage system allows duplicate files then it can have two
|
||
objects with the same name.
|
||
.PP
|
||
This confuses rclone greatly when syncing \- use the
|
||
\f[C]rclone\ dedupe\f[] command to rename or remove duplicates.
|
||
.SS MIME Type
|
||
.PP
|
||
MIME types (also known as media types) classify types of documents using
|
||
a simple text classification, eg \f[C]text/html\f[] or
|
||
\f[C]application/pdf\f[].
|
||
.PP
|
||
Some cloud storage systems support reading (\f[C]R\f[]) the MIME type of
|
||
objects and some support writing (\f[C]W\f[]) the MIME type of objects.
|
||
.PP
|
||
The MIME type can be important if you are serving files directly to HTTP
|
||
from the storage system.
|
||
.PP
|
||
If you are copying from a remote which supports reading (\f[C]R\f[]) to
|
||
a remote which supports writing (\f[C]W\f[]) then rclone will preserve
|
||
the MIME types.
|
||
Otherwise they will be guessed from the extension, or the remote itself
|
||
may assign the MIME type.
|
||
.SS Optional Features
|
||
.PP
|
||
All the remotes support a basic set of features, but there are some
|
||
optional features supported by some remotes used to make some operations
|
||
more efficient.
|
||
.PP
|
||
.TS
|
||
tab(@);
|
||
l c c c c c c c.
|
||
T{
|
||
Name
|
||
T}@T{
|
||
Purge
|
||
T}@T{
|
||
Copy
|
||
T}@T{
|
||
Move
|
||
T}@T{
|
||
DirMove
|
||
T}@T{
|
||
CleanUp
|
||
T}@T{
|
||
ListR
|
||
T}@T{
|
||
StreamUpload
|
||
T}
|
||
_
|
||
T{
|
||
Amazon Drive
|
||
T}@T{
|
||
Yes
|
||
T}@T{
|
||
No
|
||
T}@T{
|
||
Yes
|
||
T}@T{
|
||
Yes
|
||
T}@T{
|
||
No #575 (https://github.com/ncw/rclone/issues/575)
|
||
T}@T{
|
||
No
|
||
T}@T{
|
||
No
|
||
T}
|
||
T{
|
||
Amazon S3
|
||
T}@T{
|
||
No
|
||
T}@T{
|
||
Yes
|
||
T}@T{
|
||
No
|
||
T}@T{
|
||
No
|
||
T}@T{
|
||
No
|
||
T}@T{
|
||
Yes
|
||
T}@T{
|
||
Yes
|
||
T}
|
||
T{
|
||
Backblaze B2
|
||
T}@T{
|
||
No
|
||
T}@T{
|
||
No
|
||
T}@T{
|
||
No
|
||
T}@T{
|
||
No
|
||
T}@T{
|
||
Yes
|
||
T}@T{
|
||
Yes
|
||
T}@T{
|
||
Yes
|
||
T}
|
||
T{
|
||
Box
|
||
T}@T{
|
||
Yes
|
||
T}@T{
|
||
Yes
|
||
T}@T{
|
||
Yes
|
||
T}@T{
|
||
Yes
|
||
T}@T{
|
||
No #575 (https://github.com/ncw/rclone/issues/575)
|
||
T}@T{
|
||
No
|
||
T}@T{
|
||
Yes
|
||
T}
|
||
T{
|
||
Dropbox
|
||
T}@T{
|
||
Yes
|
||
T}@T{
|
||
Yes
|
||
T}@T{
|
||
Yes
|
||
T}@T{
|
||
Yes
|
||
T}@T{
|
||
No #575 (https://github.com/ncw/rclone/issues/575)
|
||
T}@T{
|
||
No
|
||
T}@T{
|
||
Yes
|
||
T}
|
||
T{
|
||
FTP
|
||
T}@T{
|
||
No
|
||
T}@T{
|
||
No
|
||
T}@T{
|
||
Yes
|
||
T}@T{
|
||
Yes
|
||
T}@T{
|
||
No
|
||
T}@T{
|
||
No
|
||
T}@T{
|
||
Yes
|
||
T}
|
||
T{
|
||
Google Cloud Storage
|
||
T}@T{
|
||
Yes
|
||
T}@T{
|
||
Yes
|
||
T}@T{
|
||
No
|
||
T}@T{
|
||
No
|
||
T}@T{
|
||
No
|
||
T}@T{
|
||
Yes
|
||
T}@T{
|
||
Yes
|
||
T}
|
||
T{
|
||
Google Drive
|
||
T}@T{
|
||
Yes
|
||
T}@T{
|
||
Yes
|
||
T}@T{
|
||
Yes
|
||
T}@T{
|
||
Yes
|
||
T}@T{
|
||
Yes
|
||
T}@T{
|
||
No
|
||
T}@T{
|
||
Yes
|
||
T}
|
||
T{
|
||
HTTP
|
||
T}@T{
|
||
No
|
||
T}@T{
|
||
No
|
||
T}@T{
|
||
No
|
||
T}@T{
|
||
No
|
||
T}@T{
|
||
No
|
||
T}@T{
|
||
No
|
||
T}@T{
|
||
No
|
||
T}
|
||
T{
|
||
Hubic
|
||
T}@T{
|
||
Yes †
|
||
T}@T{
|
||
Yes
|
||
T}@T{
|
||
No
|
||
T}@T{
|
||
No
|
||
T}@T{
|
||
No
|
||
T}@T{
|
||
Yes
|
||
T}@T{
|
||
Yes
|
||
T}
|
||
T{
|
||
Microsoft Azure Blob Storage
|
||
T}@T{
|
||
Yes
|
||
T}@T{
|
||
Yes
|
||
T}@T{
|
||
No
|
||
T}@T{
|
||
No
|
||
T}@T{
|
||
No
|
||
T}@T{
|
||
Yes
|
||
T}@T{
|
||
No
|
||
T}
|
||
T{
|
||
Microsoft OneDrive
|
||
T}@T{
|
||
Yes
|
||
T}@T{
|
||
Yes
|
||
T}@T{
|
||
Yes
|
||
T}@T{
|
||
No #197 (https://github.com/ncw/rclone/issues/197)
|
||
T}@T{
|
||
No #575 (https://github.com/ncw/rclone/issues/575)
|
||
T}@T{
|
||
No
|
||
T}@T{
|
||
No
|
||
T}
|
||
T{
|
||
Openstack Swift
|
||
T}@T{
|
||
Yes †
|
||
T}@T{
|
||
Yes
|
||
T}@T{
|
||
No
|
||
T}@T{
|
||
No
|
||
T}@T{
|
||
No
|
||
T}@T{
|
||
Yes
|
||
T}@T{
|
||
Yes
|
||
T}
|
||
T{
|
||
pCloud
|
||
T}@T{
|
||
Yes
|
||
T}@T{
|
||
Yes
|
||
T}@T{
|
||
Yes
|
||
T}@T{
|
||
Yes
|
||
T}@T{
|
||
Yes
|
||
T}@T{
|
||
No
|
||
T}@T{
|
||
No
|
||
T}
|
||
T{
|
||
QingStor
|
||
T}@T{
|
||
No
|
||
T}@T{
|
||
Yes
|
||
T}@T{
|
||
No
|
||
T}@T{
|
||
No
|
||
T}@T{
|
||
No
|
||
T}@T{
|
||
Yes
|
||
T}@T{
|
||
No
|
||
T}
|
||
T{
|
||
SFTP
|
||
T}@T{
|
||
No
|
||
T}@T{
|
||
No
|
||
T}@T{
|
||
Yes
|
||
T}@T{
|
||
Yes
|
||
T}@T{
|
||
No
|
||
T}@T{
|
||
No
|
||
T}@T{
|
||
Yes
|
||
T}
|
||
T{
|
||
WebDAV
|
||
T}@T{
|
||
Yes
|
||
T}@T{
|
||
Yes
|
||
T}@T{
|
||
Yes
|
||
T}@T{
|
||
Yes
|
||
T}@T{
|
||
No
|
||
T}@T{
|
||
No
|
||
T}@T{
|
||
Yes ‡
|
||
T}
|
||
T{
|
||
Yandex Disk
|
||
T}@T{
|
||
Yes
|
||
T}@T{
|
||
No
|
||
T}@T{
|
||
No
|
||
T}@T{
|
||
No
|
||
T}@T{
|
||
Yes
|
||
T}@T{
|
||
Yes
|
||
T}@T{
|
||
Yes
|
||
T}
|
||
T{
|
||
The local filesystem
|
||
T}@T{
|
||
Yes
|
||
T}@T{
|
||
No
|
||
T}@T{
|
||
Yes
|
||
T}@T{
|
||
Yes
|
||
T}@T{
|
||
No
|
||
T}@T{
|
||
No
|
||
T}@T{
|
||
Yes
|
||
T}
|
||
.TE
|
||
.SS Purge
|
||
.PP
|
||
This deletes a directory quicker than just deleting all the files in the
|
||
directory.
|
||
.PP
|
||
† Note Swift and Hubic implement this in order to delete directory
|
||
markers but they don\[aq]t actually have a quicker way of deleting files
|
||
other than deleting them individually.
|
||
.PP
|
||
‡ StreamUpload is not supported with Nextcloud
|
||
.SS Copy
|
||
.PP
|
||
Used when copying an object to and from the same remote.
|
||
This known as a server side copy so you can copy a file without
|
||
downloading it and uploading it again.
|
||
It is used if you use \f[C]rclone\ copy\f[] or \f[C]rclone\ move\f[] if
|
||
the remote doesn\[aq]t support \f[C]Move\f[] directly.
|
||
.PP
|
||
If the server doesn\[aq]t support \f[C]Copy\f[] directly then for copy
|
||
operations the file is downloaded then re\-uploaded.
|
||
.SS Move
|
||
.PP
|
||
Used when moving/renaming an object on the same remote.
|
||
This is known as a server side move of a file.
|
||
This is used in \f[C]rclone\ move\f[] if the server doesn\[aq]t support
|
||
\f[C]DirMove\f[].
|
||
.PP
|
||
If the server isn\[aq]t capable of \f[C]Move\f[] then rclone simulates
|
||
it with \f[C]Copy\f[] then delete.
|
||
If the server doesn\[aq]t support \f[C]Copy\f[] then rclone will
|
||
download the file and re\-upload it.
|
||
.SS DirMove
|
||
.PP
|
||
This is used to implement \f[C]rclone\ move\f[] to move a directory if
|
||
possible.
|
||
If it isn\[aq]t then it will use \f[C]Move\f[] on each file (which falls
|
||
back to \f[C]Copy\f[] then download and upload \- see \f[C]Move\f[]
|
||
section).
|
||
.SS CleanUp
|
||
.PP
|
||
This is used for emptying the trash for a remote by
|
||
\f[C]rclone\ cleanup\f[].
|
||
.PP
|
||
If the server can\[aq]t do \f[C]CleanUp\f[] then
|
||
\f[C]rclone\ cleanup\f[] will return an error.
|
||
.SS ListR
|
||
.PP
|
||
The remote supports a recursive list to list all the contents beneath a
|
||
directory quickly.
|
||
This enables the \f[C]\-\-fast\-list\f[] flag to work.
|
||
See the rclone docs (/docs/#fast-list) for more details.
|
||
.SS StreamUpload
|
||
.PP
|
||
Some remotes allow files to be uploaded without knowing the file size in
|
||
advance.
|
||
This allows certain operations to work without spooling the file to
|
||
local disk first, e.g.
|
||
\f[C]rclone\ rcat\f[].
|
||
.SS Amazon Drive
|
||
.PP
|
||
Paths are specified as \f[C]remote:path\f[]
|
||
.PP
|
||
Paths may be as deep as required, eg
|
||
\f[C]remote:directory/subdirectory\f[].
|
||
.PP
|
||
The initial setup for Amazon Drive involves getting a token from Amazon
|
||
which you need to do in your browser.
|
||
\f[C]rclone\ config\f[] walks you through it.
|
||
.PP
|
||
The configuration process for Amazon Drive may involve using an oauth
|
||
proxy (https://github.com/ncw/oauthproxy).
|
||
This is used to keep the Amazon credentials out of the source code.
|
||
The proxy runs in Google\[aq]s very secure App Engine environment and
|
||
doesn\[aq]t store any credentials which pass through it.
|
||
.PP
|
||
\f[B]NB\f[] rclone doesn\[aq]t not currently have its own Amazon Drive
|
||
credentials (see the
|
||
forum (https://forum.rclone.org/t/rclone-has-been-banned-from-amazon-drive/)
|
||
for why) so you will either need to have your own \f[C]client_id\f[] and
|
||
\f[C]client_secret\f[] with Amazon Drive, or use a a third party ouath
|
||
proxy in which case you will need to enter \f[C]client_id\f[],
|
||
\f[C]client_secret\f[], \f[C]auth_url\f[] and \f[C]token_url\f[].
|
||
.PP
|
||
Note also if you are not using Amazon\[aq]s \f[C]auth_url\f[] and
|
||
\f[C]token_url\f[], (ie you filled in something for those) then if
|
||
setting up on a remote machine you can only use the copying the config
|
||
method of
|
||
configuration (https://rclone.org/remote_setup/#configuring-by-copying-the-config-file)
|
||
\- \f[C]rclone\ authorize\f[] will not work.
|
||
.PP
|
||
Here is an example of how to make a remote called \f[C]remote\f[].
|
||
First run:
|
||
.IP
|
||
.nf
|
||
\f[C]
|
||
\ rclone\ config
|
||
\f[]
|
||
.fi
|
||
.PP
|
||
This will guide you through an interactive setup process:
|
||
.IP
|
||
.nf
|
||
\f[C]
|
||
No\ remotes\ found\ \-\ make\ a\ new\ one
|
||
n)\ New\ remote
|
||
r)\ Rename\ remote
|
||
c)\ Copy\ remote
|
||
s)\ Set\ configuration\ password
|
||
q)\ Quit\ config
|
||
n/r/c/s/q>\ n
|
||
name>\ remote
|
||
Type\ of\ storage\ to\ configure.
|
||
Choose\ a\ number\ from\ below,\ or\ type\ in\ your\ own\ value
|
||
\ 1\ /\ Amazon\ Drive
|
||
\ \ \ \\\ "amazon\ cloud\ drive"
|
||
\ 2\ /\ Amazon\ S3\ (also\ Dreamhost,\ Ceph,\ Minio)
|
||
\ \ \ \\\ "s3"
|
||
\ 3\ /\ Backblaze\ B2
|
||
\ \ \ \\\ "b2"
|
||
\ 4\ /\ Dropbox
|
||
\ \ \ \\\ "dropbox"
|
||
\ 5\ /\ Encrypt/Decrypt\ a\ remote
|
||
\ \ \ \\\ "crypt"
|
||
\ 6\ /\ FTP\ Connection
|
||
\ \ \ \\\ "ftp"
|
||
\ 7\ /\ Google\ Cloud\ Storage\ (this\ is\ not\ Google\ Drive)
|
||
\ \ \ \\\ "google\ cloud\ storage"
|
||
\ 8\ /\ Google\ Drive
|
||
\ \ \ \\\ "drive"
|
||
\ 9\ /\ Hubic
|
||
\ \ \ \\\ "hubic"
|
||
10\ /\ Local\ Disk
|
||
\ \ \ \\\ "local"
|
||
11\ /\ Microsoft\ OneDrive
|
||
\ \ \ \\\ "onedrive"
|
||
12\ /\ Openstack\ Swift\ (Rackspace\ Cloud\ Files,\ Memset\ Memstore,\ OVH)
|
||
\ \ \ \\\ "swift"
|
||
13\ /\ SSH/SFTP\ Connection
|
||
\ \ \ \\\ "sftp"
|
||
14\ /\ Yandex\ Disk
|
||
\ \ \ \\\ "yandex"
|
||
Storage>\ 1
|
||
Amazon\ Application\ Client\ Id\ \-\ required.
|
||
client_id>\ your\ client\ ID\ goes\ here
|
||
Amazon\ Application\ Client\ Secret\ \-\ required.
|
||
client_secret>\ your\ client\ secret\ goes\ here
|
||
Auth\ server\ URL\ \-\ leave\ blank\ to\ use\ Amazon\[aq]s.
|
||
auth_url>\ Optional\ auth\ URL
|
||
Token\ server\ url\ \-\ leave\ blank\ to\ use\ Amazon\[aq]s.
|
||
token_url>\ Optional\ token\ URL
|
||
Remote\ config
|
||
Make\ sure\ your\ Redirect\ URL\ is\ set\ to\ "http://127.0.0.1:53682/"\ in\ your\ custom\ config.
|
||
Use\ auto\ config?
|
||
\ *\ Say\ Y\ if\ not\ sure
|
||
\ *\ Say\ N\ if\ you\ are\ working\ on\ a\ remote\ or\ headless\ machine
|
||
y)\ Yes
|
||
n)\ No
|
||
y/n>\ y
|
||
If\ your\ browser\ doesn\[aq]t\ open\ automatically\ go\ to\ the\ following\ link:\ http://127.0.0.1:53682/auth
|
||
Log\ in\ and\ authorize\ rclone\ for\ access
|
||
Waiting\ for\ code...
|
||
Got\ code
|
||
\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-
|
||
[remote]
|
||
client_id\ =\ your\ client\ ID\ goes\ here
|
||
client_secret\ =\ your\ client\ secret\ goes\ here
|
||
auth_url\ =\ Optional\ auth\ URL
|
||
token_url\ =\ Optional\ token\ URL
|
||
token\ =\ {"access_token":"xxxxxxxxxxxxxxxxxxxxxxx","token_type":"bearer","refresh_token":"xxxxxxxxxxxxxxxxxx","expiry":"2015\-09\-06T16:07:39.658438471+01:00"}
|
||
\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-
|
||
y)\ Yes\ this\ is\ OK
|
||
e)\ Edit\ this\ remote
|
||
d)\ Delete\ this\ remote
|
||
y/e/d>\ y
|
||
\f[]
|
||
.fi
|
||
.PP
|
||
See the remote setup docs (https://rclone.org/remote_setup/) for how to
|
||
set it up on a machine with no Internet browser available.
|
||
.PP
|
||
Note that rclone runs a webserver on your local machine to collect the
|
||
token as returned from Amazon.
|
||
This only runs from the moment it opens your browser to the moment you
|
||
get back the verification code.
|
||
This is on \f[C]http://127.0.0.1:53682/\f[] and this it may require you
|
||
to unblock it temporarily if you are running a host firewall.
|
||
.PP
|
||
Once configured you can then use \f[C]rclone\f[] like this,
|
||
.PP
|
||
List directories in top level of your Amazon Drive
|
||
.IP
|
||
.nf
|
||
\f[C]
|
||
rclone\ lsd\ remote:
|
||
\f[]
|
||
.fi
|
||
.PP
|
||
List all the files in your Amazon Drive
|
||
.IP
|
||
.nf
|
||
\f[C]
|
||
rclone\ ls\ remote:
|
||
\f[]
|
||
.fi
|
||
.PP
|
||
To copy a local directory to an Amazon Drive directory called backup
|
||
.IP
|
||
.nf
|
||
\f[C]
|
||
rclone\ copy\ /home/source\ remote:backup
|
||
\f[]
|
||
.fi
|
||
.SS Modified time and MD5SUMs
|
||
.PP
|
||
Amazon Drive doesn\[aq]t allow modification times to be changed via the
|
||
API so these won\[aq]t be accurate or used for syncing.
|
||
.PP
|
||
It does store MD5SUMs so for a more accurate sync, you can use the
|
||
\f[C]\-\-checksum\f[] flag.
|
||
.SS Deleting files
|
||
.PP
|
||
Any files you delete with rclone will end up in the trash.
|
||
Amazon don\[aq]t provide an API to permanently delete files, nor to
|
||
empty the trash, so you will have to do that with one of Amazon\[aq]s
|
||
apps or via the Amazon Drive website.
|
||
As of November 17, 2016, files are automatically deleted by Amazon from
|
||
the trash after 30 days.
|
||
.SS Using with non \f[C]\&.com\f[] Amazon accounts
|
||
.PP
|
||
Let\[aq]s say you usually use \f[C]amazon.co.uk\f[].
|
||
When you authenticate with rclone it will take you to an
|
||
\f[C]amazon.com\f[] page to log in.
|
||
Your \f[C]amazon.co.uk\f[] email and password should work here just
|
||
fine.
|
||
.SS Specific options
|
||
.PP
|
||
Here are the command line options specific to this cloud storage system.
|
||
.SS \-\-acd\-templink\-threshold=SIZE
|
||
.PP
|
||
Files this size or more will be downloaded via their \f[C]tempLink\f[].
|
||
This is to work around a problem with Amazon Drive which blocks
|
||
downloads of files bigger than about 10GB.
|
||
The default for this is 9GB which shouldn\[aq]t need to be changed.
|
||
.PP
|
||
To download files above this threshold, rclone requests a
|
||
\f[C]tempLink\f[] which downloads the file through a temporary URL
|
||
directly from the underlying S3 storage.
|
||
.SS \-\-acd\-upload\-wait\-per\-gb=TIME
|
||
.PP
|
||
Sometimes Amazon Drive gives an error when a file has been fully
|
||
uploaded but the file appears anyway after a little while.
|
||
This happens sometimes for files over 1GB in size and nearly every time
|
||
for files bigger than 10GB.
|
||
This parameter controls the time rclone waits for the file to appear.
|
||
.PP
|
||
The default value for this parameter is 3 minutes per GB, so by default
|
||
it will wait 3 minutes for every GB uploaded to see if the file appears.
|
||
.PP
|
||
You can disable this feature by setting it to 0.
|
||
This may cause conflict errors as rclone retries the failed upload but
|
||
the file will most likely appear correctly eventually.
|
||
.PP
|
||
These values were determined empirically by observing lots of uploads of
|
||
big files for a range of file sizes.
|
||
.PP
|
||
Upload with the \f[C]\-v\f[] flag to see more info about what rclone is
|
||
doing in this situation.
|
||
.SS Limitations
|
||
.PP
|
||
Note that Amazon Drive is case insensitive so you can\[aq]t have a file
|
||
called "Hello.doc" and one called "hello.doc".
|
||
.PP
|
||
Amazon Drive has rate limiting so you may notice errors in the sync (429
|
||
errors).
|
||
rclone will automatically retry the sync up to 3 times by default (see
|
||
\f[C]\-\-retries\f[] flag) which should hopefully work around this
|
||
problem.
|
||
.PP
|
||
Amazon Drive has an internal limit of file sizes that can be uploaded to
|
||
the service.
|
||
This limit is not officially published, but all files larger than this
|
||
will fail.
|
||
.PP
|
||
At the time of writing (Jan 2016) is in the area of 50GB per file.
|
||
This means that larger files are likely to fail.
|
||
.PP
|
||
Unfortunately there is no way for rclone to see that this failure is
|
||
because of file size, so it will retry the operation, as any other
|
||
failure.
|
||
To avoid this problem, use \f[C]\-\-max\-size\ 50000M\f[] option to
|
||
limit the maximum size of uploaded files.
|
||
Note that \f[C]\-\-max\-size\f[] does not split files into segments, it
|
||
only ignores files over this size.
|
||
.SS Amazon S3
|
||
.PP
|
||
Paths are specified as \f[C]remote:bucket\f[] (or \f[C]remote:\f[] for
|
||
the \f[C]lsd\f[] command.) You may put subdirectories in too, eg
|
||
\f[C]remote:bucket/path/to/dir\f[].
|
||
.PP
|
||
Here is an example of making an s3 configuration.
|
||
First run
|
||
.IP
|
||
.nf
|
||
\f[C]
|
||
rclone\ config
|
||
\f[]
|
||
.fi
|
||
.PP
|
||
This will guide you through an interactive setup process.
|
||
.IP
|
||
.nf
|
||
\f[C]
|
||
No\ remotes\ found\ \-\ make\ a\ new\ one
|
||
n)\ New\ remote
|
||
s)\ Set\ configuration\ password
|
||
n/s>\ n
|
||
name>\ remote
|
||
Type\ of\ storage\ to\ configure.
|
||
Choose\ a\ number\ from\ below,\ or\ type\ in\ your\ own\ value
|
||
\ 1\ /\ Amazon\ Drive
|
||
\ \ \ \\\ "amazon\ cloud\ drive"
|
||
\ 2\ /\ Amazon\ S3\ (also\ Dreamhost,\ Ceph,\ Minio)
|
||
\ \ \ \\\ "s3"
|
||
\ 3\ /\ Backblaze\ B2
|
||
\ \ \ \\\ "b2"
|
||
\ 4\ /\ Dropbox
|
||
\ \ \ \\\ "dropbox"
|
||
\ 5\ /\ Encrypt/Decrypt\ a\ remote
|
||
\ \ \ \\\ "crypt"
|
||
\ 6\ /\ Google\ Cloud\ Storage\ (this\ is\ not\ Google\ Drive)
|
||
\ \ \ \\\ "google\ cloud\ storage"
|
||
\ 7\ /\ Google\ Drive
|
||
\ \ \ \\\ "drive"
|
||
\ 8\ /\ Hubic
|
||
\ \ \ \\\ "hubic"
|
||
\ 9\ /\ Local\ Disk
|
||
\ \ \ \\\ "local"
|
||
10\ /\ Microsoft\ OneDrive
|
||
\ \ \ \\\ "onedrive"
|
||
11\ /\ Openstack\ Swift\ (Rackspace\ Cloud\ Files,\ Memset\ Memstore,\ OVH)
|
||
\ \ \ \\\ "swift"
|
||
12\ /\ SSH/SFTP\ Connection
|
||
\ \ \ \\\ "sftp"
|
||
13\ /\ Yandex\ Disk
|
||
\ \ \ \\\ "yandex"
|
||
Storage>\ 2
|
||
Get\ AWS\ credentials\ from\ runtime\ (environment\ variables\ or\ EC2/ECS\ meta\ data\ if\ no\ env\ vars).\ Only\ applies\ if\ access_key_id\ and\ secret_access_key\ is\ blank.
|
||
Choose\ a\ number\ from\ below,\ or\ type\ in\ your\ own\ value
|
||
\ 1\ /\ Enter\ AWS\ credentials\ in\ the\ next\ step
|
||
\ \ \ \\\ "false"
|
||
\ 2\ /\ Get\ AWS\ credentials\ from\ the\ environment\ (env\ vars\ or\ IAM)
|
||
\ \ \ \\\ "true"
|
||
env_auth>\ 1
|
||
AWS\ Access\ Key\ ID\ \-\ leave\ blank\ for\ anonymous\ access\ or\ runtime\ credentials.
|
||
access_key_id>\ access_key
|
||
AWS\ Secret\ Access\ Key\ (password)\ \-\ leave\ blank\ for\ anonymous\ access\ or\ runtime\ credentials.
|
||
secret_access_key>\ secret_key
|
||
Region\ to\ connect\ to.
|
||
Choose\ a\ number\ from\ below,\ or\ type\ in\ your\ own\ value
|
||
\ \ \ /\ The\ default\ endpoint\ \-\ a\ good\ choice\ if\ you\ are\ unsure.
|
||
\ 1\ |\ US\ Region,\ Northern\ Virginia\ or\ Pacific\ Northwest.
|
||
\ \ \ |\ Leave\ location\ constraint\ empty.
|
||
\ \ \ \\\ "us\-east\-1"
|
||
\ \ \ /\ US\ West\ (Oregon)\ Region
|
||
\ 2\ |\ Needs\ location\ constraint\ us\-west\-2.
|
||
\ \ \ \\\ "us\-west\-2"
|
||
\ \ \ /\ US\ West\ (Northern\ California)\ Region
|
||
\ 3\ |\ Needs\ location\ constraint\ us\-west\-1.
|
||
\ \ \ \\\ "us\-west\-1"
|
||
\ \ \ /\ EU\ (Ireland)\ Region\ Region
|
||
\ 4\ |\ Needs\ location\ constraint\ EU\ or\ eu\-west\-1.
|
||
\ \ \ \\\ "eu\-west\-1"
|
||
\ \ \ /\ EU\ (Frankfurt)\ Region
|
||
\ 5\ |\ Needs\ location\ constraint\ eu\-central\-1.
|
||
\ \ \ \\\ "eu\-central\-1"
|
||
\ \ \ /\ Asia\ Pacific\ (Singapore)\ Region
|
||
\ 6\ |\ Needs\ location\ constraint\ ap\-southeast\-1.
|
||
\ \ \ \\\ "ap\-southeast\-1"
|
||
\ \ \ /\ Asia\ Pacific\ (Sydney)\ Region
|
||
\ 7\ |\ Needs\ location\ constraint\ ap\-southeast\-2.
|
||
\ \ \ \\\ "ap\-southeast\-2"
|
||
\ \ \ /\ Asia\ Pacific\ (Tokyo)\ Region
|
||
\ 8\ |\ Needs\ location\ constraint\ ap\-northeast\-1.
|
||
\ \ \ \\\ "ap\-northeast\-1"
|
||
\ \ \ /\ Asia\ Pacific\ (Seoul)
|
||
\ 9\ |\ Needs\ location\ constraint\ ap\-northeast\-2.
|
||
\ \ \ \\\ "ap\-northeast\-2"
|
||
\ \ \ /\ Asia\ Pacific\ (Mumbai)
|
||
10\ |\ Needs\ location\ constraint\ ap\-south\-1.
|
||
\ \ \ \\\ "ap\-south\-1"
|
||
\ \ \ /\ South\ America\ (Sao\ Paulo)\ Region
|
||
11\ |\ Needs\ location\ constraint\ sa\-east\-1.
|
||
\ \ \ \\\ "sa\-east\-1"
|
||
\ \ \ /\ If\ using\ an\ S3\ clone\ that\ only\ understands\ v2\ signatures
|
||
12\ |\ eg\ Ceph/Dreamhost
|
||
\ \ \ |\ set\ this\ and\ make\ sure\ you\ set\ the\ endpoint.
|
||
\ \ \ \\\ "other\-v2\-signature"
|
||
\ \ \ /\ If\ using\ an\ S3\ clone\ that\ understands\ v4\ signatures\ set\ this
|
||
13\ |\ and\ make\ sure\ you\ set\ the\ endpoint.
|
||
\ \ \ \\\ "other\-v4\-signature"
|
||
region>\ 1
|
||
Endpoint\ for\ S3\ API.
|
||
Leave\ blank\ if\ using\ AWS\ to\ use\ the\ default\ endpoint\ for\ the\ region.
|
||
Specify\ if\ using\ an\ S3\ clone\ such\ as\ Ceph.
|
||
endpoint>
|
||
Location\ constraint\ \-\ must\ be\ set\ to\ match\ the\ Region.\ Used\ when\ creating\ buckets\ only.
|
||
Choose\ a\ number\ from\ below,\ or\ type\ in\ your\ own\ value
|
||
\ 1\ /\ Empty\ for\ US\ Region,\ Northern\ Virginia\ or\ Pacific\ Northwest.
|
||
\ \ \ \\\ ""
|
||
\ 2\ /\ US\ West\ (Oregon)\ Region.
|
||
\ \ \ \\\ "us\-west\-2"
|
||
\ 3\ /\ US\ West\ (Northern\ California)\ Region.
|
||
\ \ \ \\\ "us\-west\-1"
|
||
\ 4\ /\ EU\ (Ireland)\ Region.
|
||
\ \ \ \\\ "eu\-west\-1"
|
||
\ 5\ /\ EU\ Region.
|
||
\ \ \ \\\ "EU"
|
||
\ 6\ /\ Asia\ Pacific\ (Singapore)\ Region.
|
||
\ \ \ \\\ "ap\-southeast\-1"
|
||
\ 7\ /\ Asia\ Pacific\ (Sydney)\ Region.
|
||
\ \ \ \\\ "ap\-southeast\-2"
|
||
\ 8\ /\ Asia\ Pacific\ (Tokyo)\ Region.
|
||
\ \ \ \\\ "ap\-northeast\-1"
|
||
\ 9\ /\ Asia\ Pacific\ (Seoul)
|
||
\ \ \ \\\ "ap\-northeast\-2"
|
||
10\ /\ Asia\ Pacific\ (Mumbai)
|
||
\ \ \ \\\ "ap\-south\-1"
|
||
11\ /\ South\ America\ (Sao\ Paulo)\ Region.
|
||
\ \ \ \\\ "sa\-east\-1"
|
||
location_constraint>\ 1
|
||
Canned\ ACL\ used\ when\ creating\ buckets\ and/or\ storing\ objects\ in\ S3.
|
||
For\ more\ info\ visit\ https://docs.aws.amazon.com/AmazonS3/latest/dev/acl\-overview.html#canned\-acl
|
||
Choose\ a\ number\ from\ below,\ or\ type\ in\ your\ own\ value
|
||
\ 1\ /\ Owner\ gets\ FULL_CONTROL.\ No\ one\ else\ has\ access\ rights\ (default).
|
||
\ \ \ \\\ "private"
|
||
\ 2\ /\ Owner\ gets\ FULL_CONTROL.\ The\ AllUsers\ group\ gets\ READ\ access.
|
||
\ \ \ \\\ "public\-read"
|
||
\ \ \ /\ Owner\ gets\ FULL_CONTROL.\ The\ AllUsers\ group\ gets\ READ\ and\ WRITE\ access.
|
||
\ 3\ |\ Granting\ this\ on\ a\ bucket\ is\ generally\ not\ recommended.
|
||
\ \ \ \\\ "public\-read\-write"
|
||
\ 4\ /\ Owner\ gets\ FULL_CONTROL.\ The\ AuthenticatedUsers\ group\ gets\ READ\ access.
|
||
\ \ \ \\\ "authenticated\-read"
|
||
\ \ \ /\ Object\ owner\ gets\ FULL_CONTROL.\ Bucket\ owner\ gets\ READ\ access.
|
||
\ 5\ |\ If\ you\ specify\ this\ canned\ ACL\ when\ creating\ a\ bucket,\ Amazon\ S3\ ignores\ it.
|
||
\ \ \ \\\ "bucket\-owner\-read"
|
||
\ \ \ /\ Both\ the\ object\ owner\ and\ the\ bucket\ owner\ get\ FULL_CONTROL\ over\ the\ object.
|
||
\ 6\ |\ If\ you\ specify\ this\ canned\ ACL\ when\ creating\ a\ bucket,\ Amazon\ S3\ ignores\ it.
|
||
\ \ \ \\\ "bucket\-owner\-full\-control"
|
||
acl>\ private
|
||
The\ server\-side\ encryption\ algorithm\ used\ when\ storing\ this\ object\ in\ S3.
|
||
Choose\ a\ number\ from\ below,\ or\ type\ in\ your\ own\ value
|
||
\ 1\ /\ None
|
||
\ \ \ \\\ ""
|
||
\ 2\ /\ AES256
|
||
\ \ \ \\\ "AES256"
|
||
server_side_encryption>
|
||
The\ storage\ class\ to\ use\ when\ storing\ objects\ in\ S3.
|
||
Choose\ a\ number\ from\ below,\ or\ type\ in\ your\ own\ value
|
||
\ 1\ /\ Default
|
||
\ \ \ \\\ ""
|
||
\ 2\ /\ Standard\ storage\ class
|
||
\ \ \ \\\ "STANDARD"
|
||
\ 3\ /\ Reduced\ redundancy\ storage\ class
|
||
\ \ \ \\\ "REDUCED_REDUNDANCY"
|
||
\ 4\ /\ Standard\ Infrequent\ Access\ storage\ class
|
||
\ \ \ \\\ "STANDARD_IA"
|
||
storage_class>
|
||
Remote\ config
|
||
\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-
|
||
[remote]
|
||
env_auth\ =\ false
|
||
access_key_id\ =\ access_key
|
||
secret_access_key\ =\ secret_key
|
||
region\ =\ us\-east\-1
|
||
endpoint\ =
|
||
location_constraint\ =
|
||
acl\ =\ private
|
||
server_side_encryption\ =
|
||
storage_class\ =
|
||
\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-
|
||
y)\ Yes\ this\ is\ OK
|
||
e)\ Edit\ this\ remote
|
||
d)\ Delete\ this\ remote
|
||
y/e/d>\ y
|
||
\f[]
|
||
.fi
|
||
.PP
|
||
This remote is called \f[C]remote\f[] and can now be used like this
|
||
.PP
|
||
See all buckets
|
||
.IP
|
||
.nf
|
||
\f[C]
|
||
rclone\ lsd\ remote:
|
||
\f[]
|
||
.fi
|
||
.PP
|
||
Make a new bucket
|
||
.IP
|
||
.nf
|
||
\f[C]
|
||
rclone\ mkdir\ remote:bucket
|
||
\f[]
|
||
.fi
|
||
.PP
|
||
List the contents of a bucket
|
||
.IP
|
||
.nf
|
||
\f[C]
|
||
rclone\ ls\ remote:bucket
|
||
\f[]
|
||
.fi
|
||
.PP
|
||
Sync \f[C]/home/local/directory\f[] to the remote bucket, deleting any
|
||
excess files in the bucket.
|
||
.IP
|
||
.nf
|
||
\f[C]
|
||
rclone\ sync\ /home/local/directory\ remote:bucket
|
||
\f[]
|
||
.fi
|
||
.SS \-\-fast\-list
|
||
.PP
|
||
This remote supports \f[C]\-\-fast\-list\f[] which allows you to use
|
||
fewer transactions in exchange for more memory.
|
||
See the rclone docs (/docs/#fast-list) for more details.
|
||
.SS Modified time
|
||
.PP
|
||
The modified time is stored as metadata on the object as
|
||
\f[C]X\-Amz\-Meta\-Mtime\f[] as floating point since the epoch accurate
|
||
to 1 ns.
|
||
.SS Multipart uploads
|
||
.PP
|
||
rclone supports multipart uploads with S3 which means that it can upload
|
||
files bigger than 5GB.
|
||
Note that files uploaded with multipart upload don\[aq]t have an MD5SUM.
|
||
.SS Buckets and Regions
|
||
.PP
|
||
With Amazon S3 you can list buckets (\f[C]rclone\ lsd\f[]) using any
|
||
region, but you can only access the content of a bucket from the region
|
||
it was created in.
|
||
If you attempt to access a bucket from the wrong region, you will get an
|
||
error,
|
||
\f[C]incorrect\ region,\ the\ bucket\ is\ not\ in\ \[aq]XXX\[aq]\ region\f[].
|
||
.SS Authentication
|
||
.PP
|
||
There are two ways to supply \f[C]rclone\f[] with a set of AWS
|
||
credentials.
|
||
In order of precedence:
|
||
.IP \[bu] 2
|
||
Directly in the rclone configuration file (as configured by
|
||
\f[C]rclone\ config\f[])
|
||
.IP \[bu] 2
|
||
set \f[C]access_key_id\f[] and \f[C]secret_access_key\f[].
|
||
\f[C]session_token\f[] can be optionally set when using AWS STS.
|
||
.IP \[bu] 2
|
||
Runtime configuration:
|
||
.IP \[bu] 2
|
||
set \f[C]env_auth\f[] to \f[C]true\f[] in the config file
|
||
.IP \[bu] 2
|
||
Exporting the following environment variables before running
|
||
\f[C]rclone\f[]
|
||
.RS 2
|
||
.IP \[bu] 2
|
||
Access Key ID: \f[C]AWS_ACCESS_KEY_ID\f[] or \f[C]AWS_ACCESS_KEY\f[]
|
||
.IP \[bu] 2
|
||
Secret Access Key: \f[C]AWS_SECRET_ACCESS_KEY\f[] or
|
||
\f[C]AWS_SECRET_KEY\f[]
|
||
.IP \[bu] 2
|
||
Session Token: \f[C]AWS_SESSION_TOKEN\f[]
|
||
.RE
|
||
.IP \[bu] 2
|
||
Running \f[C]rclone\f[] in an ECS task with an IAM role
|
||
.IP \[bu] 2
|
||
Running \f[C]rclone\f[] on an EC2 instance with an IAM role
|
||
.PP
|
||
If none of these option actually end up providing \f[C]rclone\f[] with
|
||
AWS credentials then S3 interaction will be non\-authenticated (see
|
||
below).
|
||
.SS S3 Permissions
|
||
.PP
|
||
When using the \f[C]sync\f[] subcommand of \f[C]rclone\f[] the following
|
||
minimum permissions are required to be available on the bucket being
|
||
written to:
|
||
.IP \[bu] 2
|
||
\f[C]ListBucket\f[]
|
||
.IP \[bu] 2
|
||
\f[C]DeleteObject\f[]
|
||
.IP \[bu] 2
|
||
\f[C]GetObject\f[]
|
||
.IP \[bu] 2
|
||
\f[C]PutObject\f[]
|
||
.IP \[bu] 2
|
||
\f[C]PutObjectACL\f[]
|
||
.PP
|
||
Example policy:
|
||
.IP
|
||
.nf
|
||
\f[C]
|
||
{
|
||
\ \ \ \ "Version":\ "2012\-10\-17",
|
||
\ \ \ \ "Statement":\ [
|
||
\ \ \ \ \ \ \ \ {
|
||
\ \ \ \ \ \ \ \ \ \ \ \ "Effect":\ "Allow",
|
||
\ \ \ \ \ \ \ \ \ \ \ \ "Principal":\ {
|
||
\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ "AWS":\ "arn:aws:iam::USER_SID:user/USER_NAME"
|
||
\ \ \ \ \ \ \ \ \ \ \ \ },
|
||
\ \ \ \ \ \ \ \ \ \ \ \ "Action":\ [
|
||
\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ "s3:ListBucket",
|
||
\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ "s3:DeleteObject",
|
||
\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ "s3:GetObject",
|
||
\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ "s3:PutObject",
|
||
\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ "s3:PutObjectAcl"
|
||
\ \ \ \ \ \ \ \ \ \ \ \ ],
|
||
\ \ \ \ \ \ \ \ \ \ \ \ "Resource":\ [
|
||
\ \ \ \ \ \ \ \ \ \ \ \ \ \ "arn:aws:s3:::BUCKET_NAME/*",
|
||
\ \ \ \ \ \ \ \ \ \ \ \ \ \ "arn:aws:s3:::BUCKET_NAME"
|
||
\ \ \ \ \ \ \ \ \ \ \ \ ]
|
||
\ \ \ \ \ \ \ \ }
|
||
\ \ \ \ ]
|
||
}
|
||
\f[]
|
||
.fi
|
||
.PP
|
||
Notes on above:
|
||
.IP "1." 3
|
||
This is a policy that can be used when creating bucket.
|
||
It assumes that \f[C]USER_NAME\f[] has been created.
|
||
.IP "2." 3
|
||
The Resource entry must include both resource ARNs, as one implies the
|
||
bucket and the other implies the bucket\[aq]s objects.
|
||
.PP
|
||
For reference, here\[aq]s an Ansible
|
||
script (https://gist.github.com/ebridges/ebfc9042dd7c756cd101cfa807b7ae2b)
|
||
that will generate one or more buckets that will work with
|
||
\f[C]rclone\ sync\f[].
|
||
.SS Glacier
|
||
.PP
|
||
You can transition objects to glacier storage using a lifecycle
|
||
policy (http://docs.aws.amazon.com/AmazonS3/latest/user-guide/create-lifecycle.html).
|
||
The bucket can still be synced or copied into normally, but if rclone
|
||
tries to access the data you will see an error like below.
|
||
.IP
|
||
.nf
|
||
\f[C]
|
||
2017/09/11\ 19:07:43\ Failed\ to\ sync:\ failed\ to\ open\ source\ object:\ Object\ in\ GLACIER,\ restore\ first:\ path/to/file
|
||
\f[]
|
||
.fi
|
||
.PP
|
||
In this case you need to
|
||
restore (http://docs.aws.amazon.com/AmazonS3/latest/user-guide/restore-archived-objects.html)
|
||
the object(s) in question before using rclone.
|
||
.SS Specific options
|
||
.PP
|
||
Here are the command line options specific to this cloud storage system.
|
||
.SS \-\-s3\-acl=STRING
|
||
.PP
|
||
Canned ACL used when creating buckets and/or storing objects in S3.
|
||
.PP
|
||
For more info visit the canned ACL
|
||
docs (https://docs.aws.amazon.com/AmazonS3/latest/dev/acl-overview.html#canned-acl).
|
||
.SS \-\-s3\-storage\-class=STRING
|
||
.PP
|
||
Storage class to upload new objects with.
|
||
.PP
|
||
Available options include:
|
||
.IP \[bu] 2
|
||
STANDARD \- default storage class
|
||
.IP \[bu] 2
|
||
STANDARD_IA \- for less frequently accessed data (e.g backups)
|
||
.IP \[bu] 2
|
||
REDUCED_REDUNDANCY (only for noncritical, reproducible data, has lower
|
||
redundancy)
|
||
.SS Anonymous access to public buckets
|
||
.PP
|
||
If you want to use rclone to access a public bucket, configure with a
|
||
blank \f[C]access_key_id\f[] and \f[C]secret_access_key\f[].
|
||
Eg
|
||
.IP
|
||
.nf
|
||
\f[C]
|
||
No\ remotes\ found\ \-\ make\ a\ new\ one
|
||
n)\ New\ remote
|
||
q)\ Quit\ config
|
||
n/q>\ n
|
||
name>\ anons3
|
||
What\ type\ of\ source\ is\ it?
|
||
Choose\ a\ number\ from\ below
|
||
\ 1)\ amazon\ cloud\ drive
|
||
\ 2)\ b2
|
||
\ 3)\ drive
|
||
\ 4)\ dropbox
|
||
\ 5)\ google\ cloud\ storage
|
||
\ 6)\ swift
|
||
\ 7)\ hubic
|
||
\ 8)\ local
|
||
\ 9)\ onedrive
|
||
10)\ s3
|
||
11)\ yandex
|
||
type>\ 10
|
||
Get\ AWS\ credentials\ from\ runtime\ (environment\ variables\ or\ EC2/ECS\ meta\ data\ if\ no\ env\ vars).\ Only\ applies\ if\ access_key_id\ and\ secret_access_key\ is\ blank.
|
||
Choose\ a\ number\ from\ below,\ or\ type\ in\ your\ own\ value
|
||
\ *\ Enter\ AWS\ credentials\ in\ the\ next\ step
|
||
\ 1)\ false
|
||
\ *\ Get\ AWS\ credentials\ from\ the\ environment\ (env\ vars\ or\ IAM)
|
||
\ 2)\ true
|
||
env_auth>\ 1
|
||
AWS\ Access\ Key\ ID\ \-\ leave\ blank\ for\ anonymous\ access\ or\ runtime\ credentials.
|
||
access_key_id>
|
||
AWS\ Secret\ Access\ Key\ (password)\ \-\ leave\ blank\ for\ anonymous\ access\ or\ runtime\ credentials.
|
||
secret_access_key>
|
||
\&...
|
||
\f[]
|
||
.fi
|
||
.PP
|
||
Then use it as normal with the name of the public bucket, eg
|
||
.IP
|
||
.nf
|
||
\f[C]
|
||
rclone\ lsd\ anons3:1000genomes
|
||
\f[]
|
||
.fi
|
||
.PP
|
||
You will be able to list and copy data but not upload it.
|
||
.SS Ceph
|
||
.PP
|
||
Ceph is an object storage system which presents an Amazon S3 interface.
|
||
.PP
|
||
To use rclone with ceph, you need to set the following parameters in the
|
||
config.
|
||
.IP
|
||
.nf
|
||
\f[C]
|
||
access_key_id\ =\ Whatever
|
||
secret_access_key\ =\ Whatever
|
||
endpoint\ =\ https://ceph.endpoint.goes.here/
|
||
region\ =\ other\-v2\-signature
|
||
\f[]
|
||
.fi
|
||
.PP
|
||
Note also that Ceph sometimes puts \f[C]/\f[] in the passwords it gives
|
||
users.
|
||
If you read the secret access key using the command line tools you will
|
||
get a JSON blob with the \f[C]/\f[] escaped as \f[C]\\/\f[].
|
||
Make sure you only write \f[C]/\f[] in the secret access key.
|
||
.PP
|
||
Eg the dump from Ceph looks something like this (irrelevant keys
|
||
removed).
|
||
.IP
|
||
.nf
|
||
\f[C]
|
||
{
|
||
\ \ \ \ "user_id":\ "xxx",
|
||
\ \ \ \ "display_name":\ "xxxx",
|
||
\ \ \ \ "keys":\ [
|
||
\ \ \ \ \ \ \ \ {
|
||
\ \ \ \ \ \ \ \ \ \ \ \ "user":\ "xxx",
|
||
\ \ \ \ \ \ \ \ \ \ \ \ "access_key":\ "xxxxxx",
|
||
\ \ \ \ \ \ \ \ \ \ \ \ "secret_key":\ "xxxxxx\\/xxxx"
|
||
\ \ \ \ \ \ \ \ }
|
||
\ \ \ \ ],
|
||
}
|
||
\f[]
|
||
.fi
|
||
.PP
|
||
Because this is a json dump, it is encoding the \f[C]/\f[] as
|
||
\f[C]\\/\f[], so if you use the secret key as \f[C]xxxxxx/xxxx\f[] it
|
||
will work fine.
|
||
.SS DigitalOcean Spaces
|
||
.PP
|
||
Spaces (https://www.digitalocean.com/products/object-storage/) is an
|
||
S3\-interoperable (https://developers.digitalocean.com/documentation/spaces/)
|
||
object storage service from cloud provider DigitalOcean.
|
||
.PP
|
||
To connect to DigitalOcean Spaces you will need an access key and secret
|
||
key.
|
||
These can be retrieved on the "Applications &
|
||
API (https://cloud.digitalocean.com/settings/api/tokens)" page of the
|
||
DigitalOcean control panel.
|
||
They will be needed when promted by \f[C]rclone\ config\f[] for your
|
||
\f[C]access_key_id\f[] and \f[C]secret_access_key\f[].
|
||
.PP
|
||
When prompted for a \f[C]region\f[] or \f[C]location_constraint\f[],
|
||
press enter to use the default value.
|
||
The region must be included in the \f[C]endpoint\f[] setting (e.g.
|
||
\f[C]nyc3.digitaloceanspaces.com\f[]).
|
||
The defualt values can be used for other settings.
|
||
.PP
|
||
Going through the whole process of creating a new remote by running
|
||
\f[C]rclone\ config\f[], each prompt should be answered as shown below:
|
||
.IP
|
||
.nf
|
||
\f[C]
|
||
Storage>\ 2
|
||
env_auth>\ 1
|
||
access_key_id>\ YOUR_ACCESS_KEY
|
||
secret_access_key>\ YOUR_SECRET_KEY
|
||
region>\
|
||
endpoint>\ nyc3.digitaloceanspaces.com
|
||
location_constraint>\
|
||
acl>\
|
||
storage_class>\
|
||
\f[]
|
||
.fi
|
||
.PP
|
||
The resulting configuration file should look like:
|
||
.IP
|
||
.nf
|
||
\f[C]
|
||
[spaces]
|
||
type\ =\ s3
|
||
env_auth\ =\ false
|
||
access_key_id\ =\ YOUR_ACCESS_KEY
|
||
secret_access_key\ =\ YOUR_SECRET_KEY
|
||
region\ =\
|
||
endpoint\ =\ nyc3.digitaloceanspaces.com
|
||
location_constraint\ =\
|
||
acl\ =\
|
||
server_side_encryption\ =\
|
||
storage_class\ =\
|
||
\f[]
|
||
.fi
|
||
.PP
|
||
Once configured, you can create a new Space and begin copying files.
|
||
For example:
|
||
.IP
|
||
.nf
|
||
\f[C]
|
||
rclone\ mkdir\ spaces:my\-new\-space
|
||
rclone\ copy\ /path/to/files\ spaces:my\-new\-space
|
||
\f[]
|
||
.fi
|
||
.SS Minio
|
||
.PP
|
||
Minio (https://minio.io/) is an object storage server built for cloud
|
||
application developers and devops.
|
||
.PP
|
||
It is very easy to install and provides an S3 compatible server which
|
||
can be used by rclone.
|
||
.PP
|
||
To use it, install Minio following the instructions
|
||
here (https://docs.minio.io/docs/minio-quickstart-guide).
|
||
.PP
|
||
When it configures itself Minio will print something like this
|
||
.IP
|
||
.nf
|
||
\f[C]
|
||
Endpoint:\ \ http://192.168.1.106:9000\ \ http://172.23.0.1:9000
|
||
AccessKey:\ USWUXHGYZQYFYFFIT3RE
|
||
SecretKey:\ MOJRH0mkL1IPauahWITSVvyDrQbEEIwljvmxdq03
|
||
Region:\ \ \ \ us\-east\-1
|
||
SQS\ ARNs:\ \ arn:minio:sqs:us\-east\-1:1:redis\ arn:minio:sqs:us\-east\-1:2:redis
|
||
|
||
Browser\ Access:
|
||
\ \ \ http://192.168.1.106:9000\ \ http://172.23.0.1:9000
|
||
|
||
Command\-line\ Access:\ https://docs.minio.io/docs/minio\-client\-quickstart\-guide
|
||
\ \ \ $\ mc\ config\ host\ add\ myminio\ http://192.168.1.106:9000\ USWUXHGYZQYFYFFIT3RE\ MOJRH0mkL1IPauahWITSVvyDrQbEEIwljvmxdq03
|
||
|
||
Object\ API\ (Amazon\ S3\ compatible):
|
||
\ \ \ Go:\ \ \ \ \ \ \ \ \ https://docs.minio.io/docs/golang\-client\-quickstart\-guide
|
||
\ \ \ Java:\ \ \ \ \ \ \ https://docs.minio.io/docs/java\-client\-quickstart\-guide
|
||
\ \ \ Python:\ \ \ \ \ https://docs.minio.io/docs/python\-client\-quickstart\-guide
|
||
\ \ \ JavaScript:\ https://docs.minio.io/docs/javascript\-client\-quickstart\-guide
|
||
\ \ \ .NET:\ \ \ \ \ \ \ https://docs.minio.io/docs/dotnet\-client\-quickstart\-guide
|
||
|
||
Drive\ Capacity:\ 26\ GiB\ Free,\ 165\ GiB\ Total
|
||
\f[]
|
||
.fi
|
||
.PP
|
||
These details need to go into \f[C]rclone\ config\f[] like this.
|
||
Note that it is important to put the region in as stated above.
|
||
.IP
|
||
.nf
|
||
\f[C]
|
||
env_auth>\ 1
|
||
access_key_id>\ USWUXHGYZQYFYFFIT3RE
|
||
secret_access_key>\ MOJRH0mkL1IPauahWITSVvyDrQbEEIwljvmxdq03
|
||
region>\ us\-east\-1
|
||
endpoint>\ http://192.168.1.106:9000
|
||
location_constraint>
|
||
server_side_encryption>
|
||
\f[]
|
||
.fi
|
||
.PP
|
||
Which makes the config file look like this
|
||
.IP
|
||
.nf
|
||
\f[C]
|
||
[minio]
|
||
env_auth\ =\ false
|
||
access_key_id\ =\ USWUXHGYZQYFYFFIT3RE
|
||
secret_access_key\ =\ MOJRH0mkL1IPauahWITSVvyDrQbEEIwljvmxdq03
|
||
region\ =\ us\-east\-1
|
||
endpoint\ =\ http://192.168.1.106:9000
|
||
location_constraint\ =
|
||
server_side_encryption\ =
|
||
\f[]
|
||
.fi
|
||
.PP
|
||
So once set up, for example to copy files into a bucket
|
||
.IP
|
||
.nf
|
||
\f[C]
|
||
rclone\ copy\ /path/to/files\ minio:bucket
|
||
\f[]
|
||
.fi
|
||
.SS Wasabi
|
||
.PP
|
||
Wasabi (https://wasabi.com) is a cloud\-based object storage service for
|
||
a broad range of applications and use cases.
|
||
Wasabi is designed for individuals and organizations that require a
|
||
high\-performance, reliable, and secure data storage infrastructure at
|
||
minimal cost.
|
||
.PP
|
||
Wasabi provides an S3 interface which can be configured for use with
|
||
rclone like this.
|
||
.IP
|
||
.nf
|
||
\f[C]
|
||
No\ remotes\ found\ \-\ make\ a\ new\ one
|
||
n)\ New\ remote
|
||
s)\ Set\ configuration\ password
|
||
n/s>\ n
|
||
name>\ wasabi
|
||
Type\ of\ storage\ to\ configure.
|
||
Choose\ a\ number\ from\ below,\ or\ type\ in\ your\ own\ value
|
||
\ 1\ /\ Amazon\ Drive
|
||
\ \ \ \\\ "amazon\ cloud\ drive"
|
||
\ 2\ /\ Amazon\ S3\ (also\ Dreamhost,\ Ceph,\ Minio)
|
||
\ \ \ \\\ "s3"
|
||
[snip]
|
||
Storage>\ s3
|
||
Get\ AWS\ credentials\ from\ runtime\ (environment\ variables\ or\ EC2/ECS\ meta\ data\ if\ no\ env\ vars).\ Only\ applies\ if\ access_key_id\ and\ secret_access_key\ is\ blank.
|
||
Choose\ a\ number\ from\ below,\ or\ type\ in\ your\ own\ value
|
||
\ 1\ /\ Enter\ AWS\ credentials\ in\ the\ next\ step
|
||
\ \ \ \\\ "false"
|
||
\ 2\ /\ Get\ AWS\ credentials\ from\ the\ environment\ (env\ vars\ or\ IAM)
|
||
\ \ \ \\\ "true"
|
||
env_auth>\ 1
|
||
AWS\ Access\ Key\ ID\ \-\ leave\ blank\ for\ anonymous\ access\ or\ runtime\ credentials.
|
||
access_key_id>\ YOURACCESSKEY
|
||
AWS\ Secret\ Access\ Key\ (password)\ \-\ leave\ blank\ for\ anonymous\ access\ or\ runtime\ credentials.
|
||
secret_access_key>\ YOURSECRETACCESSKEY
|
||
Region\ to\ connect\ to.
|
||
Choose\ a\ number\ from\ below,\ or\ type\ in\ your\ own\ value
|
||
\ \ \ /\ The\ default\ endpoint\ \-\ a\ good\ choice\ if\ you\ are\ unsure.
|
||
\ 1\ |\ US\ Region,\ Northern\ Virginia\ or\ Pacific\ Northwest.
|
||
\ \ \ |\ Leave\ location\ constraint\ empty.
|
||
\ \ \ \\\ "us\-east\-1"
|
||
[snip]
|
||
region>\ us\-east\-1
|
||
Endpoint\ for\ S3\ API.
|
||
Leave\ blank\ if\ using\ AWS\ to\ use\ the\ default\ endpoint\ for\ the\ region.
|
||
Specify\ if\ using\ an\ S3\ clone\ such\ as\ Ceph.
|
||
endpoint>\ s3.wasabisys.com
|
||
Location\ constraint\ \-\ must\ be\ set\ to\ match\ the\ Region.\ Used\ when\ creating\ buckets\ only.
|
||
Choose\ a\ number\ from\ below,\ or\ type\ in\ your\ own\ value
|
||
\ 1\ /\ Empty\ for\ US\ Region,\ Northern\ Virginia\ or\ Pacific\ Northwest.
|
||
\ \ \ \\\ ""
|
||
[snip]
|
||
location_constraint>\
|
||
Canned\ ACL\ used\ when\ creating\ buckets\ and/or\ storing\ objects\ in\ S3.
|
||
For\ more\ info\ visit\ https://docs.aws.amazon.com/AmazonS3/latest/dev/acl\-overview.html#canned\-acl
|
||
Choose\ a\ number\ from\ below,\ or\ type\ in\ your\ own\ value
|
||
\ 1\ /\ Owner\ gets\ FULL_CONTROL.\ No\ one\ else\ has\ access\ rights\ (default).
|
||
\ \ \ \\\ "private"
|
||
[snip]
|
||
acl>\
|
||
The\ server\-side\ encryption\ algorithm\ used\ when\ storing\ this\ object\ in\ S3.
|
||
Choose\ a\ number\ from\ below,\ or\ type\ in\ your\ own\ value
|
||
\ 1\ /\ None
|
||
\ \ \ \\\ ""
|
||
\ 2\ /\ AES256
|
||
\ \ \ \\\ "AES256"
|
||
server_side_encryption>\
|
||
The\ storage\ class\ to\ use\ when\ storing\ objects\ in\ S3.
|
||
Choose\ a\ number\ from\ below,\ or\ type\ in\ your\ own\ value
|
||
\ 1\ /\ Default
|
||
\ \ \ \\\ ""
|
||
\ 2\ /\ Standard\ storage\ class
|
||
\ \ \ \\\ "STANDARD"
|
||
\ 3\ /\ Reduced\ redundancy\ storage\ class
|
||
\ \ \ \\\ "REDUCED_REDUNDANCY"
|
||
\ 4\ /\ Standard\ Infrequent\ Access\ storage\ class
|
||
\ \ \ \\\ "STANDARD_IA"
|
||
storage_class>\
|
||
Remote\ config
|
||
\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-
|
||
[wasabi]
|
||
env_auth\ =\ false
|
||
access_key_id\ =\ YOURACCESSKEY
|
||
secret_access_key\ =\ YOURSECRETACCESSKEY
|
||
region\ =\ us\-east\-1
|
||
endpoint\ =\ s3.wasabisys.com
|
||
location_constraint\ =\
|
||
acl\ =\
|
||
server_side_encryption\ =\
|
||
storage_class\ =\
|
||
\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-
|
||
y)\ Yes\ this\ is\ OK
|
||
e)\ Edit\ this\ remote
|
||
d)\ Delete\ this\ remote
|
||
y/e/d>\ y
|
||
\f[]
|
||
.fi
|
||
.PP
|
||
This will leave the config file looking like this.
|
||
.IP
|
||
.nf
|
||
\f[C]
|
||
[wasabi]
|
||
env_auth\ =\ false
|
||
access_key_id\ =\ YOURACCESSKEY
|
||
secret_access_key\ =\ YOURSECRETACCESSKEY
|
||
region\ =\ us\-east\-1
|
||
endpoint\ =\ s3.wasabisys.com
|
||
location_constraint\ =\
|
||
acl\ =\
|
||
server_side_encryption\ =\
|
||
storage_class\ =\
|
||
\f[]
|
||
.fi
|
||
.SS Backblaze B2
|
||
.PP
|
||
B2 is Backblaze\[aq]s cloud storage
|
||
system (https://www.backblaze.com/b2/).
|
||
.PP
|
||
Paths are specified as \f[C]remote:bucket\f[] (or \f[C]remote:\f[] for
|
||
the \f[C]lsd\f[] command.) You may put subdirectories in too, eg
|
||
\f[C]remote:bucket/path/to/dir\f[].
|
||
.PP
|
||
Here is an example of making a b2 configuration.
|
||
First run
|
||
.IP
|
||
.nf
|
||
\f[C]
|
||
rclone\ config
|
||
\f[]
|
||
.fi
|
||
.PP
|
||
This will guide you through an interactive setup process.
|
||
You will need your account number (a short hex number) and key (a long
|
||
hex number) which you can get from the b2 control panel.
|
||
.IP
|
||
.nf
|
||
\f[C]
|
||
No\ remotes\ found\ \-\ make\ a\ new\ one
|
||
n)\ New\ remote
|
||
q)\ Quit\ config
|
||
n/q>\ n
|
||
name>\ remote
|
||
Type\ of\ storage\ to\ configure.
|
||
Choose\ a\ number\ from\ below,\ or\ type\ in\ your\ own\ value
|
||
\ 1\ /\ Amazon\ Drive
|
||
\ \ \ \\\ "amazon\ cloud\ drive"
|
||
\ 2\ /\ Amazon\ S3\ (also\ Dreamhost,\ Ceph,\ Minio)
|
||
\ \ \ \\\ "s3"
|
||
\ 3\ /\ Backblaze\ B2
|
||
\ \ \ \\\ "b2"
|
||
\ 4\ /\ Dropbox
|
||
\ \ \ \\\ "dropbox"
|
||
\ 5\ /\ Encrypt/Decrypt\ a\ remote
|
||
\ \ \ \\\ "crypt"
|
||
\ 6\ /\ Google\ Cloud\ Storage\ (this\ is\ not\ Google\ Drive)
|
||
\ \ \ \\\ "google\ cloud\ storage"
|
||
\ 7\ /\ Google\ Drive
|
||
\ \ \ \\\ "drive"
|
||
\ 8\ /\ Hubic
|
||
\ \ \ \\\ "hubic"
|
||
\ 9\ /\ Local\ Disk
|
||
\ \ \ \\\ "local"
|
||
10\ /\ Microsoft\ OneDrive
|
||
\ \ \ \\\ "onedrive"
|
||
11\ /\ Openstack\ Swift\ (Rackspace\ Cloud\ Files,\ Memset\ Memstore,\ OVH)
|
||
\ \ \ \\\ "swift"
|
||
12\ /\ SSH/SFTP\ Connection
|
||
\ \ \ \\\ "sftp"
|
||
13\ /\ Yandex\ Disk
|
||
\ \ \ \\\ "yandex"
|
||
Storage>\ 3
|
||
Account\ ID
|
||
account>\ 123456789abc
|
||
Application\ Key
|
||
key>\ 0123456789abcdef0123456789abcdef0123456789
|
||
Endpoint\ for\ the\ service\ \-\ leave\ blank\ normally.
|
||
endpoint>
|
||
Remote\ config
|
||
\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-
|
||
[remote]
|
||
account\ =\ 123456789abc
|
||
key\ =\ 0123456789abcdef0123456789abcdef0123456789
|
||
endpoint\ =
|
||
\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-
|
||
y)\ Yes\ this\ is\ OK
|
||
e)\ Edit\ this\ remote
|
||
d)\ Delete\ this\ remote
|
||
y/e/d>\ y
|
||
\f[]
|
||
.fi
|
||
.PP
|
||
This remote is called \f[C]remote\f[] and can now be used like this
|
||
.PP
|
||
See all buckets
|
||
.IP
|
||
.nf
|
||
\f[C]
|
||
rclone\ lsd\ remote:
|
||
\f[]
|
||
.fi
|
||
.PP
|
||
Make a new bucket
|
||
.IP
|
||
.nf
|
||
\f[C]
|
||
rclone\ mkdir\ remote:bucket
|
||
\f[]
|
||
.fi
|
||
.PP
|
||
List the contents of a bucket
|
||
.IP
|
||
.nf
|
||
\f[C]
|
||
rclone\ ls\ remote:bucket
|
||
\f[]
|
||
.fi
|
||
.PP
|
||
Sync \f[C]/home/local/directory\f[] to the remote bucket, deleting any
|
||
excess files in the bucket.
|
||
.IP
|
||
.nf
|
||
\f[C]
|
||
rclone\ sync\ /home/local/directory\ remote:bucket
|
||
\f[]
|
||
.fi
|
||
.SS \-\-fast\-list
|
||
.PP
|
||
This remote supports \f[C]\-\-fast\-list\f[] which allows you to use
|
||
fewer transactions in exchange for more memory.
|
||
See the rclone docs (/docs/#fast-list) for more details.
|
||
.SS Modified time
|
||
.PP
|
||
The modified time is stored as metadata on the object as
|
||
\f[C]X\-Bz\-Info\-src_last_modified_millis\f[] as milliseconds since
|
||
1970\-01\-01 in the Backblaze standard.
|
||
Other tools should be able to use this as a modified time.
|
||
.PP
|
||
Modified times are used in syncing and are fully supported except in the
|
||
case of updating a modification time on an existing object.
|
||
In this case the object will be uploaded again as B2 doesn\[aq]t have an
|
||
API method to set the modification time independent of doing an upload.
|
||
.SS SHA1 checksums
|
||
.PP
|
||
The SHA1 checksums of the files are checked on upload and download and
|
||
will be used in the syncing process.
|
||
.PP
|
||
Large files (bigger than the limit in \f[C]\-\-b2\-upload\-cutoff\f[])
|
||
which are uploaded in chunks will store their SHA1 on the object as
|
||
\f[C]X\-Bz\-Info\-large_file_sha1\f[] as recommended by Backblaze.
|
||
.PP
|
||
For a large file to be uploaded with an SHA1 checksum, the source needs
|
||
to support SHA1 checksums.
|
||
The local disk supports SHA1 checksums so large file transfers from
|
||
local disk will have an SHA1.
|
||
See the overview (/overview/#features) for exactly which remotes support
|
||
SHA1.
|
||
.PP
|
||
Sources which don\[aq]t support SHA1, in particular \f[C]crypt\f[] will
|
||
upload large files without SHA1 checksums.
|
||
This may be fixed in the future (see
|
||
#1767 (https://github.com/ncw/rclone/issues/1767)).
|
||
.PP
|
||
Files sizes below \f[C]\-\-b2\-upload\-cutoff\f[] will always have an
|
||
SHA1 regardless of the source.
|
||
.SS Transfers
|
||
.PP
|
||
Backblaze recommends that you do lots of transfers simultaneously for
|
||
maximum speed.
|
||
In tests from my SSD equipped laptop the optimum setting is about
|
||
\f[C]\-\-transfers\ 32\f[] though higher numbers may be used for a
|
||
slight speed improvement.
|
||
The optimum number for you may vary depending on your hardware, how big
|
||
the files are, how much you want to load your computer, etc.
|
||
The default of \f[C]\-\-transfers\ 4\f[] is definitely too low for
|
||
Backblaze B2 though.
|
||
.PP
|
||
Note that uploading big files (bigger than 200 MB by default) will use a
|
||
96 MB RAM buffer by default.
|
||
There can be at most \f[C]\-\-transfers\f[] of these in use at any
|
||
moment, so this sets the upper limit on the memory used.
|
||
.SS Versions
|
||
.PP
|
||
When rclone uploads a new version of a file it creates a new version of
|
||
it (https://www.backblaze.com/b2/docs/file_versions.html).
|
||
Likewise when you delete a file, the old version will be marked hidden
|
||
and still be available.
|
||
Conversely, you may opt in to a "hard delete" of files with the
|
||
\f[C]\-\-b2\-hard\-delete\f[] flag which would permanently remove the
|
||
file instead of hiding it.
|
||
.PP
|
||
Old versions of files, where available, are visible using the
|
||
\f[C]\-\-b2\-versions\f[] flag.
|
||
.PP
|
||
If you wish to remove all the old versions then you can use the
|
||
\f[C]rclone\ cleanup\ remote:bucket\f[] command which will delete all
|
||
the old versions of files, leaving the current ones intact.
|
||
You can also supply a path and only old versions under that path will be
|
||
deleted, eg \f[C]rclone\ cleanup\ remote:bucket/path/to/stuff\f[].
|
||
.PP
|
||
When you \f[C]purge\f[] a bucket, the current and the old versions will
|
||
be deleted then the bucket will be deleted.
|
||
.PP
|
||
However \f[C]delete\f[] will cause the current versions of the files to
|
||
become hidden old versions.
|
||
.PP
|
||
Here is a session showing the listing and retrieval of an old version
|
||
followed by a \f[C]cleanup\f[] of the old versions.
|
||
.PP
|
||
Show current version and all the versions with \f[C]\-\-b2\-versions\f[]
|
||
flag.
|
||
.IP
|
||
.nf
|
||
\f[C]
|
||
$\ rclone\ \-q\ ls\ b2:cleanup\-test
|
||
\ \ \ \ \ \ \ \ 9\ one.txt
|
||
|
||
$\ rclone\ \-q\ \-\-b2\-versions\ ls\ b2:cleanup\-test
|
||
\ \ \ \ \ \ \ \ 9\ one.txt
|
||
\ \ \ \ \ \ \ \ 8\ one\-v2016\-07\-04\-141032\-000.txt
|
||
\ \ \ \ \ \ \ 16\ one\-v2016\-07\-04\-141003\-000.txt
|
||
\ \ \ \ \ \ \ 15\ one\-v2016\-07\-02\-155621\-000.txt
|
||
\f[]
|
||
.fi
|
||
.PP
|
||
Retrieve an old version
|
||
.IP
|
||
.nf
|
||
\f[C]
|
||
$\ rclone\ \-q\ \-\-b2\-versions\ copy\ b2:cleanup\-test/one\-v2016\-07\-04\-141003\-000.txt\ /tmp
|
||
|
||
$\ ls\ \-l\ /tmp/one\-v2016\-07\-04\-141003\-000.txt
|
||
\-rw\-rw\-r\-\-\ 1\ ncw\ ncw\ 16\ Jul\ \ 2\ 17:46\ /tmp/one\-v2016\-07\-04\-141003\-000.txt
|
||
\f[]
|
||
.fi
|
||
.PP
|
||
Clean up all the old versions and show that they\[aq]ve gone.
|
||
.IP
|
||
.nf
|
||
\f[C]
|
||
$\ rclone\ \-q\ cleanup\ b2:cleanup\-test
|
||
|
||
$\ rclone\ \-q\ ls\ b2:cleanup\-test
|
||
\ \ \ \ \ \ \ \ 9\ one.txt
|
||
|
||
$\ rclone\ \-q\ \-\-b2\-versions\ ls\ b2:cleanup\-test
|
||
\ \ \ \ \ \ \ \ 9\ one.txt
|
||
\f[]
|
||
.fi
|
||
.SS Data usage
|
||
.PP
|
||
It is useful to know how many requests are sent to the server in
|
||
different scenarios.
|
||
.PP
|
||
All copy commands send the following 4 requests:
|
||
.IP
|
||
.nf
|
||
\f[C]
|
||
/b2api/v1/b2_authorize_account
|
||
/b2api/v1/b2_create_bucket
|
||
/b2api/v1/b2_list_buckets
|
||
/b2api/v1/b2_list_file_names
|
||
\f[]
|
||
.fi
|
||
.PP
|
||
The \f[C]b2_list_file_names\f[] request will be sent once for every 1k
|
||
files in the remote path, providing the checksum and modification time
|
||
of the listed files.
|
||
As of version 1.33 issue #818 (https://github.com/ncw/rclone/issues/818)
|
||
causes extra requests to be sent when using B2 with Crypt.
|
||
When a copy operation does not require any files to be uploaded, no more
|
||
requests will be sent.
|
||
.PP
|
||
Uploading files that do not require chunking, will send 2 requests per
|
||
file upload:
|
||
.IP
|
||
.nf
|
||
\f[C]
|
||
/b2api/v1/b2_get_upload_url
|
||
/b2api/v1/b2_upload_file/
|
||
\f[]
|
||
.fi
|
||
.PP
|
||
Uploading files requiring chunking, will send 2 requests (one each to
|
||
start and finish the upload) and another 2 requests for each chunk:
|
||
.IP
|
||
.nf
|
||
\f[C]
|
||
/b2api/v1/b2_start_large_file
|
||
/b2api/v1/b2_get_upload_part_url
|
||
/b2api/v1/b2_upload_part/
|
||
/b2api/v1/b2_finish_large_file
|
||
\f[]
|
||
.fi
|
||
.SS Specific options
|
||
.PP
|
||
Here are the command line options specific to this cloud storage system.
|
||
.SS \-\-b2\-chunk\-size valuee=SIZE
|
||
.PP
|
||
When uploading large files chunk the file into this size.
|
||
Note that these chunks are buffered in memory and there might a maximum
|
||
of \f[C]\-\-transfers\f[] chunks in progress at once.
|
||
5,000,000 Bytes is the minimim size (default 96M).
|
||
.SS \-\-b2\-upload\-cutoff=SIZE
|
||
.PP
|
||
Cutoff for switching to chunked upload (default 190.735 MiB == 200 MB).
|
||
Files above this size will be uploaded in chunks of
|
||
\f[C]\-\-b2\-chunk\-size\f[].
|
||
.PP
|
||
This value should be set no larger than 4.657GiB (== 5GB) as this is the
|
||
largest file size that can be uploaded.
|
||
.SS \-\-b2\-test\-mode=FLAG
|
||
.PP
|
||
This is for debugging purposes only.
|
||
.PP
|
||
Setting FLAG to one of the strings below will cause b2 to return
|
||
specific errors for debugging purposes.
|
||
.IP \[bu] 2
|
||
\f[C]fail_some_uploads\f[]
|
||
.IP \[bu] 2
|
||
\f[C]expire_some_account_authorization_tokens\f[]
|
||
.IP \[bu] 2
|
||
\f[C]force_cap_exceeded\f[]
|
||
.PP
|
||
These will be set in the \f[C]X\-Bz\-Test\-Mode\f[] header which is
|
||
documented in the b2 integrations
|
||
checklist (https://www.backblaze.com/b2/docs/integration_checklist.html).
|
||
.SS \-\-b2\-versions
|
||
.PP
|
||
When set rclone will show and act on older versions of files.
|
||
For example
|
||
.PP
|
||
Listing without \f[C]\-\-b2\-versions\f[]
|
||
.IP
|
||
.nf
|
||
\f[C]
|
||
$\ rclone\ \-q\ ls\ b2:cleanup\-test
|
||
\ \ \ \ \ \ \ \ 9\ one.txt
|
||
\f[]
|
||
.fi
|
||
.PP
|
||
And with
|
||
.IP
|
||
.nf
|
||
\f[C]
|
||
$\ rclone\ \-q\ \-\-b2\-versions\ ls\ b2:cleanup\-test
|
||
\ \ \ \ \ \ \ \ 9\ one.txt
|
||
\ \ \ \ \ \ \ \ 8\ one\-v2016\-07\-04\-141032\-000.txt
|
||
\ \ \ \ \ \ \ 16\ one\-v2016\-07\-04\-141003\-000.txt
|
||
\ \ \ \ \ \ \ 15\ one\-v2016\-07\-02\-155621\-000.txt
|
||
\f[]
|
||
.fi
|
||
.PP
|
||
Showing that the current version is unchanged but older versions can be
|
||
seen.
|
||
These have the UTC date that they were uploaded to the server to the
|
||
nearest millisecond appended to them.
|
||
.PP
|
||
Note that when using \f[C]\-\-b2\-versions\f[] no file write operations
|
||
are permitted, so you can\[aq]t upload files or delete them.
|
||
.SS Box
|
||
.PP
|
||
Paths are specified as \f[C]remote:path\f[]
|
||
.PP
|
||
Paths may be as deep as required, eg
|
||
\f[C]remote:directory/subdirectory\f[].
|
||
.PP
|
||
The initial setup for Box involves getting a token from Box which you
|
||
need to do in your browser.
|
||
\f[C]rclone\ config\f[] walks you through it.
|
||
.PP
|
||
Here is an example of how to make a remote called \f[C]remote\f[].
|
||
First run:
|
||
.IP
|
||
.nf
|
||
\f[C]
|
||
\ rclone\ config
|
||
\f[]
|
||
.fi
|
||
.PP
|
||
This will guide you through an interactive setup process:
|
||
.IP
|
||
.nf
|
||
\f[C]
|
||
No\ remotes\ found\ \-\ make\ a\ new\ one
|
||
n)\ New\ remote
|
||
s)\ Set\ configuration\ password
|
||
q)\ Quit\ config
|
||
n/s/q>\ n
|
||
name>\ remote
|
||
Type\ of\ storage\ to\ configure.
|
||
Choose\ a\ number\ from\ below,\ or\ type\ in\ your\ own\ value
|
||
\ 1\ /\ Amazon\ Drive
|
||
\ \ \ \\\ "amazon\ cloud\ drive"
|
||
\ 2\ /\ Amazon\ S3\ (also\ Dreamhost,\ Ceph,\ Minio)
|
||
\ \ \ \\\ "s3"
|
||
\ 3\ /\ Backblaze\ B2
|
||
\ \ \ \\\ "b2"
|
||
\ 4\ /\ Box
|
||
\ \ \ \\\ "box"
|
||
\ 5\ /\ Dropbox
|
||
\ \ \ \\\ "dropbox"
|
||
\ 6\ /\ Encrypt/Decrypt\ a\ remote
|
||
\ \ \ \\\ "crypt"
|
||
\ 7\ /\ FTP\ Connection
|
||
\ \ \ \\\ "ftp"
|
||
\ 8\ /\ Google\ Cloud\ Storage\ (this\ is\ not\ Google\ Drive)
|
||
\ \ \ \\\ "google\ cloud\ storage"
|
||
\ 9\ /\ Google\ Drive
|
||
\ \ \ \\\ "drive"
|
||
10\ /\ Hubic
|
||
\ \ \ \\\ "hubic"
|
||
11\ /\ Local\ Disk
|
||
\ \ \ \\\ "local"
|
||
12\ /\ Microsoft\ OneDrive
|
||
\ \ \ \\\ "onedrive"
|
||
13\ /\ Openstack\ Swift\ (Rackspace\ Cloud\ Files,\ Memset\ Memstore,\ OVH)
|
||
\ \ \ \\\ "swift"
|
||
14\ /\ SSH/SFTP\ Connection
|
||
\ \ \ \\\ "sftp"
|
||
15\ /\ Yandex\ Disk
|
||
\ \ \ \\\ "yandex"
|
||
16\ /\ http\ Connection
|
||
\ \ \ \\\ "http"
|
||
Storage>\ box
|
||
Box\ App\ Client\ Id\ \-\ leave\ blank\ normally.
|
||
client_id>\
|
||
Box\ App\ Client\ Secret\ \-\ leave\ blank\ normally.
|
||
client_secret>\
|
||
Remote\ config
|
||
Use\ auto\ config?
|
||
\ *\ Say\ Y\ if\ not\ sure
|
||
\ *\ Say\ N\ if\ you\ are\ working\ on\ a\ remote\ or\ headless\ machine
|
||
y)\ Yes
|
||
n)\ No
|
||
y/n>\ y
|
||
If\ your\ browser\ doesn\[aq]t\ open\ automatically\ go\ to\ the\ following\ link:\ http://127.0.0.1:53682/auth
|
||
Log\ in\ and\ authorize\ rclone\ for\ access
|
||
Waiting\ for\ code...
|
||
Got\ code
|
||
\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-
|
||
[remote]
|
||
client_id\ =\
|
||
client_secret\ =\
|
||
token\ =\ {"access_token":"XXX","token_type":"bearer","refresh_token":"XXX","expiry":"XXX"}
|
||
\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-
|
||
y)\ Yes\ this\ is\ OK
|
||
e)\ Edit\ this\ remote
|
||
d)\ Delete\ this\ remote
|
||
y/e/d>\ y
|
||
\f[]
|
||
.fi
|
||
.PP
|
||
See the remote setup docs (https://rclone.org/remote_setup/) for how to
|
||
set it up on a machine with no Internet browser available.
|
||
.PP
|
||
Note that rclone runs a webserver on your local machine to collect the
|
||
token as returned from Box.
|
||
This only runs from the moment it opens your browser to the moment you
|
||
get back the verification code.
|
||
This is on \f[C]http://127.0.0.1:53682/\f[] and this it may require you
|
||
to unblock it temporarily if you are running a host firewall.
|
||
.PP
|
||
Once configured you can then use \f[C]rclone\f[] like this,
|
||
.PP
|
||
List directories in top level of your Box
|
||
.IP
|
||
.nf
|
||
\f[C]
|
||
rclone\ lsd\ remote:
|
||
\f[]
|
||
.fi
|
||
.PP
|
||
List all the files in your Box
|
||
.IP
|
||
.nf
|
||
\f[C]
|
||
rclone\ ls\ remote:
|
||
\f[]
|
||
.fi
|
||
.PP
|
||
To copy a local directory to an Box directory called backup
|
||
.IP
|
||
.nf
|
||
\f[C]
|
||
rclone\ copy\ /home/source\ remote:backup
|
||
\f[]
|
||
.fi
|
||
.SS Invalid refresh token
|
||
.PP
|
||
According to the box
|
||
docs (https://developer.box.com/v2.0/docs/oauth-20#section-6-using-the-access-and-refresh-tokens):
|
||
.RS
|
||
.PP
|
||
Each refresh_token is valid for one use in 60 days.
|
||
.RE
|
||
.PP
|
||
This means that if you
|
||
.IP \[bu] 2
|
||
Don\[aq]t use the box remote for 60 days
|
||
.IP \[bu] 2
|
||
Copy the config file with a box refresh token in and use it in two
|
||
places
|
||
.IP \[bu] 2
|
||
Get an error on a token refresh
|
||
.PP
|
||
then rclone will return an error which includes the text
|
||
\f[C]Invalid\ refresh\ token\f[].
|
||
.PP
|
||
To fix this you will need to use oauth2 again to update the refresh
|
||
token.
|
||
You can use the methods in the remote setup
|
||
docs (https://rclone.org/remote_setup/), bearing in mind that if you use
|
||
the copy the config file method, you should not use that remote on the
|
||
computer you did the authentication on.
|
||
.PP
|
||
Here is how to do it.
|
||
.IP
|
||
.nf
|
||
\f[C]
|
||
$\ rclone\ config
|
||
Current\ remotes:
|
||
|
||
Name\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ Type
|
||
====\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ ====
|
||
remote\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ box
|
||
|
||
e)\ Edit\ existing\ remote
|
||
n)\ New\ remote
|
||
d)\ Delete\ remote
|
||
r)\ Rename\ remote
|
||
c)\ Copy\ remote
|
||
s)\ Set\ configuration\ password
|
||
q)\ Quit\ config
|
||
e/n/d/r/c/s/q>\ e
|
||
Choose\ a\ number\ from\ below,\ or\ type\ in\ an\ existing\ value
|
||
\ 1\ >\ remote
|
||
remote>\ remote
|
||
\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-
|
||
[remote]
|
||
type\ =\ box
|
||
token\ =\ {"access_token":"XXX","token_type":"bearer","refresh_token":"XXX","expiry":"2017\-07\-08T23:40:08.059167677+01:00"}
|
||
\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-
|
||
Edit\ remote
|
||
Value\ "client_id"\ =\ ""
|
||
Edit?\ (y/n)>
|
||
y)\ Yes
|
||
n)\ No
|
||
y/n>\ n
|
||
Value\ "client_secret"\ =\ ""
|
||
Edit?\ (y/n)>
|
||
y)\ Yes
|
||
n)\ No
|
||
y/n>\ n
|
||
Remote\ config
|
||
Already\ have\ a\ token\ \-\ refresh?
|
||
y)\ Yes
|
||
n)\ No
|
||
y/n>\ y
|
||
Use\ auto\ config?
|
||
\ *\ Say\ Y\ if\ not\ sure
|
||
\ *\ Say\ N\ if\ you\ are\ working\ on\ a\ remote\ or\ headless\ machine
|
||
y)\ Yes
|
||
n)\ No
|
||
y/n>\ y
|
||
If\ your\ browser\ doesn\[aq]t\ open\ automatically\ go\ to\ the\ following\ link:\ http://127.0.0.1:53682/auth
|
||
Log\ in\ and\ authorize\ rclone\ for\ access
|
||
Waiting\ for\ code...
|
||
Got\ code
|
||
\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-
|
||
[remote]
|
||
type\ =\ box
|
||
token\ =\ {"access_token":"YYY","token_type":"bearer","refresh_token":"YYY","expiry":"2017\-07\-23T12:22:29.259137901+01:00"}
|
||
\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-
|
||
y)\ Yes\ this\ is\ OK
|
||
e)\ Edit\ this\ remote
|
||
d)\ Delete\ this\ remote
|
||
y/e/d>\ y
|
||
\f[]
|
||
.fi
|
||
.SS Modified time and hashes
|
||
.PP
|
||
Box allows modification times to be set on objects accurate to 1 second.
|
||
These will be used to detect whether objects need syncing or not.
|
||
.PP
|
||
One drive supports SHA1 type hashes, so you can use the
|
||
\f[C]\-\-checksum\f[] flag.
|
||
.SS Transfers
|
||
.PP
|
||
For files above 50MB rclone will use a chunked transfer.
|
||
Rclone will upload up to \f[C]\-\-transfers\f[] chunks at the same time
|
||
(shared among all the multipart uploads).
|
||
Chunks are buffered in memory and are normally 8MB so increasing
|
||
\f[C]\-\-transfers\f[] will increase memory use.
|
||
.SS Deleting files
|
||
.PP
|
||
Depending on the enterprise settings for your user, the item will either
|
||
be actually deleted from Box or moved to the trash.
|
||
.SS Specific options
|
||
.PP
|
||
Here are the command line options specific to this cloud storage system.
|
||
.SS \-\-box\-upload\-cutoff=SIZE
|
||
.PP
|
||
Cutoff for switching to chunked upload \- must be >= 50MB.
|
||
The default is 50MB.
|
||
.SS Limitations
|
||
.PP
|
||
Note that Box is case insensitive so you can\[aq]t have a file called
|
||
"Hello.doc" and one called "hello.doc".
|
||
.PP
|
||
Box file names can\[aq]t have the \f[C]\\\f[] character in.
|
||
rclone maps this to and from an identical looking unicode equivalent
|
||
\f[C]\\f[].
|
||
.PP
|
||
Box only supports filenames up to 255 characters in length.
|
||
.SS Cache (BETA)
|
||
.PP
|
||
The \f[C]cache\f[] remote wraps another existing remote and stores file
|
||
structure and its data for long running tasks like
|
||
\f[C]rclone\ mount\f[].
|
||
.PP
|
||
To get started you just need to have an existing remote which can be
|
||
configured with \f[C]cache\f[].
|
||
.PP
|
||
Here is an example of how to make a remote called \f[C]test\-cache\f[].
|
||
First run:
|
||
.IP
|
||
.nf
|
||
\f[C]
|
||
\ rclone\ config
|
||
\f[]
|
||
.fi
|
||
.PP
|
||
This will guide you through an interactive setup process:
|
||
.IP
|
||
.nf
|
||
\f[C]
|
||
No\ remotes\ found\ \-\ make\ a\ new\ one
|
||
n)\ New\ remote
|
||
r)\ Rename\ remote
|
||
c)\ Copy\ remote
|
||
s)\ Set\ configuration\ password
|
||
q)\ Quit\ config
|
||
n/r/c/s/q>\ n
|
||
name>\ test\-cache
|
||
Type\ of\ storage\ to\ configure.
|
||
Choose\ a\ number\ from\ below,\ or\ type\ in\ your\ own\ value
|
||
\&...
|
||
\ 5\ /\ Cache\ a\ remote
|
||
\ \ \ \\\ "cache"
|
||
\&...
|
||
Storage>\ 5
|
||
Remote\ to\ cache.
|
||
Normally\ should\ contain\ a\ \[aq]:\[aq]\ and\ a\ path,\ eg\ "myremote:path/to/dir",
|
||
"myremote:bucket"\ or\ maybe\ "myremote:"\ (not\ recommended).
|
||
remote>\ local:/test
|
||
Optional:\ The\ URL\ of\ the\ Plex\ server
|
||
plex_url>\ http://127.0.0.1:32400
|
||
Optional:\ The\ username\ of\ the\ Plex\ user
|
||
plex_username>\ dummyusername
|
||
Optional:\ The\ password\ of\ the\ Plex\ user
|
||
y)\ Yes\ type\ in\ my\ own\ password
|
||
g)\ Generate\ random\ password
|
||
n)\ No\ leave\ this\ optional\ password\ blank
|
||
y/g/n>\ y
|
||
Enter\ the\ password:
|
||
password:
|
||
Confirm\ the\ password:
|
||
password:
|
||
The\ size\ of\ a\ chunk.\ Lower\ value\ good\ for\ slow\ connections\ but\ can\ affect\ seamless\ reading.
|
||
Default:\ 5M
|
||
Choose\ a\ number\ from\ below,\ or\ type\ in\ your\ own\ value
|
||
\ 1\ /\ 1MB
|
||
\ \ \ \\\ "1m"
|
||
\ 2\ /\ 5\ MB
|
||
\ \ \ \\\ "5M"
|
||
\ 3\ /\ 10\ MB
|
||
\ \ \ \\\ "10M"
|
||
chunk_size>\ 2
|
||
How\ much\ time\ should\ object\ info\ (file\ size,\ file\ hashes\ etc)\ be\ stored\ in\ cache.\ Use\ a\ very\ high\ value\ if\ you\ don\[aq]t\ plan\ on\ changing\ the\ source\ FS\ from\ outside\ the\ cache.
|
||
Accepted\ units\ are:\ "s",\ "m",\ "h".
|
||
Default:\ 5m
|
||
Choose\ a\ number\ from\ below,\ or\ type\ in\ your\ own\ value
|
||
\ 1\ /\ 1\ hour
|
||
\ \ \ \\\ "1h"
|
||
\ 2\ /\ 24\ hours
|
||
\ \ \ \\\ "24h"
|
||
\ 3\ /\ 24\ hours
|
||
\ \ \ \\\ "48h"
|
||
info_age>\ 2
|
||
The\ maximum\ size\ of\ stored\ chunks.\ When\ the\ storage\ grows\ beyond\ this\ size,\ the\ oldest\ chunks\ will\ be\ deleted.
|
||
Default:\ 10G
|
||
Choose\ a\ number\ from\ below,\ or\ type\ in\ your\ own\ value
|
||
\ 1\ /\ 500\ MB
|
||
\ \ \ \\\ "500M"
|
||
\ 2\ /\ 1\ GB
|
||
\ \ \ \\\ "1G"
|
||
\ 3\ /\ 10\ GB
|
||
\ \ \ \\\ "10G"
|
||
chunk_total_size>\ 3
|
||
Remote\ config
|
||
\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-
|
||
[test\-cache]
|
||
remote\ =\ local:/test
|
||
plex_url\ =\ http://127.0.0.1:32400
|
||
plex_username\ =\ dummyusername
|
||
plex_password\ =\ ***\ ENCRYPTED\ ***
|
||
chunk_size\ =\ 5M
|
||
info_age\ =\ 48h
|
||
chunk_total_size\ =\ 10G
|
||
\f[]
|
||
.fi
|
||
.PP
|
||
You can then use it like this,
|
||
.PP
|
||
List directories in top level of your drive
|
||
.IP
|
||
.nf
|
||
\f[C]
|
||
rclone\ lsd\ test\-cache:
|
||
\f[]
|
||
.fi
|
||
.PP
|
||
List all the files in your drive
|
||
.IP
|
||
.nf
|
||
\f[C]
|
||
rclone\ ls\ test\-cache:
|
||
\f[]
|
||
.fi
|
||
.PP
|
||
To start a cached mount
|
||
.IP
|
||
.nf
|
||
\f[C]
|
||
rclone\ mount\ \-\-allow\-other\ test\-cache:\ /var/tmp/test\-cache
|
||
\f[]
|
||
.fi
|
||
.SS Write Support
|
||
.PP
|
||
Writes are supported through \f[C]cache\f[].
|
||
One caveat is that a mounted cache remote does not add any retry or
|
||
fallback mechanism to the upload operation.
|
||
This will depend on the implementation of the wrapped remote.
|
||
.PP
|
||
One special case is covered with \f[C]cache\-writes\f[] which will cache
|
||
the file data at the same time as the upload when it is enabled making
|
||
it available from the cache store immediately once the upload is
|
||
finished.
|
||
.SS Read Features
|
||
.SS Multiple connections
|
||
.PP
|
||
To counter the high latency between a local PC where rclone is running
|
||
and cloud providers, the cache remote can split multiple requests to the
|
||
cloud provider for smaller file chunks and combines them together
|
||
locally where they can be available almost immediately before the reader
|
||
usually needs them.
|
||
.PP
|
||
This is similar to buffering when media files are played online.
|
||
Rclone will stay around the current marker but always try its best to
|
||
stay ahead and prepare the data before.
|
||
.SS Plex Integration
|
||
.PP
|
||
There is a direct integration with Plex which allows cache to detect
|
||
during reading if the file is in playback or not.
|
||
This helps cache to adapt how it queries the cloud provider depending on
|
||
what is needed for.
|
||
.PP
|
||
Scans will have a minimum amount of workers (1) while in a confirmed
|
||
playback cache will deploy the configured number of workers.
|
||
.PP
|
||
This integration opens the doorway to additional performance
|
||
improvements which will be explored in the near future.
|
||
.PP
|
||
\f[B]Note:\f[] If Plex options are not configured, \f[C]cache\f[] will
|
||
function with its configured options without adapting any of its
|
||
settings.
|
||
.PP
|
||
How to enable?
|
||
Run \f[C]rclone\ config\f[] and add all the Plex options (endpoint,
|
||
username and password) in your remote and it will be automatically
|
||
enabled.
|
||
.PP
|
||
Affected settings: \- \f[C]cache\-workers\f[]: \f[I]Configured value\f[]
|
||
during confirmed playback or \f[I]1\f[] all the other times
|
||
.SS Known issues
|
||
.SS Windows support \- Experimental
|
||
.PP
|
||
There are a couple of issues with Windows \f[C]mount\f[] functionality
|
||
that still require some investigations.
|
||
It should be considered as experimental thus far as fixes come in for
|
||
this OS.
|
||
.PP
|
||
Most of the issues seem to be related to the difference between
|
||
filesystems on Linux flavors and Windows as cache is heavily dependant
|
||
on them.
|
||
.PP
|
||
Any reports or feedback on how cache behaves on this OS is greatly
|
||
appreciated.
|
||
.IP \[bu] 2
|
||
https://github.com/ncw/rclone/issues/1935
|
||
.IP \[bu] 2
|
||
https://github.com/ncw/rclone/issues/1907
|
||
.IP \[bu] 2
|
||
https://github.com/ncw/rclone/issues/1834
|
||
.SS Risk of throttling
|
||
.PP
|
||
Future iterations of the cache backend will make use of the pooling
|
||
functionality of the cloud provider to synchronize and at the same time
|
||
make writing through it more tolerant to failures.
|
||
.PP
|
||
There are a couple of enhancements in track to add these but in the
|
||
meantime there is a valid concern that the expiring cache listings can
|
||
lead to cloud provider throttles or bans due to repeated queries on it
|
||
for very large mounts.
|
||
.PP
|
||
Some recommendations: \- don\[aq]t use a very small interval for entry
|
||
informations (\f[C]\-\-cache\-info\-age\f[]) \- while writes aren\[aq]t
|
||
yet optimised, you can still write through \f[C]cache\f[] which gives
|
||
you the advantage of adding the file in the cache at the same time if
|
||
configured to do so.
|
||
.PP
|
||
Future enhancements:
|
||
.IP \[bu] 2
|
||
https://github.com/ncw/rclone/issues/1937
|
||
.IP \[bu] 2
|
||
https://github.com/ncw/rclone/issues/1936
|
||
.SS cache and crypt
|
||
.PP
|
||
One common scenario is to keep your data encrypted in the cloud provider
|
||
using the \f[C]crypt\f[] remote.
|
||
\f[C]crypt\f[] uses a similar technique to wrap around an existing
|
||
remote and handles this translation in a seamless way.
|
||
.PP
|
||
There is an issue with wrapping the remotes in this order: \f[B]cloud
|
||
remote\f[] \-> \f[B]crypt\f[] \-> \f[B]cache\f[]
|
||
.PP
|
||
During testing, I experienced a lot of bans with the remotes in this
|
||
order.
|
||
I suspect it might be related to how crypt opens files on the cloud
|
||
provider which makes it think we\[aq]re downloading the full file
|
||
instead of small chunks.
|
||
Organizing the remotes in this order yelds better results: \f[B]cloud
|
||
remote\f[] \-> \f[B]cache\f[] \-> \f[B]crypt\f[]
|
||
.SS Specific options
|
||
.PP
|
||
Here are the command line options specific to this cloud storage system.
|
||
.SS \-\-cache\-chunk\-path=PATH
|
||
.PP
|
||
Path to where partial file data (chunks) is stored locally.
|
||
The remote name is appended to the final path.
|
||
.PP
|
||
This config follows the \f[C]\-\-cache\-db\-path\f[].
|
||
If you specify a custom location for \f[C]\-\-cache\-db\-path\f[] and
|
||
don\[aq]t specify one for \f[C]\-\-cache\-chunk\-path\f[] then
|
||
\f[C]\-\-cache\-chunk\-path\f[] will use the same path as
|
||
\f[C]\-\-cache\-db\-path\f[].
|
||
.PP
|
||
\f[B]Default\f[]: /cache\-backend/ \f[B]Example\f[]:
|
||
/.cache/cache\-backend/test\-cache
|
||
.SS \-\-cache\-db\-path=PATH
|
||
.PP
|
||
Path to where the file structure metadata (DB) is stored locally.
|
||
The remote name is used as the DB file name.
|
||
.PP
|
||
\f[B]Default\f[]: /cache\-backend/ \f[B]Example\f[]:
|
||
/.cache/cache\-backend/test\-cache
|
||
.SS \-\-cache\-db\-purge
|
||
.PP
|
||
Flag to clear all the cached data for this remote before.
|
||
.PP
|
||
\f[B]Default\f[]: not set
|
||
.SS \-\-cache\-chunk\-size=SIZE
|
||
.PP
|
||
The size of a chunk (partial file data).
|
||
Use lower numbers for slower connections.
|
||
.PP
|
||
\f[B]Default\f[]: 5M
|
||
.SS \-\-cache\-total\-chunk\-size=SIZE
|
||
.PP
|
||
The total size that the chunks can take up on the local disk.
|
||
If \f[C]cache\f[] exceeds this value then it will start to the delete
|
||
the oldest chunks until it goes under this value.
|
||
.PP
|
||
\f[B]Default\f[]: 10G
|
||
.SS \-\-cache\-chunk\-clean\-interval=DURATION
|
||
.PP
|
||
How often should \f[C]cache\f[] perform cleanups of the chunk storage.
|
||
The default value should be ok for most people.
|
||
If you find that \f[C]cache\f[] goes over
|
||
\f[C]cache\-total\-chunk\-size\f[] too often then try to lower this
|
||
value to force it to perform cleanups more often.
|
||
.PP
|
||
\f[B]Default\f[]: 1m
|
||
.SS \-\-cache\-info\-age=DURATION
|
||
.PP
|
||
How long to keep file structure information (directory listings, file
|
||
size, mod times etc) locally.
|
||
.PP
|
||
If all write operations are done through \f[C]cache\f[] then you can
|
||
safely make this value very large as the cache store will also be
|
||
updated in real time.
|
||
.PP
|
||
\f[B]Default\f[]: 6h
|
||
.SS \-\-cache\-read\-retries=RETRIES
|
||
.PP
|
||
How many times to retry a read from a cache storage.
|
||
.PP
|
||
Since reading from a \f[C]cache\f[] stream is independent from
|
||
downloading file data, readers can get to a point where there\[aq]s no
|
||
more data in the cache.
|
||
Most of the times this can indicate a connectivity issue if
|
||
\f[C]cache\f[] isn\[aq]t able to provide file data anymore.
|
||
.PP
|
||
For really slow connections, increase this to a point where the stream
|
||
is able to provide data but your experience will be very stuttering.
|
||
.PP
|
||
\f[B]Default\f[]: 10
|
||
.SS \-\-cache\-workers=WORKERS
|
||
.PP
|
||
How many workers should run in parallel to download chunks.
|
||
.PP
|
||
Higher values will mean more parallel processing (better CPU needed) and
|
||
more concurrent requests on the cloud provider.
|
||
This impacts several aspects like the cloud provider API limits, more
|
||
stress on the hardware that rclone runs on but it also means that
|
||
streams will be more fluid and data will be available much more faster
|
||
to readers.
|
||
.PP
|
||
\f[B]Note\f[]: If the optional Plex integration is enabled then this
|
||
setting will adapt to the type of reading performed and the value
|
||
specified here will be used as a maximum number of workers to use.
|
||
\f[B]Default\f[]: 4
|
||
.SS \-\-cache\-chunk\-no\-memory
|
||
.PP
|
||
By default, \f[C]cache\f[] will keep file data during streaming in RAM
|
||
as well to provide it to readers as fast as possible.
|
||
.PP
|
||
This transient data is evicted as soon as it is read and the number of
|
||
chunks stored doesn\[aq]t exceed the number of workers.
|
||
However, depending on other settings like \f[C]cache\-chunk\-size\f[]
|
||
and \f[C]cache\-workers\f[] this footprint can increase if there are
|
||
parallel streams too (multiple files being read at the same time).
|
||
.PP
|
||
If the hardware permits it, use this feature to provide an overall
|
||
better performance during streaming but it can also be disabled if RAM
|
||
is not available on the local machine.
|
||
.PP
|
||
\f[B]Default\f[]: not set
|
||
.SS \-\-cache\-rps=NUMBER
|
||
.PP
|
||
This setting places a hard limit on the number of requests per second
|
||
that \f[C]cache\f[] will be doing to the cloud provider remote and try
|
||
to respect that value by setting waits between reads.
|
||
.PP
|
||
If you find that you\[aq]re getting banned or limited on the cloud
|
||
provider through cache and know that a smaller number of requests per
|
||
second will allow you to work with it then you can use this setting for
|
||
that.
|
||
.PP
|
||
A good balance of all the other settings should make this setting
|
||
useless but it is available to set for more special cases.
|
||
.PP
|
||
\f[B]NOTE\f[]: This will limit the number of requests during streams but
|
||
other API calls to the cloud provider like directory listings will still
|
||
pass.
|
||
.PP
|
||
\f[B]Default\f[]: disabled
|
||
.SS \-\-cache\-writes
|
||
.PP
|
||
If you need to read files immediately after you upload them through
|
||
\f[C]cache\f[] you can enable this flag to have their data stored in the
|
||
cache store at the same time during upload.
|
||
.PP
|
||
\f[B]Default\f[]: not set
|
||
.SS Crypt
|
||
.PP
|
||
The \f[C]crypt\f[] remote encrypts and decrypts another remote.
|
||
.PP
|
||
To use it first set up the underlying remote following the config
|
||
instructions for that remote.
|
||
You can also use a local pathname instead of a remote which will encrypt
|
||
and decrypt from that directory which might be useful for encrypting
|
||
onto a USB stick for example.
|
||
.PP
|
||
First check your chosen remote is working \- we\[aq]ll call it
|
||
\f[C]remote:path\f[] in these docs.
|
||
Note that anything inside \f[C]remote:path\f[] will be encrypted and
|
||
anything outside won\[aq]t.
|
||
This means that if you are using a bucket based remote (eg S3, B2,
|
||
swift) then you should probably put the bucket in the remote
|
||
\f[C]s3:bucket\f[].
|
||
If you just use \f[C]s3:\f[] then rclone will make encrypted bucket
|
||
names too (if using file name encryption) which may or may not be what
|
||
you want.
|
||
.PP
|
||
Now configure \f[C]crypt\f[] using \f[C]rclone\ config\f[].
|
||
We will call this one \f[C]secret\f[] to differentiate it from the
|
||
\f[C]remote\f[].
|
||
.IP
|
||
.nf
|
||
\f[C]
|
||
No\ remotes\ found\ \-\ make\ a\ new\ one
|
||
n)\ New\ remote
|
||
s)\ Set\ configuration\ password
|
||
q)\ Quit\ config
|
||
n/s/q>\ n\ \ \
|
||
name>\ secret
|
||
Type\ of\ storage\ to\ configure.
|
||
Choose\ a\ number\ from\ below,\ or\ type\ in\ your\ own\ value
|
||
\ 1\ /\ Amazon\ Drive
|
||
\ \ \ \\\ "amazon\ cloud\ drive"
|
||
\ 2\ /\ Amazon\ S3\ (also\ Dreamhost,\ Ceph,\ Minio)
|
||
\ \ \ \\\ "s3"
|
||
\ 3\ /\ Backblaze\ B2
|
||
\ \ \ \\\ "b2"
|
||
\ 4\ /\ Dropbox
|
||
\ \ \ \\\ "dropbox"
|
||
\ 5\ /\ Encrypt/Decrypt\ a\ remote
|
||
\ \ \ \\\ "crypt"
|
||
\ 6\ /\ Google\ Cloud\ Storage\ (this\ is\ not\ Google\ Drive)
|
||
\ \ \ \\\ "google\ cloud\ storage"
|
||
\ 7\ /\ Google\ Drive
|
||
\ \ \ \\\ "drive"
|
||
\ 8\ /\ Hubic
|
||
\ \ \ \\\ "hubic"
|
||
\ 9\ /\ Local\ Disk
|
||
\ \ \ \\\ "local"
|
||
10\ /\ Microsoft\ OneDrive
|
||
\ \ \ \\\ "onedrive"
|
||
11\ /\ Openstack\ Swift\ (Rackspace\ Cloud\ Files,\ Memset\ Memstore,\ OVH)
|
||
\ \ \ \\\ "swift"
|
||
12\ /\ SSH/SFTP\ Connection
|
||
\ \ \ \\\ "sftp"
|
||
13\ /\ Yandex\ Disk
|
||
\ \ \ \\\ "yandex"
|
||
Storage>\ 5
|
||
Remote\ to\ encrypt/decrypt.
|
||
Normally\ should\ contain\ a\ \[aq]:\[aq]\ and\ a\ path,\ eg\ "myremote:path/to/dir",
|
||
"myremote:bucket"\ or\ maybe\ "myremote:"\ (not\ recommended).
|
||
remote>\ remote:path
|
||
How\ to\ encrypt\ the\ filenames.
|
||
Choose\ a\ number\ from\ below,\ or\ type\ in\ your\ own\ value
|
||
\ 1\ /\ Don\[aq]t\ encrypt\ the\ file\ names.\ \ Adds\ a\ ".bin"\ extension\ only.
|
||
\ \ \ \\\ "off"
|
||
\ 2\ /\ Encrypt\ the\ filenames\ see\ the\ docs\ for\ the\ details.
|
||
\ \ \ \\\ "standard"
|
||
\ 3\ /\ Very\ simple\ filename\ obfuscation.
|
||
\ \ \ \\\ "obfuscate"
|
||
filename_encryption>\ 2
|
||
Option\ to\ either\ encrypt\ directory\ names\ or\ leave\ them\ intact.
|
||
Choose\ a\ number\ from\ below,\ or\ type\ in\ your\ own\ value
|
||
\ 1\ /\ Encrypt\ directory\ names.
|
||
\ \ \ \\\ "true"
|
||
\ 2\ /\ Don\[aq]t\ encrypt\ directory\ names,\ leave\ them\ intact.
|
||
\ \ \ \\\ "false"
|
||
filename_encryption>\ 1
|
||
Password\ or\ pass\ phrase\ for\ encryption.
|
||
y)\ Yes\ type\ in\ my\ own\ password
|
||
g)\ Generate\ random\ password
|
||
y/g>\ y
|
||
Enter\ the\ password:
|
||
password:
|
||
Confirm\ the\ password:
|
||
password:
|
||
Password\ or\ pass\ phrase\ for\ salt.\ Optional\ but\ recommended.
|
||
Should\ be\ different\ to\ the\ previous\ password.
|
||
y)\ Yes\ type\ in\ my\ own\ password
|
||
g)\ Generate\ random\ password
|
||
n)\ No\ leave\ this\ optional\ password\ blank
|
||
y/g/n>\ g
|
||
Password\ strength\ in\ bits.
|
||
64\ is\ just\ about\ memorable
|
||
128\ is\ secure
|
||
1024\ is\ the\ maximum
|
||
Bits>\ 128
|
||
Your\ password\ is:\ JAsJvRcgR\-_veXNfy_sGmQ
|
||
Use\ this\ password?
|
||
y)\ Yes
|
||
n)\ No
|
||
y/n>\ y
|
||
Remote\ config
|
||
\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-
|
||
[secret]
|
||
remote\ =\ remote:path
|
||
filename_encryption\ =\ standard
|
||
password\ =\ ***\ ENCRYPTED\ ***
|
||
password2\ =\ ***\ ENCRYPTED\ ***
|
||
\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-
|
||
y)\ Yes\ this\ is\ OK
|
||
e)\ Edit\ this\ remote
|
||
d)\ Delete\ this\ remote
|
||
y/e/d>\ y
|
||
\f[]
|
||
.fi
|
||
.PP
|
||
\f[B]Important\f[] The password is stored in the config file is lightly
|
||
obscured so it isn\[aq]t immediately obvious what it is.
|
||
It is in no way secure unless you use config file encryption.
|
||
.PP
|
||
A long passphrase is recommended, or you can use a random one.
|
||
Note that if you reconfigure rclone with the same passwords/passphrases
|
||
elsewhere it will be compatible \- all the secrets used are derived from
|
||
those two passwords/passphrases.
|
||
.PP
|
||
Note that rclone does not encrypt
|
||
.IP \[bu] 2
|
||
file length \- this can be calcuated within 16 bytes
|
||
.IP \[bu] 2
|
||
modification time \- used for syncing
|
||
.SS Specifying the remote
|
||
.PP
|
||
In normal use, make sure the remote has a \f[C]:\f[] in.
|
||
If you specify the remote without a \f[C]:\f[] then rclone will use a
|
||
local directory of that name.
|
||
So if you use a remote of \f[C]/path/to/secret/files\f[] then rclone
|
||
will encrypt stuff to that directory.
|
||
If you use a remote of \f[C]name\f[] then rclone will put files in a
|
||
directory called \f[C]name\f[] in the current directory.
|
||
.PP
|
||
If you specify the remote as \f[C]remote:path/to/dir\f[] then rclone
|
||
will store encrypted files in \f[C]path/to/dir\f[] on the remote.
|
||
If you are using file name encryption, then when you save files to
|
||
\f[C]secret:subdir/subfile\f[] this will store them in the unencrypted
|
||
path \f[C]path/to/dir\f[] but the \f[C]subdir/subpath\f[] bit will be
|
||
encrypted.
|
||
.PP
|
||
Note that unless you want encrypted bucket names (which are difficult to
|
||
manage because you won\[aq]t know what directory they represent in web
|
||
interfaces etc), you should probably specify a bucket, eg
|
||
\f[C]remote:secretbucket\f[] when using bucket based remotes such as S3,
|
||
Swift, Hubic, B2, GCS.
|
||
.SS Example
|
||
.PP
|
||
To test I made a little directory of files using "standard" file name
|
||
encryption.
|
||
.IP
|
||
.nf
|
||
\f[C]
|
||
plaintext/
|
||
├──\ file0.txt
|
||
├──\ file1.txt
|
||
└──\ subdir
|
||
\ \ \ \ ├──\ file2.txt
|
||
\ \ \ \ ├──\ file3.txt
|
||
\ \ \ \ └──\ subsubdir
|
||
\ \ \ \ \ \ \ \ └──\ file4.txt
|
||
\f[]
|
||
.fi
|
||
.PP
|
||
Copy these to the remote and list them back
|
||
.IP
|
||
.nf
|
||
\f[C]
|
||
$\ rclone\ \-q\ copy\ plaintext\ secret:
|
||
$\ rclone\ \-q\ ls\ secret:
|
||
\ \ \ \ \ \ \ \ 7\ file1.txt
|
||
\ \ \ \ \ \ \ \ 6\ file0.txt
|
||
\ \ \ \ \ \ \ \ 8\ subdir/file2.txt
|
||
\ \ \ \ \ \ \ 10\ subdir/subsubdir/file4.txt
|
||
\ \ \ \ \ \ \ \ 9\ subdir/file3.txt
|
||
\f[]
|
||
.fi
|
||
.PP
|
||
Now see what that looked like when encrypted
|
||
.IP
|
||
.nf
|
||
\f[C]
|
||
$\ rclone\ \-q\ ls\ remote:path
|
||
\ \ \ \ \ \ \ 55\ hagjclgavj2mbiqm6u6cnjjqcg
|
||
\ \ \ \ \ \ \ 54\ v05749mltvv1tf4onltun46gls
|
||
\ \ \ \ \ \ \ 57\ 86vhrsv86mpbtd3a0akjuqslj8/dlj7fkq4kdq72emafg7a7s41uo
|
||
\ \ \ \ \ \ \ 58\ 86vhrsv86mpbtd3a0akjuqslj8/7uu829995du6o42n32otfhjqp4/b9pausrfansjth5ob3jkdqd4lc
|
||
\ \ \ \ \ \ \ 56\ 86vhrsv86mpbtd3a0akjuqslj8/8njh1sk437gttmep3p70g81aps
|
||
\f[]
|
||
.fi
|
||
.PP
|
||
Note that this retains the directory structure which means you can do
|
||
this
|
||
.IP
|
||
.nf
|
||
\f[C]
|
||
$\ rclone\ \-q\ ls\ secret:subdir
|
||
\ \ \ \ \ \ \ \ 8\ file2.txt
|
||
\ \ \ \ \ \ \ \ 9\ file3.txt
|
||
\ \ \ \ \ \ \ 10\ subsubdir/file4.txt
|
||
\f[]
|
||
.fi
|
||
.PP
|
||
If don\[aq]t use file name encryption then the remote will look like
|
||
this \- note the \f[C]\&.bin\f[] extensions added to prevent the cloud
|
||
provider attempting to interpret the data.
|
||
.IP
|
||
.nf
|
||
\f[C]
|
||
$\ rclone\ \-q\ ls\ remote:path
|
||
\ \ \ \ \ \ \ 54\ file0.txt.bin
|
||
\ \ \ \ \ \ \ 57\ subdir/file3.txt.bin
|
||
\ \ \ \ \ \ \ 56\ subdir/file2.txt.bin
|
||
\ \ \ \ \ \ \ 58\ subdir/subsubdir/file4.txt.bin
|
||
\ \ \ \ \ \ \ 55\ file1.txt.bin
|
||
\f[]
|
||
.fi
|
||
.SS File name encryption modes
|
||
.PP
|
||
Here are some of the features of the file name encryption modes
|
||
.PP
|
||
Off
|
||
.IP \[bu] 2
|
||
doesn\[aq]t hide file names or directory structure
|
||
.IP \[bu] 2
|
||
allows for longer file names (~246 characters)
|
||
.IP \[bu] 2
|
||
can use sub paths and copy single files
|
||
.PP
|
||
Standard
|
||
.IP \[bu] 2
|
||
file names encrypted
|
||
.IP \[bu] 2
|
||
file names can\[aq]t be as long (~156 characters)
|
||
.IP \[bu] 2
|
||
can use sub paths and copy single files
|
||
.IP \[bu] 2
|
||
directory structure visible
|
||
.IP \[bu] 2
|
||
identical files names will have identical uploaded names
|
||
.IP \[bu] 2
|
||
can use shortcuts to shorten the directory recursion
|
||
.PP
|
||
Obfuscation
|
||
.PP
|
||
This is a simple "rotate" of the filename, with each file having a rot
|
||
distance based on the filename.
|
||
We store the distance at the beginning of the filename.
|
||
So a file called "hello" may become "53.jgnnq"
|
||
.PP
|
||
This is not a strong encryption of filenames, but it may stop automated
|
||
scanning tools from picking up on filename patterns.
|
||
As such it\[aq]s an intermediate between "off" and "standard".
|
||
The advantage is that it allows for longer path segment names.
|
||
.PP
|
||
There is a possibility with some unicode based filenames that the
|
||
obfuscation is weak and may map lower case characters to upper case
|
||
equivalents.
|
||
You can not rely on this for strong protection.
|
||
.IP \[bu] 2
|
||
file names very lightly obfuscated
|
||
.IP \[bu] 2
|
||
file names can be longer than standard encryption
|
||
.IP \[bu] 2
|
||
can use sub paths and copy single files
|
||
.IP \[bu] 2
|
||
directory structure visible
|
||
.IP \[bu] 2
|
||
identical files names will have identical uploaded names
|
||
.PP
|
||
Cloud storage systems have various limits on file name length and total
|
||
path length which you are more likely to hit using "Standard" file name
|
||
encryption.
|
||
If you keep your file names to below 156 characters in length then you
|
||
should be OK on all providers.
|
||
.PP
|
||
There may be an even more secure file name encryption mode in the future
|
||
which will address the long file name problem.
|
||
.SS Directory name encryption
|
||
.PP
|
||
Crypt offers the option of encrypting dir names or leaving them intact.
|
||
There are two options:
|
||
.PP
|
||
True
|
||
.PP
|
||
Encrypts the whole file path including directory names Example:
|
||
\f[C]1/12/123.txt\f[] is encrypted to
|
||
\f[C]p0e52nreeaj0a5ea7s64m4j72s/l42g6771hnv3an9cgc8cr2n1ng/qgm4avr35m5loi1th53ato71v0\f[]
|
||
.PP
|
||
False
|
||
.PP
|
||
Only encrypts file names, skips directory names Example:
|
||
\f[C]1/12/123/txt\f[] is encrypted to
|
||
\f[C]1/12/qgm4avr35m5loi1th53ato71v0\f[]
|
||
.SS Modified time and hashes
|
||
.PP
|
||
Crypt stores modification times using the underlying remote so support
|
||
depends on that.
|
||
.PP
|
||
Hashes are not stored for crypt.
|
||
However the data integrity is protected by an extremely strong crypto
|
||
authenticator.
|
||
.PP
|
||
Note that you should use the \f[C]rclone\ cryptcheck\f[] command to
|
||
check the integrity of a crypted remote instead of
|
||
\f[C]rclone\ check\f[] which can\[aq]t check the checksums properly.
|
||
.SS Specific options
|
||
.PP
|
||
Here are the command line options specific to this cloud storage system.
|
||
.SS \-\-crypt\-show\-mapping
|
||
.PP
|
||
If this flag is set then for each file that the remote is asked to list,
|
||
it will log (at level INFO) a line stating the decrypted file name and
|
||
the encrypted file name.
|
||
.PP
|
||
This is so you can work out which encrypted names are which decrypted
|
||
names just in case you need to do something with the encrypted file
|
||
names, or for debugging purposes.
|
||
.SS Backing up a crypted remote
|
||
.PP
|
||
If you wish to backup a crypted remote, it it recommended that you use
|
||
\f[C]rclone\ sync\f[] on the encrypted files, and make sure the
|
||
passwords are the same in the new encrypted remote.
|
||
.PP
|
||
This will have the following advantages
|
||
.IP \[bu] 2
|
||
\f[C]rclone\ sync\f[] will check the checksums while copying
|
||
.IP \[bu] 2
|
||
you can use \f[C]rclone\ check\f[] between the encrypted remotes
|
||
.IP \[bu] 2
|
||
you don\[aq]t decrypt and encrypt unnecessarily
|
||
.PP
|
||
For example, let\[aq]s say you have your original remote at
|
||
\f[C]remote:\f[] with the encrypted version at \f[C]eremote:\f[] with
|
||
path \f[C]remote:crypt\f[].
|
||
You would then set up the new remote \f[C]remote2:\f[] and then the
|
||
encrypted version \f[C]eremote2:\f[] with path \f[C]remote2:crypt\f[]
|
||
using the same passwords as \f[C]eremote:\f[].
|
||
.PP
|
||
To sync the two remotes you would do
|
||
.IP
|
||
.nf
|
||
\f[C]
|
||
rclone\ sync\ remote:crypt\ remote2:crypt
|
||
\f[]
|
||
.fi
|
||
.PP
|
||
And to check the integrity you would do
|
||
.IP
|
||
.nf
|
||
\f[C]
|
||
rclone\ check\ remote:crypt\ remote2:crypt
|
||
\f[]
|
||
.fi
|
||
.SS File formats
|
||
.SS File encryption
|
||
.PP
|
||
Files are encrypted 1:1 source file to destination object.
|
||
The file has a header and is divided into chunks.
|
||
.SS Header
|
||
.IP \[bu] 2
|
||
8 bytes magic string \f[C]RCLONE\\x00\\x00\f[]
|
||
.IP \[bu] 2
|
||
24 bytes Nonce (IV)
|
||
.PP
|
||
The initial nonce is generated from the operating systems crypto strong
|
||
random number generator.
|
||
The nonce is incremented for each chunk read making sure each nonce is
|
||
unique for each block written.
|
||
The chance of a nonce being re\-used is minuscule.
|
||
If you wrote an exabyte of data (10¹⁸ bytes) you would have a
|
||
probability of approximately 2×10⁻³² of re\-using a nonce.
|
||
.SS Chunk
|
||
.PP
|
||
Each chunk will contain 64kB of data, except for the last one which may
|
||
have less data.
|
||
The data chunk is in standard NACL secretbox format.
|
||
Secretbox uses XSalsa20 and Poly1305 to encrypt and authenticate
|
||
messages.
|
||
.PP
|
||
Each chunk contains:
|
||
.IP \[bu] 2
|
||
16 Bytes of Poly1305 authenticator
|
||
.IP \[bu] 2
|
||
1 \- 65536 bytes XSalsa20 encrypted data
|
||
.PP
|
||
64k chunk size was chosen as the best performing chunk size (the
|
||
authenticator takes too much time below this and the performance drops
|
||
off due to cache effects above this).
|
||
Note that these chunks are buffered in memory so they can\[aq]t be too
|
||
big.
|
||
.PP
|
||
This uses a 32 byte (256 bit key) key derived from the user password.
|
||
.SS Examples
|
||
.PP
|
||
1 byte file will encrypt to
|
||
.IP \[bu] 2
|
||
32 bytes header
|
||
.IP \[bu] 2
|
||
17 bytes data chunk
|
||
.PP
|
||
49 bytes total
|
||
.PP
|
||
1MB (1048576 bytes) file will encrypt to
|
||
.IP \[bu] 2
|
||
32 bytes header
|
||
.IP \[bu] 2
|
||
16 chunks of 65568 bytes
|
||
.PP
|
||
1049120 bytes total (a 0.05% overhead).
|
||
This is the overhead for big files.
|
||
.SS Name encryption
|
||
.PP
|
||
File names are encrypted segment by segment \- the path is broken up
|
||
into \f[C]/\f[] separated strings and these are encrypted individually.
|
||
.PP
|
||
File segments are padded using using PKCS#7 to a multiple of 16 bytes
|
||
before encryption.
|
||
.PP
|
||
They are then encrypted with EME using AES with 256 bit key.
|
||
EME (ECB\-Mix\-ECB) is a wide\-block encryption mode presented in the
|
||
2003 paper "A Parallelizable Enciphering Mode" by Halevi and Rogaway.
|
||
.PP
|
||
This makes for deterministic encryption which is what we want \- the
|
||
same filename must encrypt to the same thing otherwise we can\[aq]t find
|
||
it on the cloud storage system.
|
||
.PP
|
||
This means that
|
||
.IP \[bu] 2
|
||
filenames with the same name will encrypt the same
|
||
.IP \[bu] 2
|
||
filenames which start the same won\[aq]t have a common prefix
|
||
.PP
|
||
This uses a 32 byte key (256 bits) and a 16 byte (128 bits) IV both of
|
||
which are derived from the user password.
|
||
.PP
|
||
After encryption they are written out using a modified version of
|
||
standard \f[C]base32\f[] encoding as described in RFC4648.
|
||
The standard encoding is modified in two ways:
|
||
.IP \[bu] 2
|
||
it becomes lower case (no\-one likes upper case filenames!)
|
||
.IP \[bu] 2
|
||
we strip the padding character \f[C]=\f[]
|
||
.PP
|
||
\f[C]base32\f[] is used rather than the more efficient \f[C]base64\f[]
|
||
so rclone can be used on case insensitive remotes (eg Windows, Amazon
|
||
Drive).
|
||
.SS Key derivation
|
||
.PP
|
||
Rclone uses \f[C]scrypt\f[] with parameters \f[C]N=16384,\ r=8,\ p=1\f[]
|
||
with an optional user supplied salt (password2) to derive the 32+32+16 =
|
||
80 bytes of key material required.
|
||
If the user doesn\[aq]t supply a salt then rclone uses an internal one.
|
||
.PP
|
||
\f[C]scrypt\f[] makes it impractical to mount a dictionary attack on
|
||
rclone encrypted data.
|
||
For full protection against this you should always use a salt.
|
||
.SS Dropbox
|
||
.PP
|
||
Paths are specified as \f[C]remote:path\f[]
|
||
.PP
|
||
Dropbox paths may be as deep as required, eg
|
||
\f[C]remote:directory/subdirectory\f[].
|
||
.PP
|
||
The initial setup for dropbox involves getting a token from Dropbox
|
||
which you need to do in your browser.
|
||
\f[C]rclone\ config\f[] walks you through it.
|
||
.PP
|
||
Here is an example of how to make a remote called \f[C]remote\f[].
|
||
First run:
|
||
.IP
|
||
.nf
|
||
\f[C]
|
||
\ rclone\ config
|
||
\f[]
|
||
.fi
|
||
.PP
|
||
This will guide you through an interactive setup process:
|
||
.IP
|
||
.nf
|
||
\f[C]
|
||
n)\ New\ remote
|
||
d)\ Delete\ remote
|
||
q)\ Quit\ config
|
||
e/n/d/q>\ n
|
||
name>\ remote
|
||
Type\ of\ storage\ to\ configure.
|
||
Choose\ a\ number\ from\ below,\ or\ type\ in\ your\ own\ value
|
||
\ 1\ /\ Amazon\ Drive
|
||
\ \ \ \\\ "amazon\ cloud\ drive"
|
||
\ 2\ /\ Amazon\ S3\ (also\ Dreamhost,\ Ceph,\ Minio)
|
||
\ \ \ \\\ "s3"
|
||
\ 3\ /\ Backblaze\ B2
|
||
\ \ \ \\\ "b2"
|
||
\ 4\ /\ Dropbox
|
||
\ \ \ \\\ "dropbox"
|
||
\ 5\ /\ Encrypt/Decrypt\ a\ remote
|
||
\ \ \ \\\ "crypt"
|
||
\ 6\ /\ Google\ Cloud\ Storage\ (this\ is\ not\ Google\ Drive)
|
||
\ \ \ \\\ "google\ cloud\ storage"
|
||
\ 7\ /\ Google\ Drive
|
||
\ \ \ \\\ "drive"
|
||
\ 8\ /\ Hubic
|
||
\ \ \ \\\ "hubic"
|
||
\ 9\ /\ Local\ Disk
|
||
\ \ \ \\\ "local"
|
||
10\ /\ Microsoft\ OneDrive
|
||
\ \ \ \\\ "onedrive"
|
||
11\ /\ Openstack\ Swift\ (Rackspace\ Cloud\ Files,\ Memset\ Memstore,\ OVH)
|
||
\ \ \ \\\ "swift"
|
||
12\ /\ SSH/SFTP\ Connection
|
||
\ \ \ \\\ "sftp"
|
||
13\ /\ Yandex\ Disk
|
||
\ \ \ \\\ "yandex"
|
||
Storage>\ 4
|
||
Dropbox\ App\ Key\ \-\ leave\ blank\ normally.
|
||
app_key>
|
||
Dropbox\ App\ Secret\ \-\ leave\ blank\ normally.
|
||
app_secret>
|
||
Remote\ config
|
||
Please\ visit:
|
||
https://www.dropbox.com/1/oauth2/authorize?client_id=XXXXXXXXXXXXXXX&response_type=code
|
||
Enter\ the\ code:\ XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX_XXXXXXXXXX
|
||
\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-
|
||
[remote]
|
||
app_key\ =
|
||
app_secret\ =
|
||
token\ =\ XXXXXXXXXXXXXXXXXXXXXXXXXXXXX_XXXX_XXXXXXXXXXXXXXXXXXXXXXXXXXXXX
|
||
\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-
|
||
y)\ Yes\ this\ is\ OK
|
||
e)\ Edit\ this\ remote
|
||
d)\ Delete\ this\ remote
|
||
y/e/d>\ y
|
||
\f[]
|
||
.fi
|
||
.PP
|
||
You can then use it like this,
|
||
.PP
|
||
List directories in top level of your dropbox
|
||
.IP
|
||
.nf
|
||
\f[C]
|
||
rclone\ lsd\ remote:
|
||
\f[]
|
||
.fi
|
||
.PP
|
||
List all the files in your dropbox
|
||
.IP
|
||
.nf
|
||
\f[C]
|
||
rclone\ ls\ remote:
|
||
\f[]
|
||
.fi
|
||
.PP
|
||
To copy a local directory to a dropbox directory called backup
|
||
.IP
|
||
.nf
|
||
\f[C]
|
||
rclone\ copy\ /home/source\ remote:backup
|
||
\f[]
|
||
.fi
|
||
.SS Modified time and Hashes
|
||
.PP
|
||
Dropbox supports modified times, but the only way to set a modification
|
||
time is to re\-upload the file.
|
||
.PP
|
||
This means that if you uploaded your data with an older version of
|
||
rclone which didn\[aq]t support the v2 API and modified times, rclone
|
||
will decide to upload all your old data to fix the modification times.
|
||
If you don\[aq]t want this to happen use \f[C]\-\-size\-only\f[] or
|
||
\f[C]\-\-checksum\f[] flag to stop it.
|
||
.PP
|
||
Dropbox supports its own hash
|
||
type (https://www.dropbox.com/developers/reference/content-hash) which
|
||
is checked for all transfers.
|
||
.SS Specific options
|
||
.PP
|
||
Here are the command line options specific to this cloud storage system.
|
||
.SS \-\-dropbox\-chunk\-size=SIZE
|
||
.PP
|
||
Any files larger than this will be uploaded in chunks of this size.
|
||
The default is 48MB.
|
||
The maximum is 150MB.
|
||
.PP
|
||
Note that chunks are buffered in memory (one at a time) so rclone can
|
||
deal with retries.
|
||
Setting this larger will increase the speed slightly (at most 10% for
|
||
128MB in tests) at the cost of using more memory.
|
||
It can be set smaller if you are tight on memory.
|
||
.SS Limitations
|
||
.PP
|
||
Note that Dropbox is case insensitive so you can\[aq]t have a file
|
||
called "Hello.doc" and one called "hello.doc".
|
||
.PP
|
||
There are some file names such as \f[C]thumbs.db\f[] which Dropbox
|
||
can\[aq]t store.
|
||
There is a full list of them in the "Ignored Files" section of this
|
||
document (https://www.dropbox.com/en/help/145).
|
||
Rclone will issue an error message
|
||
\f[C]File\ name\ disallowed\ \-\ not\ uploading\f[] if it attempt to
|
||
upload one of those file names, but the sync won\[aq]t fail.
|
||
.PP
|
||
If you have more than 10,000 files in a directory then
|
||
\f[C]rclone\ purge\ dropbox:dir\f[] will return the error
|
||
\f[C]Failed\ to\ purge:\ There\ are\ too\ many\ files\ involved\ in\ this\ operation\f[].
|
||
As a work\-around do an \f[C]rclone\ delete\ dropbox:dir\f[] followed by
|
||
an \f[C]rclone\ rmdir\ dropbox:dir\f[].
|
||
.SS FTP
|
||
.PP
|
||
FTP is the File Transfer Protocol.
|
||
FTP support is provided using the
|
||
github.com/jlaffaye/ftp (https://godoc.org/github.com/jlaffaye/ftp)
|
||
package.
|
||
.PP
|
||
Here is an example of making an FTP configuration.
|
||
First run
|
||
.IP
|
||
.nf
|
||
\f[C]
|
||
rclone\ config
|
||
\f[]
|
||
.fi
|
||
.PP
|
||
This will guide you through an interactive setup process.
|
||
An FTP remote only needs a host together with and a username and a
|
||
password.
|
||
With anonymous FTP server, you will need to use \f[C]anonymous\f[] as
|
||
username and your email address as the password.
|
||
.IP
|
||
.nf
|
||
\f[C]
|
||
No\ remotes\ found\ \-\ make\ a\ new\ one
|
||
n)\ New\ remote
|
||
r)\ Rename\ remote
|
||
c)\ Copy\ remote
|
||
s)\ Set\ configuration\ password
|
||
q)\ Quit\ config
|
||
n/r/c/s/q>\ n
|
||
name>\ remote
|
||
Type\ of\ storage\ to\ configure.
|
||
Choose\ a\ number\ from\ below,\ or\ type\ in\ your\ own\ value
|
||
\ 1\ /\ Amazon\ Drive
|
||
\ \ \ \\\ "amazon\ cloud\ drive"
|
||
\ 2\ /\ Amazon\ S3\ (also\ Dreamhost,\ Ceph,\ Minio)
|
||
\ \ \ \\\ "s3"
|
||
\ 3\ /\ Backblaze\ B2
|
||
\ \ \ \\\ "b2"
|
||
\ 4\ /\ Dropbox
|
||
\ \ \ \\\ "dropbox"
|
||
\ 5\ /\ Encrypt/Decrypt\ a\ remote
|
||
\ \ \ \\\ "crypt"
|
||
\ 6\ /\ FTP\ Connection\
|
||
\ \ \ \\\ "ftp"
|
||
\ 7\ /\ Google\ Cloud\ Storage\ (this\ is\ not\ Google\ Drive)
|
||
\ \ \ \\\ "google\ cloud\ storage"
|
||
\ 8\ /\ Google\ Drive
|
||
\ \ \ \\\ "drive"
|
||
\ 9\ /\ Hubic
|
||
\ \ \ \\\ "hubic"
|
||
10\ /\ Local\ Disk
|
||
\ \ \ \\\ "local"
|
||
11\ /\ Microsoft\ OneDrive
|
||
\ \ \ \\\ "onedrive"
|
||
12\ /\ Openstack\ Swift\ (Rackspace\ Cloud\ Files,\ Memset\ Memstore,\ OVH)
|
||
\ \ \ \\\ "swift"
|
||
13\ /\ SSH/SFTP\ Connection
|
||
\ \ \ \\\ "sftp"
|
||
14\ /\ Yandex\ Disk
|
||
\ \ \ \\\ "yandex"
|
||
Storage>\ ftp
|
||
FTP\ host\ to\ connect\ to
|
||
Choose\ a\ number\ from\ below,\ or\ type\ in\ your\ own\ value
|
||
\ 1\ /\ Connect\ to\ ftp.example.com
|
||
\ \ \ \\\ "ftp.example.com"
|
||
host>\ ftp.example.com
|
||
FTP\ username,\ leave\ blank\ for\ current\ username,\ ncw
|
||
user>
|
||
FTP\ port,\ leave\ blank\ to\ use\ default\ (21)
|
||
port>
|
||
FTP\ password
|
||
y)\ Yes\ type\ in\ my\ own\ password
|
||
g)\ Generate\ random\ password
|
||
y/g>\ y
|
||
Enter\ the\ password:
|
||
password:
|
||
Confirm\ the\ password:
|
||
password:
|
||
Remote\ config
|
||
\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-
|
||
[remote]
|
||
host\ =\ ftp.example.com
|
||
user\ =\
|
||
port\ =
|
||
pass\ =\ ***\ ENCRYPTED\ ***
|
||
\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-
|
||
y)\ Yes\ this\ is\ OK
|
||
e)\ Edit\ this\ remote
|
||
d)\ Delete\ this\ remote
|
||
y/e/d>\ y
|
||
\f[]
|
||
.fi
|
||
.PP
|
||
This remote is called \f[C]remote\f[] and can now be used like this
|
||
.PP
|
||
See all directories in the home directory
|
||
.IP
|
||
.nf
|
||
\f[C]
|
||
rclone\ lsd\ remote:
|
||
\f[]
|
||
.fi
|
||
.PP
|
||
Make a new directory
|
||
.IP
|
||
.nf
|
||
\f[C]
|
||
rclone\ mkdir\ remote:path/to/directory
|
||
\f[]
|
||
.fi
|
||
.PP
|
||
List the contents of a directory
|
||
.IP
|
||
.nf
|
||
\f[C]
|
||
rclone\ ls\ remote:path/to/directory
|
||
\f[]
|
||
.fi
|
||
.PP
|
||
Sync \f[C]/home/local/directory\f[] to the remote directory, deleting
|
||
any excess files in the directory.
|
||
.IP
|
||
.nf
|
||
\f[C]
|
||
rclone\ sync\ /home/local/directory\ remote:directory
|
||
\f[]
|
||
.fi
|
||
.SS Modified time
|
||
.PP
|
||
FTP does not support modified times.
|
||
Any times you see on the server will be time of upload.
|
||
.SS Checksums
|
||
.PP
|
||
FTP does not support any checksums.
|
||
.SS Limitations
|
||
.PP
|
||
Note that since FTP isn\[aq]t HTTP based the following flags don\[aq]t
|
||
work with it: \f[C]\-\-dump\-headers\f[], \f[C]\-\-dump\-bodies\f[],
|
||
\f[C]\-\-dump\-auth\f[]
|
||
.PP
|
||
Note that \f[C]\-\-timeout\f[] isn\[aq]t supported (but
|
||
\f[C]\-\-contimeout\f[] is).
|
||
.PP
|
||
Note that \f[C]\-\-bind\f[] isn\[aq]t supported.
|
||
.PP
|
||
FTP could support server side move but doesn\[aq]t yet.
|
||
.SS Google Cloud Storage
|
||
.PP
|
||
Paths are specified as \f[C]remote:bucket\f[] (or \f[C]remote:\f[] for
|
||
the \f[C]lsd\f[] command.) You may put subdirectories in too, eg
|
||
\f[C]remote:bucket/path/to/dir\f[].
|
||
.PP
|
||
The initial setup for google cloud storage involves getting a token from
|
||
Google Cloud Storage which you need to do in your browser.
|
||
\f[C]rclone\ config\f[] walks you through it.
|
||
.PP
|
||
Here is an example of how to make a remote called \f[C]remote\f[].
|
||
First run:
|
||
.IP
|
||
.nf
|
||
\f[C]
|
||
\ rclone\ config
|
||
\f[]
|
||
.fi
|
||
.PP
|
||
This will guide you through an interactive setup process:
|
||
.IP
|
||
.nf
|
||
\f[C]
|
||
n)\ New\ remote
|
||
d)\ Delete\ remote
|
||
q)\ Quit\ config
|
||
e/n/d/q>\ n
|
||
name>\ remote
|
||
Type\ of\ storage\ to\ configure.
|
||
Choose\ a\ number\ from\ below,\ or\ type\ in\ your\ own\ value
|
||
\ 1\ /\ Amazon\ Drive
|
||
\ \ \ \\\ "amazon\ cloud\ drive"
|
||
\ 2\ /\ Amazon\ S3\ (also\ Dreamhost,\ Ceph,\ Minio)
|
||
\ \ \ \\\ "s3"
|
||
\ 3\ /\ Backblaze\ B2
|
||
\ \ \ \\\ "b2"
|
||
\ 4\ /\ Dropbox
|
||
\ \ \ \\\ "dropbox"
|
||
\ 5\ /\ Encrypt/Decrypt\ a\ remote
|
||
\ \ \ \\\ "crypt"
|
||
\ 6\ /\ Google\ Cloud\ Storage\ (this\ is\ not\ Google\ Drive)
|
||
\ \ \ \\\ "google\ cloud\ storage"
|
||
\ 7\ /\ Google\ Drive
|
||
\ \ \ \\\ "drive"
|
||
\ 8\ /\ Hubic
|
||
\ \ \ \\\ "hubic"
|
||
\ 9\ /\ Local\ Disk
|
||
\ \ \ \\\ "local"
|
||
10\ /\ Microsoft\ OneDrive
|
||
\ \ \ \\\ "onedrive"
|
||
11\ /\ Openstack\ Swift\ (Rackspace\ Cloud\ Files,\ Memset\ Memstore,\ OVH)
|
||
\ \ \ \\\ "swift"
|
||
12\ /\ SSH/SFTP\ Connection
|
||
\ \ \ \\\ "sftp"
|
||
13\ /\ Yandex\ Disk
|
||
\ \ \ \\\ "yandex"
|
||
Storage>\ 6
|
||
Google\ Application\ Client\ Id\ \-\ leave\ blank\ normally.
|
||
client_id>
|
||
Google\ Application\ Client\ Secret\ \-\ leave\ blank\ normally.
|
||
client_secret>
|
||
Project\ number\ optional\ \-\ needed\ only\ for\ list/create/delete\ buckets\ \-\ see\ your\ developer\ console.
|
||
project_number>\ 12345678
|
||
Service\ Account\ Credentials\ JSON\ file\ path\ \-\ needed\ only\ if\ you\ want\ use\ SA\ instead\ of\ interactive\ login.
|
||
service_account_file>
|
||
Access\ Control\ List\ for\ new\ objects.
|
||
Choose\ a\ number\ from\ below,\ or\ type\ in\ your\ own\ value
|
||
\ 1\ /\ Object\ owner\ gets\ OWNER\ access,\ and\ all\ Authenticated\ Users\ get\ READER\ access.
|
||
\ \ \ \\\ "authenticatedRead"
|
||
\ 2\ /\ Object\ owner\ gets\ OWNER\ access,\ and\ project\ team\ owners\ get\ OWNER\ access.
|
||
\ \ \ \\\ "bucketOwnerFullControl"
|
||
\ 3\ /\ Object\ owner\ gets\ OWNER\ access,\ and\ project\ team\ owners\ get\ READER\ access.
|
||
\ \ \ \\\ "bucketOwnerRead"
|
||
\ 4\ /\ Object\ owner\ gets\ OWNER\ access\ [default\ if\ left\ blank].
|
||
\ \ \ \\\ "private"
|
||
\ 5\ /\ Object\ owner\ gets\ OWNER\ access,\ and\ project\ team\ members\ get\ access\ according\ to\ their\ roles.
|
||
\ \ \ \\\ "projectPrivate"
|
||
\ 6\ /\ Object\ owner\ gets\ OWNER\ access,\ and\ all\ Users\ get\ READER\ access.
|
||
\ \ \ \\\ "publicRead"
|
||
object_acl>\ 4
|
||
Access\ Control\ List\ for\ new\ buckets.
|
||
Choose\ a\ number\ from\ below,\ or\ type\ in\ your\ own\ value
|
||
\ 1\ /\ Project\ team\ owners\ get\ OWNER\ access,\ and\ all\ Authenticated\ Users\ get\ READER\ access.
|
||
\ \ \ \\\ "authenticatedRead"
|
||
\ 2\ /\ Project\ team\ owners\ get\ OWNER\ access\ [default\ if\ left\ blank].
|
||
\ \ \ \\\ "private"
|
||
\ 3\ /\ Project\ team\ members\ get\ access\ according\ to\ their\ roles.
|
||
\ \ \ \\\ "projectPrivate"
|
||
\ 4\ /\ Project\ team\ owners\ get\ OWNER\ access,\ and\ all\ Users\ get\ READER\ access.
|
||
\ \ \ \\\ "publicRead"
|
||
\ 5\ /\ Project\ team\ owners\ get\ OWNER\ access,\ and\ all\ Users\ get\ WRITER\ access.
|
||
\ \ \ \\\ "publicReadWrite"
|
||
bucket_acl>\ 2
|
||
Location\ for\ the\ newly\ created\ buckets.
|
||
Choose\ a\ number\ from\ below,\ or\ type\ in\ your\ own\ value
|
||
\ 1\ /\ Empty\ for\ default\ location\ (US).
|
||
\ \ \ \\\ ""
|
||
\ 2\ /\ Multi\-regional\ location\ for\ Asia.
|
||
\ \ \ \\\ "asia"
|
||
\ 3\ /\ Multi\-regional\ location\ for\ Europe.
|
||
\ \ \ \\\ "eu"
|
||
\ 4\ /\ Multi\-regional\ location\ for\ United\ States.
|
||
\ \ \ \\\ "us"
|
||
\ 5\ /\ Taiwan.
|
||
\ \ \ \\\ "asia\-east1"
|
||
\ 6\ /\ Tokyo.
|
||
\ \ \ \\\ "asia\-northeast1"
|
||
\ 7\ /\ Singapore.
|
||
\ \ \ \\\ "asia\-southeast1"
|
||
\ 8\ /\ Sydney.
|
||
\ \ \ \\\ "australia\-southeast1"
|
||
\ 9\ /\ Belgium.
|
||
\ \ \ \\\ "europe\-west1"
|
||
10\ /\ London.
|
||
\ \ \ \\\ "europe\-west2"
|
||
11\ /\ Iowa.
|
||
\ \ \ \\\ "us\-central1"
|
||
12\ /\ South\ Carolina.
|
||
\ \ \ \\\ "us\-east1"
|
||
13\ /\ Northern\ Virginia.
|
||
\ \ \ \\\ "us\-east4"
|
||
14\ /\ Oregon.
|
||
\ \ \ \\\ "us\-west1"
|
||
location>\ 12
|
||
The\ storage\ class\ to\ use\ when\ storing\ objects\ in\ Google\ Cloud\ Storage.
|
||
Choose\ a\ number\ from\ below,\ or\ type\ in\ your\ own\ value
|
||
\ 1\ /\ Default
|
||
\ \ \ \\\ ""
|
||
\ 2\ /\ Multi\-regional\ storage\ class
|
||
\ \ \ \\\ "MULTI_REGIONAL"
|
||
\ 3\ /\ Regional\ storage\ class
|
||
\ \ \ \\\ "REGIONAL"
|
||
\ 4\ /\ Nearline\ storage\ class
|
||
\ \ \ \\\ "NEARLINE"
|
||
\ 5\ /\ Coldline\ storage\ class
|
||
\ \ \ \\\ "COLDLINE"
|
||
\ 6\ /\ Durable\ reduced\ availability\ storage\ class
|
||
\ \ \ \\\ "DURABLE_REDUCED_AVAILABILITY"
|
||
storage_class>\ 5
|
||
Remote\ config
|
||
Use\ auto\ config?
|
||
\ *\ Say\ Y\ if\ not\ sure
|
||
\ *\ Say\ N\ if\ you\ are\ working\ on\ a\ remote\ or\ headless\ machine\ or\ Y\ didn\[aq]t\ work
|
||
y)\ Yes
|
||
n)\ No
|
||
y/n>\ y
|
||
If\ your\ browser\ doesn\[aq]t\ open\ automatically\ go\ to\ the\ following\ link:\ http://127.0.0.1:53682/auth
|
||
Log\ in\ and\ authorize\ rclone\ for\ access
|
||
Waiting\ for\ code...
|
||
Got\ code
|
||
\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-
|
||
[remote]
|
||
type\ =\ google\ cloud\ storage
|
||
client_id\ =
|
||
client_secret\ =
|
||
token\ =\ {"AccessToken":"xxxx.xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx\-xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx","RefreshToken":"x/xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx_xxxxxxxxx","Expiry":"2014\-07\-17T20:49:14.929208288+01:00","Extra":null}
|
||
project_number\ =\ 12345678
|
||
object_acl\ =\ private
|
||
bucket_acl\ =\ private
|
||
\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-
|
||
y)\ Yes\ this\ is\ OK
|
||
e)\ Edit\ this\ remote
|
||
d)\ Delete\ this\ remote
|
||
y/e/d>\ y
|
||
\f[]
|
||
.fi
|
||
.PP
|
||
Note that rclone runs a webserver on your local machine to collect the
|
||
token as returned from Google if you use auto config mode.
|
||
This only runs from the moment it opens your browser to the moment you
|
||
get back the verification code.
|
||
This is on \f[C]http://127.0.0.1:53682/\f[] and this it may require you
|
||
to unblock it temporarily if you are running a host firewall, or use
|
||
manual mode.
|
||
.PP
|
||
This remote is called \f[C]remote\f[] and can now be used like this
|
||
.PP
|
||
See all the buckets in your project
|
||
.IP
|
||
.nf
|
||
\f[C]
|
||
rclone\ lsd\ remote:
|
||
\f[]
|
||
.fi
|
||
.PP
|
||
Make a new bucket
|
||
.IP
|
||
.nf
|
||
\f[C]
|
||
rclone\ mkdir\ remote:bucket
|
||
\f[]
|
||
.fi
|
||
.PP
|
||
List the contents of a bucket
|
||
.IP
|
||
.nf
|
||
\f[C]
|
||
rclone\ ls\ remote:bucket
|
||
\f[]
|
||
.fi
|
||
.PP
|
||
Sync \f[C]/home/local/directory\f[] to the remote bucket, deleting any
|
||
excess files in the bucket.
|
||
.IP
|
||
.nf
|
||
\f[C]
|
||
rclone\ sync\ /home/local/directory\ remote:bucket
|
||
\f[]
|
||
.fi
|
||
.SS Service Account support
|
||
.PP
|
||
You can set up rclone with Google Cloud Storage in an unattended mode,
|
||
i.e.
|
||
not tied to a specific end\-user Google account.
|
||
This is useful when you want to synchronise files onto machines that
|
||
don\[aq]t have actively logged\-in users, for example build machines.
|
||
.PP
|
||
To get credentials for Google Cloud Platform IAM Service
|
||
Accounts (https://cloud.google.com/iam/docs/service-accounts), please
|
||
head to the Service
|
||
Account (https://console.cloud.google.com/permissions/serviceaccounts)
|
||
section of the Google Developer Console.
|
||
Service Accounts behave just like normal \f[C]User\f[] permissions in
|
||
Google Cloud Storage
|
||
ACLs (https://cloud.google.com/storage/docs/access-control), so you can
|
||
limit their access (e.g.
|
||
make them read only).
|
||
After creating an account, a JSON file containing the Service
|
||
Account\[aq]s credentials will be downloaded onto your machines.
|
||
These credentials are what rclone will use for authentication.
|
||
.PP
|
||
To use a Service Account instead of OAuth2 token flow, enter the path to
|
||
your Service Account credentials at the \f[C]service_account_file\f[]
|
||
prompt and rclone won\[aq]t use the browser based authentication flow.
|
||
.SS \-\-fast\-list
|
||
.PP
|
||
This remote supports \f[C]\-\-fast\-list\f[] which allows you to use
|
||
fewer transactions in exchange for more memory.
|
||
See the rclone docs (/docs/#fast-list) for more details.
|
||
.SS Modified time
|
||
.PP
|
||
Google google cloud storage stores md5sums natively and rclone stores
|
||
modification times as metadata on the object, under the "mtime" key in
|
||
RFC3339 format accurate to 1ns.
|
||
.SS Google Drive
|
||
.PP
|
||
Paths are specified as \f[C]drive:path\f[]
|
||
.PP
|
||
Drive paths may be as deep as required, eg
|
||
\f[C]drive:directory/subdirectory\f[].
|
||
.PP
|
||
The initial setup for drive involves getting a token from Google drive
|
||
which you need to do in your browser.
|
||
\f[C]rclone\ config\f[] walks you through it.
|
||
.PP
|
||
Here is an example of how to make a remote called \f[C]remote\f[].
|
||
First run:
|
||
.IP
|
||
.nf
|
||
\f[C]
|
||
\ rclone\ config
|
||
\f[]
|
||
.fi
|
||
.PP
|
||
This will guide you through an interactive setup process:
|
||
.IP
|
||
.nf
|
||
\f[C]
|
||
No\ remotes\ found\ \-\ make\ a\ new\ one
|
||
n)\ New\ remote
|
||
r)\ Rename\ remote
|
||
c)\ Copy\ remote
|
||
s)\ Set\ configuration\ password
|
||
q)\ Quit\ config
|
||
n/r/c/s/q>\ n
|
||
name>\ remote
|
||
Type\ of\ storage\ to\ configure.
|
||
Choose\ a\ number\ from\ below,\ or\ type\ in\ your\ own\ value
|
||
\ 1\ /\ Amazon\ Drive
|
||
\ \ \ \\\ "amazon\ cloud\ drive"
|
||
\ 2\ /\ Amazon\ S3\ (also\ Dreamhost,\ Ceph,\ Minio)
|
||
\ \ \ \\\ "s3"
|
||
\ 3\ /\ Backblaze\ B2
|
||
\ \ \ \\\ "b2"
|
||
\ 4\ /\ Dropbox
|
||
\ \ \ \\\ "dropbox"
|
||
\ 5\ /\ Encrypt/Decrypt\ a\ remote
|
||
\ \ \ \\\ "crypt"
|
||
\ 6\ /\ FTP\ Connection
|
||
\ \ \ \\\ "ftp"
|
||
\ 7\ /\ Google\ Cloud\ Storage\ (this\ is\ not\ Google\ Drive)
|
||
\ \ \ \\\ "google\ cloud\ storage"
|
||
\ 8\ /\ Google\ Drive
|
||
\ \ \ \\\ "drive"
|
||
\ 9\ /\ Hubic
|
||
\ \ \ \\\ "hubic"
|
||
10\ /\ Local\ Disk
|
||
\ \ \ \\\ "local"
|
||
11\ /\ Microsoft\ OneDrive
|
||
\ \ \ \\\ "onedrive"
|
||
12\ /\ Openstack\ Swift\ (Rackspace\ Cloud\ Files,\ Memset\ Memstore,\ OVH)
|
||
\ \ \ \\\ "swift"
|
||
13\ /\ SSH/SFTP\ Connection
|
||
\ \ \ \\\ "sftp"
|
||
14\ /\ Yandex\ Disk
|
||
\ \ \ \\\ "yandex"
|
||
Storage>\ 8
|
||
Google\ Application\ Client\ Id\ \-\ leave\ blank\ normally.
|
||
client_id>
|
||
Google\ Application\ Client\ Secret\ \-\ leave\ blank\ normally.
|
||
client_secret>
|
||
Service\ Account\ Credentials\ JSON\ file\ path\ \-\ needed\ only\ if\ you\ want\ use\ SA\ instead\ of\ interactive\ login.
|
||
service_account_file>
|
||
Remote\ config
|
||
Use\ auto\ config?
|
||
\ *\ Say\ Y\ if\ not\ sure
|
||
\ *\ Say\ N\ if\ you\ are\ working\ on\ a\ remote\ or\ headless\ machine\ or\ Y\ didn\[aq]t\ work
|
||
y)\ Yes
|
||
n)\ No
|
||
y/n>\ y
|
||
If\ your\ browser\ doesn\[aq]t\ open\ automatically\ go\ to\ the\ following\ link:\ http://127.0.0.1:53682/auth
|
||
Log\ in\ and\ authorize\ rclone\ for\ access
|
||
Waiting\ for\ code...
|
||
Got\ code
|
||
Configure\ this\ as\ a\ team\ drive?
|
||
y)\ Yes
|
||
n)\ No
|
||
y/n>\ n
|
||
\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-
|
||
[remote]
|
||
client_id\ =
|
||
client_secret\ =
|
||
token\ =\ {"AccessToken":"xxxx.x.xxxxx_xxxxxxxxxxx_xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx","RefreshToken":"1/xxxxxxxxxxxxxxxx_xxxxxxxxxxxxxxxxxxxxxxxxxx","Expiry":"2014\-03\-16T13:57:58.955387075Z","Extra":null}
|
||
\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-
|
||
y)\ Yes\ this\ is\ OK
|
||
e)\ Edit\ this\ remote
|
||
d)\ Delete\ this\ remote
|
||
y/e/d>\ y
|
||
\f[]
|
||
.fi
|
||
.PP
|
||
Note that rclone runs a webserver on your local machine to collect the
|
||
token as returned from Google if you use auto config mode.
|
||
This only runs from the moment it opens your browser to the moment you
|
||
get back the verification code.
|
||
This is on \f[C]http://127.0.0.1:53682/\f[] and this it may require you
|
||
to unblock it temporarily if you are running a host firewall, or use
|
||
manual mode.
|
||
.PP
|
||
You can then use it like this,
|
||
.PP
|
||
List directories in top level of your drive
|
||
.IP
|
||
.nf
|
||
\f[C]
|
||
rclone\ lsd\ remote:
|
||
\f[]
|
||
.fi
|
||
.PP
|
||
List all the files in your drive
|
||
.IP
|
||
.nf
|
||
\f[C]
|
||
rclone\ ls\ remote:
|
||
\f[]
|
||
.fi
|
||
.PP
|
||
To copy a local directory to a drive directory called backup
|
||
.IP
|
||
.nf
|
||
\f[C]
|
||
rclone\ copy\ /home/source\ remote:backup
|
||
\f[]
|
||
.fi
|
||
.SS Service Account support
|
||
.PP
|
||
You can set up rclone with Google Drive in an unattended mode, i.e.
|
||
not tied to a specific end\-user Google account.
|
||
This is useful when you want to synchronise files onto machines that
|
||
don\[aq]t have actively logged\-in users, for example build machines.
|
||
.PP
|
||
To create a service account and obtain its credentials, go to the Google
|
||
Developer Console (https://console.developers.google.com) and use the
|
||
"Create Credentials" button.
|
||
After creating an account, a JSON file containing the Service
|
||
Account\[aq]s credentials will be downloaded onto your machine.
|
||
These credentials are what rclone will use for authentication.
|
||
.PP
|
||
To use a Service Account instead of OAuth2 token flow, enter the path to
|
||
your Service Account credentials at the \f[C]service_account_file\f[]
|
||
prompt and rclone won\[aq]t use the browser based authentication flow.
|
||
.SS Team drives
|
||
.PP
|
||
If you want to configure the remote to point to a Google Team Drive then
|
||
answer \f[C]y\f[] to the question
|
||
\f[C]Configure\ this\ as\ a\ team\ drive?\f[].
|
||
.PP
|
||
This will fetch the list of Team Drives from google and allow you to
|
||
configure which one you want to use.
|
||
You can also type in a team drive ID if you prefer.
|
||
.PP
|
||
For example:
|
||
.IP
|
||
.nf
|
||
\f[C]
|
||
Configure\ this\ as\ a\ team\ drive?
|
||
y)\ Yes
|
||
n)\ No
|
||
y/n>\ y
|
||
Fetching\ team\ drive\ list...
|
||
Choose\ a\ number\ from\ below,\ or\ type\ in\ your\ own\ value
|
||
\ 1\ /\ Rclone\ Test
|
||
\ \ \ \\\ "xxxxxxxxxxxxxxxxxxxx"
|
||
\ 2\ /\ Rclone\ Test\ 2
|
||
\ \ \ \\\ "yyyyyyyyyyyyyyyyyyyy"
|
||
\ 3\ /\ Rclone\ Test\ 3
|
||
\ \ \ \\\ "zzzzzzzzzzzzzzzzzzzz"
|
||
Enter\ a\ Team\ Drive\ ID>\ 1
|
||
\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-
|
||
[remote]
|
||
client_id\ =
|
||
client_secret\ =
|
||
token\ =\ {"AccessToken":"xxxx.x.xxxxx_xxxxxxxxxxx_xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx","RefreshToken":"1/xxxxxxxxxxxxxxxx_xxxxxxxxxxxxxxxxxxxxxxxxxx","Expiry":"2014\-03\-16T13:57:58.955387075Z","Extra":null}
|
||
team_drive\ =\ xxxxxxxxxxxxxxxxxxxx
|
||
\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-
|
||
y)\ Yes\ this\ is\ OK
|
||
e)\ Edit\ this\ remote
|
||
d)\ Delete\ this\ remote
|
||
y/e/d>\ y
|
||
\f[]
|
||
.fi
|
||
.SS Modified time
|
||
.PP
|
||
Google drive stores modification times accurate to 1 ms.
|
||
.SS Revisions
|
||
.PP
|
||
Google drive stores revisions of files.
|
||
When you upload a change to an existing file to google drive using
|
||
rclone it will create a new revision of that file.
|
||
.PP
|
||
Revisions follow the standard google policy which at time of writing was
|
||
.IP \[bu] 2
|
||
They are deleted after 30 days or 100 revisions (whatever comes first).
|
||
.IP \[bu] 2
|
||
They do not count towards a user storage quota.
|
||
.SS Deleting files
|
||
.PP
|
||
By default rclone will send all files to the trash when deleting files.
|
||
If deleting them permanently is required then use the
|
||
\f[C]\-\-drive\-use\-trash=false\f[] flag, or set the equivalent
|
||
environment variable.
|
||
.SS Emptying trash
|
||
.PP
|
||
If you wish to empty your trash you can use the
|
||
\f[C]rclone\ cleanup\ remote:\f[] command which will permanently delete
|
||
all your trashed files.
|
||
This command does not take any path arguments.
|
||
.SS Specific options
|
||
.PP
|
||
Here are the command line options specific to this cloud storage system.
|
||
.SS \-\-drive\-auth\-owner\-only
|
||
.PP
|
||
Only consider files owned by the authenticated user.
|
||
.SS \-\-drive\-chunk\-size=SIZE
|
||
.PP
|
||
Upload chunk size.
|
||
Must a power of 2 >= 256k.
|
||
Default value is 8 MB.
|
||
.PP
|
||
Making this larger will improve performance, but note that each chunk is
|
||
buffered in memory one per transfer.
|
||
.PP
|
||
Reducing this will reduce memory usage but decrease performance.
|
||
.SS \-\-drive\-formats
|
||
.PP
|
||
Google documents can only be exported from Google drive.
|
||
When rclone downloads a Google doc it chooses a format to download
|
||
depending upon this setting.
|
||
.PP
|
||
By default the formats are \f[C]docx,xlsx,pptx,svg\f[] which are a
|
||
sensible default for an editable document.
|
||
.PP
|
||
When choosing a format, rclone runs down the list provided in order and
|
||
chooses the first file format the doc can be exported as from the list.
|
||
If the file can\[aq]t be exported to a format on the formats list, then
|
||
rclone will choose a format from the default list.
|
||
.PP
|
||
If you prefer an archive copy then you might use
|
||
\f[C]\-\-drive\-formats\ pdf\f[], or if you prefer
|
||
openoffice/libreoffice formats you might use
|
||
\f[C]\-\-drive\-formats\ ods,odt,odp\f[].
|
||
.PP
|
||
Note that rclone adds the extension to the google doc, so if it is
|
||
calles \f[C]My\ Spreadsheet\f[] on google docs, it will be exported as
|
||
\f[C]My\ Spreadsheet.xlsx\f[] or \f[C]My\ Spreadsheet.pdf\f[] etc.
|
||
.PP
|
||
Here are the possible extensions with their corresponding mime types.
|
||
.PP
|
||
.TS
|
||
tab(@);
|
||
lw(9.7n) lw(11.7n) lw(12.6n).
|
||
T{
|
||
Extension
|
||
T}@T{
|
||
Mime Type
|
||
T}@T{
|
||
Description
|
||
T}
|
||
_
|
||
T{
|
||
csv
|
||
T}@T{
|
||
text/csv
|
||
T}@T{
|
||
Standard CSV format for Spreadsheets
|
||
T}
|
||
T{
|
||
doc
|
||
T}@T{
|
||
application/msword
|
||
T}@T{
|
||
Micosoft Office Document
|
||
T}
|
||
T{
|
||
docx
|
||
T}@T{
|
||
application/vnd.openxmlformats\-officedocument.wordprocessingml.document
|
||
T}@T{
|
||
Microsoft Office Document
|
||
T}
|
||
T{
|
||
epub
|
||
T}@T{
|
||
application/epub+zip
|
||
T}@T{
|
||
E\-book format
|
||
T}
|
||
T{
|
||
html
|
||
T}@T{
|
||
text/html
|
||
T}@T{
|
||
An HTML Document
|
||
T}
|
||
T{
|
||
jpg
|
||
T}@T{
|
||
image/jpeg
|
||
T}@T{
|
||
A JPEG Image File
|
||
T}
|
||
T{
|
||
odp
|
||
T}@T{
|
||
application/vnd.oasis.opendocument.presentation
|
||
T}@T{
|
||
Openoffice Presentation
|
||
T}
|
||
T{
|
||
ods
|
||
T}@T{
|
||
application/vnd.oasis.opendocument.spreadsheet
|
||
T}@T{
|
||
Openoffice Spreadsheet
|
||
T}
|
||
T{
|
||
ods
|
||
T}@T{
|
||
application/x\-vnd.oasis.opendocument.spreadsheet
|
||
T}@T{
|
||
Openoffice Spreadsheet
|
||
T}
|
||
T{
|
||
odt
|
||
T}@T{
|
||
application/vnd.oasis.opendocument.text
|
||
T}@T{
|
||
Openoffice Document
|
||
T}
|
||
T{
|
||
pdf
|
||
T}@T{
|
||
application/pdf
|
||
T}@T{
|
||
Adobe PDF Format
|
||
T}
|
||
T{
|
||
png
|
||
T}@T{
|
||
image/png
|
||
T}@T{
|
||
PNG Image Format
|
||
T}
|
||
T{
|
||
pptx
|
||
T}@T{
|
||
application/vnd.openxmlformats\-officedocument.presentationml.presentation
|
||
T}@T{
|
||
Microsoft Office Powerpoint
|
||
T}
|
||
T{
|
||
rtf
|
||
T}@T{
|
||
application/rtf
|
||
T}@T{
|
||
Rich Text Format
|
||
T}
|
||
T{
|
||
svg
|
||
T}@T{
|
||
image/svg+xml
|
||
T}@T{
|
||
Scalable Vector Graphics Format
|
||
T}
|
||
T{
|
||
tsv
|
||
T}@T{
|
||
text/tab\-separated\-values
|
||
T}@T{
|
||
Standard TSV format for spreadsheets
|
||
T}
|
||
T{
|
||
txt
|
||
T}@T{
|
||
text/plain
|
||
T}@T{
|
||
Plain Text
|
||
T}
|
||
T{
|
||
xls
|
||
T}@T{
|
||
application/vnd.ms\-excel
|
||
T}@T{
|
||
Microsoft Office Spreadsheet
|
||
T}
|
||
T{
|
||
xlsx
|
||
T}@T{
|
||
application/vnd.openxmlformats\-officedocument.spreadsheetml.sheet
|
||
T}@T{
|
||
Microsoft Office Spreadsheet
|
||
T}
|
||
T{
|
||
zip
|
||
T}@T{
|
||
application/zip
|
||
T}@T{
|
||
A ZIP file of HTML, Images CSS
|
||
T}
|
||
.TE
|
||
.SS \-\-drive\-list\-chunk int
|
||
.PP
|
||
Size of listing chunk 100\-1000.
|
||
0 to disable.
|
||
(default 1000)
|
||
.SS \-\-drive\-shared\-with\-me
|
||
.PP
|
||
Only show files that are shared with me
|
||
.SS \-\-drive\-skip\-gdocs
|
||
.PP
|
||
Skip google documents in all listings.
|
||
If given, gdocs practically become invisible to rclone.
|
||
.SS \-\-drive\-trashed\-only
|
||
.PP
|
||
Only show files that are in the trash.
|
||
This will show trashed files in their original directory structure.
|
||
.SS \-\-drive\-upload\-cutoff=SIZE
|
||
.PP
|
||
File size cutoff for switching to chunked upload.
|
||
Default is 8 MB.
|
||
.SS \-\-drive\-use\-trash
|
||
.PP
|
||
Controls whether files are sent to the trash or deleted permanently.
|
||
Defaults to true, namely sending files to the trash.
|
||
Use \f[C]\-\-drive\-use\-trash=false\f[] to delete files permanently
|
||
instead.
|
||
.SS Limitations
|
||
.PP
|
||
Drive has quite a lot of rate limiting.
|
||
This causes rclone to be limited to transferring about 2 files per
|
||
second only.
|
||
Individual files may be transferred much faster at 100s of MBytes/s but
|
||
lots of small files can take a long time.
|
||
.PP
|
||
Server side copies are also subject to a separate rate limit.
|
||
If you see User rate limit exceeded errors, wait at least 24 hours and
|
||
retry.
|
||
You can disable server side copies with \f[C]\-\-disable\ copy\f[] to
|
||
download and upload the files if you prefer.
|
||
.SS Duplicated files
|
||
.PP
|
||
Sometimes, for no reason I\[aq]ve been able to track down, drive will
|
||
duplicate a file that rclone uploads.
|
||
Drive unlike all the other remotes can have duplicated files.
|
||
.PP
|
||
Duplicated files cause problems with the syncing and you will see
|
||
messages in the log about duplicates.
|
||
.PP
|
||
Use \f[C]rclone\ dedupe\f[] to fix duplicated files.
|
||
.PP
|
||
Note that this isn\[aq]t just a problem with rclone, even Google Photos
|
||
on Android duplicates files on drive sometimes.
|
||
.SS Rclone appears to be re\-copying files it shouldn\[aq]t
|
||
.PP
|
||
There are two possible reasons for rclone to recopy files which
|
||
haven\[aq]t changed to Google Drive.
|
||
.PP
|
||
The first is the duplicated file issue above \- run
|
||
\f[C]rclone\ dedupe\f[] and check your logs for duplicate object or
|
||
directory messages.
|
||
.PP
|
||
The second is that sometimes Google reports different sizes for the
|
||
Google Docs exports which will cause rclone to re\-download Google Docs
|
||
for no apparent reason.
|
||
\f[C]\-\-ignore\-size\f[] is a not very satisfactory work\-around for
|
||
this if it is causing you a lot of problems.
|
||
.SS Google docs downloads sometimes fail with "Failed to copy: read X
|
||
bytes expecting Y"
|
||
.PP
|
||
This is the same problem as above.
|
||
Google reports the google doc is one size, but rclone downloads a
|
||
different size.
|
||
Work\-around with the \f[C]\-\-ignore\-size\f[] flag or wait for rclone
|
||
to retry the download which it will.
|
||
.SS Making your own client_id
|
||
.PP
|
||
When you use rclone with Google drive in its default configuration you
|
||
are using rclone\[aq]s client_id.
|
||
This is shared between all the rclone users.
|
||
There is a global rate limit on the number of queries per second that
|
||
each client_id can do set by Google.
|
||
rclone already has a high quota and I will continue to make sure it is
|
||
high enough by contacting Google.
|
||
.PP
|
||
However you might find you get better performance making your own
|
||
client_id if you are a heavy user.
|
||
Or you may not depending on exactly how Google have been raising
|
||
rclone\[aq]s rate limit.
|
||
.PP
|
||
Here is how to create your own Google Drive client ID for rclone:
|
||
.IP "1." 3
|
||
Log into the Google API Console (https://console.developers.google.com/)
|
||
with your Google account.
|
||
It doesn\[aq]t matter what Google account you use.
|
||
(It need not be the same account as the Google Drive you want to access)
|
||
.IP "2." 3
|
||
Select a project or create a new project.
|
||
.IP "3." 3
|
||
Under "ENABLE APIS AND SERVICES" search for "Drive", and enable the then
|
||
"Google Drive API".
|
||
.IP "4." 3
|
||
Click "Credentials" in the left\-side panel (not "Create credentials",
|
||
which opens the wizard), then "Create credentials", then "OAuth client
|
||
ID".
|
||
It will prompt you to set the OAuth consent screen product name, if you
|
||
haven\[aq]t set one already.
|
||
.IP "5." 3
|
||
Choose an application type of "other", and click "Create".
|
||
(the default name is fine)
|
||
.IP "6." 3
|
||
It will show you a client ID and client secret.
|
||
Use these values in rclone config to add a new remote or edit an
|
||
existing remote.
|
||
.PP
|
||
(Thanks to \@balazer on github for these instructions.)
|
||
.SS HTTP
|
||
.PP
|
||
The HTTP remote is a read only remote for reading files of a webserver.
|
||
The webserver should provide file listings which rclone will read and
|
||
turn into a remote.
|
||
This has been tested with common webservers such as Apache/Nginx/Caddy
|
||
and will likely work with file listings from most web servers.
|
||
(If it doesn\[aq]t then please file an issue, or send a pull request!)
|
||
.PP
|
||
Paths are specified as \f[C]remote:\f[] or \f[C]remote:path/to/dir\f[].
|
||
.PP
|
||
Here is an example of how to make a remote called \f[C]remote\f[].
|
||
First run:
|
||
.IP
|
||
.nf
|
||
\f[C]
|
||
\ rclone\ config
|
||
\f[]
|
||
.fi
|
||
.PP
|
||
This will guide you through an interactive setup process:
|
||
.IP
|
||
.nf
|
||
\f[C]
|
||
No\ remotes\ found\ \-\ make\ a\ new\ one
|
||
n)\ New\ remote
|
||
s)\ Set\ configuration\ password
|
||
q)\ Quit\ config
|
||
n/s/q>\ n
|
||
name>\ remote
|
||
Type\ of\ storage\ to\ configure.
|
||
Choose\ a\ number\ from\ below,\ or\ type\ in\ your\ own\ value
|
||
\ 1\ /\ Amazon\ Drive
|
||
\ \ \ \\\ "amazon\ cloud\ drive"
|
||
\ 2\ /\ Amazon\ S3\ (also\ Dreamhost,\ Ceph,\ Minio)
|
||
\ \ \ \\\ "s3"
|
||
\ 3\ /\ Backblaze\ B2
|
||
\ \ \ \\\ "b2"
|
||
\ 4\ /\ Dropbox
|
||
\ \ \ \\\ "dropbox"
|
||
\ 5\ /\ Encrypt/Decrypt\ a\ remote
|
||
\ \ \ \\\ "crypt"
|
||
\ 6\ /\ FTP\ Connection
|
||
\ \ \ \\\ "ftp"
|
||
\ 7\ /\ Google\ Cloud\ Storage\ (this\ is\ not\ Google\ Drive)
|
||
\ \ \ \\\ "google\ cloud\ storage"
|
||
\ 8\ /\ Google\ Drive
|
||
\ \ \ \\\ "drive"
|
||
\ 9\ /\ Hubic
|
||
\ \ \ \\\ "hubic"
|
||
10\ /\ Local\ Disk
|
||
\ \ \ \\\ "local"
|
||
11\ /\ Microsoft\ OneDrive
|
||
\ \ \ \\\ "onedrive"
|
||
12\ /\ Openstack\ Swift\ (Rackspace\ Cloud\ Files,\ Memset\ Memstore,\ OVH)
|
||
\ \ \ \\\ "swift"
|
||
13\ /\ SSH/SFTP\ Connection
|
||
\ \ \ \\\ "sftp"
|
||
14\ /\ Yandex\ Disk
|
||
\ \ \ \\\ "yandex"
|
||
15\ /\ http\ Connection
|
||
\ \ \ \\\ "http"
|
||
Storage>\ http
|
||
URL\ of\ http\ host\ to\ connect\ to
|
||
Choose\ a\ number\ from\ below,\ or\ type\ in\ your\ own\ value
|
||
\ 1\ /\ Connect\ to\ example.com
|
||
\ \ \ \\\ "https://example.com"
|
||
url>\ https://beta.rclone.org
|
||
Remote\ config
|
||
\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-
|
||
[remote]
|
||
url\ =\ https://beta.rclone.org
|
||
\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-
|
||
y)\ Yes\ this\ is\ OK
|
||
e)\ Edit\ this\ remote
|
||
d)\ Delete\ this\ remote
|
||
y/e/d>\ y
|
||
Current\ remotes:
|
||
|
||
Name\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ Type
|
||
====\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ ====
|
||
remote\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ http
|
||
|
||
e)\ Edit\ existing\ remote
|
||
n)\ New\ remote
|
||
d)\ Delete\ remote
|
||
r)\ Rename\ remote
|
||
c)\ Copy\ remote
|
||
s)\ Set\ configuration\ password
|
||
q)\ Quit\ config
|
||
e/n/d/r/c/s/q>\ q
|
||
\f[]
|
||
.fi
|
||
.PP
|
||
This remote is called \f[C]remote\f[] and can now be used like this
|
||
.PP
|
||
See all the top level directories
|
||
.IP
|
||
.nf
|
||
\f[C]
|
||
rclone\ lsd\ remote:
|
||
\f[]
|
||
.fi
|
||
.PP
|
||
List the contents of a directory
|
||
.IP
|
||
.nf
|
||
\f[C]
|
||
rclone\ ls\ remote:directory
|
||
\f[]
|
||
.fi
|
||
.PP
|
||
Sync the remote \f[C]directory\f[] to \f[C]/home/local/directory\f[],
|
||
deleting any excess files.
|
||
.IP
|
||
.nf
|
||
\f[C]
|
||
rclone\ sync\ remote:directory\ /home/local/directory
|
||
\f[]
|
||
.fi
|
||
.SS Read only
|
||
.PP
|
||
This remote is read only \- you can\[aq]t upload files to an HTTP
|
||
server.
|
||
.SS Modified time
|
||
.PP
|
||
Most HTTP servers store time accurate to 1 second.
|
||
.SS Checksum
|
||
.PP
|
||
No checksums are stored.
|
||
.SS Usage without a config file
|
||
.PP
|
||
Note that since only two environment variable need to be set, it is easy
|
||
to use without a config file like this.
|
||
.IP
|
||
.nf
|
||
\f[C]
|
||
RCLONE_CONFIG_ZZ_TYPE=http\ RCLONE_CONFIG_ZZ_URL=https://beta.rclone.org\ rclone\ lsd\ zz:
|
||
\f[]
|
||
.fi
|
||
.PP
|
||
Or if you prefer
|
||
.IP
|
||
.nf
|
||
\f[C]
|
||
export\ RCLONE_CONFIG_ZZ_TYPE=http
|
||
export\ RCLONE_CONFIG_ZZ_URL=https://beta.rclone.org
|
||
rclone\ lsd\ zz:
|
||
\f[]
|
||
.fi
|
||
.SS Hubic
|
||
.PP
|
||
Paths are specified as \f[C]remote:path\f[]
|
||
.PP
|
||
Paths are specified as \f[C]remote:container\f[] (or \f[C]remote:\f[]
|
||
for the \f[C]lsd\f[] command.) You may put subdirectories in too, eg
|
||
\f[C]remote:container/path/to/dir\f[].
|
||
.PP
|
||
The initial setup for Hubic involves getting a token from Hubic which
|
||
you need to do in your browser.
|
||
\f[C]rclone\ config\f[] walks you through it.
|
||
.PP
|
||
Here is an example of how to make a remote called \f[C]remote\f[].
|
||
First run:
|
||
.IP
|
||
.nf
|
||
\f[C]
|
||
\ rclone\ config
|
||
\f[]
|
||
.fi
|
||
.PP
|
||
This will guide you through an interactive setup process:
|
||
.IP
|
||
.nf
|
||
\f[C]
|
||
n)\ New\ remote
|
||
s)\ Set\ configuration\ password
|
||
n/s>\ n
|
||
name>\ remote
|
||
Type\ of\ storage\ to\ configure.
|
||
Choose\ a\ number\ from\ below,\ or\ type\ in\ your\ own\ value
|
||
\ 1\ /\ Amazon\ Drive
|
||
\ \ \ \\\ "amazon\ cloud\ drive"
|
||
\ 2\ /\ Amazon\ S3\ (also\ Dreamhost,\ Ceph,\ Minio)
|
||
\ \ \ \\\ "s3"
|
||
\ 3\ /\ Backblaze\ B2
|
||
\ \ \ \\\ "b2"
|
||
\ 4\ /\ Dropbox
|
||
\ \ \ \\\ "dropbox"
|
||
\ 5\ /\ Encrypt/Decrypt\ a\ remote
|
||
\ \ \ \\\ "crypt"
|
||
\ 6\ /\ Google\ Cloud\ Storage\ (this\ is\ not\ Google\ Drive)
|
||
\ \ \ \\\ "google\ cloud\ storage"
|
||
\ 7\ /\ Google\ Drive
|
||
\ \ \ \\\ "drive"
|
||
\ 8\ /\ Hubic
|
||
\ \ \ \\\ "hubic"
|
||
\ 9\ /\ Local\ Disk
|
||
\ \ \ \\\ "local"
|
||
10\ /\ Microsoft\ OneDrive
|
||
\ \ \ \\\ "onedrive"
|
||
11\ /\ Openstack\ Swift\ (Rackspace\ Cloud\ Files,\ Memset\ Memstore,\ OVH)
|
||
\ \ \ \\\ "swift"
|
||
12\ /\ SSH/SFTP\ Connection
|
||
\ \ \ \\\ "sftp"
|
||
13\ /\ Yandex\ Disk
|
||
\ \ \ \\\ "yandex"
|
||
Storage>\ 8
|
||
Hubic\ Client\ Id\ \-\ leave\ blank\ normally.
|
||
client_id>
|
||
Hubic\ Client\ Secret\ \-\ leave\ blank\ normally.
|
||
client_secret>
|
||
Remote\ config
|
||
Use\ auto\ config?
|
||
\ *\ Say\ Y\ if\ not\ sure
|
||
\ *\ Say\ N\ if\ you\ are\ working\ on\ a\ remote\ or\ headless\ machine
|
||
y)\ Yes
|
||
n)\ No
|
||
y/n>\ y
|
||
If\ your\ browser\ doesn\[aq]t\ open\ automatically\ go\ to\ the\ following\ link:\ http://127.0.0.1:53682/auth
|
||
Log\ in\ and\ authorize\ rclone\ for\ access
|
||
Waiting\ for\ code...
|
||
Got\ code
|
||
\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-
|
||
[remote]
|
||
client_id\ =
|
||
client_secret\ =
|
||
token\ =\ {"access_token":"XXXXXX"}
|
||
\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-
|
||
y)\ Yes\ this\ is\ OK
|
||
e)\ Edit\ this\ remote
|
||
d)\ Delete\ this\ remote
|
||
y/e/d>\ y
|
||
\f[]
|
||
.fi
|
||
.PP
|
||
See the remote setup docs (https://rclone.org/remote_setup/) for how to
|
||
set it up on a machine with no Internet browser available.
|
||
.PP
|
||
Note that rclone runs a webserver on your local machine to collect the
|
||
token as returned from Hubic.
|
||
This only runs from the moment it opens your browser to the moment you
|
||
get back the verification code.
|
||
This is on \f[C]http://127.0.0.1:53682/\f[] and this it may require you
|
||
to unblock it temporarily if you are running a host firewall.
|
||
.PP
|
||
Once configured you can then use \f[C]rclone\f[] like this,
|
||
.PP
|
||
List containers in the top level of your Hubic
|
||
.IP
|
||
.nf
|
||
\f[C]
|
||
rclone\ lsd\ remote:
|
||
\f[]
|
||
.fi
|
||
.PP
|
||
List all the files in your Hubic
|
||
.IP
|
||
.nf
|
||
\f[C]
|
||
rclone\ ls\ remote:
|
||
\f[]
|
||
.fi
|
||
.PP
|
||
To copy a local directory to an Hubic directory called backup
|
||
.IP
|
||
.nf
|
||
\f[C]
|
||
rclone\ copy\ /home/source\ remote:backup
|
||
\f[]
|
||
.fi
|
||
.PP
|
||
If you want the directory to be visible in the official \f[I]Hubic
|
||
browser\f[], you need to copy your files to the \f[C]default\f[]
|
||
directory
|
||
.IP
|
||
.nf
|
||
\f[C]
|
||
rclone\ copy\ /home/source\ remote:default/backup
|
||
\f[]
|
||
.fi
|
||
.SS \-\-fast\-list
|
||
.PP
|
||
This remote supports \f[C]\-\-fast\-list\f[] which allows you to use
|
||
fewer transactions in exchange for more memory.
|
||
See the rclone docs (/docs/#fast-list) for more details.
|
||
.SS Modified time
|
||
.PP
|
||
The modified time is stored as metadata on the object as
|
||
\f[C]X\-Object\-Meta\-Mtime\f[] as floating point since the epoch
|
||
accurate to 1 ns.
|
||
.PP
|
||
This is a defacto standard (used in the official python\-swiftclient
|
||
amongst others) for storing the modification time for an object.
|
||
.PP
|
||
Note that Hubic wraps the Swift backend, so most of the properties of
|
||
are the same.
|
||
.SS Limitations
|
||
.PP
|
||
This uses the normal OpenStack Swift mechanism to refresh the Swift API
|
||
credentials and ignores the expires field returned by the Hubic API.
|
||
.PP
|
||
The Swift API doesn\[aq]t return a correct MD5SUM for segmented files
|
||
(Dynamic or Static Large Objects) so rclone won\[aq]t check or use the
|
||
MD5SUM for these.
|
||
.SS Microsoft Azure Blob Storage
|
||
.PP
|
||
Paths are specified as \f[C]remote:container\f[] (or \f[C]remote:\f[]
|
||
for the \f[C]lsd\f[] command.) You may put subdirectories in too, eg
|
||
\f[C]remote:container/path/to/dir\f[].
|
||
.PP
|
||
Here is an example of making a Microsoft Azure Blob Storage
|
||
configuration.
|
||
For a remote called \f[C]remote\f[].
|
||
First run:
|
||
.IP
|
||
.nf
|
||
\f[C]
|
||
\ rclone\ config
|
||
\f[]
|
||
.fi
|
||
.PP
|
||
This will guide you through an interactive setup process:
|
||
.IP
|
||
.nf
|
||
\f[C]
|
||
No\ remotes\ found\ \-\ make\ a\ new\ one
|
||
n)\ New\ remote
|
||
s)\ Set\ configuration\ password
|
||
q)\ Quit\ config
|
||
n/s/q>\ n
|
||
name>\ remote
|
||
Type\ of\ storage\ to\ configure.
|
||
Choose\ a\ number\ from\ below,\ or\ type\ in\ your\ own\ value
|
||
\ 1\ /\ Amazon\ Drive
|
||
\ \ \ \\\ "amazon\ cloud\ drive"
|
||
\ 2\ /\ Amazon\ S3\ (also\ Dreamhost,\ Ceph,\ Minio)
|
||
\ \ \ \\\ "s3"
|
||
\ 3\ /\ Backblaze\ B2
|
||
\ \ \ \\\ "b2"
|
||
\ 4\ /\ Box
|
||
\ \ \ \\\ "box"
|
||
\ 5\ /\ Dropbox
|
||
\ \ \ \\\ "dropbox"
|
||
\ 6\ /\ Encrypt/Decrypt\ a\ remote
|
||
\ \ \ \\\ "crypt"
|
||
\ 7\ /\ FTP\ Connection
|
||
\ \ \ \\\ "ftp"
|
||
\ 8\ /\ Google\ Cloud\ Storage\ (this\ is\ not\ Google\ Drive)
|
||
\ \ \ \\\ "google\ cloud\ storage"
|
||
\ 9\ /\ Google\ Drive
|
||
\ \ \ \\\ "drive"
|
||
10\ /\ Hubic
|
||
\ \ \ \\\ "hubic"
|
||
11\ /\ Local\ Disk
|
||
\ \ \ \\\ "local"
|
||
12\ /\ Microsoft\ Azure\ Blob\ Storage
|
||
\ \ \ \\\ "azureblob"
|
||
13\ /\ Microsoft\ OneDrive
|
||
\ \ \ \\\ "onedrive"
|
||
14\ /\ Openstack\ Swift\ (Rackspace\ Cloud\ Files,\ Memset\ Memstore,\ OVH)
|
||
\ \ \ \\\ "swift"
|
||
15\ /\ SSH/SFTP\ Connection
|
||
\ \ \ \\\ "sftp"
|
||
16\ /\ Yandex\ Disk
|
||
\ \ \ \\\ "yandex"
|
||
17\ /\ http\ Connection
|
||
\ \ \ \\\ "http"
|
||
Storage>\ azureblob
|
||
Storage\ Account\ Name
|
||
account>\ account_name
|
||
Storage\ Account\ Key
|
||
key>\ base64encodedkey==
|
||
Endpoint\ for\ the\ service\ \-\ leave\ blank\ normally.
|
||
endpoint>\
|
||
Remote\ config
|
||
\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-
|
||
[remote]
|
||
account\ =\ account_name
|
||
key\ =\ base64encodedkey==
|
||
endpoint\ =\
|
||
\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-
|
||
y)\ Yes\ this\ is\ OK
|
||
e)\ Edit\ this\ remote
|
||
d)\ Delete\ this\ remote
|
||
y/e/d>\ y
|
||
\f[]
|
||
.fi
|
||
.PP
|
||
See all containers
|
||
.IP
|
||
.nf
|
||
\f[C]
|
||
rclone\ lsd\ remote:
|
||
\f[]
|
||
.fi
|
||
.PP
|
||
Make a new container
|
||
.IP
|
||
.nf
|
||
\f[C]
|
||
rclone\ mkdir\ remote:container
|
||
\f[]
|
||
.fi
|
||
.PP
|
||
List the contents of a container
|
||
.IP
|
||
.nf
|
||
\f[C]
|
||
rclone\ ls\ remote:container
|
||
\f[]
|
||
.fi
|
||
.PP
|
||
Sync \f[C]/home/local/directory\f[] to the remote container, deleting
|
||
any excess files in the container.
|
||
.IP
|
||
.nf
|
||
\f[C]
|
||
rclone\ sync\ /home/local/directory\ remote:container
|
||
\f[]
|
||
.fi
|
||
.SS \-\-fast\-list
|
||
.PP
|
||
This remote supports \f[C]\-\-fast\-list\f[] which allows you to use
|
||
fewer transactions in exchange for more memory.
|
||
See the rclone docs (/docs/#fast-list) for more details.
|
||
.SS Modified time
|
||
.PP
|
||
The modified time is stored as metadata on the object with the
|
||
\f[C]mtime\f[] key.
|
||
It is stored using RFC3339 Format time with nanosecond precision.
|
||
The metadata is supplied during directory listings so there is no
|
||
overhead to using it.
|
||
.SS Hashes
|
||
.PP
|
||
MD5 hashes are stored with blobs.
|
||
However blobs that were uploaded in chunks only have an MD5 if the
|
||
source remote was capable of MD5 hashes, eg the local disk.
|
||
.SS Multipart uploads
|
||
.PP
|
||
Rclone supports multipart uploads with Azure Blob storage.
|
||
Files bigger than 256MB will be uploaded using chunked upload by
|
||
default.
|
||
.PP
|
||
The files will be uploaded in parallel in 4MB chunks (by default).
|
||
Note that these chunks are buffered in memory and there may be up to
|
||
\f[C]\-\-transfers\f[] of them being uploaded at once.
|
||
.PP
|
||
Files can\[aq]t be split into more than 50,000 chunks so by default, so
|
||
the largest file that can be uploaded with 4MB chunk size is 195GB.
|
||
Above this rclone will double the chunk size until it creates less than
|
||
50,000 chunks.
|
||
By default this will mean a maximum file size of 3.2TB can be uploaded.
|
||
This can be raised to 5TB using
|
||
\f[C]\-\-azureblob\-chunk\-size\ 100M\f[].
|
||
.PP
|
||
Note that rclone doesn\[aq]t commit the block list until the end of the
|
||
upload which means that there is a limit of 9.5TB of multipart uploads
|
||
in progress as Azure won\[aq]t allow more than that amount of
|
||
uncommitted blocks.
|
||
.SS Specific options
|
||
.PP
|
||
Here are the command line options specific to this cloud storage system.
|
||
.SS \-\-azureblob\-upload\-cutoff=SIZE
|
||
.PP
|
||
Cutoff for switching to chunked upload \- must be <= 256MB.
|
||
The default is 256MB.
|
||
.SS \-\-azureblob\-chunk\-size=SIZE
|
||
.PP
|
||
Upload chunk size.
|
||
Default 4MB.
|
||
Note that this is stored in memory and there may be up to
|
||
\f[C]\-\-transfers\f[] chunks stored at once in memory.
|
||
This can be at most 100MB.
|
||
.SS Limitations
|
||
.PP
|
||
MD5 sums are only uploaded with chunked files if the source has an MD5
|
||
sum.
|
||
This will always be the case for a local to azure copy.
|
||
.SS Microsoft OneDrive
|
||
.PP
|
||
Paths are specified as \f[C]remote:path\f[]
|
||
.PP
|
||
Paths may be as deep as required, eg
|
||
\f[C]remote:directory/subdirectory\f[].
|
||
.PP
|
||
The initial setup for OneDrive involves getting a token from Microsoft
|
||
which you need to do in your browser.
|
||
\f[C]rclone\ config\f[] walks you through it.
|
||
.PP
|
||
Here is an example of how to make a remote called \f[C]remote\f[].
|
||
First run:
|
||
.IP
|
||
.nf
|
||
\f[C]
|
||
\ rclone\ config
|
||
\f[]
|
||
.fi
|
||
.PP
|
||
This will guide you through an interactive setup process:
|
||
.IP
|
||
.nf
|
||
\f[C]
|
||
No\ remotes\ found\ \-\ make\ a\ new\ one
|
||
n)\ New\ remote
|
||
s)\ Set\ configuration\ password
|
||
n/s>\ n
|
||
name>\ remote
|
||
Type\ of\ storage\ to\ configure.
|
||
Choose\ a\ number\ from\ below,\ or\ type\ in\ your\ own\ value
|
||
\ 1\ /\ Amazon\ Drive
|
||
\ \ \ \\\ "amazon\ cloud\ drive"
|
||
\ 2\ /\ Amazon\ S3\ (also\ Dreamhost,\ Ceph,\ Minio)
|
||
\ \ \ \\\ "s3"
|
||
\ 3\ /\ Backblaze\ B2
|
||
\ \ \ \\\ "b2"
|
||
\ 4\ /\ Dropbox
|
||
\ \ \ \\\ "dropbox"
|
||
\ 5\ /\ Encrypt/Decrypt\ a\ remote
|
||
\ \ \ \\\ "crypt"
|
||
\ 6\ /\ Google\ Cloud\ Storage\ (this\ is\ not\ Google\ Drive)
|
||
\ \ \ \\\ "google\ cloud\ storage"
|
||
\ 7\ /\ Google\ Drive
|
||
\ \ \ \\\ "drive"
|
||
\ 8\ /\ Hubic
|
||
\ \ \ \\\ "hubic"
|
||
\ 9\ /\ Local\ Disk
|
||
\ \ \ \\\ "local"
|
||
10\ /\ Microsoft\ OneDrive
|
||
\ \ \ \\\ "onedrive"
|
||
11\ /\ Openstack\ Swift\ (Rackspace\ Cloud\ Files,\ Memset\ Memstore,\ OVH)
|
||
\ \ \ \\\ "swift"
|
||
12\ /\ SSH/SFTP\ Connection
|
||
\ \ \ \\\ "sftp"
|
||
13\ /\ Yandex\ Disk
|
||
\ \ \ \\\ "yandex"
|
||
Storage>\ 10
|
||
Microsoft\ App\ Client\ Id\ \-\ leave\ blank\ normally.
|
||
client_id>
|
||
Microsoft\ App\ Client\ Secret\ \-\ leave\ blank\ normally.
|
||
client_secret>
|
||
Remote\ config
|
||
Choose\ OneDrive\ account\ type?
|
||
\ *\ Say\ b\ for\ a\ OneDrive\ business\ account
|
||
\ *\ Say\ p\ for\ a\ personal\ OneDrive\ account
|
||
b)\ Business
|
||
p)\ Personal
|
||
b/p>\ p
|
||
Use\ auto\ config?
|
||
\ *\ Say\ Y\ if\ not\ sure
|
||
\ *\ Say\ N\ if\ you\ are\ working\ on\ a\ remote\ or\ headless\ machine
|
||
y)\ Yes
|
||
n)\ No
|
||
y/n>\ y
|
||
If\ your\ browser\ doesn\[aq]t\ open\ automatically\ go\ to\ the\ following\ link:\ http://127.0.0.1:53682/auth
|
||
Log\ in\ and\ authorize\ rclone\ for\ access
|
||
Waiting\ for\ code...
|
||
Got\ code
|
||
\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-
|
||
[remote]
|
||
client_id\ =
|
||
client_secret\ =
|
||
token\ =\ {"access_token":"XXXXXX"}
|
||
\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-
|
||
y)\ Yes\ this\ is\ OK
|
||
e)\ Edit\ this\ remote
|
||
d)\ Delete\ this\ remote
|
||
y/e/d>\ y
|
||
\f[]
|
||
.fi
|
||
.PP
|
||
See the remote setup docs (https://rclone.org/remote_setup/) for how to
|
||
set it up on a machine with no Internet browser available.
|
||
.PP
|
||
Note that rclone runs a webserver on your local machine to collect the
|
||
token as returned from Microsoft.
|
||
This only runs from the moment it opens your browser to the moment you
|
||
get back the verification code.
|
||
This is on \f[C]http://127.0.0.1:53682/\f[] and this it may require you
|
||
to unblock it temporarily if you are running a host firewall.
|
||
.PP
|
||
Once configured you can then use \f[C]rclone\f[] like this,
|
||
.PP
|
||
List directories in top level of your OneDrive
|
||
.IP
|
||
.nf
|
||
\f[C]
|
||
rclone\ lsd\ remote:
|
||
\f[]
|
||
.fi
|
||
.PP
|
||
List all the files in your OneDrive
|
||
.IP
|
||
.nf
|
||
\f[C]
|
||
rclone\ ls\ remote:
|
||
\f[]
|
||
.fi
|
||
.PP
|
||
To copy a local directory to an OneDrive directory called backup
|
||
.IP
|
||
.nf
|
||
\f[C]
|
||
rclone\ copy\ /home/source\ remote:backup
|
||
\f[]
|
||
.fi
|
||
.SS OneDrive for Business
|
||
.PP
|
||
There is additional support for OneDrive for Business.
|
||
Select "b" when ask
|
||
.IP
|
||
.nf
|
||
\f[C]
|
||
Choose\ OneDrive\ account\ type?
|
||
\ *\ Say\ b\ for\ a\ OneDrive\ business\ account
|
||
\ *\ Say\ p\ for\ a\ personal\ OneDrive\ account
|
||
b)\ Business
|
||
p)\ Personal
|
||
b/p>\
|
||
\f[]
|
||
.fi
|
||
.PP
|
||
After that rclone requires an authentication of your account.
|
||
The application will first authenticate your account, then query the
|
||
OneDrive resource URL and do a second (silent) authentication for this
|
||
resource URL.
|
||
.SS Modified time and hashes
|
||
.PP
|
||
OneDrive allows modification times to be set on objects accurate to 1
|
||
second.
|
||
These will be used to detect whether objects need syncing or not.
|
||
.PP
|
||
One drive supports SHA1 type hashes, so you can use
|
||
\f[C]\-\-checksum\f[] flag.
|
||
.SS Deleting files
|
||
.PP
|
||
Any files you delete with rclone will end up in the trash.
|
||
Microsoft doesn\[aq]t provide an API to permanently delete files, nor to
|
||
empty the trash, so you will have to do that with one of Microsoft\[aq]s
|
||
apps or via the OneDrive website.
|
||
.SS Specific options
|
||
.PP
|
||
Here are the command line options specific to this cloud storage system.
|
||
.SS \-\-onedrive\-chunk\-size=SIZE
|
||
.PP
|
||
Above this size files will be chunked \- must be multiple of 320k.
|
||
The default is 10MB.
|
||
Note that the chunks will be buffered into memory.
|
||
.SS \-\-onedrive\-upload\-cutoff=SIZE
|
||
.PP
|
||
Cutoff for switching to chunked upload \- must be <= 100MB.
|
||
The default is 10MB.
|
||
.SS Limitations
|
||
.PP
|
||
Note that OneDrive is case insensitive so you can\[aq]t have a file
|
||
called "Hello.doc" and one called "hello.doc".
|
||
.PP
|
||
There are quite a few characters that can\[aq]t be in OneDrive file
|
||
names.
|
||
These can\[aq]t occur on Windows platforms, but on non\-Windows
|
||
platforms they are common.
|
||
Rclone will map these names to and from an identical looking unicode
|
||
equivalent.
|
||
For example if a file has a \f[C]?\f[] in it will be mapped to
|
||
\f[C]?\f[] instead.
|
||
.PP
|
||
The largest allowed file size is 10GiB (10,737,418,240 bytes).
|
||
.SS QingStor
|
||
.PP
|
||
Paths are specified as \f[C]remote:bucket\f[] (or \f[C]remote:\f[] for
|
||
the \f[C]lsd\f[] command.) You may put subdirectories in too, eg
|
||
\f[C]remote:bucket/path/to/dir\f[].
|
||
.PP
|
||
Here is an example of making an QingStor configuration.
|
||
First run
|
||
.IP
|
||
.nf
|
||
\f[C]
|
||
rclone\ config
|
||
\f[]
|
||
.fi
|
||
.PP
|
||
This will guide you through an interactive setup process.
|
||
.IP
|
||
.nf
|
||
\f[C]
|
||
No\ remotes\ found\ \-\ make\ a\ new\ one
|
||
n)\ New\ remote
|
||
r)\ Rename\ remote
|
||
c)\ Copy\ remote
|
||
s)\ Set\ configuration\ password
|
||
q)\ Quit\ config
|
||
n/r/c/s/q>\ n
|
||
name>\ remote
|
||
Type\ of\ storage\ to\ configure.
|
||
Choose\ a\ number\ from\ below,\ or\ type\ in\ your\ own\ value
|
||
\ 1\ /\ Amazon\ Drive
|
||
\ \ \ \\\ "amazon\ cloud\ drive"
|
||
\ 2\ /\ Amazon\ S3\ (also\ Dreamhost,\ Ceph,\ Minio)
|
||
\ \ \ \\\ "s3"
|
||
\ 3\ /\ Backblaze\ B2
|
||
\ \ \ \\\ "b2"
|
||
\ 4\ /\ Dropbox
|
||
\ \ \ \\\ "dropbox"
|
||
\ 5\ /\ Encrypt/Decrypt\ a\ remote
|
||
\ \ \ \\\ "crypt"
|
||
\ 6\ /\ FTP\ Connection
|
||
\ \ \ \\\ "ftp"
|
||
\ 7\ /\ Google\ Cloud\ Storage\ (this\ is\ not\ Google\ Drive)
|
||
\ \ \ \\\ "google\ cloud\ storage"
|
||
\ 8\ /\ Google\ Drive
|
||
\ \ \ \\\ "drive"
|
||
\ 9\ /\ Hubic
|
||
\ \ \ \\\ "hubic"
|
||
10\ /\ Local\ Disk
|
||
\ \ \ \\\ "local"
|
||
11\ /\ Microsoft\ OneDrive
|
||
\ \ \ \\\ "onedrive"
|
||
12\ /\ Openstack\ Swift\ (Rackspace\ Cloud\ Files,\ Memset\ Memstore,\ OVH)
|
||
\ \ \ \\\ "swift"
|
||
13\ /\ QingStor\ Object\ Storage
|
||
\ \ \ \\\ "qingstor"
|
||
14\ /\ SSH/SFTP\ Connection
|
||
\ \ \ \\\ "sftp"
|
||
15\ /\ Yandex\ Disk
|
||
\ \ \ \\\ "yandex"
|
||
Storage>\ 13
|
||
Get\ QingStor\ credentials\ from\ runtime.\ Only\ applies\ if\ access_key_id\ and\ secret_access_key\ is\ blank.
|
||
Choose\ a\ number\ from\ below,\ or\ type\ in\ your\ own\ value
|
||
\ 1\ /\ Enter\ QingStor\ credentials\ in\ the\ next\ step
|
||
\ \ \ \\\ "false"
|
||
\ 2\ /\ Get\ QingStor\ credentials\ from\ the\ environment\ (env\ vars\ or\ IAM)
|
||
\ \ \ \\\ "true"
|
||
env_auth>\ 1
|
||
QingStor\ Access\ Key\ ID\ \-\ leave\ blank\ for\ anonymous\ access\ or\ runtime\ credentials.
|
||
access_key_id>\ access_key
|
||
QingStor\ Secret\ Access\ Key\ (password)\ \-\ leave\ blank\ for\ anonymous\ access\ or\ runtime\ credentials.
|
||
secret_access_key>\ secret_key
|
||
Enter\ a\ endpoint\ URL\ to\ connection\ QingStor\ API.
|
||
Leave\ blank\ will\ use\ the\ default\ value\ "https://qingstor.com:443"
|
||
endpoint>
|
||
Zone\ connect\ to.\ Default\ is\ "pek3a".
|
||
Choose\ a\ number\ from\ below,\ or\ type\ in\ your\ own\ value
|
||
\ \ \ /\ The\ Beijing\ (China)\ Three\ Zone
|
||
\ 1\ |\ Needs\ location\ constraint\ pek3a.
|
||
\ \ \ \\\ "pek3a"
|
||
\ \ \ /\ The\ Shanghai\ (China)\ First\ Zone
|
||
\ 2\ |\ Needs\ location\ constraint\ sh1a.
|
||
\ \ \ \\\ "sh1a"
|
||
zone>\ 1
|
||
Number\ of\ connnection\ retry.
|
||
Leave\ blank\ will\ use\ the\ default\ value\ "3".
|
||
connection_retries>
|
||
Remote\ config
|
||
\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-
|
||
[remote]
|
||
env_auth\ =\ false
|
||
access_key_id\ =\ access_key
|
||
secret_access_key\ =\ secret_key
|
||
endpoint\ =
|
||
zone\ =\ pek3a
|
||
connection_retries\ =
|
||
\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-
|
||
y)\ Yes\ this\ is\ OK
|
||
e)\ Edit\ this\ remote
|
||
d)\ Delete\ this\ remote
|
||
y/e/d>\ y
|
||
\f[]
|
||
.fi
|
||
.PP
|
||
This remote is called \f[C]remote\f[] and can now be used like this
|
||
.PP
|
||
See all buckets
|
||
.IP
|
||
.nf
|
||
\f[C]
|
||
rclone\ lsd\ remote:
|
||
\f[]
|
||
.fi
|
||
.PP
|
||
Make a new bucket
|
||
.IP
|
||
.nf
|
||
\f[C]
|
||
rclone\ mkdir\ remote:bucket
|
||
\f[]
|
||
.fi
|
||
.PP
|
||
List the contents of a bucket
|
||
.IP
|
||
.nf
|
||
\f[C]
|
||
rclone\ ls\ remote:bucket
|
||
\f[]
|
||
.fi
|
||
.PP
|
||
Sync \f[C]/home/local/directory\f[] to the remote bucket, deleting any
|
||
excess files in the bucket.
|
||
.IP
|
||
.nf
|
||
\f[C]
|
||
rclone\ sync\ /home/local/directory\ remote:bucket
|
||
\f[]
|
||
.fi
|
||
.SS \-\-fast\-list
|
||
.PP
|
||
This remote supports \f[C]\-\-fast\-list\f[] which allows you to use
|
||
fewer transactions in exchange for more memory.
|
||
See the rclone docs (/docs/#fast-list) for more details.
|
||
.SS Multipart uploads
|
||
.PP
|
||
rclone supports multipart uploads with QingStor which means that it can
|
||
upload files bigger than 5GB.
|
||
Note that files uploaded with multipart upload don\[aq]t have an MD5SUM.
|
||
.SS Buckets and Zone
|
||
.PP
|
||
With QingStor you can list buckets (\f[C]rclone\ lsd\f[]) using any
|
||
zone, but you can only access the content of a bucket from the zone it
|
||
was created in.
|
||
If you attempt to access a bucket from the wrong zone, you will get an
|
||
error,
|
||
\f[C]incorrect\ zone,\ the\ bucket\ is\ not\ in\ \[aq]XXX\[aq]\ zone\f[].
|
||
.SS Authentication
|
||
.PP
|
||
There are two ways to supply \f[C]rclone\f[] with a set of QingStor
|
||
credentials.
|
||
In order of precedence:
|
||
.IP \[bu] 2
|
||
Directly in the rclone configuration file (as configured by
|
||
\f[C]rclone\ config\f[])
|
||
.IP \[bu] 2
|
||
set \f[C]access_key_id\f[] and \f[C]secret_access_key\f[]
|
||
.IP \[bu] 2
|
||
Runtime configuration:
|
||
.IP \[bu] 2
|
||
set \f[C]env_auth\f[] to \f[C]true\f[] in the config file
|
||
.IP \[bu] 2
|
||
Exporting the following environment variables before running
|
||
\f[C]rclone\f[]
|
||
.RS 2
|
||
.IP \[bu] 2
|
||
Access Key ID: \f[C]QS_ACCESS_KEY_ID\f[] or \f[C]QS_ACCESS_KEY\f[]
|
||
.IP \[bu] 2
|
||
Secret Access Key: \f[C]QS_SECRET_ACCESS_KEY\f[] or
|
||
\f[C]QS_SECRET_KEY\f[]
|
||
.RE
|
||
.SS Swift
|
||
.PP
|
||
Swift refers to Openstack Object
|
||
Storage (https://docs.openstack.org/swift/latest/).
|
||
Commercial implementations of that being:
|
||
.IP \[bu] 2
|
||
Rackspace Cloud Files (https://www.rackspace.com/cloud/files/)
|
||
.IP \[bu] 2
|
||
Memset Memstore (https://www.memset.com/cloud/storage/)
|
||
.IP \[bu] 2
|
||
OVH Object
|
||
Storage (https://www.ovh.co.uk/public-cloud/storage/object-storage/)
|
||
.IP \[bu] 2
|
||
Oracle Cloud Storage (https://cloud.oracle.com/storage-opc)
|
||
.IP \[bu] 2
|
||
IBM Bluemix Cloud ObjectStorage
|
||
Swift (https://console.bluemix.net/docs/infrastructure/objectstorage-swift/index.html)
|
||
.PP
|
||
Paths are specified as \f[C]remote:container\f[] (or \f[C]remote:\f[]
|
||
for the \f[C]lsd\f[] command.) You may put subdirectories in too, eg
|
||
\f[C]remote:container/path/to/dir\f[].
|
||
.PP
|
||
Here is an example of making a swift configuration.
|
||
First run
|
||
.IP
|
||
.nf
|
||
\f[C]
|
||
rclone\ config
|
||
\f[]
|
||
.fi
|
||
.PP
|
||
This will guide you through an interactive setup process.
|
||
.IP
|
||
.nf
|
||
\f[C]
|
||
No\ remotes\ found\ \-\ make\ a\ new\ one
|
||
n)\ New\ remote
|
||
s)\ Set\ configuration\ password
|
||
q)\ Quit\ config
|
||
n/s/q>\ n
|
||
name>\ remote
|
||
Type\ of\ storage\ to\ configure.
|
||
Choose\ a\ number\ from\ below,\ or\ type\ in\ your\ own\ value
|
||
\ 1\ /\ Amazon\ Drive
|
||
\ \ \ \\\ "amazon\ cloud\ drive"
|
||
\ 2\ /\ Amazon\ S3\ (also\ Dreamhost,\ Ceph,\ Minio)
|
||
\ \ \ \\\ "s3"
|
||
\ 3\ /\ Backblaze\ B2
|
||
\ \ \ \\\ "b2"
|
||
\ 4\ /\ Box
|
||
\ \ \ \\\ "box"
|
||
\ 5\ /\ Cache\ a\ remote
|
||
\ \ \ \\\ "cache"
|
||
\ 6\ /\ Dropbox
|
||
\ \ \ \\\ "dropbox"
|
||
\ 7\ /\ Encrypt/Decrypt\ a\ remote
|
||
\ \ \ \\\ "crypt"
|
||
\ 8\ /\ FTP\ Connection
|
||
\ \ \ \\\ "ftp"
|
||
\ 9\ /\ Google\ Cloud\ Storage\ (this\ is\ not\ Google\ Drive)
|
||
\ \ \ \\\ "google\ cloud\ storage"
|
||
10\ /\ Google\ Drive
|
||
\ \ \ \\\ "drive"
|
||
11\ /\ Hubic
|
||
\ \ \ \\\ "hubic"
|
||
12\ /\ Local\ Disk
|
||
\ \ \ \\\ "local"
|
||
13\ /\ Microsoft\ Azure\ Blob\ Storage
|
||
\ \ \ \\\ "azureblob"
|
||
14\ /\ Microsoft\ OneDrive
|
||
\ \ \ \\\ "onedrive"
|
||
15\ /\ Openstack\ Swift\ (Rackspace\ Cloud\ Files,\ Memset\ Memstore,\ OVH)
|
||
\ \ \ \\\ "swift"
|
||
16\ /\ Pcloud
|
||
\ \ \ \\\ "pcloud"
|
||
17\ /\ QingCloud\ Object\ Storage
|
||
\ \ \ \\\ "qingstor"
|
||
18\ /\ SSH/SFTP\ Connection
|
||
\ \ \ \\\ "sftp"
|
||
19\ /\ Webdav
|
||
\ \ \ \\\ "webdav"
|
||
20\ /\ Yandex\ Disk
|
||
\ \ \ \\\ "yandex"
|
||
21\ /\ http\ Connection
|
||
\ \ \ \\\ "http"
|
||
Storage>\ swift
|
||
Get\ swift\ credentials\ from\ environment\ variables\ in\ standard\ OpenStack\ form.
|
||
Choose\ a\ number\ from\ below,\ or\ type\ in\ your\ own\ value
|
||
\ 1\ /\ Enter\ swift\ credentials\ in\ the\ next\ step
|
||
\ \ \ \\\ "false"
|
||
\ 2\ /\ Get\ swift\ credentials\ from\ environment\ vars.\ Leave\ other\ fields\ blank\ if\ using\ this.
|
||
\ \ \ \\\ "true"
|
||
env_auth>\ true
|
||
User\ name\ to\ log\ in\ (OS_USERNAME).
|
||
user>\
|
||
API\ key\ or\ password\ (OS_PASSWORD).
|
||
key>\
|
||
Authentication\ URL\ for\ server\ (OS_AUTH_URL).
|
||
Choose\ a\ number\ from\ below,\ or\ type\ in\ your\ own\ value
|
||
\ 1\ /\ Rackspace\ US
|
||
\ \ \ \\\ "https://auth.api.rackspacecloud.com/v1.0"
|
||
\ 2\ /\ Rackspace\ UK
|
||
\ \ \ \\\ "https://lon.auth.api.rackspacecloud.com/v1.0"
|
||
\ 3\ /\ Rackspace\ v2
|
||
\ \ \ \\\ "https://identity.api.rackspacecloud.com/v2.0"
|
||
\ 4\ /\ Memset\ Memstore\ UK
|
||
\ \ \ \\\ "https://auth.storage.memset.com/v1.0"
|
||
\ 5\ /\ Memset\ Memstore\ UK\ v2
|
||
\ \ \ \\\ "https://auth.storage.memset.com/v2.0"
|
||
\ 6\ /\ OVH
|
||
\ \ \ \\\ "https://auth.cloud.ovh.net/v2.0"
|
||
auth>\
|
||
User\ ID\ to\ log\ in\ \-\ optional\ \-\ most\ swift\ systems\ use\ user\ and\ leave\ this\ blank\ (v3\ auth)\ (OS_USER_ID).
|
||
user_id>\
|
||
User\ domain\ \-\ optional\ (v3\ auth)\ (OS_USER_DOMAIN_NAME)
|
||
domain>\
|
||
Tenant\ name\ \-\ optional\ for\ v1\ auth,\ this\ or\ tenant_id\ required\ otherwise\ (OS_TENANT_NAME\ or\ OS_PROJECT_NAME)
|
||
tenant>\
|
||
Tenant\ ID\ \-\ optional\ for\ v1\ auth,\ this\ or\ tenant\ required\ otherwise\ (OS_TENANT_ID)
|
||
tenant_id>\
|
||
Tenant\ domain\ \-\ optional\ (v3\ auth)\ (OS_PROJECT_DOMAIN_NAME)
|
||
tenant_domain>\
|
||
Region\ name\ \-\ optional\ (OS_REGION_NAME)
|
||
region>\
|
||
Storage\ URL\ \-\ optional\ (OS_STORAGE_URL)
|
||
storage_url>\
|
||
Auth\ Token\ from\ alternate\ authentication\ \-\ optional\ (OS_AUTH_TOKEN)
|
||
auth_token>\
|
||
AuthVersion\ \-\ optional\ \-\ set\ to\ (1,2,3)\ if\ your\ auth\ URL\ has\ no\ version\ (ST_AUTH_VERSION)
|
||
auth_version>\
|
||
Endpoint\ type\ to\ choose\ from\ the\ service\ catalogue\ (OS_ENDPOINT_TYPE)
|
||
Choose\ a\ number\ from\ below,\ or\ type\ in\ your\ own\ value
|
||
\ 1\ /\ Public\ (default,\ choose\ this\ if\ not\ sure)
|
||
\ \ \ \\\ "public"
|
||
\ 2\ /\ Internal\ (use\ internal\ service\ net)
|
||
\ \ \ \\\ "internal"
|
||
\ 3\ /\ Admin
|
||
\ \ \ \\\ "admin"
|
||
endpoint_type>\
|
||
Remote\ config
|
||
\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-
|
||
[test]
|
||
env_auth\ =\ true
|
||
user\ =\
|
||
key\ =\
|
||
auth\ =\
|
||
user_id\ =\
|
||
domain\ =\
|
||
tenant\ =\
|
||
tenant_id\ =\
|
||
tenant_domain\ =\
|
||
region\ =\
|
||
storage_url\ =\
|
||
auth_token\ =\
|
||
auth_version\ =\
|
||
endpoint_type\ =\
|
||
\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-
|
||
y)\ Yes\ this\ is\ OK
|
||
e)\ Edit\ this\ remote
|
||
d)\ Delete\ this\ remote
|
||
y/e/d>\ y
|
||
\f[]
|
||
.fi
|
||
.PP
|
||
This remote is called \f[C]remote\f[] and can now be used like this
|
||
.PP
|
||
See all containers
|
||
.IP
|
||
.nf
|
||
\f[C]
|
||
rclone\ lsd\ remote:
|
||
\f[]
|
||
.fi
|
||
.PP
|
||
Make a new container
|
||
.IP
|
||
.nf
|
||
\f[C]
|
||
rclone\ mkdir\ remote:container
|
||
\f[]
|
||
.fi
|
||
.PP
|
||
List the contents of a container
|
||
.IP
|
||
.nf
|
||
\f[C]
|
||
rclone\ ls\ remote:container
|
||
\f[]
|
||
.fi
|
||
.PP
|
||
Sync \f[C]/home/local/directory\f[] to the remote container, deleting
|
||
any excess files in the container.
|
||
.IP
|
||
.nf
|
||
\f[C]
|
||
rclone\ sync\ /home/local/directory\ remote:container
|
||
\f[]
|
||
.fi
|
||
.SS Configuration from an Openstack credentials file
|
||
.PP
|
||
An Opentstack credentials file typically looks something something like
|
||
this (without the comments)
|
||
.IP
|
||
.nf
|
||
\f[C]
|
||
export\ OS_AUTH_URL=https://a.provider.net/v2.0
|
||
export\ OS_TENANT_ID=ffffffffffffffffffffffffffffffff
|
||
export\ OS_TENANT_NAME="1234567890123456"
|
||
export\ OS_USERNAME="123abc567xy"
|
||
echo\ "Please\ enter\ your\ OpenStack\ Password:\ "
|
||
read\ \-sr\ OS_PASSWORD_INPUT
|
||
export\ OS_PASSWORD=$OS_PASSWORD_INPUT
|
||
export\ OS_REGION_NAME="SBG1"
|
||
if\ [\ \-z\ "$OS_REGION_NAME"\ ];\ then\ unset\ OS_REGION_NAME;\ fi
|
||
\f[]
|
||
.fi
|
||
.PP
|
||
The config file needs to look something like this where
|
||
\f[C]$OS_USERNAME\f[] represents the value of the \f[C]OS_USERNAME\f[]
|
||
variable \- \f[C]123abc567xy\f[] in the example above.
|
||
.IP
|
||
.nf
|
||
\f[C]
|
||
[remote]
|
||
type\ =\ swift
|
||
user\ =\ $OS_USERNAME
|
||
key\ =\ $OS_PASSWORD
|
||
auth\ =\ $OS_AUTH_URL
|
||
tenant\ =\ $OS_TENANT_NAME
|
||
\f[]
|
||
.fi
|
||
.PP
|
||
Note that you may (or may not) need to set \f[C]region\f[] too \- try
|
||
without first.
|
||
.SS Configuration from the environment
|
||
.PP
|
||
If you prefer you can configure rclone to use swift using a standard set
|
||
of OpenStack environment variables.
|
||
.PP
|
||
When you run through the config, make sure you choose \f[C]true\f[] for
|
||
\f[C]env_auth\f[] and leave everything else blank.
|
||
.PP
|
||
rclone will then set any empty config parameters from the environment
|
||
using standard OpenStack environment variables.
|
||
There is a list of the
|
||
variables (https://godoc.org/github.com/ncw/swift#Connection.ApplyEnvironment)
|
||
in the docs for the swift library.
|
||
.SS Using an alternate authentication method
|
||
.PP
|
||
If your OpenStack installation uses a non\-standard authentication
|
||
method that might not be yet supported by rclone or the underlying swift
|
||
library, you can authenticate externally (e.g.
|
||
calling manually the \f[C]openstack\f[] commands to get a token).
|
||
Then, you just need to pass the two configuration variables
|
||
\f[C]auth_token\f[] and \f[C]storage_url\f[].
|
||
If they are both provided, the other variables are ignored.
|
||
rclone will not try to authenticate but instead assume it is already
|
||
authenticated and use these two variables to access the OpenStack
|
||
installation.
|
||
.SS Using rclone without a config file
|
||
.PP
|
||
You can use rclone with swift without a config file, if desired, like
|
||
this:
|
||
.IP
|
||
.nf
|
||
\f[C]
|
||
source\ openstack\-credentials\-file
|
||
export\ RCLONE_CONFIG_MYREMOTE_TYPE=swift
|
||
export\ RCLONE_CONFIG_MYREMOTE_ENV_AUTH=true
|
||
rclone\ lsd\ myremote:
|
||
\f[]
|
||
.fi
|
||
.SS \-\-fast\-list
|
||
.PP
|
||
This remote supports \f[C]\-\-fast\-list\f[] which allows you to use
|
||
fewer transactions in exchange for more memory.
|
||
See the rclone docs (/docs/#fast-list) for more details.
|
||
.SS Specific options
|
||
.PP
|
||
Here are the command line options specific to this cloud storage system.
|
||
.SS \-\-swift\-chunk\-size=SIZE
|
||
.PP
|
||
Above this size files will be chunked into a _segments container.
|
||
The default for this is 5GB which is its maximum value.
|
||
.SS Modified time
|
||
.PP
|
||
The modified time is stored as metadata on the object as
|
||
\f[C]X\-Object\-Meta\-Mtime\f[] as floating point since the epoch
|
||
accurate to 1 ns.
|
||
.PP
|
||
This is a defacto standard (used in the official python\-swiftclient
|
||
amongst others) for storing the modification time for an object.
|
||
.SS Limitations
|
||
.PP
|
||
The Swift API doesn\[aq]t return a correct MD5SUM for segmented files
|
||
(Dynamic or Static Large Objects) so rclone won\[aq]t check or use the
|
||
MD5SUM for these.
|
||
.SS Troubleshooting
|
||
.SS Rclone gives Failed to create file system for "remote:": Bad Request
|
||
.PP
|
||
Due to an oddity of the underlying swift library, it gives a "Bad
|
||
Request" error rather than a more sensible error when the authentication
|
||
fails for Swift.
|
||
.PP
|
||
So this most likely means your username / password is wrong.
|
||
You can investigate further with the \f[C]\-\-dump\-bodies\f[] flag.
|
||
.PP
|
||
This may also be caused by specifying the region when you shouldn\[aq]t
|
||
have (eg OVH).
|
||
.SS Rclone gives Failed to create file system: Response didn\[aq]t have
|
||
storage storage url and auth token
|
||
.PP
|
||
This is most likely caused by forgetting to specify your tenant when
|
||
setting up a swift remote.
|
||
.SS pCloud
|
||
.PP
|
||
Paths are specified as \f[C]remote:path\f[]
|
||
.PP
|
||
Paths may be as deep as required, eg
|
||
\f[C]remote:directory/subdirectory\f[].
|
||
.PP
|
||
The initial setup for pCloud involves getting a token from pCloud which
|
||
you need to do in your browser.
|
||
\f[C]rclone\ config\f[] walks you through it.
|
||
.PP
|
||
Here is an example of how to make a remote called \f[C]remote\f[].
|
||
First run:
|
||
.IP
|
||
.nf
|
||
\f[C]
|
||
\ rclone\ config
|
||
\f[]
|
||
.fi
|
||
.PP
|
||
This will guide you through an interactive setup process:
|
||
.IP
|
||
.nf
|
||
\f[C]
|
||
No\ remotes\ found\ \-\ make\ a\ new\ one
|
||
n)\ New\ remote
|
||
s)\ Set\ configuration\ password
|
||
q)\ Quit\ config
|
||
n/s/q>\ n
|
||
name>\ remote
|
||
Type\ of\ storage\ to\ configure.
|
||
Choose\ a\ number\ from\ below,\ or\ type\ in\ your\ own\ value
|
||
\ 1\ /\ Amazon\ Drive
|
||
\ \ \ \\\ "amazon\ cloud\ drive"
|
||
\ 2\ /\ Amazon\ S3\ (also\ Dreamhost,\ Ceph,\ Minio)
|
||
\ \ \ \\\ "s3"
|
||
\ 3\ /\ Backblaze\ B2
|
||
\ \ \ \\\ "b2"
|
||
\ 4\ /\ Box
|
||
\ \ \ \\\ "box"
|
||
\ 5\ /\ Dropbox
|
||
\ \ \ \\\ "dropbox"
|
||
\ 6\ /\ Encrypt/Decrypt\ a\ remote
|
||
\ \ \ \\\ "crypt"
|
||
\ 7\ /\ FTP\ Connection
|
||
\ \ \ \\\ "ftp"
|
||
\ 8\ /\ Google\ Cloud\ Storage\ (this\ is\ not\ Google\ Drive)
|
||
\ \ \ \\\ "google\ cloud\ storage"
|
||
\ 9\ /\ Google\ Drive
|
||
\ \ \ \\\ "drive"
|
||
10\ /\ Hubic
|
||
\ \ \ \\\ "hubic"
|
||
11\ /\ Local\ Disk
|
||
\ \ \ \\\ "local"
|
||
12\ /\ Microsoft\ Azure\ Blob\ Storage
|
||
\ \ \ \\\ "azureblob"
|
||
13\ /\ Microsoft\ OneDrive
|
||
\ \ \ \\\ "onedrive"
|
||
14\ /\ Openstack\ Swift\ (Rackspace\ Cloud\ Files,\ Memset\ Memstore,\ OVH)
|
||
\ \ \ \\\ "swift"
|
||
15\ /\ Pcloud
|
||
\ \ \ \\\ "pcloud"
|
||
16\ /\ QingCloud\ Object\ Storage
|
||
\ \ \ \\\ "qingstor"
|
||
17\ /\ SSH/SFTP\ Connection
|
||
\ \ \ \\\ "sftp"
|
||
18\ /\ Yandex\ Disk
|
||
\ \ \ \\\ "yandex"
|
||
19\ /\ http\ Connection
|
||
\ \ \ \\\ "http"
|
||
Storage>\ pcloud
|
||
Pcloud\ App\ Client\ Id\ \-\ leave\ blank\ normally.
|
||
client_id>\
|
||
Pcloud\ App\ Client\ Secret\ \-\ leave\ blank\ normally.
|
||
client_secret>\
|
||
Remote\ config
|
||
Use\ auto\ config?
|
||
\ *\ Say\ Y\ if\ not\ sure
|
||
\ *\ Say\ N\ if\ you\ are\ working\ on\ a\ remote\ or\ headless\ machine
|
||
y)\ Yes
|
||
n)\ No
|
||
y/n>\ y
|
||
If\ your\ browser\ doesn\[aq]t\ open\ automatically\ go\ to\ the\ following\ link:\ http://127.0.0.1:53682/auth
|
||
Log\ in\ and\ authorize\ rclone\ for\ access
|
||
Waiting\ for\ code...
|
||
Got\ code
|
||
\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-
|
||
[remote]
|
||
client_id\ =\
|
||
client_secret\ =\
|
||
token\ =\ {"access_token":"XXX","token_type":"bearer","expiry":"0001\-01\-01T00:00:00Z"}
|
||
\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-
|
||
y)\ Yes\ this\ is\ OK
|
||
e)\ Edit\ this\ remote
|
||
d)\ Delete\ this\ remote
|
||
y/e/d>\ y
|
||
\f[]
|
||
.fi
|
||
.PP
|
||
See the remote setup docs (https://rclone.org/remote_setup/) for how to
|
||
set it up on a machine with no Internet browser available.
|
||
.PP
|
||
Note that rclone runs a webserver on your local machine to collect the
|
||
token as returned from pCloud.
|
||
This only runs from the moment it opens your browser to the moment you
|
||
get back the verification code.
|
||
This is on \f[C]http://127.0.0.1:53682/\f[] and this it may require you
|
||
to unblock it temporarily if you are running a host firewall.
|
||
.PP
|
||
Once configured you can then use \f[C]rclone\f[] like this,
|
||
.PP
|
||
List directories in top level of your pCloud
|
||
.IP
|
||
.nf
|
||
\f[C]
|
||
rclone\ lsd\ remote:
|
||
\f[]
|
||
.fi
|
||
.PP
|
||
List all the files in your pCloud
|
||
.IP
|
||
.nf
|
||
\f[C]
|
||
rclone\ ls\ remote:
|
||
\f[]
|
||
.fi
|
||
.PP
|
||
To copy a local directory to an pCloud directory called backup
|
||
.IP
|
||
.nf
|
||
\f[C]
|
||
rclone\ copy\ /home/source\ remote:backup
|
||
\f[]
|
||
.fi
|
||
.SS Modified time and hashes
|
||
.PP
|
||
pCloud allows modification times to be set on objects accurate to 1
|
||
second.
|
||
These will be used to detect whether objects need syncing or not.
|
||
In order to set a Modification time pCloud requires the object be
|
||
re\-uploaded.
|
||
.PP
|
||
pCloud supports MD5 and SHA1 type hashes, so you can use the
|
||
\f[C]\-\-checksum\f[] flag.
|
||
.SS Deleting files
|
||
.PP
|
||
Deleted files will be moved to the trash.
|
||
Your subscription level will determine how long items stay in the trash.
|
||
\f[C]rclone\ cleanup\f[] can be used to empty the trash.
|
||
.SS SFTP
|
||
.PP
|
||
SFTP is the Secure (or SSH) File Transfer
|
||
Protocol (https://en.wikipedia.org/wiki/SSH_File_Transfer_Protocol).
|
||
.PP
|
||
It runs over SSH v2 and is standard with most modern SSH installations.
|
||
.PP
|
||
Paths are specified as \f[C]remote:path\f[].
|
||
If the path does not begin with a \f[C]/\f[] it is relative to the home
|
||
directory of the user.
|
||
An empty path \f[C]remote:\f[] refers to the user\[aq]s home directory.
|
||
.PP
|
||
Here is an example of making an SFTP configuration.
|
||
First run
|
||
.IP
|
||
.nf
|
||
\f[C]
|
||
rclone\ config
|
||
\f[]
|
||
.fi
|
||
.PP
|
||
This will guide you through an interactive setup process.
|
||
.IP
|
||
.nf
|
||
\f[C]
|
||
No\ remotes\ found\ \-\ make\ a\ new\ one
|
||
n)\ New\ remote
|
||
s)\ Set\ configuration\ password
|
||
q)\ Quit\ config
|
||
n/s/q>\ n
|
||
name>\ remote
|
||
Type\ of\ storage\ to\ configure.
|
||
Choose\ a\ number\ from\ below,\ or\ type\ in\ your\ own\ value
|
||
\ 1\ /\ Amazon\ Drive
|
||
\ \ \ \\\ "amazon\ cloud\ drive"
|
||
\ 2\ /\ Amazon\ S3\ (also\ Dreamhost,\ Ceph,\ Minio)
|
||
\ \ \ \\\ "s3"
|
||
\ 3\ /\ Backblaze\ B2
|
||
\ \ \ \\\ "b2"
|
||
\ 4\ /\ Dropbox
|
||
\ \ \ \\\ "dropbox"
|
||
\ 5\ /\ Encrypt/Decrypt\ a\ remote
|
||
\ \ \ \\\ "crypt"
|
||
\ 6\ /\ FTP\ Connection
|
||
\ \ \ \\\ "ftp"
|
||
\ 7\ /\ Google\ Cloud\ Storage\ (this\ is\ not\ Google\ Drive)
|
||
\ \ \ \\\ "google\ cloud\ storage"
|
||
\ 8\ /\ Google\ Drive
|
||
\ \ \ \\\ "drive"
|
||
\ 9\ /\ Hubic
|
||
\ \ \ \\\ "hubic"
|
||
10\ /\ Local\ Disk
|
||
\ \ \ \\\ "local"
|
||
11\ /\ Microsoft\ OneDrive
|
||
\ \ \ \\\ "onedrive"
|
||
12\ /\ Openstack\ Swift\ (Rackspace\ Cloud\ Files,\ Memset\ Memstore,\ OVH)
|
||
\ \ \ \\\ "swift"
|
||
13\ /\ SSH/SFTP\ Connection
|
||
\ \ \ \\\ "sftp"
|
||
14\ /\ Yandex\ Disk
|
||
\ \ \ \\\ "yandex"
|
||
15\ /\ http\ Connection
|
||
\ \ \ \\\ "http"
|
||
Storage>\ sftp
|
||
SSH\ host\ to\ connect\ to
|
||
Choose\ a\ number\ from\ below,\ or\ type\ in\ your\ own\ value
|
||
\ 1\ /\ Connect\ to\ example.com
|
||
\ \ \ \\\ "example.com"
|
||
host>\ example.com
|
||
SSH\ username,\ leave\ blank\ for\ current\ username,\ ncw
|
||
user>\ sftpuser
|
||
SSH\ port,\ leave\ blank\ to\ use\ default\ (22)
|
||
port>\
|
||
SSH\ password,\ leave\ blank\ to\ use\ ssh\-agent.
|
||
y)\ Yes\ type\ in\ my\ own\ password
|
||
g)\ Generate\ random\ password
|
||
n)\ No\ leave\ this\ optional\ password\ blank
|
||
y/g/n>\ n
|
||
Path\ to\ unencrypted\ PEM\-encoded\ private\ key\ file,\ leave\ blank\ to\ use\ ssh\-agent.
|
||
key_file>\
|
||
Remote\ config
|
||
\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-
|
||
[remote]
|
||
host\ =\ example.com
|
||
user\ =\ sftpuser
|
||
port\ =\
|
||
pass\ =\
|
||
key_file\ =\
|
||
\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-
|
||
y)\ Yes\ this\ is\ OK
|
||
e)\ Edit\ this\ remote
|
||
d)\ Delete\ this\ remote
|
||
y/e/d>\ y
|
||
\f[]
|
||
.fi
|
||
.PP
|
||
This remote is called \f[C]remote\f[] and can now be used like this:
|
||
.PP
|
||
See all directories in the home directory
|
||
.IP
|
||
.nf
|
||
\f[C]
|
||
rclone\ lsd\ remote:
|
||
\f[]
|
||
.fi
|
||
.PP
|
||
Make a new directory
|
||
.IP
|
||
.nf
|
||
\f[C]
|
||
rclone\ mkdir\ remote:path/to/directory
|
||
\f[]
|
||
.fi
|
||
.PP
|
||
List the contents of a directory
|
||
.IP
|
||
.nf
|
||
\f[C]
|
||
rclone\ ls\ remote:path/to/directory
|
||
\f[]
|
||
.fi
|
||
.PP
|
||
Sync \f[C]/home/local/directory\f[] to the remote directory, deleting
|
||
any excess files in the directory.
|
||
.IP
|
||
.nf
|
||
\f[C]
|
||
rclone\ sync\ /home/local/directory\ remote:directory
|
||
\f[]
|
||
.fi
|
||
.SS SSH Authentication
|
||
.PP
|
||
The SFTP remote supports three authentication methods:
|
||
.IP \[bu] 2
|
||
Password
|
||
.IP \[bu] 2
|
||
Key file
|
||
.IP \[bu] 2
|
||
ssh\-agent
|
||
.PP
|
||
Key files should be unencrypted PEM\-encoded private key files.
|
||
For instance \f[C]/home/$USER/.ssh/id_rsa\f[].
|
||
.PP
|
||
If you don\[aq]t specify \f[C]pass\f[] or \f[C]key_file\f[] then rclone
|
||
will attempt to contact an ssh\-agent.
|
||
.SS ssh\-agent on macOS
|
||
.PP
|
||
Note that there seem to be various problems with using an ssh\-agent on
|
||
macOS due to recent changes in the OS.
|
||
The most effective work\-around seems to be to start an ssh\-agent in
|
||
each session, eg
|
||
.IP
|
||
.nf
|
||
\f[C]
|
||
eval\ `ssh\-agent\ \-s`\ &&\ ssh\-add\ \-A
|
||
\f[]
|
||
.fi
|
||
.PP
|
||
And then at the end of the session
|
||
.IP
|
||
.nf
|
||
\f[C]
|
||
eval\ `ssh\-agent\ \-k`
|
||
\f[]
|
||
.fi
|
||
.PP
|
||
These commands can be used in scripts of course.
|
||
.SS Modified time
|
||
.PP
|
||
Modified times are stored on the server to 1 second precision.
|
||
.PP
|
||
Modified times are used in syncing and are fully supported.
|
||
.SS Limitations
|
||
.PP
|
||
SFTP supports checksums if the same login has shell access and
|
||
\f[C]md5sum\f[] or \f[C]sha1sum\f[] as well as \f[C]echo\f[] are in the
|
||
remote\[aq]s PATH.
|
||
.PP
|
||
The only ssh agent supported under Windows is Putty\[aq]s pageant.
|
||
.PP
|
||
The Go SSH library disables the use of the aes128\-cbc cipher by
|
||
default, due to security concerns.
|
||
This can be re\-enabled on a per\-connection basis by setting the
|
||
\f[C]use_insecure_cipher\f[] setting in the configuration file to
|
||
\f[C]true\f[].
|
||
Further details on the insecurity of this cipher can be found [in this
|
||
paper] (http://www.isg.rhul.ac.uk/~kp/SandPfinal.pdf).
|
||
.PP
|
||
SFTP isn\[aq]t supported under plan9 until this
|
||
issue (https://github.com/pkg/sftp/issues/156) is fixed.
|
||
.PP
|
||
Note that since SFTP isn\[aq]t HTTP based the following flags don\[aq]t
|
||
work with it: \f[C]\-\-dump\-headers\f[], \f[C]\-\-dump\-bodies\f[],
|
||
\f[C]\-\-dump\-auth\f[]
|
||
.PP
|
||
Note that \f[C]\-\-timeout\f[] isn\[aq]t supported (but
|
||
\f[C]\-\-contimeout\f[] is).
|
||
.SS WebDAV
|
||
.PP
|
||
Paths are specified as \f[C]remote:path\f[]
|
||
.PP
|
||
Paths may be as deep as required, eg
|
||
\f[C]remote:directory/subdirectory\f[].
|
||
.PP
|
||
To configure the WebDAV remote you will need to have a URL for it, and a
|
||
username and password.
|
||
If you know what kind of system you are connecting to then rclone can
|
||
enable extra features.
|
||
.PP
|
||
Here is an example of how to make a remote called \f[C]remote\f[].
|
||
First run:
|
||
.IP
|
||
.nf
|
||
\f[C]
|
||
\ rclone\ config
|
||
\f[]
|
||
.fi
|
||
.PP
|
||
This will guide you through an interactive setup process:
|
||
.IP
|
||
.nf
|
||
\f[C]
|
||
No\ remotes\ found\ \-\ make\ a\ new\ one
|
||
n)\ New\ remote
|
||
s)\ Set\ configuration\ password
|
||
q)\ Quit\ config
|
||
n/s/q>\ n
|
||
name>\ remote
|
||
Type\ of\ storage\ to\ configure.
|
||
Choose\ a\ number\ from\ below,\ or\ type\ in\ your\ own\ value
|
||
\ 1\ /\ Amazon\ Drive
|
||
\ \ \ \\\ "amazon\ cloud\ drive"
|
||
\ 2\ /\ Amazon\ S3\ (also\ Dreamhost,\ Ceph,\ Minio)
|
||
\ \ \ \\\ "s3"
|
||
\ 3\ /\ Backblaze\ B2
|
||
\ \ \ \\\ "b2"
|
||
\ 4\ /\ Box
|
||
\ \ \ \\\ "box"
|
||
\ 5\ /\ Dropbox
|
||
\ \ \ \\\ "dropbox"
|
||
\ 6\ /\ Encrypt/Decrypt\ a\ remote
|
||
\ \ \ \\\ "crypt"
|
||
\ 7\ /\ FTP\ Connection
|
||
\ \ \ \\\ "ftp"
|
||
\ 8\ /\ Google\ Cloud\ Storage\ (this\ is\ not\ Google\ Drive)
|
||
\ \ \ \\\ "google\ cloud\ storage"
|
||
\ 9\ /\ Google\ Drive
|
||
\ \ \ \\\ "drive"
|
||
10\ /\ Hubic
|
||
\ \ \ \\\ "hubic"
|
||
11\ /\ Local\ Disk
|
||
\ \ \ \\\ "local"
|
||
12\ /\ Microsoft\ Azure\ Blob\ Storage
|
||
\ \ \ \\\ "azureblob"
|
||
13\ /\ Microsoft\ OneDrive
|
||
\ \ \ \\\ "onedrive"
|
||
14\ /\ Openstack\ Swift\ (Rackspace\ Cloud\ Files,\ Memset\ Memstore,\ OVH)
|
||
\ \ \ \\\ "swift"
|
||
15\ /\ Pcloud
|
||
\ \ \ \\\ "pcloud"
|
||
16\ /\ QingCloud\ Object\ Storage
|
||
\ \ \ \\\ "qingstor"
|
||
17\ /\ SSH/SFTP\ Connection
|
||
\ \ \ \\\ "sftp"
|
||
18\ /\ WebDAV
|
||
\ \ \ \\\ "webdav"
|
||
19\ /\ Yandex\ Disk
|
||
\ \ \ \\\ "yandex"
|
||
20\ /\ http\ Connection
|
||
\ \ \ \\\ "http"
|
||
Storage>\ webdav
|
||
URL\ of\ http\ host\ to\ connect\ to
|
||
Choose\ a\ number\ from\ below,\ or\ type\ in\ your\ own\ value
|
||
\ 1\ /\ Connect\ to\ example.com
|
||
\ \ \ \\\ "https://example.com"
|
||
url>\ https://example.com/remote.php/webdav/
|
||
Name\ of\ the\ WebDAV\ site/service/software\ you\ are\ using
|
||
Choose\ a\ number\ from\ below,\ or\ type\ in\ your\ own\ value
|
||
\ 1\ /\ Nextcloud
|
||
\ \ \ \\\ "nextcloud"
|
||
\ 2\ /\ Owncloud
|
||
\ \ \ \\\ "owncloud"
|
||
\ 3\ /\ Other\ site/service\ or\ software
|
||
\ \ \ \\\ "other"
|
||
vendor>\ 1
|
||
User\ name
|
||
user>\ user
|
||
Password.
|
||
y)\ Yes\ type\ in\ my\ own\ password
|
||
g)\ Generate\ random\ password
|
||
n)\ No\ leave\ this\ optional\ password\ blank
|
||
y/g/n>\ y
|
||
Enter\ the\ password:
|
||
password:
|
||
Confirm\ the\ password:
|
||
password:
|
||
Remote\ config
|
||
\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-
|
||
[remote]
|
||
url\ =\ https://example.com/remote.php/webdav/
|
||
vendor\ =\ nextcloud
|
||
user\ =\ user
|
||
pass\ =\ ***\ ENCRYPTED\ ***
|
||
\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-
|
||
y)\ Yes\ this\ is\ OK
|
||
e)\ Edit\ this\ remote
|
||
d)\ Delete\ this\ remote
|
||
y/e/d>\ y
|
||
\f[]
|
||
.fi
|
||
.PP
|
||
Once configured you can then use \f[C]rclone\f[] like this,
|
||
.PP
|
||
List directories in top level of your WebDAV
|
||
.IP
|
||
.nf
|
||
\f[C]
|
||
rclone\ lsd\ remote:
|
||
\f[]
|
||
.fi
|
||
.PP
|
||
List all the files in your WebDAV
|
||
.IP
|
||
.nf
|
||
\f[C]
|
||
rclone\ ls\ remote:
|
||
\f[]
|
||
.fi
|
||
.PP
|
||
To copy a local directory to an WebDAV directory called backup
|
||
.IP
|
||
.nf
|
||
\f[C]
|
||
rclone\ copy\ /home/source\ remote:backup
|
||
\f[]
|
||
.fi
|
||
.SS Modified time and hashes
|
||
.PP
|
||
Plain WebDAV does not support modified times.
|
||
However when used with Owncloud or Nextcloud rclone will support
|
||
modified times.
|
||
.PP
|
||
Hashes are not supported.
|
||
.SS Owncloud
|
||
.PP
|
||
Click on the settings cog in the bottom right of the page and this will
|
||
show the WebDAV URL that rclone needs in the config step.
|
||
It will look something like
|
||
\f[C]https://example.com/remote.php/webdav/\f[].
|
||
.PP
|
||
Owncloud supports modified times using the \f[C]X\-OC\-Mtime\f[] header.
|
||
.SS Nextcloud
|
||
.PP
|
||
This is configured in an identical way to Owncloud.
|
||
Note that Nextcloud does not support streaming of files (\f[C]rcat\f[])
|
||
whereas Owncloud does.
|
||
This may be
|
||
fixed (https://github.com/nextcloud/nextcloud-snap/issues/365) in the
|
||
future.
|
||
.SS Put.io
|
||
.PP
|
||
put.io can be accessed in a read only way using webdav.
|
||
.PP
|
||
Configure the \f[C]url\f[] as \f[C]https://webdav.put.io\f[] and use
|
||
your normal account username and password for \f[C]user\f[] and
|
||
\f[C]pass\f[].
|
||
Set the \f[C]vendor\f[] to \f[C]other\f[].
|
||
.PP
|
||
Your config file should end up looking like this:
|
||
.IP
|
||
.nf
|
||
\f[C]
|
||
[putio]
|
||
type\ =\ webdav
|
||
url\ =\ https://webdav.put.io
|
||
vendor\ =\ other
|
||
user\ =\ YourUserName
|
||
pass\ =\ encryptedpassword
|
||
\f[]
|
||
.fi
|
||
.PP
|
||
If you are using \f[C]put.io\f[] with \f[C]rclone\ mount\f[] then use
|
||
the \f[C]\-\-read\-only\f[] flag to signal to the OS that it can\[aq]t
|
||
write to the mount.
|
||
.PP
|
||
For more help see the put.io webdav
|
||
docs (http://help.put.io/apps-and-integrations/ftp-and-webdav).
|
||
.SS Yandex Disk
|
||
.PP
|
||
Yandex Disk (https://disk.yandex.com) is a cloud storage solution
|
||
created by Yandex (https://yandex.com).
|
||
.PP
|
||
Yandex paths may be as deep as required, eg
|
||
\f[C]remote:directory/subdirectory\f[].
|
||
.PP
|
||
Here is an example of making a yandex configuration.
|
||
First run
|
||
.IP
|
||
.nf
|
||
\f[C]
|
||
rclone\ config
|
||
\f[]
|
||
.fi
|
||
.PP
|
||
This will guide you through an interactive setup process:
|
||
.IP
|
||
.nf
|
||
\f[C]
|
||
No\ remotes\ found\ \-\ make\ a\ new\ one
|
||
n)\ New\ remote
|
||
s)\ Set\ configuration\ password
|
||
n/s>\ n
|
||
name>\ remote
|
||
Type\ of\ storage\ to\ configure.
|
||
Choose\ a\ number\ from\ below,\ or\ type\ in\ your\ own\ value
|
||
\ 1\ /\ Amazon\ Drive
|
||
\ \ \ \\\ "amazon\ cloud\ drive"
|
||
\ 2\ /\ Amazon\ S3\ (also\ Dreamhost,\ Ceph,\ Minio)
|
||
\ \ \ \\\ "s3"
|
||
\ 3\ /\ Backblaze\ B2
|
||
\ \ \ \\\ "b2"
|
||
\ 4\ /\ Dropbox
|
||
\ \ \ \\\ "dropbox"
|
||
\ 5\ /\ Encrypt/Decrypt\ a\ remote
|
||
\ \ \ \\\ "crypt"
|
||
\ 6\ /\ Google\ Cloud\ Storage\ (this\ is\ not\ Google\ Drive)
|
||
\ \ \ \\\ "google\ cloud\ storage"
|
||
\ 7\ /\ Google\ Drive
|
||
\ \ \ \\\ "drive"
|
||
\ 8\ /\ Hubic
|
||
\ \ \ \\\ "hubic"
|
||
\ 9\ /\ Local\ Disk
|
||
\ \ \ \\\ "local"
|
||
10\ /\ Microsoft\ OneDrive
|
||
\ \ \ \\\ "onedrive"
|
||
11\ /\ Openstack\ Swift\ (Rackspace\ Cloud\ Files,\ Memset\ Memstore,\ OVH)
|
||
\ \ \ \\\ "swift"
|
||
12\ /\ SSH/SFTP\ Connection
|
||
\ \ \ \\\ "sftp"
|
||
13\ /\ Yandex\ Disk
|
||
\ \ \ \\\ "yandex"
|
||
Storage>\ 13
|
||
Yandex\ Client\ Id\ \-\ leave\ blank\ normally.
|
||
client_id>
|
||
Yandex\ Client\ Secret\ \-\ leave\ blank\ normally.
|
||
client_secret>
|
||
Remote\ config
|
||
Use\ auto\ config?
|
||
\ *\ Say\ Y\ if\ not\ sure
|
||
\ *\ Say\ N\ if\ you\ are\ working\ on\ a\ remote\ or\ headless\ machine
|
||
y)\ Yes
|
||
n)\ No
|
||
y/n>\ y
|
||
If\ your\ browser\ doesn\[aq]t\ open\ automatically\ go\ to\ the\ following\ link:\ http://127.0.0.1:53682/auth
|
||
Log\ in\ and\ authorize\ rclone\ for\ access
|
||
Waiting\ for\ code...
|
||
Got\ code
|
||
\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-
|
||
[remote]
|
||
client_id\ =
|
||
client_secret\ =
|
||
token\ =\ {"access_token":"xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx","token_type":"bearer","expiry":"2016\-12\-29T12:27:11.362788025Z"}
|
||
\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-
|
||
y)\ Yes\ this\ is\ OK
|
||
e)\ Edit\ this\ remote
|
||
d)\ Delete\ this\ remote
|
||
y/e/d>\ y
|
||
\f[]
|
||
.fi
|
||
.PP
|
||
See the remote setup docs (https://rclone.org/remote_setup/) for how to
|
||
set it up on a machine with no Internet browser available.
|
||
.PP
|
||
Note that rclone runs a webserver on your local machine to collect the
|
||
token as returned from Yandex Disk.
|
||
This only runs from the moment it opens your browser to the moment you
|
||
get back the verification code.
|
||
This is on \f[C]http://127.0.0.1:53682/\f[] and this it may require you
|
||
to unblock it temporarily if you are running a host firewall.
|
||
.PP
|
||
Once configured you can then use \f[C]rclone\f[] like this,
|
||
.PP
|
||
See top level directories
|
||
.IP
|
||
.nf
|
||
\f[C]
|
||
rclone\ lsd\ remote:
|
||
\f[]
|
||
.fi
|
||
.PP
|
||
Make a new directory
|
||
.IP
|
||
.nf
|
||
\f[C]
|
||
rclone\ mkdir\ remote:directory
|
||
\f[]
|
||
.fi
|
||
.PP
|
||
List the contents of a directory
|
||
.IP
|
||
.nf
|
||
\f[C]
|
||
rclone\ ls\ remote:directory
|
||
\f[]
|
||
.fi
|
||
.PP
|
||
Sync \f[C]/home/local/directory\f[] to the remote path, deleting any
|
||
excess files in the path.
|
||
.IP
|
||
.nf
|
||
\f[C]
|
||
rclone\ sync\ /home/local/directory\ remote:directory
|
||
\f[]
|
||
.fi
|
||
.SS \-\-fast\-list
|
||
.PP
|
||
This remote supports \f[C]\-\-fast\-list\f[] which allows you to use
|
||
fewer transactions in exchange for more memory.
|
||
See the rclone docs (/docs/#fast-list) for more details.
|
||
.SS Modified time
|
||
.PP
|
||
Modified times are supported and are stored accurate to 1 ns in custom
|
||
metadata called \f[C]rclone_modified\f[] in RFC3339 with nanoseconds
|
||
format.
|
||
.SS MD5 checksums
|
||
.PP
|
||
MD5 checksums are natively supported by Yandex Disk.
|
||
.SS Emptying Trash
|
||
.PP
|
||
If you wish to empty your trash you can use the
|
||
\f[C]rclone\ cleanup\ remote:\f[] command which will permanently delete
|
||
all your trashed files.
|
||
This command does not take any path arguments.
|
||
.SS Local Filesystem
|
||
.PP
|
||
Local paths are specified as normal filesystem paths, eg
|
||
\f[C]/path/to/wherever\f[], so
|
||
.IP
|
||
.nf
|
||
\f[C]
|
||
rclone\ sync\ /home/source\ /tmp/destination
|
||
\f[]
|
||
.fi
|
||
.PP
|
||
Will sync \f[C]/home/source\f[] to \f[C]/tmp/destination\f[]
|
||
.PP
|
||
These can be configured into the config file for consistencies sake, but
|
||
it is probably easier not to.
|
||
.SS Modified time
|
||
.PP
|
||
Rclone reads and writes the modified time using an accuracy determined
|
||
by the OS.
|
||
Typically this is 1ns on Linux, 10 ns on Windows and 1 Second on OS X.
|
||
.SS Filenames
|
||
.PP
|
||
Filenames are expected to be encoded in UTF\-8 on disk.
|
||
This is the normal case for Windows and OS X.
|
||
.PP
|
||
There is a bit more uncertainty in the Linux world, but new
|
||
distributions will have UTF\-8 encoded files names.
|
||
If you are using an old Linux filesystem with non UTF\-8 file names (eg
|
||
latin1) then you can use the \f[C]convmv\f[] tool to convert the
|
||
filesystem to UTF\-8.
|
||
This tool is available in most distributions\[aq] package managers.
|
||
.PP
|
||
If an invalid (non\-UTF8) filename is read, the invalid characters will
|
||
be replaced with the unicode replacement character, \[aq]<5D>\[aq].
|
||
\f[C]rclone\f[] will emit a debug message in this case (use \f[C]\-v\f[]
|
||
to see), eg
|
||
.IP
|
||
.nf
|
||
\f[C]
|
||
Local\ file\ system\ at\ .:\ Replacing\ invalid\ UTF\-8\ characters\ in\ "gro\\xdf"
|
||
\f[]
|
||
.fi
|
||
.SS Long paths on Windows
|
||
.PP
|
||
Rclone handles long paths automatically, by converting all paths to long
|
||
UNC
|
||
paths (https://msdn.microsoft.com/en-us/library/windows/desktop/aa365247(v=vs.85).aspx#maxpath)
|
||
which allows paths up to 32,767 characters.
|
||
.PP
|
||
This is why you will see that your paths, for instance
|
||
\f[C]c:\\files\f[] is converted to the UNC path
|
||
\f[C]\\\\?\\c:\\files\f[] in the output, and \f[C]\\\\server\\share\f[]
|
||
is converted to \f[C]\\\\?\\UNC\\server\\share\f[].
|
||
.PP
|
||
However, in rare cases this may cause problems with buggy file system
|
||
drivers like EncFS (https://github.com/ncw/rclone/issues/261).
|
||
To disable UNC conversion globally, add this to your
|
||
\f[C]\&.rclone.conf\f[] file:
|
||
.IP
|
||
.nf
|
||
\f[C]
|
||
[local]
|
||
nounc\ =\ true
|
||
\f[]
|
||
.fi
|
||
.PP
|
||
If you want to selectively disable UNC, you can add it to a separate
|
||
entry like this:
|
||
.IP
|
||
.nf
|
||
\f[C]
|
||
[nounc]
|
||
type\ =\ local
|
||
nounc\ =\ true
|
||
\f[]
|
||
.fi
|
||
.PP
|
||
And use rclone like this:
|
||
.PP
|
||
\f[C]rclone\ copy\ c:\\src\ nounc:z:\\dst\f[]
|
||
.PP
|
||
This will use UNC paths on \f[C]c:\\src\f[] but not on \f[C]z:\\dst\f[].
|
||
Of course this will cause problems if the absolute path length of a file
|
||
exceeds 258 characters on z, so only use this option if you have to.
|
||
.SS Specific options
|
||
.PP
|
||
Here are the command line options specific to local storage
|
||
.SS \-\-copy\-links, \-L
|
||
.PP
|
||
Normally rclone will ignore symlinks or junction points (which behave
|
||
like symlinks under Windows).
|
||
.PP
|
||
If you supply this flag then rclone will follow the symlink and copy the
|
||
pointed to file or directory.
|
||
.PP
|
||
This flag applies to all commands.
|
||
.PP
|
||
For example, supposing you have a directory structure like this
|
||
.IP
|
||
.nf
|
||
\f[C]
|
||
$\ tree\ /tmp/a
|
||
/tmp/a
|
||
├──\ b\ \->\ ../b
|
||
├──\ expected\ \->\ ../expected
|
||
├──\ one
|
||
└──\ two
|
||
\ \ \ \ └──\ three
|
||
\f[]
|
||
.fi
|
||
.PP
|
||
Then you can see the difference with and without the flag like this
|
||
.IP
|
||
.nf
|
||
\f[C]
|
||
$\ rclone\ ls\ /tmp/a
|
||
\ \ \ \ \ \ \ \ 6\ one
|
||
\ \ \ \ \ \ \ \ 6\ two/three
|
||
\f[]
|
||
.fi
|
||
.PP
|
||
and
|
||
.IP
|
||
.nf
|
||
\f[C]
|
||
$\ rclone\ \-L\ ls\ /tmp/a
|
||
\ \ \ \ \ 4174\ expected
|
||
\ \ \ \ \ \ \ \ 6\ one
|
||
\ \ \ \ \ \ \ \ 6\ two/three
|
||
\ \ \ \ \ \ \ \ 6\ b/two
|
||
\ \ \ \ \ \ \ \ 6\ b/one
|
||
\f[]
|
||
.fi
|
||
.SS \-\-local\-no\-unicode\-normalization
|
||
.PP
|
||
This flag is deprecated now.
|
||
Rclone no longer normalizes unicode file names, but it compares them
|
||
with unicode normalization in the sync routine instead.
|
||
.SS \-\-one\-file\-system, \-x
|
||
.PP
|
||
This tells rclone to stay in the filesystem specified by the root and
|
||
not to recurse into different file systems.
|
||
.PP
|
||
For example if you have a directory hierarchy like this
|
||
.IP
|
||
.nf
|
||
\f[C]
|
||
root
|
||
├──\ disk1\ \ \ \ \ \-\ disk1\ mounted\ on\ the\ root
|
||
│\ \ \ └──\ file3\ \-\ stored\ on\ disk1
|
||
├──\ disk2\ \ \ \ \ \-\ disk2\ mounted\ on\ the\ root
|
||
│\ \ \ └──\ file4\ \-\ stored\ on\ disk12
|
||
├──\ file1\ \ \ \ \ \-\ stored\ on\ the\ root\ disk
|
||
└──\ file2\ \ \ \ \ \-\ stored\ on\ the\ root\ disk
|
||
\f[]
|
||
.fi
|
||
.PP
|
||
Using \f[C]rclone\ \-\-one\-file\-system\ copy\ root\ remote:\f[] will
|
||
only copy \f[C]file1\f[] and \f[C]file2\f[].
|
||
Eg
|
||
.IP
|
||
.nf
|
||
\f[C]
|
||
$\ rclone\ \-q\ \-\-one\-file\-system\ ls\ root
|
||
\ \ \ \ \ \ \ \ 0\ file1
|
||
\ \ \ \ \ \ \ \ 0\ file2
|
||
\f[]
|
||
.fi
|
||
.IP
|
||
.nf
|
||
\f[C]
|
||
$\ rclone\ \-q\ ls\ root
|
||
\ \ \ \ \ \ \ \ 0\ disk1/file3
|
||
\ \ \ \ \ \ \ \ 0\ disk2/file4
|
||
\ \ \ \ \ \ \ \ 0\ file1
|
||
\ \ \ \ \ \ \ \ 0\ file2
|
||
\f[]
|
||
.fi
|
||
.PP
|
||
\f[B]NB\f[] Rclone (like most unix tools such as \f[C]du\f[],
|
||
\f[C]rsync\f[] and \f[C]tar\f[]) treats a bind mount to the same device
|
||
as being on the same filesystem.
|
||
.PP
|
||
\f[B]NB\f[] This flag is only available on Unix based systems.
|
||
On systems where it isn\[aq]t supported (eg Windows) it will not appear
|
||
as an valid flag.
|
||
.SS \-\-skip\-links
|
||
.PP
|
||
This flag disables warning messages on skipped symlinks or junction
|
||
points, as you explicitly acknowledge that they should be skipped.
|
||
.SS Changelog
|
||
.IP \[bu] 2
|
||
v1.39 \- 2017\-12\-23
|
||
.RS 2
|
||
.IP \[bu] 2
|
||
New backends
|
||
.IP \[bu] 2
|
||
WebDAV
|
||
.RS 2
|
||
.IP \[bu] 2
|
||
tested with nextcloud, owncloud, put.io and others!
|
||
.RE
|
||
.IP \[bu] 2
|
||
Pcloud
|
||
.IP \[bu] 2
|
||
cache \- wraps a cache around other backends (Remus Bunduc)
|
||
.RS 2
|
||
.IP \[bu] 2
|
||
useful in combination with mount
|
||
.IP \[bu] 2
|
||
NB this feature is in beta so use with care
|
||
.RE
|
||
.IP \[bu] 2
|
||
New commands
|
||
.IP \[bu] 2
|
||
serve command with subcommands:
|
||
.RS 2
|
||
.IP \[bu] 2
|
||
serve webdav: this implements a webdav server for any rclone remote.
|
||
.IP \[bu] 2
|
||
serve http: command to serve a remote over HTTP
|
||
.RE
|
||
.IP \[bu] 2
|
||
config: add sub commands for full config file management
|
||
.RS 2
|
||
.IP \[bu] 2
|
||
create/delete/dump/edit/file/password/providers/show/update
|
||
.RE
|
||
.IP \[bu] 2
|
||
touch: to create or update the timestamp of a file (Jakub Tasiemski)
|
||
.IP \[bu] 2
|
||
New Features
|
||
.IP \[bu] 2
|
||
curl install for rclone (Filip Bartodziej)
|
||
.IP \[bu] 2
|
||
\-\-stats now shows percentage, size, rate and ETA in condensed form
|
||
(Ishuah Kariuki)
|
||
.IP \[bu] 2
|
||
\-\-exclude\-if\-present to exclude a directory if a file is present
|
||
(Iakov Davydov)
|
||
.IP \[bu] 2
|
||
rmdirs: add \-\-leave\-root flag (lewpam)
|
||
.IP \[bu] 2
|
||
move: add \-\-delete\-empty\-src\-dirs flag to remove dirs after move
|
||
(Ishuah Kariuki)
|
||
.IP \[bu] 2
|
||
Add \-\-dump flag, introduce \-\-dump requests, responses and remove
|
||
\-\-dump\-auth, \-\-dump\-filters
|
||
.RS 2
|
||
.IP \[bu] 2
|
||
Obscure X\-Auth\-Token: from headers when dumping too
|
||
.RE
|
||
.IP \[bu] 2
|
||
Document and implement exit codes for different failure modes (Ishuah
|
||
Kariuki)
|
||
.IP \[bu] 2
|
||
Compile
|
||
.IP \[bu] 2
|
||
Bug Fixes
|
||
.IP \[bu] 2
|
||
Retry lots more different types of errors to make multipart transfers
|
||
more reliable
|
||
.IP \[bu] 2
|
||
Save the config before asking for a token, fixes disappearing oauth
|
||
config
|
||
.IP \[bu] 2
|
||
Warn the user if \-\-include and \-\-exclude are used together (Ernest
|
||
Borowski)
|
||
.IP \[bu] 2
|
||
Fix duplicate files (eg on Google drive) causing spurious copies
|
||
.IP \[bu] 2
|
||
Allow trailing and leading whitespace for passwords (Jason Rose)
|
||
.IP \[bu] 2
|
||
ncdu: fix crashes on empty directories
|
||
.IP \[bu] 2
|
||
rcat: fix goroutine leak
|
||
.IP \[bu] 2
|
||
moveto/copyto: Fix to allow copying to the same name
|
||
.IP \[bu] 2
|
||
Mount
|
||
.IP \[bu] 2
|
||
\-\-vfs\-cache mode to make writes into mounts more reliable.
|
||
.RS 2
|
||
.IP \[bu] 2
|
||
this requires caching files on the disk (see \-\-cache\-dir)
|
||
.IP \[bu] 2
|
||
As this is a new feature, use with care
|
||
.RE
|
||
.IP \[bu] 2
|
||
Use sdnotify to signal systemd the mount is ready (Fabian Möller)
|
||
.IP \[bu] 2
|
||
Check if directory is not empty before mounting (Ernest Borowski)
|
||
.IP \[bu] 2
|
||
Local
|
||
.IP \[bu] 2
|
||
Add error message for cross file system moves
|
||
.IP \[bu] 2
|
||
Fix equality check for times
|
||
.IP \[bu] 2
|
||
Dropbox
|
||
.IP \[bu] 2
|
||
Rework multipart upload
|
||
.RS 2
|
||
.IP \[bu] 2
|
||
buffer the chunks when uploading large files so they can be retried
|
||
.IP \[bu] 2
|
||
change default chunk size to 48MB now we are buffering them in memory
|
||
.IP \[bu] 2
|
||
retry every error after the first chunk is done successfully
|
||
.RE
|
||
.IP \[bu] 2
|
||
Fix error when renaming directories
|
||
.IP \[bu] 2
|
||
Swift
|
||
.IP \[bu] 2
|
||
Fix crash on bad authentication
|
||
.IP \[bu] 2
|
||
Google Drive
|
||
.IP \[bu] 2
|
||
Add service account support (Tim Cooijmans)
|
||
.IP \[bu] 2
|
||
S3
|
||
.IP \[bu] 2
|
||
Make it work properly with Digital Ocean Spaces (Andrew
|
||
Starr\-Bochicchio)
|
||
.IP \[bu] 2
|
||
Fix crash if a bad listing is received
|
||
.IP \[bu] 2
|
||
Add support for ECS task IAM roles (David Minor)
|
||
.IP \[bu] 2
|
||
Backblaze B2
|
||
.IP \[bu] 2
|
||
Fix multipart upload retries
|
||
.IP \[bu] 2
|
||
Fix \-\-hard\-delete to make it work 100% of the time
|
||
.IP \[bu] 2
|
||
Swift
|
||
.IP \[bu] 2
|
||
Allow authentication with storage URL and auth key (Giovanni Pizzi)
|
||
.IP \[bu] 2
|
||
Add new fields for swift configuration to support IBM Bluemix Swift
|
||
(Pierre Carlson)
|
||
.IP \[bu] 2
|
||
Add OS_TENANT_ID and OS_USER_ID to config
|
||
.IP \[bu] 2
|
||
Allow configs with user id instead of user name
|
||
.IP \[bu] 2
|
||
Check if swift segments container exists before creating (John Leach)
|
||
.IP \[bu] 2
|
||
Fix memory leak in swift transfers (upstream fix)
|
||
.IP \[bu] 2
|
||
SFTP
|
||
.IP \[bu] 2
|
||
Add option to enable the use of aes128\-cbc cipher (Jon Fautley)
|
||
.IP \[bu] 2
|
||
Amazon cloud drive
|
||
.IP \[bu] 2
|
||
Fix download of large files failing with "Only one auth mechanism
|
||
allowed"
|
||
.IP \[bu] 2
|
||
crypt
|
||
.IP \[bu] 2
|
||
Option to encrypt directory names or leave them intact
|
||
.IP \[bu] 2
|
||
Implement DirChangeNotify (Fabian Möller)
|
||
.IP \[bu] 2
|
||
onedrive
|
||
.IP \[bu] 2
|
||
Add option to choose resourceURL during setup of OneDrive Business
|
||
account if more than one is available for user
|
||
.RE
|
||
.IP \[bu] 2
|
||
v1.38 \- 2017\-09\-30
|
||
.RS 2
|
||
.IP \[bu] 2
|
||
New backends
|
||
.IP \[bu] 2
|
||
Azure Blob Storage (thanks Andrei Dragomir)
|
||
.IP \[bu] 2
|
||
Box
|
||
.IP \[bu] 2
|
||
Onedrive for Business (thanks Oliver Heyme)
|
||
.IP \[bu] 2
|
||
QingStor from QingCloud (thanks wuyu)
|
||
.IP \[bu] 2
|
||
New commands
|
||
.IP \[bu] 2
|
||
\f[C]rcat\f[] \- read from standard input and stream upload
|
||
.IP \[bu] 2
|
||
\f[C]tree\f[] \- shows a nicely formatted recursive listing
|
||
.IP \[bu] 2
|
||
\f[C]cryptdecode\f[] \- decode crypted file names (thanks ishuah)
|
||
.IP \[bu] 2
|
||
\f[C]config\ show\f[] \- print the config file
|
||
.IP \[bu] 2
|
||
\f[C]config\ file\f[] \- print the config file location
|
||
.IP \[bu] 2
|
||
New Features
|
||
.IP \[bu] 2
|
||
Empty directories are deleted on \f[C]sync\f[]
|
||
.IP \[bu] 2
|
||
\f[C]dedupe\f[] \- implement merging of duplicate directories
|
||
.IP \[bu] 2
|
||
\f[C]check\f[] and \f[C]cryptcheck\f[] made more consistent and use less
|
||
memory
|
||
.IP \[bu] 2
|
||
\f[C]cleanup\f[] for remaining remotes (thanks ishuah)
|
||
.IP \[bu] 2
|
||
\f[C]\-\-immutable\f[] for ensuring that files don\[aq]t change (thanks
|
||
Jacob McNamee)
|
||
.IP \[bu] 2
|
||
\f[C]\-\-user\-agent\f[] option (thanks Alex McGrath Kraak)
|
||
.IP \[bu] 2
|
||
\f[C]\-\-disable\f[] flag to disable optional features
|
||
.IP \[bu] 2
|
||
\f[C]\-\-bind\f[] flag for choosing the local addr on outgoing
|
||
connections
|
||
.IP \[bu] 2
|
||
Support for zsh auto\-completion (thanks bpicode)
|
||
.IP \[bu] 2
|
||
Stop normalizing file names but do a normalized compare in \f[C]sync\f[]
|
||
.IP \[bu] 2
|
||
Compile
|
||
.IP \[bu] 2
|
||
Update to using go1.9 as the default go version
|
||
.IP \[bu] 2
|
||
Remove snapd build due to maintenance problems
|
||
.IP \[bu] 2
|
||
Bug Fixes
|
||
.IP \[bu] 2
|
||
Improve retriable error detection which makes multipart uploads better
|
||
.IP \[bu] 2
|
||
Make \f[C]check\f[] obey \f[C]\-\-ignore\-size\f[]
|
||
.IP \[bu] 2
|
||
Fix bwlimit toggle in conjunction with schedules (thanks cbruegg)
|
||
.IP \[bu] 2
|
||
\f[C]config\f[] ensures newly written config is on the same mount
|
||
.IP \[bu] 2
|
||
Local
|
||
.IP \[bu] 2
|
||
Revert to copy when moving file across file system boundaries
|
||
.IP \[bu] 2
|
||
\f[C]\-\-skip\-links\f[] to suppress symlink warnings (thanks Zhiming
|
||
Wang)
|
||
.IP \[bu] 2
|
||
Mount
|
||
.IP \[bu] 2
|
||
Re\-use \f[C]rcat\f[] internals to support uploads from all remotes
|
||
.IP \[bu] 2
|
||
Dropbox
|
||
.IP \[bu] 2
|
||
Fix "entry doesn\[aq]t belong in directory" error
|
||
.IP \[bu] 2
|
||
Stop using deprecated API methods
|
||
.IP \[bu] 2
|
||
Swift
|
||
.IP \[bu] 2
|
||
Fix server side copy to empty container with \f[C]\-\-fast\-list\f[]
|
||
.IP \[bu] 2
|
||
Google Drive
|
||
.IP \[bu] 2
|
||
Change the default for \f[C]\-\-drive\-use\-trash\f[] to \f[C]true\f[]
|
||
.IP \[bu] 2
|
||
S3
|
||
.IP \[bu] 2
|
||
Set session token when using STS (thanks Girish Ramakrishnan)
|
||
.IP \[bu] 2
|
||
Glacier docs and error messages (thanks Jan Varho)
|
||
.IP \[bu] 2
|
||
Read 1000 (not 1024) items in dir listings to fix Wasabi
|
||
.IP \[bu] 2
|
||
Backblaze B2
|
||
.IP \[bu] 2
|
||
Fix SHA1 mismatch when downloading files with no SHA1
|
||
.IP \[bu] 2
|
||
Calculate missing hashes on the fly instead of spooling
|
||
.IP \[bu] 2
|
||
\f[C]\-\-b2\-hard\-delete\f[] to permanently delete (not hide) files
|
||
(thanks John Papandriopoulos)
|
||
.IP \[bu] 2
|
||
Hubic
|
||
.IP \[bu] 2
|
||
Fix creating containers \- no longer have to use the \f[C]default\f[]
|
||
container
|
||
.IP \[bu] 2
|
||
Swift
|
||
.IP \[bu] 2
|
||
Optionally configure from a standard set of OpenStack environment vars
|
||
.IP \[bu] 2
|
||
Add \f[C]endpoint_type\f[] config
|
||
.IP \[bu] 2
|
||
Google Cloud Storage
|
||
.IP \[bu] 2
|
||
Fix bucket creation to work with limited permission users
|
||
.IP \[bu] 2
|
||
SFTP
|
||
.IP \[bu] 2
|
||
Implement connection pooling for multiple ssh connections
|
||
.IP \[bu] 2
|
||
Limit new connections per second
|
||
.IP \[bu] 2
|
||
Add support for MD5 and SHA1 hashes where available (thanks Christian
|
||
Brüggemann)
|
||
.IP \[bu] 2
|
||
HTTP
|
||
.IP \[bu] 2
|
||
Fix URL encoding issues
|
||
.IP \[bu] 2
|
||
Fix directories with \f[C]:\f[] in
|
||
.IP \[bu] 2
|
||
Fix panic with URL encoded content
|
||
.RE
|
||
.IP \[bu] 2
|
||
v1.37 \- 2017\-07\-22
|
||
.RS 2
|
||
.IP \[bu] 2
|
||
New backends
|
||
.IP \[bu] 2
|
||
FTP \- thanks to Antonio Messina
|
||
.IP \[bu] 2
|
||
HTTP \- thanks to Vasiliy Tolstov
|
||
.IP \[bu] 2
|
||
New commands
|
||
.IP \[bu] 2
|
||
rclone ncdu \- for exploring a remote with a text based user interface.
|
||
.IP \[bu] 2
|
||
rclone lsjson \- for listing with a machine readable output
|
||
.IP \[bu] 2
|
||
rclone dbhashsum \- to show Dropbox style hashes of files (local or
|
||
Dropbox)
|
||
.IP \[bu] 2
|
||
New Features
|
||
.IP \[bu] 2
|
||
Implement \-\-fast\-list flag
|
||
.RS 2
|
||
.IP \[bu] 2
|
||
This allows remotes to list recursively if they can
|
||
.IP \[bu] 2
|
||
This uses less transactions (important if you pay for them)
|
||
.IP \[bu] 2
|
||
This may or may not be quicker
|
||
.IP \[bu] 2
|
||
This will use more memory as it has to hold the listing in memory
|
||
.IP \[bu] 2
|
||
\-\-old\-sync\-method deprecated \- the remaining uses are covered by
|
||
\-\-fast\-list
|
||
.IP \[bu] 2
|
||
This involved a major re\-write of all the listing code
|
||
.RE
|
||
.IP \[bu] 2
|
||
Add \-\-tpslimit and \-\-tpslimit\-burst to limit transactions per
|
||
second
|
||
.RS 2
|
||
.IP \[bu] 2
|
||
this is useful in conjuction with \f[C]rclone\ mount\f[] to limit
|
||
external apps
|
||
.RE
|
||
.IP \[bu] 2
|
||
Add \-\-stats\-log\-level so can see \-\-stats without \-v
|
||
.IP \[bu] 2
|
||
Print password prompts to stderr \- Hraban Luyat
|
||
.IP \[bu] 2
|
||
Warn about duplicate files when syncing
|
||
.IP \[bu] 2
|
||
Oauth improvements
|
||
.RS 2
|
||
.IP \[bu] 2
|
||
allow auth_url and token_url to be set in the config file
|
||
.IP \[bu] 2
|
||
Print redirection URI if using own credentials.
|
||
.RE
|
||
.IP \[bu] 2
|
||
Don\[aq]t Mkdir at the start of sync to save transactions
|
||
.IP \[bu] 2
|
||
Compile
|
||
.IP \[bu] 2
|
||
Update build to go1.8.3
|
||
.IP \[bu] 2
|
||
Require go1.6 for building rclone
|
||
.IP \[bu] 2
|
||
Compile 386 builds with "GO386=387" for maximum compatibility
|
||
.IP \[bu] 2
|
||
Bug Fixes
|
||
.IP \[bu] 2
|
||
Fix menu selection when no remotes
|
||
.IP \[bu] 2
|
||
Config saving reworked to not kill the file if disk gets full
|
||
.IP \[bu] 2
|
||
Don\[aq]t delete remote if name does not change while renaming
|
||
.IP \[bu] 2
|
||
moveto, copyto: report transfers and checks as per move and copy
|
||
.IP \[bu] 2
|
||
Local
|
||
.IP \[bu] 2
|
||
Add \-\-local\-no\-unicode\-normalization flag \- Bob Potter
|
||
.IP \[bu] 2
|
||
Mount
|
||
.IP \[bu] 2
|
||
Now supported on Windows using cgofuse and WinFsp \- thanks to Bill
|
||
Zissimopoulos for much help
|
||
.IP \[bu] 2
|
||
Compare checksums on upload/download via FUSE
|
||
.IP \[bu] 2
|
||
Unmount when program ends with SIGINT (Ctrl+C) or SIGTERM \- Jérôme
|
||
Vizcaino
|
||
.IP \[bu] 2
|
||
On read only open of file, make open pending until first read
|
||
.IP \[bu] 2
|
||
Make \-\-read\-only reject modify operations
|
||
.IP \[bu] 2
|
||
Implement ModTime via FUSE for remotes that support it
|
||
.IP \[bu] 2
|
||
Allow modTime to be changed even before all writers are closed
|
||
.IP \[bu] 2
|
||
Fix panic on renames
|
||
.IP \[bu] 2
|
||
Fix hang on errored upload
|
||
.IP \[bu] 2
|
||
Crypt
|
||
.IP \[bu] 2
|
||
Report the name:root as specified by the user
|
||
.IP \[bu] 2
|
||
Add an "obfuscate" option for filename encryption \- Stephen Harris
|
||
.IP \[bu] 2
|
||
Amazon Drive
|
||
.IP \[bu] 2
|
||
Fix initialization order for token renewer
|
||
.IP \[bu] 2
|
||
Remove revoked credentials, allow oauth proxy config and update docs
|
||
.IP \[bu] 2
|
||
B2
|
||
.IP \[bu] 2
|
||
Reduce minimum chunk size to 5MB
|
||
.IP \[bu] 2
|
||
Drive
|
||
.IP \[bu] 2
|
||
Add team drive support
|
||
.IP \[bu] 2
|
||
Reduce bandwidth by adding fields for partial responses \- Martin
|
||
Kristensen
|
||
.IP \[bu] 2
|
||
Implement \-\-drive\-shared\-with\-me flag to view shared with me files
|
||
\- Danny Tsai
|
||
.IP \[bu] 2
|
||
Add \-\-drive\-trashed\-only to read only the files in the trash
|
||
.IP \[bu] 2
|
||
Remove obsolete \-\-drive\-full\-list
|
||
.IP \[bu] 2
|
||
Add missing seek to start on retries of chunked uploads
|
||
.IP \[bu] 2
|
||
Fix stats accounting for upload
|
||
.IP \[bu] 2
|
||
Convert / in names to a unicode equivalent (/)
|
||
.IP \[bu] 2
|
||
Poll for Google Drive changes when mounted
|
||
.IP \[bu] 2
|
||
OneDrive
|
||
.IP \[bu] 2
|
||
Fix the uploading of files with spaces
|
||
.IP \[bu] 2
|
||
Fix initialization order for token renewer
|
||
.IP \[bu] 2
|
||
Display speeds accurately when uploading \- Yoni Jah
|
||
.IP \[bu] 2
|
||
Swap to using http://localhost:53682/ as redirect URL \- Michael Ledin
|
||
.IP \[bu] 2
|
||
Retry on token expired error, reset upload body on retry \- Yoni Jah
|
||
.IP \[bu] 2
|
||
Google Cloud Storage
|
||
.IP \[bu] 2
|
||
Add ability to specify location and storage class via config and command
|
||
line \- thanks gdm85
|
||
.IP \[bu] 2
|
||
Create container if necessary on server side copy
|
||
.IP \[bu] 2
|
||
Increase directory listing chunk to 1000 to increase performance
|
||
.IP \[bu] 2
|
||
Obtain a refresh token for GCS \- Steven Lu
|
||
.IP \[bu] 2
|
||
Yandex
|
||
.IP \[bu] 2
|
||
Fix the name reported in log messages (was empty)
|
||
.IP \[bu] 2
|
||
Correct error return for listing empty directory
|
||
.IP \[bu] 2
|
||
Dropbox
|
||
.IP \[bu] 2
|
||
Rewritten to use the v2 API
|
||
.RS 2
|
||
.IP \[bu] 2
|
||
Now supports ModTime
|
||
.IP \[bu] 2
|
||
Can only set by uploading the file again
|
||
.IP \[bu] 2
|
||
If you uploaded with an old rclone, rclone may upload everything again
|
||
.IP \[bu] 2
|
||
Use \f[C]\-\-size\-only\f[] or \f[C]\-\-checksum\f[] to avoid this
|
||
.IP \[bu] 2
|
||
Now supports the Dropbox content hashing scheme
|
||
.IP \[bu] 2
|
||
Now supports low level retries
|
||
.RE
|
||
.IP \[bu] 2
|
||
S3
|
||
.IP \[bu] 2
|
||
Work around eventual consistency in bucket creation
|
||
.IP \[bu] 2
|
||
Create container if necessary on server side copy
|
||
.IP \[bu] 2
|
||
Add us\-east\-2 (Ohio) and eu\-west\-2 (London) S3 regions \- Zahiar
|
||
Ahmed
|
||
.IP \[bu] 2
|
||
Swift, Hubic
|
||
.IP \[bu] 2
|
||
Fix zero length directory markers showing in the subdirectory listing
|
||
.RS 2
|
||
.IP \[bu] 2
|
||
this caused lots of duplicate transfers
|
||
.RE
|
||
.IP \[bu] 2
|
||
Fix paged directory listings
|
||
.RS 2
|
||
.IP \[bu] 2
|
||
this caused duplicate directory errors
|
||
.RE
|
||
.IP \[bu] 2
|
||
Create container if necessary on server side copy
|
||
.IP \[bu] 2
|
||
Increase directory listing chunk to 1000 to increase performance
|
||
.IP \[bu] 2
|
||
Make sensible error if the user forgets the container
|
||
.IP \[bu] 2
|
||
SFTP
|
||
.IP \[bu] 2
|
||
Add support for using ssh key files
|
||
.IP \[bu] 2
|
||
Fix under Windows
|
||
.IP \[bu] 2
|
||
Fix ssh agent on Windows
|
||
.IP \[bu] 2
|
||
Adapt to latest version of library \- Igor Kharin
|
||
.RE
|
||
.IP \[bu] 2
|
||
v1.36 \- 2017\-03\-18
|
||
.RS 2
|
||
.IP \[bu] 2
|
||
New Features
|
||
.IP \[bu] 2
|
||
SFTP remote (Jack Schmidt)
|
||
.IP \[bu] 2
|
||
Re\-implement sync routine to work a directory at a time reducing memory
|
||
usage
|
||
.IP \[bu] 2
|
||
Logging revamped to be more inline with rsync \- now much quieter
|
||
.RS 2
|
||
.IP \[bu] 2
|
||
\-v only shows transfers
|
||
.IP \[bu] 2
|
||
\-vv is for full debug
|
||
.IP \[bu] 2
|
||
\-\-syslog to log to syslog on capable platforms
|
||
.RE
|
||
.IP \[bu] 2
|
||
Implement \-\-backup\-dir and \-\-suffix
|
||
.IP \[bu] 2
|
||
Implement \-\-track\-renames (initial implementation by Bjørn Erik
|
||
Pedersen)
|
||
.IP \[bu] 2
|
||
Add time\-based bandwidth limits (Lukas Loesche)
|
||
.IP \[bu] 2
|
||
rclone cryptcheck: checks integrity of crypt remotes
|
||
.IP \[bu] 2
|
||
Allow all config file variables and options to be set from environment
|
||
variables
|
||
.IP \[bu] 2
|
||
Add \-\-buffer\-size parameter to control buffer size for copy
|
||
.IP \[bu] 2
|
||
Make \-\-delete\-after the default
|
||
.IP \[bu] 2
|
||
Add \-\-ignore\-checksum flag (fixed by Hisham Zarka)
|
||
.IP \[bu] 2
|
||
rclone check: Add \-\-download flag to check all the data, not just
|
||
hashes
|
||
.IP \[bu] 2
|
||
rclone cat: add \-\-head, \-\-tail, \-\-offset, \-\-count and
|
||
\-\-discard
|
||
.IP \[bu] 2
|
||
rclone config: when choosing from a list, allow the value to be entered
|
||
too
|
||
.IP \[bu] 2
|
||
rclone config: allow rename and copy of remotes
|
||
.IP \[bu] 2
|
||
rclone obscure: for generating encrypted passwords for rclone\[aq]s
|
||
config (T.C.
|
||
Ferguson)
|
||
.IP \[bu] 2
|
||
Comply with XDG Base Directory specification (Dario Giovannetti)
|
||
.RS 2
|
||
.IP \[bu] 2
|
||
this moves the default location of the config file in a backwards
|
||
compatible way
|
||
.RE
|
||
.IP \[bu] 2
|
||
Release changes
|
||
.RS 2
|
||
.IP \[bu] 2
|
||
Ubuntu snap support (Dedsec1)
|
||
.IP \[bu] 2
|
||
Compile with go 1.8
|
||
.IP \[bu] 2
|
||
MIPS/Linux big and little endian support
|
||
.RE
|
||
.IP \[bu] 2
|
||
Bug Fixes
|
||
.IP \[bu] 2
|
||
Fix copyto copying things to the wrong place if the destination dir
|
||
didn\[aq]t exist
|
||
.IP \[bu] 2
|
||
Fix parsing of remotes in moveto and copyto
|
||
.IP \[bu] 2
|
||
Fix \-\-delete\-before deleting files on copy
|
||
.IP \[bu] 2
|
||
Fix \-\-files\-from with an empty file copying everything
|
||
.IP \[bu] 2
|
||
Fix sync: don\[aq]t update mod times if \-\-dry\-run set
|
||
.IP \[bu] 2
|
||
Fix MimeType propagation
|
||
.IP \[bu] 2
|
||
Fix filters to add ** rules to directory rules
|
||
.IP \[bu] 2
|
||
Local
|
||
.IP \[bu] 2
|
||
Implement \-L, \-\-copy\-links flag to allow rclone to follow symlinks
|
||
.IP \[bu] 2
|
||
Open files in write only mode so rclone can write to an rclone mount
|
||
.IP \[bu] 2
|
||
Fix unnormalised unicode causing problems reading directories
|
||
.IP \[bu] 2
|
||
Fix interaction between \-x flag and \-\-max\-depth
|
||
.IP \[bu] 2
|
||
Mount
|
||
.IP \[bu] 2
|
||
Implement proper directory handling (mkdir, rmdir, renaming)
|
||
.IP \[bu] 2
|
||
Make include and exclude filters apply to mount
|
||
.IP \[bu] 2
|
||
Implement read and write async buffers \- control with \-\-buffer\-size
|
||
.IP \[bu] 2
|
||
Fix fsync on for directories
|
||
.IP \[bu] 2
|
||
Fix retry on network failure when reading off crypt
|
||
.IP \[bu] 2
|
||
Crypt
|
||
.IP \[bu] 2
|
||
Add \-\-crypt\-show\-mapping to show encrypted file mapping
|
||
.IP \[bu] 2
|
||
Fix crypt writer getting stuck in a loop
|
||
.RS 2
|
||
.IP \[bu] 2
|
||
\f[B]IMPORTANT\f[] this bug had the potential to cause data corruption
|
||
when
|
||
.IP \[bu] 2
|
||
reading data from a network based remote and
|
||
.IP \[bu] 2
|
||
writing to a crypt on Google Drive
|
||
.IP \[bu] 2
|
||
Use the cryptcheck command to validate your data if you are concerned
|
||
.IP \[bu] 2
|
||
If syncing two crypt remotes, sync the unencrypted remote
|
||
.RE
|
||
.IP \[bu] 2
|
||
Amazon Drive
|
||
.IP \[bu] 2
|
||
Fix panics on Move (rename)
|
||
.IP \[bu] 2
|
||
Fix panic on token expiry
|
||
.IP \[bu] 2
|
||
B2
|
||
.IP \[bu] 2
|
||
Fix inconsistent listings and rclone check
|
||
.IP \[bu] 2
|
||
Fix uploading empty files with go1.8
|
||
.IP \[bu] 2
|
||
Constrain memory usage when doing multipart uploads
|
||
.IP \[bu] 2
|
||
Fix upload url not being refreshed properly
|
||
.IP \[bu] 2
|
||
Drive
|
||
.IP \[bu] 2
|
||
Fix Rmdir on directories with trashed files
|
||
.IP \[bu] 2
|
||
Fix "Ignoring unknown object" when downloading
|
||
.IP \[bu] 2
|
||
Add \-\-drive\-list\-chunk
|
||
.IP \[bu] 2
|
||
Add \-\-drive\-skip\-gdocs (Károly Oláh)
|
||
.IP \[bu] 2
|
||
OneDrive
|
||
.IP \[bu] 2
|
||
Implement Move
|
||
.IP \[bu] 2
|
||
Fix Copy
|
||
.RS 2
|
||
.IP \[bu] 2
|
||
Fix overwrite detection in Copy
|
||
.IP \[bu] 2
|
||
Fix waitForJob to parse errors correctly
|
||
.RE
|
||
.IP \[bu] 2
|
||
Use token renewer to stop auth errors on long uploads
|
||
.IP \[bu] 2
|
||
Fix uploading empty files with go1.8
|
||
.IP \[bu] 2
|
||
Google Cloud Storage
|
||
.IP \[bu] 2
|
||
Fix depth 1 directory listings
|
||
.IP \[bu] 2
|
||
Yandex
|
||
.IP \[bu] 2
|
||
Fix single level directory listing
|
||
.IP \[bu] 2
|
||
Dropbox
|
||
.IP \[bu] 2
|
||
Normalise the case for single level directory listings
|
||
.IP \[bu] 2
|
||
Fix depth 1 listing
|
||
.IP \[bu] 2
|
||
S3
|
||
.IP \[bu] 2
|
||
Added ca\-central\-1 region (Jon Yergatian)
|
||
.RE
|
||
.IP \[bu] 2
|
||
v1.35 \- 2017\-01\-02
|
||
.RS 2
|
||
.IP \[bu] 2
|
||
New Features
|
||
.IP \[bu] 2
|
||
moveto and copyto commands for choosing a destination name on copy/move
|
||
.IP \[bu] 2
|
||
rmdirs command to recursively delete empty directories
|
||
.IP \[bu] 2
|
||
Allow repeated \-\-include/\-\-exclude/\-\-filter options
|
||
.IP \[bu] 2
|
||
Only show transfer stats on commands which transfer stuff
|
||
.RS 2
|
||
.IP \[bu] 2
|
||
show stats on any command using the \f[C]\-\-stats\f[] flag
|
||
.RE
|
||
.IP \[bu] 2
|
||
Allow overlapping directories in move when server side dir move is
|
||
supported
|
||
.IP \[bu] 2
|
||
Add \-\-stats\-unit option \- thanks Scott McGillivray
|
||
.IP \[bu] 2
|
||
Bug Fixes
|
||
.IP \[bu] 2
|
||
Fix the config file being overwritten when two rclones are running
|
||
.IP \[bu] 2
|
||
Make rclone lsd obey the filters properly
|
||
.IP \[bu] 2
|
||
Fix compilation on mips
|
||
.IP \[bu] 2
|
||
Fix not transferring files that don\[aq]t differ in size
|
||
.IP \[bu] 2
|
||
Fix panic on nil retry/fatal error
|
||
.IP \[bu] 2
|
||
Mount
|
||
.IP \[bu] 2
|
||
Retry reads on error \- should help with reliability a lot
|
||
.IP \[bu] 2
|
||
Report the modification times for directories from the remote
|
||
.IP \[bu] 2
|
||
Add bandwidth accounting and limiting (fixes \-\-bwlimit)
|
||
.IP \[bu] 2
|
||
If \-\-stats provided will show stats and which files are transferring
|
||
.IP \[bu] 2
|
||
Support R/W files if truncate is set.
|
||
.IP \[bu] 2
|
||
Implement statfs interface so df works
|
||
.IP \[bu] 2
|
||
Note that write is now supported on Amazon Drive
|
||
.IP \[bu] 2
|
||
Report number of blocks in a file \- thanks Stefan Breunig
|
||
.IP \[bu] 2
|
||
Crypt
|
||
.IP \[bu] 2
|
||
Prevent the user pointing crypt at itself
|
||
.IP \[bu] 2
|
||
Fix failed to authenticate decrypted block errors
|
||
.RS 2
|
||
.IP \[bu] 2
|
||
these will now return the underlying unexpected EOF instead
|
||
.RE
|
||
.IP \[bu] 2
|
||
Amazon Drive
|
||
.IP \[bu] 2
|
||
Add support for server side move and directory move \- thanks Stefan
|
||
Breunig
|
||
.IP \[bu] 2
|
||
Fix nil pointer deref on size attribute
|
||
.IP \[bu] 2
|
||
B2
|
||
.IP \[bu] 2
|
||
Use new prefix and delimiter parameters in directory listings
|
||
.RS 2
|
||
.IP \[bu] 2
|
||
This makes \-\-max\-depth 1 dir listings as used in mount much faster
|
||
.RE
|
||
.IP \[bu] 2
|
||
Reauth the account while doing uploads too \- should help with token
|
||
expiry
|
||
.IP \[bu] 2
|
||
Drive
|
||
.IP \[bu] 2
|
||
Make DirMove more efficient and complain about moving the root
|
||
.IP \[bu] 2
|
||
Create destination directory on Move()
|
||
.RE
|
||
.IP \[bu] 2
|
||
v1.34 \- 2016\-11\-06
|
||
.RS 2
|
||
.IP \[bu] 2
|
||
New Features
|
||
.IP \[bu] 2
|
||
Stop single file and \f[C]\-\-files\-from\f[] operations iterating
|
||
through the source bucket.
|
||
.IP \[bu] 2
|
||
Stop removing failed upload to cloud storage remotes
|
||
.IP \[bu] 2
|
||
Make ContentType be preserved for cloud to cloud copies
|
||
.IP \[bu] 2
|
||
Add support to toggle bandwidth limits via SIGUSR2 \- thanks Marco
|
||
Paganini
|
||
.IP \[bu] 2
|
||
\f[C]rclone\ check\f[] shows count of hashes that couldn\[aq]t be
|
||
checked
|
||
.IP \[bu] 2
|
||
\f[C]rclone\ listremotes\f[] command
|
||
.IP \[bu] 2
|
||
Support linux/arm64 build \- thanks Fredrik Fornwall
|
||
.IP \[bu] 2
|
||
Remove \f[C]Authorization:\f[] lines from \f[C]\-\-dump\-headers\f[]
|
||
output
|
||
.IP \[bu] 2
|
||
Bug Fixes
|
||
.IP \[bu] 2
|
||
Ignore files with control characters in the names
|
||
.IP \[bu] 2
|
||
Fix \f[C]rclone\ move\f[] command
|
||
.RS 2
|
||
.IP \[bu] 2
|
||
Delete src files which already existed in dst
|
||
.IP \[bu] 2
|
||
Fix deletion of src file when dst file older
|
||
.RE
|
||
.IP \[bu] 2
|
||
Fix \f[C]rclone\ check\f[] on crypted file systems
|
||
.IP \[bu] 2
|
||
Make failed uploads not count as "Transferred"
|
||
.IP \[bu] 2
|
||
Make sure high level retries show with \f[C]\-q\f[]
|
||
.IP \[bu] 2
|
||
Use a vendor directory with godep for repeatable builds
|
||
.IP \[bu] 2
|
||
\f[C]rclone\ mount\f[] \- FUSE
|
||
.IP \[bu] 2
|
||
Implement FUSE mount options
|
||
.RS 2
|
||
.IP \[bu] 2
|
||
\f[C]\-\-no\-modtime\f[], \f[C]\-\-debug\-fuse\f[],
|
||
\f[C]\-\-read\-only\f[], \f[C]\-\-allow\-non\-empty\f[],
|
||
\f[C]\-\-allow\-root\f[], \f[C]\-\-allow\-other\f[]
|
||
.IP \[bu] 2
|
||
\f[C]\-\-default\-permissions\f[], \f[C]\-\-write\-back\-cache\f[],
|
||
\f[C]\-\-max\-read\-ahead\f[], \f[C]\-\-umask\f[], \f[C]\-\-uid\f[],
|
||
\f[C]\-\-gid\f[]
|
||
.RE
|
||
.IP \[bu] 2
|
||
Add \f[C]\-\-dir\-cache\-time\f[] to control caching of directory
|
||
entries
|
||
.IP \[bu] 2
|
||
Implement seek for files opened for read (useful for video players)
|
||
.RS 2
|
||
.IP \[bu] 2
|
||
with \f[C]\-no\-seek\f[] flag to disable
|
||
.RE
|
||
.IP \[bu] 2
|
||
Fix crash on 32 bit ARM (alignment of 64 bit counter)
|
||
.IP \[bu] 2
|
||
\&...and many more internal fixes and improvements!
|
||
.IP \[bu] 2
|
||
Crypt
|
||
.IP \[bu] 2
|
||
Don\[aq]t show encrypted password in configurator to stop confusion
|
||
.IP \[bu] 2
|
||
Amazon Drive
|
||
.IP \[bu] 2
|
||
New wait for upload option \f[C]\-\-acd\-upload\-wait\-per\-gb\f[]
|
||
.RS 2
|
||
.IP \[bu] 2
|
||
upload timeouts scale by file size and can be disabled
|
||
.RE
|
||
.IP \[bu] 2
|
||
Add 502 Bad Gateway to list of errors we retry
|
||
.IP \[bu] 2
|
||
Fix overwriting a file with a zero length file
|
||
.IP \[bu] 2
|
||
Fix ACD file size warning limit \- thanks Felix Bünemann
|
||
.IP \[bu] 2
|
||
Local
|
||
.IP \[bu] 2
|
||
Unix: implement \f[C]\-x\f[]/\f[C]\-\-one\-file\-system\f[] to stay on a
|
||
single file system
|
||
.RS 2
|
||
.IP \[bu] 2
|
||
thanks Durval Menezes and Luiz Carlos Rumbelsperger Viana
|
||
.RE
|
||
.IP \[bu] 2
|
||
Windows: ignore the symlink bit on files
|
||
.IP \[bu] 2
|
||
Windows: Ignore directory based junction points
|
||
.IP \[bu] 2
|
||
B2
|
||
.IP \[bu] 2
|
||
Make sure each upload has at least one upload slot \- fixes strange
|
||
upload stats
|
||
.IP \[bu] 2
|
||
Fix uploads when using crypt
|
||
.IP \[bu] 2
|
||
Fix download of large files (sha1 mismatch)
|
||
.IP \[bu] 2
|
||
Return error when we try to create a bucket which someone else owns
|
||
.IP \[bu] 2
|
||
Update B2 docs with Data usage, and Crypt section \- thanks Tomasz Mazur
|
||
.IP \[bu] 2
|
||
S3
|
||
.IP \[bu] 2
|
||
Command line and config file support for
|
||
.RS 2
|
||
.IP \[bu] 2
|
||
Setting/overriding ACL \- thanks Radek Senfeld
|
||
.IP \[bu] 2
|
||
Setting storage class \- thanks Asko Tamm
|
||
.RE
|
||
.IP \[bu] 2
|
||
Drive
|
||
.IP \[bu] 2
|
||
Make exponential backoff work exactly as per Google specification
|
||
.IP \[bu] 2
|
||
add \f[C]\&.epub\f[], \f[C]\&.odp\f[] and \f[C]\&.tsv\f[] as export
|
||
formats.
|
||
.IP \[bu] 2
|
||
Swift
|
||
.IP \[bu] 2
|
||
Don\[aq]t read metadata for directory marker objects
|
||
.RE
|
||
.IP \[bu] 2
|
||
v1.33 \- 2016\-08\-24
|
||
.RS 2
|
||
.IP \[bu] 2
|
||
New Features
|
||
.IP \[bu] 2
|
||
Implement encryption
|
||
.RS 2
|
||
.IP \[bu] 2
|
||
data encrypted in NACL secretbox format
|
||
.IP \[bu] 2
|
||
with optional file name encryption
|
||
.RE
|
||
.IP \[bu] 2
|
||
New commands
|
||
.RS 2
|
||
.IP \[bu] 2
|
||
rclone mount \- implements FUSE mounting of remotes (EXPERIMENTAL)
|
||
.IP \[bu] 2
|
||
works on Linux, FreeBSD and OS X (need testers for the last 2!)
|
||
.IP \[bu] 2
|
||
rclone cat \- outputs remote file or files to the terminal
|
||
.IP \[bu] 2
|
||
rclone genautocomplete \- command to make a bash completion script for
|
||
rclone
|
||
.RE
|
||
.IP \[bu] 2
|
||
Editing a remote using \f[C]rclone\ config\f[] now goes through the
|
||
wizard
|
||
.IP \[bu] 2
|
||
Compile with go 1.7 \- this fixes rclone on macOS Sierra and on 386
|
||
processors
|
||
.IP \[bu] 2
|
||
Use cobra for sub commands and docs generation
|
||
.IP \[bu] 2
|
||
drive
|
||
.IP \[bu] 2
|
||
Document how to make your own client_id
|
||
.IP \[bu] 2
|
||
s3
|
||
.IP \[bu] 2
|
||
User\-configurable Amazon S3 ACL (thanks Radek Šenfeld)
|
||
.IP \[bu] 2
|
||
b2
|
||
.IP \[bu] 2
|
||
Fix stats accounting for upload \- no more jumping to 100% done
|
||
.IP \[bu] 2
|
||
On cleanup delete hide marker if it is the current file
|
||
.IP \[bu] 2
|
||
New B2 API endpoint (thanks Per Cederberg)
|
||
.IP \[bu] 2
|
||
Set maximum backoff to 5 Minutes
|
||
.IP \[bu] 2
|
||
onedrive
|
||
.IP \[bu] 2
|
||
Fix URL escaping in file names \- eg uploading files with \f[C]+\f[] in
|
||
them.
|
||
.IP \[bu] 2
|
||
amazon cloud drive
|
||
.IP \[bu] 2
|
||
Fix token expiry during large uploads
|
||
.IP \[bu] 2
|
||
Work around 408 REQUEST_TIMEOUT and 504 GATEWAY_TIMEOUT errors
|
||
.IP \[bu] 2
|
||
local
|
||
.IP \[bu] 2
|
||
Fix filenames with invalid UTF\-8 not being uploaded
|
||
.IP \[bu] 2
|
||
Fix problem with some UTF\-8 characters on OS X
|
||
.RE
|
||
.IP \[bu] 2
|
||
v1.32 \- 2016\-07\-13
|
||
.RS 2
|
||
.IP \[bu] 2
|
||
Backblaze B2
|
||
.IP \[bu] 2
|
||
Fix upload of files large files not in root
|
||
.RE
|
||
.IP \[bu] 2
|
||
v1.31 \- 2016\-07\-13
|
||
.RS 2
|
||
.IP \[bu] 2
|
||
New Features
|
||
.IP \[bu] 2
|
||
Reduce memory on sync by about 50%
|
||
.IP \[bu] 2
|
||
Implement \-\-no\-traverse flag to stop copy traversing the destination
|
||
remote.
|
||
.RS 2
|
||
.IP \[bu] 2
|
||
This can be used to reduce memory usage down to the smallest possible.
|
||
.IP \[bu] 2
|
||
Useful to copy a small number of files into a large destination folder.
|
||
.RE
|
||
.IP \[bu] 2
|
||
Implement cleanup command for emptying trash / removing old versions of
|
||
files
|
||
.RS 2
|
||
.IP \[bu] 2
|
||
Currently B2 only
|
||
.RE
|
||
.IP \[bu] 2
|
||
Single file handling improved
|
||
.RS 2
|
||
.IP \[bu] 2
|
||
Now copied with \-\-files\-from
|
||
.IP \[bu] 2
|
||
Automatically sets \-\-no\-traverse when copying a single file
|
||
.RE
|
||
.IP \[bu] 2
|
||
Info on using installing with ansible \- thanks Stefan Weichinger
|
||
.IP \[bu] 2
|
||
Implement \-\-no\-update\-modtime flag to stop rclone fixing the remote
|
||
modified times.
|
||
.IP \[bu] 2
|
||
Bug Fixes
|
||
.IP \[bu] 2
|
||
Fix move command \- stop it running for overlapping Fses \- this was
|
||
causing data loss.
|
||
.IP \[bu] 2
|
||
Local
|
||
.IP \[bu] 2
|
||
Fix incomplete hashes \- this was causing problems for B2.
|
||
.IP \[bu] 2
|
||
Amazon Drive
|
||
.IP \[bu] 2
|
||
Rename Amazon Cloud Drive to Amazon Drive \- no changes to config file
|
||
needed.
|
||
.IP \[bu] 2
|
||
Swift
|
||
.IP \[bu] 2
|
||
Add support for non\-default project domain \- thanks Antonio Messina.
|
||
.IP \[bu] 2
|
||
S3
|
||
.IP \[bu] 2
|
||
Add instructions on how to use rclone with minio.
|
||
.IP \[bu] 2
|
||
Add ap\-northeast\-2 (Seoul) and ap\-south\-1 (Mumbai) regions.
|
||
.IP \[bu] 2
|
||
Skip setting the modified time for objects > 5GB as it isn\[aq]t
|
||
possible.
|
||
.IP \[bu] 2
|
||
Backblaze B2
|
||
.IP \[bu] 2
|
||
Add \-\-b2\-versions flag so old versions can be listed and retreived.
|
||
.IP \[bu] 2
|
||
Treat 403 errors (eg cap exceeded) as fatal.
|
||
.IP \[bu] 2
|
||
Implement cleanup command for deleting old file versions.
|
||
.IP \[bu] 2
|
||
Make error handling compliant with B2 integrations notes.
|
||
.IP \[bu] 2
|
||
Fix handling of token expiry.
|
||
.IP \[bu] 2
|
||
Implement \-\-b2\-test\-mode to set \f[C]X\-Bz\-Test\-Mode\f[] header.
|
||
.IP \[bu] 2
|
||
Set cutoff for chunked upload to 200MB as per B2 guidelines.
|
||
.IP \[bu] 2
|
||
Make upload multi\-threaded.
|
||
.IP \[bu] 2
|
||
Dropbox
|
||
.IP \[bu] 2
|
||
Don\[aq]t retry 461 errors.
|
||
.RE
|
||
.IP \[bu] 2
|
||
v1.30 \- 2016\-06\-18
|
||
.RS 2
|
||
.IP \[bu] 2
|
||
New Features
|
||
.IP \[bu] 2
|
||
Directory listing code reworked for more features and better error
|
||
reporting (thanks to Klaus Post for help).
|
||
This enables
|
||
.RS 2
|
||
.IP \[bu] 2
|
||
Directory include filtering for efficiency
|
||
.IP \[bu] 2
|
||
\-\-max\-depth parameter
|
||
.IP \[bu] 2
|
||
Better error reporting
|
||
.IP \[bu] 2
|
||
More to come
|
||
.RE
|
||
.IP \[bu] 2
|
||
Retry more errors
|
||
.IP \[bu] 2
|
||
Add \-\-ignore\-size flag \- for uploading images to onedrive
|
||
.IP \[bu] 2
|
||
Log \-v output to stdout by default
|
||
.IP \[bu] 2
|
||
Display the transfer stats in more human readable form
|
||
.IP \[bu] 2
|
||
Make 0 size files specifiable with \f[C]\-\-max\-size\ 0b\f[]
|
||
.IP \[bu] 2
|
||
Add \f[C]b\f[] suffix so we can specify bytes in \-\-bwlimit,
|
||
\-\-min\-size etc
|
||
.IP \[bu] 2
|
||
Use "password:" instead of "password>" prompt \- thanks Klaus Post and
|
||
Leigh Klotz
|
||
.IP \[bu] 2
|
||
Bug Fixes
|
||
.IP \[bu] 2
|
||
Fix retry doing one too many retries
|
||
.IP \[bu] 2
|
||
Local
|
||
.IP \[bu] 2
|
||
Fix problems with OS X and UTF\-8 characters
|
||
.IP \[bu] 2
|
||
Amazon Drive
|
||
.IP \[bu] 2
|
||
Check a file exists before uploading to help with 408 Conflict errors
|
||
.IP \[bu] 2
|
||
Reauth on 401 errors \- this has been causing a lot of problems
|
||
.IP \[bu] 2
|
||
Work around spurious 403 errors
|
||
.IP \[bu] 2
|
||
Restart directory listings on error
|
||
.IP \[bu] 2
|
||
Google Drive
|
||
.IP \[bu] 2
|
||
Check a file exists before uploading to help with duplicates
|
||
.IP \[bu] 2
|
||
Fix retry of multipart uploads
|
||
.IP \[bu] 2
|
||
Backblaze B2
|
||
.IP \[bu] 2
|
||
Implement large file uploading
|
||
.IP \[bu] 2
|
||
S3
|
||
.IP \[bu] 2
|
||
Add AES256 server\-side encryption for \- thanks Justin R.
|
||
Wilson
|
||
.IP \[bu] 2
|
||
Google Cloud Storage
|
||
.IP \[bu] 2
|
||
Make sure we don\[aq]t use conflicting content types on upload
|
||
.IP \[bu] 2
|
||
Add service account support \- thanks Michal Witkowski
|
||
.IP \[bu] 2
|
||
Swift
|
||
.IP \[bu] 2
|
||
Add auth version parameter
|
||
.IP \[bu] 2
|
||
Add domain option for openstack (v3 auth) \- thanks Fabian Ruff
|
||
.RE
|
||
.IP \[bu] 2
|
||
v1.29 \- 2016\-04\-18
|
||
.RS 2
|
||
.IP \[bu] 2
|
||
New Features
|
||
.IP \[bu] 2
|
||
Implement \f[C]\-I,\ \-\-ignore\-times\f[] for unconditional upload
|
||
.IP \[bu] 2
|
||
Improve \f[C]dedupe\f[]command
|
||
.RS 2
|
||
.IP \[bu] 2
|
||
Now removes identical copies without asking
|
||
.IP \[bu] 2
|
||
Now obeys \f[C]\-\-dry\-run\f[]
|
||
.IP \[bu] 2
|
||
Implement \f[C]\-\-dedupe\-mode\f[] for non interactive running
|
||
.IP \[bu] 2
|
||
\f[C]\-\-dedupe\-mode\ interactive\f[] \- interactive the default.
|
||
.IP \[bu] 2
|
||
\f[C]\-\-dedupe\-mode\ skip\f[] \- removes identical files then skips
|
||
anything left.
|
||
.IP \[bu] 2
|
||
\f[C]\-\-dedupe\-mode\ first\f[] \- removes identical files then keeps
|
||
the first one.
|
||
.IP \[bu] 2
|
||
\f[C]\-\-dedupe\-mode\ newest\f[] \- removes identical files then keeps
|
||
the newest one.
|
||
.IP \[bu] 2
|
||
\f[C]\-\-dedupe\-mode\ oldest\f[] \- removes identical files then keeps
|
||
the oldest one.
|
||
.IP \[bu] 2
|
||
\f[C]\-\-dedupe\-mode\ rename\f[] \- removes identical files then
|
||
renames the rest to be different.
|
||
.RE
|
||
.IP \[bu] 2
|
||
Bug fixes
|
||
.IP \[bu] 2
|
||
Make rclone check obey the \f[C]\-\-size\-only\f[] flag.
|
||
.IP \[bu] 2
|
||
Use "application/octet\-stream" if discovered mime type is invalid.
|
||
.IP \[bu] 2
|
||
Fix missing "quit" option when there are no remotes.
|
||
.IP \[bu] 2
|
||
Google Drive
|
||
.IP \[bu] 2
|
||
Increase default chunk size to 8 MB \- increases upload speed of big
|
||
files
|
||
.IP \[bu] 2
|
||
Speed up directory listings and make more reliable
|
||
.IP \[bu] 2
|
||
Add missing retries for Move and DirMove \- increases reliability
|
||
.IP \[bu] 2
|
||
Preserve mime type on file update
|
||
.IP \[bu] 2
|
||
Backblaze B2
|
||
.IP \[bu] 2
|
||
Enable mod time syncing
|
||
.RS 2
|
||
.IP \[bu] 2
|
||
This means that B2 will now check modification times
|
||
.IP \[bu] 2
|
||
It will upload new files to update the modification times
|
||
.IP \[bu] 2
|
||
(there isn\[aq]t an API to just set the mod time.)
|
||
.IP \[bu] 2
|
||
If you want the old behaviour use \f[C]\-\-size\-only\f[].
|
||
.RE
|
||
.IP \[bu] 2
|
||
Update API to new version
|
||
.IP \[bu] 2
|
||
Fix parsing of mod time when not in metadata
|
||
.IP \[bu] 2
|
||
Swift/Hubic
|
||
.IP \[bu] 2
|
||
Don\[aq]t return an MD5SUM for static large objects
|
||
.IP \[bu] 2
|
||
S3
|
||
.IP \[bu] 2
|
||
Fix uploading files bigger than 50GB
|
||
.RE
|
||
.IP \[bu] 2
|
||
v1.28 \- 2016\-03\-01
|
||
.RS 2
|
||
.IP \[bu] 2
|
||
New Features
|
||
.IP \[bu] 2
|
||
Configuration file encryption \- thanks Klaus Post
|
||
.IP \[bu] 2
|
||
Improve \f[C]rclone\ config\f[] adding more help and making it easier to
|
||
understand
|
||
.IP \[bu] 2
|
||
Implement \f[C]\-u\f[]/\f[C]\-\-update\f[] so creation times can be used
|
||
on all remotes
|
||
.IP \[bu] 2
|
||
Implement \f[C]\-\-low\-level\-retries\f[] flag
|
||
.IP \[bu] 2
|
||
Optionally disable gzip compression on downloads with
|
||
\f[C]\-\-no\-gzip\-encoding\f[]
|
||
.IP \[bu] 2
|
||
Bug fixes
|
||
.IP \[bu] 2
|
||
Don\[aq]t make directories if \f[C]\-\-dry\-run\f[] set
|
||
.IP \[bu] 2
|
||
Fix and document the \f[C]move\f[] command
|
||
.IP \[bu] 2
|
||
Fix redirecting stderr on unix\-like OSes when using
|
||
\f[C]\-\-log\-file\f[]
|
||
.IP \[bu] 2
|
||
Fix \f[C]delete\f[] command to wait until all finished \- fixes missing
|
||
deletes.
|
||
.IP \[bu] 2
|
||
Backblaze B2
|
||
.IP \[bu] 2
|
||
Use one upload URL per go routine fixes
|
||
\f[C]more\ than\ one\ upload\ using\ auth\ token\f[]
|
||
.IP \[bu] 2
|
||
Add pacing, retries and reauthentication \- fixes token expiry problems
|
||
.IP \[bu] 2
|
||
Upload without using a temporary file from local (and remotes which
|
||
support SHA1)
|
||
.IP \[bu] 2
|
||
Fix reading metadata for all files when it shouldn\[aq]t have been
|
||
.IP \[bu] 2
|
||
Drive
|
||
.IP \[bu] 2
|
||
Fix listing drive documents at root
|
||
.IP \[bu] 2
|
||
Disable copy and move for Google docs
|
||
.IP \[bu] 2
|
||
Swift
|
||
.IP \[bu] 2
|
||
Fix uploading of chunked files with non ASCII characters
|
||
.IP \[bu] 2
|
||
Allow setting of \f[C]storage_url\f[] in the config \- thanks Xavier
|
||
Lucas
|
||
.IP \[bu] 2
|
||
S3
|
||
.IP \[bu] 2
|
||
Allow IAM role and credentials from environment variables \- thanks
|
||
Brian Stengaard
|
||
.IP \[bu] 2
|
||
Allow low privilege users to use S3 (check if directory exists during
|
||
Mkdir) \- thanks Jakub Gedeon
|
||
.IP \[bu] 2
|
||
Amazon Drive
|
||
.IP \[bu] 2
|
||
Retry on more things to make directory listings more reliable
|
||
.RE
|
||
.IP \[bu] 2
|
||
v1.27 \- 2016\-01\-31
|
||
.RS 2
|
||
.IP \[bu] 2
|
||
New Features
|
||
.IP \[bu] 2
|
||
Easier headless configuration with \f[C]rclone\ authorize\f[]
|
||
.IP \[bu] 2
|
||
Add support for multiple hash types \- we now check SHA1 as well as MD5
|
||
hashes.
|
||
.IP \[bu] 2
|
||
\f[C]delete\f[] command which does obey the filters (unlike
|
||
\f[C]purge\f[])
|
||
.IP \[bu] 2
|
||
\f[C]dedupe\f[] command to deduplicate a remote.
|
||
Useful with Google Drive.
|
||
.IP \[bu] 2
|
||
Add \f[C]\-\-ignore\-existing\f[] flag to skip all files that exist on
|
||
destination.
|
||
.IP \[bu] 2
|
||
Add \f[C]\-\-delete\-before\f[], \f[C]\-\-delete\-during\f[],
|
||
\f[C]\-\-delete\-after\f[] flags.
|
||
.IP \[bu] 2
|
||
Add \f[C]\-\-memprofile\f[] flag to debug memory use.
|
||
.IP \[bu] 2
|
||
Warn the user about files with same name but different case
|
||
.IP \[bu] 2
|
||
Make \f[C]\-\-include\f[] rules add their implict exclude * at the end
|
||
of the filter list
|
||
.IP \[bu] 2
|
||
Deprecate compiling with go1.3
|
||
.IP \[bu] 2
|
||
Amazon Drive
|
||
.IP \[bu] 2
|
||
Fix download of files > 10 GB
|
||
.IP \[bu] 2
|
||
Fix directory traversal ("Next token is expired") for large directory
|
||
listings
|
||
.IP \[bu] 2
|
||
Remove 409 conflict from error codes we will retry \- stops very long
|
||
pauses
|
||
.IP \[bu] 2
|
||
Backblaze B2
|
||
.IP \[bu] 2
|
||
SHA1 hashes now checked by rclone core
|
||
.IP \[bu] 2
|
||
Drive
|
||
.IP \[bu] 2
|
||
Add \f[C]\-\-drive\-auth\-owner\-only\f[] to only consider files owned
|
||
by the user \- thanks Björn Harrtell
|
||
.IP \[bu] 2
|
||
Export Google documents
|
||
.IP \[bu] 2
|
||
Dropbox
|
||
.IP \[bu] 2
|
||
Make file exclusion error controllable with \-q
|
||
.IP \[bu] 2
|
||
Swift
|
||
.IP \[bu] 2
|
||
Fix upload from unprivileged user.
|
||
.IP \[bu] 2
|
||
S3
|
||
.IP \[bu] 2
|
||
Fix updating of mod times of files with \f[C]+\f[] in.
|
||
.IP \[bu] 2
|
||
Local
|
||
.IP \[bu] 2
|
||
Add local file system option to disable UNC on Windows.
|
||
.RE
|
||
.IP \[bu] 2
|
||
v1.26 \- 2016\-01\-02
|
||
.RS 2
|
||
.IP \[bu] 2
|
||
New Features
|
||
.IP \[bu] 2
|
||
Yandex storage backend \- thank you Dmitry Burdeev ("dibu")
|
||
.IP \[bu] 2
|
||
Implement Backblaze B2 storage backend
|
||
.IP \[bu] 2
|
||
Add \-\-min\-age and \-\-max\-age flags \- thank you Adriano Aurélio
|
||
Meirelles
|
||
.IP \[bu] 2
|
||
Make ls/lsl/md5sum/size/check obey includes and excludes
|
||
.IP \[bu] 2
|
||
Fixes
|
||
.IP \[bu] 2
|
||
Fix crash in http logging
|
||
.IP \[bu] 2
|
||
Upload releases to github too
|
||
.IP \[bu] 2
|
||
Swift
|
||
.IP \[bu] 2
|
||
Fix sync for chunked files
|
||
.IP \[bu] 2
|
||
OneDrive
|
||
.IP \[bu] 2
|
||
Re\-enable server side copy
|
||
.IP \[bu] 2
|
||
Don\[aq]t mask HTTP error codes with JSON decode error
|
||
.IP \[bu] 2
|
||
S3
|
||
.IP \[bu] 2
|
||
Fix corrupting Content\-Type on mod time update (thanks Joseph Spurrier)
|
||
.RE
|
||
.IP \[bu] 2
|
||
v1.25 \- 2015\-11\-14
|
||
.RS 2
|
||
.IP \[bu] 2
|
||
New features
|
||
.IP \[bu] 2
|
||
Implement Hubic storage system
|
||
.IP \[bu] 2
|
||
Fixes
|
||
.IP \[bu] 2
|
||
Fix deletion of some excluded files without \-\-delete\-excluded
|
||
.RS 2
|
||
.IP \[bu] 2
|
||
This could have deleted files unexpectedly on sync
|
||
.IP \[bu] 2
|
||
Always check first with \f[C]\-\-dry\-run\f[]!
|
||
.RE
|
||
.IP \[bu] 2
|
||
Swift
|
||
.IP \[bu] 2
|
||
Stop SetModTime losing metadata (eg X\-Object\-Manifest)
|
||
.RS 2
|
||
.IP \[bu] 2
|
||
This could have caused data loss for files > 5GB in size
|
||
.RE
|
||
.IP \[bu] 2
|
||
Use ContentType from Object to avoid lookups in listings
|
||
.IP \[bu] 2
|
||
OneDrive
|
||
.IP \[bu] 2
|
||
disable server side copy as it seems to be broken at Microsoft
|
||
.RE
|
||
.IP \[bu] 2
|
||
v1.24 \- 2015\-11\-07
|
||
.RS 2
|
||
.IP \[bu] 2
|
||
New features
|
||
.IP \[bu] 2
|
||
Add support for Microsoft OneDrive
|
||
.IP \[bu] 2
|
||
Add \f[C]\-\-no\-check\-certificate\f[] option to disable server
|
||
certificate verification
|
||
.IP \[bu] 2
|
||
Add async readahead buffer for faster transfer of big files
|
||
.IP \[bu] 2
|
||
Fixes
|
||
.IP \[bu] 2
|
||
Allow spaces in remotes and check remote names for validity at creation
|
||
time
|
||
.IP \[bu] 2
|
||
Allow \[aq]&\[aq] and disallow \[aq]:\[aq] in Windows filenames.
|
||
.IP \[bu] 2
|
||
Swift
|
||
.IP \[bu] 2
|
||
Ignore directory marker objects where appropriate \- allows working with
|
||
Hubic
|
||
.IP \[bu] 2
|
||
Don\[aq]t delete the container if fs wasn\[aq]t at root
|
||
.IP \[bu] 2
|
||
S3
|
||
.IP \[bu] 2
|
||
Don\[aq]t delete the bucket if fs wasn\[aq]t at root
|
||
.IP \[bu] 2
|
||
Google Cloud Storage
|
||
.IP \[bu] 2
|
||
Don\[aq]t delete the bucket if fs wasn\[aq]t at root
|
||
.RE
|
||
.IP \[bu] 2
|
||
v1.23 \- 2015\-10\-03
|
||
.RS 2
|
||
.IP \[bu] 2
|
||
New features
|
||
.IP \[bu] 2
|
||
Implement \f[C]rclone\ size\f[] for measuring remotes
|
||
.IP \[bu] 2
|
||
Fixes
|
||
.IP \[bu] 2
|
||
Fix headless config for drive and gcs
|
||
.IP \[bu] 2
|
||
Tell the user they should try again if the webserver method failed
|
||
.IP \[bu] 2
|
||
Improve output of \f[C]\-\-dump\-headers\f[]
|
||
.IP \[bu] 2
|
||
S3
|
||
.IP \[bu] 2
|
||
Allow anonymous access to public buckets
|
||
.IP \[bu] 2
|
||
Swift
|
||
.IP \[bu] 2
|
||
Stop chunked operations logging "Failed to read info: Object Not Found"
|
||
.IP \[bu] 2
|
||
Use Content\-Length on uploads for extra reliability
|
||
.RE
|
||
.IP \[bu] 2
|
||
v1.22 \- 2015\-09\-28
|
||
.RS 2
|
||
.IP \[bu] 2
|
||
Implement rsync like include and exclude flags
|
||
.IP \[bu] 2
|
||
swift
|
||
.IP \[bu] 2
|
||
Support files > 5GB \- thanks Sergey Tolmachev
|
||
.RE
|
||
.IP \[bu] 2
|
||
v1.21 \- 2015\-09\-22
|
||
.RS 2
|
||
.IP \[bu] 2
|
||
New features
|
||
.IP \[bu] 2
|
||
Display individual transfer progress
|
||
.IP \[bu] 2
|
||
Make lsl output times in localtime
|
||
.IP \[bu] 2
|
||
Fixes
|
||
.IP \[bu] 2
|
||
Fix allowing user to override credentials again in Drive, GCS and ACD
|
||
.IP \[bu] 2
|
||
Amazon Drive
|
||
.IP \[bu] 2
|
||
Implement compliant pacing scheme
|
||
.IP \[bu] 2
|
||
Google Drive
|
||
.IP \[bu] 2
|
||
Make directory reads concurrent for increased speed.
|
||
.RE
|
||
.IP \[bu] 2
|
||
v1.20 \- 2015\-09\-15
|
||
.RS 2
|
||
.IP \[bu] 2
|
||
New features
|
||
.IP \[bu] 2
|
||
Amazon Drive support
|
||
.IP \[bu] 2
|
||
Oauth support redone \- fix many bugs and improve usability
|
||
.RS 2
|
||
.IP \[bu] 2
|
||
Use "golang.org/x/oauth2" as oauth libary of choice
|
||
.IP \[bu] 2
|
||
Improve oauth usability for smoother initial signup
|
||
.IP \[bu] 2
|
||
drive, googlecloudstorage: optionally use auto config for the oauth
|
||
token
|
||
.RE
|
||
.IP \[bu] 2
|
||
Implement \-\-dump\-headers and \-\-dump\-bodies debug flags
|
||
.IP \[bu] 2
|
||
Show multiple matched commands if abbreviation too short
|
||
.IP \[bu] 2
|
||
Implement server side move where possible
|
||
.IP \[bu] 2
|
||
local
|
||
.IP \[bu] 2
|
||
Always use UNC paths internally on Windows \- fixes a lot of bugs
|
||
.IP \[bu] 2
|
||
dropbox
|
||
.IP \[bu] 2
|
||
force use of our custom transport which makes timeouts work
|
||
.IP \[bu] 2
|
||
Thanks to Klaus Post for lots of help with this release
|
||
.RE
|
||
.IP \[bu] 2
|
||
v1.19 \- 2015\-08\-28
|
||
.RS 2
|
||
.IP \[bu] 2
|
||
New features
|
||
.IP \[bu] 2
|
||
Server side copies for s3/swift/drive/dropbox/gcs
|
||
.IP \[bu] 2
|
||
Move command \- uses server side copies if it can
|
||
.IP \[bu] 2
|
||
Implement \-\-retries flag \- tries 3 times by default
|
||
.IP \[bu] 2
|
||
Build for plan9/amd64 and solaris/amd64 too
|
||
.IP \[bu] 2
|
||
Fixes
|
||
.IP \[bu] 2
|
||
Make a current version download with a fixed URL for scripting
|
||
.IP \[bu] 2
|
||
Ignore rmdir in limited fs rather than throwing error
|
||
.IP \[bu] 2
|
||
dropbox
|
||
.IP \[bu] 2
|
||
Increase chunk size to improve upload speeds massively
|
||
.IP \[bu] 2
|
||
Issue an error message when trying to upload bad file name
|
||
.RE
|
||
.IP \[bu] 2
|
||
v1.18 \- 2015\-08\-17
|
||
.RS 2
|
||
.IP \[bu] 2
|
||
drive
|
||
.IP \[bu] 2
|
||
Add \f[C]\-\-drive\-use\-trash\f[] flag so rclone trashes instead of
|
||
deletes
|
||
.IP \[bu] 2
|
||
Add "Forbidden to download" message for files with no downloadURL
|
||
.IP \[bu] 2
|
||
dropbox
|
||
.IP \[bu] 2
|
||
Remove datastore
|
||
.RS 2
|
||
.IP \[bu] 2
|
||
This was deprecated and it caused a lot of problems
|
||
.IP \[bu] 2
|
||
Modification times and MD5SUMs no longer stored
|
||
.RE
|
||
.IP \[bu] 2
|
||
Fix uploading files > 2GB
|
||
.IP \[bu] 2
|
||
s3
|
||
.IP \[bu] 2
|
||
use official AWS SDK from github.com/aws/aws\-sdk\-go
|
||
.IP \[bu] 2
|
||
\f[B]NB\f[] will most likely require you to delete and recreate remote
|
||
.IP \[bu] 2
|
||
enable multipart upload which enables files > 5GB
|
||
.IP \[bu] 2
|
||
tested with Ceph / RadosGW / S3 emulation
|
||
.IP \[bu] 2
|
||
many thanks to Sam Liston and Brian Haymore at the Utah Center for High
|
||
Performance Computing (https://www.chpc.utah.edu/) for a Ceph test
|
||
account
|
||
.IP \[bu] 2
|
||
misc
|
||
.IP \[bu] 2
|
||
Show errors when reading the config file
|
||
.IP \[bu] 2
|
||
Do not print stats in quiet mode \- thanks Leonid Shalupov
|
||
.IP \[bu] 2
|
||
Add FAQ
|
||
.IP \[bu] 2
|
||
Fix created directories not obeying umask
|
||
.IP \[bu] 2
|
||
Linux installation instructions \- thanks Shimon Doodkin
|
||
.RE
|
||
.IP \[bu] 2
|
||
v1.17 \- 2015\-06\-14
|
||
.RS 2
|
||
.IP \[bu] 2
|
||
dropbox: fix case insensitivity issues \- thanks Leonid Shalupov
|
||
.RE
|
||
.IP \[bu] 2
|
||
v1.16 \- 2015\-06\-09
|
||
.RS 2
|
||
.IP \[bu] 2
|
||
Fix uploading big files which was causing timeouts or panics
|
||
.IP \[bu] 2
|
||
Don\[aq]t check md5sum after download with \-\-size\-only
|
||
.RE
|
||
.IP \[bu] 2
|
||
v1.15 \- 2015\-06\-06
|
||
.RS 2
|
||
.IP \[bu] 2
|
||
Add \-\-checksum flag to only discard transfers by MD5SUM \- thanks Alex
|
||
Couper
|
||
.IP \[bu] 2
|
||
Implement \-\-size\-only flag to sync on size not checksum & modtime
|
||
.IP \[bu] 2
|
||
Expand docs and remove duplicated information
|
||
.IP \[bu] 2
|
||
Document rclone\[aq]s limitations with directories
|
||
.IP \[bu] 2
|
||
dropbox: update docs about case insensitivity
|
||
.RE
|
||
.IP \[bu] 2
|
||
v1.14 \- 2015\-05\-21
|
||
.RS 2
|
||
.IP \[bu] 2
|
||
local: fix encoding of non utf\-8 file names \- fixes a duplicate file
|
||
problem
|
||
.IP \[bu] 2
|
||
drive: docs about rate limiting
|
||
.IP \[bu] 2
|
||
google cloud storage: Fix compile after API change in
|
||
"google.golang.org/api/storage/v1"
|
||
.RE
|
||
.IP \[bu] 2
|
||
v1.13 \- 2015\-05\-10
|
||
.RS 2
|
||
.IP \[bu] 2
|
||
Revise documentation (especially sync)
|
||
.IP \[bu] 2
|
||
Implement \-\-timeout and \-\-conntimeout
|
||
.IP \[bu] 2
|
||
s3: ignore etags from multipart uploads which aren\[aq]t md5sums
|
||
.RE
|
||
.IP \[bu] 2
|
||
v1.12 \- 2015\-03\-15
|
||
.RS 2
|
||
.IP \[bu] 2
|
||
drive: Use chunked upload for files above a certain size
|
||
.IP \[bu] 2
|
||
drive: add \-\-drive\-chunk\-size and \-\-drive\-upload\-cutoff
|
||
parameters
|
||
.IP \[bu] 2
|
||
drive: switch to insert from update when a failed copy deletes the
|
||
upload
|
||
.IP \[bu] 2
|
||
core: Log duplicate files if they are detected
|
||
.RE
|
||
.IP \[bu] 2
|
||
v1.11 \- 2015\-03\-04
|
||
.RS 2
|
||
.IP \[bu] 2
|
||
swift: add region parameter
|
||
.IP \[bu] 2
|
||
drive: fix crash on failed to update remote mtime
|
||
.IP \[bu] 2
|
||
In remote paths, change native directory separators to /
|
||
.IP \[bu] 2
|
||
Add synchronization to ls/lsl/lsd output to stop corruptions
|
||
.IP \[bu] 2
|
||
Ensure all stats/log messages to go stderr
|
||
.IP \[bu] 2
|
||
Add \-\-log\-file flag to log everything (including panics) to file
|
||
.IP \[bu] 2
|
||
Make it possible to disable stats printing with \-\-stats=0
|
||
.IP \[bu] 2
|
||
Implement \-\-bwlimit to limit data transfer bandwidth
|
||
.RE
|
||
.IP \[bu] 2
|
||
v1.10 \- 2015\-02\-12
|
||
.RS 2
|
||
.IP \[bu] 2
|
||
s3: list an unlimited number of items
|
||
.IP \[bu] 2
|
||
Fix getting stuck in the configurator
|
||
.RE
|
||
.IP \[bu] 2
|
||
v1.09 \- 2015\-02\-07
|
||
.RS 2
|
||
.IP \[bu] 2
|
||
windows: Stop drive letters (eg C:) getting mixed up with remotes (eg
|
||
drive:)
|
||
.IP \[bu] 2
|
||
local: Fix directory separators on Windows
|
||
.IP \[bu] 2
|
||
drive: fix rate limit exceeded errors
|
||
.RE
|
||
.IP \[bu] 2
|
||
v1.08 \- 2015\-02\-04
|
||
.RS 2
|
||
.IP \[bu] 2
|
||
drive: fix subdirectory listing to not list entire drive
|
||
.IP \[bu] 2
|
||
drive: Fix SetModTime
|
||
.IP \[bu] 2
|
||
dropbox: adapt code to recent library changes
|
||
.RE
|
||
.IP \[bu] 2
|
||
v1.07 \- 2014\-12\-23
|
||
.RS 2
|
||
.IP \[bu] 2
|
||
google cloud storage: fix memory leak
|
||
.RE
|
||
.IP \[bu] 2
|
||
v1.06 \- 2014\-12\-12
|
||
.RS 2
|
||
.IP \[bu] 2
|
||
Fix "Couldn\[aq]t find home directory" on OSX
|
||
.IP \[bu] 2
|
||
swift: Add tenant parameter
|
||
.IP \[bu] 2
|
||
Use new location of Google API packages
|
||
.RE
|
||
.IP \[bu] 2
|
||
v1.05 \- 2014\-08\-09
|
||
.RS 2
|
||
.IP \[bu] 2
|
||
Improved tests and consequently lots of minor fixes
|
||
.IP \[bu] 2
|
||
core: Fix race detected by go race detector
|
||
.IP \[bu] 2
|
||
core: Fixes after running errcheck
|
||
.IP \[bu] 2
|
||
drive: reset root directory on Rmdir and Purge
|
||
.IP \[bu] 2
|
||
fs: Document that Purger returns error on empty directory, test and fix
|
||
.IP \[bu] 2
|
||
google cloud storage: fix ListDir on subdirectory
|
||
.IP \[bu] 2
|
||
google cloud storage: re\-read metadata in SetModTime
|
||
.IP \[bu] 2
|
||
s3: make reading metadata more reliable to work around eventual
|
||
consistency problems
|
||
.IP \[bu] 2
|
||
s3: strip trailing / from ListDir()
|
||
.IP \[bu] 2
|
||
swift: return directories without / in ListDir
|
||
.RE
|
||
.IP \[bu] 2
|
||
v1.04 \- 2014\-07\-21
|
||
.RS 2
|
||
.IP \[bu] 2
|
||
google cloud storage: Fix crash on Update
|
||
.RE
|
||
.IP \[bu] 2
|
||
v1.03 \- 2014\-07\-20
|
||
.RS 2
|
||
.IP \[bu] 2
|
||
swift, s3, dropbox: fix updated files being marked as corrupted
|
||
.IP \[bu] 2
|
||
Make compile with go 1.1 again
|
||
.RE
|
||
.IP \[bu] 2
|
||
v1.02 \- 2014\-07\-19
|
||
.RS 2
|
||
.IP \[bu] 2
|
||
Implement Dropbox remote
|
||
.IP \[bu] 2
|
||
Implement Google Cloud Storage remote
|
||
.IP \[bu] 2
|
||
Verify Md5sums and Sizes after copies
|
||
.IP \[bu] 2
|
||
Remove times from "ls" command \- lists sizes only
|
||
.IP \[bu] 2
|
||
Add add "lsl" \- lists times and sizes
|
||
.IP \[bu] 2
|
||
Add "md5sum" command
|
||
.RE
|
||
.IP \[bu] 2
|
||
v1.01 \- 2014\-07\-04
|
||
.RS 2
|
||
.IP \[bu] 2
|
||
drive: fix transfer of big files using up lots of memory
|
||
.RE
|
||
.IP \[bu] 2
|
||
v1.00 \- 2014\-07\-03
|
||
.RS 2
|
||
.IP \[bu] 2
|
||
drive: fix whole second dates
|
||
.RE
|
||
.IP \[bu] 2
|
||
v0.99 \- 2014\-06\-26
|
||
.RS 2
|
||
.IP \[bu] 2
|
||
Fix \-\-dry\-run not working
|
||
.IP \[bu] 2
|
||
Make compatible with go 1.1
|
||
.RE
|
||
.IP \[bu] 2
|
||
v0.98 \- 2014\-05\-30
|
||
.RS 2
|
||
.IP \[bu] 2
|
||
s3: Treat missing Content\-Length as 0 for some ceph installations
|
||
.IP \[bu] 2
|
||
rclonetest: add file with a space in
|
||
.RE
|
||
.IP \[bu] 2
|
||
v0.97 \- 2014\-05\-05
|
||
.RS 2
|
||
.IP \[bu] 2
|
||
Implement copying of single files
|
||
.IP \[bu] 2
|
||
s3 & swift: support paths inside containers/buckets
|
||
.RE
|
||
.IP \[bu] 2
|
||
v0.96 \- 2014\-04\-24
|
||
.RS 2
|
||
.IP \[bu] 2
|
||
drive: Fix multiple files of same name being created
|
||
.IP \[bu] 2
|
||
drive: Use o.Update and fs.Put to optimise transfers
|
||
.IP \[bu] 2
|
||
Add version number, \-V and \-\-version
|
||
.RE
|
||
.IP \[bu] 2
|
||
v0.95 \- 2014\-03\-28
|
||
.RS 2
|
||
.IP \[bu] 2
|
||
rclone.org: website, docs and graphics
|
||
.IP \[bu] 2
|
||
drive: fix path parsing
|
||
.RE
|
||
.IP \[bu] 2
|
||
v0.94 \- 2014\-03\-27
|
||
.RS 2
|
||
.IP \[bu] 2
|
||
Change remote format one last time
|
||
.IP \[bu] 2
|
||
GNU style flags
|
||
.RE
|
||
.IP \[bu] 2
|
||
v0.93 \- 2014\-03\-16
|
||
.RS 2
|
||
.IP \[bu] 2
|
||
drive: store token in config file
|
||
.IP \[bu] 2
|
||
cross compile other versions
|
||
.IP \[bu] 2
|
||
set strict permissions on config file
|
||
.RE
|
||
.IP \[bu] 2
|
||
v0.92 \- 2014\-03\-15
|
||
.RS 2
|
||
.IP \[bu] 2
|
||
Config fixes and \-\-config option
|
||
.RE
|
||
.IP \[bu] 2
|
||
v0.91 \- 2014\-03\-15
|
||
.RS 2
|
||
.IP \[bu] 2
|
||
Make config file
|
||
.RE
|
||
.IP \[bu] 2
|
||
v0.90 \- 2013\-06\-27
|
||
.RS 2
|
||
.IP \[bu] 2
|
||
Project named rclone
|
||
.RE
|
||
.IP \[bu] 2
|
||
v0.00 \- 2012\-11\-18
|
||
.RS 2
|
||
.IP \[bu] 2
|
||
Project started
|
||
.RE
|
||
.SS Bugs and Limitations
|
||
.SS Empty directories are left behind / not created
|
||
.PP
|
||
With remotes that have a concept of directory, eg Local and Drive, empty
|
||
directories may be left behind, or not created when one was expected.
|
||
.PP
|
||
This is because rclone doesn\[aq]t have a concept of a directory \- it
|
||
only works on objects.
|
||
Most of the object storage systems can\[aq]t actually store a directory
|
||
so there is nowhere for rclone to store anything about directories.
|
||
.PP
|
||
You can work round this to some extent with the\f[C]purge\f[] command
|
||
which will delete everything under the path, \f[B]inluding\f[] empty
|
||
directories.
|
||
.PP
|
||
This may be fixed at some point in Issue
|
||
#100 (https://github.com/ncw/rclone/issues/100)
|
||
.SS Directory timestamps aren\[aq]t preserved
|
||
.PP
|
||
For the same reason as the above, rclone doesn\[aq]t have a concept of a
|
||
directory \- it only works on objects, therefore it can\[aq]t preserve
|
||
the timestamps of directories.
|
||
.SS Frequently Asked Questions
|
||
.SS Do all cloud storage systems support all rclone commands
|
||
.PP
|
||
Yes they do.
|
||
All the rclone commands (eg \f[C]sync\f[], \f[C]copy\f[] etc) will work
|
||
on all the remote storage systems.
|
||
.SS Can I copy the config from one machine to another
|
||
.PP
|
||
Sure! Rclone stores all of its config in a single file.
|
||
If you want to find this file, the simplest way is to run
|
||
\f[C]rclone\ \-h\f[] and look at the help for the \f[C]\-\-config\f[]
|
||
flag which will tell you where it is.
|
||
.PP
|
||
See the remote setup docs (https://rclone.org/remote_setup/) for more
|
||
info.
|
||
.SS How do I configure rclone on a remote / headless box with no
|
||
browser?
|
||
.PP
|
||
This has now been documented in its own remote setup
|
||
page (https://rclone.org/remote_setup/).
|
||
.SS Can rclone sync directly from drive to s3
|
||
.PP
|
||
Rclone can sync between two remote cloud storage systems just fine.
|
||
.PP
|
||
Note that it effectively downloads the file and uploads it again, so the
|
||
node running rclone would need to have lots of bandwidth.
|
||
.PP
|
||
The syncs would be incremental (on a file by file basis).
|
||
.PP
|
||
Eg
|
||
.IP
|
||
.nf
|
||
\f[C]
|
||
rclone\ sync\ drive:Folder\ s3:bucket
|
||
\f[]
|
||
.fi
|
||
.SS Using rclone from multiple locations at the same time
|
||
.PP
|
||
You can use rclone from multiple places at the same time if you choose
|
||
different subdirectory for the output, eg
|
||
.IP
|
||
.nf
|
||
\f[C]
|
||
Server\ A>\ rclone\ sync\ /tmp/whatever\ remote:ServerA
|
||
Server\ B>\ rclone\ sync\ /tmp/whatever\ remote:ServerB
|
||
\f[]
|
||
.fi
|
||
.PP
|
||
If you sync to the same directory then you should use rclone copy
|
||
otherwise the two rclones may delete each others files, eg
|
||
.IP
|
||
.nf
|
||
\f[C]
|
||
Server\ A>\ rclone\ copy\ /tmp/whatever\ remote:Backup
|
||
Server\ B>\ rclone\ copy\ /tmp/whatever\ remote:Backup
|
||
\f[]
|
||
.fi
|
||
.PP
|
||
The file names you upload from Server A and Server B should be different
|
||
in this case, otherwise some file systems (eg Drive) may make
|
||
duplicates.
|
||
.SS Why doesn\[aq]t rclone support partial transfers / binary diffs like
|
||
rsync?
|
||
.PP
|
||
Rclone stores each file you transfer as a native object on the remote
|
||
cloud storage system.
|
||
This means that you can see the files you upload as expected using
|
||
alternative access methods (eg using the Google Drive web interface).
|
||
There is a 1:1 mapping between files on your hard disk and objects
|
||
created in the cloud storage system.
|
||
.PP
|
||
Cloud storage systems (at least none I\[aq]ve come across yet) don\[aq]t
|
||
support partially uploading an object.
|
||
You can\[aq]t take an existing object, and change some bytes in the
|
||
middle of it.
|
||
.PP
|
||
It would be possible to make a sync system which stored binary diffs
|
||
instead of whole objects like rclone does, but that would break the 1:1
|
||
mapping of files on your hard disk to objects in the remote cloud
|
||
storage system.
|
||
.PP
|
||
All the cloud storage systems support partial downloads of content, so
|
||
it would be possible to make partial downloads work.
|
||
However to make this work efficiently this would require storing a
|
||
significant amount of metadata, which breaks the desired 1:1 mapping of
|
||
files to objects.
|
||
.SS Can rclone do bi\-directional sync?
|
||
.PP
|
||
No, not at present.
|
||
rclone only does uni\-directional sync from A \-> B.
|
||
It may do in the future though since it has all the primitives \- it
|
||
just requires writing the algorithm to do it.
|
||
.SS Can I use rclone with an HTTP proxy?
|
||
.PP
|
||
Yes.
|
||
rclone will use the environment variables \f[C]HTTP_PROXY\f[],
|
||
\f[C]HTTPS_PROXY\f[] and \f[C]NO_PROXY\f[], similar to cURL and other
|
||
programs.
|
||
.PP
|
||
\f[C]HTTPS_PROXY\f[] takes precedence over \f[C]HTTP_PROXY\f[] for https
|
||
requests.
|
||
.PP
|
||
The environment values may be either a complete URL or a "host[:port]",
|
||
in which case the "http" scheme is assumed.
|
||
.PP
|
||
The \f[C]NO_PROXY\f[] allows you to disable the proxy for specific
|
||
hosts.
|
||
Hosts must be comma separated, and can contain domains or parts.
|
||
For instance "foo.com" also matches "bar.foo.com".
|
||
.SS Rclone gives x509: failed to load system roots and no roots provided
|
||
error
|
||
.PP
|
||
This means that \f[C]rclone\f[] can\[aq]t file the SSL root
|
||
certificates.
|
||
Likely you are running \f[C]rclone\f[] on a NAS with a cut\-down Linux
|
||
OS, or possibly on Solaris.
|
||
.PP
|
||
Rclone (via the Go runtime) tries to load the root certificates from
|
||
these places on Linux.
|
||
.IP
|
||
.nf
|
||
\f[C]
|
||
"/etc/ssl/certs/ca\-certificates.crt",\ //\ Debian/Ubuntu/Gentoo\ etc.
|
||
"/etc/pki/tls/certs/ca\-bundle.crt",\ \ \ //\ Fedora/RHEL
|
||
"/etc/ssl/ca\-bundle.pem",\ \ \ \ \ \ \ \ \ \ \ \ \ //\ OpenSUSE
|
||
"/etc/pki/tls/cacert.pem",\ \ \ \ \ \ \ \ \ \ \ \ //\ OpenELEC
|
||
\f[]
|
||
.fi
|
||
.PP
|
||
So doing something like this should fix the problem.
|
||
It also sets the time which is important for SSL to work properly.
|
||
.IP
|
||
.nf
|
||
\f[C]
|
||
mkdir\ \-p\ /etc/ssl/certs/
|
||
curl\ \-o\ /etc/ssl/certs/ca\-certificates.crt\ https://raw.githubusercontent.com/bagder/ca\-bundle/master/ca\-bundle.crt
|
||
ntpclient\ \-s\ \-h\ pool.ntp.org
|
||
\f[]
|
||
.fi
|
||
.PP
|
||
Note that you may need to add the \f[C]\-\-insecure\f[] option to the
|
||
\f[C]curl\f[] command line if it doesn\[aq]t work without.
|
||
.IP
|
||
.nf
|
||
\f[C]
|
||
curl\ \-\-insecure\ \-o\ /etc/ssl/certs/ca\-certificates.crt\ https://raw.githubusercontent.com/bagder/ca\-bundle/master/ca\-bundle.crt
|
||
\f[]
|
||
.fi
|
||
.SS Rclone gives Failed to load config file: function not implemented
|
||
error
|
||
.PP
|
||
Likely this means that you are running rclone on Linux version not
|
||
supported by the go runtime, ie earlier than version 2.6.23.
|
||
.PP
|
||
See the system requirements section in the go install
|
||
docs (https://golang.org/doc/install) for full details.
|
||
.SS All my uploaded docx/xlsx/pptx files appear as archive/zip
|
||
.PP
|
||
This is caused by uploading these files from a Windows computer which
|
||
hasn\[aq]t got the Microsoft Office suite installed.
|
||
The easiest way to fix is to install the Word viewer and the Microsoft
|
||
Office Compatibility Pack for Word, Excel, and PowerPoint 2007 and later
|
||
versions\[aq] file formats
|
||
.SS tcp lookup some.domain.com no such host
|
||
.PP
|
||
This happens when rclone cannot resolve a domain.
|
||
Please check that your DNS setup is generally working, e.g.
|
||
.IP
|
||
.nf
|
||
\f[C]
|
||
#\ both\ should\ print\ a\ long\ list\ of\ possible\ IP\ addresses
|
||
dig\ www.googleapis.com\ \ \ \ \ \ \ \ \ \ #\ resolve\ using\ your\ default\ DNS
|
||
dig\ www.googleapis.com\ \@8.8.8.8\ #\ resolve\ with\ Google\[aq]s\ DNS\ server
|
||
\f[]
|
||
.fi
|
||
.PP
|
||
If you are using \f[C]systemd\-resolved\f[] (default on Arch Linux),
|
||
ensure it is at version 233 or higher.
|
||
Previous releases contain a bug which causes not all domains to be
|
||
resolved properly.
|
||
.SS License
|
||
.PP
|
||
This is free software under the terms of MIT the license (check the
|
||
COPYING file included with the source code).
|
||
.IP
|
||
.nf
|
||
\f[C]
|
||
Copyright\ (C)\ 2012\ by\ Nick\ Craig\-Wood\ https://www.craig\-wood.com/nick/
|
||
|
||
Permission\ is\ hereby\ granted,\ free\ of\ charge,\ to\ any\ person\ obtaining\ a\ copy
|
||
of\ this\ software\ and\ associated\ documentation\ files\ (the\ "Software"),\ to\ deal
|
||
in\ the\ Software\ without\ restriction,\ including\ without\ limitation\ the\ rights
|
||
to\ use,\ copy,\ modify,\ merge,\ publish,\ distribute,\ sublicense,\ and/or\ sell
|
||
copies\ of\ the\ Software,\ and\ to\ permit\ persons\ to\ whom\ the\ Software\ is
|
||
furnished\ to\ do\ so,\ subject\ to\ the\ following\ conditions:
|
||
|
||
The\ above\ copyright\ notice\ and\ this\ permission\ notice\ shall\ be\ included\ in
|
||
all\ copies\ or\ substantial\ portions\ of\ the\ Software.
|
||
|
||
THE\ SOFTWARE\ IS\ PROVIDED\ "AS\ IS",\ WITHOUT\ WARRANTY\ OF\ ANY\ KIND,\ EXPRESS\ OR
|
||
IMPLIED,\ INCLUDING\ BUT\ NOT\ LIMITED\ TO\ THE\ WARRANTIES\ OF\ MERCHANTABILITY,
|
||
FITNESS\ FOR\ A\ PARTICULAR\ PURPOSE\ AND\ NONINFRINGEMENT.\ IN\ NO\ EVENT\ SHALL\ THE
|
||
AUTHORS\ OR\ COPYRIGHT\ HOLDERS\ BE\ LIABLE\ FOR\ ANY\ CLAIM,\ DAMAGES\ OR\ OTHER
|
||
LIABILITY,\ WHETHER\ IN\ AN\ ACTION\ OF\ CONTRACT,\ TORT\ OR\ OTHERWISE,\ ARISING\ FROM,
|
||
OUT\ OF\ OR\ IN\ CONNECTION\ WITH\ THE\ SOFTWARE\ OR\ THE\ USE\ OR\ OTHER\ DEALINGS\ IN
|
||
THE\ SOFTWARE.
|
||
\f[]
|
||
.fi
|
||
.SS Authors
|
||
.IP \[bu] 2
|
||
Nick Craig\-Wood <nick@craig-wood.com>
|
||
.SS Contributors
|
||
.IP \[bu] 2
|
||
Alex Couper <amcouper@gmail.com>
|
||
.IP \[bu] 2
|
||
Leonid Shalupov <leonid@shalupov.com> <shalupov@diverse.org.ru>
|
||
.IP \[bu] 2
|
||
Shimon Doodkin <helpmepro1@gmail.com>
|
||
.IP \[bu] 2
|
||
Colin Nicholson <colin@colinn.com>
|
||
.IP \[bu] 2
|
||
Klaus Post <klauspost@gmail.com>
|
||
.IP \[bu] 2
|
||
Sergey Tolmachev <tolsi.ru@gmail.com>
|
||
.IP \[bu] 2
|
||
Adriano Aurélio Meirelles <adriano@atinge.com>
|
||
.IP \[bu] 2
|
||
C.
|
||
Bess <cbess@users.noreply.github.com>
|
||
.IP \[bu] 2
|
||
Dmitry Burdeev <dibu28@gmail.com>
|
||
.IP \[bu] 2
|
||
Joseph Spurrier <github@josephspurrier.com>
|
||
.IP \[bu] 2
|
||
Björn Harrtell <bjorn@wololo.org>
|
||
.IP \[bu] 2
|
||
Xavier Lucas <xavier.lucas@corp.ovh.com>
|
||
.IP \[bu] 2
|
||
Werner Beroux <werner@beroux.com>
|
||
.IP \[bu] 2
|
||
Brian Stengaard <brian@stengaard.eu>
|
||
.IP \[bu] 2
|
||
Jakub Gedeon <jgedeon@sofi.com>
|
||
.IP \[bu] 2
|
||
Jim Tittsler <jwt@onjapan.net>
|
||
.IP \[bu] 2
|
||
Michal Witkowski <michal@improbable.io>
|
||
.IP \[bu] 2
|
||
Fabian Ruff <fabian.ruff@sap.com>
|
||
.IP \[bu] 2
|
||
Leigh Klotz <klotz@quixey.com>
|
||
.IP \[bu] 2
|
||
Romain Lapray <lapray.romain@gmail.com>
|
||
.IP \[bu] 2
|
||
Justin R.
|
||
Wilson <jrw972@gmail.com>
|
||
.IP \[bu] 2
|
||
Antonio Messina <antonio.s.messina@gmail.com>
|
||
.IP \[bu] 2
|
||
Stefan G.
|
||
Weichinger <office@oops.co.at>
|
||
.IP \[bu] 2
|
||
Per Cederberg <cederberg@gmail.com>
|
||
.IP \[bu] 2
|
||
Radek Šenfeld <rush@logic.cz>
|
||
.IP \[bu] 2
|
||
Fredrik Fornwall <fredrik@fornwall.net>
|
||
.IP \[bu] 2
|
||
Asko Tamm <asko@deekit.net>
|
||
.IP \[bu] 2
|
||
xor\-zz <xor@gstocco.com>
|
||
.IP \[bu] 2
|
||
Tomasz Mazur <tmazur90@gmail.com>
|
||
.IP \[bu] 2
|
||
Marco Paganini <paganini@paganini.net>
|
||
.IP \[bu] 2
|
||
Felix Bünemann <buenemann@louis.info>
|
||
.IP \[bu] 2
|
||
Durval Menezes <jmrclone@durval.com>
|
||
.IP \[bu] 2
|
||
Luiz Carlos Rumbelsperger Viana <maxd13_luiz_carlos@hotmail.com>
|
||
.IP \[bu] 2
|
||
Stefan Breunig <stefan-github@yrden.de>
|
||
.IP \[bu] 2
|
||
Alishan Ladhani <ali-l@users.noreply.github.com>
|
||
.IP \[bu] 2
|
||
0xJAKE <0xJAKE@users.noreply.github.com>
|
||
.IP \[bu] 2
|
||
Thibault Molleman <thibaultmol@users.noreply.github.com>
|
||
.IP \[bu] 2
|
||
Scott McGillivray <scott.mcgillivray@gmail.com>
|
||
.IP \[bu] 2
|
||
Bjørn Erik Pedersen <bjorn.erik.pedersen@gmail.com>
|
||
.IP \[bu] 2
|
||
Lukas Loesche <lukas@mesosphere.io>
|
||
.IP \[bu] 2
|
||
emyarod <allllaboutyou@gmail.com>
|
||
.IP \[bu] 2
|
||
T.C.
|
||
Ferguson <tcf909@gmail.com>
|
||
.IP \[bu] 2
|
||
Brandur <brandur@mutelight.org>
|
||
.IP \[bu] 2
|
||
Dario Giovannetti <dev@dariogiovannetti.net>
|
||
.IP \[bu] 2
|
||
Károly Oláh <okaresz@aol.com>
|
||
.IP \[bu] 2
|
||
Jon Yergatian <jon@macfanatic.ca>
|
||
.IP \[bu] 2
|
||
Jack Schmidt <github@mowsey.org>
|
||
.IP \[bu] 2
|
||
Dedsec1 <Dedsec1@users.noreply.github.com>
|
||
.IP \[bu] 2
|
||
Hisham Zarka <hzarka@gmail.com>
|
||
.IP \[bu] 2
|
||
Jérôme Vizcaino <jerome.vizcaino@gmail.com>
|
||
.IP \[bu] 2
|
||
Mike Tesch <mjt6129@rit.edu>
|
||
.IP \[bu] 2
|
||
Marvin Watson <marvwatson@users.noreply.github.com>
|
||
.IP \[bu] 2
|
||
Danny Tsai <danny8376@gmail.com>
|
||
.IP \[bu] 2
|
||
Yoni Jah <yonjah+git@gmail.com> <yonjah+github@gmail.com>
|
||
.IP \[bu] 2
|
||
Stephen Harris <github@spuddy.org>
|
||
.IP \[bu] 2
|
||
Ihor Dvoretskyi <ihor.dvoretskyi@gmail.com>
|
||
.IP \[bu] 2
|
||
Jon Craton <jncraton@gmail.com>
|
||
.IP \[bu] 2
|
||
Hraban Luyat <hraban@0brg.net>
|
||
.IP \[bu] 2
|
||
Michael Ledin <mledin89@gmail.com>
|
||
.IP \[bu] 2
|
||
Martin Kristensen <me@azgul.com>
|
||
.IP \[bu] 2
|
||
Too Much IO <toomuchio@users.noreply.github.com>
|
||
.IP \[bu] 2
|
||
Anisse Astier <anisse@astier.eu>
|
||
.IP \[bu] 2
|
||
Zahiar Ahmed <zahiar@live.com>
|
||
.IP \[bu] 2
|
||
Igor Kharin <igorkharin@gmail.com>
|
||
.IP \[bu] 2
|
||
Bill Zissimopoulos <billziss@navimatics.com>
|
||
.IP \[bu] 2
|
||
Bob Potter <bobby.potter@gmail.com>
|
||
.IP \[bu] 2
|
||
Steven Lu <tacticalazn@gmail.com>
|
||
.IP \[bu] 2
|
||
Sjur Fredriksen <sjurtf@ifi.uio.no>
|
||
.IP \[bu] 2
|
||
Ruwbin <hubus12345@gmail.com>
|
||
.IP \[bu] 2
|
||
Fabian Möller <fabianm88@gmail.com>
|
||
.IP \[bu] 2
|
||
Edward Q.
|
||
Bridges <github@eqbridges.com>
|
||
.IP \[bu] 2
|
||
Vasiliy Tolstov <v.tolstov@selfip.ru>
|
||
.IP \[bu] 2
|
||
Harshavardhana <harsha@minio.io>
|
||
.IP \[bu] 2
|
||
sainaen <sainaen@gmail.com>
|
||
.IP \[bu] 2
|
||
gdm85 <gdm85@users.noreply.github.com>
|
||
.IP \[bu] 2
|
||
Yaroslav Halchenko <debian@onerussian.com>
|
||
.IP \[bu] 2
|
||
John Papandriopoulos <jpap@users.noreply.github.com>
|
||
.IP \[bu] 2
|
||
Zhiming Wang <zmwangx@gmail.com>
|
||
.IP \[bu] 2
|
||
Andy Pilate <cubox@cubox.me>
|
||
.IP \[bu] 2
|
||
Oliver Heyme <olihey@googlemail.com>
|
||
.IP \[bu] 2
|
||
wuyu <wuyu@yunify.com>
|
||
.IP \[bu] 2
|
||
Andrei Dragomir <adragomi@adobe.com>
|
||
.IP \[bu] 2
|
||
Christian Brüggemann <mail@cbruegg.com>
|
||
.IP \[bu] 2
|
||
Alex McGrath Kraak <amkdude@gmail.com>
|
||
.IP \[bu] 2
|
||
bpicode <bjoern.pirnay@googlemail.com>
|
||
.IP \[bu] 2
|
||
Daniel Jagszent <daniel@jagszent.de>
|
||
.IP \[bu] 2
|
||
Josiah White <thegenius2009@gmail.com>
|
||
.IP \[bu] 2
|
||
Ishuah Kariuki <kariuki@ishuah.com> <ishuah91@gmail.com>
|
||
.IP \[bu] 2
|
||
Jan Varho <jan@varho.org>
|
||
.IP \[bu] 2
|
||
Girish Ramakrishnan <girish@cloudron.io>
|
||
.IP \[bu] 2
|
||
LingMan <LingMan@users.noreply.github.com>
|
||
.IP \[bu] 2
|
||
Jacob McNamee <jacobmcnamee@gmail.com>
|
||
.IP \[bu] 2
|
||
jersou <jertux@gmail.com>
|
||
.IP \[bu] 2
|
||
thierry <thierry@substantiel.fr>
|
||
.IP \[bu] 2
|
||
Simon Leinen <simon.leinen@gmail.com> <ubuntu@s3-test.novalocal>
|
||
.IP \[bu] 2
|
||
Dan Dascalescu <ddascalescu+github@gmail.com>
|
||
.IP \[bu] 2
|
||
Jason Rose <jason@jro.io>
|
||
.IP \[bu] 2
|
||
Andrew Starr\-Bochicchio <a.starr.b@gmail.com>
|
||
.IP \[bu] 2
|
||
John Leach <john@johnleach.co.uk>
|
||
.IP \[bu] 2
|
||
Corban Raun <craun@instructure.com>
|
||
.IP \[bu] 2
|
||
Pierre Carlson <mpcarl@us.ibm.com>
|
||
.IP \[bu] 2
|
||
Ernest Borowski <er.borowski@gmail.com>
|
||
.IP \[bu] 2
|
||
Remus Bunduc <remus.bunduc@gmail.com>
|
||
.IP \[bu] 2
|
||
Iakov Davydov <iakov.davydov@unil.ch>
|
||
.IP \[bu] 2
|
||
Fabian Möller <f.moeller@nynex.de>
|
||
.IP \[bu] 2
|
||
Jakub Tasiemski <tasiemski@gmail.com>
|
||
.IP \[bu] 2
|
||
David Minor <dminor@saymedia.com>
|
||
.IP \[bu] 2
|
||
Tim Cooijmans <cooijmans.tim@gmail.com>
|
||
.IP \[bu] 2
|
||
Laurence <liuxy6@gmail.com>
|
||
.IP \[bu] 2
|
||
Giovanni Pizzi <gio.piz@gmail.com>
|
||
.IP \[bu] 2
|
||
Filip Bartodziej <filipbartodziej@gmail.com>
|
||
.IP \[bu] 2
|
||
Jon Fautley <jon@dead.li>
|
||
.IP \[bu] 2
|
||
lewapm <32110057+lewapm@users.noreply.github.com>
|
||
.IP \[bu] 2
|
||
Yassine Imounachen <yassine256@gmail.com>
|
||
.SH Contact the rclone project
|
||
.SS Forum
|
||
.PP
|
||
Forum for general discussions and questions:
|
||
.IP \[bu] 2
|
||
https://forum.rclone.org
|
||
.SS Gitub project
|
||
.PP
|
||
The project website is at:
|
||
.IP \[bu] 2
|
||
https://github.com/ncw/rclone
|
||
.PP
|
||
There you can file bug reports, ask for help or contribute pull
|
||
requests.
|
||
.SS Google+
|
||
.PP
|
||
Rclone has a Google+ page which announcements are posted to
|
||
.IP \[bu] 2
|
||
Google+ page for general comments
|
||
.SS Twitter
|
||
.PP
|
||
You can also follow me on twitter for rclone announcements
|
||
.IP \[bu] 2
|
||
[\@njcw](https://twitter.com/njcw)
|
||
.SS Email
|
||
.PP
|
||
Or if all else fails or you want to ask something private or
|
||
confidential email Nick Craig\-Wood (mailto:nick@craig-wood.com)
|
||
.SH AUTHORS
|
||
Nick Craig\-Wood.
|