1
mirror of https://github.com/rclone/rclone synced 2024-11-25 02:47:14 +01:00
Go to file
nielash 810644e873 bisync: add --resync-mode for customizing --resync - fixes #5681
Before this change, the path1 version of a file always prevailed during
--resync, and many users requested options to automatically select the winner
based on characteristics such as newer, older, larger, and smaller. This change
adds support for such options.

Note that ideally this feature would have been implemented by allowing the
existing `--resync` flag to optionally accept string values such as `--resync
newer`. However, this would have been a breaking change, as the existing flag
is a `bool` and it does not seem to be possible to have a `string` flag that
accepts both `--resync newer` and `--resync` (with no argument.) (`NoOptDefVal`
does not work for this, as it would force an `=` like `--resync=newer`.) So
instead, the best compromise to avoid a breaking change was to add a new
`--resync-mode CHOICE` flag that implies `--resync`, while maintaining the
existing behavior of `--resync` (which implies `--resync-mode path1`. i.e. both
flags are now valid, and either can be used without the other.

--resync-mode CHOICE

In the event that a file differs on both sides during a `--resync`,
`--resync-mode` controls which version will overwrite the other. The supported
options are similar to `--conflict-resolve`. For all of the following options,
the version that is kept is referred to as the "winner", and the version that
is overwritten (deleted) is referred to as the "loser". The options are named
after the "winner":

- `path1` - (the default) - the version from Path1 is unconditionally
considered the winner (regardless of `modtime` and `size`, if any). This can be
useful if one side is more trusted or up-to-date than the other, at the time of
the `--resync`.
- `path2` - same as `path1`, except the path2 version is considered the winner.
- `newer` - the newer file (by `modtime`) is considered the winner, regardless
of which side it came from. This may result in having a mix of some winners
from Path1, and some winners from Path2. (The implementation is analagous to
running `rclone copy --update` in both directions.)
- `older` - same as `newer`, except the older file is considered the winner,
and the newer file is considered the loser.
- `larger` - the larger file (by `size`) is considered the winner (regardless
of `modtime`, if any). This can be a useful option for remotes without
`modtime` support, or with the kinds of files (such as logs) that tend to grow
but not shrink, over time.
- `smaller` - the smaller file (by `size`) is considered the winner (regardless
of `modtime`, if any).

For all of the above options, note the following:
- If either of the underlying remotes lacks support for the chosen method, it
will be ignored and will fall back to the default of `path1`. (For example, if
`--resync-mode newer` is set, but one of the paths uses a remote that doesn't
support `modtime`.)
- If a winner can't be determined because the chosen method's attribute is
missing or equal, it will be ignored, and bisync will instead try to determine
whether the files differ by looking at the other `--compare` methods in effect.
(For example, if `--resync-mode newer` is set, but the Path1 and Path2 modtimes
are identical, bisync will compare the sizes.) If bisync concludes that they
differ, preference is given to whichever is the "source" at that moment. (In
practice, this gives a slight advantage to Path2, as the 2to1 copy comes before
the 1to2 copy.) If the files _do not_ differ, nothing is copied (as both sides
are already correct).
- These options apply only to files that exist on both sides (with the same
name and relative path). Files that exist *only* on one side and not the other
are *always* copied to the other, during `--resync` (this is one of the main
differences between resync and non-resync runs.).
- `--conflict-resolve`, `--conflict-loser`, and `--conflict-suffix` do not
apply during `--resync`, and unlike these flags, nothing is renamed during
`--resync`. When a file differs on both sides during `--resync`, one version
always overwrites the other (much like in `rclone copy`.) (Consider using
`--backup-dir` to retain a backup of the losing version.)
- Unlike for `--conflict-resolve`, `--resync-mode none` is not a valid option
(or rather, it will be interpreted as "no resync", unless `--resync` has also
been specified, in which case it will be ignored.)
- Winners and losers are decided at the individual file-level only (there is
not currently an option to pick an entire winning directory atomically,
although the `path1` and `path2` options typically produce a similar result.)
- To maintain backward-compatibility, the `--resync` flag implies
`--resync-mode path1` unless a different `--resync-mode` is explicitly
specified. Similarly, all `--resync-mode` options (except `none`) imply
`--resync`, so it is not necessary to use both the `--resync` and
`--resync-mode` flags simultaneously -- either one is sufficient without the
other.
2024-01-20 17:17:01 -05:00
.github build(deps): bump actions/cache from 3 to 4 2024-01-19 17:19:08 +00:00
backend local: fix cleanRootPath on Windows after go1.21.4 stdlib update 2024-01-20 14:50:08 -05:00
bin amazonclouddrive: remove Amazon Drive backend code and docs #7539 2024-01-04 17:05:54 +00:00
cmd bisync: add --resync-mode for customizing --resync - fixes #5681 2024-01-20 17:17:01 -05:00
cmdtest cmdtest: fix typos found by codespell 2023-03-24 20:44:25 +00:00
contrib docker volume plugin: add missing fuse3 dependency #6844 2023-03-15 15:57:53 +00:00
docs bisync: add --resync-mode for customizing --resync - fixes #5681 2024-01-20 17:17:01 -05:00
fs bisync: full support for comparing checksum, size, modtime - fixes #5679 fixes #5683 fixes #5684 fixes #5675 2024-01-20 16:08:06 -05:00
fstest build: remove random.Seed since random generator is seeded automatically in go1.20 2024-01-15 16:22:07 +00:00
graphics build: Add file properties and icon to Windows executable (fixes #4304) 2020-06-11 09:26:14 +01:00
lib build: remove random.Seed since random generator is seeded automatically in go1.20 2024-01-15 16:22:07 +00:00
librclone docs/librclone: the newer and recommended ucrt64 subsystem of msys2 can now be used for building on windows 2024-01-01 21:56:45 +01:00
vfs build: remove random.Seed since random generator is seeded automatically in go1.20 2024-01-15 16:22:07 +00:00
.gitattributes build: add azure pipelines build 2019-08-06 10:31:32 +01:00
.gitignore serve s3: let rclone act as an S3 compatible server 2023-11-16 16:59:55 +00:00
.golangci.yml build: re-enable revive linters 2023-08-29 13:01:15 +01:00
CONTRIBUTING.md Refresh CONTRIBUTING.md 2023-11-19 12:48:27 +00:00
COPYING Initial commit - some small parts working 2012-11-18 17:32:31 +00:00
Dockerfile build: fix docker build on arm/v6 2024-01-03 17:43:23 +00:00
go.mod operations: add logger to log list of sync results -- fixes #7282 2024-01-20 14:50:08 -05:00
go.sum operations: add logger to log list of sync results -- fixes #7282 2024-01-20 14:50:08 -05:00
MAINTAINERS.md Add @nielash as bisync maintainer 2023-09-05 04:05:39 -04:00
Makefile build: use API when fetching golangci-lint as it is more reliable 2024-01-15 16:22:07 +00:00
MANUAL.html Version v1.65.0 2023-11-26 16:07:39 +00:00
MANUAL.md Version v1.65.0 2023-11-26 16:07:39 +00:00
MANUAL.txt Version v1.65.0 2023-11-26 16:07:39 +00:00
notes.txt docs: spelling: etc. 2020-10-28 18:16:23 +00:00
rclone.1 Version v1.65.0 2023-11-26 16:07:39 +00:00
rclone.go lib: add plugin support 2019-09-29 11:05:10 +01:00
README.md docs: add bisync to index 2024-01-20 14:50:08 -05:00
RELEASE.md build: fix docker build on arm/v6 2024-01-03 17:43:23 +00:00
VERSION Start v1.66.0-DEV development 2023-11-26 17:14:38 +00:00

rclone logo rclone logo

Website | Documentation | Download | Contributing | Changelog | Installation | Forum

Build Status Go Report Card GoDoc Docker Pulls

Rclone

Rclone ("rsync for cloud storage") is a command-line program to sync files and directories to and from different cloud storage providers.

Storage providers

  • 1Fichier 📄
  • Akamai Netstorage 📄
  • Alibaba Cloud (Aliyun) Object Storage System (OSS) 📄
  • Amazon S3 📄
  • ArvanCloud Object Storage (AOS) 📄
  • Backblaze B2 📄
  • Box 📄
  • Ceph 📄
  • China Mobile Ecloud Elastic Object Storage (EOS) 📄
  • Cloudflare R2 📄
  • Citrix ShareFile 📄
  • DigitalOcean Spaces 📄
  • Digi Storage 📄
  • Dreamhost 📄
  • Dropbox 📄
  • Enterprise File Fabric 📄
  • Fastmail Files 📄
  • FTP 📄
  • Google Cloud Storage 📄
  • Google Drive 📄
  • Google Photos 📄
  • HDFS (Hadoop Distributed Filesystem) 📄
  • HiDrive 📄
  • HTTP 📄
  • Huawei Cloud Object Storage Service(OBS) 📄
  • ImageKit 📄
  • Internet Archive 📄
  • Jottacloud 📄
  • IBM COS S3 📄
  • IONOS Cloud 📄
  • Koofr 📄
  • Leviia Object Storage 📄
  • Liara Object Storage 📄
  • Linkbox 📄
  • Linode Object Storage 📄
  • Mail.ru Cloud 📄
  • Memset Memstore 📄
  • Mega 📄
  • Memory 📄
  • Microsoft Azure Blob Storage 📄
  • Microsoft Azure Files Storage 📄
  • Microsoft OneDrive 📄
  • Minio 📄
  • Nextcloud 📄
  • OVH 📄
  • Blomp Cloud Storage 📄
  • OpenDrive 📄
  • OpenStack Swift 📄
  • Oracle Cloud Storage 📄
  • Oracle Object Storage 📄
  • ownCloud 📄
  • pCloud 📄
  • Petabox 📄
  • PikPak 📄
  • premiumize.me 📄
  • put.io 📄
  • Proton Drive 📄
  • QingStor 📄
  • Qiniu Cloud Object Storage (Kodo) 📄
  • Quatrix 📄
  • Rackspace Cloud Files 📄
  • RackCorp Object Storage 📄
  • Scaleway 📄
  • Seafile 📄
  • SeaweedFS 📄
  • SFTP 📄
  • SMB / CIFS 📄
  • StackPath 📄
  • Storj 📄
  • SugarSync 📄
  • Synology C2 Object Storage 📄
  • Tencent Cloud Object Storage (COS) 📄
  • Wasabi 📄
  • WebDAV 📄
  • Yandex Disk 📄
  • Zoho WorkDrive 📄
  • The local filesystem 📄

Please see the full list of all storage providers and their features

Virtual storage providers

These backends adapt or modify other storage providers

  • Alias: rename existing remotes 📄
  • Cache: cache remotes (DEPRECATED) 📄
  • Chunker: split large files 📄
  • Combine: combine multiple remotes into a directory tree 📄
  • Compress: compress files 📄
  • Crypt: encrypt files 📄
  • Hasher: hash files 📄
  • Union: join multiple remotes to work together 📄

Features

  • MD5/SHA-1 hashes checked at all times for file integrity
  • Timestamps preserved on files
  • Partial syncs supported on a whole file basis
  • Copy mode to just copy new/changed files
  • Sync (one way) mode to make a directory identical
  • Bisync (two way) to keep two directories in sync bidirectionally
  • Check mode to check for file hash equality
  • Can sync to and from network, e.g. two different cloud accounts
  • Optional large file chunking (Chunker)
  • Optional transparent compression (Compress)
  • Optional encryption (Crypt)
  • Optional FUSE mount (rclone mount)
  • Multi-threaded downloads to local disk
  • Can serve local or remote files over HTTP/WebDAV/FTP/SFTP/DLNA

Installation & documentation

Please see the rclone website for:

Downloads

License

This is free software under the terms of the MIT license (check the COPYING file included in this package).