Commit Graph

7735 Commits

Author SHA1 Message Date
Nick Craig-Wood 15890b7ce7 cmd: make auto completion work for all shells and reduce the size
This updates the bash completion to work with GenBashCompletionV2
which cuts down the size of the completion file dramatically.

See: https://forum.rclone.org/t/request-make-remote-path-completion-work-for-fish-and-zsh/42982/
See: #7000
2024-02-29 14:46:50 +00:00
Nick Craig-Wood 186bb85c44 crypt: add missing error check spotted by linter 2024-02-29 14:46:50 +00:00
nielash 4c6d2c5410 crypt: improve handling of undecryptable file names - fixes #5787 fixes #6439 fixes #6437
Before this change, undecryptable file names would be skipped very quietly
(there was a log warning, but only at DEBUG level),
failing to alert users of a potentially serious issue that needs attention.

After this change, the log level is raised to NOTICE by default and a new
--crypt-strict-names flag allows raising an error, for users who may prefer not
to proceed if such an issue is detected.

See https://forum.rclone.org/t/skipping-undecryptable-file-name-should-be-an-error/27115
https://github.com/rclone/rclone/issues/5787
2024-02-29 12:11:02 +00:00
Nick Craig-Wood f5f86786b2 sync: implement directory sync for mod times and metadata
Directory mod times are synced by default if the backend is capable
and directory metadata is synced if the --metadata flag is provided
and the backend is capable.

This updates the bisync golden tests also which were affected by
--dry-run setting of directory modtimes.

Fixes #6685
2024-02-28 16:26:14 +00:00
Nick Craig-Wood 15579c2195 fstests: factor out fstest.NewObject function 2024-02-28 16:26:14 +00:00
Nick Craig-Wood e8fe0b0553 operations: Implement CopyDirMetadata, CopyDirModTime and SetDirModTime 2024-02-28 16:26:14 +00:00
Nick Craig-Wood 09953d77b5 lsjson,lsf: make sure metadata appears for directories 2024-02-28 16:26:14 +00:00
Nick Craig-Wood e4d0055b3e drive: implement modtime and metadata setting for directories 2024-02-28 16:26:14 +00:00
Nick Craig-Wood a60da2ef38 local: fix setting of btime on directories on Windows
Before this change this would give errors like this

    failed to set metadata on directory: failed to set birth (creation) time: Access is denied.

This was caused by opening the directory in the wrong mode.
2024-02-28 16:25:59 +00:00
Nick Craig-Wood 7b01564f83 local: implement modtime and metadata for directories
A consequence of this is that fs.Directory returned by the local
backend will now have a correct size in (rather than -1). Some tests
depended on this and have been fixed by this commit too.
2024-02-28 16:09:04 +00:00
Nick Craig-Wood 39db8caff1 cache,chunker,combine,compress,crypt,hasher,union: implement MkdirMetadata and related Features 2024-02-28 16:09:04 +00:00
nielash 0297542f6b cache,chunker,combine,compress,crypt,hasher,union: implement DirSetModTime (if supported by wrapped remote) 2024-02-28 16:09:04 +00:00
nielash 17c0ecc72c sftp: implement DirSetModTime 2024-02-28 16:09:04 +00:00
nielash cbcb295185 drive: implement DirSetModTime 2024-02-27 19:59:13 +00:00
nielash 67e3725205 local: implement DirSetModTime 2024-02-27 19:59:13 +00:00
Nick Craig-Wood 61d76ae47d fstests: add integration tests for Directory Metadata and ModTime 2024-02-27 19:59:13 +00:00
Nick Craig-Wood fd1ca2dfe8 fs: allow Metadata calls to be called with Directory or Object
This involved adding the Fs() method to DirEntry as it is needed in
the metadata mapper.

Unspecialised fs.Dir objects will return a new fs.Unknown from their
Fs() methods as they are not specific to any given Fs.
2024-02-27 10:56:19 +00:00
Nick Craig-Wood e1032f693f fs: add DirWrapper for wrapping Directory-s with optional methods 2024-02-27 10:56:19 +00:00
Nick Craig-Wood a4cadd1128 fs: add Directory Metadata flags for backends and interfaces
Add backend flags
- ReadDirMetadata
- WriteDirMetadata
- WriteDirSetModTime
- UserDirMetadata
- DirModTimeUpdatesOnWrite

Add Metadata/SetMetadata for directories.

Add MkdirMetadata optional feature
2024-02-27 10:56:19 +00:00
nielash 6da52d76a7 fs: implement DirSetModTime optional feature 2024-02-22 11:13:54 +00:00
Nick Craig-Wood 71a1bbb2be errcount: factor errcount abstraction from operations 2024-02-22 11:13:54 +00:00
Nick Craig-Wood 8f0e9f9f6b mega: fix panic with go1.22
Before this fix rclone would crash with

    panic: encoding alphabet includes duplicate symbols

When compiled with go1.22. This was fixed upstream in

https://github.com/t3rm1n4l/go-mega/issues/48

And this just pulls in the fix.

Fixes #7639
2024-02-21 18:41:44 +00:00
Nick Craig-Wood 072d1f10ab serve webdav: fix --baseurl without leading /
The webdav server needs the prefix passed to it with a leading /
otherwise it does not remove it properly.

The docs state that a leading slash is optional so this patch adds one
if not present.

See: https://forum.rclone.org/t/cant-rename-files-in-rclone-serve-webdav-with-baseurl-maybe-wrong-handling-of-move-request-method/44637
2024-02-21 18:08:44 +00:00
Nick Craig-Wood 5014348229 Add Anders Swanson to contributors 2024-02-21 18:08:44 +00:00
Nick Craig-Wood ed78ac7c92 Add Joe Cai to contributors 2024-02-21 18:08:44 +00:00
Nick Craig-Wood 53d873d60d Add Dan McArdle to contributors 2024-02-21 18:08:44 +00:00
Nick Craig-Wood f2c35fdec6 Add Gabriel Ramos to contributors 2024-02-21 18:08:44 +00:00
Nick Craig-Wood 1c69b20ed7 Add Jack Provance to contributors 2024-02-21 18:08:44 +00:00
nielash 547c635552 mailru: add override for TestApplyTransforms - #7591
mailru is unable to handle filenames with certain combining characters (for
example: йěáñ), and is therefore incapable of testing ApplyTransforms. (It is
also therefore incapable of fully supporting --no-unicode-normalization.)

The same override is applied to chunker when wrapping mailru.
2024-02-21 18:02:19 +00:00
nielash f0d9117ff3 linkbox: add override for TestFixCase - #7591
linkbox already has an override for TestCaseInsensitiveMoveFile, and being able
to handle case-insensitive moves is a prerequisite for TestFixCase.
2024-02-21 18:02:19 +00:00
nielash 9d2bd163c7 opendrive: fix moving file/folder within the same parent dir - #7591
Before this change, moving (renaming) a file or folder to a different name
within the same parent directory would fail, due to using the wrong API
operation ("/file/move_copy.json" and "/folder/move_copy.json", instead of the
separate "/file/rename.json" and "/folder/rename.json" that opendrive has for
this purpose.)

After this change, Move and DirMove check whether the move is within the same
parent dir. If so, "rename" is used. If not, "move_copy" is used, like before.
2024-02-21 18:02:19 +00:00
Anders Swanson db8fb5ceda oracleobjectstorage: supports workload identity authentication for OKE
Signed-off-by: Anders Swanson <anders.swanson@oracle.com>
2024-02-20 16:25:59 +00:00
Joe Cai a1e66cc5e8 swift: Avoid unnecessary container versioning check
Container versioning check is only needed for non-empty large objects.
2024-02-20 15:52:25 +00:00
nielash 7b8bbe531e nfsmount: fix --volname being ignored #7503
Before this change, nfsmount ignored the --volname flag. After this change, the --
volname flag is respected, making it possible to set a custom volume name.

macOS users should note that Finder will show the correct volume name in most
places, but a notable exception is the sidebar, which will show "localhost".
This seems to be a system limitation (at least without `sudo`), but see the
discussion at https://github.com/rclone/rclone/issues/7503#issuecomment-1933997678
for some possible workarounds.
2024-02-18 05:08:59 -05:00
nielash 0e2f1d64e3 nfsmount: fix exit after external unmount #7503
Before this change, if a user unmounted externally (for example, via the Finder
UI), rclone would not be aware of this and wait forever to exit -- effectively
causing a deadlock that would require Ctrl+C to terminate.

After this change, when the handler detects an external unmount, it calls a
function which allows rclone to cleanly shutdown the VFS and exit.
2024-02-18 05:08:59 -05:00
nielash 5638a3841f serve nfs: fix writing files via Finder on macOS - fixes #7503
Before this change, writing files to an `nfsmount` via Finder on macOS would
cause critical errors, rendering `nfsmount` effectively unusable on macOS. This
change fixes the issue so that writes via Finder should be possible.

The issue was primarily caused by the handler's HandleLimit being set to -1. -1 is
the correct default for a NullAuthHandler, but not for a CachingHandler, which
interprets -1 not as "no limit" but as "no cache".

This change sets a high default of 1000000, and gives the user control over it
with a new --nfs-cache-handle-limit flag (available in both `serve nfs` and
`nfsmount`. A minimum of 5 is enforced, as any lower than this will be
insufficient to support directory listing.
2024-02-18 05:08:59 -05:00
Dan McArdle 6986a43b68 bisync: delete flushCache() function from tests
The flushCache() function has a bug that causes it to never actually
flush the cache. Specifically, it checks whether DirCacheFlush is nil,
but never calls it.

The tests are already passing without flushing the dir cache, so this
commit just deletes flushCache() and its call sites.

Fixes rclone/rclone#7623
2024-02-18 04:14:51 -05:00
Oksana Zhykina 11c6489fd1 quatrix: add option to skip project folders 2024-02-18 07:38:19 +01:00
Gabriel Ramos 43823bc925
webdav: reduce priority of chunks upload log 2024-02-18 07:29:23 +01:00
dependabot[bot] a3b661be0d build(deps): bump golangci/golangci-lint-action from 3 to 4
Bumps [golangci/golangci-lint-action](https://github.com/golangci/golangci-lint-action) from 3 to 4.
- [Release notes](https://github.com/golangci/golangci-lint-action/releases)
- [Commits](https://github.com/golangci/golangci-lint-action/compare/v3...v4)

---
updated-dependencies:
- dependency-name: golangci/golangci-lint-action
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
2024-02-18 07:25:50 +01:00
Jack Provance f113c68b13
docs: Fix a heading level in webdav.md documentation (#7631)
This fixes a heading problem under the "Provider Notes" section.
2024-02-18 07:16:23 +01:00
nielash 137f7f62fb sync: use operations.DirMove instead of sync.MoveDir for --fix-case - #7591
This should be more efficient for the purposes of --fix-case, as operations.DirMove
accepts `srcRemote` and `dstRemote` arguments, while sync.MoveDir does not.

This also factors the two-step-move logic to operations.DirMoveCaseInsensitive, so
that it is reusable by other commands.
2024-02-13 15:07:41 -05:00
nielash dfe76570a1 operations: skip backends incapable of testing TestApplyTransforms - #7591
This adds a step to detect whether the backend is capable of supporting the
feature, and skips the test if not. A backend can be incapable if, for example,
it is non-case-preserving or automatically converts NFD to NFC.
2024-02-13 15:07:41 -05:00
nielash f4c058e13e bisync: use global --retries and --retries-sleep flags instead of overriding 2024-02-12 13:24:54 -05:00
nielash 407a0f3733 cmd: refactor --retries and --retries-sleep to global config
This change moves the --retries and --retries-sleep flags/variables from cmd to
config (consistent with --low-level-retries), so that they can be more easily
referenced from subcommands.
2024-02-12 13:24:54 -05:00
nielash b14269fd23 bisync: add support for --retries-sleep - fixes #7555
Before this change, bisync supported --retries but not --retries-sleep.
This change adds support for --retries-sleep.
2024-02-12 13:24:54 -05:00
nielash 76b7bcd4d7 bisync: reset errors between retries
Before this change, in the event of a retryable error, bisync would always retry
the maximum number of times allowed by the `--retries` flag, even if one of the
retries was successful. This change fixes the issue, so that bisync moves on
after the first successful retry.
2024-02-12 13:24:54 -05:00
nielash 782ab3f582 bisync: clean up docs
(as the flags in docs/content/bisync.md do not update automatically, unlike
docs/content/commands/rclone_bisync.md)
2024-02-12 13:24:54 -05:00
nielash 9c6325c131 backend: rename variables to fix CI lint test failures 2024-02-12 12:49:00 -05:00
Volodymyr 2abeda5961
quatrix: fix Content-Range header
This change does not actually affect uploads. Just to be right according to definition of Content-Range in
https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Content-Range#range-end
2024-02-09 16:44:45 +00:00