1
mirror of https://github.com/rclone/rclone synced 2024-11-26 04:07:22 +01:00
rclone/backend/s3
Nick Craig-Wood 83f61a9cfb s3: GCS provider: fix server side copy of files bigger than 5G
GCS gives NotImplemented errors for multi-part server side copies. The
threshold for these is currently set just below 5G so any files bigger
than 5G that rclone attempts to server side copy will fail.

This patch works around the problem by adding a quirk for GCS raising
--s3-copy-cutoff to the maximum. This means that rclone will never use
multi-part copies for files in GCS. This includes files bigger than
5GB which (according to AWS documentation) must be copied with
multi-part copy. However this seems to work with GCS.

See: https://forum.rclone.org/t/chunker-uploads-to-gcs-s3-fail-if-the-chunk-size-is-greater-than-the-max-part-size/44349/
See: https://issuetracker.google.com/issues/323465186
2024-02-08 14:53:30 +00:00
..
gen_setfrom.go s3: refactor to use generated code instead of reflection to copy structs 2022-08-05 16:45:23 +01:00
s3_internal_test.go s3: fix no error being returned when creating a bucket we don't own 2023-10-09 18:15:02 +01:00
s3_test.go s3: S3 IPv6 support with option "use_dual_stack" (bool) 2023-12-08 11:11:47 +00:00
s3.go s3: GCS provider: fix server side copy of files bigger than 5G 2024-02-08 14:53:30 +00:00
setfrom.go s3: refactor to use generated code instead of reflection to copy structs 2022-08-05 16:45:23 +01:00
v2sign.go s3: v3sign: add missing subresource delete 2023-05-14 11:25:52 +01:00