github
/
rclone
mirror of https://github.com/rclone/rclone
1
Fork 0
Code Releases Activity
rclone/backend
History
Nick Craig-Wood 98e81a6c2b azureblob: fix data corruption bug #7590 
It was reported that rclone copy occasionally uploaded corrupted data
to azure blob.

This turned out to be a race condition updating the block count which
caused blocks to be duplicated.

This bug was introduced in this commit in v1.64.0 and will be fixed in v1.65.2

0427177857 azureblob: implement OpenChunkWriter and multi-thread uploads #7056

This race only seems to happen if `--checksum` is used but can happen otherwise.

Unfortunately Azure blob does not check the MD5 that we send them so
despite sending incorrect data this corruption is not detected. The
corruption is detected when rclone tries to download the file, so
attempting to copy the files back to local disk will result in errors
such as:

    ERROR : file.pokosuf5.partial: corrupted on transfer: md5 hash differ "XXX" vs "YYY"

This adds a check to test the blocklist we upload is as we expected
which would have caught the problem had it been in place earlier.
 		2024-01-24 11:46:47 +00:00
..
alias build: add package comments to silence revive linter 2022-08-28 13:43:51 +02:00
all imagekit: Added ImageKit backend 2023-11-24 18:18:01 +00:00
amazonclouddrive build: reformat comments to pass go1.19 vet 2022-08-05 16:35:41 +01:00
azureblob azureblob: fix data corruption bug #7590 2024-01-24 11:46:47 +00:00
azurefiles azure-files: fix storage base url 2024-01-05 16:19:19 +00:00
b2 b2: fix streaming chunked files an exact multiple of chunk size 2023-11-24 14:32:01 +00:00
box box: fix performance problem reading metadata for single files 2023-11-20 18:07:05 +00:00
cache cache: fix integration tests by adding missing OpenChunkWriter exclude 2023-09-04 19:26:14 +01:00
chunker chunker,compress,crypt,hasher,union: fix rclone move a file over itself deleting the file 2024-01-05 16:19:19 +00:00
combine build: fix lint errors when re-enabling revive exported & package-comments 2023-08-29 13:03:13 +01:00
compress chunker,compress,crypt,hasher,union: fix rclone move a file over itself deleting the file 2024-01-05 16:19:19 +00:00
crypt chunker,compress,crypt,hasher,union: fix rclone move a file over itself deleting the file 2024-01-05 16:19:19 +00:00
drive drive: fix integration tests by enabling metadata support from the context 2023-11-19 12:48:27 +00:00
dropbox dropbox: fix used space on dropbox team accounts 2024-01-05 16:19:19 +00:00
fichier fichier: fix error code parsing 2023-08-11 00:47:01 +09:00
filefabric filefabric: use atomic types 2023-08-22 12:52:13 +01:00
ftp ftp: fix 425 "TLS session of data connection not resumed" errors 2023-08-18 14:44:13 +01:00
googlecloudstorage gcs: fix 400 Bad request errors when using multi-thread copy 2023-11-20 18:07:05 +00:00
googlephotos googlephotos: fix nil pointer exception when batch failed 2024-01-05 16:19:19 +00:00
hasher hasher: fix invalid memory address error when MaxAge == 0 2024-01-05 16:19:19 +00:00
hdfs hdfs: added support for list of namenodes in hdfs remote config 2023-11-13 15:55:52 +00:00
hidrive hidrive: remove unused code (fixes issue reported by the unused linter) 2023-03-26 14:28:15 +02:00
http http: implement set backend command to update running backend 2023-11-20 18:07:05 +00:00
imagekit imagekit: Added ImageKit backend 2023-11-24 18:18:01 +00:00
internetarchive fs: implement --metadata-mapper to transform metatadata with a user supplied program 2023-11-18 17:49:35 +00:00
jottacloud fs: implement --metadata-mapper to transform metatadata with a user supplied program 2023-11-18 17:49:35 +00:00
koofr rclone config redacted: implement support mechanism for showing redacted config 2023-07-07 16:25:14 +01:00
linkbox linkbox: pre-merge fixes 2023-11-18 17:14:45 +00:00
local fs: implement --metadata-mapper to transform metatadata with a user supplied program 2023-11-18 17:49:35 +00:00
mailru rclone config redacted: implement support mechanism for showing redacted config 2023-07-07 16:25:14 +01:00
mega docs: fix typos found by codespell in docs and code comments 2023-09-23 12:20:01 +01:00
memory Replace deprecated ioutil 2022-11-07 11:41:47 +00:00
netstorage rclone config redacted: implement support mechanism for showing redacted config 2023-07-07 16:25:14 +01:00
onedrive onedrive: fix "unauthenticated: Unauthenticated" errors when uploading 2024-01-07 11:24:55 +00:00
opendrive opendrive: fix List on a just deleted and remade directory 2023-09-05 17:59:03 +01:00
oracleobjectstorage oracle object storage: fix object storage endpoint for custom endpoints 2024-01-05 16:19:19 +00:00
pcloud rclone config redacted: implement support mechanism for showing redacted config 2023-07-07 16:25:14 +01:00
pikpak docs: fix typos found by codespell in docs and code comments 2023-09-23 12:20:01 +01:00
premiumizeme rclone config redacted: implement support mechanism for showing redacted config 2023-07-07 16:25:14 +01:00
protondrive protondrive: make cached keys rclone style and not show with `rclone config redacted` 2023-09-11 15:57:08 +01:00
putio rclone config redacted: implement support mechanism for showing redacted config 2023-07-07 16:25:14 +01:00
qingstor rclone config redacted: implement support mechanism for showing redacted config 2023-07-07 16:25:14 +01:00
quatrix quatrix: overwrite files on conflict during server-side move 2023-11-16 17:14:00 +00:00
s3 s3: fix crash if no UploadId in multipart upload 2024-01-05 16:19:19 +00:00
seafile seafile: use atomic types 2023-08-22 12:52:13 +01:00
sftp sftp: implement --sftp-copy-is-hardlink to server side copy as hardlink 2023-10-16 12:08:22 +01:00
sharefile rclone config redacted: implement support mechanism for showing redacted config 2023-07-07 16:25:14 +01:00
sia rclone config redacted: implement support mechanism for showing redacted config 2023-07-07 16:25:14 +01:00
smb smb: fix shares not listed by updating go-smb2 2024-01-05 16:19:19 +00:00
storj storj: update storj.io/uplink to v1.12.0 2023-09-14 14:01:35 +01:00
sugarsync rclone config redacted: implement support mechanism for showing redacted config 2023-07-07 16:25:14 +01:00
swift swift: fix HEADing 0-length objects when --swift-no-large-objects set 2023-08-03 08:38:39 +01:00
union chunker,compress,crypt,hasher,union: fix rclone move a file over itself deleting the file 2024-01-05 16:19:19 +00:00
uptobox rclone config redacted: implement support mechanism for showing redacted config 2023-07-07 16:25:14 +01:00
webdav webdav: added an rclone vendor to work with rclone serve webdav 2023-11-05 12:37:25 +00:00
yandex yandex: fix 400 Bad Request on transfer failure 2023-06-11 11:11:43 +01:00
zoho zoho: remove Range requests workarounds to fix integration tests 2023-09-05 18:21:15 +01:00