github/rclone
1
Fork 0
mirror of https://github.com/rclone/rclone synced 2024-11-25 02:47:14 +01:00
Code Releases Activity

s3: fix Cloudflare R2 integration tests after SDKv2 update #4989

Browse Source 
Cloudflare will normally automatically decompress files with
`Content-Encoding: gzip` when downloaded. This is not what AWS S3 does
and it breaks the integration tests.

This fudges the integration tests to upload the test file with
`Cache-Control: no-transform` on Cloudflare R2 and puts a note in the
docs about this problem.
This commit is contained in:
Nick Craig-Wood 2024-08-06 11:04:40 +01:00
parent cb611b8330
commit 8aef1de695
2 changed files with 18 additions and 1 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

10
backend/s3/s3_internal_test.go
View File

@ -60,6 +60,16 @@ func (f *Fs) InternalTestMetadata(t *testing.T) {
// "tier" - read only
// "btime" - read only
}
// Cloudflare insists on decompressing `Content-Encoding: gzip` unless
// `Cache-Control: no-transform` is supplied. This is a deviation from
// AWS but we fudge the tests here rather than breaking peoples
// expectations of what Cloudflare does.
//
// This can always be overridden by using
// `--header-upload "Cache-Control: no-transform"`
if f.opt.Provider == "Cloudflare" {
metadata["cache-control"] = "no-transform"
}
obj := fstests.PutTestContentsMetadata(ctx, t, f, &item, true, contents, true, "text/html", metadata)
defer func() {
assert.NoError(t, obj.Remove(ctx))

9
docs/content/s3.md
View File

@ -2388,7 +2388,14 @@ acl = private
Now run `rclone lsf r2:` to see your buckets and `rclone lsf
r2:bucket` to look within a bucket.
For R2 tokens with the "Object Read & Write" permission, you may also need to add `no_check_bucket = true` for object uploads to work correctly.
For R2 tokens with the "Object Read & Write" permission, you may also
need to add `no_check_bucket = true` for object uploads to work
correctly.
Note that Cloudflare decompresses files uploaded with
`Content-Encoding: gzip` by default which is a deviation from what AWS
does. If this is causing a problem then upload the files with
`--header-upload "Cache-Control: no-transform"`
### Dreamhost