Mirrors/sftpgo
1
0
Fork 0
mirror of https://github.com/drakkan/sftpgo.git synced 2025-12-06 22:30:56 +03:00
Code Issues Packages Projects Releases Wiki Activity

s3: set upload concurrency to 2

Browse Source 
we upload a file while receiving it via SFTP not a file stored on a local
disk. We use concurrent uploads only to be able to send files of arbitrary
size, so concurrency is not really useful here. Setting the concurrency to
2 we have a max difference of 10 MB between the writer (sftp client) and
the reader (aws sdk), with the default concurrency value this difference
is 25MB.
This commit is contained in:
Nicola Murino
2020-02-04 23:14:55 +01:00
parent 8b039e0447
commit 9359669cd4
1 changed files with 2 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

2
vfs/s3fs.go
View File

@@ -188,6 +188,8 @@ func (fs S3Fs) Create(name string, flag int) (*os.File, *pipeat.PipeWriterAt, fu
Key: aws.String(key), Key: aws.String(key),
Body: r, Body: r,
StorageClass: utils.NilIfEmpty(fs.config.StorageClass), StorageClass: utils.NilIfEmpty(fs.config.StorageClass),
}, func(u *s3manager.Uploader) {
u.Concurrency = 2
}) })
r.CloseWithError(err) r.CloseWithError(err)
fsLog(fs, logger.LevelDebug, "upload completed, path: %#v, response: %v, readed bytes: %v, err: %v", fsLog(fs, logger.LevelDebug, "upload completed, path: %#v, response: %v, readed bytes: %v, err: %v",