Commit Graph

89 Commits

Author SHA1 Message Date
Nicola Murino
2f56375121 improve SFTP loop detection 2021-04-01 18:53:48 +02:00
Nicola Murino
183bedd6ed webui: add responsive extension 2021-03-28 11:02:11 +02:00
Nicola Murino
2a89a8f664 webui: minor improvements 2021-03-27 22:23:01 +01:00
Nicola Murino
9ad750da54 WebDAV: try to preserve the lock fs as much as possible 2021-03-27 19:10:27 +01:00
Nicola Murino
5e375f56dd kms: add a lock, secrets could be modified concurrently for cached users
also reduce the size of the JSON payload omitting empty secrets
2021-03-22 19:03:25 +01:00
Nicola Murino
54c0c1b80d Windows: manually check if we can bind on the configured port/ports
Windows allows the coexistence of three types of sockets on the same
transport-layer service port, for example, 127.0.0.1:8080, [::1]:8080
and [::ffff:0.0.0.0]:8080

Go don't properly handles this, so we use a ugly hack

Fixes #350
2021-03-21 22:21:04 +01:00
Nicola Murino
d6dc3a507e extend virtual folders support to all storage backends
Fixes #241
2021-03-21 19:15:47 +01:00
Nicola Murino
0286da2356 try to auto create virtual folders if missing 2021-03-10 22:30:56 +01:00
Nicola Murino
055506e518 sftpfs: add an option to disable concurrent reads 2021-03-06 15:41:40 +01:00
Nicola Murino
88122ba2f8 update jwtauth to v5 2021-03-05 18:50:45 +01:00
Nicola Murino
ca3e15578e Use new methods in the io and os packages instead of ioutil ones
ioutil is deprecated in Go 1.16 and SFTPGo is an application, not
a library, we have no reason to keep compatibility with old Go
versions.

Go 1.16 fix some cifs related issues too.
2021-02-25 21:53:04 +01:00
Nicola Murino
2146b83343 data providers: add filesystem to folder ...
... and some descriptive fields.
The filesystem support for virtual folders will be implemented in
future commits
2021-02-24 19:40:29 +01:00
Nicola Murino
3e1b07324d GCS: remove compat code 2021-02-22 22:06:23 +01:00
Nicola Murino
78a837e8f1 remove other compat code 2021-02-22 09:13:26 +01:00
Nicola Murino
be9230e85b micro optimizations spotted using the go-critic linter 2021-02-16 19:11:36 +01:00
Nicola Murino
46176a54b4 minor doc fixes 2021-02-14 22:08:08 +01:00
Nicola Murino
1ac66d27b6 Use IEC units for byte counting everywhere 2021-02-12 22:16:35 +01:00
Nicola Murino
51f110bc7b sftpd: add statvfs@openssh.com support 2021-02-11 19:45:52 +01:00
Nicola Murino
70bb3c34ce sftpfs: improve endpoint validation
Validation will fail if the endpoint is not specified as host:port
2021-02-03 11:29:04 +01:00
Nicola Murino
78bf808322 virtual folders: change dataprovider structure
This way we no longer depend on the local file system path and so we can
add support for cloud backends in future updates
2021-02-01 19:04:15 +01:00
Nicola Murino
ea096db8e4 sftpfs: set the correct file mode 2021-01-23 10:32:15 +01:00
Nicola Murino
0caeb68680 sftpfs: fix stat info 2021-01-23 09:42:49 +01:00
Giorgio Pellero
d42fcc3786 s3: don't paginate to find zero-byte-keyed dirs (#277)
Fixes #275
2021-01-14 12:01:25 +01:00
Nicola Murino
5d4f758c47 GCS: don't paginate to find compat "dirs" 2021-01-12 19:22:12 +01:00
Nicola Murino
2a95d031ea FTP: add support for AVBL command 2020-12-25 11:14:08 +01:00
Jochen Munz
5b1d8666b3 S3fs: Handle non-ascii filename in rename operations (#257)
SFTP is based on UTF-8 filenames, so non-ASCII filenames get transported with utf-8 escaped character sequences.
At least for the S3fs provider, if such a file is stored in a nested path it cannot be used as the source for a rename operations.

This adds the necessary escaping of the path fragments.

The patch is not required for MinIO but it doesn't hurt
2020-12-24 11:13:42 +01:00
Nicola Murino
7ab7941ddd sftpfs: fix race condition 2020-12-23 17:15:55 +01:00
Nicola Murino
f876c728ad add support for the latest ftpserverlib and azblob versions 2020-12-17 13:40:36 +01:00
Nicola Murino
a6985075b9 add sftpfs storage backend
Fixes #224
2020-12-12 10:31:09 +01:00
Nicola Murino
034d89876d webdav: fix proppatch handling
also respect login delay for cached webdav users and check the home dir as
soon as the user authenticates

Fixes #239
2020-12-06 08:19:41 +01:00
Nicola Murino
4a88ea5c03 add Data At Rest Encryption support 2020-12-05 13:48:13 +01:00
Nicola Murino
634b723b5d add KMS support
Fixes #226
2020-11-30 21:46:34 +01:00
Nicola Murino
99cd1ccfe5 S3: fix empty directory detection
when listing empty directory MinIO returns no contents while S3 returns
1 object with the key equal to the prefix. Make detection work in both
cases

Fixes #227
2020-11-23 15:36:42 +01:00
Nicola Murino
dccc583b5d add a dedicated struct to store encrypted credentials
also gcs credentials are now encrypted, both on disk and inside the
provider.

Data provider is automatically migrated and load data will accept
old format too but you should upgrade to the new format to avoid future
issues
2020-11-22 21:53:04 +01:00
Nicola Murino
ca0ff0d630 add a File interface so we can avoid to use os.File directly 2020-11-17 19:36:39 +01:00
Nicola Murino
1a6863f4b1 GCS uploads: check Close() error
some code simplification too
2020-11-13 18:40:18 +01:00
Nicola Murino
5720d40fee add setstat_mode 2
in this mode chmod/chtimes/chown can be silently ignored only for cloud
based file systems

Fixes #223
2020-11-12 10:39:46 +01:00
Nicola Murino
c6ab6f94e7 azblob: container level SAS cannot access container properties
so return the root directory without checking if the bucket exists
2020-11-05 15:03:35 +01:00
Nicola Murino
0a14297b48 webdav: performance improvements and bug fixes
we need my custom golang/x/net/webdav fork for now

https://github.com/drakkan/net/tree/sftpgo
2020-11-04 19:11:40 +01:00
Nicola Murino
6ad4cc317c cloud backends: stat and other performance improvements 2020-11-02 19:16:12 +01:00
Nicola Murino
fcfdd633f6 Azure Blob: update SDK and add access tier support 2020-10-30 22:17:17 +01:00
Nicola Murino
e54828a7b8 add metrics for Azure Blob storage 2020-10-26 19:01:17 +01:00
Nicola Murino
f2acde789d portable mode: add Azure Blob support 2020-10-25 21:42:43 +01:00
Nicola Murino
9b49f63a97 azure: implement multipart uploads using low level API
The high level wrapper seems to hang if there are network issues
2020-10-25 17:41:04 +01:00
Nicola Murino
14bcc6f2fc s3, azblob: check upper limit for part size 2020-10-25 12:10:11 +01:00
Nicola Murino
5ff8f75917 add Azure Blob support 2020-10-25 08:18:48 +01:00
Sean Hildebrand
db7e81e9d0 add prefer_database_credentials configuration parameter
When true, users' Google Cloud Storage credentials will be written to
the data provider instead of disk.
Pre-existing credentials on disk will be used as a fallback

Fixes #201
2020-10-22 10:42:40 +02:00
Nicola Murino
5eabaf98e0 gcs: remove a superfluous debug log 2020-09-29 09:17:08 +02:00
Nicola Murino
d1f0e9ae9f CGS: implement MimeTyper interface 2020-09-28 22:12:46 +02:00
Thomas Blommaert
cd56039ab7 GCS mime-type detection (#179)
Fixes #178
2020-09-28 21:52:18 +02:00