Commit Graph

154 Commits

Author SHA1 Message Date
Nicola Murino
0540b8780e redact credentials within hooks
go-retryablehttp does not redact credentials, so we still log them
when we use it

https://github.com/hashicorp/go-retryablehttp/pull/133
2021-05-12 22:44:17 +02:00
Nicola Murino
23d9ebfc91 add a basic front-end web interface for end-users
Fixes #339 #321 #398
2021-05-06 21:35:43 +02:00
Nicola Murino
1275328fdf Authentication errors: try to avoid user enumeration
Fixes #395
2021-04-26 19:48:21 +02:00
Nicola Murino
46998252e5 use bcrypt as default password hashing algo
argon2id has a high memory cost and, if not properly tuned, it can lead to
resource starvation.

Advanced users can still configure and use argon2id.
Passwords stored as argon2id will continue to work
2021-04-25 09:38:33 +02:00
Nicola Murino
92638ce93d add support for hashing password using bcrypt
argon2id remains the default
2021-04-20 13:55:09 +02:00
Nicola Murino
6ef85d6026 add, optional, in memory password caching
Verifying argon2 passwords has a high memory and computational cost,
by enabling, in memory, password caching you reduce this cost
2021-04-20 09:39:36 +02:00
Nicola Murino
2ffefbeb33 add sql_tables_prefix also to indexes and constraints
This allows you to reuse the same database for multiple SFTPGo instances

Fixes #372
2021-04-12 20:00:49 +02:00
Nicola Murino
c844fc7477 add support for delayed quota update
If there are a lot of close uploads, accumulating quota updates can
save you many queries to the data provider
2021-04-11 08:38:43 +02:00
Nicola Murino
fdf3f23df5 allow to disable some hooks on a per-user basis
This way you can, for example, mix external and internal users
2021-04-04 22:32:25 +02:00
Nicola Murino
2f56375121 improve SFTP loop detection 2021-04-01 18:53:48 +02:00
Nicola Murino
3bfd7e4d17 sftpfs: try to detect if an SFTP user point to itself
this will cause an infinite loop on login. The check should be improved
2021-03-29 21:53:44 +02:00
Nicola Murino
5cd27ce529 document Cockroach driver name 2021-03-27 19:41:00 +01:00
Nicola Murino
9ad750da54 WebDAV: try to preserve the lock fs as much as possible 2021-03-27 19:10:27 +01:00
Nicola Murino
5f49af1780 external auth: allow to inspect and preserve an existing user 2021-03-26 15:19:01 +01:00
Nicola Murino
70e035315e data provider: add CockroachDB support 2021-03-23 19:14:15 +01:00
Nicola Murino
5e375f56dd kms: add a lock, secrets could be modified concurrently for cached users
also reduce the size of the JSON payload omitting empty secrets
2021-03-22 19:03:25 +01:00
Nicola Murino
f7c7e2951d initialize argon params before creating the data provider
Fixes #349
2021-03-21 19:58:57 +01:00
Nicola Murino
d6dc3a507e extend virtual folders support to all storage backends
Fixes #241
2021-03-21 19:15:47 +01:00
Nicola Murino
df41f0c556 add a setting to skip natural keys validation
Enabling the "skip_natural_keys_validation" data provider setting,
the natural keys for REST API/Web Admin as usernames, admin names,
folder names are not restricted to unreserved URI chars

Fixes #334 #308
2021-03-04 09:48:53 +01:00
Nicola Murino
534b253c20 WebDAV: improve TLS certificate authentication
For each user you can now configure:

- TLS certificate auth
- TLS certificate auth and password
- Password auth

For TLS certificate auth, the certificate common name is used as
username
2021-03-01 19:28:11 +01:00
Nicola Murino
a6e36e7cad FTP: improve TLS certificate authentication
For each user you can now configure:

- TLS certificate auth
- TLS certificate auth and password
- Password auth

For TLS auth, the certificate common name must match the name provided
using the "USER" FTP command
2021-02-28 12:10:40 +01:00
Nicola Murino
ca3e15578e Use new methods in the io and os packages instead of ioutil ones
ioutil is deprecated in Go 1.16 and SFTPGo is an application, not
a library, we have no reason to keep compatibility with old Go
versions.

Go 1.16 fix some cifs related issues too.
2021-02-25 21:53:04 +01:00
Nicola Murino
2146b83343 data providers: add filesystem to folder ...
... and some descriptive fields.
The filesystem support for virtual folders will be implemented in
future commits
2021-02-24 19:40:29 +01:00
Nicola Murino
8cc2dfe5c2 update pkg/sftp
we don't need my branch anymore now that all the required features for
the sftpfs are available upstream too
2021-02-22 16:27:45 +01:00
Nicola Murino
49830516be squash database migrations and remove compat code 2021-02-22 08:37:50 +01:00
Nicola Murino
be9230e85b micro optimizations spotted using the go-critic linter 2021-02-16 19:11:36 +01:00
Nicola Murino
b1ce6eb85b web admin: allow to set an empty password for SFTPGo users 2021-02-15 19:38:53 +01:00
Nicola Murino
6a6e8fffbc web hooks: improve resilience by adding a configurable retry
the retryable http client is used for hooks that notify events
2021-02-12 21:42:49 +01:00
Nicola Murino
51f110bc7b sftpd: add statvfs@openssh.com support 2021-02-11 19:45:52 +01:00
Nicola Murino
db80781716 validation: improve error message for invalid chars 2021-02-08 21:32:59 +01:00
Nicola Murino
267d9f1831 web ui: allow to create folders from a template 2021-02-04 19:09:43 +01:00
Nicola Murino
78bf808322 virtual folders: change dataprovider structure
This way we no longer depend on the local file system path and so we can
add support for cloud backends in future updates
2021-02-01 19:04:15 +01:00
Nicola Murino
46ab8f8d78 post-login hook: add the full user JSON serialized
Fixes #284
2021-01-26 18:05:44 +01:00
Nicola Murino
54321c5240 web ui: allow to create multiple users from a template 2021-01-25 21:31:33 +01:00
Nicola Murino
778ec9b88f REST API v2
- add JWT authentication
- admins are now stored inside the data provider
- admin access can be restricted based on the source IP: both proxy
  header and connection IP are checked
- deprecate REST API CLI: it is not relevant anymore

Some other changes to the REST API can still happen before releasing
SFTPGo 2.0.0

Fixes #197
2021-01-17 22:29:08 +01:00
Nicola Murino
daac90c4e1 fix a potential race condition for pre-login and ext auth
hooks

doing something like this:

err = provider.updateUser(u)
...
return provider.userExists(username)

could be racy if another update happen before

provider.userExists(username)

also pass a pointer to updateUser so if the user is modified inside
"validateUser" we can just return the modified user without do a new
query
2021-01-05 09:50:22 +01:00
Nicola Murino
037d89a320 add support for a basic built-in defender
It can help to prevent DoS and brute force password guessing
2021-01-02 14:05:09 +01:00
Nicola Murino
a6985075b9 add sftpfs storage backend
Fixes #224
2020-12-12 10:31:09 +01:00
Nicola Murino
50982229e1 REST API: add a method to get the status of the services
added a status page to the built-in web admin
2020-12-08 11:18:34 +01:00
Nicola Murino
034d89876d webdav: fix proppatch handling
also respect login delay for cached webdav users and check the home dir as
soon as the user authenticates

Fixes #239
2020-12-06 08:19:41 +01:00
Nicola Murino
4a88ea5c03 add Data At Rest Encryption support 2020-12-05 13:48:13 +01:00
Nicola Murino
634b723b5d add KMS support
Fixes #226
2020-11-30 21:46:34 +01:00
Nicola Murino
224ce5fe81 add revertprovider subcommand
Fixes #233
2020-11-26 22:08:33 +01:00
Nicola Murino
2054dfd83d create the credential directory when needed
The credentials dir is currently required only for GCS users if
prefer database credential setting is false, so defer its creation
and don't fail to start the services if this directory is missing
2020-11-25 14:18:12 +01:00
Nicola Murino
6699f5c2cc initial data loading: an error is no longer fatal
therefore it does not prevent the services from starting
2020-11-25 09:18:36 +01:00
Nicola Murino
dccc583b5d add a dedicated struct to store encrypted credentials
also gcs credentials are now encrypted, both on disk and inside the
provider.

Data provider is automatically migrated and load data will accept
old format too but you should upgrade to the new format to avoid future
issues
2020-11-22 21:53:04 +01:00
Nicola Murino
e3eca424f1 web admin: allow both allowed and denied extensions/patterns for a dir
this fix a regression introduced in the previous commit
2020-11-16 19:21:50 +01:00
Nicola Murino
a6355e298e add support for limit files using shell like patterns
Fixes #209
2020-11-15 22:04:48 +01:00
Nicola Murino
0a14297b48 webdav: performance improvements and bug fixes
we need my custom golang/x/net/webdav fork for now

https://github.com/drakkan/net/tree/sftpgo
2020-11-04 19:11:40 +01:00
Nicola Murino
6ad4cc317c cloud backends: stat and other performance improvements 2020-11-02 19:16:12 +01:00