add Data At Rest Encryption support

docs/account.md

@@ -50,7 +50,7 @@ For each account, the following properties can be configured:
   - `allowed_patterns`, list of, case insensitive, allowed file patterns. Examples: `*.jpg`, `a*b?.png`. Any non matching file will be denied
   - `denied_patterns`, list of, case insensitive, denied file patterns. Denied file patterns are evaluated before the allowed ones
   - `path`, exposed virtual path, if no other specific filter is defined, the filter apply for sub directories too. For example if filters are defined for the paths `/` and `/sub` then the filters for `/` are applied for any file outside the `/sub` directory
-- `fs_provider`, filesystem to serve via SFTP. Local filesystem (0), S3 Compatible Object Storage (1), Google Cloud Storage (2) and Azure Blob Storage (3) are supported
+- `fs_provider`, filesystem to serve via SFTP. Local filesystem (0), S3 Compatible Object Storage (1), Google Cloud Storage (2), Azure Blob Storage (3) and encrypted local filesystem (4) are supported
 - `s3_bucket`, required for S3 filesystem
 - `s3_region`, required for S3 filesystem. Must match the region for your bucket. You can find here the list of available [AWS regions](https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/using-regions-availability-zones.html#concepts-available-regions). For example if your bucket is at `Frankfurt` you have to set the region to `eu-central-1`
 - `s3_access_key`
@@ -74,6 +74,7 @@ For each account, the following properties can be configured:
 - `az_upload_concurrency`,  how many parts are uploaded in parallel. Zero means the default (2)
 - `az_key_prefix`,  allows to restrict access to the folder identified by this prefix and its contents
 - `az_use_emulator`, boolean
+- `crypt_passphrase`, passphrase to use for local encryption
 - `additional_info`, string. Free text field
 
 These properties are stored inside the data provider.

docs/dare.md

@@ -0,0 +1,19 @@
+# Data At Rest Encryption (DARE)
+
+SFTPGo supports data at-rest encryption via the `cryptfs` virtual file system, in this mode SFTPGo transparently encrypts and decrypts data (to/from the disk) on-the-fly during uploads and/or downloads, making sure that the files at-rest on the server-side are always encrypted.
+
+So, because of the way it works, as described here above, when you set up an encrypted filesystem for a user you need to make sure it points to an empty path/directory (that has no files in it). Otherwise, it would try to decrypt existing files that are not encrypted in the first place and fail.
+
+The SFTPGo's `cryptfs` is a tiny wrapper around [sio](https://github.com/minio/sio) therefore data is encrypted and authenticated using `AES-256-GCM` or `ChaCha20-Poly1305`. AES-GCM will be used if the CPU provides hardware support for it.
+
+The only required configuration parameter is a `passphrase`, each file will be encrypted using an unique, randomly generated secret key derived from the given passphrase using the HMAC-based Extract-and-Expand Key Derivation Function (HKDF) as defined in [RFC 5869](http://tools.ietf.org/html/rfc5869). It is important to note that the per-object encryption key is never stored anywhere: it is derived from your `passphrase` and a randomly generated initialization vector just before encryption/decryption. The initialization vector is stored with the file.
+
+The passphrase is stored encrypted itself according to your [KMS configuration](./kms.md) and is required to decrypt any file encrypted using an encryption key derived from it.
+
+The encrypted filesystem has some limitations compared to the local, unencrypted, one:
+
+- Upload resume is not supported.
+- Opening a file for both reading and writing at the same time is not supported and so clients that require advanced filesystem-like features such as `sshfs` are not supported too.
+- Truncate is not supported.
+- System commands such as `git` or `rsync` are not supported: they will store data unencrypted.
+- Virtual folders are not implemented for now, if you are interested in this feature, please consider submitting a well written pull request (fully covered by test cases) or sponsoring this development. We could add a filesystem configuration to each virtual folder so we can mount encrypted or cloud backends as subfolders for local filesystems and vice versa.

docs/portable-mode.md

@@ -41,6 +41,7 @@ Flags:
       --az-upload-part-size int         The buffer size for multipart uploads
                                         (MB) (default 4)
       --az-use-emulator
+      --crypto-passphrase string        Passphrase for encryption/decryption
       --denied-patterns stringArray     Denied file patterns case insensitive.
                                         The format is:
                                         /dir::pattern1,pattern2.
@@ -53,6 +54,7 @@ Flags:
                                         1 => AWS S3 compatible
                                         2 => Google Cloud Storage
                                         3 => Azure Blob Storage
+                                        4 => Encrypted local filesystem
       --ftpd-cert string                Path to the certificate file for FTPS
       --ftpd-key string                 Path to the key file for FTPS
       --ftpd-port int                   0 means a random unprivileged port,

docs/ssh-commands.md

@@ -9,6 +9,7 @@ For system commands we have no direct control on file creation/deletion and so t
 - we cannot avoid to leak real filesystem paths
 - quota check is suboptimal
 - maximum size restriction on single file is not respected
+- data at-rest encryption is not supported
 
  If quota is enabled and SFTPGo receives a system command, the used size and number of files are checked at the command start and not while new files are created/deleted. While the command is running the number of files is not checked, the remaining size is calculated as the difference between the max allowed quota and the used one, and it is checked against the bytes transferred via SSH. The command is aborted if it uploads more bytes than the remaining allowed size calculated at the command start. Anyway, we only see the bytes that the remote command sends to the local one via SSH. These bytes contain both protocol commands and files, and so the size of the files is different from the size transferred via SSH: for example, a command can send compressed files, or a protocol command (few bytes) could delete a big file. To mitigate these issues, quotas are recalculated at the command end with a full scan of the directory specified for the system command. This could be heavy for big directories. If you need system commands and quotas you could consider disabling quota restrictions and periodically update quota usage yourself using the REST API.