skip availability check when updating the node timestamp.
If the update succeeds, the data provider is healthy
Signed-off-by: Nicola Murino <nicola.murino@gmail.com>
We replaced the jwtauth and jwx libraries with a minimal custom wrapper
around go-jose because we don’t need the full feature set provided by jwx.
Implementing our own wrapper simplifies the codebase and improves
maintainability.
Moreover, go-jose depends only on the standard library, resulting in a
leaner dependency that still meets all our requirements.
This change also reduces the SFTPGo binary size by approximately 1MB
Signed-off-by: Nicola Murino <nicola.murino@gmail.com>
rsync was executed as an external command, which means we have no insight
into or control over what it actually does.
From a security perspective, this is far from ideal.
To be clear, there's nothing inherently wrong with rsync itself. However,
if we were to support it properly within SFTPGo, we would need to implement
the low-level protocol internally rather than relying on launching an external
process. This would ensure it works seamlessly with any storage backend,
just as SFTP does, for example.
We recommend using one of the many alternatives that rely on the SFTP
protocol, such as rclone
Signed-off-by: Nicola Murino <nicola.murino@gmail.com>
The Azure Trusted Signing certificate is expiring soon, and renewal is no
longer available for individuals or organizations outside of Canada and USA.
Due to this limitation, we are removing the Trusted Signing step from our
CI pipeline.
Signed-off-by: Nicola Murino <nicola.murino@gmail.com>
EnabledCommands are initialized after the migration so allow any
command, they will be denied if not allowed and this is temporary.
The migration will be removed in the future
Signed-off-by: Nicola Murino <nicola.murino@gmail.com>
Azure login tokens now appear to expire after 5 minutes.
To avoid authentication issues, the login step is now performed
immediately before signing the binaries.
Signed-off-by: Nicola Murino <nicola.murino@gmail.com>
- Updated the "Sponsors" section to reflect the current open-core model
- Clarified that sponsorship supports the open-source edition
- Improved "Support" section to distinguish community vs. Enterprise support
- Added instructions on selecting the correct documentation version
Signed-off-by: Nicola Murino <nicola.murino@gmail.com>
As per the documentation for external authentication, provider actions
should not be executed post-authentication.
Signed-off-by: Nicola Murino <nicola.murino@gmail.com>
Folders and groups now retain their initial order, improving compatibility
and predictability when used with Terraform
Signed-off-by: Nicola Murino <nicola.murino@gmail.com>
Git integration has been removed as it is out of scope for a file transfer
solution like SFTPGo.
Maintaining Git support introduces unnecessary complexity and potential
security risks due to reliance on system commands.
In particular, allowing Git operations could enable authorized users to
upload repositories containing hooks, which might then be executed and abused.
To reduce the attack surface and simplify the codebase, Git support has been
fully dropped.
Signed-off-by: Nicola Murino <nicola.murino@gmail.com>
Data retention is now managed via the EventManager, introduced in v2.4.0.
This allows scheduling retention checks and sending email or HTTP notifications,
making the old API redundant.
Signed-off-by: Nicola Murino <nicola.murino@gmail.com>
Avoid clearing the file list every time the modal is opened, as
the modal might be closed unintentionally (e.g., by clicking
outside it)
Fixes#1981
Signed-off-by: Nicola Murino <nicola.murino@gmail.com>
removed legacy workaround code that was likely introduced to mask
a bug in the DataTable component.
This underlying issue has since been resolved and this code cause
issues now.
Fixes#1971
Signed-off-by: Nicola Murino <nicola.murino@gmail.com>
Some language translations may contain longer text, requiring
additional space in the menu
Fixes#1972
Signed-off-by: Nicola Murino <nicola.murino@gmail.com>
This improves compatibility with external authentication providers that
allow such characters in passwords.
Passwords created via the WebAdmin UI are still sanitized to prevent user
confusion.
Signed-off-by: Nicola Murino <nicola.murino@gmail.com>
SFTPGo is not designed to be used as an OpenSSH subsystem — many
features do not work correctly in subsystem mode. The functionality
was added after a user request in the pkg/sftp repository to
demonstrate that it was feasible, not for actual practical use.
Signed-off-by: Nicola Murino <nicola.murino@gmail.com>
Our permissions system for admin users is too granular and some
permissions overlap. For example, you can define an administrator
with the "manage_system" permission and not with the "manage_admins"
or "manage_user" permission, but the "manage_system" permission
allows you to restore a backup and then create users and
administrators. The following permissions will be removed:
"manage_admins", "manage_apikeys", "manage_system", "retention_checks",
"manage_event_rules", "manage_roles", "manage_ip_lists". Now you
need to add the "*" permission to replace the removed granular
permissions because the removed permissions allow actions that
should only be allowed to super administrators.
There is no point in having separate, overlapping permissions.
Signed-off-by: Nicola Murino <nicola.murino@gmail.com>
Failing to read from rand.Reader essentially can't happen, and if it
does is not possible to fallback securely, so just panic
Signed-off-by: Nicola Murino <nicola.murino@gmail.com>
Ugly hack to prevent to start uploading files before the upload
queue is fully populated.
We should investigate if there is a better way
Signed-off-by: Nicola Murino <nicola.murino@gmail.com>
token should never be null here because we have an authenticated user
however add the same check as elsewhere
Signed-off-by: Nicola Murino <nicola.murino@gmail.com>
we should special case path errors and replace the fs path with the
virtual path.
Thanks to @nezzzumi for reporting this issue
Signed-off-by: Nicola Murino <nicola.murino@gmail.com>
many clients rely on the version string to enable/disable some features.
We only allow to hide the version number, clients must be able to reliably
identify SFTPGo
Signed-off-by: Nicola Murino <nicola.murino@gmail.com>
we originally stored these values as MB but since we use bytes now,
an integer field is not enough.
Fixes#1575
Signed-off-by: Nicola Murino <nicola.murino@gmail.com>
Required permissions are now automatically granted as for any other
filesystem action
Fixes#1564
Signed-off-by: Nicola Murino <nicola.murino@gmail.com>
This change should fix the random failure in TestSCPTransferQuotaLimits
because the quota is already updated when the scp command ends.
Signed-off-by: Nicola Murino <nicola.murino@gmail.com>
also returns an error if file or directory names contain a slash
instead of silently replacing slashes with a similar symbol
Signed-off-by: Nicola Murino <nicola.murino@gmail.com>
the performance difference are no longer relavant.
We can restore this support if anyone reports performance regression
on any particular hardware
Signed-off-by: Nicola Murino <nicola.murino@gmail.com>
The -c flag is no longer used for SSH commands.
This is a backward incompatible change
Fixes#1423
Signed-off-by: Nicola Murino <nicola.murino@gmail.com>
if we don't have a global TLS configuration, advertise TLS only on the
bindings where it is configured instead of failing at runtime
Signed-off-by: Nicola Murino <nicola.murino@gmail.com>
* Expand reference to http clients in hooks docs
Before, it wasn't really clear where clients were configured and that their configuration also affected headers.
Signed-off-by: Bruce Weirdan <weirdan@gmail.com>
* Update more mentions of HTTP clients
Signed-off-by: Bruce Weirdan <weirdan@gmail.com>
---------
Signed-off-by: Bruce Weirdan <weirdan@gmail.com>
* update check for multiple inverse matches
Signed-off-by: David Stäheli <mistrdave@gmail.com>
* after match, direct return true
Signed-off-by: David Stäheli <mistrdave@gmail.com>
* apply same behaviour to checkEventGroupConditionPatterns
Signed-off-by: David Stäheli <mistrdave@gmail.com>
* fix spellmistake of function name
Signed-off-by: David Stäheli <mistrdave@gmail.com>
---------
Signed-off-by: David Stäheli <mistrdave@gmail.com>
so the service is stopped before the installation starts and
we avoid the force close app warning
Signed-off-by: Nicola Murino <nicola.murino@gmail.com>
The deny policy only applies to denied patterns, showing an allowed
pattern as hidden will confuse users
Signed-off-by: Nicola Murino <nicola.murino@gmail.com>
SQLite: remove AUTOINCREMENT from primary keys. It is not needed.
Postgres: switch from serial to identity for primary keys.
This means Postgres < 10 will not work in v2.6.x
Signed-off-by: Nicola Murino <nicola.murino@gmail.com>
start the countdown, let's see how long it takes for your favorite
Go-based proprietary SFTP server to notice this change, copy the SFTPGo
code and thus violate its license, and announce the same feature :)
Signed-off-by: Nicola Murino <nicola.murino@gmail.com>
This is probably something that changed in the recent datatables update,
before it was handled automatically
Signed-off-by: Nicola Murino <nicola.murino@gmail.com>
A structure similar to the one used for secrets would be better,
but we don't want to break backwards compatibility.
Also document that omitting the password field in the request body
will preserve the current password when updating a user using the
REST API. Added a test case for this.
Signed-off-by: Nicola Murino <nicola.murino@gmail.com>
This is a breaking change, but it is necessary to make JSON serialization of
users more compatible.
For example, Terraform does not allow JSON fields starting with numbers
Signed-off-by: Nicola Murino <nicola.murino@gmail.com>
This regression was introduced after recent changes to allow setting the SMTP
settings from the WebAdmin UI.
Fixes#1217
Signed-off-by: Nicola Murino <nicola.murino@gmail.com>
remove an unnecessary nil check in tlsutils added as workaround
to make telemetry test cases work
Signed-off-by: Nicola Murino <nicola.murino@gmail.com>
Setting configurations is an experimental feature and is not currently
supported in the REST API
Signed-off-by: Nicola Murino <nicola.murino@gmail.com>
this is a backward incompatible change, all previous file based IP/network
lists will not work anymore
Signed-off-by: Nicola Murino <nicola.murino@gmail.com>
This change is required to be able to set multiple domains for the same
certificate using env vars.
The change is backward compatible for general use cases but may be
backward incompatible in some edge cases, for example:
- "sftpgo.com,www.sftpgo.com" will work as before
- "sftpgo.com, www.sftpgo.com" will not work anymore
Check the logs to see if you are affected and rename the certificate and key
to fix
Signed-off-by: Nicola Murino <nicola.murino@gmail.com>
Useful in restricted environments where commands like curl and such
are not available.
Fixes#1129
Signed-off-by: Nicola Murino <nicola.murino@gmail.com>
the OpenAPI docs should really be improved, but nobody seems interested
enough to sponsor this work
Fixes#1141
Signed-off-by: Nicola Murino <nicola.murino@gmail.com>
I do not know which version removed /metrics from the HTTP server, but it does not seem to be available in 2.4.2, so I updated the metrics documentation to reflect this. Replaced with links to telemetry configuration.
Signed-off-by: Jon Bendtsen <github@jonb.dk>
Dockers default grace period is only 10 seconds, so added a warning to alert users to those cases where their SFTPGO_GRACE_TIME is larger than the docker grace
This introduce a backward incompatible change for filesystem path matching
in the Event Manager, now patterns like "*.txt" will no longer match any
file with the "txt" suffix, you need to change them to "/**/*.txt".
Also change pre-delete behaviour, now if an error is returned the client
will get a permission denied error. This is the same as the other pre-*
action. Previously it was not possible to deny deletion of a file.
Signed-off-by: Nicola Murino <nicola.murino@gmail.com>
WebAdmin does not work properly is trimming trailing and leading white
spaces is disabled
Fixes#1119
Signed-off-by: Nicola Murino <nicola.murino@gmail.com>
* Added graceful shutdown description to docker
Describing how to use the graceful shutdown period in a docker SFTPGO container and giving some examples of what happens with both existing and new connections.
Signed-off-by: Jon Bendtsen <github@jonb.dk>
we use PUT verb not PATCH. We keep merging only to allow to preserve
hidden/encrypted fields.
This is a backward incompatible change, but is necessary to avoid unexpected
issues.
You have to pass complete objects on updates.
Fixes#1088
Signed-off-by: Nicola Murino <nicola.murino@gmail.com>
* Added charset=utf-8 to application/json content type
This change is linked to https://github.com/drakkan/sftpgo/issues/1101 and should partially alleviate the need to change the content type in the files generated by openapi-generator-cli
Signed-off-by: Jon Bendtsen <github@jonb.dk>
* extra newline
Signed-off-by: Jon Bendtsen <github@jonb.dk>
* Signed-off-by: Jon Bendtsen github@jonb.dk
Signed-off-by: Jon Bendtsen github@jonb.dk
Signed-off-by: Jon Bendtsen <github@jonb.dk>
* This change is linked to #1101 and should partially alleviate the need to change the content type in the files generated by openapi-generator-cli.
Signed-off-by: Jon Bendtsen <github@jonb.dk>
Signed-off-by: Jon Bendtsen <github@jonb.dk>
Signed-off-by: Jon Bendtsen github@jonb.dk
Performance improves if there are many active connections.
For a few connections there is a small (unnoticeable) performance
degradation
Signed-off-by: Nicola Murino <nicola.murino@gmail.com>
add groups section in the getting started guide.
Suggest to prefer configuration with env vars instead of modifying
the default configuration file
Signed-off-by: Nicola Murino <nicola.murino@gmail.com>
This makes it easier to set environment variables on some operating systems.
Setting configuration options from environment variables is recommended if
you want to avoid the time-consuming task of merging your changes with the
default configuration file after upgrading SFTPGo
Signed-off-by: Nicola Murino <nicola.murino@gmail.com>
now that the event manager can create files, self connections may create
even more issues than before
Signed-off-by: Nicola Murino <nicola.murino@gmail.com>
they will be converted to the configured password hashing algorithm after
the first user login
Fixes#1000
Signed-off-by: Nicola Murino <nicola.murino@gmail.com>
this allows you to define a group deny policy that can be overridden
on a per-user basis.
This is a backward incompatible change
Signed-off-by: Nicola Murino <nicola.murino@gmail.com>
Behind a load balancer with no sticky sessions enabled is not enough to check
the home dir only when the client logs in
Signed-off-by: Nicola Murino <nicola.murino@gmail.com>
The old log-verbose flag is not appropriate anymore.
You should now use the log-level flag to set your preferred log level.
The default level is "debug" as before, you can also set "info", "warn",
"error"
Signed-off-by: Nicola Murino <nicola.murino@gmail.com>
... and to better explain why they are required.
Please don't say "someone else will help the project, I'll just use it"
Signed-off-by: Nicola Murino <nicola.murino@gmail.com>
Remove AWS SDK Transport ResponseHeaderTimeout (finer-grained timeout are already handled by the callers)
Lower the threshold for MultipartCopy (5GB -> 500MB) to improve copy performance and reduce chance of hitting Single part copy timeout
Fixes#898
Signed-off-by: Maxime Thébault <contact@maximethebault.me>
so we can persist password reset codes, OIDC auth sessions and tokens.
These features will also work in multi-node setups without sicky
sessions now
Signed-off-by: Nicola Murino <nicola.murino@gmail.com>
For historical reasons we send the json serialized user as a string field.
I Initially copied the code used in the script hook where it is appropriate
to convert the JSON user to string.
After some time I have noticed this error, I know that changing it now might
break existing external authentication hooks but we cannot continue with
this mistake, new users are surprised by this behavior, sorry
Fixes#836
Signed-off-by: Nicola Murino <nicola.murino@gmail.com>
so migrations cannot be executed concurrently if you run them from multiple
SFTPGo instances at the same time.
CockroachDB doesn't support database-level locks
Signed-off-by: Nicola Murino <nicola.murino@gmail.com>
if the virtual quota limit is greater than the filesystem available space,
we need to return the filesystem limits
Signed-off-by: Nicola Murino <nicola.murino@gmail.com>
they are negotiated according to the order.
Restrictions are generally configured server side.
I want to avoid to expose other settings for now.
Fixes#817
Signed-off-by: Nicola Murino <nicola.murino@gmail.com>
this is the minimal implementation to allow mget and similar commands with
wildcards.
We only support wildcard for the last path level, for example:
- mget *.xml is supported
- mget dir/*.xml is supported
- mget */*.xml is not supported
Removed . and .. from FTP directory listing
Signed-off-by: Nicola Murino <nicola.murino@gmail.com>
Google Cloud Storage credentials are now always stored within the data
provider.
Added a migration to read credentials from disk and store them inside the
data provider.
After v2.3 we can also remove credentials_path
Signed-off-by: Nicola Murino <nicola.murino@gmail.com>
Using groups simplifies the administration of multiple accounts by
letting you assign settings once to a group, instead of multiple
times to each individual user.
Signed-off-by: Nicola Murino <nicola.murino@gmail.com>
there are better alternatives and rsync will only work on local
filesystem, but it can still be useful to some people
Signed-off-by: Nicola Murino <nicola.murino@gmail.com>
if defined only the listed IPs/networks can access the configured
services, all other client connections will be dropped before they
even try to authenticate
Signed-off-by: Nicola Murino <nicola.murino@gmail.com>
When an host is banned but the updated_time field is in the past the ui
didn't show the record.
Fixes#758
Signed-off-by: lucatiozzo91 <luca.tiozzo91@gmail.com>
... and deprecate this setting.
In the future we'll remove prefer_database_credentials and
credentials_path and we will not allow the credentials to be saved on
the filesystem
Signed-off-by: Nicola Murino <nicola.murino@gmail.com>
Automatic backup are enabled by default, a new backup will be saved
each day at midnight.
The backups_path setting was moved from the httpd section to the
data_provider one, please adjust your configuration file and or your
env vars
Signed-off-by: Nicola Murino <nicola.murino@gmail.com>
The purpose of this code is to prevent anyone who can access to
the initial setup screen from creating an admin user
Signed-off-by: Nicola Murino <nicola.murino@gmail.com>
before this patch we allow a rename in the following cases:
- the user has rename permission on both source and target path
- the user has delete permission on source path and create/upload on
target path
we now check only the rename/rename_files/rename_dirs permissions.
This is what SFTPGo users expect.
This is a backward incompatible change and it will not backported to
the 2.2.x branch
Signed-off-by: Nicola Murino <nicola.murino@gmail.com>
The check could be expensive with some backends and is generally
only required the first time that a user logs in
Signed-off-by: Nicola Murino <nicola.murino@gmail.com>
we know if the client asks to remove a file or directory so let's
use the appropriate command without letting the sftp library guess
the appropriate behavior
Signed-off-by: Nicola Murino <nicola.murino@gmail.com>
if you share a single folder with read scope, you can now browse the share
and download single files
Fixes#674
See #677
Signed-off-by: Nicola Murino <nicola.murino@gmail.com>
The updated version avoid to always create a socket connected to the
journald on application start.
Now the socket is only created if we log to the journald
Signed-off-by: Nicola Murino <nicola.murino@gmail.com>
naming rules allow to support case insensitive usernames, trim trailing
and leading white spaces, and accept any valid UTF-8 characters in
usernames.
If you were enabling `skip_natural_keys_validation` now you need to
set `naming_rules` to `1`
Fixes#687
Signed-off-by: Nicola Murino <nicola.murino@gmail.com>
with total limit or separate settings for uploads and downloads and
overrides based on the client's IP address.
Limits can be reset using the REST API
Signed-off-by: Nicola Murino <nicola.murino@gmail.com>
301 MovedPermanently is cached by the browser which can
be annoying when it is is on base path like / while one
may reuse the domain (e.g. localhost) for other apps/tests.
Fixes#695
Signed-off-by: Jeremy Clerc <jeremy@clerc.io>
Disallowed files/dirs can be completly hidden. This may cause performance
issues for large directories
Signed-off-by: Nicola Murino <nicola.murino@gmail.com>
We can tighten security by adding the following to
the systemd service file:
* NoNewPrivileges: should never be needed
* DevicePolicy: only basics required
* PrivateDevices: only needs mounted stuff, never devs
* ProtectSystem: no need to change boot
* RestrictAddressFamilies: INET, UNIX only
Signed-off-by: Marc <mail@lpcvoid.com>
the clone button is not needed anymore, you can select a user and
click on template to generate one or more similar users or you can
create users/folders from an empty template
Signed-off-by: Nicola Murino <nicola.murino@gmail.com>
The page to add/edit users should be less less intimidating now.
All the advanced settings are hidden by default. Permissions are set
to any, so if you also have a users base dir set, to add a user
you have to simply set username, password or public key and save
Signed-off-by: Nicola Murino <nicola.murino@gmail.com>
if a user is updated using pre-login or external auth hook we need to
preserve the MFA related configs in the same way we do if the user is
updated using the REST API
so enabling debug level is not required, for example only to understand
that a user exceeded the allowed sessions.
Also set the cache update frequency as documented
In order to reset the password from the admin/client user interface,
an SMTP configuration must be added and the user/admin must have an email
address.
You can prohibit the reset functionality on a per-user basis by using a
specific restriction.
Fixes#597
SFTPGo is now fully auditable, all fs and provider events that change
something are notified and can be collected using hooks/plugins.
There are some backward incompatible changes for command hooks
Support 'virtual' users who have no homeDirectory, uidNumber or gidNumber.
Permit read-only access by a user named "anonymous", with any password.
Assume a conventional DIT with users under ou=people,dc=example,dc=com.
Read the LDAP bindPassword from a file (not baked into the code).
Log progress and problems to syslog.
building docker images now takes too long and often fails with random
errors. I have to restart the build several times to be able to push
the images to docker hub and gcr
it is quite common for HTTP clients to send a first request without
the Authorization header and then send the credentials after receiving
a 401 response. We don't want to generate defender events in this case
The builtin two-factor authentication is based on time-based one time
passwords (RFC 6238) which works with Authy, Google Authenticator and
other compatible apps.
you need to load initial data or set "create_default_admin" to true
and the appropriate env vars if you don't want to use the web admin
setup screen to create the default admin
This hack will no longer be needed once Ceph tags a new version and vendors
using it update their servers.
This code is taken from rclone, thank you!
Fixes#483
We now return 552 code for quota exceeded errors and 553 in the following
cases:
- filename denied by a filter
- no upload permission
- no overwrite permission
- pre upload hook error
Fixes#442
These new APIs match the web client features.
I'm aware that some API do not follow REST best practises.
I want to avoid things likes "/user/folders/<path>"
where "path" must be encoded and making it optional create issues, so
I defined resources as query parameters instead of path parameters
combined with server side processing I can now list a directory with
about 100.000 files in less than 2 seconds without losing client side
filtering and pagination
WinSCP does not set the APPEND flag while resuming a file upload,
so we detect a file upload resume if the TRUNCATE flag is not set.
The APPEND flag is now ignored.
Fixes#420
If you prefer to auto-create the first admin you can enable the
"create_default_admin" configuration key and SFTPGo will work as before.
You can also create the first admin by loading initial data: now you can
set both username and password, before you could only change the password
The actions to run synchronously can be configured via the `execute_sync`
configuration key.
Executing an action synchronously means that SFTPGo will not return a result
code to the client until your hook have completed its execution.
Fixes#409
X-Forwarded-For, X-Real-IP and X-Forwarded-Proto headers will be ignored
for hosts not included in this list.
This is a backward incompatible change, before the proxy headers were
always used
argon2id has a high memory cost and, if not properly tuned, it can lead to
resource starvation.
Advanced users can still configure and use argon2id.
Passwords stored as argon2id will continue to work
currently xgo don't allow to choose the building OS, this could cause
unexpected issues, for example v2.0.3 packages for arm64 and ppc64
don't run on Ubuntu 18.04
SFTPGo requires that the user's home directory, virtual folder root,
and intermediate paths to virtual folders exist to work properly.
If you already know that the required directories exist, disabling
these checks will speed up login.
Windows allows the coexistence of three types of sockets on the same
transport-layer service port, for example, 127.0.0.1:8080, [::1]:8080
and [::ffff:0.0.0.0]:8080
Go don't properly handles this, so we use a ugly hack
Fixes#350
on Windows x509.SystemCertPool is not implemented and therefore we end
uo with an empty certificate pool if we load the CA certificates
unconditionally
Enabling the "skip_natural_keys_validation" data provider setting,
the natural keys for REST API/Web Admin as usernames, admin names,
folder names are not restricted to unreserved URI chars
Fixes#334#308
For each user you can now configure:
- TLS certificate auth
- TLS certificate auth and password
- Password auth
For TLS certificate auth, the certificate common name is used as
username
For each user you can now configure:
- TLS certificate auth
- TLS certificate auth and password
- Password auth
For TLS auth, the certificate common name must match the name provided
using the "USER" FTP command
ioutil is deprecated in Go 1.16 and SFTPGo is an application, not
a library, we have no reason to keep compatibility with old Go
versions.
Go 1.16 fix some cifs related issues too.
This way data and backup dirs can be mounted as separate volumes.
Based on the proof of concept submitted by
Mark Sagi-Kazar <mark.sagikazar@gmail.com>
See #305
The underlying golang webdav library returns Method Not Allowed for
HEAD requests on directories:
https://github.com/golang/net/blob/master/webdav/webdav.go#L210
let's see if we can workaround this inside SFTPGo itself in a similar
way as we do for GET.
The HEAD response will not return a Content-Length, we cannot handle
this inside SFTPGo.
Fixes#294
- add JWT authentication
- admins are now stored inside the data provider
- admin access can be restricted based on the source IP: both proxy
header and connection IP are checked
- deprecate REST API CLI: it is not relevant anymore
Some other changes to the REST API can still happen before releasing
SFTPGo 2.0.0
Fixes#197
hooks
doing something like this:
err = provider.updateUser(u)
...
return provider.userExists(username)
could be racy if another update happen before
provider.userExists(username)
also pass a pointer to updateUser so if the user is modified inside
"validateUser" we can just return the modified user without do a new
query
This test cannot work using memory provider, we cannot change the provider
for a kms secrete without reloading it from JSON and the memory provider
will never reload users
- allow to disable active mode
- allow to disable SITE commands
- add optional support for calculating hash value of files
- add optional support for the non standard COMB command
SFTP is based on UTF-8 filenames, so non-ASCII filenames get transported with utf-8 escaped character sequences.
At least for the S3fs provider, if such a file is stored in a nested path it cannot be used as the source for a rename operations.
This adds the necessary escaping of the path fragments.
The patch is not required for MinIO but it doesn't hurt
Config name is a Viper concept used for searching a specific file
in various paths with various extensions.
Making it configurable is usually not a useful feature
as users mostly want to define a full or relative path
to a config file.
This change replaces config name with config file.
The credentials dir is currently required only for GCS users if
prefer database credential setting is false, so defer its creation
and don't fail to start the services if this directory is missing
when listing empty directory MinIO returns no contents while S3 returns
1 object with the key equal to the prefix. Make detection work in both
cases
Fixes#227
also gcs credentials are now encrypted, both on disk and inside the
provider.
Data provider is automatically migrated and load data will accept
old format too but you should upgrade to the new format to avoid future
issues
Include this patch:
https://github.com/pkg/sftp/pull/393
to avoid request accumulation (no underlying fd) if we return an error.
Before this patch the accumulated requests are released only when the
client disconnects.
We use our fork for now to include
https://github.com/pkg/sftp/pull/392
too
remove the VOLUME instruction from the Dockerfile so you can change
user using your own image like this:
FROM drakkan/sftpgo:tag
USER root
RUN chown -R 1100:1100 /etc/sftpgo && chown 1100:1100 /var/lib/sftpgo /srv/sftpgo
USER 1100:1100
When true, users' Google Cloud Storage credentials will be written to
the data provider instead of disk.
Pre-existing credentials on disk will be used as a fallback
Fixes#201
This way you can force the user to login again and so to use the updated
configuration.
A deleted user will be automatically disconnected.
Fixes#163
Improved some docs too.
for Cloud FS the folders are virtual and they, generally, disappear when the
last file is removed.
This fix doesn't work for FTP protocol for now.
Fixes#149
It failed sometime due to a bug in the ftp client library used in test
cases. The failure was more frequent on FreeBSD but it could happen in
any supported OS. It was not systematic since we use small files in
test cases.
See https://github.com/jlaffaye/ftp/pull/192
The common package defines the interfaces that a protocol must implement
and contain code that can be shared among supported protocols.
This way should be easier to support new protocols
Allow to enable rename permission in a more controlled way granting "delete"
permission on source directory and "upload" permission on target directory
some external auth users want to map multiple login usernames with a single
SGTPGo account.
For example an SFTP user logins using "user1" or "user2" and the external auth
returns "user" in both cases, so we use the username returned from external auth
and not the one used to login
Fixes#125
host_key defines the private host keys as plain list of strings.
Remove the other deprecated config params from the default config too.
Signed-off-by: Nicola Murino <nicola.murino@gmail.com>
HTTP clients are used for executing hooks such as the ones used for custom
actions, external authentication and pre-login user modifications.
This allows, for example, to use self-signed certificate without defeating the
purpose of using TLS
The underlying filesystem errors for permissions and non-existing files
can contain the real storage path.
Map these errors to more generic ones to avoid to leak this info
Fixes#109
This way we can import the default passwords format used in 389ds.
See TestPasswordsHashPbkdf2Sha256_389DS test case to learn how to convert
389ds passwords
The HTTP notifications are now invoked as POST and the notification is
a JSON inside the POST body.
This is a backward incompatible change but this way the actions can be
extended more easily, sorry for the trouble
Fixes#101
profiling is now available via the HTTP base URL /debug/pprof/
examples, use this URL to start and download a 30 seconds CPU profile:
/debug/pprof/profile?seconds=30
use this URL to profile used memory:
/debug/pprof/heap?gc=1
use this URL to profile allocated memory:
/debug/pprof/allocs?gc=1
Full docs here:
https://golang.org/pkg/net/http/pprof/
- use latest pkg/sftp that contains my latest performance patch
- replace default crypto with my branch that use minio sha256-simd
instead of Golang SHA256 implementation, this improve performance on
some hardware
Please note that if the upload bandwidth between the SFTP client and
SFTPGo is greater than the upload bandwidth between SFTPGo and S3 then
the SFTP client have to wait for the upload of the last parts to S3
after it ends the file upload to SFTPGo, and it may time out.
Keep this in mind if you customize parts size and upload concurrency
we try to remove the trailing double quote for user input such as this one
sftpgo.exe serve -c "C:\ProgramData\SFTPGO\"
the value for the -c flag is parsed as:
C:\ProgramData\SFTPGO"
this is what the user specified, but the user want this value:
C:\ProgramData\SFTPGO
so we try to remove the trailing double quote.
Please note that we cannot do anything for something like this:
-c "C:\ProgramData\SFTPGO\" -l "sftpgo.log"
in this case the -l flag will be ignored and the value for the c flag is:
C:\ProgramData\SFTPGO" -l sftpgo.log
and so probably it is invalid. This is definitely a bad user input
"proxy_allowed" setting allows to specify the allowed IP address and IP
ranges that can send the proxy header. This setting combined with
"proxy_protocol" allows to ignore the header or to reject connections
that send the proxy header from a non listed IP
scp now properly handles virtual folders.
rsync is disabled for users with virtual folders: we execute a system
command and it is not aware about virtual folders.
git is not allowed if the repo path is inside a virtual folder
A custom program can be executed before the users login to modify the
configurations for the user trying to login.
You can, for example, allow login based on time range.
Fixes#77
This simplify the common pattern where the user password and a one time
token is requested: now the external program can delegate password check
to SFTPGo and verify the token itself
some clients, for example rclone can request only part of a file, we have
no way to detect this so we haven't return an error if the downloaded size
does not match the file size
added the "initprovider" command to initialize the database structure.
If we change the database schema the required changes will be checked
at startup and automatically applyed.
HTTPS certificate can be reloaded on demand sending a SIGHUP signal on
Unix based systems and a "paramchange" request to the running service on
Windows
we upload a file while receiving it via SFTP not a file stored on a local
disk. We use concurrent uploads only to be able to send files of arbitrary
size, so concurrency is not really useful here. Setting the concurrency to
2 we have a max difference of 10 MB between the writer (sftp client) and
the reader (aws sdk), with the default concurrency value this difference
is 25MB.
We have to rework TestRelativePaths and TestResolvePaths if we want to run
them for Cloud Storage on Windows too: we use filesystem path while Cloud
Storage providers expect Unix paths.
On Windows is important to check the local filesystem so skip Cloud Storage
providers test cases for now
The `memory` provider can load users from a dump obtained using the
`dumpdata` REST API. This dump file can be configured using the
dataprovider `name` configuration key. It will be loaded at startup
and can be reloaded on demand using a `SIGHUP` on Unix based systems
and a `paramchange` request to the running service on Windows.
Fixes#66
Here are the main improvements:
- unliked files works on windows too
- the uploads are now synced on the lower speed between the SFTP client write
and the upload speed to S3
This commit increase the external auth timeout to 60 seconds too
... now that it contains all the needed patches.
Remove an hack for setstat with empty attrs, it is now handled in pkg/sftp.
Update other dependencies too.
Use os.Environ() as a base instead of empty variable. Currently the environment of executed external auth program only contains SFTPGO_AUTHD* variables and therefore the program lacks additional context when started.
Login can be restricted to specific ranges of IP address or to a specific IP
address.
Please apply the appropriate SQL upgrade script to add the filter field to your
database.
The filter database field will allow to add other filters without requiring a
new database migration
Sometime we can have this error:
read |0: file already closed
reading from the command standard error, this means that the command is
already finished so we don't need to do nothing.
This happen randomically while running the test cases on travis.
The git push test sometime fails when running on travis.
The issue cannot be replicated locally so print the logs to try to
understand what is happening
currently we support:
- Linux/Unix users stored in shadow/passwd files
- Pure-FTPd virtual users generated using `pure-pw` CLI
- ProFTPD users generated using `ftpasswd` CLI
using something like this:
update-user <user-id> <username> --public-keys ''
the public keys auth will be disabled
using something like this:
update-user <user-id> <username> --password ''
the password auth will be disabled
SysProcAttr.Credential is not available on Windows we need to move the
WrapCmd test in a separate file to be able to build test cases on Windows,
skipping the test is not enough
It was possible to remove an empty root dir or create a symlink to it.
We now return a Permission Denied error if we detect an attempt to remove,
renaming or symlinking the root directory
we can now have permissions such as these ones
{"/":["*"],"/somedir":["list","download"]}
The old permissions are automatically converted to the new structure,
no database migration is needed
It seems that there are some clients that sends Setstat requests with
no attrs:
https://github.com/pkg/sftp/issues/325
I haven't never seen this myself, anyway we now return ErrSSHFxBadMessage
and log the client version in such cases
and better document quota management issues for system commands.
rsync and git are not enabled in the default config so don't install
them in sample Dockerfiles, simply add a comment to facilitate their
installation if needed
Fixes#44
we only need to wait for the write from the local command to
the ssh channel. There is no need to wait for the write from ssh
channel to the local command stdin
We use the system commands "git-receive-pack", "git-upload-pack" and
"git-upload-archive". they need to be installed and in your system's
PATH. Since we execute system commands we have no direct control on
file creation/deletion and so quota check is suboptimal: if quota is
enabled, the number of files is checked at the command begin and not
while new files are created.
The allowed size is calculated as the difference between the max quota
and the used one. The command is aborted if it uploads more bytes than
the remaining allowed size calculated at the command start. Quotas are
recalculated at the command end with a full home directory scan, this
could be heavy for big directories.
md5sum, sha1sum are used by rclone.
cd, pwd improve the support for RemoteFiles mobile app.
These commands are all implemented inside SFTPGo so they work even
if the matching system commands are not available, for example on Windows
added matching permissions too and a new setting "setstat_mode".
Setting setstat_mode to 1 you can keep the previous behaviour that
silently ignore setstat requests
an user can now be disabled or expired.
If you are using an SQL database as dataprovider please remember to
execute the sql update script inside "sql" folder.
Fixes#57
- remove a non empty directory. Before: the directory contents were
removed recursively. Now: removing a non empty directory fails.
- make a directory in a non existent path: Before: any necessary parents
were created. Now: it fails.
- remove a file. Before: files, directories and symlinks were removed.
Now: only files and symlink are removed, removing a directory using "Remove"
instead of "Rmdir" fails.
Upload a file in a non existent directory. Before: any necessary parents
were created. Now: it fails.
Now SFTPGo behaves as OpenSSH.
we have not known bugs with the previous implementation anyway this one
is cleaner: the underlying network connection is directly related with
SFTP/SCP connections.
This should better protect us against buggy clients and edge cases
we support resume only if the client sets the correct offset while resuming
the upload.
Based on the specs the offset is optional for resume, but all the tested
clients sets a right offset.
If an invalid offset is given we interrupt the transfer with the error
"Invalid write offset ..."
See https://github.com/pkg/sftp/issues/295
This commit add a new upload mode: "atomic with resume support", this acts
as atomic but if there is an upload error the temporary file is renamed
to the requested path and not deleted, this way a client can reconnect
and resume the upload
The update is atomic so no transaction is needed.
Addionally a transaction will ask for a new connection to the pool
and this can deadlock if the pool has a max connection limit too low.
Also make configurable the pool size instead of hard code to the cpu number.
Fixes#47
The volume configuration is not required anymore, we have
a default config in /etc/sftpgo and the volume config is
mounted on /srv/sftpgo/config so we always have a default
config in /etc/sftpgo/sftpgo.json.
Anyway to persist server's keys after an image rebuild a
configuration volume is highly recommended
detect upload/download errors and don't execute actions if a transfer
error happen.
To detect SFTP errors this patch is needed:
https://github.com/pkg/sftp/pull/307
* created a "Log" function for type "Connection"
* created a "log" function for type "Provider"
* replace logger calls to Log/log where possible
I also renamed PGSSQL to PGSQL, as this seemed to be a typo
Signed-off-by: Jo Vandeginste <Jo.Vandeginste@kuleuven.be>
This will show the key fingerprint and the associated comment, or
"password" when password was used, during login.
Eg.:
```
message":"User id: 1, logged in with: \"public_key:SHA256:FV3+wlAKGzYy7+J02786fh8N8c06+jga/mdiSOSPT7g:jo@desktop\",
```
or
```
message":"User id: 1, logged in with: \"password\",
...`
Signed-off-by: Jo Vandeginste <Jo.Vandeginste@kuleuven.be>
viper will not use env vars if no configuration file is found
https://github.com/spf13/viper/issues/584
As workaround we could manually bind/set a default for each configuration
option using viper.SetDefault("key") and then generate a default config
using viper.Get("key").
This manual solution is error prone and it will become increasingly
difficult to maintain since the configuration options will grow, so
we avoid it for now.
Let's see if viper will solve this issue
Fixes#35
- sqlite3 sftpgo.db 'CREATE TABLE "users" ("id" integer NOT NULL PRIMARY KEY AUTOINCREMENT, "username" varchar(255) NOT NULL UNIQUE, "password" varchar(255) NULL, "public_keys" text NULL, "home_dir" varchar(255) NOT NULL, "uid" integer NOT NULL, "gid" integer NOT NULL, "max_sessions" integer NOT NULL, "quota_size" bigint NOT NULL, "quota_files" integer NOT NULL, "permissions" text NOT NULL, "used_quota_size" bigint NOT NULL, "used_quota_files" integer NOT NULL, "last_quota_update" bigint NOT NULL, "upload_bandwidth" integer NOT NULL, "download_bandwidth" integer NOT NULL);'
install:
- go get -v -t ./...
script:
- go test -v ./... -coverprofile=coverage.txt -covermode=atomic
[](https://github.com/avelino/awesome-go)
## Features
Full-featured and highly configurable event-driven file transfer solution.
Server protocols: SFTP, HTTP/S, FTP/S, WebDAV.
Storage backends: local filesystem, encrypted local filesystem, S3 (compatible) Object Storage, Google Cloud Storage, Azure Blob Storage, other SFTP servers.
- Each account is chrooted to his Home Dir.
- SFTP accounts are virtual accounts stored in a "data provider".
- SQLite, MySQL, PostgreSQL and bbolt (key/value store in pure Go) data providers are supported.
- Public key and password authentication. Multiple public keys per user are supported.
- Quota support: accounts can have individual quota expressed as max total size and/or max number of files.
- Bandwidth throttling is supported, with distinct settings for upload and download.
- Per user maximum concurrent sessions.
- Per user permissions: list directories content, upload, download, delete, rename, create directories, create symlinks can be enabled or disabled.
- Per user files/folders ownership: you can map all the users to the system account that runs SFTPGo (all platforms are supported) or you can run SFTPGo as root user and map each user or group of users to a different system account (*NIX only).
- Configurable custom commands and/or HTTP notifications on upload, download, delete or rename.
- Automatically terminating idle connections.
- Atomic uploads are configurable.
- Optional SCP support.
- REST API for users and quota management and real time reports for the active connections with possibility of forcibly closing a connection.
- Configuration is a your choice: JSON, TOML, YAML, HCL, envfile are supported.
- Log files are accurate and they are saved in the easily parsable JSON format.
With SFTPGo you can leverage local and cloud storage backends for exchanging and storing files internally or with business partners using the same tools and processes you are already familiar with.
## Platforms
The WebAdmin UI allows to easily create and manage your users, folders, groups and other resources.
SFTPGo is developed and tested on Linux. After each commit the code is automatically built and tested on Linux and macOS using Travis CI.
Regularly the test cases are manually executed and pass on Windows. Other UNIX variants such as *BSD should work too.
The WebClient UI allows end users to change their credentials, browse and manage their files in the browser and setup two-factor authentication which works with Microsoft Authenticator, Google Authenticator, Authy and other compatible apps.
## Requirements
## Sponsors
- Go 1.12 or higher.
- A suitable SQL server or key/value store to use as data provider: PostreSQL 9.4+ or MySQL 5.6+ or SQLite 3.x or bbolt 1.3.x
SFTPGo remains committed to open source. The core features are freely available and maintained.
If you rely on SFTPGo in your projects, consider becoming a [sponsor](https://github.com/sponsors/drakkan) to help ensure its long-term sustainability.
## Installation
Your sponsorship helps cover maintenance, security updates and ongoing development of the open-source edition.
Simple install the package to your [$GOPATH](https://github.com/golang/go/wiki/GOPATH "GOPATH") with the [go tool](https://golang.org/cmd/go/ "go command") from shell:
### Thank you to our sponsors
```bash
$ go get -u github.com/drakkan/sftpgo
```
#### Platinum sponsors
Make sure [Git is installed](https://git-scm.com/downloads) on your machine and in your system's `PATH`.
SFTPGo depends on [go-sqlite3](https://github.com/mattn/go-sqlite3) that is a CGO package and so it requires a `C` compiler at build time.
On Linux and macOS a compiler is easy to install or already installed, on Windows you need to download [MinGW-w64](https://sourceforge.net/projects/mingw-w64/files/) and build SFTPGo from its command prompt.
#### Silver sponsors
The compiler is a build time only dependency, it is not not required at runtime.
If you don't need SQLite, you can also get/build SFTPGo setting the environment variable `GCO_ENABLED` to 0, this way SQLite support will be disabled but PostgreSQL, MySQL and bbolt will work and you don't need a `C` compiler for building.
#### Bronze sponsors
Version info, such as git commit and build date, can be embedded setting the following string variables at build time:
For example you can build using the following command:
The open-source version of SFTPGo is free to use under the terms of its [license](#license). We are proud to provide a robust and fully functional edition that meets the needs of many production environments.
While we do not offer direct free support, community support is available. You can use [GitHub Discussions](https://github.com/drakkan/sftpgo/discussions) to ask questions, share feedback and engage with other users of the project.
and you will get a version that includes git commit and build date like this one:
If you require guaranteed support, expert guidance, or advanced features, consider using SFTPGo Enterprise: a commercially licensed edition of SFTPGo that extends the open source version with enterprise-only features and full support.
SFTPGo Enterprise is available in two deployment options:
For Linux, a systemd sample [service](https://github.com/drakkan/sftpgo/tree/master/init/sftpgo.service "systemd service") can be found inside the source tree.
- On-premises: Deploy on your own infrastructure with full control and commercial-grade support. More details: [sftpgo.com/on-premises](https://sftpgo.com/on-premises)
- Fully managed SaaS: Let us handle the infrastructure. Ideal for teams that want a secure, scalable, and maintenance-free setup with full support included. More details: [sftpgo.com/saas](https://sftpgo.com/saas)
Alternately you can use distro packages:
## Documentation
- Arch Linux PKGBUILD is available on [AUR](https://aur.archlinux.org/packages/sftpgo-git/ "SFTPGo")
You can explore all supported features and configuration options at [docs.sftpgo.com](https://docs.sftpgo.com/).
## Configuration
You can select the version you're using from the top-left corner of the documentation site.
If you're using the open-source edition, please make sure to switch to the corresponding documentation. The "Enterprise" docs, shown by default, include features that may only be available in the licensed edition or our SaaS offerings.
The `sftpgo` executable can be used this way:
## Internationalization
```bash
Usage:
sftpgo [command]
The translations are available via [Crowdin](https://crowdin.com/project/sftpgo), who have granted us an open source license.
Available Commands:
help Help about any command
serve Start the SFTP Server
Before start translating please take a look at our contribution [guidelines](https://sftpgo.github.io/latest/web-interfaces/#internationalization).
Flags:
-h, --help helpfor sftpgo
-v, --version
```
## Release Cadence
The `serve` subcommand supports the following flags:
SFTPGo follows a feature-driven release cycle rather than a fixed, time-based schedule. Currently, our primary development efforts are focused on the [Enterprise edition](https://docs.sftpgo.com/enterprise/#enterprise-edition), which benefits from a faster release cadence and receives major new features (see [changelog](https://docs.sftpgo.com/enterprise/changelog/)).
-`--config-dir` string. Location of the config dir. This directory should contain the `sftpgo` configuration file and is used as the base for files with a relative path (eg. the private keys for the SFTP server, the SQLite or bblot database if you use SQLite or bbolt as data provider). The default value is "." or the value of `SFTPGO_CONFIG_DIR` environment variable.
-`--config-file` string. Name of the configuration file. It must be the name of a file stored in config-dir not the absolute path to the configuration file. The specified file name must have no extension we automatically load JSON, YAML, TOML, HCL and Java properties. The default value is "sftpgo" (and therefore `sftpgo.json`, `sftpgo.yaml` and so on are searched) or the value of `SFTPGO_CONFIG_FILE` environment variable.
-`--log-compress` boolean. Determine if the rotated log files should be compressed using gzip. Default `false` or the value of `SFTPGO_LOG_COMPRESS` environment variable (1 or `true`, 0 or `false`).
-`--log-file-path` string. Location for the log file, default "sftpgo.log" or the value of `SFTPGO_LOG_FILE_PATH` environment variable.
-`--log-max-age` int. Maximum number of days to retain old log files. Default 28 or the value of `SFTPGO_LOG_MAX_AGE` environment variable.
-`--log-max-backups` int. Maximum number of old log files to retain. Default 5 or the value of `SFTPGO_LOG_MAX_BACKUPS` environment variable.
-`--log-max-size` int. Maximum size in megabytes of the log file before it gets rotated. Default 10 or the value of `SFTPGO_LOG_MAX_SIZE` environment variable.
-`--log-verbose` boolean. Enable verbose logs. Default `true` or the value of `SFTPGO_LOG_VERBOSE` environment variable (1 or `true`, 0 or `false`).
If you don't configure any private host keys, the daemon will use `id_rsa` in the configuration directory. If that file doesn't exist, the daemon will attempt to autogenerate it (if the user that executes SFTPGo has write access to the config-dir). The server supports any private key format supported by [`crypto/ssh`](https://github.com/golang/crypto/blob/master/ssh/keys.go#L32).
Before starting `sftpgo` a dataprovider must be configured.
Sample SQL scripts to create the required database structure can be found inside the source tree [sql](https://github.com/drakkan/sftpgo/tree/master/sql "sql") directory. The SQL scripts filename's is, by convention, the date as `YYYYMMDD` and the suffix `.sql`. You need to apply all the SQL scripts for your database ordered by name, for example `20190706.sql` must be applied before `20190728.sql` and so on.
The `sftpgo` configuration file contains the following sections:
- **"sftpd"**, the configuration for the SFTP server
-`bind_port`, integer. The port used for serving SFTP requests. Default: 2022
-`bind_address`, string. Leave blank to listen on all available network interfaces. Default: ""
-`idle_timeout`, integer. Time in minutes after which an idle client will be disconnected. Default: 15
-`max_auth_tries` integer. Maximum number of authentication attempts permitted per connection. If set to a negative number, the number of attempts are unlimited. If set to zero, the number of attempts are limited to 6.
-`umask`, string. Umask for the new files and directories. This setting has no effect on Windows. Default: "0022"
-`banner`, string. Identification string used by the server. Default "SFTPGo"
-`upload_mode` integer. 0 means standard, the files are uploaded directly to the requested path. 1 means atomic: files are uploaded to a temporary path and renamed to the requested path when the client ends the upload. Atomic mode avoids problems such as a web server that serves partial files when the files are being uploaded
-`actions`, struct. It contains the command to execute and/or the HTTP URL to notify and the trigger conditions
-`execute_on`, list of strings. Valid values are `download`, `upload`, `delete`, `rename`. On folder deletion a `delete` notification will be sent for each deleted file. Leave empty to disable actions.
-`command`, string. Absolute path to the command to execute. Leave empty to disable. The command is invoked with the following arguments:
-`action`, any valid `execute_on` string
-`username`, user who did the action
-`path` to the affected file. For `rename` action this is the old file name
-`target_path`, non empty for `rename` action, this is the new file name
-`http_notification_url`, a valid URL. An HTTP GET request will be executed to this URL. Leave empty to disable. The query string will contain the following parameters that have the same meaning of the command's arguments:
-`action`
-`username`
-`path`
-`target_path`, added for `rename` action only
-`keys`, struct array. It contains the daemon's private keys. If empty or missing the daemon will search or try to generate `id_rsa` in the configuration directory.
-`private_key`, path to the private key file. It can be a path relative to the config dir or an absolute one.
-`enable_scp`, boolean. Default disabled. Set to `true` to enable SCP support. SCP is an experimental feature, we have our own SCP implementation since we can't rely on `scp` system command to proper handle permissions, quota and user's home dir restrictions. The SCP protocol is quite simple but there is no official docs about it, so we need more testing and feedbacks before enabling it by default. We may not handle some borderline cases or have sneaky bugs. Please do accurate tests yourself before enabling SCP and let us known if something does not work as expected for your use cases. SCP between two remote hosts is supported using the `-3` scp option.
- **"data_provider"**, the configuration for the data provider
-`driver`, string. Supported drivers are `sqlite`, `mysql`, `postgresql`, `bolt`
-`name`, string. Database name. For driver `sqlite` this can be the database name relative to the config dir or the absolute path to the SQLite database.
-`host`, string. Database host. Leave empty for driver `sqlite` and `bolt`
-`port`, integer. Database port. Leave empty for driver `sqlite` and `bolt`
-`username`, string. Database user. Leave empty for driver `sqlite` and `bolt`
-`password`, string. Database password. Leave empty for driver `sqlite` and `bolt`
-`sslmode`, integer. Used for drivers `mysql` and `postgresql`. 0 disable SSL/TLS connections, 1 require ssl, 2 set ssl mode to `verify-ca` for driver `postgresql` and `skip-verify` for driver `mysql`, 3 set ssl mode to `verify-full` for driver `postgresql` and `preferred` for driver `mysql`
-`connectionstring`, string. Provide a custom database connection string. If not empty this connection string will be used instead of build one using the previous parameters. Leave empty for driver `bolt`
-`users_table`, string. Database table for SFTP users
-`manage_users`, integer. Set to 0 to disable users management, 1 to enable
-`track_quota`, integer. Set the preferred way to track users quota between the following choices:
- 0, disable quota tracking. REST API to scan user dir and update quota will do nothing
- 1, quota is updated each time a user upload or delete a file even if the user has no quota restrictions
- 2, quota is updated each time a user upload or delete a file but only for users with quota restrictions. With this configuration the "quota scan" REST API can still be used to periodically update space usage for users without quota restrictions
- **"httpd"**, the configuration for the HTTP server used to serve REST API
-`bind_port`, integer. The port used for serving HTTP requests. Set to 0 to disable HTTP server. Default: 8080
-`bind_address`, string. Leave blank to listen on all available network interfaces. Default: "127.0.0.1"
Here is a full example showing the default config in JSON format:
```json
{
"sftpd":{
"bind_port":2022,
"bind_address":"",
"idle_timeout":15,
"max_auth_tries":0,
"umask":"0022",
"banner":"SFTPGo",
"actions":{
"execute_on":[],
"command":"",
"http_notification_url":""
},
"keys":[],
"enable_scp":false
},
"data_provider":{
"driver":"sqlite",
"name":"sftpgo.db",
"host":"",
"port":5432,
"username":"",
"password":"",
"sslmode":0,
"connection_string":"",
"users_table":"users",
"manage_users":1,
"track_quota":2
},
"httpd":{
"bind_port":8080,
"bind_address":"127.0.0.1"
}
}
```
If you want to use a private key that use an algorithm different from RSA or more than one private key then replace the empty `keys` array with something like this:
```json
"keys":[
{
"private_key":"id_rsa"
},
{
"private_key":"id_ecdsa"
}
]
```
The configuration can be read from JSON, TOML, YAML, HCL, envfile and Java properties config files, if your `config-file` flag is set to `sftpgo` (default value) you need to create a configuration file called `sftpgo.json` or `sftpgo.yaml` and so on inside `config-dir`.
You can also configure all the available options using environment variables, sftpgo will check for environment variables with a name matching the key uppercased and prefixed with the `SFTPGO_`. You need to use `__` to traverse a struct.
Let's see some examples:
- To set sftpd `bind_port` you need to define the env var `SFTPGO_SFTPD__BIND_PORT`
- To set the `execute_on` actions you need to define the env var `SFTPGO_SFTPD__ACTIONS__EXECUTE_ON` for example `SFTPGO_SFTPD__ACTIONS__EXECUTE_ON=upload,download`
To start the SFTP Server with the default values for the command line flags simply use:
```bash
sftpgo serve
```
## Account's configuration properties
For each account the following properties can be configured:
-`username`
-`password` used for password authentication. For users created using SFTPGo REST API if the password has no known hashing algo prefix it will be stored using argon2id. SFTPGo supports checking passwords stored with bcrypt and pbkdf2 too. For pbkdf2 the supported format is `$<algo>$<iterations>$<salt>$<hashed pwd base64 encoded>`, where algo is `pbkdf2-sha1` or `pbkdf2-sha256` or `pbkdf2-sha512`. For example the `pbkdf2-sha256` of the word `password` using 150000 iterations and `E86a9YMX3zC7` as salt must be stored as `$pbkdf2-sha256$150000$E86a9YMX3zC7$R5J62hsSq+pYw00hLLPKBbcGXmq7fj5+/M0IFoYtZbo=`. For bcrypt the format must be the one supported by golang's [crypto/bcrypt](https://godoc.org/golang.org/x/crypto/bcrypt) package, for example the password `secret` with cost `14` must be stored as `$2a$14$ajq8Q7fbtFRQvXpdCq7Jcuy.Rx1h/L4J60Otx.gyNLbAYctGMJ9tK`. Using the REST API you can send a password hashed as bcrypt or pbkdf2 and it will be stored as is.
-`public_keys` array of public keys. At least one public key or the password is mandatory.
-`home_dir` The user cannot upload or download files outside this directory. Must be an absolute path
-`uid`, `gid`. If sftpgo runs as root system user then the created files and directories will be assigned to this system uid/gid. Ignored on windows and if sftpgo runs as non root user: in this case files and directories for all SFTP users will be owned by the system user that runs sftpgo.
-`max_sessions` maximum concurrent sessions. 0 means unlimited
-`quota_size` maximum size allowed as bytes. 0 means unlimited
-`quota_files` maximum number of files allowed. 0 means unlimited
-`permissions` the following permissions are supported:
-`*` all permission are granted
-`list` list items is allowed
-`download` download files is allowed
-`upload` upload files is allowed
-`delete` delete files or directories is allowed
-`rename` rename files or directories is allowed
-`create_dirs` create directories is allowed
-`create_symlinks` create symbolic links is allowed
-`upload_bandwidth` maximum upload bandwidth as KB/s, 0 means unlimited
-`download_bandwidth` maximum download bandwidth as KB/s, 0 means unlimited
These properties are stored inside the data provider. If you want to use your existing accounts, you can create a database view. Since a view is read only, you have to disable user management and quota tracking so SFTPGo will never try to write to the view.
## REST API
SFTPGo exposes REST API to manage users and quota and to get real time reports for the active connections with possibility of forcibly closing a connection.
If quota tracking is enabled in `sftpgo` configuration file, then the used size and number of files are updated each time a file is added/removed. If files are added/removed not using SFTP or if you change `track_quota` from `2` to `1`, you can rescan the user home dir and update the used quota using the REST API.
REST API is designed to run on localhost or on a trusted network, if you need HTTPS or authentication you can setup a reverse proxy using an HTTP Server such as Apache or NGNIX.
For example you can keep SFTPGo listening on localhost and expose it externally configuring a reverse proxy using Apache HTTP Server this way:
and you can add authentication with something like this:
```
<Location /api/v1>
AuthType Digest
AuthName "Private"
AuthDigestDomain "/api/v1"
AuthDigestProvider file
AuthUserFile "/etc/httpd/conf/auth_digest"
Require valid-user
</Location>
```
and, of course, you can configure the web server to use HTTPS.
The OpenAPI 3 schema for the exposed API can be found inside the source tree: [openapi.yaml](https://github.com/drakkan/sftpgo/tree/master/api/schema/openapi.yaml "OpenAPI 3 specs").
A sample CLI client for the REST API can be found inside the source tree [scripts](https://github.com/drakkan/sftpgo/tree/master/scripts "scripts") directory.
You can also generate your own REST client, in your preferred programming language or even bash scripts, using an OpenAPI generator such as [swagger-codegen](https://github.com/swagger-api/swagger-codegen) or [OpenAPI Generator](https://openapi-generator.tech/)
## Logs
Inside the log file each line is a JSON struct, each struct has a `sender` fields that identify the log type.
The logs can be divided into the following categories:
- **"app logs"**, internal logs used to debug `sftpgo`:
-`sender` string. This is generally the package name that emits the log
-`time` string. Date/time with millisecond precision
-`level` string
-`message` string
- **"transfer logs"**, SFTP/SCP transfer logs:
-`sender` string. `Upload` or `Download`
-`time` string. Date/time with millisecond precision
-`level` string
-`elapsed_ms`, int64. Elapsed time, as milliseconds, for the upload/download
-`size_bytes`, int64. Size, as bytes, of the download/upload
-`resp_size` integer. Size in bytes of the HTTP response
-`elapsed_ms` int64. Elapsed time, as milliseconds, to complete the request
-`request_id` string. Unique request identifier
This open-source version of SFTPGo remains maintained and will continue to receive bug fixes and essential updates. However, not all enhancements introduced in the Enterprise edition will be available.
SFTPGo makes use of the third party libraries listed inside [go.mod](./go.mod).
Some code was initially taken from [Pterodactyl sftp server](https://github.com/pterodactyl/sftp-server)
We are very grateful to all the people who contributed with ideas and/or pull requests.
Thank you to [ysura](https://www.ysura.com/) for granting us stable access to a test AWS S3 account.
Thank you to [KeenThemes](https://keenthemes.com/) for granting us a custom license to use their amazing [themes](https://keenthemes.com/bootstrap-templates) for the SFTPGo WebAdmin and WebClient user interfaces, across both the Open Source and Open Core versions.
Thank you to [Crowdin](https://crowdin.com/) for granting us an Open Source License.
Thank you to [Incode](https://www.incode.it/) for helping us to improve the UI/UX.
## License
GNU GPLv3
SFTPGo source code is licensed under the GNU AGPL-3.0-only with [additional terms](./NOTICE).
The [theme](https://keenthemes.com/bootstrap-templates) used in WebAdmin and WebClient user interfaces is proprietary, this means:
- KeenThemes HTML/CSS/JS components are allowed for use only within the SFTPGo product and restricted to be used in a resealable HTML template that can compete with KeenThemes products anyhow.
- The SFTPGo WebAdmin and WebClient user interfaces (HTML, CSS and JS components) based on this theme are allowed for use only within the SFTPGo product and therefore cannot be used in derivative works/products without an explicit grant from the [SFTPGo Team](mailto:support@sftpgo.com).
More information about [compliance](https://sftpgo.com/compliance.html).
**Note:** We do not provide legal advice. If you have questions about license compliance or whether your use case is permitted under the license terms, please consult your legal team.
Only the current release of the software is actively supported.
[Contact us](mailto:support@sftpgo.com) if you need early security patches and enterprise-grade security.
## Reporting a Vulnerability
To report (possible) security issues in SFTPGo, please either send a mail to the [SFTPGo Team](mailto:support@sftpgo.com) or use Github's [private reporting feature](https://github.com/drakkan/sftpgo/security/advisories/new).
summary:Get the active users and info about their uploads/downloads
operationId:get_connections
responses:
200:
description:successful operation
content:
application/json:
schema:
type:array
items:
$ref :'#/components/schemas/ConnectionStatus'
/connection/{connectionID}:
delete:
tags:
- connections
summary:Terminate an active connection
operationId:close_connection
parameters:
- name:connectionID
in:path
description:ID of the connection to close
required:true
schema:
type:string
responses:
200:
description:successful operation
content:
application/json:
schema:
$ref:'#/components/schemas/ApiResponse'
example:
status:200
message:"Connection closed"
error:""
400:
description:Bad request
content:
application/json:
schema:
$ref:'#/components/schemas/ApiResponse'
example:
status:400
message:""
error:"Error description if any"
404:
description:Not Found
content:
application/json:
schema:
$ref:'#/components/schemas/ApiResponse'
example:
status:404
message:""
error:"Error description if any"
500:
description:Internal Server Error
content:
application/json:
schema:
$ref:'#/components/schemas/ApiResponse'
example:
status:500
message:""
error:"Error description if any"
/quota_scan:
get:
tags:
- quota
summary:Get the active quota scans
operationId:get_quota_scans
responses:
200:
description:successful operation
content:
application/json:
schema:
type:array
items:
$ref :'#/components/schemas/QuotaScan'
post:
tags:
- quota
summary:start a new quota scan
description:A quota scan update the number of files and their total size for the given user
operationId:start_quota_scan
requestBody:
required:true
content:
application/json:
schema:
$ref :'#/components/schemas/User'
responses:
201:
description:successful operation
content:
application/json:
schema:
$ref:'#/components/schemas/ApiResponse'
example:
status:201
message:"Scan started"
error:""
400:
description:Bad request
content:
application/json:
schema:
$ref:'#/components/schemas/ApiResponse'
example:
status:400
message:""
error:"Error description if any"
403:
description:Forbidden
content:
application/json:
schema:
$ref:'#/components/schemas/ApiResponse'
example:
status:403
message:""
error:"Error description if any"
404:
description:Not Found
content:
application/json:
schema:
$ref:'#/components/schemas/ApiResponse'
example:
status:404
message:""
error:"Error description if any"
409:
description:Another scan is already in progress for this user
content:
application/json:
schema:
$ref:'#/components/schemas/ApiResponse'
example:
status:409
message:"Another scan is already in progress"
error:"Error description if any"
500:
description:Internal Server Error
content:
application/json:
schema:
$ref:'#/components/schemas/ApiResponse'
example:
status:500
message:""
error:"Error description if any"
/user:
get:
tags:
- users
summary:Returns an array with one or more users
description:For security reasons password and public key are empty in the response
operationId:get_users
parameters:
- in:query
name:offset
schema:
type:integer
minimum:0
default:0
required:false
- in:query
name:limit
schema:
type:integer
minimum:1
maximum:500
default:100
required:false
description:The maximum number of items to return. Max value is 500, default is 100
- in:query
name:order
required:false
description:Ordering users by username
schema:
type:string
enum:
- ASC
- DESC
example:ASC
- in:query
name:username
required:false
description:Filter by username, extact match case sensitive
schema:
type:string
responses:
200:
description:successful operation
content:
application/json:
schema:
type:array
items:
$ref :'#/components/schemas/User'
400:
description:Bad request
content:
application/json:
schema:
$ref:'#/components/schemas/ApiResponse'
example:
status:400
message:""
error:"Error description if any"
403:
description:Forbidden
content:
application/json:
schema:
$ref:'#/components/schemas/ApiResponse'
example:
status:403
message:""
error:"Error description if any"
500:
description:Internal Server Error
content:
application/json:
schema:
$ref:'#/components/schemas/ApiResponse'
example:
status:500
message:""
error:"Error description if any"
post:
tags:
- users
summary:Adds a new SFTP/SCP user
operationId:add_user
requestBody:
required:true
content:
application/json:
schema:
$ref :'#/components/schemas/User'
responses:
200:
description:successful operation
content:
application/json:
schema:
$ref :'#/components/schemas/User'
400:
description:Bad request
content:
application/json:
schema:
$ref:'#/components/schemas/ApiResponse'
example:
status:400
message:""
error:"Error description if any"
403:
description:Forbidden
content:
application/json:
schema:
$ref:'#/components/schemas/ApiResponse'
example:
status:403
message:""
error:"Error description if any"
500:
description:Internal Server Error
content:
application/json:
schema:
$ref:'#/components/schemas/ApiResponse'
example:
status:500
message:""
error:"Error description if any"
/user/{userID}:
get:
tags:
- users
summary:Find user by ID
description:For security reasons password and public key are empty in the response
operationId:get_user_by_id
parameters:
- name:userID
in:path
description:ID of the user to retrieve
required:true
schema:
type:integer
format:int32
responses:
200:
description:successful operation
content:
application/json:
schema:
$ref :'#/components/schemas/User'
400:
description:Bad request
content:
application/json:
schema:
$ref:'#/components/schemas/ApiResponse'
example:
status:400
message:""
error:"Error description if any"
403:
description:Forbidden
content:
application/json:
schema:
$ref:'#/components/schemas/ApiResponse'
example:
status:403
message:""
error:"Error description if any"
404:
description:Not Found
content:
application/json:
schema:
$ref:'#/components/schemas/ApiResponse'
example:
status:404
message:""
error:"Error description if any"
500:
description:Internal Server Error
content:
application/json:
schema:
$ref:'#/components/schemas/ApiResponse'
example:
status:500
message:""
error:"Error description if any"
put:
tags:
- users
summary:Update an existing user
operationId:update_user
parameters:
- name:userID
in:path
description:ID of the user to update
required:true
schema:
type:integer
format:int32
requestBody:
required:true
content:
application/json:
schema:
$ref :'#/components/schemas/User'
responses:
200:
description:successful operation
content:
application/json:
schema:
$ref :'#/components/schemas/ApiResponse'
example:
status:200
message:"User updated"
error:""
400:
description:Bad request
content:
application/json:
schema:
$ref:'#/components/schemas/ApiResponse'
example:
status:400
message:""
error:"Error description if any"
403:
description:Forbidden
content:
application/json:
schema:
$ref:'#/components/schemas/ApiResponse'
example:
status:403
message:""
error:"Error description if any"
404:
description:Not Found
content:
application/json:
schema:
$ref:'#/components/schemas/ApiResponse'
example:
status:404
message:""
error:"Error description if any"
500:
description:Internal Server Error
content:
application/json:
schema:
$ref:'#/components/schemas/ApiResponse'
example:
status:500
message:""
error:"Error description if any"
delete:
tags:
- users
summary:Delete an existing user
operationId:delete_user
parameters:
- name:userID
in:path
description:ID of the user to delete
required:true
schema:
type:integer
format:int32
responses:
200:
description:successful operation
content:
application/json:
schema:
$ref :'#/components/schemas/ApiResponse'
example:
status:200
message:"User deleted"
error:""
400:
description:Bad request
content:
application/json:
schema:
$ref:'#/components/schemas/ApiResponse'
example:
status:400
message:""
error:"Error description if any"
403:
description:Forbidden
content:
application/json:
schema:
$ref:'#/components/schemas/ApiResponse'
example:
status:403
message:""
error:"Error description if any"
404:
description:Not Found
content:
application/json:
schema:
$ref:'#/components/schemas/ApiResponse'
example:
status:404
message:""
error:"Error description if any"
500:
description:Internal Server Error
content:
application/json:
schema:
$ref:'#/components/schemas/ApiResponse'
example:
status:500
message:""
error:"Error description if any"
components:
schemas:
Permission:
type:string
enum:
- '*'
- list
- download
- upload
- delete
- rename
- create_dirs
- create_symlinks
description:>
Permissions:
* `*` - all permission are granted
* `list` - list items is allowed
* `download` - download files is allowed
* `upload` - upload files is allowed
* `delete` - delete files or directories is allowed
* `rename` - rename files or directories is allowed
* `create_dirs` - create directories is allowed
* `create_symlinks` - create links is allowed
User:
type:object
properties:
id:
type:integer
format:int32
minimum:1
username:
type:string
password:
type:string
nullable:true
description:password or public key are mandatory. If the password has no known hashing algo prefix it will be stored using argon2id. You can send a password hashed as bcrypt or pbkdf2 and it will be stored as is. For security reasons this field is omitted when you search/get users
public_keys:
type:array
items:
type:string
nullable:true
description:a password or at least one public key are mandatory. For security reasons this field is omitted when you search/get users.
home_dir:
type:string
description:path to the user home directory. The user cannot upload or download files outside this directory. SFTPGo tries to automatically create this folder if missing. Must be an absolute path
uid:
type:integer
format:int32
minimum:0
maximum:65535
description:if you run sftpgo as root user the created files and directories will be assigned to this uid. 0 means no change, the owner will be the user that runs sftpgo. Ignored on windows
gid:
type:integer
format:int32
minimum:0
maximum:65535
description:if you run sftpgo as root user the created files and directories will be assigned to this gid. 0 means no change, the group will be the one of the user that runs sftpgo. Ignored on windows
max_sessions:
type:integer
format:int32
description:limit the sessions that an user can open. 0 means unlimited
quota_size:
type:integer
format:int64
description:quota as size. 0 menas unlimited. Please note that quota is updated if files are added/removed via SFTP/SCP otherwise a quota scan is needed
quota_files:
type:integer
format:int32
description:quota as number of files. 0 menas unlimited. Please note that quota is updated if files are added/removed via SFTP/SCP otherwise a quota scan is needed
permissions:
type:array
items:
$ref:'#/components/schemas/Permission'
minItems:1
used_quota_size:
type:integer
format:int64
used_quota_file:
type:integer
format:int32
last_quota_update:
type:integer
format:int64
description:last quota update as unix timestamp in milliseconds
upload_bandwidth:
type:integer
format:int32
description:Maximum upload bandwidth as KB/s, 0 means unlimited
download_bandwidth:
type:integer
format:int32
description:Maximum download bandwidth as KB/s, 0 means unlimited
Transfer:
type:object
properties:
operation_type:
type:string
enum:
- upload
- download
path:
type:string
description:SFTP/SCP file path for the upload/download
start_time:
type:integer
format:int64
description:start time as unix timestamp in milliseconds
size:
type:integer
format:int64
description:bytes transferred
last_activity:
type:integer
format:int64
description:last transfer activity as unix timestamp in milliseconds
ConnectionStatus:
type:object
properties:
username:
type:string
description:connected username
connection_id:
type:string
description:unique connection identifier
client_version:
type:string
description:SFTP/SCP client version
remote_address:
type:string
description:Remote address for the connected SFTP/SCP client
connection_time:
type:integer
format:int64
description:connection time as unix timestamp in milliseconds
last_activity:
type:integer
format:int64
description:last client activity as unix timestamp in milliseconds
protocol:
type:string
enum:
- SFTP
- SCP
active_transfers:
type:array
items:
$ref :'#/components/schemas/Transfer'
QuotaScan:
type:object
properties:
username:
type:string
description:username with an active scan
start_time:
type:integer
format:int64
description:scan start time as unix timestamp in milliseconds
ApiResponse:
type:object
properties:
status:
type:integer
format:int32
minimum:200
maximum:500
example:200
description:HTTP Status code, for example 200 OK, 400 Bad request and so on
These example show how-to integrate [Twilio Authy API](https://www.twilio.com/docs/authy/api) for One-Time-Password logins.
The examples assume that the user has the free [Authy app](https://authy.com/) installed and uses it to generate offline [TOTP](https://en.wikipedia.org/wiki/Time-based_One-time_Password_algorithm) codes (soft tokens).
You first need to [create an Authy Application in the Twilio Console](https://twilio.com/console/authy/applications?_ga=2.205553366.451688189.1597667213-1526360003.1597667213), then you can create a new Authy user and store a reference to the matching SFTPGo account.
Verify that your Authy application is successfully registered:
{"message":"User created successfully.","user":{"id":xxxxxxxx},"success":true}
```
Save the user id somewhere and add a reference to the matching SFTPGo account. You could also store this ID in the `additional_info` SFTPGo user field.
After this step you can use the Authy app installed on your phone to generate TOTP codes.
Now you can verify the token using an HTTP GET request:
- the HTTP response code for the verify request, it must be `200`
- the JSON response body, it must contains the key `success` with the value `true` (as string)
If these conditions are met the token is valid and you allow the user to login.
We provide the following examples:
- [Keyboard interactive authentication](./keyint/README.md) for 2FA using password + Authy one time token.
- [External authentication](./extauth/README.md) using Authy one time tokens as passwords.
- [Check password hook](./checkpwd/README.md) for 2FA using a password consisting of a fixed string and a One Time Token.
Please note that these are sample programs not intended for production use, you should write your own hook based on them and you should prefer HTTP based hooks if performance is a concern.
This example shows how to use 2FA via the check password hook using a password consisting of a fixed part and an Authy TOTP token. The hook will check the TOTP token using the Authy API and SFTPGo will check the fixed part. Please read the [sample code](./main.go), it should be self explanatory.
# Authy 2FA using keyboard interactive authentication
This example shows how to authenticate SFTP users using 2FA (password + Authy token). Please read the [sample code](./main.go), it should be self explanatory.
The provided example tries to connect to an SFTPGo instance running on `127.0.0.1:8080` using the following credentials:
- username: `admin`
- password: `password`
and, if you execute it daily, it saves a different backup file for each day of the week. The backups will be saved within the configured `backups_path`.
The `bulkuserupdate` example script shows how to use the SFTPGo REST API to easily update some common parameters for multiple users while preserving the others.
The script is written in Python and has the following requirements:
The generated json file can be used as input for the `loaddata` REST API.
Please note that when importing Linux/Unix users the input file is not required: `/etc/passwd` and `/etc/shadow` are automatically parsed. `/etc/shadow` read permission is typically granted to the `root` user only, so you need to execute `convertusers` as `root`.
:warning: SFTPGo does not currently support `yescrypt` hashed passwords.
This is an example for an external authentication program. It performs authentication against an LDAP server.
It is tested against [389ds](https://directory.fedoraproject.org/) and can be used as starting point to authenticate using any LDAP server including Active Directory.
You need to change the LDAP connection parameters and the user search query to match your environment.
You can build this example using the following command:
```console
go build -ldflags "-s -w" -o ldapauth
```
This program assumes that the 389ds schema was extended to add support for public keys using the following ldif file placed in `/etc/dirsrv/schema/98openssh-ldap.ldif`:
```console
dn: cn=schema
changetype: modify
add: attributetypes
attributetypes: ( 1.3.6.1.4.1.24552.500.1.1.1.13 NAME 'sshPublicKey' DESC 'MANDATORY: OpenSSH Public key' EQUALITY octetStringMatch SYNTAX 1.3.6.1.4.1.1466.115.121.1.40 )
-
add: objectclasses
objectClasses: ( 1.3.6.1.4.1.24552.500.1.1.2.0 NAME 'ldapPublicKey' SUP top AUXILIARY DESC 'MANDATORY: OpenSSH LPK objectclass' MUST ( uid ) MAY ( sshPublicKey ) )
description: Members of this group gain the ability to edit their own sshPublicKey field
dn: dc=example,dc=com
changetype: modify
add: aci
aci: (targetattr = "sshPublicKey") (version 3.0; acl "Allow members of sshpublickey_self_manage to edit their keys"; allow(write) (groupdn = "ldap:///cn=sshpublickey_self_manage,ou=groups,dc=example,dc=com" and userdn="ldap:///self" ); )
-
```
:warning: A plugin for LDAP/Active Directory authentication is also [available](https://github.com/sftpgo/sftpgo-plugin-auth).
This is an example for an HTTP server to use as external authentication HTTP hook. It performs authentication against an LDAP server.
It is tested against [389ds](https://directory.fedoraproject.org/) and can be used as starting point to authenticate using any LDAP server including Active Directory.
You can configure the server using the [ldapauth.toml](./ldapauth.toml) configuration file.
You can build this example using the following command:
```console
go build -ldflags "-s -w" -o ldapauthserver
```
:warning: A plugin for LDAP/Active Directory authentication is also [available](https://github.com/sftpgo/sftpgo-plugin-auth).
# SFTPGo on Windows with Active Directory Integration + Caddy Static File Server Example
[](https://www.youtube.com/watch?v=M5UcJI8t4AI)
This is similar to the ldapauthserver example, but is more specific to using Active Directory along with using SFTPGo on a Windows Server.
The Youtube Walkthrough/Tutorial video above goes into considerable more detail, but in short, it walks through setting up SFTPGo on a new Windows Server, and enables the External Authentication feature within SFTPGo, along with my `sftpgo-ldap-http-server` project, to allow for user authentication into SFTPGo to occur through one or more Active Directory connections.
Additionally, I go through using the Caddy web server, to help enable serving of static files, if this is something that would be of interest for you.
To get started, you'll want to download the latest release ZIP package from the [sftpgo-ldap-http-server repository](https://github.com/orware/sftpgo-ldap-http-server).
The ZIP itself contains the `sftpgo-ldap-http-server.exe` file, along with an `OpenLDAP` folder (mainly to help if you want to use TLS for your LDAP connections), and a `Data` which contains a logs folder, a configuration.example.php file, a functions.php file, and the LICENSE and README files.
The video above goes through the whole process, but to get started you'll want to install SFTPGo on your server, and then extract the `sftpgo-ldap-http-server` ZIP file on the server as well into a separate folder. Then you'll want to copy the configuration.example.php file and name it `configuration.php` and begin customizing the settings (e.g. add in your own LDAP settings, along with how you may want to have your folders be created). At the very minimum you'll want to make sure that the home directories are set correctly to how you want the folders to be created for your environment (you don't have to use the virtual folders or really any of the other functionality if you don't need it).
Once configured, from a command prompt window, if you are already in the same folder as where you extracted the `sftpgo-ldap-http-server` ZIP, you may simply call the `sftpgo-ldap-http-server.exe` and it should start up a simple HTTP server on Port 9001 running on localhost (the port can be adjusted via the `configuration.php` file as well). Now all you have to do is point SFTPGo's `external_auth_hook` option to point to `http://localhost:9001/` and you should be able to run some authentication tests (assuming you have all of your settings correct and there are no intermediate issues).
The video above definitely goes through some troubleshooting situations you might find yourself coming across, so while it is long (at about 1 hour, 42 minutes), it may be helpful to review and avoid some issues and just to learn a bit more about SFTPGo and the integration above.
## Example Virtual Folders Configuration (Allowing for Both a Public and Private Folder)
The following can be utilized if you'd like to assign your users both a Private Virtual Folder and Public Virtual Folder.
By itself, the Public Virtual Folder isn't necessarily public, so keep that in mind. Only by combining things together with the Caddy web server (and Caddyfile example configuration down below) can you be successful in making the `F:\files\public` folder from the example public.
```php
$virtual_folders['example']=[
[
//"id" => 0,
"name"=>"private-#USERNAME#",
"mapped_path"=>'F:\files\private\#USERNAME#',
//"used_quota_size" => 0,
//"used_quota_files" => 0,
//"last_quota_update" => 0,
"virtual_path"=>"/_private",
"quota_size"=>-1,
"quota_files"=>-1
],
[
//"id" => 0,
"name"=>"public-#USERNAME#",
"mapped_path"=>'F:\files\public\#USERNAME#',
//"used_quota_size" => 0,
//"used_quota_files" => 0,
//"last_quota_update" => 0,
"virtual_path"=>"/_public",
"quota_size"=>-1,
"quota_files"=>-1
]
];
```
## Example Connection "Output Object" Allowing For No Files in the User's Home Directory ("Root Directory") but Allowing for Files in the Public/Private Virtual Folders
The magic here happens in the "permissions" value, by limiting the root/home directory to just the list/download permissions, and then allowing all permissions on the Public/Private virtual folders.
```php
$connection_output_objects['example']=[
'status'=>1,
'username'=>'',
'expiration_date'=>0,
'home_dir'=>'',
'uid'=>0,
'gid'=>0,
'max_sessions'=>0,
'quota_size'=>0,
'quota_files'=>100000,
'permissions'=>[
"/"=>["list","download"],
"/_private"=>["*"],
"/_public"=>["*"],
],
'upload_bandwidth'=>0,
'download_bandwidth'=>0,
'filters'=>[
'allowed_ip'=>[],
'denied_ip'=>[],
],
'public_keys'=>[],
];
```
## Recommended Usage of Automatic Groups Mode (Limiting by Group Prefix)
The `sftpgo-ldap-http-server` project is able to automatically create virtual folders for any groups your user is a memberof if the automatic mode is turned on. However, by having a specific set of allowed prefixes defined, you can limit things to just those groups that begin with the prefixes you've listed, which can be helpful. The prefix itself will be removed from the group name when added as a virtual folder for the user.
```php
// If automatic groups mode is disabled, then you have to manually add the allowed groups into $allowed_groups down below:
// If enabled, then any groups you are a memberof will automatically be added in using the template below.
$auto_groups_mode=true;
$auto_groups_mode_virtual_folder_template=[
[
//"id" => 0,
"name"=>"groups-#GROUP#",
"mapped_path"=>'F:\files\groups\#GROUP#',
//"used_quota_size" => 0,
//"used_quota_files" => 0,
//"last_quota_update" => 0,
"virtual_path"=>"/groups/#GROUP#",
"quota_size"=>0,
"quota_files"=>100000
]
];
// Used only when auto groups mode is enabled and will help prevent all your groups from being
// added into SFTPGo since only groups with the prefixes defined here will be automatically added
// with prefixes automatically removed when listed as a virtual folder (e.g. a group with name
// "sftpgo-example" would simply become "example").
$allowed_group_prefixes=[
'sftpgo-'
];
```
## Example Caddyfile Configuration You Can Adapt for Your Needs
```shell
### Re-usable snippets:
(add_static_file_serving_features){
# Allow accessing files without requiring .html:
try_files {path}{path}.html
# Enable Static File Server and Directory Browsing:
:warning: Since v2.4.0 you can use the [EventManager](https://docs.sftpgo.com/latest/eventmanager/) to schedule quota scans.
The `scanuserquota` example script shows how to use the SFTPGo REST API to update the users' quota.
The stored quota may be incorrect for several reasons, such as an unexpected shutdown while uploading files, temporary provider failures, files copied outside of SFTPGo, and so on.
A quota scan updates the number of files and their total size for the specified user and the virtual folders, if any, included in his quota.
If you want to track quotas, a scheduled quota scan is recommended. You can use this example as a starting point.
The script is written in Python and has the following requirements:
Some files were not shown because too many files have changed in this diff
Show More
Reference in New Issue
Block a user
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.