memory provider: load users from a dump file

The `memory` provider can load users from a dump obtained using the
`dumpdata` REST API. This dump file can be configured using the
dataprovider `name` configuration key. It will be loaded at startup
and can be reloaded on demand using a `SIGHUP` on Unix based systems
and a `paramchange` request to the running service on Windows.

Fixes #66
This commit is contained in:
Nicola Murino
2020-02-02 22:20:39 +01:00
parent 31a433cda2
commit bcaf283c35
22 changed files with 284 additions and 60 deletions

View File

@@ -157,7 +157,7 @@ The `sftpgo` configuration file contains the following sections:
- `keyboard_interactive_auth_program`, string. Absolute path to an external program to use for keyboard interactive authentication. See the "Keyboard Interactive Authentication" paragraph for more details. - `keyboard_interactive_auth_program`, string. Absolute path to an external program to use for keyboard interactive authentication. See the "Keyboard Interactive Authentication" paragraph for more details.
- **"data_provider"**, the configuration for the data provider - **"data_provider"**, the configuration for the data provider
- `driver`, string. Supported drivers are `sqlite`, `mysql`, `postgresql`, `bolt`, `memory` - `driver`, string. Supported drivers are `sqlite`, `mysql`, `postgresql`, `bolt`, `memory`
- `name`, string. Database name. For driver `sqlite` this can be the database name relative to the config dir or the absolute path to the SQLite database. - `name`, string. Database name. For driver `sqlite` this can be the database name relative to the config dir or the absolute path to the SQLite database. For driver `memory` this is the (optional) path relative to the config dir or the absolute path to the users dump to load.
- `host`, string. Database host. Leave empty for drivers `sqlite`, `bolt` and `memory` - `host`, string. Database host. Leave empty for drivers `sqlite`, `bolt` and `memory`
- `port`, integer. Database port. Leave empty for drivers `sqlite`, `bolt` and `memory` - `port`, integer. Database port. Leave empty for drivers `sqlite`, `bolt` and `memory`
- `username`, string. Database user. Leave empty for drivers `sqlite`, `bolt` and `memory` - `username`, string. Database user. Leave empty for drivers `sqlite`, `bolt` and `memory`
@@ -277,7 +277,9 @@ Before starting `sftpgo serve` please ensure that the configured dataprovider is
SQL based data providers (SQLite, MySQL, PostgreSQL) requires the creation of a database containing the required tables. Memory and bolt data providers does not require an initialization. SQL based data providers (SQLite, MySQL, PostgreSQL) requires the creation of a database containing the required tables. Memory and bolt data providers does not require an initialization.
SQL scripts to create the required database structure can be found inside the source tree [sql](./sql "sql") directory. The SQL scripts filename is, by convention, the date as `YYYYMMDD` and the suffix `.sql`. You need to apply all the SQL scripts for your database ordered by name, for example `20190828.sql` must be applied before `20191112.sql` and so on. SQL scripts to create the required database structure can be found inside the source tree [sql](./sql "sql") directory. The SQL scripts filename is, by convention, the date as `YYYYMMDD` and the suffix `.sql`. You need to apply all the SQL scripts for your database ordered by name, for example `20190828.sql` must be applied before `20191112.sql` and so on.
Example for `SQLite`: `find sql/sqlite/ -type f -iname '*.sql' -print | sort -n |xargs cat | sqlite3 sftpgo.db` Example for SQLite: `find sql/sqlite/ -type f -iname '*.sql' -print | sort -n |xargs cat | sqlite3 sftpgo.db`.
The `memory` provider can load users from a dump obtained using the `dumpdata` REST API. This dump file can be configured using the dataprovider `name` configuration key. It will be loaded at startup and can be reloaded on demand using a `SIGHUP` on Unix based systems and a `paramchange` request to the running service on Windows.
### Starting SFTGo in server mode ### Starting SFTGo in server mode
@@ -291,13 +293,14 @@ On Windows you can register `SFTPGo` as Windows Service, take a look at the CLI
``` ```
sftpgo.exe service --help sftpgo.exe service --help
Install, Uninstall, Start, Stop and retrieve status for SFTPGo Windows Service Install, Uninstall, Start, Stop, Reload and retrieve status for SFTPGo Windows Service
Usage: Usage:
sftpgo service [command] sftpgo service [command]
Available Commands: Available Commands:
install Install SFTPGo as Windows Service install Install SFTPGo as Windows Service
reload Reload the SFTPGo Windows Service sending a `paramchange` request
start Start SFTPGo Windows Service start Start SFTPGo Windows Service
status Retrieve the status for the SFTPGo Windows Service status Retrieve the status for the SFTPGo Windows Service
stop Stop SFTPGo Windows Service stop Stop SFTPGo Windows Service

32
cmd/reload_windows.go Normal file
View File

@@ -0,0 +1,32 @@
package cmd
import (
"fmt"
"github.com/drakkan/sftpgo/service"
"github.com/spf13/cobra"
)
var (
reloadCmd = &cobra.Command{
Use: "reload",
Short: "Reload the SFTPGo Windows Service sending a \"paramchange\" request",
Run: func(cmd *cobra.Command, args []string) {
s := service.WindowsService{
Service: service.Service{
Shutdown: make(chan bool),
},
}
err := s.Reload()
if err != nil {
fmt.Printf("Error reloading service: %v\r\n", err)
} else {
fmt.Printf("Service reloaded!\r\n")
}
},
}
)
func init() {
serviceCmd.AddCommand(reloadCmd)
}

View File

@@ -7,7 +7,7 @@ import (
var ( var (
serviceCmd = &cobra.Command{ serviceCmd = &cobra.Command{
Use: "service", Use: "service",
Short: "Install, Uninstall, Start, Stop and retrieve status for SFTPGo Windows Service", Short: "Install, Uninstall, Start, Stop, Reload and retrieve status for SFTPGo Windows Service",
} }
) )

View File

@@ -392,6 +392,10 @@ func (p BoltProvider) close() error {
return p.dbHandle.Close() return p.dbHandle.Close()
} }
func (p BoltProvider) reloadConfig() error {
return nil
}
// itob returns an 8-byte big endian representation of v. // itob returns an 8-byte big endian representation of v.
func itob(v int64) []byte { func itob(v int64) []byte {
b := make([]byte, 8) b := make([]byte, 8)

View File

@@ -187,6 +187,11 @@ type Config struct {
CredentialsPath string `json:"credentials_path" mapstructure:"credentials_path"` CredentialsPath string `json:"credentials_path" mapstructure:"credentials_path"`
} }
// BackupData defines the structure for the backup/restore files
type BackupData struct {
Users []User `json:"users"`
}
type keyboardAuthProgramResponse struct { type keyboardAuthProgramResponse struct {
Instruction string `json:"instruction"` Instruction string `json:"instruction"`
Questions []string `json:"questions"` Questions []string `json:"questions"`
@@ -251,6 +256,7 @@ type Provider interface {
updateLastLogin(username string) error updateLastLogin(username string) error
checkAvailability() error checkAvailability() error
close() error close() error
reloadConfig() error
} }
func init() { func init() {
@@ -287,7 +293,7 @@ func Initialize(cnf Config, basePath string) error {
} else if config.Driver == BoltDataProviderName { } else if config.Driver == BoltDataProviderName {
err = initializeBoltProvider(basePath) err = initializeBoltProvider(basePath)
} else if config.Driver == MemoryDataProviderName { } else if config.Driver == MemoryDataProviderName {
err = initializeMemoryProvider() err = initializeMemoryProvider(basePath)
} else { } else {
err = fmt.Errorf("unsupported data provider: %v", config.Driver) err = fmt.Errorf("unsupported data provider: %v", config.Driver)
} }
@@ -417,6 +423,13 @@ func DumpUsers(p Provider) ([]User, error) {
return p.dumpUsers() return p.dumpUsers()
} }
// ReloadConfig reloads provider configuration.
// Currently only implemented for memory provider, allows to reload the users
// from the configured file, if defined
func ReloadConfig() error {
return provider.reloadConfig()
}
// GetUsers returns an array of users respecting limit and offset and filtered by username exact match if not empty // GetUsers returns an array of users respecting limit and offset and filtered by username exact match if not empty
func GetUsers(p Provider, limit int, offset int, order string, username string) ([]User, error) { func GetUsers(p Provider, limit int, offset int, order string, username string) ([]User, error) {
return p.getUsers(limit, offset, order, username) return p.getUsers(limit, offset, order, username)

View File

@@ -1,8 +1,12 @@
package dataprovider package dataprovider
import ( import (
"encoding/json"
"errors" "errors"
"fmt" "fmt"
"io/ioutil"
"os"
"path/filepath"
"sort" "sort"
"sync" "sync"
"time" "time"
@@ -23,6 +27,8 @@ type memoryProviderHandle struct {
usersIdx map[int64]string usersIdx map[int64]string
// map for users, username is the key // map for users, username is the key
users map[string]User users map[string]User
// configuration file to use for loading users
configFile string
lock *sync.Mutex lock *sync.Mutex
} }
@@ -31,17 +37,25 @@ type MemoryProvider struct {
dbHandle *memoryProviderHandle dbHandle *memoryProviderHandle
} }
func initializeMemoryProvider() error { func initializeMemoryProvider(basePath string) error {
configFile := ""
if len(config.Name) > 0 {
configFile = config.Name
if !filepath.IsAbs(configFile) {
configFile = filepath.Join(basePath, configFile)
}
}
provider = MemoryProvider{ provider = MemoryProvider{
dbHandle: &memoryProviderHandle{ dbHandle: &memoryProviderHandle{
isClosed: false, isClosed: false,
usernames: []string{}, usernames: []string{},
usersIdx: make(map[int64]string), usersIdx: make(map[int64]string),
users: make(map[string]User), users: make(map[string]User),
configFile: configFile,
lock: new(sync.Mutex), lock: new(sync.Mutex),
}, },
} }
return nil return provider.reloadConfig()
} }
func (p MemoryProvider) checkAvailability() error { func (p MemoryProvider) checkAvailability() error {
@@ -308,3 +322,71 @@ func (p MemoryProvider) getNextID() int64 {
} }
return nextID return nextID
} }
func (p MemoryProvider) clearUsers() {
p.dbHandle.lock.Lock()
defer p.dbHandle.lock.Unlock()
p.dbHandle.usernames = []string{}
p.dbHandle.usersIdx = make(map[int64]string)
p.dbHandle.users = make(map[string]User)
}
func (p MemoryProvider) reloadConfig() error {
if len(p.dbHandle.configFile) == 0 {
providerLog(logger.LevelDebug, "no users configuration file defined")
return nil
}
providerLog(logger.LevelDebug, "loading users from file: %#v", p.dbHandle.configFile)
fi, err := os.Stat(p.dbHandle.configFile)
if err != nil {
providerLog(logger.LevelWarn, "error loading users: %v", err)
return err
}
if fi.Size() == 0 {
err = errors.New("users configuration file is invalid, its size must be > 0")
providerLog(logger.LevelWarn, "error loading users: %v", err)
return err
}
if fi.Size() > 10485760 {
err = errors.New("users configuration file is invalid, its size must be <= 10485760 bytes")
providerLog(logger.LevelWarn, "error loading users: %v", err)
return err
}
content, err := ioutil.ReadFile(p.dbHandle.configFile)
if err != nil {
providerLog(logger.LevelWarn, "error loading users: %v", err)
return err
}
var dump BackupData
err = json.Unmarshal(content, &dump)
if err != nil {
providerLog(logger.LevelWarn, "error loading users: %v", err)
return err
}
p.clearUsers()
for _, user := range dump.Users {
u, err := p.userExists(user.Username)
if err == nil {
user.ID = u.ID
user.LastLogin = u.LastLogin
user.UsedQuotaSize = u.UsedQuotaSize
user.UsedQuotaFiles = u.UsedQuotaFiles
err = p.updateUser(user)
if err != nil {
providerLog(logger.LevelWarn, "error updating user %#v: %v", user.Username, err)
return err
}
} else {
user.LastLogin = 0
user.UsedQuotaSize = 0
user.UsedQuotaFiles = 0
err = p.addUser(user)
if err != nil {
providerLog(logger.LevelWarn, "error adding user %#v: %v", user.Username, err)
return err
}
}
}
providerLog(logger.LevelDebug, "users loaded from file: %#v", p.dbHandle.configFile)
return nil
}

View File

@@ -99,3 +99,7 @@ func (p MySQLProvider) getUsers(limit int, offset int, order string, username st
func (p MySQLProvider) close() error { func (p MySQLProvider) close() error {
return p.dbHandle.Close() return p.dbHandle.Close()
} }
func (p MySQLProvider) reloadConfig() error {
return nil
}

View File

@@ -98,3 +98,7 @@ func (p PGSQLProvider) getUsers(limit int, offset int, order string, username st
func (p PGSQLProvider) close() error { func (p PGSQLProvider) close() error {
return p.dbHandle.Close() return p.dbHandle.Close()
} }
func (p PGSQLProvider) reloadConfig() error {
return nil
}

View File

@@ -105,3 +105,7 @@ func (p SQLiteProvider) getUsers(limit int, offset int, order string, username s
func (p SQLiteProvider) close() error { func (p SQLiteProvider) close() error {
return p.dbHandle.Close() return p.dbHandle.Close()
} }
func (p SQLiteProvider) reloadConfig() error {
return nil
}

View File

@@ -17,10 +17,13 @@ import (
) )
func dumpData(w http.ResponseWriter, r *http.Request) { func dumpData(w http.ResponseWriter, r *http.Request) {
var outputFile string var outputFile, indent string
if _, ok := r.URL.Query()["output_file"]; ok { if _, ok := r.URL.Query()["output_file"]; ok {
outputFile = strings.TrimSpace(r.URL.Query().Get("output_file")) outputFile = strings.TrimSpace(r.URL.Query().Get("output_file"))
} }
if _, ok := r.URL.Query()["indent"]; ok {
indent = strings.TrimSpace(r.URL.Query().Get("indent"))
}
if len(outputFile) == 0 { if len(outputFile) == 0 {
sendAPIResponse(w, r, errors.New("Invalid or missing output_file"), "", http.StatusBadRequest) sendAPIResponse(w, r, errors.New("Invalid or missing output_file"), "", http.StatusBadRequest)
return return
@@ -42,12 +45,19 @@ func dumpData(w http.ResponseWriter, r *http.Request) {
sendAPIResponse(w, r, err, "", getRespStatus(err)) sendAPIResponse(w, r, err, "", getRespStatus(err))
return return
} }
dump, err := json.Marshal(BackupData{ var dump []byte
if indent == "1" {
dump, err = json.MarshalIndent(dataprovider.BackupData{
Users: users,
}, "", " ")
} else {
dump, err = json.Marshal(dataprovider.BackupData{
Users: users, Users: users,
}) })
}
if err == nil { if err == nil {
os.MkdirAll(filepath.Dir(outputFile), 0777) os.MkdirAll(filepath.Dir(outputFile), 0700)
err = ioutil.WriteFile(outputFile, dump, 0666) err = ioutil.WriteFile(outputFile, dump, 0600)
} }
if err != nil { if err != nil {
logger.Warn(logSender, "", "dumping data error: %v, output file: %#v", err, outputFile) logger.Warn(logSender, "", "dumping data error: %v, output file: %#v", err, outputFile)
@@ -74,8 +84,8 @@ func loadData(w http.ResponseWriter, r *http.Request) {
return return
} }
if fi.Size() > maxRestoreSize { if fi.Size() > maxRestoreSize {
sendAPIResponse(w, r, err, fmt.Sprintf("Unable to restore input file: %#v size too big: %v/%v", inputFile, fi.Size(), sendAPIResponse(w, r, err, fmt.Sprintf("Unable to restore input file: %#v size too big: %v/%v bytes",
maxRestoreSize), http.StatusBadRequest) inputFile, fi.Size(), maxRestoreSize), http.StatusBadRequest)
return return
} }
@@ -84,7 +94,7 @@ func loadData(w http.ResponseWriter, r *http.Request) {
sendAPIResponse(w, r, err, "", getRespStatus(err)) sendAPIResponse(w, r, err, "", getRespStatus(err))
return return
} }
var dump BackupData var dump dataprovider.BackupData
err = json.Unmarshal(content, &dump) err = json.Unmarshal(content, &dump)
if err != nil { if err != nil {
sendAPIResponse(w, r, err, fmt.Sprintf("Unable to parse input file: %#v", inputFile), http.StatusBadRequest) sendAPIResponse(w, r, err, fmt.Sprintf("Unable to parse input file: %#v", inputFile), http.StatusBadRequest)
@@ -95,7 +105,7 @@ func loadData(w http.ResponseWriter, r *http.Request) {
u, err := dataprovider.UserExists(dataProvider, user.Username) u, err := dataprovider.UserExists(dataProvider, user.Username)
if err == nil { if err == nil {
if mode == 1 { if mode == 1 {
logger.Debug(logSender, "", "loaddata mode = 1 existing user: %#v not updated", u.Username) logger.Debug(logSender, "", "loaddata mode 1, existing user %#v not updated", u.Username)
continue continue
} }
user.ID = u.ID user.ID = u.ID

View File

@@ -309,7 +309,7 @@ func GetProviderStatus(expectedStatusCode int) (map[string]interface{}, []byte,
// Dumpdata requests a backup to outputFile. // Dumpdata requests a backup to outputFile.
// outputFile is relative to the configured backups_path // outputFile is relative to the configured backups_path
func Dumpdata(outputFile string, expectedStatusCode int) (map[string]interface{}, []byte, error) { func Dumpdata(outputFile, indent string, expectedStatusCode int) (map[string]interface{}, []byte, error) {
var response map[string]interface{} var response map[string]interface{}
var body []byte var body []byte
url, err := url.Parse(buildURLRelativeToBase(dumpDataPath)) url, err := url.Parse(buildURLRelativeToBase(dumpDataPath))
@@ -318,6 +318,9 @@ func Dumpdata(outputFile string, expectedStatusCode int) (map[string]interface{}
} }
q := url.Query() q := url.Query()
q.Add("output_file", outputFile) q.Add("output_file", outputFile)
if len(indent) > 0 {
q.Add("indent", indent)
}
url.RawQuery = q.Encode() url.RawQuery = q.Encode()
resp, err := getHTTPClient().Get(url.String()) resp, err := getHTTPClient().Get(url.String())
if err != nil { if err != nil {

View File

@@ -56,11 +56,6 @@ type Conf struct {
BackupsPath string `json:"backups_path" mapstructure:"backups_path"` BackupsPath string `json:"backups_path" mapstructure:"backups_path"`
} }
// BackupData defines the structure for the backup/restore files
type BackupData struct {
Users []dataprovider.User `json:"users"`
}
type apiResponse struct { type apiResponse struct {
Error string `json:"error"` Error string `json:"error"`
Message string `json:"message"` Message string `json:"message"`

View File

@@ -716,13 +716,13 @@ func TestProviderErrors(t *testing.T) {
if err != nil { if err != nil {
t.Errorf("get provider status with provider closed must fail: %v", err) t.Errorf("get provider status with provider closed must fail: %v", err)
} }
_, _, err = httpd.Dumpdata("backup.json", http.StatusInternalServerError) _, _, err = httpd.Dumpdata("backup.json", "", http.StatusInternalServerError)
if err != nil { if err != nil {
t.Errorf("get provider status with provider closed must fail: %v", err) t.Errorf("get provider status with provider closed must fail: %v", err)
} }
user := getTestUser() user := getTestUser()
user.ID = 1 user.ID = 1
backupData := httpd.BackupData{} backupData := dataprovider.BackupData{}
backupData.Users = append(backupData.Users, user) backupData.Users = append(backupData.Users, user)
backupContent, _ := json.Marshal(backupData) backupContent, _ := json.Marshal(backupData)
backupFilePath := filepath.Join(backupsPath, "backup.json") backupFilePath := filepath.Join(backupsPath, "backup.json")
@@ -755,26 +755,30 @@ func TestDumpdata(t *testing.T) {
} }
httpd.SetDataProvider(dataprovider.GetProvider()) httpd.SetDataProvider(dataprovider.GetProvider())
sftpd.SetDataProvider(dataprovider.GetProvider()) sftpd.SetDataProvider(dataprovider.GetProvider())
_, _, err = httpd.Dumpdata("", http.StatusBadRequest) _, _, err = httpd.Dumpdata("", "", http.StatusBadRequest)
if err != nil { if err != nil {
t.Errorf("unexpected error: %v", err) t.Errorf("unexpected error: %v", err)
} }
_, _, err = httpd.Dumpdata(filepath.Join(backupsPath, "backup.json"), http.StatusBadRequest) _, _, err = httpd.Dumpdata(filepath.Join(backupsPath, "backup.json"), "", http.StatusBadRequest)
if err != nil { if err != nil {
t.Errorf("unexpected error: %v", err) t.Errorf("unexpected error: %v", err)
} }
_, _, err = httpd.Dumpdata("../backup.json", http.StatusBadRequest) _, _, err = httpd.Dumpdata("../backup.json", "", http.StatusBadRequest)
if err != nil { if err != nil {
t.Errorf("unexpected error: %v", err) t.Errorf("unexpected error: %v", err)
} }
_, _, err = httpd.Dumpdata("backup.json", http.StatusOK) _, _, err = httpd.Dumpdata("backup.json", "0", http.StatusOK)
if err != nil {
t.Errorf("unexpected error: %v", err)
}
_, _, err = httpd.Dumpdata("backup.json", "1", http.StatusOK)
if err != nil { if err != nil {
t.Errorf("unexpected error: %v", err) t.Errorf("unexpected error: %v", err)
} }
os.Remove(filepath.Join(backupsPath, "backup.json")) os.Remove(filepath.Join(backupsPath, "backup.json"))
if runtime.GOOS != "windows" { if runtime.GOOS != "windows" {
os.Chmod(backupsPath, 0001) os.Chmod(backupsPath, 0001)
_, _, err = httpd.Dumpdata("bck.json", http.StatusInternalServerError) _, _, err = httpd.Dumpdata("bck.json", "", http.StatusInternalServerError)
if err != nil { if err != nil {
t.Errorf("unexpected error: %v", err) t.Errorf("unexpected error: %v", err)
} }
@@ -795,7 +799,7 @@ func TestLoaddata(t *testing.T) {
user := getTestUser() user := getTestUser()
user.ID = 1 user.ID = 1
user.Username = "test_user_restore" user.Username = "test_user_restore"
backupData := httpd.BackupData{} backupData := dataprovider.BackupData{}
backupData.Users = append(backupData.Users, user) backupData.Users = append(backupData.Users, user)
backupContent, _ := json.Marshal(backupData) backupContent, _ := json.Marshal(backupData)
backupFilePath := filepath.Join(backupsPath, "backup.json") backupFilePath := filepath.Join(backupsPath, "backup.json")
@@ -865,7 +869,7 @@ func TestLoaddataMode(t *testing.T) {
user := getTestUser() user := getTestUser()
user.ID = 1 user.ID = 1
user.Username = "test_user_restore" user.Username = "test_user_restore"
backupData := httpd.BackupData{} backupData := dataprovider.BackupData{}
backupData.Users = append(backupData.Users, user) backupData.Users = append(backupData.Users, user)
backupContent, _ := json.Marshal(backupData) backupContent, _ := json.Marshal(backupData)
backupFilePath := filepath.Join(backupsPath, "backup.json") backupFilePath := filepath.Join(backupsPath, "backup.json")

View File

@@ -336,7 +336,7 @@ func TestApiCallsWithBadURL(t *testing.T) {
if err == nil { if err == nil {
t.Error("request with invalid URL must fail") t.Error("request with invalid URL must fail")
} }
_, _, err = Dumpdata("backup.json", http.StatusBadRequest) _, _, err = Dumpdata("backup.json", "", http.StatusBadRequest)
if err == nil { if err == nil {
t.Error("request with invalid URL must fail") t.Error("request with invalid URL must fail")
} }
@@ -395,7 +395,7 @@ func TestApiCallToNotListeningServer(t *testing.T) {
if err == nil { if err == nil {
t.Errorf("request to an inactive URL must fail") t.Errorf("request to an inactive URL must fail")
} }
_, _, err = Dumpdata("backup.json", http.StatusOK) _, _, err = Dumpdata("backup.json", "0", http.StatusOK)
if err == nil { if err == nil {
t.Errorf("request to an inactive URL must fail") t.Errorf("request to an inactive URL must fail")
} }

View File

@@ -2,7 +2,7 @@ openapi: 3.0.1
info: info:
title: SFTPGo title: SFTPGo
description: 'SFTPGo REST API' description: 'SFTPGo REST API'
version: 1.6.1 version: 1.6.2
servers: servers:
- url: /api/v1 - url: /api/v1
@@ -543,6 +543,17 @@ paths:
type: string type: string
required: true required: true
description: Path for the file to write the JSON serialized data to. This path is relative to the configured "backups_path". If this file already exists it will be overwritten description: Path for the file to write the JSON serialized data to. This path is relative to the configured "backups_path". If this file already exists it will be overwritten
- in: query
name: indent
schema:
type: integer
enum:
- 0
- 1
description: >
indent:
* `0` no indentation. This is the default
* `1` format the output JSON
responses: responses:
200: 200:
description: successful operation description: successful operation

View File

@@ -368,7 +368,7 @@ Output:
Command: Command:
``` ```
python sftpgo_api_cli.py dumpdata backup.json python sftpgo_api_cli.py dumpdata backup.json --indent 1
``` ```
Output: Output:
@@ -386,7 +386,7 @@ Output:
Command: Command:
``` ```
python sftpgo_api_cli.py loaddata /app/data/backups/backup.json --scan-quota 2 python sftpgo_api_cli.py loaddata /app/data/backups/backup.json --scan-quota 2 --mode 0
``` ```
Output: Output:

View File

@@ -209,9 +209,9 @@ class SFTPGoApiRequests:
r = requests.get(self.providerStatusPath, auth=self.auth, verify=self.verify) r = requests.get(self.providerStatusPath, auth=self.auth, verify=self.verify)
self.printResponse(r) self.printResponse(r)
def dumpData(self, output_file): def dumpData(self, output_file, indent):
r = requests.get(self.dumpDataPath, params={'output_file':output_file}, auth=self.auth, r = requests.get(self.dumpDataPath, params={'output_file':output_file, 'indent':indent},
verify=self.verify) auth=self.auth, verify=self.verify)
self.printResponse(r) self.printResponse(r)
def loadData(self, input_file, scan_quota, mode): def loadData(self, input_file, scan_quota, mode):
@@ -514,6 +514,8 @@ if __name__ == '__main__':
parserDumpData = subparsers.add_parser('dumpdata', help='Backup SFTPGo data serializing them as JSON') parserDumpData = subparsers.add_parser('dumpdata', help='Backup SFTPGo data serializing them as JSON')
parserDumpData.add_argument('output_file', type=str) parserDumpData.add_argument('output_file', type=str)
parserDumpData.add_argument('-I', '--indent', type=int, choices=[0, 1], default=0,
help='0 means no indentation. 1 means format the output JSON. Default: %(default)s')
parserLoadData = subparsers.add_parser('loaddata', help='Restore SFTPGo data from a JSON backup') parserLoadData = subparsers.add_parser('loaddata', help='Restore SFTPGo data from a JSON backup')
parserLoadData.add_argument('input_file', type=str) parserLoadData.add_argument('input_file', type=str)
@@ -584,7 +586,7 @@ if __name__ == '__main__':
elif args.command == 'get-provider-status': elif args.command == 'get-provider-status':
api.getProviderStatus() api.getProviderStatus()
elif args.command == 'dumpdata': elif args.command == 'dumpdata':
api.dumpData(args.output_file) api.dumpData(args.output_file, args.indent)
elif args.command == 'loaddata': elif args.command == 'loaddata':
api.loadData(args.input_file, args.scan_quota, args.mode) api.loadData(args.input_file, args.scan_quota, args.mode)
elif args.command == 'convert-users': elif args.command == 'convert-users':

View File

@@ -114,6 +114,9 @@ func (s *Service) Start() error {
logger.DebugToConsole("HTTP server not started, disabled in config file") logger.DebugToConsole("HTTP server not started, disabled in config file")
} }
} }
if s.PortableMode != 1 {
registerSigHup()
}
return nil return nil
} }

View File

@@ -7,6 +7,7 @@ import (
"strings" "strings"
"time" "time"
"github.com/drakkan/sftpgo/dataprovider"
"github.com/drakkan/sftpgo/logger" "github.com/drakkan/sftpgo/logger"
"golang.org/x/sys/windows/svc" "golang.org/x/sys/windows/svc"
@@ -61,7 +62,7 @@ func (s Status) String() string {
} }
func (s *WindowsService) Execute(args []string, r <-chan svc.ChangeRequest, changes chan<- svc.Status) (ssec bool, errno uint32) { func (s *WindowsService) Execute(args []string, r <-chan svc.ChangeRequest, changes chan<- svc.Status) (ssec bool, errno uint32) {
const cmdsAccepted = svc.AcceptStop | svc.AcceptShutdown const cmdsAccepted = svc.AcceptStop | svc.AcceptShutdown | svc.AcceptParamChange
changes <- svc.Status{State: svc.StartPending} changes <- svc.Status{State: svc.StartPending}
if err := s.Service.Start(); err != nil { if err := s.Service.Start(); err != nil {
return true, 1 return true, 1
@@ -79,6 +80,9 @@ loop:
changes <- svc.Status{State: svc.StopPending} changes <- svc.Status{State: svc.StopPending}
s.Service.Stop() s.Service.Stop()
break loop break loop
case svc.ParamChange:
logger.Debug(logSender, "", "Received reload request")
dataprovider.ReloadConfig()
default: default:
continue loop continue loop
} }
@@ -127,6 +131,24 @@ func (s *WindowsService) Start() error {
return nil return nil
} }
func (s *WindowsService) Reload() error {
m, err := mgr.Connect()
if err != nil {
return err
}
defer m.Disconnect()
service, err := m.OpenService(serviceName)
if err != nil {
return fmt.Errorf("could not access service: %v", err)
}
defer service.Close()
_, err = service.Control(svc.ParamChange)
if err != nil {
return fmt.Errorf("could not send control=%d: %v", svc.ParamChange, err)
}
return nil
}
func (s *WindowsService) Install(args ...string) error { func (s *WindowsService) Install(args ...string) error {
exePath, err := s.getExePath() exePath, err := s.getExePath()
if err != nil { if err != nil {

23
service/sighup_unix.go Normal file
View File

@@ -0,0 +1,23 @@
// +build !windows
package service
import (
"os"
"os/signal"
"syscall"
"github.com/drakkan/sftpgo/dataprovider"
"github.com/drakkan/sftpgo/logger"
)
func registerSigHup() {
sig := make(chan os.Signal, 1)
signal.Notify(sig, syscall.SIGHUP)
go func() {
for range sig {
logger.Debug(logSender, "", "Received reload request")
dataprovider.ReloadConfig()
}
}()
}

View File

@@ -0,0 +1,4 @@
package service
func registerSigHup() {
}

View File

@@ -3035,12 +3035,13 @@ func TestRelativePaths(t *testing.T) {
user := getTestUser(true) user := getTestUser(true)
var path, rel string var path, rel string
filesystems := []vfs.Fs{vfs.NewOsFs("", user.GetHomeDir())} filesystems := []vfs.Fs{vfs.NewOsFs("", user.GetHomeDir())}
keyPrefix := strings.TrimPrefix(user.GetHomeDir(), "/") + "/"
s3config := vfs.S3FsConfig{ s3config := vfs.S3FsConfig{
KeyPrefix: strings.TrimPrefix(user.GetHomeDir(), "/") + "/", KeyPrefix: keyPrefix,
} }
s3fs, _ := vfs.NewS3Fs("", user.GetHomeDir(), s3config) s3fs, _ := vfs.NewS3Fs("", user.GetHomeDir(), s3config)
gcsConfig := vfs.GCSFsConfig{ gcsConfig := vfs.GCSFsConfig{
KeyPrefix: strings.TrimPrefix(user.GetHomeDir(), "/") + "/", KeyPrefix: keyPrefix,
} }
gcsfs, _ := vfs.NewGCSFs("", user.GetHomeDir(), gcsConfig) gcsfs, _ := vfs.NewGCSFs("", user.GetHomeDir(), gcsConfig)
filesystems = append(filesystems, s3fs, gcsfs) filesystems = append(filesystems, s3fs, gcsfs)
@@ -3048,52 +3049,52 @@ func TestRelativePaths(t *testing.T) {
path = filepath.Join(user.HomeDir, "/") path = filepath.Join(user.HomeDir, "/")
rel = fs.GetRelativePath(path) rel = fs.GetRelativePath(path)
if rel != "/" { if rel != "/" {
t.Errorf("Unexpected relative path: %v", rel) t.Errorf("Unexpected relative path: %v fs: %v", rel, fs.Name())
} }
path = filepath.Join(user.HomeDir, "//") path = filepath.Join(user.HomeDir, "//")
rel = fs.GetRelativePath(path) rel = fs.GetRelativePath(path)
if rel != "/" { if rel != "/" {
t.Errorf("Unexpected relative path: %v", rel) t.Errorf("Unexpected relative path: %v fs: %v", rel, fs.Name())
} }
path = filepath.Join(user.HomeDir, "../..") path = filepath.Join(user.HomeDir, "../..")
rel = fs.GetRelativePath(path) rel = fs.GetRelativePath(path)
if rel != "/" { if rel != "/" {
t.Errorf("Unexpected relative path: %v path: %v", rel, path) t.Errorf("Unexpected relative path: %v path: %v fs: %v", rel, path, fs.Name())
} }
path = filepath.Join(user.HomeDir, "../../../../../") path = filepath.Join(user.HomeDir, "../../../../../")
rel = fs.GetRelativePath(path) rel = fs.GetRelativePath(path)
if rel != "/" { if rel != "/" {
t.Errorf("Unexpected relative path: %v", rel) t.Errorf("Unexpected relative path: %v fs: %v", rel, fs.Name())
} }
path = filepath.Join(user.HomeDir, "/..") path = filepath.Join(user.HomeDir, "/..")
rel = fs.GetRelativePath(path) rel = fs.GetRelativePath(path)
if rel != "/" { if rel != "/" {
t.Errorf("Unexpected relative path: %v path: %v", rel, path) t.Errorf("Unexpected relative path: %v path: %v fs: %v", rel, path, fs.Name())
} }
path = filepath.Join(user.HomeDir, "/../../../..") path = filepath.Join(user.HomeDir, "/../../../..")
rel = fs.GetRelativePath(path) rel = fs.GetRelativePath(path)
if rel != "/" { if rel != "/" {
t.Errorf("Unexpected relative path: %v", rel) t.Errorf("Unexpected relative path: %v fs: %v", rel, fs.Name())
} }
path = filepath.Join(user.HomeDir, "") path = filepath.Join(user.HomeDir, "")
rel = fs.GetRelativePath(path) rel = fs.GetRelativePath(path)
if rel != "/" { if rel != "/" {
t.Errorf("Unexpected relative path: %v", rel) t.Errorf("Unexpected relative path: %v fs: %v", rel, fs.Name())
} }
path = filepath.Join(user.HomeDir, ".") path = filepath.Join(user.HomeDir, ".")
rel = fs.GetRelativePath(path) rel = fs.GetRelativePath(path)
if rel != "/" { if rel != "/" {
t.Errorf("Unexpected relative path: %v", rel) t.Errorf("Unexpected relative path: %v fs: %v", rel, fs.Name())
} }
path = filepath.Join(user.HomeDir, "somedir") path = filepath.Join(user.HomeDir, "somedir")
rel = fs.GetRelativePath(path) rel = fs.GetRelativePath(path)
if rel != "/somedir" { if rel != "/somedir" {
t.Errorf("Unexpected relative path: %v", rel) t.Errorf("Unexpected relative path: %v fs: %v", rel, fs.Name())
} }
path = filepath.Join(user.HomeDir, "/somedir/subdir") path = filepath.Join(user.HomeDir, "/somedir/subdir")
rel = fs.GetRelativePath(path) rel = fs.GetRelativePath(path)
if rel != "/somedir/subdir" { if rel != "/somedir/subdir" {
t.Errorf("Unexpected relative path: %v", rel) t.Errorf("Unexpected relative path: %v fs: %v", rel, fs.Name())
} }
} }
} }