#69513 [PATCH] services: Add restic-backup service.

Message #5 received at submit@debbugs.gnu.org (full text, mbox, reply):

From: Giacomo Leidi <goodoldpaul@autistici.org>

To: guix-patches@gnu.org

Cc: Giacomo Leidi <goodoldpaul@autistici.org>

Subject: [PATCH] services: Add restic-backup service.

Date: Sat, 2 Mar 2024 21:51:24 +0100

* gnu/services/backup.scm: New file.
* gnu/local.mk: Add this.
* doc/guix.texi: Document this.

Change-Id: I9efd5559bb445b484107a7c27c2d0a65ccad1e66
---
 doc/guix.texi           |  95 +++++++++++++++++++++++-
 gnu/local.mk            |   1 +
 gnu/services/backup.scm | 160 ++++++++++++++++++++++++++++++++++++++++
 3 files changed, 255 insertions(+), 1 deletion(-)
 create mode 100644 gnu/services/backup.scm

diff --git a/doc/guix.texi b/doc/guix.texi
index 87fe9f803c..4e53d22c5a 100644
--- a/doc/guix.texi
+++ b/doc/guix.texi
@@ -111,7 +111,7 @@
 Copyright @copyright{} 2022 John Kehayias@*
 Copyright @copyright{} 2022⁠–⁠2023 Bruno Victal@*
 Copyright @copyright{} 2022 Ivan Vilata-i-Balaguer@*
-Copyright @copyright{} 2023 Giacomo Leidi@*
+Copyright @copyright{} 2023, 2024 Giacomo Leidi@*
 Copyright @copyright{} 2022 Antero Mejr@*
 Copyright @copyright{} 2023 Karl Hallsby@*
 Copyright @copyright{} 2023 Nathaniel Nicandro@*
@@ -41045,6 +41045,99 @@ Miscellaneous Services
 
 @c End of auto-generated fail2ban documentation.
 
+@cindex Backup
+@subsubheading Backup services
+
+The @code{(gnu services backup)} module offers services for backing up
+file system trees. For now, it provides the @code{restic-backup-service-type}.
+
+To backup a list of file system trees to a pre-initialized, end-to-end
+encrypted, deduplicated data repository, you could so with the
+@code{restic-backup-service-type}. For example with the following
+configuration:
+
+@lisp
+(service restic-backup-service-type
+         (restic-backup-configuration
+           (jobs
+             (list (restic-backup-job
+                     (repository "rclone:remote-ftp:backup/restic")
+                     (password-file "/root/.restic")
+                     ;; Every day at 23.
+                     (specification "0 23 * * *")
+                     (included '("/root/.restic"
+                                 "/root/.config/rclone"
+                                 "/etc/ssh/ssh_host_rsa_key"
+                                 "/etc/ssh/ssh_host_rsa_key.pub"
+                                 "/etc/guix/signing-key.pub"
+                                 "/etc/guix/signing-key.sec")))))))
+@end lisp
+
+Each @code{restic-backup-job} translates to an mcron job which sets the
+@code{RESTIC_PASSWORD} environment variable by reading the first line of
+@code{password-file} and runs @command{restic backup}.
+
+@c %start of fragment
+
+@deftp {Data Type} restic-backup-configuration
+Available @code{restic-backup-configuration} fields are:
+
+@table @asis
+@item @code{jobs} (default: @code{'()}) (type: list-of-restic-backup-jobs)
+The list of backup jobs for the current system.
+
+@end table
+
+@end deftp
+
+
+@c %end of fragment
+
+@c %start of fragment
+
+@deftp {Data Type} restic-backup-job
+Available @code{restic-backup-job} fields are:
+
+@table @asis
+@item @code{restic} (default: @code{restic}) (type: package)
+The restic package to be used for the current job.
+
+@item @code{user} (default: @code{"root"}) (type: string)
+The user used for running the current job.
+
+@item @code{repository} (type: string)
+The restic repository target of this job.
+
+@item @code{password-file} (type: string)
+The path of a password file, readable by the configured @code{user},
+that will be used to set the @code{RESTIC_PASSWORD} environment variable
+for the current job.
+
+@item @code{specification} (type: gexp-or-string)
+A string or a gexp that will be passed as time specification in the
+mcron job specification (@pxref{Syntax, mcron job specifications,,
+mcron,GNU@tie{}mcron}).
+
+@item @code{included} (default: @code{'()}) (type: list-of-lowerables)
+A list of values that are lowered to strings representing filesystem
+paths.  These are the paths that will be recursively included in the
+current job.
+
+@item @code{verbose?} (default: @code{#f}) (type: boolean)
+Whether to enable verbose output for the current backup job.
+
+@item @code{extra-flags} (default: @code{'()}) (type: list-of-lowerables)
+A list of values that are lowered to strings.  These will be passed as
+command-line arguments to the current job @command{restic backup}
+invokation.
+
+@end table
+
+@end deftp
+
+
+@c %end of fragment
+
 @node Setuid Programs
 @section Setuid Programs
 
diff --git a/gnu/local.mk b/gnu/local.mk
index cabd82f532..bf911327f4 100644
--- a/gnu/local.mk
+++ b/gnu/local.mk
@@ -693,6 +693,7 @@ GNU_SYSTEM_MODULES =				\
   %D%/services/auditd.scm			\
   %D%/services/avahi.scm			\
   %D%/services/base.scm				\
+  %D%/services/backup.scm			\
   %D%/services/certbot.scm			\
   %D%/services/cgit.scm			\
   %D%/services/ci.scm				\
diff --git a/gnu/services/backup.scm b/gnu/services/backup.scm
new file mode 100644
index 0000000000..e9172af8c4
--- /dev/null
+++ b/gnu/services/backup.scm
@@ -0,0 +1,160 @@
+;;; GNU Guix --- Functional package management for GNU
+;;; Copyright © 2024 Giacomo Leidi <goodoldpaul@autistici.org>
+;;;
+;;; This file is part of GNU Guix.
+;;;
+;;; GNU Guix is free software; you can redistribute it and/or modify it
+;;; under the terms of the GNU General Public License as published by
+;;; the Free Software Foundation; either version 3 of the License, or (at
+;;; your option) any later version.
+;;;
+;;; GNU Guix is distributed in the hope that it will be useful, but
+;;; WITHOUT ANY WARRANTY; without even the implied warranty of
+;;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+;;; GNU General Public License for more details.
+;;;
+;;; You should have received a copy of the GNU General Public License
+;;; along with GNU Guix.  If not, see <http://www.gnu.org/licenses/>.
+
+(define-module (gnu services backup)
+  #:use-module (gnu packages backup)
+  #:use-module (gnu services)
+  #:use-module (gnu services configuration)
+  #:use-module (gnu services mcron)
+  #:use-module (guix gexp)
+  #:use-module (guix modules)
+  #:use-module (guix packages)
+  #:use-module (srfi srfi-1)
+  #:export (restic-backup-job
+            restic-backup-job?
+            restic-backup-job-fields
+            restic-backup-job-restic
+            restic-backup-job-user
+            restic-backup-job-repository
+            restic-backup-job-password-file
+            restic-backup-job-included
+            restic-backup-job-verbose?
+            restic-backup-job-extra-flags
+
+            restic-backup-configuration
+            restic-backup-configuration?
+            restic-backup-configuration-fields
+            restic-backup-configuration-jobs
+
+            restic-backup-job-program
+            restic-backup-job->mcron-job
+            restic-backup-service-type))
+
+(define (gexp-or-string? value)
+  (or (gexp? value)
+      (string? value)))
+
+(define (lowerable? value)
+  (or (file-like? value)
+      (gexp-or-string? value)))
+
+(define list-of-lowerables?
+  (list-of lowerable?))
+
+(define-configuration/no-serialization restic-backup-job
+  (restic
+   (package restic)
+   "The restic package to be used for the current job.")
+  (user
+   (string "root")
+   "The user used for running the current job.")
+  (repository
+   (string)
+   "The restic repository target of this job.")
+  (password-file
+   (string)
+   "The path of a password file, readable by the configured @code{user}, that
+will be used to set the @code{RESTIC_PASSWORD} environment variable for the
+current job.")
+  (specification
+   (gexp-or-string)
+   "A string or a gexp that will be passed as time specification in the mcron
+job specification (@pxref{Syntax, mcron job specifications,, mcron,
+GNU@tie{}mcron}).")
+  (included
+   (list-of-lowerables '())
+   "A list of values that are lowered to strings representing filesystem paths.
+These are the paths that will be recursively included in the current job.")
+  (verbose?
+   (boolean #f)
+   "Whether to enable verbose output for the current backup job.")
+  (extra-flags
+   (list-of-lowerables '())
+   "A list of values that are lowered to strings.  These will be passed as
+command-line arguments to the current job @command{restic backup} invokation."))
+
+(define list-of-restic-backup-jobs?
+  (list-of restic-backup-job?))
+
+(define-configuration/no-serialization restic-backup-configuration
+  (jobs
+   (list-of-restic-backup-jobs '())
+   "The list of backup jobs for the current system."))
+
+(define (restic-backup-job-program config)
+  (let ((restic
+         (file-append (restic-backup-job-restic config) "/bin/restic"))
+        (repository
+         (restic-backup-job-repository config))
+        (password-file
+         (restic-backup-job-password-file config))
+        (included
+         (restic-backup-job-included config))
+        (extra-flags
+         (restic-backup-job-extra-flags config))
+        (verbose
+         (if (restic-backup-job-verbose? config)
+             '("--verbose")
+             '())))
+    (program-file
+     "restic-backup-job.scm"
+     (with-imported-modules (source-module-closure
+                             '((guix build utils)))
+       #~(begin
+           (use-modules (guix build utils)
+                        (ice-9 popen)
+                        (ice-9 rdelim))
+           (setenv "RESTIC_PASSWORD"
+                   (with-input-from-file #$password-file read-line))
+
+           (execlp #$restic #$@verbose
+                   "-r" #$repository
+                   #$@extra-flags
+                   "backup" #$@included))))))
+
+(define (restic-backup-job->mcron-job config)
+  (let ((user
+         (restic-backup-job-user config))
+        (specification
+         (restic-backup-job-specification config))
+        (program
+         (restic-backup-job-program config)))
+    #~(job #$specification
+           #$program
+           #:user #$user)))
+
+(define restic-backup-service-type
+  (service-type (name 'restic-backup)
+                (extensions
+                 (list
+                  (service-extension mcron-service-type
+                                     (lambda (config)
+                                       (map restic-backup-job->mcron-job
+                                            (restic-backup-configuration-jobs
+                                             config))))))
+                (compose concatenate)
+                (extend
+                 (lambda (config jobs)
+                   (restic-backup-configuration
+                    (inherit config)
+                    (jobs (append (restic-backup-configuration-jobs config)
+                                  jobs)))))
+                (default-value (restic-backup-configuration))
+                (description
+                 "This service configures @code{mcron} jobs for running backups
+with @code{restic}.")))

base-commit: 6f5ea7ac1acb3d1c53baf7620cca66cc87fe5a73
-- 
2.41.0

Message #8 received at 69513@debbugs.gnu.org (full text, mbox, reply):

From: Ludovic Courtès <ludo@gnu.org>

To: Giacomo Leidi <goodoldpaul@autistici.org>

Cc: 69513@debbugs.gnu.org

Subject: Re: [bug#69513] [PATCH] services: Add restic-backup service.

Date: Fri, 29 Mar 2024 23:36:27 +0100

Hi,

Giacomo Leidi <goodoldpaul@autistici.org> skribis:

> * gnu/services/backup.scm: New file.
> * gnu/local.mk: Add this.
> * doc/guix.texi: Document this.
>
> Change-Id: I9efd5559bb445b484107a7c27c2d0a65ccad1e66

[...]

> +@subsubheading Backup services

Please capitalize headings: “Backup Services”.

We should probably move documentation of ‘syncthing-service-type’ here,
even if they live in different modules for now.

> +The @code{(gnu services backup)} module offers services for backing up
> +file system trees. For now, it provides the @code{restic-backup-service-type}.
                    ^
Nitpick: Please leave two spaces after an end-of-sentence period (for
easier Emacs navigation, readability, and consistency).

> +To backup a list of file system trees to a pre-initialized, end-to-end
> +encrypted, deduplicated data repository, you could so with the
> +@code{restic-backup-service-type}. For example with the following
> +configuration:

How about:

  With @code{restic-backup-service-type}, you can periodically back up
  directories and files with @uref{https://restic.net/, Restic}, which
  supports end-to-end encryption and deduplication.  Consider the
  following configuration:

?

> +Each @code{restic-backup-job} translates to an mcron job which sets the
> +@code{RESTIC_PASSWORD} environment variable by reading the first line of

@env{RESTIC_PASSWORD}

> +@item @code{specification} (type: gexp-or-string)
> +A string or a gexp that will be passed as time specification in the
> +mcron job specification (@pxref{Syntax, mcron job specifications,,
> +mcron,GNU@tie{}mcron}).

Maybe ‘schedule’ rather than ‘specification’, to clarify what’s being
specified?  (That’s the name I chose in <unattended-upgrade-configuration>.)

> +@item @code{included} (default: @code{'()}) (type: list-of-lowerables)
> +A list of values that are lowered to strings representing filesystem
> +paths.  These are the paths that will be recursively included in the
> +current job.

In GNU and Guix, “path” is used to denote “search paths”; in other
cases, we write “file name” or “file”.  So I’d suggest something like:

  The list of files or directories to be backed up.

The ‘files-to-backup’ (or ‘files’?) may be more descriptive that
‘included’.

> +  (password-file
> +   (string)
> +   "The path of a password file, readable by the configured @code{user}, that

“Name of the password file”

> +will be used to set the @code{RESTIC_PASSWORD} environment variable for the

s/@code/@env/

> +    (program-file
> +     "restic-backup-job.scm"
> +     (with-imported-modules (source-module-closure
> +                             '((guix build utils)))
> +       #~(begin
> +           (use-modules (guix build utils)
> +                        (ice-9 popen)
> +                        (ice-9 rdelim))
> +           (setenv "RESTIC_PASSWORD"
> +                   (with-input-from-file #$password-file read-line))
> +
> +           (execlp #$restic #$@verbose
> +                   "-r" #$repository
> +                   #$@extra-flags
> +                   "backup" #$@included))))))

I believe (guix build utils) is unused, in which case you can remove it.

The ‘execlp’ call lacks argv[0]; it should look like this:

  (execlp #$restic #$restic #$@verbose "-r" …)

(Notice that #$restic appears twice.)

Could you send an updated patch?

Thanks,
Ludo’.

Message #11 received at 69513@debbugs.gnu.org (full text, mbox, reply):

From: paul <goodoldpaul@autistici.org>

To: Ludovic Courtès <ludo@gnu.org>

Cc: 69513@debbugs.gnu.org

Subject: Re: [bug#69513] [PATCH] services: Add restic-backup service.

Date: Tue, 2 Apr 2024 22:33:17 +0200

Hello Ludo',

thank you for your insight, I should have addressed all of your 
comments. I'm sending an updated patch.


cheers

giacomo

Message #14 received at 69513@debbugs.gnu.org (full text, mbox, reply):

From: Giacomo Leidi <goodoldpaul@autistici.org>

To: 69513@debbugs.gnu.org

Cc: Giacomo Leidi <goodoldpaul@autistici.org>

Subject: [PATCH v2] services: Add restic-backup service.

Date: Tue, 2 Apr 2024 22:34:07 +0200

* gnu/services/backup.scm: New file.
* gnu/local.mk: Add this.
* doc/guix.texi: Document this.

Change-Id: I9efd5559bb445b484107a7c27c2d0a65ccad1e66
---
 doc/guix.texi           |  98 +++++++++++++++++++++++++
 gnu/local.mk            |   1 +
 gnu/services/backup.scm | 158 ++++++++++++++++++++++++++++++++++++++++
 3 files changed, 257 insertions(+)
 create mode 100644 gnu/services/backup.scm

diff --git a/doc/guix.texi b/doc/guix.texi
index 69a904473c..a13efbff7b 100644
--- a/doc/guix.texi
+++ b/doc/guix.texi
@@ -41129,6 +41129,104 @@ Miscellaneous Services
 
 @c End of auto-generated fail2ban documentation.
 
+@cindex Backup
+@subsubheading Backup Services
+
+The @code{(gnu services backup)} module offers services for backing up
+file system trees.  For now, it provides the @code{restic-backup-service-type}.
+
+With @code{restic-backup-service-type}, you can periodically back up
+directories and files with @uref{https://restic.net/, Restic}, which
+supports end-to-end encryption and deduplication.  Consider the
+following configuration:
+
+@lisp
+(operating-system
+
+  (packages (list "rclone"))
+
+  (services
+    (list
+      (service restic-backup-service-type
+               (restic-backup-configuration
+                 (jobs
+                   (list (restic-backup-job
+                           (repository "rclone:remote-ftp:backup/restic")
+                           (password-file "/root/.restic")
+                           ;; Every day at 23.
+                           (schedule "0 23 * * *")
+                           (files '("/root/.restic"
+                                    "/root/.config/rclone"
+                                    "/etc/ssh/ssh_host_rsa_key"
+                                    "/etc/ssh/ssh_host_rsa_key.pub"
+                                    "/etc/guix/signing-key.pub"
+                                    "/etc/guix/signing-key.sec"))))))))))
+@end lisp
+
+Each @code{restic-backup-job} translates to an mcron job which sets the
+@env{RESTIC_PASSWORD} environment variable by reading the first line of
+@code{password-file} and runs @command{restic backup}.
+
+@c %start of fragment
+
+@deftp {Data Type} restic-backup-configuration
+Available @code{restic-backup-configuration} fields are:
+
+@table @asis
+@item @code{jobs} (default: @code{'()}) (type: list-of-restic-backup-jobs)
+The list of backup jobs for the current system.
+
+@end table
+
+@end deftp
+
+
+@c %end of fragment
+
+@c %start of fragment
+
+@deftp {Data Type} restic-backup-job
+Available @code{restic-backup-job} fields are:
+
+@table @asis
+@item @code{restic} (default: @code{restic}) (type: package)
+The restic package to be used for the current job.
+
+@item @code{user} (default: @code{"root"}) (type: string)
+The user used for running the current job.
+
+@item @code{repository} (type: string)
+The restic repository target of this job.
+
+@item @code{password-file} (type: string)
+Name of the password file, readable by the configured @code{user},
+that will be used to set the @env{RESTIC_PASSWORD} environment variable
+for the current job.
+
+@item @code{schedule} (type: gexp-or-string)
+A string or a gexp that will be passed as time specification in the
+mcron job specification (@pxref{Syntax, mcron job specifications,,
+mcron,GNU@tie{}mcron}).
+
+@item @code{files} (default: @code{'()}) (type: list-of-lowerables)
+The list of files or directories to be backed up.  It must be a list of
+values that can be lowered to strings.
+
+@item @code{verbose?} (default: @code{#f}) (type: boolean)
+Whether to enable verbose output for the current backup job.
+
+@item @code{extra-flags} (default: @code{'()}) (type: list-of-lowerables)
+A list of values that are lowered to strings.  These will be passed as
+command-line arguments to the current job @command{restic backup}
+invokation.
+
+@end table
+
+@end deftp
+
+
+@c %end of fragment
+
 @node Setuid Programs
 @section Setuid Programs
 
diff --git a/gnu/local.mk b/gnu/local.mk
index f2b480bded..be7a968459 100644
--- a/gnu/local.mk
+++ b/gnu/local.mk
@@ -696,6 +696,7 @@ GNU_SYSTEM_MODULES =				\
   %D%/services/auditd.scm			\
   %D%/services/avahi.scm			\
   %D%/services/base.scm				\
+  %D%/services/backup.scm			\
   %D%/services/certbot.scm			\
   %D%/services/cgit.scm			\
   %D%/services/ci.scm				\
diff --git a/gnu/services/backup.scm b/gnu/services/backup.scm
new file mode 100644
index 0000000000..2bd9e2f29a
--- /dev/null
+++ b/gnu/services/backup.scm
@@ -0,0 +1,158 @@
+;;; GNU Guix --- Functional package management for GNU
+;;; Copyright © 2024 Giacomo Leidi <goodoldpaul@autistici.org>
+;;;
+;;; This file is part of GNU Guix.
+;;;
+;;; GNU Guix is free software; you can redistribute it and/or modify it
+;;; under the terms of the GNU General Public License as published by
+;;; the Free Software Foundation; either version 3 of the License, or (at
+;;; your option) any later version.
+;;;
+;;; GNU Guix is distributed in the hope that it will be useful, but
+;;; WITHOUT ANY WARRANTY; without even the implied warranty of
+;;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+;;; GNU General Public License for more details.
+;;;
+;;; You should have received a copy of the GNU General Public License
+;;; along with GNU Guix.  If not, see <http://www.gnu.org/licenses/>.
+
+(define-module (gnu services backup)
+  #:use-module (gnu packages backup)
+  #:use-module (gnu services)
+  #:use-module (gnu services configuration)
+  #:use-module (gnu services mcron)
+  #:use-module (guix gexp)
+  #:use-module (guix modules)
+  #:use-module (guix packages)
+  #:use-module (srfi srfi-1)
+  #:export (restic-backup-job
+            restic-backup-job?
+            restic-backup-job-fields
+            restic-backup-job-restic
+            restic-backup-job-user
+            restic-backup-job-repository
+            restic-backup-job-password-file
+            restic-backup-job-schedule
+            restic-backup-job-files
+            restic-backup-job-verbose?
+            restic-backup-job-extra-flags
+
+            restic-backup-configuration
+            restic-backup-configuration?
+            restic-backup-configuration-fields
+            restic-backup-configuration-jobs
+
+            restic-backup-job-program
+            restic-backup-job->mcron-job
+            restic-backup-service-type))
+
+(define (gexp-or-string? value)
+  (or (gexp? value)
+      (string? value)))
+
+(define (lowerable? value)
+  (or (file-like? value)
+      (gexp-or-string? value)))
+
+(define list-of-lowerables?
+  (list-of lowerable?))
+
+(define-configuration/no-serialization restic-backup-job
+  (restic
+   (package restic)
+   "The restic package to be used for the current job.")
+  (user
+   (string "root")
+   "The user used for running the current job.")
+  (repository
+   (string)
+   "The restic repository target of this job.")
+  (password-file
+   (string)
+   "Name of the password file, readable by the configured @code{user}, that
+will be used to set the @code{RESTIC_PASSWORD} environment variable for the
+current job.")
+  (schedule
+   (gexp-or-string)
+   "A string or a gexp that will be passed as time specification in the mcron
+job specification (@pxref{Syntax, mcron job specifications,, mcron,
+GNU@tie{}mcron}).")
+  (files
+   (list-of-lowerables '())
+   "The list of files or directories to be backed up.  It must be a list of
+values that can be lowered to strings.")
+  (verbose?
+   (boolean #f)
+   "Whether to enable verbose output for the current backup job.")
+  (extra-flags
+   (list-of-lowerables '())
+   "A list of values that are lowered to strings.  These will be passed as
+command-line arguments to the current job @command{restic backup} invokation."))
+
+(define list-of-restic-backup-jobs?
+  (list-of restic-backup-job?))
+
+(define-configuration/no-serialization restic-backup-configuration
+  (jobs
+   (list-of-restic-backup-jobs '())
+   "The list of backup jobs for the current system."))
+
+(define (restic-backup-job-program config)
+  (let ((restic
+         (file-append (restic-backup-job-restic config) "/bin/restic"))
+        (repository
+         (restic-backup-job-repository config))
+        (password-file
+         (restic-backup-job-password-file config))
+        (files
+         (restic-backup-job-files config))
+        (extra-flags
+         (restic-backup-job-extra-flags config))
+        (verbose
+         (if (restic-backup-job-verbose? config)
+             '("--verbose")
+             '())))
+    (program-file
+     "restic-backup-job.scm"
+     #~(begin
+         (use-modules (ice-9 popen)
+                      (ice-9 rdelim))
+         (setenv "RESTIC_PASSWORD"
+                 (with-input-from-file #$password-file read-line))
+
+         (execlp #$restic #$restic #$@verbose
+                 "-r" #$repository
+                 #$@extra-flags
+                 "backup" #$@files)))))
+
+(define (restic-backup-job->mcron-job config)
+  (let ((user
+         (restic-backup-job-user config))
+        (schedule
+         (restic-backup-job-schedule config))
+        (program
+         (restic-backup-job-program config)))
+    #~(job #$schedule
+           #$program
+           #:user #$user)))
+
+(define restic-backup-service-type
+  (service-type (name 'restic-backup)
+                (extensions
+                 (list
+                  (service-extension mcron-service-type
+                                     (lambda (config)
+                                       (map restic-backup-job->mcron-job
+                                            (restic-backup-configuration-jobs
+                                             config))))))
+                (compose concatenate)
+                (extend
+                 (lambda (config jobs)
+                   (restic-backup-configuration
+                    (inherit config)
+                    (jobs (append (restic-backup-configuration-jobs config)
+                                  jobs)))))
+                (default-value (restic-backup-configuration))
+                (description
+                 "This service configures @code{mcron} jobs for running backups
+with @code{restic}.")))

base-commit: 7af70efd7633b0d70091762cf43ce01a86176e8e
-- 
2.41.0

Message #17 received at 69513@debbugs.gnu.org (full text, mbox, reply):

From: paul <goodoldpaul@autistici.org>

To: Ludovic Courtès <ludo@gnu.org>

Cc: 69513@debbugs.gnu.org

Subject: Re: [bug#69513] [PATCH] services: Add restic-backup service.

Date: Wed, 1 May 2024 23:14:29 +0200

Hello Ludo' ,


I'm sending a v3 rebased on current master. In this revision I added the 
possibility to manually trigger a backup without waiting for the 
scheduled job to run.

Thank you for your work,


giacomo

Message #20 received at 69513@debbugs.gnu.org (full text, mbox, reply):

From: Giacomo Leidi <goodoldpaul@autistici.org>

To: 69513@debbugs.gnu.org

Cc: Giacomo Leidi <goodoldpaul@autistici.org>

Subject: [PATCH v3] services: Add restic-backup service.

Date: Wed, 1 May 2024 23:15:07 +0200

* gnu/services/backup.scm: New file.
* gnu/local.mk: Add this.
* doc/guix.texi: Document this.

Change-Id: I9efd5559bb445b484107a7c27c2d0a65ccad1e66
---
 doc/guix.texi           | 112 +++++++++++++++++++
 gnu/local.mk            |   1 +
 gnu/services/backup.scm | 236 ++++++++++++++++++++++++++++++++++++++++
 3 files changed, 349 insertions(+)
 create mode 100644 gnu/services/backup.scm

diff --git a/doc/guix.texi b/doc/guix.texi
index 3f5d4e7f0d..1b52c43c34 100644
--- a/doc/guix.texi
+++ b/doc/guix.texi
@@ -40946,6 +40946,118 @@ Miscellaneous Services
 
 @c End of auto-generated fail2ban documentation.
 
+@cindex Backup
+@subsubheading Backup Services
+
+The @code{(gnu services backup)} module offers services for backing up
+file system trees.  For now, it provides the @code{restic-backup-service-type}.
+
+With @code{restic-backup-service-type}, you can periodically back up
+directories and files with @uref{https://restic.net/, Restic}, which
+supports end-to-end encryption and deduplication.  Consider the
+following configuration:
+
+@lisp
+(operating-system
+
+  (packages (list "rclone"))
+
+  (services
+    (list
+      (service restic-backup-service-type
+               (restic-backup-configuration
+                 (jobs
+                   (list (restic-backup-job
+                           (name "remote-ftp")
+                           (repository "rclone:remote-ftp:backup/restic")
+                           (password-file "/root/.restic")
+                           ;; Every day at 23.
+                           (schedule "0 23 * * *")
+                           (files '("/root/.restic"
+                                    "/root/.config/rclone"
+                                    "/etc/ssh/ssh_host_rsa_key"
+                                    "/etc/ssh/ssh_host_rsa_key.pub"
+                                    "/etc/guix/signing-key.pub"
+                                    "/etc/guix/signing-key.sec"))))))))))
+@end lisp
+
+Each @code{restic-backup-job} translates to an mcron job which sets the
+@env{RESTIC_PASSWORD} environment variable by reading the first line of
+@code{password-file} and runs @command{restic backup}.
+
+The @code{restic-backup-service-type} installs as well @code{restic-guix}
+to the system profile, a @code{restic} utility wrapper that allows for easier
+interaction with the Guix configured backup jobs.  For example the following
+could be used to instantaneusly trigger a backup for the above shown
+configuration, without waiting for the scheduled job:
+
+@example
+restic-guix backup remote-ftp
+@end example
+
+@c %start of fragment
+
+@deftp {Data Type} restic-backup-configuration
+Available @code{restic-backup-configuration} fields are:
+
+@table @asis
+@item @code{jobs} (default: @code{'()}) (type: list-of-restic-backup-jobs)
+The list of backup jobs for the current system.
+
+@end table
+
+@end deftp
+
+
+@c %end of fragment
+
+@c %start of fragment
+
+@deftp {Data Type} restic-backup-job
+Available @code{restic-backup-job} fields are:
+
+@table @asis
+@item @code{restic} (default: @code{restic}) (type: package)
+The restic package to be used for the current job.
+
+@item @code{user} (default: @code{"root"}) (type: string)
+The user used for running the current job.
+
+@item @code{repository} (type: string)
+The restic repository target of this job.
+
+@item @code{name} (type: string)
+A string denoting a name for this job.
+
+@item @code{password-file} (type: string)
+Name of the password file, readable by the configured @code{user},
+that will be used to set the @env{RESTIC_PASSWORD} environment variable
+for the current job.
+
+@item @code{schedule} (type: gexp-or-string)
+A string or a gexp that will be passed as time specification in the
+mcron job specification (@pxref{Syntax, mcron job specifications,,
+mcron,GNU@tie{}mcron}).
+
+@item @code{files} (default: @code{'()}) (type: list-of-lowerables)
+The list of files or directories to be backed up.  It must be a list of
+values that can be lowered to strings.
+
+@item @code{verbose?} (default: @code{#f}) (type: boolean)
+Whether to enable verbose output for the current backup job.
+
+@item @code{extra-flags} (default: @code{'()}) (type: list-of-lowerables)
+A list of values that are lowered to strings.  These will be passed as
+command-line arguments to the current job @command{restic backup}
+invokation.
+
+@end table
+
+@end deftp
+
+
+@c %end of fragment
+
 @node Setuid Programs
 @section Setuid Programs
 
diff --git a/gnu/local.mk b/gnu/local.mk
index f1dab53f2b..bb17ef182a 100644
--- a/gnu/local.mk
+++ b/gnu/local.mk
@@ -699,6 +699,7 @@ GNU_SYSTEM_MODULES =				\
   %D%/services/auditd.scm			\
   %D%/services/avahi.scm			\
   %D%/services/base.scm				\
+  %D%/services/backup.scm			\
   %D%/services/certbot.scm			\
   %D%/services/cgit.scm			\
   %D%/services/ci.scm				\
diff --git a/gnu/services/backup.scm b/gnu/services/backup.scm
new file mode 100644
index 0000000000..555e9fc959
--- /dev/null
+++ b/gnu/services/backup.scm
@@ -0,0 +1,236 @@
+;;; GNU Guix --- Functional package management for GNU
+;;; Copyright © 2024 Giacomo Leidi <goodoldpaul@autistici.org>
+;;;
+;;; This file is part of GNU Guix.
+;;;
+;;; GNU Guix is free software; you can redistribute it and/or modify it
+;;; under the terms of the GNU General Public License as published by
+;;; the Free Software Foundation; either version 3 of the License, or (at
+;;; your option) any later version.
+;;;
+;;; GNU Guix is distributed in the hope that it will be useful, but
+;;; WITHOUT ANY WARRANTY; without even the implied warranty of
+;;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+;;; GNU General Public License for more details.
+;;;
+;;; You should have received a copy of the GNU General Public License
+;;; along with GNU Guix.  If not, see <http://www.gnu.org/licenses/>.
+
+(define-module (gnu services backup)
+  #:use-module (gnu packages backup)
+  #:use-module (gnu services)
+  #:use-module (gnu services configuration)
+  #:use-module (gnu services mcron)
+  #:use-module (guix build-system copy)
+  #:use-module (guix gexp)
+  #:use-module ((guix licenses)
+                #:prefix license:)
+  #:use-module (guix modules)
+  #:use-module (guix packages)
+  #:use-module (srfi srfi-1)
+  #:export (restic-backup-job
+            restic-backup-job?
+            restic-backup-job-fields
+            restic-backup-job-restic
+            restic-backup-job-user
+            restic-backup-job-name
+            restic-backup-job-repository
+            restic-backup-job-password-file
+            restic-backup-job-schedule
+            restic-backup-job-files
+            restic-backup-job-verbose?
+            restic-backup-job-extra-flags
+
+            restic-backup-configuration
+            restic-backup-configuration?
+            restic-backup-configuration-fields
+            restic-backup-configuration-jobs
+
+            restic-backup-job-program
+            restic-backup-job->mcron-job
+            restic-guix
+            restic-guix-wrapper-package
+            restic-backup-service-profile
+            restic-backup-service-type))
+
+(define (gexp-or-string? value)
+  (or (gexp? value)
+      (string? value)))
+
+(define (lowerable? value)
+  (or (file-like? value)
+      (gexp-or-string? value)))
+
+(define list-of-lowerables?
+  (list-of lowerable?))
+
+(define-configuration/no-serialization restic-backup-job
+  (restic
+   (package restic)
+   "The restic package to be used for the current job.")
+  (user
+   (string "root")
+   "The user used for running the current job.")
+  (name
+   (string)
+   "A string denoting a name for this job.")
+  (repository
+   (string)
+   "The restic repository target of this job.")
+  (password-file
+   (string)
+   "Name of the password file, readable by the configured @code{user}, that
+will be used to set the @code{RESTIC_PASSWORD} environment variable for the
+current job.")
+  (schedule
+   (gexp-or-string)
+   "A string or a gexp that will be passed as time specification in the mcron
+job specification (@pxref{Syntax, mcron job specifications,, mcron,
+GNU@tie{}mcron}).")
+  (files
+   (list-of-lowerables '())
+   "The list of files or directories to be backed up.  It must be a list of
+values that can be lowered to strings.")
+  (verbose?
+   (boolean #f)
+   "Whether to enable verbose output for the current backup job.")
+  (extra-flags
+   (list-of-lowerables '())
+   "A list of values that are lowered to strings.  These will be passed as
+command-line arguments to the current job @command{restic backup} invokation."))
+
+(define list-of-restic-backup-jobs?
+  (list-of restic-backup-job?))
+
+(define-configuration/no-serialization restic-backup-configuration
+  (jobs
+   (list-of-restic-backup-jobs '())
+   "The list of backup jobs for the current system."))
+
+(define (restic-backup-job-program config)
+  (let ((restic
+         (file-append (restic-backup-job-restic config) "/bin/restic"))
+        (repository
+         (restic-backup-job-repository config))
+        (password-file
+         (restic-backup-job-password-file config))
+        (files
+         (restic-backup-job-files config))
+        (extra-flags
+         (restic-backup-job-extra-flags config))
+        (verbose
+         (if (restic-backup-job-verbose? config)
+             '("--verbose")
+             '())))
+    (program-file
+     "restic-backup-job.scm"
+     #~(begin
+         (use-modules (ice-9 popen)
+                      (ice-9 rdelim))
+         (setenv "RESTIC_PASSWORD"
+                 (with-input-from-file #$password-file read-line))
+
+         (execlp #$restic #$restic #$@verbose
+                 "-r" #$repository
+                 #$@extra-flags
+                 "backup" #$@files)))))
+
+(define (restic-guix jobs)
+  (program-file
+   "restic-guix"
+   #~(begin
+       (use-modules (ice-9 match)
+                    (srfi srfi-1))
+
+       (define names '#$(map restic-backup-job-name jobs))
+       (define programs '#$(map restic-backup-job-program jobs))
+
+       (define (get-program name)
+         (define idx
+           (list-index (lambda (n) (string=? n name)) names))
+         (unless idx
+           (error (string-append "Unknown job name " name "\n\n"
+                                 "Possible job names are: "
+                                 (string-join names " "))))
+         (list-ref programs idx))
+
+       (define (backup args)
+         (define name (third args))
+         (define program (get-program name))
+         (execlp program program))
+
+       (define (validate-args args)
+         (when (not (>= (length args) 3))
+           (error (string-append "Usage: " (basename (car args))
+                                 " backup NAME"))))
+
+       (define (main args)
+         (validate-args args)
+         (define action (second args))
+         (match action
+           ("backup"
+            (backup args))
+           (_
+            (error (string-append "Unknown action: " action)))))
+
+       (main (command-line)))))
+
+(define (restic-backup-job->mcron-job config)
+  (let ((user
+         (restic-backup-job-user config))
+        (schedule
+         (restic-backup-job-schedule config))
+        (name
+         (restic-backup-job-name config)))
+    #~(job #$schedule
+           #$(string-append "restic-guix backup " name)
+           #:user #$user)))
+
+(define (restic-guix-wrapper-package jobs)
+  (package
+    (name "restic-backup-service-wrapper")
+    (version "0.0.0")
+    (source (restic-guix jobs))
+    (build-system copy-build-system)
+    (arguments
+     (list #:install-plan #~'(("./" "/bin"))))
+    (home-page "https://restic.net")
+    (synopsis
+     "Easily interact from the CLI with Guix configured backups")
+    (description
+     "This package provides a simple wrapper around @code{restic}, handled
+by the @code{restic-backup-service-type}.  It allows for easily interacting
+with Guix configured backup jobs, for example for manually triggering a backup
+without waiting for the scheduled job to run.")
+    (license license:gpl3+)))
+
+(define restic-backup-service-profile
+  (lambda (config)
+    (define jobs (restic-backup-configuration-jobs config))
+    (if (> (length jobs) 0)
+        (list
+         (restic-guix-wrapper-package jobs))
+        '())))
+
+(define restic-backup-service-type
+  (service-type (name 'restic-backup)
+                (extensions
+                 (list
+                  (service-extension profile-service-type
+                                     restic-backup-service-profile)
+                  (service-extension mcron-service-type
+                                     (lambda (config)
+                                       (map restic-backup-job->mcron-job
+                                            (restic-backup-configuration-jobs
+                                             config))))))
+                (compose concatenate)
+                (extend
+                 (lambda (config jobs)
+                   (restic-backup-configuration
+                    (inherit config)
+                    (jobs (append (restic-backup-configuration-jobs config)
+                                  jobs)))))
+                (default-value (restic-backup-configuration))
+                (description
+                 "This service configures @code{mcron} jobs for running backups
+with @code{restic}.")))

base-commit: 7d4ae2fca723114fb1df56de33b82177fbc4d0a6
-- 
2.41.0

Message #25 received at 69513-done@debbugs.gnu.org (full text, mbox, reply):

From: Ludovic Courtès <ludo@gnu.org>

To: Giacomo Leidi <goodoldpaul@autistici.org>

Cc: 69513-done@debbugs.gnu.org

Subject: Re: [bug#69513] [PATCH v3] services: Add restic-backup service.

Date: Sat, 25 May 2024 15:23:57 +0200

[Message part 1 (text/plain, inline)]

Hello,

Giacomo Leidi <goodoldpaul@autistici.org> skribis:

> * gnu/services/backup.scm: New file.
> * gnu/local.mk: Add this.
> * doc/guix.texi: Document this.
>
> Change-Id: I9efd5559bb445b484107a7c27c2d0a65ccad1e66

Please consider adding a system test for this: as previously discussed,
we try hard to have tests for every system service.

I’ve applied it with the minor doc changes below.

There might be more work that could be done to ensure the doc is
self-contained.  For instance, I merely guessed that ‘rclone’ needed to
be in the system profile so ‘restic’ would use it, and I cannot tell
what those repository URIs should look like.  Perhaps this can be solved
with a few more words, examples, and/or links to the upstream doc?

Thank you!

Ludo’.

[Message part 2 (text/x-patch, inline)]

diff --git a/doc/guix.texi b/doc/guix.texi
index acf35357a60..d2643cf7fd9 100644
--- a/doc/guix.texi
+++ b/doc/guix.texi
@@ -41102,10 +41102,13 @@ Miscellaneous Services
 following configuration:
 
 @lisp
+(use-service-modules backup @dots{}) ;for 'restic-backup-service-type'
+(use-package-modules sync @dots{})   ;for 'rclone'
+
 (operating-system
-
-  (packages (list "rclone"))
-
+  ;; @dots{}
+  (packages (append (list rclone)    ;for use by restic
+                    %base-packages))
   (services
     (list
       (service restic-backup-service-type
@@ -41127,7 +41130,8 @@ Miscellaneous Services
 
 Each @code{restic-backup-job} translates to an mcron job which sets the
 @env{RESTIC_PASSWORD} environment variable by reading the first line of
-@code{password-file} and runs @command{restic backup}.
+@code{password-file} and runs @command{restic backup}, creating backups
+using rclone of all the files listed in the @code{files} field.
 
 The @code{restic-backup-service-type} installs as well @code{restic-guix}
 to the system profile, a @code{restic} utility wrapper that allows for easier

Message #28 received at 69513@debbugs.gnu.org (full text, mbox, reply):

From: Richard Sent <richard@freakingpenguin.com>

To: 69513@debbugs.gnu.org

Cc: goodoldpaul@autistici.org

Subject: Adding a couple new features and tests

Date: Thu, 30 May 2024 15:23:46 -0400

Hi all,

> Please consider adding a system test for this: as previously
> discussed, we try hard to have tests for every system service.

I have a few changes I plan on submitting soon regarding the restic
service. FYSA Giacomo, I'm writing system tests as well, although those
tests will obviously be limited to local repositories.

For the curious I'm working on the following changes:

1. Add an init? field to attempt to bootstrap local repository
initialization, ala https://github.com/NixOS/nixpkgs/pull/307962

2. Add support for Restic's password command feature in addition to the
existing password file

3. Either add rclone to the profile or add an extra-packages field to
restic-backup-job so restic-required packages can be located in one
spot.

4. The aforementioned tests

I haven't used restic before so this might be harder than I think it'll
be. So far things seem to be going well.

-- 
Take it easy,
Richard Sent
Making my computer weirder one commit at a time.

Send a report that this bug log contains spam.