GNU bug report logs

#61363 [PATCH 0/2] self: Apply grafts to the outputs of the guix derivation.

PackageSource(s)Maintainer(s)
guix-patches PTS Buildd Popcon
Reply or subscribe to this bug. View this bug as an mbox, status mbox, or maintainer mbox

Report forwarded to guix-patches@gnu.org:
bug#61363; Package guix-patches. (Wed, 08 Feb 2023 07:49:02 GMT) (full text, mbox, link).


Acknowledgement sent to Christopher Baines <mail@cbaines.net>:
New bug report received and forwarded. Copy sent to guix-patches@gnu.org. (Wed, 08 Feb 2023 07:49:02 GMT) (full text, mbox, link).


Message #5 received at submit@debbugs.gnu.org (full text, mbox, reply):

From: Christopher Baines <mail@cbaines.net>
To: guix-patches@gnu.org
Subject: [PATCH 0/2] self: Apply grafts to the outputs of the guix derivation.
Date: Wed, 08 Feb 2023 08:46:17 +0100
[Message part 1 (text/plain, inline)]
These patches mean that grafts apply to the outputs of the guix
derivation, rather than having grafts apply to the derivation
itself. This moves grafting here to work like grafting for packages,
where you can think of the grafted outputs as a transformed variant of
the ungrafted outputs.

I'm looking at this as it'll allow the Guix Data Service to compute the
derivations without grafts, and for these to be useful for substitutes
regardless of whether users are using grafts.


Christopher Baines (2):
  packages: Add explicit-grafting record type to assist with grafts.
  self: Apply grafts to the outputs of the guix derivation.

 build-aux/build-self.scm |  4 ++-
 guix/packages.scm        | 45 +++++++++++++++++++++++++++-
 guix/self.scm            | 65 ++++++++++++++++++++++++++--------------
 3 files changed, 89 insertions(+), 25 deletions(-)

-- 
2.38.1
[signature.asc (application/pgp-signature, inline)]

Information forwarded to ludo@gnu.org, guix-patches@gnu.org:
bug#61363; Package guix-patches. (Wed, 08 Feb 2023 07:55:01 GMT) (full text, mbox, link).


Message #8 received at 61363@debbugs.gnu.org (full text, mbox, reply):

From: Christopher Baines <mail@cbaines.net>
To: 61363@debbugs.gnu.org
Subject: [PATCH 1/2] packages: Add explicit-grafting record type to assist with grafts.
Date: Wed, 8 Feb 2023 08:54:02 +0100
Normally the grafting takes place when lowering packages, but this record
assists with applying the same transformation to arbitrary objects/store
items.

I'm adding this to allow grafting the channel instance derivation outputs.

* guix/packages.scm (explicit-grafting, explicit-grafting?,
explicit-grafting-obj, explicit-grafting-grafts): New procedures.
---
 guix/packages.scm | 45 ++++++++++++++++++++++++++++++++++++++++++++-
 1 file changed, 44 insertions(+), 1 deletion(-)

diff --git a/guix/packages.scm b/guix/packages.scm
index 041a872f9d..877bf89522 100644
--- a/guix/packages.scm
+++ b/guix/packages.scm
@@ -188,7 +188,12 @@ (define-module (guix packages)
             package-file
             package->derivation
             package->cross-derivation
-            origin->derivation))
+            origin->derivation
+
+            explicit-grafting
+            explicit-grafting?
+            explicit-grafting-obj
+            explicit-grafting-grafts))
 
 ;; The 'source-module-closure' procedure ca. 1.2.0 did not recognize
 ;; #:re-export-and-replace: <https://issues.guix.gnu.org/52694>.
@@ -2093,3 +2098,41 @@ (define package-source-derivation                 ;somewhat deprecated
          (add-to-store store (basename file) #t "sha256" file))
         (_
          (lower store source system))))))
+
+;; Apply grafts explicitly
+(define-immutable-record-type <explicit-grafting>
+  (%explicit-grafting obj packages)
+  explicit-grafting?
+  (obj      explicit-grafting-obj)       ;obj
+  (packages explicit-grafting-packages)) ;list of <package>s
+
+(define (write-explicit-grafting rec port)
+  (match rec
+    (($ <explicit-grafting> obj packages)
+     (format port "#<explicit-grafting ~s ~s>" obj packages))))
+
+(define (explicit-grafting obj packages)
+  (%explicit-grafting obj packages))
+
+(define-gexp-compiler (explicit-grafting-compiler (explicit-grafting <explicit-grafting>)
+                                                  system target)
+  (match explicit-grafting
+    (($ <explicit-grafting> obj packages)
+     (mlet* %store-monad ((drv (without-grafting
+                                (lower-object obj system #:target target)))
+                          (grafts
+                           (mapm %store-monad
+                                 (lambda (pkg)
+                                   (package-grafts* pkg system #:target target))
+                                 packages)))
+       (match (delete-duplicates
+               (concatenate grafts))
+         (()
+          (return drv))
+         (grafts
+          (mlet %store-monad ((guile (package->derivation
+                                      (guile-for-grafts)
+                                      system #:graft? #f)))
+            (graft-derivation* drv grafts
+                               #:system system
+                               #:guile guile))))))))
-- 
2.38.1





Information forwarded to ludo@gnu.org, guix-patches@gnu.org:
bug#61363; Package guix-patches. (Wed, 08 Feb 2023 07:55:01 GMT) (full text, mbox, link).


Message #11 received at 61363@debbugs.gnu.org (full text, mbox, reply):

From: Christopher Baines <mail@cbaines.net>
To: 61363@debbugs.gnu.org
Subject: [PATCH 2/2] self: Apply grafts to the outputs of the guix derivation.
Date: Wed, 8 Feb 2023 08:54:03 +0100
Rather than having grafts apply to the derivation itself. This moves grafting
here to work like grafting for packages, where you can think of the grafted
outputs as a transformed variant of the ungrafted outputs.

I'm looking at this as it'll allow the Guix Data Service to compute the
derivations without grafts, and for these to be useful for substitutes
regardless of whether users are using grafts.

* guix/self.scm (compiled-guix, guix-derivation): Add a #:graft? keyword
argument, to control grafting when computing the guix derivation.
* build-aux/build-self.scm (build-program): Call guix-derivation with
 #:graft? (%graft?) to make the compute-guix-derivation script use or not use
grafts as desired.
---
 build-aux/build-self.scm |  4 ++-
 guix/self.scm            | 65 ++++++++++++++++++++++++++--------------
 2 files changed, 45 insertions(+), 24 deletions(-)

diff --git a/build-aux/build-self.scm b/build-aux/build-self.scm
index 02822a2ee8..6d0037f20c 100644
--- a/build-aux/build-self.scm
+++ b/build-aux/build-self.scm
@@ -353,7 +353,9 @@ (define fake-git
                                                   #:channel-metadata
                                                   '#$channel-metadata
                                                   #:pull-version
-                                                  #$pull-version)
+                                                  #$pull-version
+                                                  #:graft?
+                                                  #$(%graft?))
                                  #:system system))
                              derivation-file-name))))))
                   #:module-path (list source))))
diff --git a/guix/self.scm b/guix/self.scm
index 93019e1c64..c944dbe9ce 100644
--- a/guix/self.scm
+++ b/guix/self.scm
@@ -752,7 +752,8 @@ (define* (compiled-guix source #:key
                         (gzip (specification->package "gzip"))
                         (bzip2 (specification->package "bzip2"))
                         (xz (specification->package "xz"))
-                        (guix (specification->package "guix")))
+                        (guix (specification->package "guix"))
+                        (graft? #t))
   "Return a file-like object that contains a compiled Guix."
   (define guile-avahi
     (specification->package "guile-avahi"))
@@ -802,6 +803,12 @@ (define dependencies
                       guile-json guile-semver guile-ssh guile-sqlite3
                       guile-lib guile-zlib guile-lzlib guile-zstd)))
 
+  (define packages
+    (cons* gzip
+           bzip2
+           xz
+           dependencies))
+
   (define *core-modules*
     (scheme-node "guix-core"
                  '((guix)
@@ -1022,28 +1029,35 @@ (define (built-modules node-subset)
                                                guile-lzma
                                                dependencies)
                                         #:guile guile-for-build
-                                        #:guile-version guile-version)))
-           (whole-package name modules dependencies
-                          #:command command
-                          #:guile guile-for-build
-
-                          ;; Include 'guix-daemon'.  XXX: Here we inject an
-                          ;; older snapshot of guix-daemon, but that's a good
-                          ;; enough approximation for now.
-                          #:daemon (module-ref (resolve-interface
-                                                '(gnu packages
-                                                      package-management))
-                                               'guix-daemon)
-
-                          #:info (info-manual source)
-                          #:miscellany (miscellaneous-files source)
-                          #:guile-version guile-version)))
+                                        #:guile-version guile-version))
+                (obj
+                 (whole-package name modules dependencies
+                                #:command command
+                                #:guile guile-for-build
+
+                                ;; Include 'guix-daemon'.  XXX: Here we inject
+                                ;; an older snapshot of guix-daemon, but
+                                ;; that's a good enough approximation for now.
+                                #:daemon (module-ref (resolve-interface
+                                                      '(gnu packages
+                                                            package-management))
+                                                     'guix-daemon)
+
+                                #:info (info-manual source)
+                                #:miscellany (miscellaneous-files source)
+                                #:guile-version guile-version)))
+           (if graft?
+               (explicit-grafting obj packages)
+               obj)))
         ((= 0 pull-version)
          ;; Legacy 'guix pull': return the .scm and .go files as one
          ;; directory.
-         (built-modules (lambda (node)
-                          (list (node-source node)
-                                (node-compiled node)))))
+         (let ((obj (built-modules (lambda (node)
+                                     (list (node-source node)
+                                           (node-compiled node))))))
+           (if graft?
+               (explicit-grafting obj packages)
+               obj)))
         (else
          ;; Unsupported 'guix pull' version.
          #f)))
@@ -1273,7 +1287,8 @@ (define (process-directory directory files output)
 (define* (guix-derivation source version
                           #:optional (guile-version (effective-version))
                           #:key (pull-version 0)
-                          channel-metadata)
+                          channel-metadata
+                          (graft? #t))
   "Return, as a monadic value, the derivation to build the Guix from SOURCE
 for GUILE-VERSION.  Use VERSION as the version string.  Use CHANNEL-METADATA
 as the channel metadata sexp to include in (guix config).
@@ -1310,7 +1325,11 @@ (define guile
                                #:pull-version pull-version
                                #:guile-version (if (>= pull-version 1)
                                                    "3.0" guile-version)
-                               #:guile-for-build guile)))
+                               #:guile-for-build guile
+                               #:graft? graft?)))
       (if guix
-          (lower-object guix)
+          (if graft?
+              (lower-object guix)
+              (without-grafting
+               (lower-object guix)))
           (return #f)))))
-- 
2.38.1





Information forwarded to guix-patches@gnu.org:
bug#61363; Package guix-patches. (Fri, 10 Feb 2023 09:20:02 GMT) (full text, mbox, link).


Message #14 received at 61363@debbugs.gnu.org (full text, mbox, reply):

From: Christopher Baines <mail@cbaines.net>
To: Christopher Baines <mail@cbaines.net>
Cc: 61363@debbugs.gnu.org
Subject: Re: [bug#61363] [PATCH 0/2] self: Apply grafts to the outputs of the guix derivation.
Date: Fri, 10 Feb 2023 09:16:12 +0000
[Message part 1 (text/plain, inline)]
The data service comparison is now available for this, and while there
are no differences in the packages, you can see some information.

This is the channel instances before:

  https://data.qa.guix.gnu.org/revision/a582d863465990642d331bc05bf073f47fb80908/channel-instances

and this is after:

  https://data.qa.guix.gnu.org/revision/9cfbb22b556d28a0af345824ae5b3e00eb3f4a15/channel-instances
[signature.asc (application/pgp-signature, inline)]

Information forwarded to guix-patches@gnu.org:
bug#61363; Package guix-patches. (Wed, 22 Feb 2023 09:17:01 GMT) (full text, mbox, link).


Message #17 received at 61363@debbugs.gnu.org (full text, mbox, reply):

From: Ludovic Courtès <ludo@gnu.org>
To: Christopher Baines <mail@cbaines.net>
Cc: 61363@debbugs.gnu.org
Subject: Re: [bug#61363] [PATCH 2/2] self: Apply grafts to the outputs of the guix derivation.
Date: Wed, 22 Feb 2023 10:16:14 +0100
Hi,

Christopher Baines <mail@cbaines.net> skribis:

> Rather than having grafts apply to the derivation itself. This moves grafting
> here to work like grafting for packages, where you can think of the grafted
> outputs as a transformed variant of the ungrafted outputs.

Hmm.

> I'm looking at this as it'll allow the Guix Data Service to compute the
> derivations without grafts, and for these to be useful for substitutes
> regardless of whether users are using grafts.

How does it help exactly?  By disabling grafts in that context?

> +++ b/guix/self.scm
> @@ -752,7 +752,8 @@ (define* (compiled-guix source #:key
>                          (gzip (specification->package "gzip"))
>                          (bzip2 (specification->package "bzip2"))
>                          (xz (specification->package "xz"))
> -                        (guix (specification->package "guix")))
> +                        (guix (specification->package "guix"))
> +                        (graft? #t))
>    "Return a file-like object that contains a compiled Guix."
>    (define guile-avahi
>      (specification->package "guile-avahi"))
> @@ -802,6 +803,12 @@ (define dependencies
>                        guile-json guile-semver guile-ssh guile-sqlite3
>                        guile-lib guile-zlib guile-lzlib guile-zstd)))
>  
> +  (define packages
> +    (cons* gzip
> +           bzip2
> +           xz
> +           dependencies))
> +

[...]

> +         (let ((obj (built-modules (lambda (node)
> +                                     (list (node-source node)
> +                                           (node-compiled node))))))
> +           (if graft?
> +               (explicit-grafting obj packages)
> +               obj)))

There are two things I’m not comfortable with:

  1. Having <explicit-grafting> in (guix packages); it looks misplaced.

  2. More importantly, manually listing packages that might require
     grafting looks like a slippery slope (“oops! we’re not getting the
     GnuTLS graft for that CVE, too bad”).

I designed and implemented several variants to try and delay grafting.
One of them consisted in carrying graft information in gexps:

  https://git.savannah.gnu.org/cgit/guix.git/log?h=wip-gexp-grafts

It’s kinda similar to what you’re proposing in that graft information is
carried as far as possible.  The main difference is that it’s automated.

Hmm needs more thought.

Ludo’.




Information forwarded to guix-patches@gnu.org:
bug#61363; Package guix-patches. (Wed, 22 Feb 2023 15:08:01 GMT) (full text, mbox, link).


Message #20 received at 61363@debbugs.gnu.org (full text, mbox, reply):

From: Christopher Baines <mail@cbaines.net>
To: Ludovic Courtès <ludo@gnu.org>
Cc: 61363@debbugs.gnu.org
Subject: Re: [bug#61363] [PATCH 2/2] self: Apply grafts to the outputs of the guix derivation.
Date: Wed, 22 Feb 2023 11:17:48 +0000
[Message part 1 (text/plain, inline)]
Ludovic Courtès <ludo@gnu.org> writes:

>> I'm looking at this as it'll allow the Guix Data Service to compute the
>> derivations without grafts, and for these to be useful for substitutes
>> regardless of whether users are using grafts.
>
> How does it help exactly?  By disabling grafts in that context?

So the Guix Data Service is somewhat built on the assumption that it's
cheap to compute derivations, at least with grafts disabled. That's
always been the case for packages, but for channel instance derivations
it's not reliably the case, since currently disabling grafts doesn't
apply to the whole process, and even if it did, the derivations you'd
get out wouldn't be that useful (since you can't transform the outputs
from those derivations to the outputs you'd get if using grafts).

With these changes, it's always relatively cheap to compute the channel
instance derivations, and it's always possible to compute the
derivations for any system without needing to be able to perform builds
for that system.

You can see this in how the data service has processed Guix before and
after these patches.

This is the channel instances before:

  https://data.qa.guix.gnu.org/revision/a582d863465990642d331bc05bf073f47fb80908/channel-instances

and this is after:

  https://data.qa.guix.gnu.org/revision/9cfbb22b556d28a0af345824ae5b3e00eb3f4a15/channel-instances

Given data.qa.guix.gnu.org is running on an x86_64-linux system, that
and i686-linux isn't generally a problem, but I'm guessing it only
managed to compute the powerpc64le-linux and aarch64-linux derivations
because it was able to substitute the necessary store items. For other
system computing the derivations would have failed.

I believe this change will also mean that the build farms will go from
performing the grafting for these builds, to being able to not do so, in
line with how builds for packages are handled. This isn't a big thing,
but I think it makes sense.

>> +++ b/guix/self.scm
>> @@ -752,7 +752,8 @@ (define* (compiled-guix source #:key
>>                          (gzip (specification->package "gzip"))
>>                          (bzip2 (specification->package "bzip2"))
>>                          (xz (specification->package "xz"))
>> -                        (guix (specification->package "guix")))
>> +                        (guix (specification->package "guix"))
>> +                        (graft? #t))
>>    "Return a file-like object that contains a compiled Guix."
>>    (define guile-avahi
>>      (specification->package "guile-avahi"))
>> @@ -802,6 +803,12 @@ (define dependencies
>>                        guile-json guile-semver guile-ssh guile-sqlite3
>>                        guile-lib guile-zlib guile-lzlib guile-zstd)))
>>  
>> +  (define packages
>> +    (cons* gzip
>> +           bzip2
>> +           xz
>> +           dependencies))
>> +
>
> [...]
>
>> +         (let ((obj (built-modules (lambda (node)
>> +                                     (list (node-source node)
>> +                                           (node-compiled node))))))
>> +           (if graft?
>> +               (explicit-grafting obj packages)
>> +               obj)))
>
> There are two things I’m not comfortable with:
>
>   1. Having <explicit-grafting> in (guix packages); it looks misplaced.

I didn't put it there at first, but I think it makes sense since
grafting is currently specific to packages, as is this additional code.

>   2. More importantly, manually listing packages that might require
>      grafting looks like a slippery slope (“oops! we’re not getting the
>      GnuTLS graft for that CVE, too bad”).
>
> I designed and implemented several variants to try and delay grafting.
> One of them consisted in carrying graft information in gexps:
>
>   https://git.savannah.gnu.org/cgit/guix.git/log?h=wip-gexp-grafts
>
> It’s kinda similar to what you’re proposing in that graft information is
> carried as far as possible.  The main difference is that it’s automated.

That's interesting, I think that making grafting not specific to
packages, and something where the replacement is handled at a lower
level (e.g. gexps) would be an alternative way to handle this.

Given that this approach works though, maybe the explicit-grafting
functionality could just sit and be used inside of (guix self). Given
that module is very explicit about what packages are used, it should be
possible to arrange the code so it's very hard to miss a package out,
which should address your concern about manually listing packages (maybe
specification->package can be tweaked so that it's possible to get all
the packages, and that can be the list considered for grafting).

I don't know of any other places where this approach would be useful, so
while it would be nice to have a more general grafting mechanism
eventually, I'd also like to be able to make these changes to channel
instance grafts sooner rather than later.
[signature.asc (application/pgp-signature, inline)]

Information forwarded to guix-patches@gnu.org:
bug#61363; Package guix-patches. (Tue, 28 Feb 2023 15:48:01 GMT) (full text, mbox, link).


Message #23 received at 61363@debbugs.gnu.org (full text, mbox, reply):

From: Christopher Baines <mail@cbaines.net>
To: 61363@debbugs.gnu.org
Subject: [PATCH v2 1/3] packages: Export guile-for-grafts.
Date: Tue, 28 Feb 2023 15:47:01 +0000
So this can be used in (guix self).

* guix/packages.scm (guile-for-grafts): Export.
---
 guix/packages.scm | 1 +
 1 file changed, 1 insertion(+)

diff --git a/guix/packages.scm b/guix/packages.scm
index 041a872f9d..2f81ad0284 100644
--- a/guix/packages.scm
+++ b/guix/packages.scm
@@ -183,6 +183,7 @@ (define-module (guix packages)
             package-closure
 
             default-guile
+            guile-for-grafts
             default-guile-derivation
             set-guile-for-build
             package-file
-- 
2.39.1





Information forwarded to guix-patches@gnu.org:
bug#61363; Package guix-patches. (Tue, 28 Feb 2023 15:48:02 GMT) (full text, mbox, link).


Message #26 received at 61363@debbugs.gnu.org (full text, mbox, reply):

From: Christopher Baines <mail@cbaines.net>
To: 61363@debbugs.gnu.org
Subject: [PATCH v2 2/3] self: Restructure accessing packages.
Date: Tue, 28 Feb 2023 15:47:02 +0000
Both for consistency (always use specification->package as defined in this
module) and so that all the packages that are used can be accessed (which
comes in useful when applying grafts).

* guix/self.scm (%packages): New variable.
(specification->package): Use %packages.
(locale-data, translate-texi-manuals, info-manual, guix-command,
compiled-guix): Use specification->package.
---
 guix/self.scm | 97 +++++++++++++++++++++++++--------------------------
 1 file changed, 48 insertions(+), 49 deletions(-)

diff --git a/guix/self.scm b/guix/self.scm
index 93019e1c64..c5de3ab8fc 100644
--- a/guix/self.scm
+++ b/guix/self.scm
@@ -44,34 +44,42 @@ (define-module (guix self)
 ;;; Dependency handling.
 ;;;
 
-(define specification->package
+(define %packages
+  (let ((ref (lambda (module variable)
+               (delay
+                 (module-ref (resolve-interface
+                              `(gnu packages ,module))
+                             variable)))))
+    `(("guile"              . ,(ref 'guile 'guile-3.0-latest))
+      ("guile-avahi"        . ,(ref 'guile-xyz 'guile-avahi))
+      ("guile-json"         . ,(ref 'guile 'guile-json-4))
+      ("guile-ssh"          . ,(ref 'ssh   'guile-ssh))
+      ("guile-git"          . ,(ref 'guile 'guile-git))
+      ("guile-semver"       . ,(ref 'guile-xyz 'guile-semver))
+      ("guile-lib"          . ,(ref 'guile-xyz 'guile-lib))
+      ("guile-sqlite3"      . ,(ref 'guile 'guile-sqlite3))
+      ("guile-zlib"         . ,(ref 'guile 'guile-zlib))
+      ("guile-lzlib"        . ,(ref 'guile 'guile-lzlib))
+      ("guile-zstd"         . ,(ref 'guile 'guile-zstd))
+      ("guile-gcrypt"       . ,(ref 'gnupg 'guile-gcrypt))
+      ("guile-gnutls"       . ,(ref 'tls 'guile-gnutls))
+      ("guix-daemon"        . ,(ref 'package-management 'guix-daemon))
+      ("disarchive"         . ,(ref 'backup 'disarchive))
+      ("guile-lzma"         . ,(ref 'guile 'guile-lzma))
+      ("gzip"               . ,(ref 'compression 'gzip))
+      ("bzip2"              . ,(ref 'compression 'bzip2))
+      ("xz"                 . ,(ref 'compression 'xz))
+      ("po4a"               . ,(ref 'gettext 'po4a))
+      ("gettext-minimal"    . ,(ref 'gettext 'gettext-minimal))
+      ("gcc-toolchain"      . ,(ref 'commencement 'gcc-toolchain))
+      ("glibc-utf8-locales" . ,(ref 'base 'glibc-utf8-locales))
+      ("graphviz"           . ,(ref 'graphviz 'graphviz))
+      ("texinfo"            . ,(ref 'texinfo 'texinfo)))))
+
+(define (specification->package name)
   ;; Use our own variant of that procedure because that of (gnu packages)
   ;; would traverse all the .scm files, which is wasteful.
-  (let ((ref (lambda (module variable)
-               (module-ref (resolve-interface module) variable))))
-    (match-lambda
-      ("guile"      (ref '(gnu packages guile) 'guile-3.0-latest))
-      ("guile-avahi" (ref '(gnu packages guile-xyz) 'guile-avahi))
-      ("guile-json" (ref '(gnu packages guile) 'guile-json-4))
-      ("guile-ssh"  (ref '(gnu packages ssh)   'guile-ssh))
-      ("guile-git"  (ref '(gnu packages guile) 'guile-git))
-      ("guile-semver"  (ref '(gnu packages guile-xyz) 'guile-semver))
-      ("guile-lib"  (ref '(gnu packages guile-xyz) 'guile-lib))
-      ("guile-sqlite3" (ref '(gnu packages guile) 'guile-sqlite3))
-      ("guile-zlib" (ref '(gnu packages guile) 'guile-zlib))
-      ("guile-lzlib" (ref '(gnu packages guile) 'guile-lzlib))
-      ("guile-zstd" (ref '(gnu packages guile) 'guile-zstd))
-      ("guile-gcrypt"  (ref '(gnu packages gnupg) 'guile-gcrypt))
-      ("guile-gnutls"  (ref '(gnu packages tls) 'guile-gnutls))
-      ("disarchive" (ref '(gnu packages backup) 'disarchive))
-      ("guile-lzma" (ref '(gnu packages guile) 'guile-lzma))
-      ("gzip"       (ref '(gnu packages compression) 'gzip))
-      ("bzip2"      (ref '(gnu packages compression) 'bzip2))
-      ("xz"         (ref '(gnu packages compression) 'xz))
-      ("po4a"       (ref '(gnu packages gettext) 'po4a))
-      ("gettext"       (ref '(gnu packages gettext) 'gettext-minimal))
-      ("gcc-toolchain" (ref '(gnu packages commencement) 'gcc-toolchain))
-      (_            #f))))                        ;no such package
+  (and=> (assoc-ref %packages name) force))
 
 
 ;;;
@@ -240,9 +248,8 @@ (define* (locale-data source domain
                       #:optional (directory domain))
   "Return the locale data from 'po/DIRECTORY' in SOURCE, corresponding to
 DOMAIN, a gettext domain."
-  (define gettext
-    (module-ref (resolve-interface '(gnu packages gettext))
-                'gettext-minimal))
+  (define gettext-minimal
+    (specification->package "gettext-minimal"))
 
   (define build
     (with-imported-modules '((guix build utils))
@@ -258,7 +265,7 @@ (define (compile language)
             (let ((gmo (string-append #$output "/" language "/LC_MESSAGES/"
                                       #$domain ".mo")))
               (mkdir-p (dirname gmo))
-              (invoke #+(file-append gettext "/bin/msgfmt")
+              (invoke #+(file-append gettext-minimal "/bin/msgfmt")
                       "-c" "--statistics" "--verbose"
                       "-o" gmo
                       (string-append po-directory "/" language ".po"))))
@@ -280,20 +287,19 @@ (define (translate-texi-manuals source)
   "Return the translated texinfo manuals built from SOURCE."
   (define po4a
     (specification->package "po4a"))
-  
-  (define gettext
-    (specification->package "gettext"))
+
+  (define gettext-minimal
+    (specification->package "gettext-minimal"))
 
   (define glibc-utf8-locales
-    (module-ref (resolve-interface '(gnu packages base))
-                'glibc-utf8-locales))
+    (specification->package "glibc-utf8-locales"))
 
   (define documentation
     (file-append* source "doc"))
 
   (define documentation-po
     (file-append* source "po/doc"))
-  
+
   (define build
     (with-imported-modules '((guix build utils) (guix build po))
       #~(begin
@@ -365,7 +371,7 @@ (define parallel-jobs
 
           (setenv "GUIX_LOCPATH"
                   #+(file-append glibc-utf8-locales "/lib/locale"))
-          (setenv "PATH" #+(file-append gettext "/bin"))
+          (setenv "PATH" #+(file-append gettext-minimal "/bin"))
           (setenv "LC_ALL" "en_US.UTF-8")
           (setlocale LC_ALL "en_US.UTF-8")
 
@@ -394,16 +400,13 @@ (define parallel-jobs
 (define (info-manual source)
   "Return the Info manual built from SOURCE."
   (define texinfo
-    (module-ref (resolve-interface '(gnu packages texinfo))
-                'texinfo))
+    (specification->package "texinfo"))
 
   (define graphviz
-    (module-ref (resolve-interface '(gnu packages graphviz))
-                'graphviz))
+    (specification->package "graphviz"))
 
   (define glibc-utf8-locales
-    (module-ref (resolve-interface '(gnu packages base))
-                'glibc-utf8-locales))
+    (specification->package "glibc-utf8-locales"))
 
   (define documentation
     (file-append* source "doc"))
@@ -586,8 +589,7 @@ (define* (guix-command modules
   "Return the 'guix' command such that it adds MODULES and DEPENDENCIES in its
 load path."
   (define glibc-utf8-locales
-    (module-ref (resolve-interface '(gnu packages base))
-                'glibc-utf8-locales))
+    (specification->package "glibc-utf8-locales"))
 
   (define module-directory
     ;; To minimize the number of 'stat' calls needed to locate a module,
@@ -1030,10 +1032,7 @@ (define (built-modules node-subset)
                           ;; Include 'guix-daemon'.  XXX: Here we inject an
                           ;; older snapshot of guix-daemon, but that's a good
                           ;; enough approximation for now.
-                          #:daemon (module-ref (resolve-interface
-                                                '(gnu packages
-                                                      package-management))
-                                               'guix-daemon)
+                          #:daemon (specification->package "guix-daemon")
 
                           #:info (info-manual source)
                           #:miscellany (miscellaneous-files source)
-- 
2.39.1





Information forwarded to guix-patches@gnu.org:
bug#61363; Package guix-patches. (Tue, 28 Feb 2023 15:48:02 GMT) (full text, mbox, link).


Message #29 received at 61363@debbugs.gnu.org (full text, mbox, reply):

From: Christopher Baines <mail@cbaines.net>
To: 61363@debbugs.gnu.org
Subject: [PATCH v2 3/3] self: Apply grafts to the outputs of the guix derivation.
Date: Tue, 28 Feb 2023 15:47:03 +0000
Rather than having grafts apply to the derivation itself. This moves grafting
here to work like grafting for packages, where you can think of the grafted
outputs as a transformed variant of the ungrafted outputs.

I'm looking at this as it'll allow the Guix Data Service to compute the
derivations without grafts, and for these to be useful for substitutes
regardless of whether users are using grafts.

* guix/self.scm (compiled-guix, guix-derivation): Add a #:graft? keyword
argument, to control grafting when computing the guix derivation.
* build-aux/build-self.scm (build-program): Call guix-derivation with
 #:graft? (%graft?) to make the compute-guix-derivation script use or not use
grafts as desired.
---
 build-aux/build-self.scm |   4 +-
 guix/self.scm            | 101 +++++++++++++++++++++++++++++++--------
 2 files changed, 84 insertions(+), 21 deletions(-)

diff --git a/build-aux/build-self.scm b/build-aux/build-self.scm
index 02822a2ee8..6d0037f20c 100644
--- a/build-aux/build-self.scm
+++ b/build-aux/build-self.scm
@@ -353,7 +353,9 @@ (define fake-git
                                                   #:channel-metadata
                                                   '#$channel-metadata
                                                   #:pull-version
-                                                  #$pull-version)
+                                                  #$pull-version
+                                                  #:graft?
+                                                  #$(%graft?))
                                  #:system system))
                              derivation-file-name))))))
                   #:module-path (list source))))
diff --git a/guix/self.scm b/guix/self.scm
index c5de3ab8fc..8842275ff8 100644
--- a/guix/self.scm
+++ b/guix/self.scm
@@ -22,6 +22,7 @@ (define-module (guix self)
   #:use-module (guix i18n)
   #:use-module (guix modules)
   #:use-module (guix gexp)
+  #:use-module (guix grafts)
   #:use-module (guix store)
   #:use-module (guix monads)
   #:use-module (guix discovery)
@@ -32,6 +33,7 @@ (define-module (guix self)
   #:use-module ((guix build utils) #:select (find-files))
   #:use-module (srfi srfi-1)
   #:use-module (srfi srfi-9)
+  #:use-module (srfi srfi-9 gnu)
   #:use-module (srfi srfi-35)
   #:use-module (ice-9 match)
   #:export (make-config.scm
@@ -244,6 +246,50 @@ (define* (file-append* item file #:key (recursive? #t))
      ;; which isn't great.
      (file-append item "/" file))))
 
+(define graft-derivation*
+  (store-lift graft-derivation))
+
+(define package-grafts*
+  (store-lift package-grafts))
+
+;; Apply grafts explicitly
+(define-immutable-record-type <explicit-grafting>
+  (%explicit-grafting obj packages)
+  explicit-grafting?
+  (obj      explicit-grafting-obj)       ;obj
+  (packages explicit-grafting-packages)) ;list of <package>s
+
+(define (write-explicit-grafting rec port)
+  (match rec
+    (($ <explicit-grafting> obj packages)
+     (format port "#<explicit-grafting ~s ~s>" obj packages))))
+
+(define (explicit-grafting obj packages)
+  (%explicit-grafting obj packages))
+
+(define-gexp-compiler (explicit-grafting-compiler (explicit-grafting <explicit-grafting>)
+                                                  system target)
+  (match explicit-grafting
+    (($ <explicit-grafting> obj packages)
+     (mlet* %store-monad ((drv (without-grafting
+                                (lower-object obj system #:target target)))
+                          (grafts
+                           (mapm %store-monad
+                                 (lambda (pkg)
+                                   (package-grafts* pkg system #:target target))
+                                 packages)))
+       (match (delete-duplicates
+               (concatenate grafts))
+         (()
+          (return drv))
+         (grafts
+          (mlet %store-monad ((guile (package->derivation
+                                      (guile-for-grafts)
+                                      system #:graft? #f)))
+            (graft-derivation* drv grafts
+                               #:system system
+                               #:guile guile))))))))
+
 (define* (locale-data source domain
                       #:optional (directory domain))
   "Return the locale data from 'po/DIRECTORY' in SOURCE, corresponding to
@@ -754,7 +800,8 @@ (define* (compiled-guix source #:key
                         (gzip (specification->package "gzip"))
                         (bzip2 (specification->package "bzip2"))
                         (xz (specification->package "xz"))
-                        (guix (specification->package "guix")))
+                        (guix (specification->package "guix"))
+                        (graft? #t))
   "Return a file-like object that contains a compiled Guix."
   (define guile-avahi
     (specification->package "guile-avahi"))
@@ -1024,25 +1071,34 @@ (define (built-modules node-subset)
                                                guile-lzma
                                                dependencies)
                                         #:guile guile-for-build
-                                        #:guile-version guile-version)))
-           (whole-package name modules dependencies
-                          #:command command
-                          #:guile guile-for-build
-
-                          ;; Include 'guix-daemon'.  XXX: Here we inject an
-                          ;; older snapshot of guix-daemon, but that's a good
-                          ;; enough approximation for now.
-                          #:daemon (specification->package "guix-daemon")
-
-                          #:info (info-manual source)
-                          #:miscellany (miscellaneous-files source)
-                          #:guile-version guile-version)))
+                                        #:guile-version guile-version))
+                (obj
+                 (whole-package name modules dependencies
+                                #:command command
+                                #:guile guile-for-build
+
+                                ;; Include 'guix-daemon'.  XXX: Here we inject
+                                ;; an older snapshot of guix-daemon, but
+                                ;; that's a good enough approximation for now.
+                                #:daemon (specification->package "guix-daemon")
+
+                                #:info (info-manual source)
+                                #:miscellany (miscellaneous-files source)
+                                #:guile-version guile-version)))
+           (if graft?
+               (explicit-grafting obj
+                                  (map (compose force cdr) %packages))
+               obj)))
         ((= 0 pull-version)
          ;; Legacy 'guix pull': return the .scm and .go files as one
          ;; directory.
-         (built-modules (lambda (node)
-                          (list (node-source node)
-                                (node-compiled node)))))
+         (let ((obj (built-modules (lambda (node)
+                                     (list (node-source node)
+                                           (node-compiled node))))))
+           (if graft?
+               (explicit-grafting obj
+                                  (map (compose force cdr) %packages))
+               obj)))
         (else
          ;; Unsupported 'guix pull' version.
          #f)))
@@ -1272,7 +1328,8 @@ (define (process-directory directory files output)
 (define* (guix-derivation source version
                           #:optional (guile-version (effective-version))
                           #:key (pull-version 0)
-                          channel-metadata)
+                          channel-metadata
+                          (graft? #t))
   "Return, as a monadic value, the derivation to build the Guix from SOURCE
 for GUILE-VERSION.  Use VERSION as the version string.  Use CHANNEL-METADATA
 as the channel metadata sexp to include in (guix config).
@@ -1309,7 +1366,11 @@ (define guile
                                #:pull-version pull-version
                                #:guile-version (if (>= pull-version 1)
                                                    "3.0" guile-version)
-                               #:guile-for-build guile)))
+                               #:guile-for-build guile
+                               #:graft? graft?)))
       (if guix
-          (lower-object guix)
+          (if graft?
+              (lower-object guix)
+              (without-grafting
+               (lower-object guix)))
           (return #f)))))
-- 
2.39.1





Information forwarded to guix-patches@gnu.org:
bug#61363; Package guix-patches. (Tue, 28 Feb 2023 15:51:01 GMT) (full text, mbox, link).


Message #32 received at 61363@debbugs.gnu.org (full text, mbox, reply):

From: Christopher Baines <mail@cbaines.net>
To: Ludovic Courtès <ludo@gnu.org>
Cc: 61363@debbugs.gnu.org
Subject: Re: [bug#61363] [PATCH 2/2] self: Apply grafts to the outputs of the guix derivation.
Date: Tue, 28 Feb 2023 15:47:11 +0000
[Message part 1 (text/plain, inline)]
Christopher Baines <mail@cbaines.net> writes:

> Ludovic Courtès <ludo@gnu.org> writes:
>
>>   2. More importantly, manually listing packages that might require
>>      grafting looks like a slippery slope (“oops! we’re not getting the
>>      GnuTLS graft for that CVE, too bad”).
>>
>> I designed and implemented several variants to try and delay grafting.
>> One of them consisted in carrying graft information in gexps:
>>
>>   https://git.savannah.gnu.org/cgit/guix.git/log?h=wip-gexp-grafts
>>
>> It’s kinda similar to what you’re proposing in that graft information is
>> carried as far as possible.  The main difference is that it’s automated.
>
> That's interesting, I think that making grafting not specific to
> packages, and something where the replacement is handled at a lower
> level (e.g. gexps) would be an alternative way to handle this.
>
> Given that this approach works though, maybe the explicit-grafting
> functionality could just sit and be used inside of (guix self). Given
> that module is very explicit about what packages are used, it should be
> possible to arrange the code so it's very hard to miss a package out,
> which should address your concern about manually listing packages (maybe
> specification->package can be tweaked so that it's possible to get all
> the packages, and that can be the list considered for grafting).
>
> I don't know of any other places where this approach would be useful, so
> while it would be nice to have a more general grafting mechanism
> eventually, I'd also like to be able to make these changes to channel
> instance grafts sooner rather than later.

I've sent a v2 series which changes along the above lines. The explicit
grafting stuff just sits in (guix self), and (guix self) more
rigeriously uses it's own definition of specification->package, which
should provide some protection against missing packages out. Obviously
it's not quite as rigerous as moving the grafting functionality in to
gexps, but hopefully it's rigerous enough for now.
[signature.asc (application/pgp-signature, inline)]

Information forwarded to guix-patches@gnu.org:
bug#61363; Package guix-patches. (Mon, 17 Apr 2023 15:00:02 GMT) (full text, mbox, link).


Message #35 received at 61363@debbugs.gnu.org (full text, mbox, reply):

From: Christopher Baines <mail@cbaines.net>
To: 61363@debbugs.gnu.org
Cc: Christopher Baines <mail@cbaines.net>
Subject: [PATCH v3] self: Apply grafts to the outputs of the guix derivation.
Date: Mon, 17 Apr 2023 15:59:28 +0100
Rather than having grafts apply to the derivation itself. This moves grafting
here to work like grafting for packages, where you can think of the grafted
outputs as a transformed variant of the ungrafted outputs.

I'm looking at this as it'll allow the Guix Data Service to compute the
derivations without grafts, and for these to be useful for substitutes
regardless of whether users are using grafts.

* guix/self.scm (compiled-guix, guix-derivation): Add a #:graft? keyword
argument, to control grafting when computing the guix derivation.
* build-aux/build-self.scm (build-program): Call guix-derivation with
 #:graft? (%graft?) to make the compute-guix-derivation script use or not use
grafts as desired.

Signed-off-by: Christopher Baines <mail@cbaines.net>
---
 build-aux/build-self.scm |   4 +-
 guix/self.scm            | 101 +++++++++++++++++++++++++++++++--------
 2 files changed, 84 insertions(+), 21 deletions(-)

diff --git a/build-aux/build-self.scm b/build-aux/build-self.scm
index 02822a2ee8..6d0037f20c 100644
--- a/build-aux/build-self.scm
+++ b/build-aux/build-self.scm
@@ -353,7 +353,9 @@ (define fake-git
                                                   #:channel-metadata
                                                   '#$channel-metadata
                                                   #:pull-version
-                                                  #$pull-version)
+                                                  #$pull-version
+                                                  #:graft?
+                                                  #$(%graft?))
                                  #:system system))
                              derivation-file-name))))))
                   #:module-path (list source))))
diff --git a/guix/self.scm b/guix/self.scm
index 74c953bd50..bbc0beaca8 100644
--- a/guix/self.scm
+++ b/guix/self.scm
@@ -21,6 +21,7 @@ (define-module (guix self)
   #:use-module (guix config)
   #:use-module (guix modules)
   #:use-module (guix gexp)
+  #:use-module (guix grafts)
   #:use-module (guix store)
   #:use-module (guix monads)
   #:use-module (guix discovery)
@@ -31,6 +32,7 @@ (define-module (guix self)
   #:use-module ((guix build utils) #:select (find-files))
   #:use-module (srfi srfi-1)
   #:use-module (srfi srfi-9)
+  #:use-module (srfi srfi-9 gnu)
   #:use-module (srfi srfi-35)
   #:use-module (ice-9 match)
   #:export (make-config.scm
@@ -243,6 +245,50 @@ (define* (file-append* item file #:key (recursive? #t))
      ;; which isn't great.
      (file-append item "/" file))))
 
+(define graft-derivation*
+  (store-lift graft-derivation))
+
+(define package-grafts*
+  (store-lift package-grafts))
+
+;; Apply grafts explicitly
+(define-immutable-record-type <explicit-grafting>
+  (%explicit-grafting obj packages)
+  explicit-grafting?
+  (obj      explicit-grafting-obj)       ;obj
+  (packages explicit-grafting-packages)) ;list of <package>s
+
+(define (write-explicit-grafting rec port)
+  (match rec
+    (($ <explicit-grafting> obj packages)
+     (format port "#<explicit-grafting ~s ~s>" obj packages))))
+
+(define (explicit-grafting obj packages)
+  (%explicit-grafting obj packages))
+
+(define-gexp-compiler (explicit-grafting-compiler (explicit-grafting <explicit-grafting>)
+                                                  system target)
+  (match explicit-grafting
+    (($ <explicit-grafting> obj packages)
+     (mlet* %store-monad ((drv (without-grafting
+                                (lower-object obj system #:target target)))
+                          (grafts
+                           (mapm %store-monad
+                                 (lambda (pkg)
+                                   (package-grafts* pkg system #:target target))
+                                 packages)))
+       (match (delete-duplicates
+               (concatenate grafts))
+         (()
+          (return drv))
+         (grafts
+          (mlet %store-monad ((guile (package->derivation
+                                      (guile-for-grafts)
+                                      system #:graft? #f)))
+            (graft-derivation* drv grafts
+                               #:system system
+                               #:guile guile))))))))
+
 (define* (locale-data source domain
                       #:optional (directory domain))
   "Return the locale data from 'po/DIRECTORY' in SOURCE, corresponding to
@@ -753,7 +799,8 @@ (define* (compiled-guix source #:key
                         (gzip (specification->package "gzip"))
                         (bzip2 (specification->package "bzip2"))
                         (xz (specification->package "xz"))
-                        (guix (specification->package "guix")))
+                        (guix (specification->package "guix"))
+                        (graft? #t))
   "Return a file-like object that contains a compiled Guix."
   (define guile-avahi
     (specification->package "guile-avahi"))
@@ -1023,25 +1070,34 @@ (define (built-modules node-subset)
                                                guile-lzma
                                                dependencies)
                                         #:guile guile-for-build
-                                        #:guile-version guile-version)))
-           (whole-package name modules dependencies
-                          #:command command
-                          #:guile guile-for-build
-
-                          ;; Include 'guix-daemon'.  XXX: Here we inject an
-                          ;; older snapshot of guix-daemon, but that's a good
-                          ;; enough approximation for now.
-                          #:daemon (specification->package "guix-daemon")
-
-                          #:info (info-manual source)
-                          #:miscellany (miscellaneous-files source)
-                          #:guile-version guile-version)))
+                                        #:guile-version guile-version))
+                (obj
+                 (whole-package name modules dependencies
+                                #:command command
+                                #:guile guile-for-build
+
+                                ;; Include 'guix-daemon'.  XXX: Here we inject
+                                ;; an older snapshot of guix-daemon, but
+                                ;; that's a good enough approximation for now.
+                                #:daemon (specification->package "guix-daemon")
+
+                                #:info (info-manual source)
+                                #:miscellany (miscellaneous-files source)
+                                #:guile-version guile-version)))
+           (if graft?
+               (explicit-grafting obj
+                                  (map (compose force cdr) %packages))
+               obj)))
         ((= 0 pull-version)
          ;; Legacy 'guix pull': return the .scm and .go files as one
          ;; directory.
-         (built-modules (lambda (node)
-                          (list (node-source node)
-                                (node-compiled node)))))
+         (let ((obj (built-modules (lambda (node)
+                                     (list (node-source node)
+                                           (node-compiled node))))))
+           (if graft?
+               (explicit-grafting obj
+                                  (map (compose force cdr) %packages))
+               obj)))
         (else
          ;; Unsupported 'guix pull' version.
          #f)))
@@ -1271,7 +1327,8 @@ (define (process-directory directory files output)
 (define* (guix-derivation source version
                           #:optional (guile-version (effective-version))
                           #:key (pull-version 0)
-                          channel-metadata)
+                          channel-metadata
+                          (graft? #t))
   "Return, as a monadic value, the derivation to build the Guix from SOURCE
 for GUILE-VERSION.  Use VERSION as the version string.  Use CHANNEL-METADATA
 as the channel metadata sexp to include in (guix config).
@@ -1308,7 +1365,11 @@ (define guile
                                #:pull-version pull-version
                                #:guile-version (if (>= pull-version 1)
                                                    "3.0" guile-version)
-                               #:guile-for-build guile)))
+                               #:guile-for-build guile
+                               #:graft? graft?)))
       (if guix
-          (lower-object guix)
+          (if graft?
+              (lower-object guix)
+              (without-grafting
+               (lower-object guix)))
           (return #f)))))
-- 
2.39.1





Information forwarded to guix-patches@gnu.org:
bug#61363; Package guix-patches. (Mon, 17 Apr 2023 15:08:01 GMT) (full text, mbox, link).


Message #38 received at 61363@debbugs.gnu.org (full text, mbox, reply):

From: Christopher Baines <mail@cbaines.net>
To: 61363@debbugs.gnu.org
Subject: Re: [bug#61363] [PATCH 2/2] self: Apply grafts to the outputs of the guix derivation.
Date: Mon, 17 Apr 2023 16:06:16 +0100
[Message part 1 (text/plain, inline)]
Christopher Baines <mail@cbaines.net> writes:

> [[PGP Signed Part:Undecided]]
>
> Christopher Baines <mail@cbaines.net> writes:
>
>> Ludovic Courtès <ludo@gnu.org> writes:
>>
>>>   2. More importantly, manually listing packages that might require
>>>      grafting looks like a slippery slope (“oops! we’re not getting the
>>>      GnuTLS graft for that CVE, too bad”).
>>>
>>> I designed and implemented several variants to try and delay grafting.
>>> One of them consisted in carrying graft information in gexps:
>>>
>>>   https://git.savannah.gnu.org/cgit/guix.git/log?h=wip-gexp-grafts
>>>
>>> It’s kinda similar to what you’re proposing in that graft information is
>>> carried as far as possible.  The main difference is that it’s automated.
>>
>> That's interesting, I think that making grafting not specific to
>> packages, and something where the replacement is handled at a lower
>> level (e.g. gexps) would be an alternative way to handle this.
>>
>> Given that this approach works though, maybe the explicit-grafting
>> functionality could just sit and be used inside of (guix self). Given
>> that module is very explicit about what packages are used, it should be
>> possible to arrange the code so it's very hard to miss a package out,
>> which should address your concern about manually listing packages (maybe
>> specification->package can be tweaked so that it's possible to get all
>> the packages, and that can be the list considered for grafting).
>>
>> I don't know of any other places where this approach would be useful, so
>> while it would be nice to have a more general grafting mechanism
>> eventually, I'd also like to be able to make these changes to channel
>> instance grafts sooner rather than later.
>
> I've sent a v2 series which changes along the above lines. The explicit
> grafting stuff just sits in (guix self), and (guix self) more
> rigeriously uses it's own definition of specification->package, which
> should provide some protection against missing packages out. Obviously
> it's not quite as rigerous as moving the grafting functionality in to
> gexps, but hopefully it's rigerous enough for now.

This has stalled a bit, but it would be good to try and get things
merged. I've gone ahead and pushed the first two patches in the series I
last sent, these just make minor changes to prepare for the functional
change here. I've also resent that patch as as v3.
[signature.asc (application/pgp-signature, inline)]

Information forwarded to guix-patches@gnu.org:
bug#61363; Package guix-patches. (Tue, 16 May 2023 15:07:02 GMT) (full text, mbox, link).


Message #41 received at 61363@debbugs.gnu.org (full text, mbox, reply):

From: Simon Tournier <zimon.toutoune@gmail.com>
To: Christopher Baines <mail@cbaines.net>, 61363@debbugs.gnu.org
Cc: Christopher Baines <mail@cbaines.net>
Subject: Re: [bug#61363] [PATCH v3] self: Apply grafts to the outputs of the guix derivation.
Date: Tue, 16 May 2023 15:25:37 +0200
Hi Chris,

I am late to the party and probably do not well understand all that
part.  Just a quick comment in the same direction as Ludo.

On Mon, 17 Apr 2023 at 15:59, Christopher Baines <mail@cbaines.net> wrote:

> diff --git a/guix/self.scm b/guix/self.scm
> index 74c953bd50..bbc0beaca8 100644
> --- a/guix/self.scm
> +++ b/guix/self.scm

[...]

> +           (if graft?
> +               (explicit-grafting obj
> +                                  (map (compose force cdr) %packages))
> +               obj)))

[...]

> +           (if graft?
> +               (explicit-grafting obj
> +                                  (map (compose force cdr) %packages))
> +               obj)))

It means that the grafts are only applied to %packages, right?

Other said, defined by:

--8<---------------cut here---------------start------------->8---
(define %packages
  (let ((ref (lambda (module variable)
               (delay
                 (module-ref (resolve-interface
                              `(gnu packages ,module))
                             variable)))))
    `(("guile"              . ,(ref 'guile 'guile-3.0-latest))
      ("guile-avahi"        . ,(ref 'guile-xyz 'guile-avahi))
      ("guile-json"         . ,(ref 'guile 'guile-json-4))
      ("guile-ssh"          . ,(ref 'ssh   'guile-ssh))
      ("guile-git"          . ,(ref 'guile 'guile-git))
      ("guile-semver"       . ,(ref 'guile-xyz 'guile-semver))
      ("guile-lib"          . ,(ref 'guile-xyz 'guile-lib))
      ("guile-sqlite3"      . ,(ref 'guile 'guile-sqlite3))
      ("guile-zlib"         . ,(ref 'guile 'guile-zlib))
      ("guile-lzlib"        . ,(ref 'guile 'guile-lzlib))
      ("guile-zstd"         . ,(ref 'guile 'guile-zstd))
      ("guile-gcrypt"       . ,(ref 'gnupg 'guile-gcrypt))
      ("guile-gnutls"       . ,(ref 'tls 'guile-gnutls))
      ("guix-daemon"        . ,(ref 'package-management 'guix-daemon))
      ("disarchive"         . ,(ref 'backup 'disarchive))
      ("guile-lzma"         . ,(ref 'guile 'guile-lzma))
      ("gzip"               . ,(ref 'compression 'gzip))
      ("bzip2"              . ,(ref 'compression 'bzip2))
      ("xz"                 . ,(ref 'compression 'xz))
      ("po4a"               . ,(ref 'gettext 'po4a))
      ("gettext-minimal"    . ,(ref 'gettext 'gettext-minimal))
      ("gcc-toolchain"      . ,(ref 'commencement 'gcc-toolchain))
      ("glibc-utf8-locales" . ,(ref 'base 'glibc-utf8-locales))
      ("graphviz"           . ,(ref 'graphviz 'graphviz))
      ("texinfo"            . ,(ref 'texinfo 'texinfo)))))
--8<---------------cut here---------------end--------------->8---

tweaked by e5c33837cbee98d460d9ae09b463501de6f15d97.  And there is a
slippery slope: the manual addition.  These had been added with
e5c33837cbee98d460d9ae09b463501de6f15d97:

    + ("glibc-utf8-locales" . ,(ref 'base               'glibc-utf8-locales))
    + ("graphviz"           . ,(ref 'graphviz           'graphviz))
    + ("guix-daemon"        . ,(ref 'package-management 'guix-daemon))
    + ("texinfo"            . ,(ref 'texinfo            'texinfo)))))

Other said, what does it happen if we forget to manually update this
list?


Cheers,
simon




Information forwarded to guix-patches@gnu.org:
bug#61363; Package guix-patches. (Sat, 03 Jun 2023 11:45:01 GMT) (full text, mbox, link).


Message #44 received at 61363@debbugs.gnu.org (full text, mbox, reply):

From: Christopher Baines <mail@cbaines.net>
To: Simon Tournier <zimon.toutoune@gmail.com>
Cc: 61363@debbugs.gnu.org
Subject: Re: [bug#61363] [PATCH v3] self: Apply grafts to the outputs of the guix derivation.
Date: Sat, 03 Jun 2023 12:41:57 +0100
[Message part 1 (text/plain, inline)]
Simon Tournier <zimon.toutoune@gmail.com> writes:

> Hi Chris,
>
> I am late to the party and probably do not well understand all that
> part.  Just a quick comment in the same direction as Ludo.
>
> On Mon, 17 Apr 2023 at 15:59, Christopher Baines <mail@cbaines.net> wrote:
>
>> diff --git a/guix/self.scm b/guix/self.scm
>> index 74c953bd50..bbc0beaca8 100644
>> --- a/guix/self.scm
>> +++ b/guix/self.scm
>
> [...]
>
>> +           (if graft?
>> +               (explicit-grafting obj
>> +                                  (map (compose force cdr) %packages))
>> +               obj)))
>
> [...]
>
>> +           (if graft?
>> +               (explicit-grafting obj
>> +                                  (map (compose force cdr) %packages))
>> +               obj)))
>
> It means that the grafts are only applied to %packages, right?
>
> Other said, defined by:
>
> (define %packages
>   (let ((ref (lambda (module variable)
>                (delay
>                  (module-ref (resolve-interface
>                               `(gnu packages ,module))
>                              variable)))))
>     `(("guile"              . ,(ref 'guile 'guile-3.0-latest))
>       ("guile-avahi"        . ,(ref 'guile-xyz 'guile-avahi))
>       ("guile-json"         . ,(ref 'guile 'guile-json-4))
>       ("guile-ssh"          . ,(ref 'ssh   'guile-ssh))
>       ("guile-git"          . ,(ref 'guile 'guile-git))
>       ("guile-semver"       . ,(ref 'guile-xyz 'guile-semver))
>       ("guile-lib"          . ,(ref 'guile-xyz 'guile-lib))
>       ("guile-sqlite3"      . ,(ref 'guile 'guile-sqlite3))
>       ("guile-zlib"         . ,(ref 'guile 'guile-zlib))
>       ("guile-lzlib"        . ,(ref 'guile 'guile-lzlib))
>       ("guile-zstd"         . ,(ref 'guile 'guile-zstd))
>       ("guile-gcrypt"       . ,(ref 'gnupg 'guile-gcrypt))
>       ("guile-gnutls"       . ,(ref 'tls 'guile-gnutls))
>       ("guix-daemon"        . ,(ref 'package-management 'guix-daemon))
>       ("disarchive"         . ,(ref 'backup 'disarchive))
>       ("guile-lzma"         . ,(ref 'guile 'guile-lzma))
>       ("gzip"               . ,(ref 'compression 'gzip))
>       ("bzip2"              . ,(ref 'compression 'bzip2))
>       ("xz"                 . ,(ref 'compression 'xz))
>       ("po4a"               . ,(ref 'gettext 'po4a))
>       ("gettext-minimal"    . ,(ref 'gettext 'gettext-minimal))
>       ("gcc-toolchain"      . ,(ref 'commencement 'gcc-toolchain))
>       ("glibc-utf8-locales" . ,(ref 'base 'glibc-utf8-locales))
>       ("graphviz"           . ,(ref 'graphviz 'graphviz))
>       ("texinfo"            . ,(ref 'texinfo 'texinfo)))))
>
> tweaked by e5c33837cbee98d460d9ae09b463501de6f15d97.  And there is a
> slippery slope: the manual addition.  These had been added with
> e5c33837cbee98d460d9ae09b463501de6f15d97:
>
>     + ("glibc-utf8-locales" . ,(ref 'base               'glibc-utf8-locales))
>     + ("graphviz"           . ,(ref 'graphviz           'graphviz))
>     + ("guix-daemon"        . ,(ref 'package-management 'guix-daemon))
>     + ("texinfo"            . ,(ref 'texinfo            'texinfo)))))
>
> Other said, what does it happen if we forget to manually update this
> list?

Well, specification->package in (guix self) won't work for the missing
packages.

It's possible to use packages outside of this list, but that doesn't
happen currently.
[signature.asc (application/pgp-signature, inline)]

Added tag(s) moreinfo. Request was from Christopher Baines <mail@cbaines.net> to control@debbugs.gnu.org. (Sat, 03 Jun 2023 11:45:02 GMT) (full text, mbox, link).


Send a report that this bug log contains spam.


debbugs.gnu.org maintainers <help-debbugs@gnu.org>. Last modified: Fri Oct 25 19:22:36 2024; Machine Name: wallace-server

GNU bug tracking system

Debbugs is free software and licensed under the terms of the GNU Public License version 2. The current version can be obtained from https://bugs.debian.org/debbugs-source/.

Copyright © 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson, 2005-2017 Don Armstrong, and many other contributors.