Received: (at 47115) by debbugs.gnu.org; 16 Mar 2021 09:19:57 +0000
From debbugs-submit-bounces@debbugs.gnu.org Tue Mar 16 05:19:57 2021
Received: from localhost ([127.0.0.1]:38205 helo=debbugs.gnu.org)
by debbugs.gnu.org with esmtp (Exim 4.84_2)
(envelope-from <debbugs-submit-bounces@debbugs.gnu.org>)
id 1lM5ro-0005j0-Ud
for submit@debbugs.gnu.org; Tue, 16 Mar 2021 05:19:57 -0400
Received: from world.peace.net ([64.112.178.59]:49600)
by debbugs.gnu.org with esmtp (Exim 4.84_2)
(envelope-from <mhw@netris.org>)
id 1lM5rn-0005ig-DL; Tue, 16 Mar 2021 05:19:55 -0400
Received: from mhw by world.peace.net with esmtpsa
(TLS1.3:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92)
(envelope-from <mhw@netris.org>)
id 1lM5rg-0006Oo-Rf; Tue, 16 Mar 2021 05:19:49 -0400
From: Mark H Weaver <mhw@netris.org>
To: Jack Hill <jackhill@jackhill.us>
Subject: Re: bug#47115: Redundant library grafts leads to breakage (was:
Failure building grub-img.png when reconfiguring)
In-Reply-To: <875z1rjyfn.fsf@netris.org>
References: <alpine.DEB.2.21.2103121747190.8138@marsh.hcoop.net>
<YEvznooRt4wUjDtA@jasmine.lan> <87a6r7294k.fsf@netris.org>
<alpine.DEB.2.21.2103121931030.8138@marsh.hcoop.net>
<alpine.DEB.2.21.2103122301220.8138@marsh.hcoop.net>
<877dmb1owa.fsf@netris.org>
<alpine.DEB.2.21.2103131506140.8138@marsh.hcoop.net>
<alpine.DEB.2.21.2103132302490.8138@marsh.hcoop.net>
<874khds84o.fsf@netris.org>
<alpine.DEB.2.21.2103141914430.8138@marsh.hcoop.net>
<87y2epqnq8.fsf@netris.org>
<alpine.DEB.2.21.2103142334250.8138@marsh.hcoop.net>
<alpine.DEB.2.21.2103142350500.8138@marsh.hcoop.net>
<alpine.DEB.2.21.2103151642290.8138@marsh.hcoop.net>
<alpine.DEB.2.21.2103152139000.8138@marsh.hcoop.net>
<alpine.DEB.2.21.2103152233590.8138@marsh.hcoop.net>
<875z1rjyfn.fsf@netris.org>
Date: Tue, 16 Mar 2021 05:18:15 -0400
Message-ID: <8735wvjw25.fsf@netris.org>
MIME-Version: 1.0
Content-Type: text/plain
X-Spam-Score: 0.0 (/)
X-Debbugs-Envelope-To: 47115
Cc: 47115@debbugs.gnu.org
X-BeenThere: debbugs-submit@debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>,
<mailto:debbugs-submit-request@debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit@debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request@debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>,
<mailto:debbugs-submit-request@debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces@debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces@debbugs.gnu.org>
X-Spam-Score: -1.0 (-)
retitle 47115 Redundant library grafts leads to breakage
thanks
Hi,
I looked a bit deeper, and now I think I finally know what's going on.
It turns out that the grafting process is creating two redundant
variants of the replacement guile-cairo.
All of the relevant information is in
/gnu/store/07xw2pp63xin4c4y8ndrcdn3n8z1vmx2-grub-image.png.drv and its
dependents, which fails to build if deduplication is disabled.
If you look through the output of "guix size
/gnu/store/07xw2pp63xin4c4y8ndrcdn3n8z1vmx2-grub-image.png.drv", you'll
find three distinct guile-cairo derivations:
(1) /gnu/store/vz4yw7zkm73diy95mdzywgixal3nf2s2-guile-cairo-1.11.2.drv,
=> /gnu/store/5nmzfnxk8kp85xwma2r585fgyz3jfw56-guile-cairo-1.11.2
(the original, ungrafted, cairo)
(2) /gnu/store/rcl324yiq7a56rwkqwgqx097dwc5mgni-guile-cairo-1.11.2.drv,
=> /gnu/store/vjn7ygzzqshvsfzck8hq5lp5pfrr2xp5-guile-cairo-1.11.2
(the first graft)
(3) /gnu/store/9mha4bzbji8iql50prmq9br4j1c51sjn-guile-cairo-1.11.2.drv,
=> /gnu/store/j69k9n0g3h9ppqi7dmqypwy3lrhxvb97-guile-cairo-1.11.2
(the second graft)
In the 'guile-builder' files referenced from the two graft derivations,
we see that they have the same inputs and perform the same rewrites, but
list them in a different order. Graft 1 has this guile-builder:
--8<---------------cut here---------------start------------->8---
(begin
(use-modules (guix build graft) (guix build utils) (ice-9 match))
(define %output (getenv "out"))
(define %outputs (map (lambda (o) (cons o (getenv o))) (quote ("out"))))
(define %build-inputs
(quote
(("x" . "/gnu/store/5nmzfnxk8kp85xwma2r585fgyz3jfw56-guile-cairo-1.11.2")
("x" . "/gnu/store/fx3979c88s9yxdbchyf36qryawgzpwb5-libx11-1.6.10")
("x" . "/gnu/store/na0x00biq02fm5cyj5a8r67qwsnsskw8-cairo-1.16.0")
("x" . "/gnu/store/cw69is9wbbllwx95wky4pmbcsk4vvbpd-libxrender-0.9.10")
("x" . "/gnu/store/qrs0p8j3wq6q5a4dm0ndjdavk9gyal5q-libxext-1.3.4")
("x" . "/gnu/store/rwkqxykm91a75w9afhb41saj0dmf30hw-libx11-1.6.12")
("x" . "/gnu/store/p51dv37zj24q8001zghc3wxhxz8i3c50-cairo-1.16.0")
("x" . "/gnu/store/pzj036f1nmxdrbza6cqy419ddsn9bydp-libxrender-0.9.10")
("x" . "/gnu/store/3rmazp46f6g8w9qs8n3w7qcg8hhs1lig-libxext-1.3.4"))))
(unsetenv "GUILE_LOAD_COMPILED_PATH")
(unsetenv "LD_LIBRARY_PATH"))
(exit
(begin
(let* ((old-outputs
(quote
(("out" . "/gnu/store/5nmzfnxk8kp85xwma2r585fgyz3jfw56-guile-cairo-1.11.2"))))
(mapping
(append (quote
(("/gnu/store/fx3979c88s9yxdbchyf36qryawgzpwb5-libx11-1.6.10"
. "/gnu/store/rwkqxykm91a75w9afhb41saj0dmf30hw-libx11-1.6.12")
("/gnu/store/na0x00biq02fm5cyj5a8r67qwsnsskw8-cairo-1.16.0"
. "/gnu/store/p51dv37zj24q8001zghc3wxhxz8i3c50-cairo-1.16.0")
("/gnu/store/cw69is9wbbllwx95wky4pmbcsk4vvbpd-libxrender-0.9.10"
. "/gnu/store/pzj036f1nmxdrbza6cqy419ddsn9bydp-libxrender-0.9.10")
("/gnu/store/qrs0p8j3wq6q5a4dm0ndjdavk9gyal5q-libxext-1.3.4"
. "/gnu/store/3rmazp46f6g8w9qs8n3w7qcg8hhs1lig-libxext-1.3.4")))
(map (match-lambda ((name . file)
(cons (assoc-ref old-outputs name) file)))
%outputs))))
(graft old-outputs %outputs mapping))))
--8<---------------cut here---------------end--------------->8---
Graft 2 has this guile-builder:
--8<---------------cut here---------------start------------->8---
(begin
(use-modules (guix build graft) (guix build utils) (ice-9 match))
(define %output (getenv "out"))
(define %outputs (map (lambda (o) (cons o (getenv o))) (quote ("out"))))
(define %build-inputs
(quote (("x" . "/gnu/store/5nmzfnxk8kp85xwma2r585fgyz3jfw56-guile-cairo-1.11.2")
("x" . "/gnu/store/na0x00biq02fm5cyj5a8r67qwsnsskw8-cairo-1.16.0")
("x" . "/gnu/store/fx3979c88s9yxdbchyf36qryawgzpwb5-libx11-1.6.10")
("x" . "/gnu/store/cw69is9wbbllwx95wky4pmbcsk4vvbpd-libxrender-0.9.10")
("x" . "/gnu/store/qrs0p8j3wq6q5a4dm0ndjdavk9gyal5q-libxext-1.3.4")
("x" . "/gnu/store/p51dv37zj24q8001zghc3wxhxz8i3c50-cairo-1.16.0")
("x" . "/gnu/store/rwkqxykm91a75w9afhb41saj0dmf30hw-libx11-1.6.12")
("x" . "/gnu/store/pzj036f1nmxdrbza6cqy419ddsn9bydp-libxrender-0.9.10")
("x" . "/gnu/store/3rmazp46f6g8w9qs8n3w7qcg8hhs1lig-libxext-1.3.4"))))
(unsetenv "GUILE_LOAD_COMPILED_PATH")
(unsetenv "LD_LIBRARY_PATH"))
(exit
(begin
(let* ((old-outputs
(quote
(("out" . "/gnu/store/5nmzfnxk8kp85xwma2r585fgyz3jfw56-guile-cairo-1.11.2"))))
(mapping
(append (quote
(("/gnu/store/na0x00biq02fm5cyj5a8r67qwsnsskw8-cairo-1.16.0"
. "/gnu/store/p51dv37zj24q8001zghc3wxhxz8i3c50-cairo-1.16.0")
("/gnu/store/fx3979c88s9yxdbchyf36qryawgzpwb5-libx11-1.6.10"
. "/gnu/store/rwkqxykm91a75w9afhb41saj0dmf30hw-libx11-1.6.12")
("/gnu/store/cw69is9wbbllwx95wky4pmbcsk4vvbpd-libxrender-0.9.10"
. "/gnu/store/pzj036f1nmxdrbza6cqy419ddsn9bydp-libxrender-0.9.10")
("/gnu/store/qrs0p8j3wq6q5a4dm0ndjdavk9gyal5q-libxext-1.3.4"
. "/gnu/store/3rmazp46f6g8w9qs8n3w7qcg8hhs1lig-libxext-1.3.4")))
(map (match-lambda ((name . file)
(cons (assoc-ref old-outputs name) file)))
%outputs))))
(graft old-outputs %outputs mapping))))
--8<---------------cut here---------------end--------------->8---
I think that my last hypothesis was on the right track, but not quite
right:
* Instead of 'libcairo' being loaded twice, I now suspect that
"libguile-cairo.so" is being loaded twice.
* Instead of the original and replacement libraries being loaded, I now
suspect that two different variants of the replacement "guile-cairo"
are being loaded.
* Instead of libcairo type tags being duplicated, I now suspect that
duplicated smob tags are being allocated.
However, *if* deduplication is enabled, two redundant replacements
created by grafting _should_ occupy the same inodes, assuming that the
replacement mappings are the same (modulo ordering), and assuming that
/gnu/store/.links doesn't hit a directory size limit (which can happen
on ext3/4, leading to missed deduplication opportunities).
I've known about these redundant replacements in Guix for many years,
but was not aware of any significant practical problems arising from
them until now.
Mark
Debbugs is free software and licensed under the terms of the
GNU Public License version 2. The current version can be
obtained from https://bugs.debian.org/debbugs-source/.
