Report forwarded
to bug-guix@gnu.org: bug#66866; Package guix.
(Wed, 01 Nov 2023 01:57:01 GMT) (full text, mbox, link).
Acknowledgement sent
to Denis 'GNUtoo' Carikli <GNUtoo@cyberdimension.org>:
New bug report received and forwarded. Copy sent to bug-guix@gnu.org.
(Wed, 01 Nov 2023 01:57:02 GMT) (full text, mbox, link).
Hi dan,
dan <i@dan.games> writes:
> with git bisect i found after this commit the cross-compilation
> couldn't work anymore:
Could you post some more details about what you mean by "couldn't work"?
What exactly is failing, and with what logs?
Best,
--
Josselin Poiret
Subject: aarch64 system cross compilation + pinebook pro image broken?
Date: Sun, 31 Dec 2023 14:23:07 +0100
Hello, bumping this issue since there has been no activity for a while
and this error is still present. I get the exact same error as Denis
when building any Pine64 images. Building with --system=aarch64-linux
instead of --target=aarch64-linux-gnu works, but this is very much
slower.
This is what I get:
> checking for string.h... (cached) yes
> checking whether NLS is requested... yes
> checking for msgfmt... no
> checking for gmsgfmt... :
> checking for xgettext... no
> checking for msgmerge... no
> checking build system type... x86_64-unknown-linux-gnu
> checking host system type... Invalid configuration `aarch64-linux-gnu': machine `aarch64' not recognized
> configure: error: /gnu/store/rb75igdc6daly1mz2ivz7rs8hd85imdz-gash-boot-0.3.0/bin/bash ./config.sub aarch64-linux-gnu failed
> error: in phase 'configure': uncaught exception:
> srfi-34 #<condition &invoke-error [program: "/gnu/store/rb75igdc6daly1mz2ivz7rs8hd85imdz-gash-boot-0.3.0/bin/bash" arguments: ("./configure" "CC_FOR_BUILD=gcc" "CONFIG_SHELL=/gnu/store/rb75igdc6daly1mz2ivz7rs8hd85imdz-gash-boot-0.3.0/bin/bash" "SHELL=/gnu/store/rb75igdc6daly1mz2ivz7rs8hd85imdz-gash-boot-0.3.0/bin/bash" "--prefix=/gnu/store/bl3aq7fnpyxq9w2a7bqa4zqgd8z88y8x-gawk-mesboot-3.1.8" "--enable-fast-install" "--build=x86_64-unknown-linux-gnu" "--host=aarch64-linux-gnu" "ac_cv_func_connect=no") exit-status: 1 term-signal: #f stop-signal: #f] 152c4c0>
> phase `configure' failed after 38.7 seconds
> command "/gnu/store/rb75igdc6daly1mz2ivz7rs8hd85imdz-gash-boot-0.3.0/bin/bash" "./configure" "CC_FOR_BUILD=gcc" "CONFIG_SHELL=/gnu/store/rb75igdc6daly1mz2ivz7rs8hd85imdz-gash-boot-0.3.0/bin/bash" "SHELL=/gnu/store/rb75igdc6daly1mz2ivz7rs8hd85imdz-gash-boot-0.3.0/bin/bash" "--prefix=/gnu/store/bl3aq7fnpyxq9w2a7bqa4zqgd8z88y8x-gawk-mesboot-3.1.8" "--enable-fast-install" "--build=x86_64-unknown-linux-gnu" "--host=aarch64-linux-gnu" "ac_cv_func_connect=no" failed with status 1
Other cross-compiled images like Raspberry Pi work fine for me.
Information forwarded
to bug-guix@gnu.org: bug#66866; Package guix.
(Wed, 03 Jan 2024 09:06:02 GMT) (full text, mbox, link).
Cc: Josselin Poiret <dev@jpoiret.xyz>, Jan Nieuwenhuizen <janneke@gnu.org>,
66866@debbugs.gnu.org
Subject: Re: bug#66866: aarch64 system cross compilation + pinebook pro
image broken?
Date: Wed, 03 Jan 2024 10:05:39 +0100
Hello,
I can reproduce the error by running:
--8<---------------cut here---------------start------------->8---
./pre-inst-env guix system build --target=aarch64-linux-gnu gnu/system/images/pine64.scm
--8<---------------cut here---------------end--------------->8---
which cause an issue in gawk-mesboot:
--8<---------------cut here---------------start------------->8---
checking host system type... Invalid configuration `aarch64-linux-gnu': machine `aarch64' not recognized
configure: error: /gnu/store/rb75igdc6daly1mz2ivz7rs8hd85imdz-gash-boot-0.3.0/bin/bash ./config.sub aarch64-linux-gnu failed
--8<---------------cut here---------------end--------------->8---
Janneke, do you know what could have caused this regression?
Thanks,
Mathieu
Information forwarded
to bug-guix@gnu.org: bug#66866; Package guix.
(Sat, 13 Jan 2024 17:56:01 GMT) (full text, mbox, link).
Cc: Josselin Poiret <dev@jpoiret.xyz>, 66866@debbugs.gnu.org,
Jan Nieuwenhuizen <janneke@gnu.org>
Subject: Re: bug#66866: aarch64 system cross compilation + pinebook pro
image broken?
Date: Sat, 13 Jan 2024 18:55:23 +0100
Hey,
> ./pre-inst-env guix system build --target=aarch64-linux-gnu gnu/system/images/pine64.scm
>
>
> which cause an issue in gawk-mesboot:
>
> checking host system type... Invalid configuration `aarch64-linux-gnu': machine `aarch64' not recognized
> configure: error: /gnu/store/rb75igdc6daly1mz2ivz7rs8hd85imdz-gash-boot-0.3.0/bin/bash ./config.sub aarch64-linux-gnu failed
>
> Janneke, do you know what could have caused this regression?
This probably has nothing to do with the bootstrap packages. I noticed
that the CI succeeds in building the pine64 image:
https://ci.guix.gnu.org/build/3265001/details.
The difference is that the CI is building without grafting. Disabling
grafting locally, seems to do the trick as well:
--8<---------------cut here---------------start------------->8---
./pre-inst-env guix system image gnu/system/images/pine64.scm --no-grafts
--8<---------------cut here---------------end--------------->8---
The issue seems to be that grafting ends-up dragging the bootstrap
packages into the closure when cross-compiling which is quite scary.
Mathieu
Severity set to 'important' from 'normal'
Request was from Mathieu Othacehe <mathieu@meije.mail-host-address-is-not-set>
to control@debbugs.gnu.org.
(Sat, 13 Jan 2024 17:57:02 GMT) (full text, mbox, link).
Changed bug title to 'Grafting breaks cross-compilation' from 'aarch64 system cross compilation + pinebook pro image broken?'
Request was from Mathieu Othacehe <mathieu@meije.mail-host-address-is-not-set>
to control@debbugs.gnu.org.
(Sat, 13 Jan 2024 17:58:02 GMT) (full text, mbox, link).
Information forwarded
to bug-guix@gnu.org: bug#66866; Package guix.
(Sat, 13 Jan 2024 18:43:02 GMT) (full text, mbox, link).
Subject: Re: bug#66866: aarch64 system cross compilation + pinebook pro
image broken?
Date: Sat, 13 Jan 2024 19:41:51 +0100
> The issue seems to be that grafting ends-up dragging the bootstrap
> packages into the closure when cross-compiling which is quite scary.
I could narrow it down somehow.
This commands drags the bootstrap and fails:
--8<---------------cut here---------------start------------->8---
guix build --target=<any-target> alsa-lib
--8<---------------cut here---------------end--------------->8---
while this one doesn't:
--8<---------------cut here---------------start------------->8---
guix build --target=<any-target> alsa-lib --no-grafts
--8<---------------cut here---------------end--------------->8---
Not sure how to go further. Adding Ludo and Efraim.
Thanks,
Mathieu
Information forwarded
to bug-guix@gnu.org: bug#66866; Package guix.
(Sun, 11 Feb 2024 19:35:02 GMT) (full text, mbox, link).
Subject: Re: bug#66866: aarch64 system cross compilation + pinebook pro
image broken?
Date: Sun, 11 Feb 2024 19:34:27 +0000
Hello,
Mathieu Othacehe <othacehe@gnu.org> writes:
> I could narrow it down somehow.
Starting from alsa-lib, I narrowed it down further. I found that the
problem is actually when an input of the package uses copy-build-system.
In the following example, the cross-compilation (with
--target=x86_64-linux-gnu) for testp2 and testp3 works fine. Their
respective inputs are testinput2 (built with trivial-build-system) and
the hello package (built with gnu-build-system).
For testp1 however, whose input testinput1 is created
with copy-build-system, the same thing as for alsa-lib (which depends on
two packages built with copy-build-system) happens: Bootstrap packages
are starting to be built if grafts are used. The outputs of both
testinput1 and testinput2 do not contain store references, so that is
not the issue. It is also no problem to build the packages testinput1,
testinput2 or hello with grafts.
Is there something special about copy-build-system which is not the case
for trivial-build-system and gnu-build-system? I noticed that in
(guix build-system trival) and (guix build-system gnu), there is a
special case for cross-compilation, while in (guix build-system copy),
the same code is used as for a normal build. I don't see why this could
cause an issue with grafts though.
--8<---------------cut here---------------start------------->8---
(define-module (cc-test)
#:use-module (guix build-system copy)
#:use-module (guix build-system gnu)
#:use-module (guix build-system trivial)
#:use-module (gnu packages base)
#:use-module (guix packages)
#:use-module (guix gexp))
(define-public testinput1
(package
(name "testinput1")
(version "0")
(source
(computed-file
"testp1-src"
#~(begin
(mkdir #$output)
(mkdir (string-append #$output "/dummy")))))
(build-system copy-build-system)
(arguments (list #:install-plan ''(("dummy" "dummy"))))
(home-page "")
(synopsis "")
(description "")
(license #f)))
(define-public testinput2
(package
(name "testinput2")
(version "0")
(source #f)
(build-system trivial-build-system)
(arguments
(list
#:builder
#~(begin
(mkdir #$output)
(mkdir (string-append #$output "/dummy")))))
(home-page "")
(synopsis "")
(description "")
(license #f)))
(define-public testp1
(package
(name "testp1")
(version "0")
(source #f)
(build-system trivial-build-system)
(arguments
(list
#:builder #~(mkdir #$output)))
(inputs (list testinput1))
(home-page "")
(synopsis "")
(description "")
(license #f)))
(define-public testp2
(package
(name "testp2")
(version "0")
(source #f)
(build-system trivial-build-system)
(arguments
(list
#:builder #~(mkdir #$output)))
(inputs (list testinput2))
(home-page "")
(synopsis "")
(description "")
(license #f)))
(define-public testp3
(package
(name "testp3")
(version "0")
(source #f)
(build-system trivial-build-system)
(arguments
(list
#:builder #~(mkdir #$output)))
(inputs (list hello))
(home-page "")
(synopsis "")
(description "")
(license #f)))
--8<---------------cut here---------------end--------------->8---
Best,
David
Information forwarded
to bug-guix@gnu.org: bug#66866; Package guix.
(Mon, 18 Mar 2024 18:11:01 GMT) (full text, mbox, link).
Subject: Re: bug#66866: aarch64 system cross compilation + pinebook pro
image broken?
Date: Tue, 19 Mar 2024 01:30:31 +0800
Hi David,
Thanks for sharing your discovery.
David Elsing <david.elsing@posteo.net> writes:
> Starting from alsa-lib, I narrowed it down further. I found that
> the
> problem is actually when an input of the package uses
> copy-build-system.
I spent some time digging into the rabbit hole. After changing
the lower function of the copy-build-system to look more like the
lower function of the gnu-build-system, I'm able to cross compile
alsa-lib without the --no-grafts flag. The changes I made are
like:
--8<---------------cut here---------------start------------->8---
diff --git a/guix/build-system/copy.scm
b/guix/build-system/copy.scm
index d58931b33c..74304b4bfb 100644
--- a/guix/build-system/copy.scm
+++ b/guix/build-system/copy.scm
@@ -66,13 +66,13 @@ (define* (lower name
(bag
(name name)
(system system)
- (host-inputs `(,@(if source
+ (build-inputs `(,@(if source
`(("source" ,source))
'())
- ,@inputs
+ ,@native-inputs
;; Keep the standard inputs of
'gnu-build-system'.
,@(standard-packages)))
- (build-inputs native-inputs)
+ (host-inputs inputs)
(outputs outputs)
(build copy-build)
(arguments (strip-keyword-arguments private-keywords
arguments))))
--8<---------------cut here---------------end--------------->8---
Can we put everything inside build-inputs? From my understanding,
copy-build-system shouldn't care about cross-compilation at all.
Any feedback on this would be really helpful.
Best,
--
dan
Information forwarded
to bug-guix@gnu.org: bug#66866; Package guix.
(Wed, 10 Apr 2024 06:46:03 GMT) (full text, mbox, link).
Subject: Re: bug#66866: aarch64 system cross compilation + pinebook pro
image broken?
Date: Wed, 10 Apr 2024 20:49:55 +0000
Hi dan,
sorry for the late reply. I didn't yet find the reason why different
bootstrap packages are built for the cross build when grafts are
applied. This seems possibly like another bug to me, as the package
inputs which use copy-build-system do not contain store references and
it does not happen when they themselves are built with grafts.
dan <i@dan.games> writes:
> Can we put everything inside build-inputs? From my understanding,
> copy-build-system shouldn't care about cross-compilation at all.
I understand it that way too. In guix/build-system.scm, it is mentioned
that build/host/target are used in the sense of the GNU toolchain [1].
There, "build" is the system used for building and "host" is the target
system for which the package is built. I guess this was confused when
copy-build-system was added [2].
Cheers,
David
[1] https://www.gnu.org/savannah-checkouts/gnu/autoconf/manual/autoconf-2.72/html_node/Specifying-Target-Triplets.html
[2] https://issues.guix.gnu.org/39599
Information forwarded
to bug-guix@gnu.org: bug#66866; Package guix.
(Thu, 11 Apr 2024 05:01:01 GMT) (full text, mbox, link).
Subject: Re: bug#66866: aarch64 system cross compilation + pinebook pro
image broken?
Date: Thu, 11 Apr 2024 12:59:34 +0800 (GMT+08:00)
Hi David,
Thanks for the reply.
Apr 11, 2024 04:50:03 David Elsing <david.elsing@posteo.net>:
>> Can we put everything inside build-inputs? From my understanding,
>> copy-build-system shouldn't care about cross-compilation at all.
> I understand it that way too. In guix/build-system.scm, it is mentioned
> that build/host/target are used in the sense of the GNU toolchain [1].
> There, "build" is the system used for building and "host" is the target
> system for which the package is built. I guess this was confused when
> copy-build-system was added [2].
Then I think I can submit a patch to guix-patches, hoping to bring more
attention to this bug.
Information forwarded
to bug-guix@gnu.org: bug#66866; Package guix.
(Sun, 21 Apr 2024 07:14:05 GMT) (full text, mbox, link).
Hi,
dan <i@dan.games> skribis:
> I spent some time digging into the rabbit hole. After changing
> the lower function of the copy-build-system to look more like the
> lower function of the gnu-build-system, I'm able to cross compile
> alsa-lib without the --no-grafts flag. The changes I made are
> like:
>
> diff --git a/guix/build-system/copy.scm
> b/guix/build-system/copy.scm
> index d58931b33c..74304b4bfb 100644
> --- a/guix/build-system/copy.scm
> +++ b/guix/build-system/copy.scm
> @@ -66,13 +66,13 @@ (define* (lower name
> (bag
> (name name)
> (system system)
> - (host-inputs `(,@(if source
> + (build-inputs `(,@(if source
> `(("source" ,source))
> '())
> - ,@inputs
> + ,@native-inputs
> ;; Keep the standard inputs of
> 'gnu-build-system'.
> ,@(standard-packages)))
> - (build-inputs native-inputs)
> + (host-inputs inputs)
> (outputs outputs)
> (build copy-build)
> (arguments (strip-keyword-arguments private-keywords
> arguments))))
>
> Can we put everything inside build-inputs? From my understanding,
> copy-build-system shouldn't care about cross-compilation at all.
Intuitively, if ‘copy-build-system’ is about copying
architecture-independent files, then it should do the same thing whether
or not we are cross-compiling.
However, users can and do add phases whose result is
architecture-dependent. Small sample:
• ‘desec-certbot-hook’ captures a reference to curl, so it would get
the wrong one when cross-compiling if we assumed build-inputs =
host-inputs.
• ‘chez-scheme-for-racket-bootstrap-bootfiles’ builds stuff when
cross-compiling. Philip, could you explain the intent and what you
expect here?
So it would seem we can’t just assume everything is a native input like
https://issues.guix.gnu.org/70492 does.
Now, as David and you found out, the use of inputs in
build-system/copy.scm:lower is bogus. It seems that it can be fixed by
following the intended definition of build/host inputs, as David
suggested:
But wait! That’s all theoretical because the bag always has (target #f)
and ‘copy-build’ bundles build and host inputs together, as if doing a
native build.
So it seems like https://issues.guix.gnu.org/70492 (putting everything
in ‘build-inputs’) is OK, after all.
But still, there seem to be some expectation that ‘copy-build-system’
can support cross-compilation for real, so maybe we should add a
‘copy-cross-build’ procedure in addition to the patch above.
Thoughts?
Ludo’.
Information forwarded
to bug-guix@gnu.org: bug#66866; Package guix.
(Sun, 12 May 2024 15:40:02 GMT) (full text, mbox, link).
Hello,
I think I finally understand why the problem only occurs with grafts if
a dependency uses copy-build-system. It is actually somewhat
complicated.
When a package is lowered into a derivation by 'package->derivation' in
guix/packages.scm, a list of potentially applicable grafts is created by
the 'bag-grafts' procedure.
This works by first traversing the bag for native packages (where
bag-build-inputs and bag-target-inputs are followed recursively; and
also bag-host-inputs if not cross building) and building the native
derivations of the grafted versions of the packages which have a
replacement (such as glibc at the time of writing). Building the grafted
package (in the 'input-graft' procedure) is done again with
'package->derivation', which eventually calls 'graft-derivation' in
guix/grafts.scm, which in turn calls 'non-self-references', where the
ungrafted package is actually built (ignoring the store monad
indirection), not just its derivation.
In the case of cross builds, the bag is also traversed for the target
dependencies, where the bag-host-inputs are followed recursively.
Analogously, this causes the ungrafted packages and their ungrafted
replacement to be cross built to compute the grafted derivations.
As dan found out, the 'lower' procedure in guix/build-system/copy.scm
incorrectly puts standard-packages into the host inputs. They contain
the glibc-final package in gnu/packages/commencement.scm (which inherits
the replacement of glibc). Because copy-build-system puts them into
bag-host-inputs, the package replacement of the glibc-final package is
then cross built without grafts when calculating its grafted derivation,
which it does not support. Note that the glibc-final package and its
package replacement are distinct from the glibc package and its
replacement in gnu/packages/base.scm. I confirmed this by running
--8<---------------cut here---------------start------------->8---
,use (guix)
(with-store store (run-with-store store (package->cross-derivation (package-replacement (@@ (gnu packages commencement) glibc-final)) "i686-linux-gnu" #:graft? #f)))
--8<---------------cut here---------------end--------------->8---
in the REPL, which returns the same incorrect glibc derivation that is
attempted to be built as a dependency when running
`guix build alsa-lib --target=i686-linux-gnu`.
It is actually possible to compute the graft derivations only when
needed by a store reference (which does not change the derivation of the
grafted package). Then, this problem does not occur even without the fix
for the copy-build-system, as the invalid derivations are never actually
built. With substitutes or after running GC, this may even prevent
building a package with replacement when the requested package output
which (transitively) depends on it has no (transitive) store reference
to it. I made a patch for that here: https://issues.guix.gnu.org/70895
Regarding the changes to the copy-build-system:
Ludovic Courtès <ludo@gnu.org> writes:
> But still, there seem to be some expectation that ‘copy-build-system’
> can support cross-compilation for real, so maybe we should add a
> ‘copy-cross-build’ procedure in addition to the patch above.
In guix/build-system/copy.scm, it is described as an extension of
gnu-build-system and the manual says "It adds much of the
gnu-build-system packages to the set of inputs. Because of this, the
copy-build-system does not require all the boilerplate code often needed
for the trivial-build-system."
Therefore, I think it makes sense to add the copy-cross-build procedure
so that copy-build-system actually behaves like gnu-build-system.
Cheers,
David
Debbugs is free software and licensed under the terms of the
GNU Public License version 2. The current version can be
obtained from https://bugs.debian.org/debbugs-source/.