GNU bug report logs

#76850 tests/pack.scm failure (AppImage)

Package	Source(s)		Maintainer(s)
guix		PTS Buildd Popcon

Reported by Reepca Russelstein <reepca@russelstein.xyz>
Date 2025-03-08T03:47:02
Severity normal

Reply or subscribe to this bug. View this bug as an mbox, status mbox, or maintainer mbox

Report forwarded to bug-guix@gnu.org:
bug#76850; Package guix. (Sat, 08 Mar 2025 03:47:02 GMT) (full text, mbox, link).

Acknowledgement sent to Reepca Russelstein <reepca@russelstein.xyz>:
New bug report received and forwarded. Copy sent to bug-guix@gnu.org. (Sat, 08 Mar 2025 03:47:02 GMT) (full text, mbox, link).

Message #5 received at submit@debbugs.gnu.org (full text, mbox, reply):

[Message part 1 (text/plain, inline)]

Or perhaps it would be more accurate to say that it is nondeterministic.
This is because it currently succeeds more or less by accident: the
AppRun symlink points to ./gnu/store/...-profile/bin/hello, which points
to /gnu/store/...-hello-2.12.1/bin/hello (note: absolute path!).  There
shouldn't be any reason for that to exist inside the chroot, except that
the daemon's reference scanner has noticed that some of the inputs to
the appimage-building derivation *may* have their hashes visible in the
built appimage.  I say "may" because the appimage is compressed with
squashfs, so it's a matter of luck whether a hash is actually directly
visible.  Currently, in the master branch, it happens to be visible, but
in my local repository it isn't, and so it fails for me.

To demonstrate this without relying on the fickle compression, find the
store path of the appimage built during the tests/pack.scm "appimage"
test case after running "make check TESTS=tests/pack.scm" (the
"check-appimage" derivation is printed into tests/pack.log, from there
you can find the appimage derivation and its output path), and call it
$IMAGE.  Then:

$ IMAGE=/gnu/store/2c8m9in2pkgkf8p9qgv17dqz19jfxmmm-hello-appimage.AppImage
$ mkdir test-root
$ mkdir test-root/proc
$ mkdir test-root/tmp
$ cp "$IMAGE" test-root/test-image
$ unshare --user --mount --map-root-user

then, in the subshell spawned by unshare:

$ mount --bind /proc test-root/proc
$ chroot ./test-root /test-image --appimage-extract-and-run

you should see "Failed to run
/tmp/appimage_extracted_e331827d4eb2f579cccf6fb79143c261/AppRun: No such
file or directory" or something like it.

- reepca

[signature.asc (application/pgp-signature, inline)]

Information forwarded to bug-guix@gnu.org:
bug#76850; Package guix. (Mon, 17 Mar 2025 19:49:04 GMT) (full text, mbox, link).

Message #8 received at 76850@debbugs.gnu.org (full text, mbox, reply):

Hello,

Thanks for the bug report, Reepca.

Noé, could you take a look?

Thanks,
Ludo’.

Reepca Russelstein <reepca@russelstein.xyz> skribis:

> Or perhaps it would be more accurate to say that it is nondeterministic.
> This is because it currently succeeds more or less by accident: the
> AppRun symlink points to ./gnu/store/...-profile/bin/hello, which points
> to /gnu/store/...-hello-2.12.1/bin/hello (note: absolute path!).  There
> shouldn't be any reason for that to exist inside the chroot, except that
> the daemon's reference scanner has noticed that some of the inputs to
> the appimage-building derivation *may* have their hashes visible in the
> built appimage.  I say "may" because the appimage is compressed with
> squashfs, so it's a matter of luck whether a hash is actually directly
> visible.  Currently, in the master branch, it happens to be visible, but
> in my local repository it isn't, and so it fails for me.
>
> To demonstrate this without relying on the fickle compression, find the
> store path of the appimage built during the tests/pack.scm "appimage"
> test case after running "make check TESTS=tests/pack.scm" (the
> "check-appimage" derivation is printed into tests/pack.log, from there
> you can find the appimage derivation and its output path), and call it
> $IMAGE.  Then:
>
> $ IMAGE=/gnu/store/2c8m9in2pkgkf8p9qgv17dqz19jfxmmm-hello-appimage.AppImage
> $ mkdir test-root
> $ mkdir test-root/proc
> $ mkdir test-root/tmp
> $ cp "$IMAGE" test-root/test-image
> $ unshare --user --mount --map-root-user
>
> then, in the subshell spawned by unshare:
>
> $ mount --bind /proc test-root/proc
> $ chroot ./test-root /test-image --appimage-extract-and-run
>
> you should see "Failed to run
> /tmp/appimage_extracted_e331827d4eb2f579cccf6fb79143c261/AppRun: No such
> file or directory" or something like it.
>
> - reepca

Information forwarded to bug-guix@gnu.org:
bug#76850; Package guix. (Sat, 05 Apr 2025 22:30:02 GMT) (full text, mbox, link).

Message #11 received at 76850@debbugs.gnu.org (full text, mbox, reply):

[Message part 1 (text/plain, inline)]

Ludovic Courtès <ludo@gnu.org> writes:

> Hello,
>
> Thanks for the bug report, Reepca.
>
> Noé, could you take a look?
>
> Thanks,
> Ludo’.
>

Hi Reepca and Ludo,

Thanks for sending this to me Ludo.

This bug report is very well detailed, thank you.

> Reepca Russelstein <reepca@russelstein.xyz> skribis:
>
>> Or perhaps it would be more accurate to say that it is nondeterministic.
>> This is because it currently succeeds more or less by accident: the
>> AppRun symlink points to ./gnu/store/...-profile/bin/hello, which points
>> to /gnu/store/...-hello-2.12.1/bin/hello (note: absolute path!).  There
>> shouldn't be any reason for that to exist inside the chroot, except that
>> the daemon's reference scanner has noticed that some of the inputs to
>> the appimage-building derivation *may* have their hashes visible in the
>> built appimage.  I say "may" because the appimage is compressed with
>> squashfs, so it's a matter of luck whether a hash is actually directly
>> visible.  Currently, in the master branch, it happens to be visible, but
>> in my local repository it isn't, and so it fails for me.

So you’re saying that the daemon is scanning through the files to find
references to add?  As we see here that is unreliable, do you know what
is the reasoning for this?

Indeed, there are no reasons for that absolute path to exist, since we
have not yet entered the AppImage’s chroot/usermount.  This is a
weakness in our current AppImage design, the contents are always
designed to be ran with a store in /gnu/store but this is handled only
by the AppImage runtime (without --appimage-extract).
--appimage-extract bypasses this and we then rely on the --relocatable
binaries, but ideally there would be no need for relocatable.

This is where our issue lies: in our tests we use #:relocatable? on the
self-contained-appimage function but the profile we give it has not
been made relocatably.

>>
>> To demonstrate this without relying on the fickle compression, find the
>> store path of the appimage built during the tests/pack.scm "appimage"
>> test case after running "make check TESTS=tests/pack.scm" (the
>> "check-appimage" derivation is printed into tests/pack.log, from there
>> you can find the appimage derivation and its output path), and call it
>> $IMAGE.  Then:
>>
>> $ IMAGE=/gnu/store/2c8m9in2pkgkf8p9qgv17dqz19jfxmmm-hello-appimage.AppImage
>> $ mkdir test-root
>> $ mkdir test-root/proc
>> $ mkdir test-root/tmp
>> $ cp "$IMAGE" test-root/test-image
>> $ unshare --user --mount --map-root-user
>>
>> then, in the subshell spawned by unshare:
>>
>> $ mount --bind /proc test-root/proc
>> $ chroot ./test-root /test-image --appimage-extract-and-run
>>
>> you should see "Failed to run
>> /tmp/appimage_extracted_e331827d4eb2f579cccf6fb79143c261/AppRun: No such
>> file or directory" or something like it.
>>
>> - reepca

I’m sending a patch with the updated tests.  Sorry for the long wait!

Have a nice day,
Noé

[signature.asc (application/pgp-signature, inline)]

Information forwarded to guix@cbaines.net, dev@jpoiret.xyz, ludo@gnu.org, othacehe@gnu.org, zimon.toutoune@gmail.com, me@tobias.gr, bug-guix@gnu.org:
bug#76850; Package guix. (Sat, 05 Apr 2025 22:33:02 GMT) (full text, mbox, link).

Message #14 received at 76850@debbugs.gnu.org (full text, mbox, reply):

As reported in #76850, the tested AppImages were not actually relocatable and
would rely on items being available on the environment’s store (apart from
glibc).

* guix/scripts/pack.scm (wrapped-manifest): New function.
(guix-pack): Extract relocatable manifest to wrapped-manifest.
* tests/pack.scm: Use relocatable profiles in AppImage tests.

Change-Id: Ib3123054913fce903d215dc0629d806e9fceebc7
---
 guix/scripts/pack.scm | 11 ++++++++---
 tests/pack.scm        |  8 ++++++--
 2 files changed, 14 insertions(+), 5 deletions(-)

diff --git a/guix/scripts/pack.scm b/guix/scripts/pack.scm
index 7ab2c0d447..c0b7216921 100644
--- a/guix/scripts/pack.scm
+++ b/guix/scripts/pack.scm
@@ -1406,6 +1406,13 @@ (define (wrapped-manifest-entry entry . args)
                          (apply wrapped-manifest-entry entry args))
                        (manifest-entry-dependencies entry)))))
 
+(define*-public (wrapped-manifest manifest #:rest args)
+  "Return the MANIFEST with its entries wrapped such that they are
+relocatable. Extra arguments are passed to wrapped-package."
+  (map-manifest-entries
+   (lambda (entry) (apply wrapped-manifest-entry entry args))
+   manifest))
+
 
 ;;;
 ;;; Command-line options.
@@ -1801,9 +1808,7 @@ (define-command (guix-pack . args)
                                   ;; Note: We cannot honor '--bootstrap' here because
                                   ;; 'glibc-bootstrap' lacks 'libc.a'.
                                   (if relocatable?
-                                      (map-manifest-entries
-                                       (cut wrapped-manifest-entry <> #:proot? proot?)
-                                       manifest)
+                                      (wrapped-manifest manifest #:proot? proot?)
                                       manifest)))
                    (pack-format (assoc-ref opts 'format))
                    (extra-options (match pack-format
diff --git a/tests/pack.scm b/tests/pack.scm
index 9c7e0a50ba..1d1aef024f 100644
--- a/tests/pack.scm
+++ b/tests/pack.scm
@@ -350,7 +350,9 @@ (define rpm-for-tests
          (profile -> (profile
                       ;; When using '--appimage-extract-and-run', the dynamic
                       ;; linker is necessary, hence glibc below.
-                      (content (packages->manifest (list hello glibc)))
+                      (content (wrapped-manifest
+                                (packages->manifest (list hello glibc))))
+                      (relative-symlinks? #t)
                       (hooks '())
                       (locales? #f)))
          (image   (self-contained-appimage "hello-appimage" profile
@@ -382,7 +384,9 @@ (define rpm-for-tests
          (profile -> (profile
                       ;; When using '--appimage-extract-and-run', the dynamic
                       ;; linker is necessary, hence glibc below.
-                      (content (packages->manifest (list guile-3.0 glibc)))
+                      (content (wrapped-manifest
+                                (packages->manifest (list guile-3.0 glibc))))
+                      (relative-symlinks? #t)
                       (hooks '())
                       (locales? #f)))
          (image   (self-contained-appimage "guile-appimage" profile

base-commit: 80826c6e038997dc47eb455888f0feaa38c08bf5
-- 
2.49.0

Information forwarded to bug-guix@gnu.org:
bug#76850; Package guix. (Tue, 08 Apr 2025 10:27:02 GMT) (full text, mbox, link).

Message #17 received at 76850@debbugs.gnu.org (full text, mbox, reply):

Hi Noé,

Noé Lopez <noelopez@free.fr> skribis:

> As reported in #76850, the tested AppImages were not actually relocatable and
> would rely on items being available on the environment’s store (apart from
> glibc).
>
> * guix/scripts/pack.scm (wrapped-manifest): New function.
> (guix-pack): Extract relocatable manifest to wrapped-manifest.
> * tests/pack.scm: Use relocatable profiles in AppImage tests.
>
> Change-Id: Ib3123054913fce903d215dc0629d806e9fceebc7

Please add a “Fixes” line and a “Reported-by” tag as is usually done.

> +(define*-public (wrapped-manifest manifest #:rest args)

Simply ‘define*’.

Otherwise LGTM, thanks for fixing it!

Ludo’.

Information forwarded to bug-guix@gnu.org:
bug#76850; Package guix. (Tue, 08 Apr 2025 19:12:02 GMT) (full text, mbox, link).

Message #20 received at 76850@debbugs.gnu.org (full text, mbox, reply):

[Message part 1 (text/plain, inline)]

Ludovic Courtès <ludo@gnu.org> writes:

> Hi Noé,
>
> Noé Lopez <noelopez@free.fr> skribis:
>
>> As reported in #76850, the tested AppImages were not actually relocatable and
>> would rely on items being available on the environment’s store (apart from
>> glibc).
>>
>> * guix/scripts/pack.scm (wrapped-manifest): New function.
>> (guix-pack): Extract relocatable manifest to wrapped-manifest.
>> * tests/pack.scm: Use relocatable profiles in AppImage tests.
>>
>> Change-Id: Ib3123054913fce903d215dc0629d806e9fceebc7
>
> Please add a “Fixes” line and a “Reported-by” tag as is usually done.
>
>> +(define*-public (wrapped-manifest manifest #:rest args)
>
> Simply ‘define*’.
>

Then how do I use it in the test? @@ won’t work, I think because the
module is declarative and guile is inlining it.

Or did you mean I should use define* and add it to the module exports in
define-module?

> Otherwise LGTM, thanks for fixing it!
>
> Ludo’.

Thanks for the review,
Noé

[signature.asc (application/pgp-signature, inline)]

Display info messages

Send a report that this bug log contains spam.

debbugs.gnu.org maintainers <help-debbugs@gnu.org>. Last modified: Wed Apr 16 03:55:17 2025; Machine Name: wallace-server

GNU bug tracking system

Debbugs is free software and licensed under the terms of the GNU Public License version 2. The current version can be obtained from https://bugs.debian.org/debbugs-source/.