tests/pack.scm failure (AppImage)

Open

Details

4 participants

Ludovic Courtès
Noé Lopez
Noé Lopez
Reepca Russelstein

Owner: unassigned

Submitted by: Reepca Russelstein

Severity: normal

Debbugs page

Reepca Russelstein wrote on 7 Mar 19:45 -0800

Recipients:(address . bug-guix@gnu.org)

Message-ID:87v7sk5god.fsf@russelstein.xyz

Or perhaps it would be more accurate to say that it is nondeterministic.
This is because it currently succeeds more or less by accident: the
AppRun symlink points to ./gnu/store/...-profile/bin/hello, which points
to /gnu/store/...-hello-2.12.1/bin/hello (note: absolute path!).  There
shouldn't be any reason for that to exist inside the chroot, except that
the daemon's reference scanner has noticed that some of the inputs to
the appimage-building derivation *may* have their hashes visible in the
built appimage.  I say "may" because the appimage is compressed with
squashfs, so it's a matter of luck whether a hash is actually directly
visible.  Currently, in the master branch, it happens to be visible, but
in my local repository it isn't, and so it fails for me.

To demonstrate this without relying on the fickle compression, find the
store path of the appimage built during the tests/pack.scm "appimage"
test case after running "make check TESTS=tests/pack.scm" (the
"check-appimage" derivation is printed into tests/pack.log, from there
you can find the appimage derivation and its output path), and call it
$IMAGE.  Then:

$ IMAGE=/gnu/store/2c8m9in2pkgkf8p9qgv17dqz19jfxmmm-hello-appimage.AppImage
$ mkdir test-root
$ mkdir test-root/proc
$ mkdir test-root/tmp
$ cp "$IMAGE" test-root/test-image
$ unshare --user --mount --map-root-user

then, in the subshell spawned by unshare:

$ mount --bind /proc test-root/proc
$ chroot ./test-root /test-image --appimage-extract-and-run

you should see "Failed to run
/tmp/appimage_extracted_e331827d4eb2f579cccf6fb79143c261/AppRun: No such
file or directory" or something like it.

- reepca

Ludovic Courtès wrote on 17 Mar 12:48 -0700

Recipients:(name . Reepca Russelstein)(address . reepca@russelstein.xyz)

Message-ID:87tt7rl9qm.fsf@gnu.org

Hello,

Thanks for the bug report, Reepca.

Noé, could you take a look?

Thanks,

Ludo’.

Reepca Russelstein <reepca@russelstein.xyz> skribis:

Toggle quote (36 lines)> Or perhaps it would be more accurate to say that it is nondeterministic.
> This is because it currently succeeds more or less by accident: the
> AppRun symlink points to ./gnu/store/...-profile/bin/hello, which points
> to /gnu/store/...-hello-2.12.1/bin/hello (note: absolute path!).  There
> shouldn't be any reason for that to exist inside the chroot, except that
> the daemon's reference scanner has noticed that some of the inputs to
> the appimage-building derivation *may* have their hashes visible in the
> built appimage.  I say "may" because the appimage is compressed with
> squashfs, so it's a matter of luck whether a hash is actually directly
> visible.  Currently, in the master branch, it happens to be visible, but
> in my local repository it isn't, and so it fails for me.
>
> To demonstrate this without relying on the fickle compression, find the
> store path of the appimage built during the tests/pack.scm "appimage"
> test case after running "make check TESTS=tests/pack.scm" (the
> "check-appimage" derivation is printed into tests/pack.log, from there
> you can find the appimage derivation and its output path), and call it
> $IMAGE.  Then:
>
> $ IMAGE=/gnu/store/2c8m9in2pkgkf8p9qgv17dqz19jfxmmm-hello-appimage.AppImage
> $ mkdir test-root
> $ mkdir test-root/proc
> $ mkdir test-root/tmp
> $ cp "$IMAGE" test-root/test-image
> $ unshare --user --mount --map-root-user
>
> then, in the subshell spawned by unshare:
>
> $ mount --bind /proc test-root/proc
> $ chroot ./test-root /test-image --appimage-extract-and-run
>
> you should see "Failed to run
> /tmp/appimage_extracted_e331827d4eb2f579cccf6fb79143c261/AppRun: No such
> file or directory" or something like it.
>
> - reepca

Noé Lopez wrote 22 hours ago

Recipients:(address . 76850@debbugs.gnu.org)

Message-ID:87r026xmwy.fsf@xn--no-cja.eu

Ludovic Courtès <ludo@gnu.org> writes:

Toggle quote (10 lines)> Hello,
>
> Thanks for the bug report, Reepca.
>
> Noé, could you take a look?
>
> Thanks,
> Ludo’.
>

Hi Reepca and Ludo,

Thanks for sending this to me Ludo.

This bug report is very well detailed, thank you.

Toggle quote (14 lines)> Reepca Russelstein <reepca@russelstein.xyz> skribis:
>
>> Or perhaps it would be more accurate to say that it is nondeterministic.
>> This is because it currently succeeds more or less by accident: the
>> AppRun symlink points to ./gnu/store/...-profile/bin/hello, which points
>> to /gnu/store/...-hello-2.12.1/bin/hello (note: absolute path!).  There
>> shouldn't be any reason for that to exist inside the chroot, except that
>> the daemon's reference scanner has noticed that some of the inputs to
>> the appimage-building derivation *may* have their hashes visible in the
>> built appimage.  I say "may" because the appimage is compressed with
>> squashfs, so it's a matter of luck whether a hash is actually directly
>> visible.  Currently, in the master branch, it happens to be visible, but
>> in my local repository it isn't, and so it fails for me.

So you’re saying that the daemon is scanning through the files to find
references to add?  As we see here that is unreliable, do you know what
is the reasoning for this?

Indeed, there are no reasons for that absolute path to exist, since we
have not yet entered the AppImage’s chroot/usermount.  This is a
weakness in our current AppImage design, the contents are always
designed to be ran with a store in /gnu/store but this is handled only
by the AppImage runtime (without --appimage-extract).
--appimage-extract bypasses this and we then rely on the --relocatable
binaries, but ideally there would be no need for relocatable.

This is where our issue lies: in our tests we use #:relocatable? on the
self-contained-appimage function but the profile we give it has not
been made relocatably.

Toggle quote (26 lines)>>
>> To demonstrate this without relying on the fickle compression, find the
>> store path of the appimage built during the tests/pack.scm "appimage"
>> test case after running "make check TESTS=tests/pack.scm" (the
>> "check-appimage" derivation is printed into tests/pack.log, from there
>> you can find the appimage derivation and its output path), and call it
>> $IMAGE.  Then:
>>
>> $ IMAGE=/gnu/store/2c8m9in2pkgkf8p9qgv17dqz19jfxmmm-hello-appimage.AppImage
>> $ mkdir test-root
>> $ mkdir test-root/proc
>> $ mkdir test-root/tmp
>> $ cp "$IMAGE" test-root/test-image
>> $ unshare --user --mount --map-root-user
>>
>> then, in the subshell spawned by unshare:
>>
>> $ mount --bind /proc test-root/proc
>> $ chroot ./test-root /test-image --appimage-extract-and-run
>>
>> you should see "Failed to run
>> /tmp/appimage_extracted_e331827d4eb2f579cccf6fb79143c261/AppRun: No such
>> file or directory" or something like it.
>>
>> - reepca

I’m sending a patch with the updated tests. Sorry for the long wait!

Have a nice day,

Noé

Noé Lopez wrote 22 hours ago

[PATCH] tests: pack: Fix nondeterministic AppImage tests.

Recipients:(address . 76850@debbugs.gnu.org)(name . Noé Lopez)(address . noelopez@free.fr)

Message-ID:187fec823285b71c80fff33a86227554575289f7.1743892064.git.noelopez@free.fr

As reported in #76850, the tested AppImages were not actually relocatable and

would rely on items being available on the environment’s store (apart from

glibc).

* guix/scripts/pack.scm (wrapped-manifest): New function.

(guix-pack): Extract relocatable manifest to wrapped-manifest.

* tests/pack.scm: Use relocatable profiles in AppImage tests.

Change-Id: Ib3123054913fce903d215dc0629d806e9fceebc7

---

guix/scripts/pack.scm | 11 ++++++++---

tests/pack.scm | 8 ++++++--

2 files changed, 14 insertions(+), 5 deletions(-)

Toggle diff (59 lines)diff --git a/guix/scripts/pack.scm b/guix/scripts/pack.scm
index 7ab2c0d447..c0b7216921 100644
--- a/guix/scripts/pack.scm
+++ b/guix/scripts/pack.scm
@@ -1406,6 +1406,13 @@ (define (wrapped-manifest-entry entry . args)
                          (apply wrapped-manifest-entry entry args))
                        (manifest-entry-dependencies entry)))))
 
+(define*-public (wrapped-manifest manifest #:rest args)
+  "Return the MANIFEST with its entries wrapped such that they are
+relocatable. Extra arguments are passed to wrapped-package."
+  (map-manifest-entries
+   (lambda (entry) (apply wrapped-manifest-entry entry args))
+   manifest))
+
 
 ;;;
 ;;; Command-line options.
@@ -1801,9 +1808,7 @@ (define-command (guix-pack . args)
                                   ;; Note: We cannot honor '--bootstrap' here because
                                   ;; 'glibc-bootstrap' lacks 'libc.a'.
                                   (if relocatable?
-                                      (map-manifest-entries
-                                       (cut wrapped-manifest-entry <> #:proot? proot?)
-                                       manifest)
+                                      (wrapped-manifest manifest #:proot? proot?)
                                       manifest)))
                    (pack-format (assoc-ref opts 'format))
                    (extra-options (match pack-format
diff --git a/tests/pack.scm b/tests/pack.scm
index 9c7e0a50ba..1d1aef024f 100644
--- a/tests/pack.scm
+++ b/tests/pack.scm
@@ -350,7 +350,9 @@ (define rpm-for-tests
          (profile -> (profile
                       ;; When using '--appimage-extract-and-run', the dynamic
                       ;; linker is necessary, hence glibc below.
-                      (content (packages->manifest (list hello glibc)))
+                      (content (wrapped-manifest
+                                (packages->manifest (list hello glibc))))
+                      (relative-symlinks? #t)
                       (hooks '())
                       (locales? #f)))
          (image   (self-contained-appimage "hello-appimage" profile
@@ -382,7 +384,9 @@ (define rpm-for-tests
          (profile -> (profile
                       ;; When using '--appimage-extract-and-run', the dynamic
                       ;; linker is necessary, hence glibc below.
-                      (content (packages->manifest (list guile-3.0 glibc)))
+                      (content (wrapped-manifest
+                                (packages->manifest (list guile-3.0 glibc))))
+                      (relative-symlinks? #t)
                       (hooks '())
                       (locales? #f)))
          (image   (self-contained-appimage "guile-appimage" profile

base-commit: 80826c6e038997dc47eb455888f0feaa38c08bf5
-- 
2.49.0

Your comment

Commenting via the web interface is currently disabled.

To comment on this conversation send an email to 76850@patchwise.org

To respond to this issue using the mumi CLI, first switch to it

mumi current 76850

Then, you may apply the latest patchset in this issue (with sign off)

mumi am -- -s

Or, compose a reply to this issue

mumi compose

Or, send patches to this issue

mumi send-email *.patch

is:open	open issues
is:done	closed issues
submitter:<who>	search issue submitter
author:<who>	search by message author
date:yesterday..now	search by issue date
mdate:3m..2d	search by message date