GNU bug report logs

#48907 Grafts cause discrepancies in debug symbols file names (debug symbols missing in GDB).

Package	Source(s)		Maintainer(s)
guix		PTS Buildd Popcon

Reported by Maxim Cournoyer <maxim.cournoyer@gmail.com>
Date 2021-06-07T18:20:02
Severity important

Reply or subscribe to this bug. View this bug as an mbox, status mbox, or maintainer mbox

Report forwarded to bug-guix@gnu.org:
bug#48907; Package guix. (Mon, 07 Jun 2021 18:20:02 GMT) (full text, mbox, link).

Acknowledgement sent to Maxim Cournoyer <maxim.cournoyer@gmail.com>:
New bug report received and forwarded. Copy sent to bug-guix@gnu.org. (Mon, 07 Jun 2021 18:20:02 GMT) (full text, mbox, link).

Message #5 received at submit@debbugs.gnu.org (full text, mbox, reply):

Hello,

While attempting to debug a crash in jami-qt, I've noticed that GDB
would fail to load the symbol tables of the shared libraries it uses,
for example qtdeclarative.

It seems that grafts are to blame.  Let's start by looking at the debug
files found for the qtdeclarative libQt5Qml.so.5 shared library:

--8<---------------cut here---------------start------------->8---
$ guix build qtdeclarative | xargs -I{} find -L {} -name *libQt5Qml.so.5*
substitute: updating substitutes from 'http://127.0.0.1:8080'... 100.0%
substitute: updating substitutes from 'https://ci.guix.gnu.org'... 100.0%
The following files will be downloaded:
   /gnu/store/g1gxfbkyxilnx7s6mjdlj697y5n5wazn-qtdeclarative-5.15.2-debug
   /gnu/store/nvzvrr137qfqsn2875yrs9ilfd12wi96-qtdeclarative-5.15.2
substituting /gnu/store/g1gxfbkyxilnx7s6mjdlj697y5n5wazn-qtdeclarative-5.15.2-debug...
downloading from https://ci.guix.gnu.org/nar/lzip/g1gxfbkyxilnx7s6mjdlj697y5n5wazn-qtdeclarative-5.15.2-debug ...
 qtdeclarative-5.15.2-debug  94.9MiB                            1.2MiB/s 01:21 [##################] 100.0%

The following graft will be made:
   /gnu/store/djhcai9rixm2j3jlamwdhsgwgidg7w74-qtdeclarative-5.15.2.drv
applying 2 grafts for /gnu/store/djhcai9rixm2j3jlamwdhsgwgidg7w74-qtdeclarative-5.15.2.drv ...
grafting '/gnu/store/g1gxfbkyxilnx7s6mjdlj697y5n5wazn-qtdeclarative-5.15.2-debug' -> '/gnu/store/l3h4ka7v3j1yhik0f1phwch08a09p0bx-qtdeclarative-5.15.2-debug'...
grafting '/gnu/store/nvzvrr137qfqsn2875yrs9ilfd12wi96-qtdeclarative-5.15.2' -> '/gnu/store/pryhgzb6cwnzsskqwldwc6dxr6nwnywf-qtdeclarative-5.15.2'...
updating '.gnu_debuglink' CRC in '/gnu/store/pryhgzb6cwnzsskqwldwc6dxr6nwnywf-qtdeclarative-5.15.2/bin/qml'
updating '.gnu_debuglink' CRC in '/gnu/store/pryhgzb6cwnzsskqwldwc6dxr6nwnywf-qtdeclarative-5.15.2/bin/q
[...]
updating '.gnu_debuglink' CRC in '/gnu/store/pryhgzb6cwnzsskqwldwc6dxr6nwnywf-qtdeclarative-5.15.2/lib/qt5/qml/QtQuick/Window.2/libwindowplugin.so'
updating '.gnu_debuglink' CRC in '/gnu/store/pryhgzb6cwnzsskqwldwc6dxr6nwnywf-qtdeclarative-5.15.2/lib/qt5/qml/QtTest/libqmltestplugin.so'
successfully built /gnu/store/djhcai9rixm2j3jlamwdhsgwgidg7w74-qtdeclarative-5.15.2.drv
/gnu/store/l3h4ka7v3j1yhik0f1phwch08a09p0bx-qtdeclarative-5.15.2-debug/lib/debug/gnu/store/pryhgzb6cwnzsskqwldwc6dxr6nwnywf-qtdeclarative-5.15.2/lib/libQt5Qml.so.5.15.2.debug
/gnu/store/pryhgzb6cwnzsskqwldwc6dxr6nwnywf-qtdeclarative-5.15.2/lib/libQt5Qml.so.5.15.2
/gnu/store/pryhgzb6cwnzsskqwldwc6dxr6nwnywf-qtdeclarative-5.15.2/lib/libQt5Qml.so.5
/gnu/store/pryhgzb6cwnzsskqwldwc6dxr6nwnywf-qtdeclarative-5.15.2/lib/libQt5Qml.so.5.15
--8<---------------cut here---------------end--------------->8---

So far so good.  The file hierarchy under the debug output matches the
actual shared library file name.  Next, let's verify which qtdeclarative
shared libraries jami-qt is dynamically linked against:

--8<---------------cut here---------------start------------->8---
$ guix build jami-qt | tail -1 | xargs -I{} ldd {}/bin/.jami-qt-real | grep qtdeclarative
        libQt5QuickWidgets.so.5 => /gnu/store/mjl02yma4r5xjark6d8pp5h2j0a2vccs-qtdeclarative-5.15.2/lib/libQt5QuickWidgets.so.5 (0x00007fb9e38a8000)
        libQt5Quick.so.5 => /gnu/store/mjl02yma4r5xjark6d8pp5h2j0a2vccs-qtdeclarative-5.15.2/lib/libQt5Quick.so.5 (0x00007fb9dba47000)
        libQt5QmlModels.so.5 => /gnu/store/mjl02yma4r5xjark6d8pp5h2j0a2vccs-qtdeclarative-5.15.2/lib/libQt5QmlModels.so.5 (0x00007fb9db9c3000)
        libQt5Qml.so.5 => /gnu/store/mjl02yma4r5xjark6d8pp5h2j0a2vccs-qtdeclarative-5.15.2/lib/libQt5Qml.so.5 (0x00007fb9dae4e000)
--8<---------------cut here---------------end--------------->8---

Oops!  The actual store file name of the libQt5Qml.so.5 known to jami-qt
is *not* the same as the one obtained earlier, which explains why GDB
doesn't find its symbols.  Without grafts, the first command gives:

--8<---------------cut here---------------start------------->8---
$ guix build --no-grafts qtdeclarative | xargs -I{} find -L {} -name *libQt5Qml.so.5*
/gnu/store/g1gxfbkyxilnx7s6mjdlj697y5n5wazn-qtdeclarative-5.15.2-debug/lib/debug/gnu/store/nvzvrr137qfqsn2875yrs9ilfd12wi96-qtdeclarative-5.15.2/lib/libQt5Qml.so.5.15.2.debug
/gnu/store/nvzvrr137qfqsn2875yrs9ilfd12wi96-qtdeclarative-5.15.2/lib/libQt5Qml.so.5
/gnu/store/nvzvrr137qfqsn2875yrs9ilfd12wi96-qtdeclarative-5.15.2/lib/libQt5Qml.so.5.15
/gnu/store/nvzvrr137qfqsn2875yrs9ilfd12wi96-qtdeclarative-5.15.2/lib/libQt5Qml.so.5.15.2
--8<---------------cut here---------------end--------------->8---

Which still doesn't match the libraries jami-qt is linked with.

I'm out of ideas for now.  Would someone have a clue?

Thank you!

Maxim

Information forwarded to bug-guix@gnu.org:
bug#48907; Package guix. (Mon, 07 Jun 2021 19:27:02 GMT) (full text, mbox, link).

Message #8 received at 48907@debbugs.gnu.org (full text, mbox, reply):

Hello again,

Building everything without grafts does resolve the file name
discrepancy issue, e.g.:

guix build --no-grafts qtdeclarative | xargs -I{} find -L {} -name *libQt5Qml.so.5*
/gnu/store/g1gxfbkyxilnx7s6mjdlj697y5n5wazn-qtdeclarative-5.15.2-debug/lib/debug/gnu/store/nvzvrr137qfqsn2875yrs9ilfd12wi96-qtdeclarative-5.15.2/lib/libQt5Qml.so.5.15.2.debug
/gnu/store/nvzvrr137qfqsn2875yrs9ilfd12wi96-qtdeclarative-5.15.2/lib/libQt5Qml.so.5
/gnu/store/nvzvrr137qfqsn2875yrs9ilfd12wi96-qtdeclarative-5.15.2/lib/libQt5Qml.so.5.15
/gnu/store/nvzvrr137qfqsn2875yrs9ilfd12wi96-qtdeclarative-5.15.2/lib/libQt5Qml.so.5.15.2

guix build --no-grafts jami-qt | tail -1 | xargs -I{} ldd {}/bin/.jami-qt-real | grep qtdeclarative
        libQt5QuickWidgets.so.5 => /gnu/store/nvzvrr137qfqsn2875yrs9ilfd12wi96-qtdeclarative-5.15.2/lib/libQt5QuickWidgets.so.5 (0x00007f42c79b2000)
        libQt5Quick.so.5 => /gnu/store/nvzvrr137qfqsn2875yrs9ilfd12wi96-qtdeclarative-5.15.2/lib/libQt5Quick.so.5 (0x00007f42bfb52000)
        libQt5QmlModels.so.5 => /gnu/store/nvzvrr137qfqsn2875yrs9ilfd12wi96-qtdeclarative-5.15.2/lib/libQt5QmlModels.so.5 (0x00007f42bface000)
        libQt5Qml.so.5 => /gnu/store/nvzvrr137qfqsn2875yrs9ilfd12wi96-qtdeclarative-5.15.2/lib/libQt5Qml.so.5 (0x00007f42bef59000)

And the debug symbols are now discovered and usable by GDB.  So this
issue is indeed caused by grafts.

Maxim

Changed bug title to 'Grafts cause discrepancies in debug symbols file names (debug symbols missing in GDB).' from 'Debug symbols file name discrepancies' Request was from Maxim Cournoyer <maxim.cournoyer@gmail.com> to control@debbugs.gnu.org. (Mon, 07 Jun 2021 19:29:01 GMT) (full text, mbox, link).

Information forwarded to bug-guix@gnu.org:
bug#48907; Package guix. (Fri, 18 Jun 2021 09:31:01 GMT) (full text, mbox, link).

Message #13 received at 48907@debbugs.gnu.org (full text, mbox, reply):

Hi,

Maxim Cournoyer <maxim.cournoyer@gmail.com> skribis:

> $ guix build qtdeclarative | xargs -I{} find -L {} -name *libQt5Qml.so.5*
> substitute: updating substitutes from 'http://127.0.0.1:8080'... 100.0%
> substitute: updating substitutes from 'https://ci.guix.gnu.org'... 100.0%
> The following files will be downloaded:
>    /gnu/store/g1gxfbkyxilnx7s6mjdlj697y5n5wazn-qtdeclarative-5.15.2-debug
>    /gnu/store/nvzvrr137qfqsn2875yrs9ilfd12wi96-qtdeclarative-5.15.2
> substituting /gnu/store/g1gxfbkyxilnx7s6mjdlj697y5n5wazn-qtdeclarative-5.15.2-debug...
> downloading from https://ci.guix.gnu.org/nar/lzip/g1gxfbkyxilnx7s6mjdlj697y5n5wazn-qtdeclarative-5.15.2-debug ...
>  qtdeclarative-5.15.2-debug  94.9MiB                            1.2MiB/s 01:21 [##################] 100.0%
>
> The following graft will be made:
>    /gnu/store/djhcai9rixm2j3jlamwdhsgwgidg7w74-qtdeclarative-5.15.2.drv
> applying 2 grafts for /gnu/store/djhcai9rixm2j3jlamwdhsgwgidg7w74-qtdeclarative-5.15.2.drv ...
> grafting '/gnu/store/g1gxfbkyxilnx7s6mjdlj697y5n5wazn-qtdeclarative-5.15.2-debug' -> '/gnu/store/l3h4ka7v3j1yhik0f1phwch08a09p0bx-qtdeclarative-5.15.2-debug'...
> grafting '/gnu/store/nvzvrr137qfqsn2875yrs9ilfd12wi96-qtdeclarative-5.15.2' -> '/gnu/store/pryhgzb6cwnzsskqwldwc6dxr6nwnywf-qtdeclarative-5.15.2'...
> updating '.gnu_debuglink' CRC in '/gnu/store/pryhgzb6cwnzsskqwldwc6dxr6nwnywf-qtdeclarative-5.15.2/bin/qml'
> updating '.gnu_debuglink' CRC in '/gnu/store/pryhgzb6cwnzsskqwldwc6dxr6nwnywf-qtdeclarative-5.15.2/bin/q
> [...]
> updating '.gnu_debuglink' CRC in '/gnu/store/pryhgzb6cwnzsskqwldwc6dxr6nwnywf-qtdeclarative-5.15.2/lib/qt5/qml/QtQuick/Window.2/libwindowplugin.so'
> updating '.gnu_debuglink' CRC in '/gnu/store/pryhgzb6cwnzsskqwldwc6dxr6nwnywf-qtdeclarative-5.15.2/lib/qt5/qml/QtTest/libqmltestplugin.so'
> successfully built /gnu/store/djhcai9rixm2j3jlamwdhsgwgidg7w74-qtdeclarative-5.15.2.drv
> /gnu/store/l3h4ka7v3j1yhik0f1phwch08a09p0bx-qtdeclarative-5.15.2-debug/lib/debug/gnu/store/pryhgzb6cwnzsskqwldwc6dxr6nwnywf-qtdeclarative-5.15.2/lib/libQt5Qml.so.5.15.2.debug
> /gnu/store/pryhgzb6cwnzsskqwldwc6dxr6nwnywf-qtdeclarative-5.15.2/lib/libQt5Qml.so.5.15.2
> /gnu/store/pryhgzb6cwnzsskqwldwc6dxr6nwnywf-qtdeclarative-5.15.2/lib/libQt5Qml.so.5
> /gnu/store/pryhgzb6cwnzsskqwldwc6dxr6nwnywf-qtdeclarative-5.15.2/lib/libQt5Qml.so.5.15
>
>
> So far so good.  The file hierarchy under the debug output matches the
> actual shared library file name.  Next, let's verify which qtdeclarative
> shared libraries jami-qt is dynamically linked against:
>
> $ guix build jami-qt | tail -1 | xargs -I{} ldd {}/bin/.jami-qt-real | grep qtdeclarative
>         libQt5QuickWidgets.so.5 => /gnu/store/mjl02yma4r5xjark6d8pp5h2j0a2vccs-qtdeclarative-5.15.2/lib/libQt5QuickWidgets.so.5 (0x00007fb9e38a8000)
>         libQt5Quick.so.5 => /gnu/store/mjl02yma4r5xjark6d8pp5h2j0a2vccs-qtdeclarative-5.15.2/lib/libQt5Quick.so.5 (0x00007fb9dba47000)
>         libQt5QmlModels.so.5 => /gnu/store/mjl02yma4r5xjark6d8pp5h2j0a2vccs-qtdeclarative-5.15.2/lib/libQt5QmlModels.so.5 (0x00007fb9db9c3000)
>         libQt5Qml.so.5 => /gnu/store/mjl02yma4r5xjark6d8pp5h2j0a2vccs-qtdeclarative-5.15.2/lib/libQt5Qml.so.5 (0x00007fb9dae4e000)

This is due to the fact that, when you run ‘guix build jami-qt’, the
grafting derivation dismisses the “debug” output of qtdeclarative, since
jami-qt does not depend on it.  That way it doesn’t have to
build/download and graft qtdeclarative:debug.

Conversely, when you run ‘guix build qtdeclarative’, it grafts both
outputs of qtdeclarative.  Thus, this grafting derivation is different
from the one jami-qt.drv depends on.

This behavior was implemented in commit
482fda2729c3e76999892cb8f9a0391a7bd37119.  Not sure what a good solution
would be.

Thoughts?

Ludo’.

Information forwarded to bug-guix@gnu.org:
bug#48907; Package guix. (Fri, 24 Sep 2021 02:33:01 GMT) (full text, mbox, link).

Message #16 received at 48907@debbugs.gnu.org (full text, mbox, reply):

Hi!

Ludovic Courtès <ludo@gnu.org> writes:

> Hi,
>
> Maxim Cournoyer <maxim.cournoyer@gmail.com> skribis:
>
>> $ guix build qtdeclarative | xargs -I{} find -L {} -name *libQt5Qml.so.5*
>> substitute: updating substitutes from 'http://127.0.0.1:8080'... 100.0%
>> substitute: updating substitutes from 'https://ci.guix.gnu.org'... 100.0%
>> The following files will be downloaded:
>>    /gnu/store/g1gxfbkyxilnx7s6mjdlj697y5n5wazn-qtdeclarative-5.15.2-debug
>>    /gnu/store/nvzvrr137qfqsn2875yrs9ilfd12wi96-qtdeclarative-5.15.2
>> substituting /gnu/store/g1gxfbkyxilnx7s6mjdlj697y5n5wazn-qtdeclarative-5.15.2-debug...
>> downloading from https://ci.guix.gnu.org/nar/lzip/g1gxfbkyxilnx7s6mjdlj697y5n5wazn-qtdeclarative-5.15.2-debug ...
>>  qtdeclarative-5.15.2-debug  94.9MiB                            1.2MiB/s 01:21 [##################] 100.0%
>>
>> The following graft will be made:
>>    /gnu/store/djhcai9rixm2j3jlamwdhsgwgidg7w74-qtdeclarative-5.15.2.drv
>> applying 2 grafts for /gnu/store/djhcai9rixm2j3jlamwdhsgwgidg7w74-qtdeclarative-5.15.2.drv ...
>> grafting '/gnu/store/g1gxfbkyxilnx7s6mjdlj697y5n5wazn-qtdeclarative-5.15.2-debug' -> '/gnu/store/l3h4ka7v3j1yhik0f1phwch08a09p0bx-qtdeclarative-5.15.2-debug'...
>> grafting '/gnu/store/nvzvrr137qfqsn2875yrs9ilfd12wi96-qtdeclarative-5.15.2' -> '/gnu/store/pryhgzb6cwnzsskqwldwc6dxr6nwnywf-qtdeclarative-5.15.2'...
>> updating '.gnu_debuglink' CRC in '/gnu/store/pryhgzb6cwnzsskqwldwc6dxr6nwnywf-qtdeclarative-5.15.2/bin/qml'
>> updating '.gnu_debuglink' CRC in '/gnu/store/pryhgzb6cwnzsskqwldwc6dxr6nwnywf-qtdeclarative-5.15.2/bin/q
>> [...]
>> updating '.gnu_debuglink' CRC in '/gnu/store/pryhgzb6cwnzsskqwldwc6dxr6nwnywf-qtdeclarative-5.15.2/lib/qt5/qml/QtQuick/Window.2/libwindowplugin.so'
>> updating '.gnu_debuglink' CRC in '/gnu/store/pryhgzb6cwnzsskqwldwc6dxr6nwnywf-qtdeclarative-5.15.2/lib/qt5/qml/QtTest/libqmltestplugin.so'
>> successfully built /gnu/store/djhcai9rixm2j3jlamwdhsgwgidg7w74-qtdeclarative-5.15.2.drv
>> /gnu/store/l3h4ka7v3j1yhik0f1phwch08a09p0bx-qtdeclarative-5.15.2-debug/lib/debug/gnu/store/pryhgzb6cwnzsskqwldwc6dxr6nwnywf-qtdeclarative-5.15.2/lib/libQt5Qml.so.5.15.2.debug
>> /gnu/store/pryhgzb6cwnzsskqwldwc6dxr6nwnywf-qtdeclarative-5.15.2/lib/libQt5Qml.so.5.15.2
>> /gnu/store/pryhgzb6cwnzsskqwldwc6dxr6nwnywf-qtdeclarative-5.15.2/lib/libQt5Qml.so.5
>> /gnu/store/pryhgzb6cwnzsskqwldwc6dxr6nwnywf-qtdeclarative-5.15.2/lib/libQt5Qml.so.5.15
>>
>>
>> So far so good.  The file hierarchy under the debug output matches the
>> actual shared library file name.  Next, let's verify which qtdeclarative
>> shared libraries jami-qt is dynamically linked against:
>>
>> $ guix build jami-qt | tail -1 | xargs -I{} ldd {}/bin/.jami-qt-real | grep qtdeclarative
>>         libQt5QuickWidgets.so.5 => /gnu/store/mjl02yma4r5xjark6d8pp5h2j0a2vccs-qtdeclarative-5.15.2/lib/libQt5QuickWidgets.so.5 (0x00007fb9e38a8000)
>>         libQt5Quick.so.5 => /gnu/store/mjl02yma4r5xjark6d8pp5h2j0a2vccs-qtdeclarative-5.15.2/lib/libQt5Quick.so.5 (0x00007fb9dba47000)
>>         libQt5QmlModels.so.5 => /gnu/store/mjl02yma4r5xjark6d8pp5h2j0a2vccs-qtdeclarative-5.15.2/lib/libQt5QmlModels.so.5 (0x00007fb9db9c3000)
>>         libQt5Qml.so.5 => /gnu/store/mjl02yma4r5xjark6d8pp5h2j0a2vccs-qtdeclarative-5.15.2/lib/libQt5Qml.so.5 (0x00007fb9dae4e000)
>
> This is due to the fact that, when you run ‘guix build jami-qt’, the
> grafting derivation dismisses the “debug” output of qtdeclarative, since
> jami-qt does not depend on it.  That way it doesn’t have to
> build/download and graft qtdeclarative:debug.
>
> Conversely, when you run ‘guix build qtdeclarative’, it grafts both
> outputs of qtdeclarative.  Thus, this grafting derivation is different
> from the one jami-qt.drv depends on.
>
> This behavior was implemented in commit
> 482fda2729c3e76999892cb8f9a0391a7bd37119.  Not sure what a good solution
> would be.
>
> Thoughts?

Yikes!  This means that debugging with grafts (with the aid of debugging
symbols) is no longer possible, right?

I remember reading about a 2nd option to locate the separate debug
symbol files with GDB in info '(gdb) Separate Debug Files':


   * The executable contains a "build ID", a unique bit string that is
     also present in the corresponding debug info file.  (This is
     supported only on some operating systems, when using the ELF or PE
     file formats for binary files and the GNU Binutils.)  For more
     details about this feature, see the description of the '--build-id'
     command-line option in *note Command Line Options: (ld)Options.
     The debug info file's name is not specified explicitly by the build
     ID, but can be computed from the build ID, see below.

   [...]
   
   * For the "build ID" method, GDB looks in the '.build-id'
     subdirectory of each one of the global debug directories for a file
     named 'NN/NNNNNNNN.debug', where NN are the first 2 hex characters
     of the build ID bit string, and NNNNNNNN are the rest of the bit
     string.  (Real build ID strings are 32 or more hex characters, not
     10.)

What may help us here, compared to debug links, is that it seems to be
file name agnostic; the debug files would be matched by an internal ID
that they got at build time rather than from their file names (which
doesn't work with the current different derivations in the presence of
grafts).

Perhaps it'd also lift the need to recompute the CRC checksum of the
debug links produced when grafting!

HTH!

Maxim

Severity set to 'important' from 'normal' Request was from Ludovic Courtès <ludo@gnu.org> to control@debbugs.gnu.org. (Fri, 24 Sep 2021 14:08:01 GMT) (full text, mbox, link).

Information forwarded to bug-guix@gnu.org:
bug#48907; Package guix. (Fri, 24 Sep 2021 14:15:01 GMT) (full text, mbox, link).

Message #21 received at 48907@debbugs.gnu.org (full text, mbox, reply):

Hi,

Maxim Cournoyer <maxim.cournoyer@gmail.com> skribis:

> Ludovic Courtès <ludo@gnu.org> writes:

[...]

>> This is due to the fact that, when you run ‘guix build jami-qt’, the
>> grafting derivation dismisses the “debug” output of qtdeclarative, since
>> jami-qt does not depend on it.  That way it doesn’t have to
>> build/download and graft qtdeclarative:debug.

[...]

> Yikes!  This means that debugging with grafts (with the aid of debugging
> symbols) is no longer possible, right?

It depends on whether the separate “debug” output gets grafted or not,
but yeah, if a dependency tree has this shape (app -> lib + lib:debug),
running ‘guix install app’ alone will prevent you from getting debugging
symbols from ‘lib:debug’ I believe.  That sucks.

I wonder if we should revert 482fda2729c3e76999892cb8f9a0391a7bd37119.
It’s often not very helpful anyway (we often find ourselves downloading
unnecessary package outputs because of grafting).

> I remember reading about a 2nd option to locate the separate debug
> symbol files with GDB in info '(gdb) Separate Debug Files':
>
>
>    * The executable contains a "build ID", a unique bit string that is

We’d have to check if this is applicable.  Looking at the ld manual
(info "(ld) Options"), it seems that the UUID “style” is ruled out
because it’s non-deterministic, and the md5 and sha1 styles would
require us to rewrite build IDs IIUC, similar to how we rewrite CRCs.

Thanks,
Ludo’.

Information forwarded to bug-guix@gnu.org:
bug#48907; Package guix. (Tue, 28 Sep 2021 02:26:01 GMT) (full text, mbox, link).

Message #24 received at 48907@debbugs.gnu.org (full text, mbox, reply):

Heya,

Ludovic Courtès <ludo@gnu.org> writes:

[...]

>> Yikes!  This means that debugging with grafts (with the aid of debugging
>> symbols) is no longer possible, right?
>
> It depends on whether the separate “debug” output gets grafted or not,
> but yeah, if a dependency tree has this shape (app -> lib + lib:debug),
> running ‘guix install app’ alone will prevent you from getting debugging
> symbols from ‘lib:debug’ I believe.  That sucks.
>
> I wonder if we should revert 482fda2729c3e76999892cb8f9a0391a7bd37119.
> It’s often not very helpful anyway (we often find ourselves downloading
> unnecessary package outputs because of grafting).

Hmm.  Perhaps.  But it'd also suck to have to download 1 GiB of unneeded
debugging symbols to just apply a graft to Qt, for example.

>> I remember reading about a 2nd option to locate the separate debug
>> symbol files with GDB in info '(gdb) Separate Debug Files':
>>
>>
>>    * The executable contains a "build ID", a unique bit string that is
>
> We’d have to check if this is applicable.  Looking at the ld manual
> (info "(ld) Options"), it seems that the UUID “style” is ruled out
> because it’s non-deterministic, and the md5 and sha1 styles would
> require us to rewrite build IDs IIUC, similar to how we rewrite CRCs.

Seems like it could work?  simark from #gdb says it should be
deterministic for reproducible builds.  We'd need to fixup the grafted
debug output, but they could being done in a separate derivation would
no longer matter (as the debug symbols would be matched on a unique ID
that is not linked to that derivation, not on their file name, which
is).

Did I get the above right?

Thanks,

Maxim

Information forwarded to bug-guix@gnu.org:
bug#48907; Package guix. (Tue, 28 Sep 2021 09:46:01 GMT) (full text, mbox, link).

Message #27 received at 48907@debbugs.gnu.org (full text, mbox, reply):

Hi,

Maxim Cournoyer <maxim.cournoyer@gmail.com> skribis:

> Ludovic Courtès <ludo@gnu.org> writes:
>
> [...]
>
>>> Yikes!  This means that debugging with grafts (with the aid of debugging
>>> symbols) is no longer possible, right?
>>
>> It depends on whether the separate “debug” output gets grafted or not,
>> but yeah, if a dependency tree has this shape (app -> lib + lib:debug),
>> running ‘guix install app’ alone will prevent you from getting debugging
>> symbols from ‘lib:debug’ I believe.  That sucks.
>>
>> I wonder if we should revert 482fda2729c3e76999892cb8f9a0391a7bd37119.
>> It’s often not very helpful anyway (we often find ourselves downloading
>> unnecessary package outputs because of grafting).
>
> Hmm.  Perhaps.  But it'd also suck to have to download 1 GiB of unneeded
> debugging symbols to just apply a graft to Qt, for example.

Yeah.  That’s already the case in some cases though, that’s what I
meant.

>>> I remember reading about a 2nd option to locate the separate debug
>>> symbol files with GDB in info '(gdb) Separate Debug Files':
>>>
>>>
>>>    * The executable contains a "build ID", a unique bit string that is
>>
>> We’d have to check if this is applicable.  Looking at the ld manual
>> (info "(ld) Options"), it seems that the UUID “style” is ruled out
>> because it’s non-deterministic, and the md5 and sha1 styles would
>> require us to rewrite build IDs IIUC, similar to how we rewrite CRCs.
>
> Seems like it could work?  simark from #gdb says it should be
> deterministic for reproducible builds.  We'd need to fixup the grafted
> debug output, but they could being done in a separate derivation would
> no longer matter (as the debug symbols would be matched on a unique ID
> that is not linked to that derivation, not on their file name, which
> is).
>
> Did I get the above right?

To summarize, ‘.gnu_debuglink’ in executables/libraries contains the
CRC of the debug file.

Conversely, IIUC what the “normative parts of the output contents” (info
"(ld) Options") really are, build IDs are computed on the code, not on
debug info.

But the problems remains the same I think: if you have
/gnu/store/abc…/libfoo.so and /gnu/store/xyz…/libfoo.so, chances are
that they are different due to embedded store file names, and thus get a
different build ID.

Am I right?

(BTW, I just noticed build IDs were also discussed at
<https://issues.guix.gnu.org/25752>.)

Ludo’.

Information forwarded to bug-guix@gnu.org:
bug#48907; Package guix. (Tue, 28 Sep 2021 10:30:02 GMT) (full text, mbox, link).

Message #30 received at 48907@debbugs.gnu.org (full text, mbox, reply):

Ludovic Courtès <ludo@gnu.org> skribis:

> Conversely, IIUC what the “normative parts of the output contents” (info
> "(ld) Options") really are, build IDs are computed on the code, not on
> debug info.
>
> But the problems remains the same I think: if you have
> /gnu/store/abc…/libfoo.so and /gnu/store/xyz…/libfoo.so, chances are
> that they are different due to embedded store file names, and thus get a
> different build ID.

We discussed this with Mark Wielaard on #guix¹, and one important
takeaway is:

--8<---------------cut here---------------start------------->8---
<civodul> so gdb just checks that the separate debug file has the same
	  build-id as the code, right?  [12:16]
<civodul> it doesn't matter whether it really is the sha1 of all those
	  sections, does it?
<mjw> that is kind of the whole point of the build-id, that it captures the
      whole build environment, not just the generated code, but also how it
      was generated (which is what the .debug sections kind of represent)
<civodul> ok  [12:17]
<mjw> civodul, no, it doesn't need to be a hash over the actual bits
      produced. It can be a completely different hash, it can be a different
      number of bits (but not too short, they need to be globally unique).
<civodul> ok, so we could have our own way of caculating build IDs  [12:18]
<mjw> civodul, all that really matters is that it uniquely identifies this
      binary blob. If any input, source, compiler, flags, etc. changes, it
      should be unique.
--8<---------------cut here---------------end--------------->8---

So I suspect that we would not need to rewrite build IDs upon grafting,
and we could use the ungrafted debug info with grafted code and vice
versa.

We should try it out to test the hypothesis, but if that works, that’d
be great.

Ludo’.

¹ https://logs.guix.gnu.org/guix/2021-09-28.log#114610

Information forwarded to bug-guix@gnu.org:
bug#48907; Package guix. (Mon, 04 Oct 2021 13:15:02 GMT) (full text, mbox, link).

Message #33 received at 48907@debbugs.gnu.org (full text, mbox, reply):

Hi,

Maxim Cournoyer <maxim.cournoyer@gmail.com> skribis:

> Ludovic Courtès <ludo@gnu.org> writes:
>
> [...]
>
>>> Yikes!  This means that debugging with grafts (with the aid of debugging
>>> symbols) is no longer possible, right?
>>
>> It depends on whether the separate “debug” output gets grafted or not,
>> but yeah, if a dependency tree has this shape (app -> lib + lib:debug),
>> running ‘guix install app’ alone will prevent you from getting debugging
>> symbols from ‘lib:debug’ I believe.  That sucks.
>>
>> I wonder if we should revert 482fda2729c3e76999892cb8f9a0391a7bd37119.
>> It’s often not very helpful anyway (we often find ourselves downloading
>> unnecessary package outputs because of grafting).
>
> Hmm.  Perhaps.  But it'd also suck to have to download 1 GiB of unneeded
> debugging symbols to just apply a graft to Qt, for example.

Yeah.  That’s already the case in some cases though, that’s what I
meant.

>>> I remember reading about a 2nd option to locate the separate debug
>>> symbol files with GDB in info '(gdb) Separate Debug Files':
>>>
>>>
>>>    * The executable contains a "build ID", a unique bit string that is
>>
>> We’d have to check if this is applicable.  Looking at the ld manual
>> (info "(ld) Options"), it seems that the UUID “style” is ruled out
>> because it’s non-deterministic, and the md5 and sha1 styles would
>> require us to rewrite build IDs IIUC, similar to how we rewrite CRCs.
>
> Seems like it could work?  simark from #gdb says it should be
> deterministic for reproducible builds.  We'd need to fixup the grafted
> debug output, but they could being done in a separate derivation would
> no longer matter (as the debug symbols would be matched on a unique ID
> that is not linked to that derivation, not on their file name, which
> is).
>
> Did I get the above right?

To summarize, ‘.gnu_debuglink’ in executables/libraries contains the
CRC of the debug file.

Conversely, IIUC what the “normative parts of the output contents” (info
"(ld) Options") really are, build IDs are computed on the code, not on
debug info.

But the problems remains the same I think: if you have
/gnu/store/abc…/libfoo.so and /gnu/store/xyz…/libfoo.so, chances are
that they are different due to embedded store file names, and thus get a
different build ID.

Am I right?

(BTW, I just noticed build IDs were also discussed at
<https://issues.guix.gnu.org/25752>.)

Ludo’.

Information forwarded to bug-guix@gnu.org:
bug#48907; Package guix. (Sat, 27 Apr 2024 08:05:06 GMT) (full text, mbox, link).

Message #36 received at 48907@debbugs.gnu.org (full text, mbox, reply):

Hello all.

Ludovic Courtès <ludo@gnu.org> writes:
>>> I wonder if we should revert 482fda2729c3e76999892cb8f9a0391a7bd37119.
>>> It’s often not very helpful anyway (we often find ourselves downloading
>>> unnecessary package outputs because of grafting).
>>
>> Hmm.  Perhaps.  But it'd also suck to have to download 1 GiB of unneeded
>> debugging symbols to just apply a graft to Qt, for example.
>
> Yeah.  That’s already the case in some cases though, that’s what I
> meant.

I believe I got bitten by this 482fda2729c3e76999892cb8f9a0391a7bd37119
too, not only for debug symbols, which annoys me, and not like the Go
users, but for me the issue surfaced when trying to use guile-g-golf,
whose hello-world.scm errors out with “cannot register existing type
'GdkPixbuf'” and GDB debugging shows (I think) it is because a variant
of gdk-pixbuf without a debug output is used by Gtk while guile-g-golf
loads gdk-pixbuf with debug output before.  `guix shell --no-grafts` is
a workaround.

Perhaps guile-gi have the same issue.  Quoting
<https://github.com/spk121/guile-gi/issues/96>:
> It is pretty clear to me, that the main culprit here is a different
> version of GLib being linked to Guile-GI than the one that should be
> loaded through Guile-GI.

In essence it seems to me 482fda2729c3e76999892cb8f9a0391a7bd37119 is
implemented at the wrong level.  It changed the grafting derivation and
thus the hash, when it could probably be done in guix-daemon directly.

Is it better to wait with fixing this until the guix-daemon is rewritten
in Guile?  And use `--no-grafts' in the meantime?

Regards,
Florian

Information forwarded to bug-guix@gnu.org:
bug#48907; Package guix. (Thu, 30 May 2024 01:13:02 GMT) (full text, mbox, link).

Message #39 received at 48907@debbugs.gnu.org (full text, mbox, reply):

Hi Florian,

"pelzflorian (Florian Pelz)" <pelzflorian@pelzflorian.de> writes:

> Hello all.
>
> Ludovic Courtès <ludo@gnu.org> writes:
>>>> I wonder if we should revert 482fda2729c3e76999892cb8f9a0391a7bd37119.
>>>> It’s often not very helpful anyway (we often find ourselves downloading
>>>> unnecessary package outputs because of grafting).
>>>
>>> Hmm.  Perhaps.  But it'd also suck to have to download 1 GiB of unneeded
>>> debugging symbols to just apply a graft to Qt, for example.
>>
>> Yeah.  That’s already the case in some cases though, that’s what I
>> meant.
>
> I believe I got bitten by this 482fda2729c3e76999892cb8f9a0391a7bd37119
> too, not only for debug symbols, which annoys me, and not like the Go
> users, but for me the issue surfaced when trying to use guile-g-golf,
> whose hello-world.scm errors out with “cannot register existing type
> 'GdkPixbuf'” and GDB debugging shows (I think) it is because a variant
> of gdk-pixbuf without a debug output is used by Gtk while guile-g-golf
> loads gdk-pixbuf with debug output before.  `guix shell --no-grafts` is
> a workaround.

I'm guessing bug #64836 ("pygobject GTK modules lookup fails following
CUPS graft") may be another manifestation of this problem.

-- 
Thanks,
Maxim

Information forwarded to bug-guix@gnu.org:
bug#48907; Package guix. (Thu, 30 May 2024 02:40:02 GMT) (full text, mbox, link).

Message #42 received at 48907@debbugs.gnu.org (full text, mbox, reply):

Hi all,

I believe https://issues.guix.gnu.org/62890 is another example of this
problem. Possibly https://issues.guix.gnu.org/47115#23 as well.

One idea I had was to somehow graft each output as a unique derivation
via repeated calls to graft-derivation. From what I've seen so far
though that looks much easier said than done. It hasn't progressed
beyond the "huh, I wonder" phase.

Funny, I just started investigating the grafting code myself to see if I
could understand what's going on and I stumble upon this 3 year old
issue just an hour after Maxim comments.

-- 
Take it easy,
Richard Sent
Making my computer weirder one commit at a time.

Display info messages

Send a report that this bug log contains spam.

debbugs.gnu.org maintainers <help-debbugs@gnu.org>. Last modified: Sun Sep 8 05:03:11 2024; Machine Name: wallace-server

GNU bug tracking system

Debbugs is free software and licensed under the terms of the GNU Public License version 2. The current version can be obtained from https://bugs.debian.org/debbugs-source/.