#33848 - Store references in SBCL-compiled code are "invisible"

Package	Source(s)		Maintainer(s)
guix		PTS Buildd Popcon

Message #112 received at 33848@debbugs.gnu.org (full text, mbox, reply):

Received: (at 33848) by debbugs.gnu.org; 1 Apr 2021 09:07:38 +0000
From debbugs-submit-bounces@debbugs.gnu.org Thu Apr 01 05:07:38 2021
Received: from localhost ([127.0.0.1]:56289 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.84_2)
	(envelope-from <debbugs-submit-bounces@debbugs.gnu.org>)
	id 1lRtIg-00017c-Fn
	for submit@debbugs.gnu.org; Thu, 01 Apr 2021 05:07:38 -0400
Received: from mout01.posteo.de ([185.67.36.65]:55957)
 by debbugs.gnu.org with esmtp (Exim 4.84_2)
 (envelope-from <glv@posteo.net>) id 1lRtIe-00017M-G5
 for 33848@debbugs.gnu.org; Thu, 01 Apr 2021 05:07:37 -0400
Received: from submission (posteo.de [89.146.220.130]) 
 by mout01.posteo.de (Postfix) with ESMTPS id 34E82160060
 for <33848@debbugs.gnu.org>; Thu,  1 Apr 2021 11:07:30 +0200 (CEST)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=posteo.net; s=2017;
 t=1617268050; bh=pFX2Hdgq596McSZNUd1m2x4kbOv/dl3wNhLj0AS/cRA=;
 h=From:To:Cc:Subject:Date:From;
 b=JKb9yiqao5+HNkGlxV95zwKrHSeyw0WRcMfvZTAuyb+zX2evsRKpuvbJK39N5MNmy
 yaGTbrn64lEvNDzSUT/yb9DwkXzR3TGCHFpFbNMTZI0l+Z5IAZMlmDl/1RjHOr1DAm
 3ytGOapAUM3OzpqAktv+nkbEasB+Nspnrdy1XIHECQR2nOU8Iy1hpOos8KD2TJ0qeA
 yRlxTVd4ccHn1ttAWFNQzZuUgzJfT7jxO7mRt1One4XfRqocfV6kpJi6Xi+Me8wfgn
 KuMS0DMngRaerWjWtRhJljy08HkyZIbJNLx00/nXosu3bEqG3Kvpyr6moYRYZyVcmM
 IqGa7zWXy4WXg==
Received: from customer (localhost [127.0.0.1])
 by submission (posteo.de) with ESMTPSA id 4F9y4c5vw3z9rxV;
 Thu,  1 Apr 2021 11:07:28 +0200 (CEST)
References: <87r2e8jpfx.fsf@gnu.org> <87d0psi1xo.fsf@gnu.org>
 <874lb3kin6.fsf@ambrevar.xyz> <87sgynezha.fsf@gnu.org>
 <87tvj2yesd.fsf@netris.org> <877efwe04u.fsf@gnu.org>
 <8736qji7c1.fsf@ambrevar.xyz> <87tvizvzgk.fsf@netris.org>
 <87o9979gfn.fsf@gnu.org> <87tvizgghs.fsf@ambrevar.xyz>
 <87k1juaomo.fsf@gnu.org> <87muoqhk62.fsf@ambrevar.xyz>
 <87zhsq8wkj.fsf@gnu.org> <87d0pmhbgn.fsf@ambrevar.xyz>
 <87r2e28tkv.fsf@gnu.org> <874laygkiy.fsf@ambrevar.xyz>
 <87lfa5eymf.fsf@ambrevar.xyz> <87tuoscsk9.fsf@gnu.org>
 <87im57b8u7.fsf@ambrevar.xyz> <87czvebky2.fsf@netris.org>
 <87eefu30a4.fsf@gnu.org>
User-agent: mu4e 1.4.15; emacs 27.2
From: Guillaume Le Vaillant <glv@posteo.net>
To: Ludovic Courtès <ludo@gnu.org>
Subject: Re: bug#33848: Store references in SBCL-compiled code are "invisible"
In-reply-to: <87eefu30a4.fsf@gnu.org>
Date: Thu, 01 Apr 2021 11:07:23 +0200
Message-ID: <87im56l6es.fsf@yamatai>
MIME-Version: 1.0
Content-Type: multipart/signed; boundary="=-=-=";
 micalg=pgp-sha512; protocol="application/pgp-signature"
X-Spam-Score: -0.3 (/)
X-Debbugs-Envelope-To: 33848
Cc: Mark H Weaver <mhw@netris.org>, Pierre Neidhardt <mail@ambrevar.xyz>,
 33848@debbugs.gnu.org
X-BeenThere: debbugs-submit@debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request@debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit@debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request@debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request@debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces@debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces@debbugs.gnu.org>
X-Spam-Score: -1.3 (-)

[Message part 1 (text/plain, inline)]

Pierre Neidhardt <mail@ambrevar.xyz> skribis:

> If we are going for a SBCL-specific solution, I believe that all
> references are stored in plain text files at some points (the .asd files
> and the .lisp files) which are often encoded in ASCII / UTF-8, so
> searching these files without having to deal with their encoding might
> be enough.  But of course this is less general and more brittle.

A store reference to a C library in a standalone Lisp binary can come
either from the current package or from some dependencies
(cl+ssl, cl-cffi-gtk, etc.). So we would need to scan the source
code of all the Lisp dependencies recursively to get the full list of
store refrences.
And as Mark wrote below, with the current grafting code, this list of
store references will not solve grafting for paths that are in UTF-32 or
both in ASCII and UTF-32 in the binary file.


Ludovic Courtès <ludo@gnu.org> skribis:

> Mark H Weaver <mhw@netris.org> skribis:
>
>> Pierre Neidhardt <mail@ambrevar.xyz> writes:
>>> - The main recommendation for an easy fix without updating the scanner
>>>   is that we tweaked our build system to dump the store reference to a
>>>   separate ASCII file.
>>
>> Sounds good.  I made a similar proposal in Dec 2018, earlier in this
>> thread <https://bugs.gnu.org/33848#31>.  I wrote:
>>
>>   If you don't want to change the daemon, it could be worked around in our
>>   build-side code as follows: we could add a new phase to certain build
>>   systems (or possibly gnu-build-system) that scans each output for
>>   UTF-16/32 encoded store references that are never referenced in UTF-8.
>>   If such references exist, a file with an unobtrusive name would be added
>>   to that output containing those references encoded in UTF-8.  This would
>>   enable our daemon's existing reference scanner to find all of the
>>   references.
>>
>>   Our grafting code would then need to be extended to recognize and
>>   transform store references encoded in UTF-16/32 as well as UTF-8.
>
> Oh thanks for the reminder.
>
> The separate ASCII file doesn’t solve it all because, as you write, we’d
> need to change the grafting code as well.
>
> Then it might be simpler to use a “byte vector” data type for those
> strings.  We’ll have to wait for Pierre’s patches to get a better idea.
> :-)

I'm not sure what you mean with the "byte vector" data type here. Could
you explain what you're thinking about with a few more details?

[signature.asc (application/pgp-signature, inline)]

Display info messages

Send a report that this bug log contains spam.

debbugs.gnu.org maintainers <help-debbugs@gnu.org>. Last modified: Sat Apr 12 15:00:50 2025; Machine Name: wallace-server

GNU bug tracking system

Debbugs is free software and licensed under the terms of the GNU Public License version 2. The current version can be obtained from https://bugs.debian.org/debbugs-source/.

#33848 Store references in SBCL-compiled code are "invisible"