#65665 package-mapping with #:deep? #t doesn't get all the implicit inputs

Message #73 received at 65665@debbugs.gnu.org (full text, mbox, reply):

Received: (at 65665) by debbugs.gnu.org; 21 Oct 2023 22:23:00 +0000
From debbugs-submit-bounces@debbugs.gnu.org Sat Oct 21 18:23:00 2023
Received: from localhost ([127.0.0.1]:45211 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.84_2)
	(envelope-from <debbugs-submit-bounces@debbugs.gnu.org>)
	id 1quKN2-0001Dj-DZ
	for submit@debbugs.gnu.org; Sat, 21 Oct 2023 18:23:00 -0400
Received: from tilde.club ([142.44.150.184]:42476 ident=postfix)
 by debbugs.gnu.org with esmtp (Exim 4.84_2)
 (envelope-from <striness@tilde.club>) id 1quKN0-0001DZ-1g
 for 65665@debbugs.gnu.org; Sat, 21 Oct 2023 18:22:59 -0400
Received: by tilde.club (Postfix, from userid 5378)
 id 0C2122202A78C; Sat, 21 Oct 2023 22:22:29 +0000 (UTC)
DKIM-Filter: OpenDKIM Filter v2.11.0 tilde.club 0C2122202A78C
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=tilde.club; s=mail;
 t=1697926949; bh=6Q8HA8Wx5YGg0cCQNm7UZUl75eAErrZG49vYkQMVSpw=;
 h=From:To:Cc:Subject:In-Reply-To:References:Date:From;
 b=RMdsTQ0NaE5uYk9MH3rHWn9tAOgnU9MEsL17uH24uOkX/APzv8Up7GnX4/kR/Nq8z
 8PkpMWEhNFL5zH7+F4pRArDIoF0r37sx5kkO2wjkjMor+f6D6UcztmPkN/jsWqfpf7
 zv4vUjJQxn8wKYqp/cUfI3Sg1zjpdSkK+u6Klbbw=
From: Ulf Herrman <striness@tilde.club>
To: Ludovic Courtès <ludo@gnu.org>
Subject: Re: bug#65665: package-mapping with #:deep? #t doesn't get all the
 implicit inputs
In-Reply-To: <87wmvgoxn7.fsf@gnu.org> ("Ludovic Courtès"'s message of "Sat, 21 Oct 2023 16:31:08 +0200")
References: <87h6ofufy5.fsf@tilde.club> <87msxmqwng.fsf@tilde.club>
 <871qe0lytk.fsf_-_@gnu.org> <87wmvrdxnf.fsf@tilde.club>
 <87y1g5fd1e.fsf@gnu.org> <87o7h0e4us.fsf@tilde.club>
 <87wmvgoxn7.fsf@gnu.org>
User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/28.2 (gnu/linux)
Date: Sat, 21 Oct 2023 17:22:10 -0500
Message-ID: <8734y3r4z1.fsf@tilde.club>
MIME-Version: 1.0
Content-Type: multipart/signed; boundary="=-=-=";
 micalg=pgp-sha256; protocol="application/pgp-signature"
X-Spam-Score: -0.0 (/)
X-Debbugs-Envelope-To: 65665
Cc: Ulf Herrman <striness@tilde.club>, 65665@debbugs.gnu.org
X-BeenThere: debbugs-submit@debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request@debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit@debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request@debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request@debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces@debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces@debbugs.gnu.org>
X-Spam-Score: -1.0 (-)

[Message part 1 (text/plain, inline)]

Ludovic Courtès <ludo@gnu.org> writes:

> I don’t know, should we start by having a proper bug report for this and
> study how this happen?

Does that mean opening a new issue, or what?  The bug you've described
is the one this issue was initially opened for.

> Again I’m sorry if I’m slow to understand, but I’d like to make sure we
> have a good understanding of the problem before we start discussing
> solutions.

Okay, I suppose I should have given a concrete example of the behavior.
The qgit example can fill that role:

$ cat $(./pre-inst-env guix build qgit -n --derivations --no-grafts --with-latest=qtbase)
<observe that an untransformed qtbase is present>

The way this happens is that qt-build-system's bag-builder
(e.g. qt-build, qt-cross-build)) introduces a reference to the qtbase
from its #:qtbase argument into the gexp it creates a derivation from,
completely independently of any inputs.  qgit doesn't explicitly specify
#:qtbase, and qt-build-system's 'lower' procedure doesn't add one, so
the default value for that keyword argument, (default-qtbase), is used
in qt-build.

This default value can only be overridden by explicitly passing #:qtbase
as a package or bag argument.  This requirement doesn't map well to a
generic package transformation interface at all - it requires that every
transformer (e.g. package-mapping) check if the package it's
transforming is using qt-build-system, and if so explicitly supply a
#:qtbase that is the result of transforming the original implicit or
explicit value, after somehow figuring out what that may be (currently
only doable by manually reading guix/build-system/qt.scm).

This behavior is also currently exhibited by all build systems' handling
of #:guile.  Here's a concrete example of that, taken from another
mailing I sent to this issue (https://issues.guix.gnu.org/65665#15):

---------------------------------
(use-modules (guix packages)
             (guix profiles)
             (gnu packages base))

(define guile-named-lyle
  (package
    (inherit (default-guile))
    (name "lyle")))

;; contrived example that only replaces hello's immediate dependencies
(define hello-transformer
  (package-mapping (lambda (p0)
                     (if (eq? p0 (default-guile))
                         guile-named-lyle
                         p0))
                   (lambda (p)
                     (not (eq? p hello)))
                   #:deep? #t))

(define hello-with-lyle
  (hello-transformer hello))

(packages->manifest (list hello hello-with-lyle))

;; $ guix build --derivations --manifest=THISFILE
;; Expectation: build for hello-with-lyle is done with guile-named-lyle.
;; Reality: derivation for hello-with-lyle is the same as hello.
---------------------------------

Hopefully that makes it clear why this is happening.

As for solutions, some options that come to mind:

1. guile and qtbase, instead of being passed to bag builders as a
   separate argument, are passed as a bag input that is looked up by
   magic input label, e.g. "guile-for-build", "qtbase-for-build", etc.
   Seems fragile, but requires no changes to package-mapping, etc.

2. Modify the build systems so that these kinds of implicit arguments
   that are packages are always present in bags, and the defaults for
   those keyword arguments in e.g. qt-build are never used.  This is the
   approach I've taken with
   https://issues.guix.gnu.org/issue/65665/attachment/4/0/3.  This still
   requires that bag arguments that are packages are transformed in
   addition to inputs.

I hope that counts as a proper bug report.

- Ulf

[signature.asc (application/pgp-signature, inline)]

Send a report that this bug log contains spam.