#42053 hidden derivation build with inferiors

Message #5 received at submit@debbugs.gnu.org (full text, mbox, reply):

From: Julien Lepiller <julien@lepiller.eu>

To: bug-guix@gnu.org

Subject: hidden derivation build with inferiors

Date: Fri, 26 Jun 2020 01:47:12 +0200

[Message part 1 (text/plain, inline)]

Hi,

someone (kernel-help) on IRC today was having issues using an inferior
kernel with their os configuration and reported that guix was
"hanging". It turns out that guix was actually building the kernel in
the background, but did not report anything to the user. kernel-help
was kind enough to share their config, and I can confirm that guix is
building the kernel, but doesn't tell anything.

I was able to track this done to a call to operating-system-derivation,
which is supposed to return the path to a .drv file. I did not expect
it to have to build anything, since it's only computing a derivation
name.

Using the attached operating system declaration, defining it as "os",
you can see that calling ((operating-system-derivation os)
(open-connection)) "hangs" with no message, and a build user is
building the kernel (unless you already have the kernel built).

Note that the build process continues even if you interrupt that
procedure, and only stops after you exit guix repl entirely.

Going one step further, operating-system-derivation calls fold-services
(which is fine), then service-value (which is also fine) and calls that
value with the store, which builds the kernel without saying anything.
That value comes from the root service type, system-service-type and its
system-derivation procedure.

With some pk, I was able to find that mapm/accumulate-builds was the
culprit, but I don't know how to investigate further.

[test-inferior.scm (text/x-scheme, attachment)]

Message #10 received at 42053@debbugs.gnu.org (full text, mbox, reply):

From: Ludovic Courtès <ludo@gnu.org>

To: Julien Lepiller <julien@lepiller.eu>

Cc: 42053@debbugs.gnu.org

Subject: Re: bug#42053: hidden derivation build with inferiors

Date: Fri, 26 Jun 2020 12:17:28 +0200

Hi,

Julien Lepiller <julien@lepiller.eu> skribis:

> someone (kernel-help) on IRC today was having issues using an inferior
> kernel with their os configuration and reported that guix was
> "hanging". It turns out that guix was actually building the kernel in
> the background, but did not report anything to the user. kernel-help
> was kind enough to share their config, and I can confirm that guix is
> building the kernel, but doesn't tell anything.

Aaah right.  What happens here is that grafts lead the inferior to start
building the kernel (this is expected).  However, ‘proxy’ in (guix
inferior) doesn’t forward build output to ‘current-build-output-port’.
That’s why there’s no message saying what’s being built, no spinner, etc.

I’ll see what can be done.

The problem has always been there but it was perhaps less visible before
710854304b1ab29332edcb76f3de532e0724c197, at least when substitutes were
available, because then the inferior wouldn’t start building things.

Ludo’.

Message #13 received at 42053@debbugs.gnu.org (full text, mbox, reply):

From: Julien Lepiller <julien@lepiller.eu>

To: Ludovic Courtès <ludo@gnu.org>

Cc: 42053@debbugs.gnu.org

Subject: Re: bug#42053: hidden derivation build with inferiors

Date: Fri, 26 Jun 2020 06:19:56 -0400

Le 26 juin 2020 06:17:28 GMT-04:00, "Ludovic Courtès" <ludo@gnu.org> a écrit :
>Hi,
>
>Julien Lepiller <julien@lepiller.eu> skribis:
>
>> someone (kernel-help) on IRC today was having issues using an
>inferior
>> kernel with their os configuration and reported that guix was
>> "hanging". It turns out that guix was actually building the kernel in
>> the background, but did not report anything to the user. kernel-help
>> was kind enough to share their config, and I can confirm that guix is
>> building the kernel, but doesn't tell anything.
>
>Aaah right.  What happens here is that grafts lead the inferior to
>start
>building the kernel (this is expected).  However, ‘proxy’ in (guix
>inferior) doesn’t forward build output to ‘current-build-output-port’.
>That’s why there’s no message saying what’s being built, no spinner,
>etc.
>
>I’ll see what can be done.
>
>The problem has always been there but it was perhaps less visible
>before
>710854304b1ab29332edcb76f3de532e0724c197, at least when substitutes
>were
>available, because then the inferior wouldn’t start building things.
>
>Ludo’.

Thanks for the answer. The build also starts with -n, which I didn't expect either. I suppose this is also because the inferior doesn't report anything?

Message #16 received at 42053@debbugs.gnu.org (full text, mbox, reply):

From: Ludovic Courtès <ludo@gnu.org>

To: Julien Lepiller <julien@lepiller.eu>

Cc: 42053@debbugs.gnu.org

Subject: Re: bug#42053: hidden derivation build with inferiors

Date: Fri, 26 Jun 2020 14:08:00 +0200

Hi,

Julien Lepiller <julien@lepiller.eu> skribis:

> Le 26 juin 2020 06:17:28 GMT-04:00, "Ludovic Courtès" <ludo@gnu.org> a écrit :

[...]

>>The problem has always been there but it was perhaps less visible
>>before
>>710854304b1ab29332edcb76f3de532e0724c197, at least when substitutes
>>were
>>available, because then the inferior wouldn’t start building things.
>>
>>Ludo’.
>
> Thanks for the answer. The build also starts with -n, which I didn't expect either. I suppose this is also because the inferior doesn't report anything?

Yes, that and the fact that ‘-n’ no longer implies ‘--no-grafts’.

Ludo’.

Message #19 received at 42053@debbugs.gnu.org (full text, mbox, reply):

From: Ludovic Courtès <ludo@gnu.org>

To: Julien Lepiller <julien@lepiller.eu>

Cc: 42053@debbugs.gnu.org

Subject: Re: bug#42053: hidden derivation build with inferiors

Date: Sun, 28 Jun 2020 22:43:12 +0200

Ludovic Courtès <ludo@gnu.org> skribis:

> Julien Lepiller <julien@lepiller.eu> skribis:
>
>> someone (kernel-help) on IRC today was having issues using an inferior
>> kernel with their os configuration and reported that guix was
>> "hanging". It turns out that guix was actually building the kernel in
>> the background, but did not report anything to the user. kernel-help
>> was kind enough to share their config, and I can confirm that guix is
>> building the kernel, but doesn't tell anything.
>
> Aaah right.  What happens here is that grafts lead the inferior to start
> building the kernel (this is expected).  However, ‘proxy’ in (guix
> inferior) doesn’t forward build output to ‘current-build-output-port’.
> That’s why there’s no message saying what’s being built, no spinner, etc.
>
> I’ll see what can be done.

Not surprisingly, to fix it, we need to implement a proxy that
interprets all the messages exchanged so that it can write to
‘current-build-output-port’ when appropriate (that’s the usual
all-or-nothing problem with proxies.)

For the daemon rewrite in Scheme, we’ll need to have server stubs
anyway, so perhaps that’s an opportunity to fiddle with that.  Caleb?

Ludo’.

Send a report that this bug log contains spam.