Build coordiantor "Signals delivery fails constantly" crashes

  • Open
  • quality assurance status badge
Details
2 participants
  • Ludovic Courtès
  • Christopher Baines
Owner
unassigned
Submitted by
Christopher Baines
Severity
important

Debbugs page

C
C
Christopher Baines wrote on 8 May 2023 03:45
(address . bug-guix@gnu.org)
87mt2fytxh.fsf@cbaines.net
Since the recent core-updates merge, I've seen the build coordinator
using less memory, but it's also been crashing in a new way, up to 10
times a day.

In the log, you see something like:

2023-05-07 09:15:42 Signals delivery fails constantly at GC #71051
2023-05-07 09:15:42 Signals delivery fails constantly

I'm guessing the switch from libgc-8.0.4 to libgc-8.2.2 has something to
do with this.
-----BEGIN PGP SIGNATURE-----

iQKlBAEBCgCPFiEEPonu50WOcg2XVOCyXiijOwuE9XcFAmRY1PpfFIAAAAAALgAo
aXNzdWVyLWZwckBub3RhdGlvbnMub3BlbnBncC5maWZ0aGhvcnNlbWFuLm5ldDNF
ODlFRUU3NDU4RTcyMEQ5NzU0RTBCMjVFMjhBMzNCMEI4NEY1NzcRHG1haWxAY2Jh
aW5lcy5uZXQACgkQXiijOwuE9XeOqw//UN9MXhHtF+pwIIVhQE7Al0LYHq/92RQq
utN3xu7iQt9KZVfvc+rcjwujcMOja8XRuikO9pBCFGyV/NdblyCjerlPgxlTSF4l
FPuzUTl9Omotvxw+d2fJ4lsEKcruBiLO+jy89OMHUgB/DWP06xXgrgG7cOF2t6Pf
+nOXd2LsYjGAGnqWfJnwQiouk/8MlzzHQgmLw28Spcu4GL81NlcPoQwRz8Q7Plu+
hsZ28uaWw57TgxQUeEESGvMl0KBHTR1shc5FWc5HW/5KqOMI0FqByON6mheoa7fU
wm/TUEASbvBSk9Sqka0i2aNdexnKGhZatkMcypRutWmuVnAHNXgb266+elMmWQya
eMFHH5Zyf9fkhDQU+46rnM45MvCViAbredUpcrLAffmk0pb4v2IAl5+XdLgSmBdg
N3R5VwxG+dXFs6EEQ3kTotIWvdyfBMwLM1VSYe6mPaGRr8kn3ni6uFUPjjyfkGLH
4txlR7XzpvKh705jFFz8WolZtkQLeFnAuvlrx2/n5bD6USJYD774gZ19Pq4iszzT
2GEDTmeBeasy4x/gAxPw7iwXgCFYALFBNW+kmN5Vp+gQwXPlyXa+dxE81Nq9iBx9
ni7hDCgub4pOuaBxWcpBt+0W4Va/Kz/IdQf1WwTt9LI6SlnTAAoeQqGR2ITPotk3
4xk6TbKwLVw=
=zJ0V
-----END PGP SIGNATURE-----

C
C
Christopher Baines wrote on 10 May 2023 05:47
(address . 63368@debbugs.gnu.org)
87jzxg1hd4.fsf@cbaines.net
Christopher Baines <mail@cbaines.net> writes:

Toggle quote (12 lines)
> Since the recent core-updates merge, I've seen the build coordinator
> using less memory, but it's also been crashing in a new way, up to 10
> times a day.
>
> In the log, you see something like:
>
> 2023-05-07 09:15:42 Signals delivery fails constantly at GC #71051
> 2023-05-07 09:15:42 Signals delivery fails constantly
>
> I'm guessing the switch from libgc-8.0.4 to libgc-8.2.2 has something to
> do with this.

I think I've found a workaround. I found a list of environment variables
[1] you can set to affect the GC behaviour, and the first one I tried
(GC_RETRY_SIGNALS=0) seems to have had the desired affect, in that the
crashes/restarts have stopped.


I've sent a patch [2] to apply this setting as part of the service.

-----BEGIN PGP SIGNATURE-----

iQKlBAEBCgCPFiEEPonu50WOcg2XVOCyXiijOwuE9XcFAmRbktdfFIAAAAAALgAo
aXNzdWVyLWZwckBub3RhdGlvbnMub3BlbnBncC5maWZ0aGhvcnNlbWFuLm5ldDNF
ODlFRUU3NDU4RTcyMEQ5NzU0RTBCMjVFMjhBMzNCMEI4NEY1NzcRHG1haWxAY2Jh
aW5lcy5uZXQACgkQXiijOwuE9XdYkg/8DpG7BBKKwfYX5PGHoSC0Y0AfIgVYW3Gc
AX22dRpHkKi4H0GEPLpG6SLRCP7koA8AiJrW2j7qnCARoKGibQZRN7HAL+LIkv3S
D5Cu8WkdOMRm4mYysFeCnsbUh5O1LsiBFLdMrjVtYLxZJpRH9RJqBUqjWo1Ovn7f
S7av+mE3m4jkcHpplJEaS5QJfKGSOml8bP8aM7D8LxbWen9H6bnnPpoTeo1NMPBE
X6NnGjC1IrjQRWZYjGzZzhk3ofaGyNi/nF+hLmHG7wltkyUHYhm5jvJXUhjawCwN
ppBQ4GX2dFyO8qS1MthHzWM9cv9YeUuhuQYX3hi+GKRVnlxVbcUAxhAbsmLD696K
fBEwweBz7Bbf4g2gK6D/grBMvgttPFXYelgOfvAMXBkHyt6U1DImgttKDnTVa8RX
1v/HHq/UXQ/CN+vOX9XCBIhIEVEjxEVxqk2ZQDR3bkihOdji4BBoPMXLpeGeRhIn
uUpxsXKZ7rMF2sfIgw7hxk7FLG043RRlLeUshtfWAZu/x0ZJ9bCYgo7HeyZcLJ7f
OkiTT2xggoYbHi9522D6R1yucn6tkJwmjPri5ojxJkYF6vxDyT6pbcem8X6hv8/y
E30KZKenxI+bfobxakb5AXjHvGdFCO1ega/FKdlPwOmaDHOcunakvHEFo8f6yBlC
SplyFZiUAEk=
=TM2C
-----END PGP SIGNATURE-----

L
L
Ludovic Courtès wrote on 25 May 2023 08:24
(name . Christopher Baines)(address . mail@cbaines.net)(address . 63368@debbugs.gnu.org)
87cz2ocu2f.fsf@gnu.org
Hi,

Christopher Baines <mail@cbaines.net> skribis:

Toggle quote (12 lines)
> Since the recent core-updates merge, I've seen the build coordinator
> using less memory, but it's also been crashing in a new way, up to 10
> times a day.
>
> In the log, you see something like:
>
> 2023-05-07 09:15:42 Signals delivery fails constantly at GC #71051
> 2023-05-07 09:15:42 Signals delivery fails constantly
>
> I'm guessing the switch from libgc-8.0.4 to libgc-8.2.2 has something to
> do with this.

Normally on GNU/Linux libgc has:

#define SIG_SUSPEND SIGPWR

The Coordinator fiddles with SIGALRM, SIGUSR1, SIGINT, and SIGPIPE,
which should normally be fine.

Is there anything else that might interfere with libgc?

Ludo’.
C
C
Christopher Baines wrote on 25 May 2023 08:26
(name . Ludovic Courtès)(address . ludo@gnu.org)(address . 63368@debbugs.gnu.org)
877csw4dvt.fsf@cbaines.net
Ludovic Courtès <ludo@gnu.org> writes:

Toggle quote (23 lines)
> Christopher Baines <mail@cbaines.net> skribis:
>
>> Since the recent core-updates merge, I've seen the build coordinator
>> using less memory, but it's also been crashing in a new way, up to 10
>> times a day.
>>
>> In the log, you see something like:
>>
>> 2023-05-07 09:15:42 Signals delivery fails constantly at GC #71051
>> 2023-05-07 09:15:42 Signals delivery fails constantly
>>
>> I'm guessing the switch from libgc-8.0.4 to libgc-8.2.2 has something to
>> do with this.
>
> Normally on GNU/Linux libgc has:
>
> #define SIG_SUSPEND SIGPWR
>
> The Coordinator fiddles with SIGALRM, SIGUSR1, SIGINT, and SIGPIPE,
> which should normally be fine.
>
> Is there anything else that might interfere with libgc?

I've seen this issue in both the build coordinator and nar-herder, both
of which use guile-sqlite, so I wonder if that could have something to
do with it.
-----BEGIN PGP SIGNATURE-----

iQKlBAEBCgCPFiEEPonu50WOcg2XVOCyXiijOwuE9XcFAmRvgbZfFIAAAAAALgAo
aXNzdWVyLWZwckBub3RhdGlvbnMub3BlbnBncC5maWZ0aGhvcnNlbWFuLm5ldDNF
ODlFRUU3NDU4RTcyMEQ5NzU0RTBCMjVFMjhBMzNCMEI4NEY1NzcRHG1haWxAY2Jh
aW5lcy5uZXQACgkQXiijOwuE9Xe9Ag/+LfCuSCqPT+J7Gvyj/ITn0dA+FbBUxLIT
iYf6oOhY87H2YvwaC/NtzzS17uIiPg3NE75XWFlIaV94E5oDX/YU1EavsuwZswwv
sGa0dN7cSkII994OMve8jOsJ9EELotwMyE6aPEwlPiuYCfydubhHfFrnNDr0ySrT
lW+TRWbB2TNSGDShfpwOOrTaAr8LZkyu86h3PIGLL+Huupu9YpTXH4x8f/OK+5A0
R22eRcniI87ayrdvIe+ByVbt2u907A2xC2M+IZezhkVbI80VTI33gObkKd57kLSP
pb0PfD4Wu9f3KTHoy5OjI7IS0V//IRBGxDqLF2njZFcIzdDkqUPYluO8p1w5d9UG
9ME5HItEgB6Ad9k6ml3AOOMe40/u3JMjt5zKHWbDPGUYEPlb8NK526sp03BgsfAo
aeeod7Gepycox+HwM63JnGAipxXk/DqXVUTeQzhWNLJpc1G+AvdoRZR2kSq4AgrH
rZQTiT9Mni50lQ+7Tax7s9F7X6xd2seAq8/GAkkYu+nFohQwpvtEPF+T1GGiafSL
BPl16R/lW+j2LPOLbDvfNE9L1iSJuib4gBkFD8B3B06w5m2j12dlmj1t6Rh3eBVe
kCdgK5qQZiPKAJD0oDki5pQepZT8EC7eGKqLnW7x631J3vJMCafyEhzlONcr5UlB
Ji7fCv/vrmc=
=dr0R
-----END PGP SIGNATURE-----

C
C
Christopher Baines wrote on 2 Jun 2023 10:07
(address . 63368@debbugs.gnu.org)(name . Ludovic Courtès)(address . ludo@gnu.org)
871qit7pq7.fsf@cbaines.net
Christopher Baines <mail@cbaines.net> writes:

Toggle quote (29 lines)
> Ludovic Courtès <ludo@gnu.org> writes:
>
>> Christopher Baines <mail@cbaines.net> skribis:
>>
>>> Since the recent core-updates merge, I've seen the build coordinator
>>> using less memory, but it's also been crashing in a new way, up to 10
>>> times a day.
>>>
>>> In the log, you see something like:
>>>
>>> 2023-05-07 09:15:42 Signals delivery fails constantly at GC #71051
>>> 2023-05-07 09:15:42 Signals delivery fails constantly
>>>
>>> I'm guessing the switch from libgc-8.0.4 to libgc-8.2.2 has something to
>>> do with this.
>>
>> Normally on GNU/Linux libgc has:
>>
>> #define SIG_SUSPEND SIGPWR
>>
>> The Coordinator fiddles with SIGALRM, SIGUSR1, SIGINT, and SIGPIPE,
>> which should normally be fine.
>>
>> Is there anything else that might interfere with libgc?
>
> I've seen this issue in both the build coordinator and nar-herder, both
> of which use guile-sqlite, so I wonder if that could have something to
> do with it.

I've seen this happen with the build coordinator agent now (on
milano-guix-1):

2023-06-02 18:59:55 2023-06-02 18:59:55 (DEBUG): fb9f06cf-cc1d-4493-88b8-3eac9437f5d4: checking the availability of build inputs
2023-06-02 18:59:55 2023-06-02 18:59:55 (INFO ): fb9f06cf-cc1d-4493-88b8-3eac9437f5d4: setup successful, building: /gnu/store/7fbrli2a8nzn676q8gz2b0i0y0lr9nxv-r-quasr-1.40.0.drv
2023-06-02 19:00:46 Signals delivery fails constantly at GC #55
2023-06-02 19:01:22 Signals delivery fails constantly
2023-06-02 19:01:29 locale is en_US.utf8
2023-06-02 19:01:29 (gnutls version: 3.7.7, guix version: 1.4.0-6.dc5430c)

Which is a bit more concerning, since the build coordinator agent is
intentionally quite simple (no SQLite for example).
-----BEGIN PGP SIGNATURE-----

iQKlBAEBCgCPFiEEPonu50WOcg2XVOCyXiijOwuE9XcFAmR6IwBfFIAAAAAALgAo
aXNzdWVyLWZwckBub3RhdGlvbnMub3BlbnBncC5maWZ0aGhvcnNlbWFuLm5ldDNF
ODlFRUU3NDU4RTcyMEQ5NzU0RTBCMjVFMjhBMzNCMEI4NEY1NzcRHG1haWxAY2Jh
aW5lcy5uZXQACgkQXiijOwuE9XfoWg/9GKzwsXsuotKjopavcBGMcbA0Tit6B8is
UtpHVGqlPiswgJHzPR/Cpb06ILAwN7CV89FNwLQe+CIZvoRkb693ZRPaqXQh67TO
dS9v+TLmnp7iCVrwmxX8ji5OunuXcKB39vdA42C1S9i4hcZvbNAK+TILLUFKj+w3
UGE/cjJddm7Y+yJenUjz27Up1PfMa412mplWq9cgY0y4Y2P20/rCxTQIhp5Ej/8w
tWeJwUhSjWIkW0FstJXuEzr6kfVdyghOtRdW+mHZDbfar2Xs0orANXfxhNb+u9Gw
nN0/oZHFJb0X4iHtBDM8KYuSsJeZEMfr1nIERdNSEGjIDOdTq+TLyjPiZB9a9lBz
yyq2rJCwHKA+yGv/EfvrgUhw6CBTHjcMv3nHUbpFSNP2jm8cV/QXELgQr9HkHzCc
eHRlEu1tpzVTTOtRpDhk5FH/s/oogvJq4pawvzAoaWI+aWeLEzYmt122hyXtqS4P
4Cs2VRlCW2BLe3YLkq4TNbg+9N4puTdS7mHvBxZeHO9k7ANQgCqqLJof06M3zZ0Z
JkXX2Qur1Hwad/1/h2YzNNIJCNgqCrgn5qCFhkkz2oTh/hGec2rShpl1XryGVHAS
uL5bHdz5e52yhwTexWAbRtNNr23UzgteeE8llQ+jJmxSlb5W0pTw4VFd9/LGHCEF
zRl9pn4Oux8=
=YlHb
-----END PGP SIGNATURE-----

L
L
Ludovic Courtès wrote on 6 Jun 2023 08:09
(name . Christopher Baines)(address . mail@cbaines.net)(address . 63368@debbugs.gnu.org)
8735348w6o.fsf@gnu.org
Christopher Baines <mail@cbaines.net> skribis:

Toggle quote (13 lines)
> I've seen this happen with the build coordinator agent now (on
> milano-guix-1):
>
> 2023-06-02 18:59:55 2023-06-02 18:59:55 (DEBUG): fb9f06cf-cc1d-4493-88b8-3eac9437f5d4: checking the availability of build inputs
> 2023-06-02 18:59:55 2023-06-02 18:59:55 (INFO ): fb9f06cf-cc1d-4493-88b8-3eac9437f5d4: setup successful, building: /gnu/store/7fbrli2a8nzn676q8gz2b0i0y0lr9nxv-r-quasr-1.40.0.drv
> 2023-06-02 19:00:46 Signals delivery fails constantly at GC #55
> 2023-06-02 19:01:22 Signals delivery fails constantly
> 2023-06-02 19:01:29 locale is en_US.utf8
> 2023-06-02 19:01:29 (gnutls version: 3.7.7, guix version: 1.4.0-6.dc5430c)
>
> Which is a bit more concerning, since the build coordinator agent is
> intentionally quite simple (no SQLite for example).

The closure of (guix-build-coordinator agent) seems to be quite large
still.

Could you check what .so files are loaded by that code, perhaps via
/proc/PID/maps?

Thanks,
Ludo’.
C
C
Christopher Baines wrote on 6 Jun 2023 08:19
(name . Ludovic Courtès)(address . ludo@gnu.org)(address . 63368@debbugs.gnu.org)
87wn0g1utn.fsf@cbaines.net
Ludovic Courtès <ludo@gnu.org> writes:

Toggle quote (25 lines)
> Christopher Baines <mail@cbaines.net> skribis:
>
>> I've seen this happen with the build coordinator agent now (on
>> milano-guix-1):
>>
>> 2023-06-02 18:59:55 2023-06-02 18:59:55 (DEBUG):
>> fb9f06cf-cc1d-4493-88b8-3eac9437f5d4: checking the availability of
>> build inputs
>> 2023-06-02 18:59:55 2023-06-02 18:59:55 (INFO ):
>> fb9f06cf-cc1d-4493-88b8-3eac9437f5d4: setup successful, building:
>> /gnu/store/7fbrli2a8nzn676q8gz2b0i0y0lr9nxv-r-quasr-1.40.0.drv
>> 2023-06-02 19:00:46 Signals delivery fails constantly at GC #55
>> 2023-06-02 19:01:22 Signals delivery fails constantly
>> 2023-06-02 19:01:29 locale is en_US.utf8
>> 2023-06-02 19:01:29 (gnutls version: 3.7.7, guix version: 1.4.0-6.dc5430c)
>>
>> Which is a bit more concerning, since the build coordinator agent is
>> intentionally quite simple (no SQLite for example).
>
> The closure of (guix-build-coordinator agent) seems to be quite large
> still.
>
> Could you check what .so files are loaded by that code, perhaps via
> /proc/PID/maps?

I think I see these (that's on milano-guix-1 currently):

/gnu/store/0i81lpfnn05pmjc5f43q4nfvd27r08f7-guile-gnutls-3.7.12/lib/guile/3.0/extensions/guile-gnutls-v-2.so.0.0.0
/gnu/store/0jk7sl5xqwwdkzjpp9sxgz9z0d48a3vy-libunistring-1.0/lib/libunistring.so.2.2.0
/gnu/store/1r1azdi4hvfypnx14d01n60p4aa7g2im-libidn2-2.3.4/lib/libidn2.so.0.3.8
/gnu/store/1w1r6r56z9lhg8ghcb7lxss6mkn7d5l1-libgc-8.2.2/lib/libgc.so.1.5.1
/gnu/store/4gvgcfdiz67wv04ihqfa8pqwzsb0qpv5-guile-3.0.9/lib/libguile-3.0.so.1.6.0
/gnu/store/8y0pwifz8a3d7zbdfzsawa1amf4afx1s-libgcrypt-1.10.1/lib/libgcrypt.so.20.4.1
/gnu/store/930nwsiysdvy2x5zv1sf6v7ym75z8ayk-gcc-11.3.0-lib/lib/libgcc_s.so.1
/gnu/store/c2fx42ial6lr60s96xcbml5hd8vwaxq3-nettle-3.8.1/lib/libhogweed.so.6.6
/gnu/store/c2fx42ial6lr60s96xcbml5hd8vwaxq3-nettle-3.8.1/lib/libnettle.so.8.6
/gnu/store/gsjczqir1wbz8p770zndrpw4rnppmxi3-glibc-2.35/lib/ld-linux-x86-64.so.2
/gnu/store/gsjczqir1wbz8p770zndrpw4rnppmxi3-glibc-2.35/lib/libcrypt.so.1
/gnu/store/gsjczqir1wbz8p770zndrpw4rnppmxi3-glibc-2.35/lib/libc.so.6
/gnu/store/gsjczqir1wbz8p770zndrpw4rnppmxi3-glibc-2.35/lib/libm.so.6
/gnu/store/ib2n2vzqpchc3bhh9i712w5sq9zapn8d-gmp-6.2.1/lib/libgmp.so.10.4.1
/gnu/store/j5kzdjan6mnf2ngmkc50fia8vrbpqi9b-libtasn1-4.19.0/lib/libtasn1.so.6.6.3
/gnu/store/k0p01a6b7hsxjfr65ga4f2gh6lh92aiq-lzlib-1.13/lib/liblz.so.1.13
/gnu/store/m9wi9hcrf7f9dm4ri32vw1jrbh1csywi-libgpg-error-1.45/lib/libgpg-error.so.0.33.0
/gnu/store/slzq3zqwj75lbrg4ly51hfhbv2vhryv5-zlib-1.2.13/lib/libz.so.1.2.13
/gnu/store/vq7dxp5la2lnhsvniwv38j0ggvsmzim7-p11-kit-0.24.1/lib/libp11-kit.so.0.3.0
/gnu/store/w8b0l8hk6g0fahj4fvmc4qqm3cvaxnmv-libffi-3.4.4/lib/libffi.so.8.1.2
/gnu/store/yr4lbvdyc4dgs76yij1dw2w2z8s84af8-gnutls-3.7.7/lib/libgnutls.so.30.34.1
-----BEGIN PGP SIGNATURE-----

iQKlBAEBCgCPFiEEPonu50WOcg2XVOCyXiijOwuE9XcFAmR/TrRfFIAAAAAALgAo
aXNzdWVyLWZwckBub3RhdGlvbnMub3BlbnBncC5maWZ0aGhvcnNlbWFuLm5ldDNF
ODlFRUU3NDU4RTcyMEQ5NzU0RTBCMjVFMjhBMzNCMEI4NEY1NzcRHG1haWxAY2Jh
aW5lcy5uZXQACgkQXiijOwuE9Xdm9g//TlHIgMkENeno3k4l5r3jYjN0JozNN4hq
wc7pCMmt1W6Sruqpl2xtUZFfxHiDqFlgIHv1vCv59xgbupWEFbi1p+c7GEAkQdPZ
vmQiUHPqr1LU87AdPkdXcT5BQBqNCQkQwRveassEGrscK1wUZ21hy4KDQYnBqOKN
kghFvNSjcleiYUpqFKPjqx3GUbluBagluD7riMoGilQcJ3DWSmIiCqU0AK4dgpib
SAnYGbjhUMHlQIzgHx9VvVJ5D0h8ONVyLJsnCfaRW+8PQ5vAqJhMbYsvrg+dW+Jq
KJkIra5QKF3kUKbfMAtiK4GYzw46g3MKqyUc8a6Zkr/pSeUXnifijy5RArH8u5Xd
U7hUFTbfZiKO8DpMlGBH35nZte2qCpI4GmvhBvllw1hDbxpLq0B2stxqi5+l9eNc
BFHsNO24Qg1ah213ttruJKjbn/dI6kl4syucqqxLqa/pcBG0zNkN9i7hVck8Fx1S
VaizOmHt+n/pYCP6mW2YidjI/C5hGvSdJzA2i8qEI8oFOx8hd4KrfWqcvpkIWFxN
eXYqlAVFYCpROCBG6eNdVzlP2JLF2htn213okRL4J+lKcky68OfYVmvcLYistGZt
dwYVLr7aF8Bl+kwygvmBkuZPwoAnicuTFSl5G9sZUpzuPC34ewolbj+qxGbl9zMx
glW2yrTNgcQ=
=KiIx
-----END PGP SIGNATURE-----

L
L
Ludovic Courtès wrote on 9 Jun 2023 06:14
(name . Christopher Baines)(address . mail@cbaines.net)(address . 63368@debbugs.gnu.org)
87pm64wzf5.fsf@gnu.org
Christopher Baines <mail@cbaines.net> skribis:

Toggle quote (4 lines)
> Ludovic Courtès <ludo@gnu.org> writes:
>
>> Christopher Baines <mail@cbaines.net> skribis:

[...]

Toggle quote (38 lines)
>>> 2023-06-02 19:01:22 Signals delivery fails constantly
>>> 2023-06-02 19:01:29 locale is en_US.utf8
>>> 2023-06-02 19:01:29 (gnutls version: 3.7.7, guix version: 1.4.0-6.dc5430c)
>>>
>>> Which is a bit more concerning, since the build coordinator agent is
>>> intentionally quite simple (no SQLite for example).
>>
>> The closure of (guix-build-coordinator agent) seems to be quite large
>> still.
>>
>> Could you check what .so files are loaded by that code, perhaps via
>> /proc/PID/maps?
>
> I think I see these (that's on milano-guix-1 currently):
>
> /gnu/store/0i81lpfnn05pmjc5f43q4nfvd27r08f7-guile-gnutls-3.7.12/lib/guile/3.0/extensions/guile-gnutls-v-2.so.0.0.0
> /gnu/store/0jk7sl5xqwwdkzjpp9sxgz9z0d48a3vy-libunistring-1.0/lib/libunistring.so.2.2.0
> /gnu/store/1r1azdi4hvfypnx14d01n60p4aa7g2im-libidn2-2.3.4/lib/libidn2.so.0.3.8
> /gnu/store/1w1r6r56z9lhg8ghcb7lxss6mkn7d5l1-libgc-8.2.2/lib/libgc.so.1.5.1
> /gnu/store/4gvgcfdiz67wv04ihqfa8pqwzsb0qpv5-guile-3.0.9/lib/libguile-3.0.so.1.6.0
> /gnu/store/8y0pwifz8a3d7zbdfzsawa1amf4afx1s-libgcrypt-1.10.1/lib/libgcrypt.so.20.4.1
> /gnu/store/930nwsiysdvy2x5zv1sf6v7ym75z8ayk-gcc-11.3.0-lib/lib/libgcc_s.so.1
> /gnu/store/c2fx42ial6lr60s96xcbml5hd8vwaxq3-nettle-3.8.1/lib/libhogweed.so.6.6
> /gnu/store/c2fx42ial6lr60s96xcbml5hd8vwaxq3-nettle-3.8.1/lib/libnettle.so.8.6
> /gnu/store/gsjczqir1wbz8p770zndrpw4rnppmxi3-glibc-2.35/lib/ld-linux-x86-64.so.2
> /gnu/store/gsjczqir1wbz8p770zndrpw4rnppmxi3-glibc-2.35/lib/libcrypt.so.1
> /gnu/store/gsjczqir1wbz8p770zndrpw4rnppmxi3-glibc-2.35/lib/libc.so.6
> /gnu/store/gsjczqir1wbz8p770zndrpw4rnppmxi3-glibc-2.35/lib/libm.so.6
> /gnu/store/ib2n2vzqpchc3bhh9i712w5sq9zapn8d-gmp-6.2.1/lib/libgmp.so.10.4.1
> /gnu/store/j5kzdjan6mnf2ngmkc50fia8vrbpqi9b-libtasn1-4.19.0/lib/libtasn1.so.6.6.3
> /gnu/store/k0p01a6b7hsxjfr65ga4f2gh6lh92aiq-lzlib-1.13/lib/liblz.so.1.13
> /gnu/store/m9wi9hcrf7f9dm4ri32vw1jrbh1csywi-libgpg-error-1.45/lib/libgpg-error.so.0.33.0
> /gnu/store/slzq3zqwj75lbrg4ly51hfhbv2vhryv5-zlib-1.2.13/lib/libz.so.1.2.13
> /gnu/store/vq7dxp5la2lnhsvniwv38j0ggvsmzim7-p11-kit-0.24.1/lib/libp11-kit.so.0.3.0
> /gnu/store/w8b0l8hk6g0fahj4fvmc4qqm3cvaxnmv-libffi-3.4.4/lib/libffi.so.8.1.2
> /gnu/store/yr4lbvdyc4dgs76yij1dw2w2z8s84af8-gnutls-3.7.7/lib/libgnutls.so.30.34.1


Hmm no idea. I’ve never seen “Signals delivery fails” before so I
really wonder what could be causing this. Would be great if you could
come up with a reduced test case, but I guess that won’t be easy.

Or perhaps you could run a Coordinator agent under ‘strace -f’ to see if
we get hints?

Ludo’.
L
L
Ludovic Courtès wrote on 1 Dec 06:23 -0800
control message for bug #63368
(address . control@debbugs.gnu.org)
87jzcjtrc5.fsf@gnu.org
severity 63368 important
quit
L
L
Ludovic Courtès wrote on 1 Dec 06:26 -0800
Re: bug#63368: Build coordiantor "Signals delivery fails constantly" crashes
(name . Christopher Baines)(address . mail@cbaines.net)(address . 63368@debbugs.gnu.org)
87cyibtr6i.fsf@gnu.org
Christopher Baines <mail@cbaines.net> skribis:

Toggle quote (9 lines)
> Since the recent core-updates merge, I've seen the build coordinator
> using less memory, but it's also been crashing in a new way, up to 10
> times a day.
>
> In the log, you see something like:
>
> 2023-05-07 09:15:42 Signals delivery fails constantly at GC #71051
> 2023-05-07 09:15:42 Signals delivery fails constantly

Same with ‘guix publish’: https://issues.guix.gnu.org/74632

Toggle quote (3 lines)
> I'm guessing the switch from libgc-8.0.4 to libgc-8.2.2 has something to
> do with this.

I’m not sure when these started to happen for ‘guix publish’.

Data point: the ‘guix publish’ instance at guix.bordeaux.inria.fr never
encountered this problem. The main difference compared to ci.guix is
that it does not produce lzip archives. (I see the Coordinator uses
Guile-Lzlib; maybe that’s a lead.)

Ludo’.
?
Your comment

Commenting via the web interface is currently disabled.

To comment on this conversation send an email to 63368@patchwise.org

To respond to this issue using the mumi CLI, first switch to it
mumi current 63368
Then, you may apply the latest patchset in this issue (with sign off)
mumi am -- -s
Or, compose a reply to this issue
mumi compose
Or, send patches to this issue
mumi send-email *.patch