GNU bug report logs

#32182 Login fail after core-update without reboot

PackageSource(s)Maintainer(s)
guix PTS Buildd Popcon
Reply or subscribe to this bug. View this bug as an mbox, status mbox, or maintainer mbox

Report forwarded to bug-guix@gnu.org:
bug#32182; Package guix. (Tue, 17 Jul 2018 08:31:02 GMT) (full text, mbox, link).


Acknowledgement sent to Pierre-Antoine Rouby <pierre-antoine.rouby@inria.fr>:
New bug report received and forwarded. Copy sent to bug-guix@gnu.org. (Tue, 17 Jul 2018 08:31:02 GMT) (full text, mbox, link).


Message #5 received at submit@debbugs.gnu.org (full text, mbox, reply):

From: Pierre-Antoine Rouby <pierre-antoine.rouby@inria.fr>
To: bug-guix@gnu.org
Subject: Login fail after core-update without reboot
Date: Tue, 17 Jul 2018 10:30:08 +0200 (CEST)
Hi Guix,

I found a problem with 'guix reconfigure' and core-update. After
reconfigure it's impossible to connect in tty, 'login' segfault
with this error:

----------------------------------------------------------------------
login[30083]: segfault at 968 ip 00007f6ae6168ec8 sp 00007ffc7bd0f420 error 4 in libpthread-2.27.so[7f6ae6163000+19000]
----------------------------------------------------------------------

I think login try to use glibc-2.27 but it's still configured to use
glib-2.26. It's possible this issue come from '/etc/pam.d/login'.

A had to reboot my system.

gdb trace:
----------------------------------------------------------------------
process 24717 is executing new program: /gnu/store/31qbd404pmlm5bmb0l0r147mnjxzpq3y-shadow-4.6/bin/login
warning: Could not load shared library symbols for linux-vdso.so.1.
Do you need "set solib-search-path" or "set sysroot"?
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/gnu/store/l4lr0f5cjd0nbsaaf8b5dmcw1a1yypr3-glibc-2.27/lib/libthread_db.so.1".

Program received signal SIGSEGV, Segmentation fault.
0x00007fb95cabaec8 in __pthread_initialize_minimal_internal () from /gnu/store/l4lr0f5cjd0nbsaaf8b5dmcw1a1yypr3-glibc-2.27/lib/libpthread.so.0
(gdb) bt
#0  0x00007fb95cabaec8 in __pthread_initialize_minimal_internal () from /gnu/store/l4lr0f5cjd0nbsaaf8b5dmcw1a1yypr3-glibc-2.27/lib/libpthread.so.0
#1  0x00007fb95caba621 in _init () from /gnu/store/l4lr0f5cjd0nbsaaf8b5dmcw1a1yypr3-glibc-2.27/lib/libpthread.so.0
#2  0x00007fb95d8dcaa0 in ?? () from /gnu/store/gwyb3679v49ljisgkvzay2xa3njgq4ii-linux-pam-1.3.0/lib/security/pam_env.so
#3  0x00007fb95eb1f33a in call_init.part () from /gnu/store/4sqaib7c2dfjv62ivrg9b8wa7bh226la-glibc-2.26.105-g0890d5379c/lib/ld-linux-x86-64.so.2
#4  0x00007fb95eb1f4f5 in _dl_init () from /gnu/store/4sqaib7c2dfjv62ivrg9b8wa7bh226la-glibc-2.26.105-g0890d5379c/lib/ld-linux-x86-64.so.2
#5  0x00007fb95eb23980 in dl_open_worker () from /gnu/store/4sqaib7c2dfjv62ivrg9b8wa7bh226la-glibc-2.26.105-g0890d5379c/lib/ld-linux-x86-64.so.2
#6  0x00007fb95e058901 in _dl_catch_error () from /gnu/store/4sqaib7c2dfjv62ivrg9b8wa7bh226la-glibc-2.26.105-g0890d5379c/lib/libc.so.6
#7  0x00007fb95eb23127 in _dl_open () from /gnu/store/4sqaib7c2dfjv62ivrg9b8wa7bh226la-glibc-2.26.105-g0890d5379c/lib/ld-linux-x86-64.so.2
#8  0x00007fb95e4f9f96 in dlopen_doit () from /gnu/store/4sqaib7c2dfjv62ivrg9b8wa7bh226la-glibc-2.26.105-g0890d5379c/lib/libdl.so.2
#9  0x00007fb95e058901 in _dl_catch_error () from /gnu/store/4sqaib7c2dfjv62ivrg9b8wa7bh226la-glibc-2.26.105-g0890d5379c/lib/libc.so.6
#10 0x00007fb95e4fa5a9 in _dlerror_run () from /gnu/store/4sqaib7c2dfjv62ivrg9b8wa7bh226la-glibc-2.26.105-g0890d5379c/lib/libdl.so.2
#11 0x00007fb95e4fa021 in dlopen@@GLIBC_2.2.5 () from /gnu/store/4sqaib7c2dfjv62ivrg9b8wa7bh226la-glibc-2.26.105-g0890d5379c/lib/libdl.so.2
#12 0x00007fb95e701f4d in _pam_load_module () from /gnu/store/gwyb3679v49ljisgkvzay2xa3njgq4ii-linux-pam-1.3.0/lib/libpam.so.0
#13 0x00007fb95e7025d9 in _pam_add_handler () from /gnu/store/gwyb3679v49ljisgkvzay2xa3njgq4ii-linux-pam-1.3.0/lib/libpam.so.0
#14 0x00007fb95e702cd6 in _pam_parse_conf_file () from /gnu/store/gwyb3679v49ljisgkvzay2xa3njgq4ii-linux-pam-1.3.0/lib/libpam.so.0
#15 0x00007fb95e7033d7 in _pam_init_handlers () from /gnu/store/gwyb3679v49ljisgkvzay2xa3njgq4ii-linux-pam-1.3.0/lib/libpam.so.0
#16 0x00007fb95e704bc1 in pam_start () from /gnu/store/gwyb3679v49ljisgkvzay2xa3njgq4ii-linux-pam-1.3.0/lib/libpam.so.0
#17 0x0000000000402f2c in main ()
----------------------------------------------------------------------

----------------------------------------------------------------------
(gdb) info sharedlibrary 
From                To                  Syms Read   Shared Object Library
0x00007fb95eb10cc0  0x00007fb95eb2c990  Yes (*)     /gnu/store/4sqaib7c2dfjv62ivrg9b8wa7bh226la-glibc-2.26.105-g0890d5379c/lib/ld-linux-x86-64.so.2
                                        No          linux-vdso.so.1
0x00007fb95e90d1b0  0x00007fb95e90de59  Yes (*)     /gnu/store/gwyb3679v49ljisgkvzay2xa3njgq4ii-linux-pam-1.3.0/lib/libpam_misc.so.0
0x00007fb95e6ff9d0  0x00007fb95e706ae5  Yes (*)     /gnu/store/gwyb3679v49ljisgkvzay2xa3njgq4ii-linux-pam-1.3.0/lib/libpam.so.0
0x00007fb95e4f9de0  0x00007fb95e4faa27  Yes (*)     /gnu/store/4sqaib7c2dfjv62ivrg9b8wa7bh226la-glibc-2.26.105-g0890d5379c/lib/libdl.so.2
0x00007fb95e2e4a40  0x00007fb95e2f4775  Yes (*)     /gnu/store/2ifmksc425qcysl5rkxkbv6yrgc1w9cs-gcc-5.5.0-lib/lib/libgcc_s.so.1
0x00007fb95df50750  0x00007fb95e088fac  Yes (*)     /gnu/store/4sqaib7c2dfjv62ivrg9b8wa7bh226la-glibc-2.26.105-g0890d5379c/lib/libc.so.6
0x00007fb95dd1a6a0  0x00007fb95dd20af8  Yes (*)     /gnu/store/gwyb3679v49ljisgkvzay2xa3njgq4ii-linux-pam-1.3.0/lib/security/pam_unix.so
0x00007fb95dae0ba0  0x00007fb95dae5f47  Yes (*)     /gnu/store/4sqaib7c2dfjv62ivrg9b8wa7bh226la-glibc-2.26.105-g0890d5379c/lib/libcrypt.so.1
0x00007fb95cabab40  0x00007fb95cac8657  Yes (*)     /gnu/store/l4lr0f5cjd0nbsaaf8b5dmcw1a1yypr3-glibc-2.27/lib/libpthread.so.0
0x00007fb95d8dcf30  0x00007fb95d8de421  Yes (*)     /gnu/store/gwyb3679v49ljisgkvzay2xa3njgq4ii-linux-pam-1.3.0/lib/security/pam_env.so
(*): Shared library is missing debugging information.
----------------------------------------------------------------------

--
Pierre-Antoine Rouby




Information forwarded to bug-guix@gnu.org:
bug#32182; Package guix. (Mon, 23 Jul 2018 13:18:01 GMT) (full text, mbox, link).


Message #8 received at 32182@debbugs.gnu.org (full text, mbox, reply):

From: ludo@gnu.org (Ludovic Courtès)
To: Pierre-Antoine Rouby <pierre-antoine.rouby@inria.fr>
Cc: 32182@debbugs.gnu.org
Subject: Re: bug#32182: Login fail after core-update without reboot
Date: Mon, 23 Jul 2018 15:17:07 +0200
Hello!

Pierre-Antoine Rouby <pierre-antoine.rouby@inria.fr> skribis:

> I think login try to use glibc-2.27 but it's still configured to use
> glib-2.26. It's possible this issue come from '/etc/pam.d/login'.

Indeed.  The problem here is that ‘reconfigure’ updates /etc/pam.d, but
does not change the service definition of ‘login’, etc.  Thus, when
‘login’ restarts, it reads the new /etc/pam.d/login, which contains a
line like:

  session required /gnu/store/…-elogind-232.4/lib/security/pam_elogind.so 

Consequently, ‘login’ dlopens pam_elogind.so, which is linked against
the new libc, which eventually causes it to crash.

It’s a real issue on headless servers because you could lock yourself
out (‘sshd’ could have the same problem.)

I can think of several solutions:

  1. Arrange for services to refer to /gnu/store/…-pam.d instead of
     /etc/pam.d.  This can maybe be achieved by modifying PAM such that
     these applications honor $PAM_DIRECTORY or something like that.

  2. Add support for “service chain-loading” in the Shepherd and/or
     GuixSD.  The idea is that, for services that cannot be restarted
     right away because they are currently running, register code to
     upgrade the service next time it is restarted (see
     <https://bugs.gnu.org/30706>).  That way, when ‘login’ restarts
     after ‘reconfigure’, it’s the new ‘login’ service that would be
     restarted.

Thoughts?

Ludo’.




Severity set to 'important' from 'normal' Request was from ludo@gnu.org (Ludovic Courtès) to control@debbugs.gnu.org. (Sat, 08 Sep 2018 21:06:02 GMT) (full text, mbox, link).


Information forwarded to bug-guix@gnu.org:
bug#32182; Package guix. (Sat, 02 May 2020 14:38:01 GMT) (full text, mbox, link).


Message #13 received at 32182@debbugs.gnu.org (full text, mbox, reply):

From: Ludovic Courtès <ludo@gnu.org>
To: Pierre-Antoine Rouby <pierre-antoine.rouby@inria.fr>
Cc: 32182@debbugs.gnu.org
Subject: Re: bug#32182: Login fail after core-update without reboot
Date: Sat, 02 May 2020 16:37:48 +0200
Hi, old bug!  :-)

ludo@gnu.org (Ludovic Courtès) skribis:

> I can think of several solutions:
>
>   1. Arrange for services to refer to /gnu/store/…-pam.d instead of
>      /etc/pam.d.  This can maybe be achieved by modifying PAM such that
>      these applications honor $PAM_DIRECTORY or something like that.

We should look into that.

>   2. Add support for “service chain-loading” in the Shepherd and/or
>      GuixSD.  The idea is that, for services that cannot be restarted
>      right away because they are currently running, register code to
>      upgrade the service next time it is restarted (see
>      <https://bugs.gnu.org/30706>).  That way, when ‘login’ restarts
>      after ‘reconfigure’, it’s the new ‘login’ service that would be
>      restarted.

That bit was implemented long ago with Shepherd service replacements.
So at least, now, one can run ‘herd start term-tty1’ or similar to get a
working login:

  https://lists.gnu.org/archive/html/guix-devel/2020-04/msg00496.html

Ludo’.




Information forwarded to bug-guix@gnu.org:
bug#32182; Package guix. (Thu, 16 Dec 2021 15:57:02 GMT) (full text, mbox, link).


Message #16 received at 32182@debbugs.gnu.org (full text, mbox, reply):

From: Maxim Cournoyer <maxim.cournoyer@gmail.com>
To: Ludovic Courtès <ludo@gnu.org>
Cc: 32182@debbugs.gnu.org, Pierre-Antoine Rouby <pierre-antoine.rouby@inria.fr>
Subject: Re: bug#32182: Login fail after core-update without reboot
Date: Thu, 16 Dec 2021 10:56:06 -0500
Hello,

Ludovic Courtès <ludo@gnu.org> writes:

[...]

>> I can think of several solutions:
>>
>>   1. Arrange for services to refer to /gnu/store/…-pam.d instead of
>>      /etc/pam.d.  This can maybe be achieved by modifying PAM such that
>>      these applications honor $PAM_DIRECTORY or something like that.
>>
>>   2. Add support for “service chain-loading” in the Shepherd and/or
>>      GuixSD.  The idea is that, for services that cannot be restarted
>>      right away because they are currently running, register code to
>>      upgrade the service next time it is restarted (see
>>      <https://bugs.gnu.org/30706>).  That way, when ‘login’ restarts
>>      after ‘reconfigure’, it’s the new ‘login’ service that would be
>>      restarted.
>
> That bit was implemented long ago with Shepherd service replacements.
> So at least, now, one can run ‘herd start term-tty1’ or similar to get a
> working login:

Point 2 doesn't seem to help in working around or fixing the related
#52533 though, correct?  Restarting the remote elogind or even
ssh-daemon doesn't work there, perhaps because 'guix deploy' wasn't able
to complete in the first place.

I guess that means we should look into fixing point 1., as you already
suggested.  On top of that I'd propose disabling PAM unless there's a
good reason to have it on by default; as I wrote in the other issue,
`man sshd_config' documents that by default in OpenSSH it is disabled.

Thanks,

Maxim




Information forwarded to bug-guix@gnu.org:
bug#32182; Package guix. (Thu, 16 Dec 2021 16:16:02 GMT) (full text, mbox, link).


Message #19 received at 32182@debbugs.gnu.org (full text, mbox, reply):

From: Maxim Cournoyer <maxim.cournoyer@gmail.com>
To: Ludovic Courtès <ludo@gnu.org>
Cc: 32182@debbugs.gnu.org, Pierre-Antoine Rouby <pierre-antoine.rouby@inria.fr>
Subject: Re: bug#32182: Login fail after core-update without reboot
Date: Thu, 16 Dec 2021 11:15:44 -0500
Hi again,

Maxim Cournoyer <maxim.cournoyer@gmail.com> writes:

> Hello,
>
> Ludovic Courtès <ludo@gnu.org> writes:
>
> [...]
>
>>> I can think of several solutions:
>>>
>>>   1. Arrange for services to refer to /gnu/store/…-pam.d instead of
>>>      /etc/pam.d.  This can maybe be achieved by modifying PAM such that
>>>      these applications honor $PAM_DIRECTORY or something like that.
>>>
>>>   2. Add support for “service chain-loading” in the Shepherd and/or
>>>      GuixSD.  The idea is that, for services that cannot be restarted
>>>      right away because they are currently running, register code to
>>>      upgrade the service next time it is restarted (see
>>>      <https://bugs.gnu.org/30706>).  That way, when ‘login’ restarts
>>>      after ‘reconfigure’, it’s the new ‘login’ service that would be
>>>      restarted.
>>
>> That bit was implemented long ago with Shepherd service replacements.
>> So at least, now, one can run ‘herd start term-tty1’ or similar to get a
>> working login:
>
> Point 2 doesn't seem to help in working around or fixing the related
> #52533 though, correct?  Restarting the remote elogind or even
> ssh-daemon doesn't work there, perhaps because 'guix deploy' wasn't able
> to complete in the first place.

Another bit that probably played a role here: the above failure to
complete is perhaps caused/made worst by #41238 (guix deploy close ssh
session after each store items sent), which doesn't reuse the same
stable SSH session to do the whole of what it needs to do.

Maxim




Send a report that this bug log contains spam.


debbugs.gnu.org maintainers <help-debbugs@gnu.org>. Last modified: Fri Jan 3 06:40:24 2025; Machine Name: wallace-server

GNU bug tracking system

Debbugs is free software and licensed under the terms of the GNU Public License version 2. The current version can be obtained from https://bugs.debian.org/debbugs-source/.

Copyright © 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson, 2005-2017 Don Armstrong, and many other contributors.