GNU bug report logs

#59365 make-dynamic-linker-cache OOMs for LLVM 15 on i686-linux

PackageSource(s)Maintainer(s)
guix PTS Buildd Popcon
Full log

Message #11 received at 59365@debbugs.gnu.org (full text, mbox, reply):

Received: (at 59365) by debbugs.gnu.org; 22 Nov 2022 10:22:58 +0000
From debbugs-submit-bounces@debbugs.gnu.org Tue Nov 22 05:22:58 2022
Received: from localhost ([127.0.0.1]:49921 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.84_2)
	(envelope-from <debbugs-submit-bounces@debbugs.gnu.org>)
	id 1oxQQc-0008Ga-6e
	for submit@debbugs.gnu.org; Tue, 22 Nov 2022 05:22:58 -0500
Received: from mail-qk1-f170.google.com ([209.85.222.170]:36670)
 by debbugs.gnu.org with esmtp (Exim 4.84_2)
 (envelope-from <maxim.cournoyer@gmail.com>) id 1oxQQZ-0008GL-Gc
 for 59365@debbugs.gnu.org; Tue, 22 Nov 2022 05:22:56 -0500
Received: by mail-qk1-f170.google.com with SMTP id d7so9891074qkk.3
 for <59365@debbugs.gnu.org>; Tue, 22 Nov 2022 02:22:55 -0800 (PST)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112;
 h=content-transfer-encoding:mime-version:user-agent:message-id
 :in-reply-to:date:references:subject:cc:to:from:from:to:cc:subject
 :date:message-id:reply-to;
 bh=uF3Z4Qoxr0txlqMqI3SqVBuR80tu5lxgTIvHDXTSHIk=;
 b=kkUBO6ccGZb+LeQlfS1s7t2CFg1rXZyyRbuL6IAlZ2qd8bBq7LPWjE+Tkg64Qj5REs
 AOHYzwezpd3xn5BKESFsJwKHhMZ6GebfkSoak8ovTLvuD2rcNyGhM/m+6cuUXguPPXsf
 ALpUNJNWeQEgvOD9nzvEJ6fgRuiEB7diQY9VfYdO07S0+h8fKK5VcapUN/EmxFx4TbLP
 JRfMZEvtNiEu+CGRrH08iIV3wuljHYUSwfZUkF2aA0FqnPlRRjYfpWu/kXz3QBlLDdnD
 pmH5tcnlER8/jAPO4mReIm/2kNmDNz3KJ9Yu2Lf9+uTXGmbZC5u2m0Y9xXHSBZSTULXz
 W13w==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
 d=1e100.net; s=20210112;
 h=content-transfer-encoding:mime-version:user-agent:message-id
 :in-reply-to:date:references:subject:cc:to:from:x-gm-message-state
 :from:to:cc:subject:date:message-id:reply-to;
 bh=uF3Z4Qoxr0txlqMqI3SqVBuR80tu5lxgTIvHDXTSHIk=;
 b=aX3RBT9AVUvgTefeFjfT1DBXnhVce2cgbl/AW7ptgbS3gKDl34ByjgPMxAp00t06b4
 asEDy8o3QPO+W+nqH7heHHX7cuQNKvik+kORqOg+fNfRBw1MTvQkDf29q02Grn9S9moC
 56hSIa79C14KRTIpAEgQen1o7/LN5Kj7JJ6tOQR9U0XO/WyNlvu2hkUOB8S133lWf1Fw
 /DcxKaqGBXdTMPLkzyxncbiPGBrVtAPp/iqTl0oM8AqSa4ro0BXV8XTaOyv63pcgbi4U
 ZTyzoW1uEgMlMBZyp8bdmuZm5r234gyKXn6uTnjXAC+zit1oxkmFXoRiY//Rp0en6W1y
 C8NA==
X-Gm-Message-State: ANoB5pkDgiLaMR6WsBXIoIYT63jBZJh4GE9SX0iBSoHdStDSm5XJ2DcB
 2pD9dqxQNEOr6Qbf+f7lszH2xmIhBJ9CjQ==
X-Google-Smtp-Source: AA0mqf7FFGiknGYnM4n2Ki2O8V/weHyCTzDRMgu5WThSrlTFtgM5zEpejK3kBh7ErwgKMWFhytfcIQ==
X-Received: by 2002:a37:bc9:0:b0:6fa:163f:dad8 with SMTP id
 192-20020a370bc9000000b006fa163fdad8mr20106095qkl.575.1669112569791; 
 Tue, 22 Nov 2022 02:22:49 -0800 (PST)
Received: from hurd ([2607:fad8:4:3::1003]) by smtp.gmail.com with ESMTPSA id
 s3-20020a05620a0bc300b006fa84082b6dsm10071454qki.128.2022.11.22.02.22.49
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Tue, 22 Nov 2022 02:22:49 -0800 (PST)
From: Maxim Cournoyer <maxim.cournoyer@gmail.com>
To: Ludovic Courtès <ludo@gnu.org>
Subject: Re: bug#59365: make-dynamic-linker-cache OOMs for LLVM 15 on
 i686-linux
References: <87mt8o14xd.fsf@gnu.org> <877cznidn8.fsf@gnu.org>
Date: Tue, 22 Nov 2022 05:22:48 -0500
In-Reply-To: <877cznidn8.fsf@gnu.org> ("Ludovic Courtès"'s message of "Tue, 22 Nov 2022 09:42:19 +0100")
Message-ID: <874jurp9tz.fsf@gmail.com>
User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/28.2 (gnu/linux)
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable
X-Spam-Score: 0.0 (/)
X-Debbugs-Envelope-To: 59365
Cc: 59365@debbugs.gnu.org, Greg Hogan <code@greghogan.com>,
 Marius Bakke <marius@gnu.org>
X-BeenThere: debbugs-submit@debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request@debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit@debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request@debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request@debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces@debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces@debbugs.gnu.org>
X-Spam-Score: -1.0 (-)

Hi Ludovic,

Ludovic Courtès <ludo@gnu.org> writes:

> Hi,
>
> (Cc: Maxim and Greg for LLVM packaging questions below.)
>
> Marius Bakke <marius@gnu.org> skribis:
>
>> LLVM 15.0.4 fails on i686-linux:
>>
>>   https://ci.guix.gnu.org/build/1702995/details
>>
>> Because the 'make-dynamic-linker-cache' phase runs out of memory:
>>
>>   starting phase `make-dynamic-linker-cache'
>>   GC Warning: Repeated allocation of very large block (appr. size 268439552):
>> 	May lead to memory leak and poor performance
>>   GC Warning: Repeated allocation of very large block (appr. size 134221824):
>> 	May lead to memory leak and poor performance
>>   GC Warning: Repeated allocation of very large block (appr. size 268439552):
>>   	May lead to memory leak and poor performance
>>   GC Warning: Failed to expand heap by 285216768 bytes
>>   GC Warning: Failed to expand heap by 268439552 bytes
>>   GC Warning: Out of Memory! Heap size: 3620 MiB. Returning NULL!
>>   Warning: Unwind-only out of memory exception; skipping pre-unwind handler.
>>   Warning: Unwind-only out of memory exception; skipping pre-unwind handler.
>>   Warning: Unwind-only out of memory exception; skipping pre-unwind handler.
>>
>> (excerpt from https://ci.guix.gnu.org/build/1702995/log/raw)
>>
>> Not sure why this phase uses so much memory.  Ideas?
>
> Yes: the gremlin.scm code uses ‘file-dynamic-info’, which loads the
> whole file in memory.  Ridiculous.

If it loaded just that file, it should be fine, no?  It weighs 133 MiB,
as you've shown below:

> But the crux of the problem is that llvm@15 has a single huge shared
> library, unlike previous versions:
>
> $ du -hL  /gnu/store/bgqdvvi7k6l255332rfawgjmn2hpn13r-llvm-15.0.4/lib/*.so
> 133M    /gnu/store/bgqdvvi7k6l255332rfawgjmn2hpn13r-llvm-15.0.4/lib/libLLVM-15.0.4.so
> 96K     /gnu/store/bgqdvvi7k6l255332rfawgjmn2hpn13r-llvm-15.0.4/lib/libLTO.so
> 16K     /gnu/store/bgqdvvi7k6l255332rfawgjmn2hpn13r-llvm-15.0.4/lib/libRemarks.so
> (It also has tons of .a files, which shouldn’t be there.)

The static files are needed at least to build the clang runtime.  I had
tried to get rid of them without success.  Perhaps they could be moved
to a "static" output if they're needed only at that time.

> Is that big LLVM.so due to different build options on our side?  Or is
> it a radical upstream change (sounds unlikely, but who knows)?

It's caused by -DLLVM_LINK_LLVM_DYLIB=ON and -DLLVM_BUILD_LLVM_DYLIB=ON,
which is the supported configuration to build a shared library of LLVM
(-DBUILD_SHARED_LIBS=ON is obsolete/deprecated) [0].

It also makes things conveniently easy to link to LLVM; you just need to
link to '-lLLVM', and everything it needs is available.

[0]  https://llvm.org/docs/BuildingADistribution.html#special-notes-for-library-only-distributions

-- 
Thanks,
Maxim

Send a report that this bug log contains spam.

debbugs.gnu.org maintainers <help-debbugs@gnu.org>. Last modified: Mon Sep 8 11:35:24 2025; Machine Name: wallace-server

GNU bug tracking system

Debbugs is free software and licensed under the terms of the GNU Public License version 2. The current version can be obtained from https://bugs.debian.org/debbugs-source/.

Copyright © 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson, 2005-2017 Don Armstrong, and many other contributors.