#24937 - "deleting unused links" GC phase is too slow

Package	Source(s)		Maintainer(s)
guix		PTS Buildd Popcon

Message #79 received at 24937@debbugs.gnu.org (full text, mbox, reply):

Received: (at 24937) by debbugs.gnu.org; 22 Nov 2021 02:30:50 +0000
From debbugs-submit-bounces@debbugs.gnu.org Sun Nov 21 21:30:50 2021
Received: from localhost ([127.0.0.1]:46433 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.84_2)
	(envelope-from <debbugs-submit-bounces@debbugs.gnu.org>)
	id 1moz6Y-0000g2-7t
	for submit@debbugs.gnu.org; Sun, 21 Nov 2021 21:30:50 -0500
Received: from mail-40134.protonmail.ch ([185.70.40.134]:10814)
 by debbugs.gnu.org with esmtp (Exim 4.84_2)
 (envelope-from <john.kehayias@protonmail.com>) id 1moz6W-0000fo-BY
 for 24937@debbugs.gnu.org; Sun, 21 Nov 2021 21:30:48 -0500
Date: Mon, 22 Nov 2021 02:30:36 +0000
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=protonmail.com;
 s=protonmail; t=1637548237;
 bh=NEesTcyW+2GEzH5kneN1fmQP2U4YEf6SqRUo1gf5928=;
 h=Date:To:From:Reply-To:Subject:From;
 b=ZEfhiQlJD8/KsKAkWBx7FNEoIR8trh/OkVG7qAXLcWRJyZO/+tMbBbMQIuJ6nkdvA
 69w+o8unxPlORsiFVEYlJi3F9dAf+u5CzoSIkUm3muMjtq1vTL8C8QYul4/g6A2nZE
 94OwC+H/hlz5+PIEeOk5UYRxQvP6A+5Rwwp08UdE=
To: "24937@debbugs.gnu.org" <24937@debbugs.gnu.org>
From: John Kehayias <john.kehayias@protonmail.com>
Subject: Re: "deleting unused links" GC phase is too slow
Message-ID: <QAT6Oklv6MJDMioc_c1DjSQKV_v-3AWmaSFX9mnO5dqeHMJ-v9W43_z4orvQ0cp5awD2GfvFlO8rBIh5jwnHyj1ZxXGMBUsTUsOf5nYq_l8=@protonmail.com>
MIME-Version: 1.0
Content-Type: multipart/mixed;
 boundary="b1_uiddqzn2gcURBO3qy5L6W4nX4WZphrwn3gokecSEo"
X-Spam-Status: No, score=-1.2 required=10.0 tests=ALL_TRUSTED,DKIM_SIGNED,
 DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,FREEMAIL_FROM shortcircuit=no
 autolearn=disabled version=3.4.4
X-Spam-Checker-Version: SpamAssassin 3.4.4 (2020-01-24) on
 mailout.protonmail.ch
X-Debbugs-Envelope-To: 24937
X-BeenThere: debbugs-submit@debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request@debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit@debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request@debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request@debbugs.gnu.org?subject=subscribe>
Reply-To: John Kehayias <john.kehayias@protonmail.com>
Errors-To: debbugs-submit-bounces@debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces@debbugs.gnu.org>

[Message part 1 (text/plain, inline)]

Hello,

A little late, but wanted to add my results here, from before the commit was made. I had reported some of this on IRC before and had some outlying results. Since then I finally did some generation deleting and gc-ing, though perhaps still have a bit of cruft. I've been on core-updates-frozen for a while, so keeping a lot from before making the branch switch, as well as lots of stuff piling up in trying to fix things or trying out changes in core-updates-frozen.

Anyway, attached are the plots from the above code. Running

ls -1A /gnu/store/.links | wc -l

showed 15776256 links at the time. Still quite a bit I think, but I've had 1.5-2x as much in the past, easily. (This had caused some earlier warnings on ext4 and enabling large_dir, which will make a system unbootable due to Grub not being up to speed on this old feature. I'm now on btrfs.)

John

[nlink.png (image/png, attachment)]

[space-saving-contribution.png (image/png, attachment)]

[size-deduplicated.png (image/png, attachment)]

[nlink-small.png (image/png, attachment)]

[size-savings.png (image/png, attachment)]

[size.png (image/png, attachment)]

Display info messages

Send a report that this bug log contains spam.

debbugs.gnu.org maintainers <help-debbugs@gnu.org>. Last modified: Sun Sep 7 15:43:38 2025; Machine Name: wallace-server

GNU bug tracking system

Debbugs is free software and licensed under the terms of the GNU Public License version 2. The current version can be obtained from https://bugs.debian.org/debbugs-source/.

#24937 "deleting unused links" GC phase is too slow