GNU bug report logs

#24937 "deleting unused links" GC phase is too slow

PackageSource(s)Maintainer(s)
guix PTS Buildd Popcon
Full log

Message #37 received at 24937@debbugs.gnu.org (full text, mbox, reply):

Received: (at 24937) by debbugs.gnu.org; 13 Dec 2016 17:19:28 +0000
From debbugs-submit-bounces@debbugs.gnu.org Tue Dec 13 12:19:28 2016
Received: from localhost ([127.0.0.1]:40959 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.84_2)
	(envelope-from <debbugs-submit-bounces@debbugs.gnu.org>)
	id 1cGqjg-0001cF-Ge
	for submit@debbugs.gnu.org; Tue, 13 Dec 2016 12:19:28 -0500
Received: from sinope.bbbm.mdc-berlin.de ([141.80.25.23]:42371)
 by debbugs.gnu.org with esmtp (Exim 4.84_2)
 (envelope-from <Ricardo.Wurmus@mdc-berlin.de>) id 1cGqjf-0001c6-8A
 for 24937@debbugs.gnu.org; Tue, 13 Dec 2016 12:19:28 -0500
Received: from localhost (localhost [127.0.0.1])
 by sinope.bbbm.mdc-berlin.de (Postfix) with ESMTP id F2CF0A894A;
 Tue, 13 Dec 2016 18:19:25 +0100 (CET)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=mdc-berlin.de; h=
 content-transfer-encoding:content-type:content-type:mime-version
 :message-id:date:date:in-reply-to:subject:subject:from:from
 :user-agent:references:received:received:received; s=mdc; t=
 1481649559; x=1483463960; bh=YMu6NtX1ZnmlGcwmhJ2Cw4MUCmkE/LX0JLt
 rrnfnVCA=; b=LMdVFxVUKLaAX98PEKDm5bptcC18Qbdd8k4Rs4wyaQ1oQJOVnvS
 RhedB4HAWFnceREd0dzTi66UAe++MtIBBYgvqBLV0EL1hSHwQBfWsO6quk0H3V3u
 C8uEqu7lyTzNE7tXjB4fvZJXzZSr43Sqt0naO2L1cYp210tGpG8BqlUI=
X-Virus-Scanned: amavisd-new at mdc-berlin.de
Received: from sinope.bbbm.mdc-berlin.de ([127.0.0.1])
 by localhost (sinope.bbbm.mdc-berlin.de [127.0.0.1]) (amavisd-new, port 10024)
 with ESMTP id USUQSi6KjIHv; Tue, 13 Dec 2016 18:19:19 +0100 (CET)
Received: from HTCAONE.mdc-berlin.net (puck.citx.mdc-berlin.de [141.80.36.101])
 (using TLSv1 with cipher ECDHE-RSA-AES256-SHA (256/256 bits))
 (No client certificate requested)
 by sinope.bbbm.mdc-berlin.de (Postfix) with ESMTPS;
 Tue, 13 Dec 2016 18:19:19 +0100 (CET)
Received: from localhost (141.80.180.135) by HTCAONE.mdc-berlin.net
 (141.80.180.125) with Microsoft SMTP Server (TLS) id 14.3.319.2; Tue, 13 Dec
 2016 18:19:18 +0100
References: <87wpg7ffbm.fsf@gnu.org> <87lgvm4lzu.fsf@gnu.org>
 <87twaaa6j9.fsf@netris.org> <87twaa2vjx.fsf@gnu.org>
 <87lgvm9sgq.fsf@netris.org> <87d1gwvgu0.fsf@gnu.org>
 <87wpf4yoz0.fsf@netris.org> <87fulrsqxx.fsf@gnu.org>
User-agent: mu4e 0.9.16; emacs 25.1.1
From: Ricardo Wurmus <ricardo.wurmus@mdc-berlin.de>
To: Ludovic Courtès <ludo@gnu.org>
Subject: Re: bug#24937: "deleting unused links" GC phase is too slow
In-Reply-To: <87fulrsqxx.fsf@gnu.org>
Date: Tue, 13 Dec 2016 18:18:57 +0100
Message-ID: <87vaunbvcu.fsf@mdc-berlin.de>
MIME-Version: 1.0
Content-Type: text/plain; charset="utf-8"
Content-Transfer-Encoding: 8bit
X-Originating-IP: [141.80.180.135]
X-TM-AS-Product-Ver: SMEX-11.0.0.4283-8.000.1202-22758.006
X-TM-AS-Result: No--0.114700-0.000000-31
X-TM-AS-MatchedID: 150567-703786-703731-139010-706249-139705-707904-861157-7
 03385-105700-711664-701202-710480-701719-709009-702942-701604-700862-701296
 -700756-148004-148133-42000-42003
X-TM-AS-User-Approved-Sender: Yes
X-TM-AS-User-Blocked-Sender: No
X-Spam-Score: -8.1 (--------)
X-Debbugs-Envelope-To: 24937
Cc: Mark H Weaver <mhw@netris.org>, 24937@debbugs.gnu.org
X-BeenThere: debbugs-submit@debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request@debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit@debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request@debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request@debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces@debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces@debbugs.gnu.org>
X-Spam-Score: -8.1 (--------)
Ludovic Courtès <ludo@gnu.org> writes:

> Ricardo, Roel: would you be able to run that links-traversal.c from
> <https://debbugs.gnu.org/cgi/bugreport.cgi?filename=links-traversal.c;bug=24937;msg=25;att=1>
> on a machine with a big store, as described at
> <https://debbugs.gnu.org/cgi/bugreport.cgi?bug=24937#25>?

I just ran this on my workstation in the office where I regularly build
packages.  Here’s the output of “df -i /gnu”

  Filesystem               Inodes   IUsed   IFree IUse% Mounted on
  /dev/mapper/fedora-root 3301376 1098852 2202524   34% /

Probably not large enough to derive conclusions about hydra’s behaviour.

[I can’t run it on the shared store at the MDC because NFS performance is
too poor.  I recently ran “guix gc --optimize” to dedupe the shared
store (post-build deduplication is disabled since a few weeks) and it’s
at 3,197,489 used inodes.]

Here are the results of running the link-traversal code on my
workstation:

--8<---------------cut here---------------start------------->8---
rwurmus in ~: gcc -std=gnu99 -Wall links-traversal.c  -DMODE=3
rwurmus in ~: sudo sh -c 'echo 3 > /proc/sys/vm/drop_caches'
rwurmus in ~: time ./a.out 
412825 dir_entries, 107 seconds
stat took 0 seconds

real	1m47.264s
user	0m0.214s
sys	0m1.314s

rwurmus in ~: gcc -std=gnu99 -Wall links-traversal.c  -DMODE=2
rwurmus in ~: sudo sh -c 'echo 3 > /proc/sys/vm/drop_caches'
rwurmus in ~: time ./a.out 
12821 dir_entries, 107 seconds (including stat)

real	1m46.475s
user	0m0.201s
sys	0m1.309s
--8<---------------cut here---------------end--------------->8---


-- 
Ricardo




Send a report that this bug log contains spam.


debbugs.gnu.org maintainers <help-debbugs@gnu.org>. Last modified: Sun Sep 7 12:02:18 2025; Machine Name: wallace-server

GNU bug tracking system

Debbugs is free software and licensed under the terms of the GNU Public License version 2. The current version can be obtained from https://bugs.debian.org/debbugs-source/.

Copyright © 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson, 2005-2017 Don Armstrong, and many other contributors.