#41702 - `guix environment` performance issues

Package	Source(s)		Maintainer(s)
guix		PTS Buildd Popcon

Message #5 received at submit@debbugs.gnu.org (full text, mbox, reply):

Received: (at submit) by debbugs.gnu.org; 4 Jun 2020 08:23:25 +0000
From debbugs-submit-bounces@debbugs.gnu.org Thu Jun 04 04:23:25 2020
Received: from localhost ([127.0.0.1]:45057 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.84_2)
	(envelope-from <debbugs-submit-bounces@debbugs.gnu.org>)
	id 1jgl9o-0001rh-TY
	for submit@debbugs.gnu.org; Thu, 04 Jun 2020 04:23:25 -0400
Received: from lists.gnu.org ([209.51.188.17]:35836)
 by debbugs.gnu.org with esmtp (Exim 4.84_2)
 (envelope-from <ldb@leibniz-psychology.org>) id 1jgl9m-0001rY-TE
 for submit@debbugs.gnu.org; Thu, 04 Jun 2020 04:23:23 -0400
Received: from eggs.gnu.org ([2001:470:142:3::10]:48448)
 by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256)
 (Exim 4.90_1) (envelope-from <ldb@leibniz-psychology.org>)
 id 1jgl9m-0001xr-N6
 for bug-guix@gnu.org; Thu, 04 Jun 2020 04:23:22 -0400
Received: from mail-ed1-x532.google.com ([2a00:1450:4864:20::532]:41745)
 by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128)
 (Exim 4.90_1) (envelope-from <ldb@leibniz-psychology.org>)
 id 1jgl9k-0003uP-Mo
 for bug-guix@gnu.org; Thu, 04 Jun 2020 04:23:22 -0400
Received: by mail-ed1-x532.google.com with SMTP id ce8so3952525edb.8
 for <bug-guix@gnu.org>; Thu, 04 Jun 2020 01:23:19 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
 d=leibniz-psychology-org.20150623.gappssmtp.com; s=20150623;
 h=date:from:to:subject:message-id:mime-version:content-disposition
 :user-agent; bh=GUsCJnysXURzM+btb8b0onoT10tehoeNWU2ITasfoqo=;
 b=F0ioEYnIrxb2P6DUvkTx5mjIHZmeaSfhuZZpxVV+WXLHmcg7zdsl35OvXazTvREdDQ
 h8qi9pnGsC77d/A1Ftm5bcCYJ7Yl5RoS/fZOmjAD6rPwpJndVZBEFeL5OLNu3NWNGyoj
 z+uvUNiK/KBI7c35dM/L5FTizzo9mGq7Qk+O/akSmovt6fTblQN8chVceJumA6B2hdxF
 WgWF6zZmT5EAVxJ29H+C1fYSES4dNLcOqxypFFfvh5HR2vvvLQSVdDtn3uyo2N385lb/
 UZ/aK1u2yc0l+Wi0yt2+/46Zup4uOGFPvmIcL/0NVQ9zoXcuNIzFD42z12D+mQa1du2o
 ulIg==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
 d=1e100.net; s=20161025;
 h=x-gm-message-state:date:from:to:subject:message-id:mime-version
 :content-disposition:user-agent;
 bh=GUsCJnysXURzM+btb8b0onoT10tehoeNWU2ITasfoqo=;
 b=GXXbcDTL/5s43ExdeNp8ZxO1UxnhrZdI0NfNU3JFuAy0vagLfCQ5YFIl6PORxkUMjb
 ZQiiB34tRld2rB9BkSa7E14C4U8iHMg37x0VYbxQbCtS22muUi0OnWewzPiqMIq8HhUP
 OPl/FQ8UqemTo+pM52rHUJq2gAQK+5cf0ygdMNu79tXbBiUvZ9/rAIInwZ5EFMKtCJxV
 3Bvbn6U65p5HWLQyf2aZDl0kzjjXKhJWx7am//ikth8eIlEOmZuvcCutxwh9H0vaDEFK
 jYPAJukqVPOyZ7s37FglCc1kW84KvQYXDkvXXhHVlUdSR+KEeCOjD97VWVPNtCCvkm8x
 hFdg==
X-Gm-Message-State: AOAM530c6T6z+gu6dTEOUpRwEeeO1cLqZW2J98GFOM3YbAqd1Ez5/Gme
 x0p21YRfL2V47kC476u8pU1aAETxfkbDSx8BTRF4HVNQW8PE9yeMxoLOzIMKVTWdcr8nhlEwLMQ
 X9Jp/ZD8k5acUtiec84pL+d/kiQSTMHegKDvunPNILi0roG3YkPPFH372feW8rsyfcoWC2LRQ/I
 oCKw==
X-Google-Smtp-Source: ABdhPJx985fJ7VBpf5/94I81LK81huj7+wGLOedoy4RDvfx8qH+XfFSfQdRQqlJ9NK8tgL7SSTwQBw==
X-Received: by 2002:a05:6402:3ca:: with SMTP id
 t10mr3339066edw.128.1591258998105; 
 Thu, 04 Jun 2020 01:23:18 -0700 (PDT)
Received: from localhost
 (dynamic-2a01-0c23-8458-f500-08f2-7cdd-6690-abd3.c23.pool.telefonica.de.
 [2a01:c23:8458:f500:8f2:7cdd:6690:abd3])
 by smtp.gmail.com with ESMTPSA id v12sm1687474eda.39.2020.06.04.01.23.17
 for <bug-guix@gnu.org>
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Thu, 04 Jun 2020 01:23:17 -0700 (PDT)
Date: Thu, 4 Jun 2020 10:23:16 +0200
From: Lars-Dominik Braun <ldb@leibniz-psychology.org>
To: bug-guix@gnu.org
Subject: `guix environment` performance issues
Message-ID: <20200604082316.GA3146@zpidnp36>
MIME-Version: 1.0
Content-Type: multipart/signed; micalg=pgp-sha512;
 protocol="application/pgp-signature"; boundary="UugvWAfsgieZRqgk"
Content-Disposition: inline
User-Agent: Mutt/1.10.1 (2018-07-13)
Received-SPF: none client-ip=2a00:1450:4864:20::532;
 envelope-from=ldb@leibniz-psychology.org; helo=mail-ed1-x532.google.com
X-detected-operating-system: by eggs.gnu.org: No matching host in p0f cache.
 That's all we know.
X-Spam_score_int: -18
X-Spam_score: -1.9
X-Spam_bar: -
X-Spam_report: (-1.9 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1,
 DKIM_VALID=-0.1, RCVD_IN_DNSWL_NONE=-0.0001,
 URIBL_BLOCKED=0.001 autolearn=_AUTOLEARN
X-Spam_action: no action
X-Spam-Score: -2.3 (--)
X-Debbugs-Envelope-To: submit
X-BeenThere: debbugs-submit@debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request@debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit@debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request@debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request@debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces@debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces@debbugs.gnu.org>
X-Spam-Score: -3.3 (---)

[Message part 1 (text/plain, inline)]

Hi,

I’ve noticed that `guix environment` can be very very slow for some packages.
Whereas usually a call like

	time guix environment --ad-hoc <package> -- true

takes 600ms to 1.5s, it takes 8.4s for the package r-learnr on my Ryzen 5 3600
with NVMe SSD, 32G of RAM and a warm cache. This seems to accumulate when
trying to install multiple unrelated “slow” packages into the same environment,
i.e. jupyter (2.9s) + r-rlearnr (8.4s) + r-stm (2.9s) = 12.1s. What these
packages seem to have in common is a deep chain of propagated inputs.

And in fact, if I create artificial test packages where test-999 propagates
test-998, test-998→test-997 and so on the command above takes 1m 9s. A single
package propagating 999 packages takes 4.5s.

Instrumenting guix-environment’s with-store call with statprof I see a lot of
time is spent in the garbage collector:

Total time: 24.672604202 seconds (19.431122691 seconds in GC)
./pre-inst-env guix environment --ad-hoc r-learnr -- true  25,18s user 0,24s system 308% cpu 8,248 total

More specifically in an anonymous function and reap-pipes, which is a gc hook,
I believe:

%     cumulative   self
time   seconds    seconds   calls   procedure
 33.41     14.49      8.24                            anon #xbb8480
 27.95      6.90      6.90                            ice-9/popen.scm:145:0:reap-pipes
  4.37      1.08      1.08                            anon #xbbdcd8
  3.28      0.86      0.81                            ice-9/vlist.scm:539:0:vhash-assq
  2.40      2.37      0.59                            guix/grafts.scm:202:22

Not sure if that helps tracking the issue down though.

Cheers,
Lars

PS: Here’s `guix describe`:
Generation 1	Jun 03 2020 10:38:17	(current)
  guix 9ab2790
    repository URL: https://git.savannah.gnu.org/git/guix.git
    branch: master
    commit: 9ab2790e2acf872c0f2cf41ec180de80d1eee3f3
  zpid bab33c1
    repository URL: https://github.com/leibniz-psychology/guix-zpid.git
    branch: master
    commit: bab33c19e0f0304d7aa82751653eb85d8cad8c0a

[signature.asc (application/pgp-signature, inline)]

Display info messages

Send a report that this bug log contains spam.

debbugs.gnu.org maintainers <help-debbugs@gnu.org>. Last modified: Sun Dec 22 14:53:55 2024; Machine Name: wallace-server

GNU bug tracking system

Debbugs is free software and licensed under the terms of the GNU Public License version 2. The current version can be obtained from https://bugs.debian.org/debbugs-source/.

#41702 `guix environment` performance issues