GNU bug report logs

#41702 `guix environment` performance issues

PackageSource(s)Maintainer(s)
guix PTS Buildd Popcon
Full log

Message #22 received at 41702@debbugs.gnu.org (full text, mbox, reply):

Received: (at 41702) by debbugs.gnu.org; 9 Jun 2020 09:15:32 +0000
From debbugs-submit-bounces@debbugs.gnu.org Tue Jun 09 05:15:32 2020
Received: from localhost ([127.0.0.1]:58937 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.84_2)
	(envelope-from <debbugs-submit-bounces@debbugs.gnu.org>)
	id 1jiaM0-0000O6-6n
	for submit@debbugs.gnu.org; Tue, 09 Jun 2020 05:15:32 -0400
Received: from mail-wm1-f49.google.com ([209.85.128.49]:53192)
 by debbugs.gnu.org with esmtp (Exim 4.84_2)
 (envelope-from <ldb@leibniz-psychology.org>) id 1jiaLy-0000Ns-98
 for 41702@debbugs.gnu.org; Tue, 09 Jun 2020 05:15:30 -0400
Received: by mail-wm1-f49.google.com with SMTP id r9so2095910wmh.2
 for <41702@debbugs.gnu.org>; Tue, 09 Jun 2020 02:15:30 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
 d=leibniz-psychology-org.20150623.gappssmtp.com; s=20150623;
 h=date:from:to:cc:subject:message-id:references:mime-version
 :content-disposition:in-reply-to:user-agent;
 bh=qz4Fu3nZaZG2CLuasY5u1szhQ0Dyk5hOv98/XWZ5f1I=;
 b=jP3zCMyf+vrHZiMZ0et32GFF4IeY7qtzwFGaS9kzDVwzKpR4C4Pq5LBBZ1tIX1Ltq0
 3/IGb9m26WjiOYTax6fw6rXH4bCF+Wv29imYKoV6bIWbqOe7daYNYLdgQgq7yi2VJKLR
 Jgj6BUGa3+z6fMUlZ7AagJkFFVi4A0ujK++jyr73jnTLqgjWCepC4K3bDoth2OBMSZLc
 BKPL68T+ZAgWyH5uG+nhwPtFZqVeXz13SblFnba+XAMmmelUGrhIz0a/TTDeAF2SoDpY
 ruTRpOYZvAQfdPoX6ZukaPCx4Wol7ZE0KGwskRAP33YlUC/yi/etozTE4D7vPRMN+Kfz
 h4hQ==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
 d=1e100.net; s=20161025;
 h=x-gm-message-state:date:from:to:cc:subject:message-id:references
 :mime-version:content-disposition:in-reply-to:user-agent;
 bh=qz4Fu3nZaZG2CLuasY5u1szhQ0Dyk5hOv98/XWZ5f1I=;
 b=rzIzad18CtNbriiEpVqMr2comC4pEnNQXWoJ0WLb8LYX+8F905YeILSe/IxJ5cgv+t
 k2fecfMbAyDuyjTV+q/bE3nkuV2Vn7zwYKMgmtQwJFjyi7HGS/oOBXQj5vM+yIZXdm5t
 TxnIe54klIl04SxZnbavdQEWBu0ZJMkkCw92NB8xHWJ56P1L5mf6+IChxp7LaKcRYsbQ
 TuzV1SE/SRHmnAzIBJxo6lvQ9gqZQUcoeGiiulog7gCxbWrCVRzcCkYiWTCiVC/sczXu
 ZNzKqPa1aoDT6E0Wa4Wvb26l5uaM91Ff2CW3ADg7/zFAckl4ikDb/762ZD7Y9RLUFi5+
 MrgA==
X-Gm-Message-State: AOAM530kEJIIvRddXlfoJgHjhNiknSDKlO50mjQ1vOkwoUyEJRSGjYFM
 bK2kapAObljXTagm/AE2j5+nygtBZNeZLWHdzZ6MJ7Ak1OefOUgxfIzwzlRWfMADvC+xupUkQ6C
 ZxLmP3fL/Hp9+VOwFqs68mwwpN+8J91r4QBBF20Ujj+icrrRDhbSVOp2t1fuAk3XLKOuZ9+BsE9
 WGAbo=
X-Google-Smtp-Source: ABdhPJzlQ3huw0HzeLj0aBtVws0x62rRoHHH8RdbCo+jm94YcF1nhx9VjXOLOa8enzfdEKbhAMTIVw==
X-Received: by 2002:a1c:98cc:: with SMTP id a195mr2967897wme.89.1591694124024; 
 Tue, 09 Jun 2020 02:15:24 -0700 (PDT)
Received: from localhost
 (dynamic-2a01-0c23-7c5e-9e00-3d61-c8ee-9cd9-ef49.c23.pool.telefonica.de.
 [2a01:c23:7c5e:9e00:3d61:c8ee:9cd9:ef49])
 by smtp.gmail.com with ESMTPSA id o18sm2276486wme.19.2020.06.09.02.15.22
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Tue, 09 Jun 2020 02:15:23 -0700 (PDT)
Date: Tue, 9 Jun 2020 11:15:22 +0200
From: Lars-Dominik Braun <ldb@leibniz-psychology.org>
To: Ludovic Courtès <ludo@gnu.org>
Subject: Re: bug#41702: `guix environment` performance issues
Message-ID: <20200609091522.GB8597@zpidnp36>
References: <20200604082316.GA3146@zpidnp36> <87mu5gtbwg.fsf@gnu.org>
 <20200608090453.GC3166@zpidnp36> <87k10hp6ba.fsf@gnu.org>
MIME-Version: 1.0
Content-Type: multipart/signed; micalg=pgp-sha512;
 protocol="application/pgp-signature"; boundary="Pk6IbRAofICFmK5e"
Content-Disposition: inline
In-Reply-To: <87k10hp6ba.fsf@gnu.org>
User-Agent: Mutt/1.10.1 (2018-07-13)
X-Spam-Score: 0.0 (/)
X-Debbugs-Envelope-To: 41702
Cc: 41702@debbugs.gnu.org
X-BeenThere: debbugs-submit@debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request@debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit@debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request@debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request@debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces@debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces@debbugs.gnu.org>
X-Spam-Score: -1.0 (-)

[Message part 1 (text/plain, inline)]
Hey,

> That’s over SSH, right?
correct, the worst possible case: Inside two VM’s on a Laptop, SSH transport
between them and /gnu+/var/guix on an NFS share (nfsd is in the same VM as
guix-daemon).

> Probably what’s killing us is the round-trip time for all these small
> RPCs.  We would need pipelining but the RPC protocol is not designed to
> make that easy.
That would have been my best guess too, but it does not seem to be the biggest
problem right now. Looking at the numbers again (both patches applied) with the
attached manifest[1], I see that:

---snip---
Local UNIX socket with and without --no-grafts:
    N           Min           Max        Median           Avg        Stddev
x  10          6.07          6.35         6.145          6.16    0.08232726
+  10         17.47         17.89        17.545        17.602    0.14351152
Difference at 99.0% confidence
        11.442 +/- 0.150576
        185.747% +/- 4.07133%

Local UNIX socket vs. guix://localhost transport:
    N           Min           Max        Median           Avg        Stddev
x  10         17.47         17.89        17.545        17.602    0.14351152
+  10         17.43          18.1         17.61        17.642    0.20131788
No difference proven at 99.0% confidence

Local UNIX socket vs ssh://localhost transport:
    N           Min           Max        Median           Avg        Stddev
x  10         17.47         17.89        17.545        17.602    0.14351152
+  10         33.46         35.27        34.315        34.359    0.53873205
Difference at 99.0% confidence
        16.757 +/- 0.5074
        95.1994% +/- 3.13957%
---snap---

So I would conclude:

1) Grafting still takes a lot of time and needs more work
2) Linux optimizes localhost networking pretty well
3) Our SSH transport is terribly slow

Moving to non-localhost communication between two VM’s:

---snip---
guix://localhost vs. guix://remote-host transport:
    N           Min           Max        Median           Avg        Stddev
x  10         17.43          18.1         17.61        17.642    0.20131788
+  10         20.88         22.58        21.095        21.222    0.49689704
Difference at 99.0% confidence
        3.58 +/- 0.487934
        20.2925% +/- 2.85159%

guix://remote-host vs. ssh://remote-host:
    N           Min           Max        Median           Avg        Stddev
x  10         20.88         22.58        21.095        21.222    0.49689704
+  10          30.1         32.56        31.005        31.093    0.70740606
Difference at 99.0% confidence
        9.871 +/- 0.786769
        46.5131% +/- 4.35326%
---snap---

Conclusion here is the same: Not alot of impact of networking/NFS and SSH
transport is still terribly slow. (Confusingly faster than localhost though.)

> Perhaps you could “strace -Tt” the thing to check whether this
> hypothesis is correct by looking at the time we spend waiting for
> replies?
I’m not sure this will yield meaningful data for SSH, so I analyzet it for
guix://localhost vs. guix://remote-host. Takeaway is, yes, of course there is a
statistically significant difference and it’s about 40%±50%, which means this
method is pretty useless, because we can’t bin RPC’s by type.

So, I guess it would make sense for me to look at the SSH transport itself
again and see if there are any other low-hanging fruit. Not sure how much I can
help with profiling guile/guix itself. A different/better RPC protocol is
probably GSoC/v2.0-worthy?

Sorry for all the lengthy emails,
Lars

[1] You’ll need this channel: https://github.com/leibniz-psychology/guix-zpid


[manifest.scm (text/plain, attachment)]
[signature.asc (application/pgp-signature, inline)]

Send a report that this bug log contains spam.

debbugs.gnu.org maintainers <help-debbugs@gnu.org>. Last modified: Sun Dec 22 15:47:42 2024; Machine Name: wallace-server

GNU bug tracking system

Debbugs is free software and licensed under the terms of the GNU Public License version 2. The current version can be obtained from https://bugs.debian.org/debbugs-source/.

Copyright © 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson, 2005-2017 Don Armstrong, and many other contributors.