#26608 Provide --only-substitutes flag to "guix package --upgrade"

Message #43 received at 26608@debbugs.gnu.org (full text, mbox, reply):

Received: (at 26608) by debbugs.gnu.org; 4 Sep 2018 08:02:41 +0000
From debbugs-submit-bounces@debbugs.gnu.org Tue Sep 04 04:02:41 2018
Received: from localhost ([127.0.0.1]:44990 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.84_2)
	(envelope-from <debbugs-submit-bounces@debbugs.gnu.org>)
	id 1fx6IL-0007Zt-Cn
	for submit@debbugs.gnu.org; Tue, 04 Sep 2018 04:02:41 -0400
Received: from [82.153.16.8] (port=36569 helo=ronja.pompo.co)
 by debbugs.gnu.org with esmtp (Exim 4.84_2)
 (envelope-from <alex@pompo.co>)
 id 1fx6II-0007ZX-TH; Tue, 04 Sep 2018 04:02:39 -0400
Received: from rosser (vodsl-8997.vo.lu [85.93.202.37])
 (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits))
 (No client certificate requested)
 by ronja.pompo.co (Postfix) with ESMTPSA id 9658A402FA;
 Tue,  4 Sep 2018 08:02:32 +0000 (UTC)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=pompo.co; s=mail;
 t=1536048152; bh=GQP2IdXMJuN0f50IPjTghGMdDO6YbwC5eOFHkgfT8Bk=;
 h=References:From:To:Cc:Subject:Reply-To:In-reply-to:Date:From;
 b=WNBWWt3tv15EGYX8rnS4a6Yvkbk31Yj2Mmbxy+iBal9MgYUv980Jpj9E9HZaOuoL5
 lRrMYpPrYxIfCf462cRpBPcR7FjrTiajjW0QrHkwGjIqeKRZoFqR93p8aOhR+Nfb+U
 Po3vHezlCJflOKTaxo4bfKJKV96rpAis2nSGTDPM=
References: <87vb5vsffd.fsf@gnu.org> <87pny2iks2.fsf@gnu.org>
 <877ekagtg9.fsf@netris.org> <87zhx5msfl.fsf@pompo.co>
 <87lg8pccys.fsf_-_@netris.org> <87zhx59gh3.fsf@elephly.net>
 <m1wos82y70.fsf@ordinateur-de-catherine--konrad.home>
 <875zzs9wzl.fsf@netris.org> <m1d0u0qi4v.fsf@fastmail.net>
 <874lfcxd2v.fsf_-_@gnu.org> <87wos8lzcj.fsf@pompo.co>
 <878t4nqzqv.fsf@gnu.org> <87r2iau0wz.fsf@pompo.co> <87zhwywe8v.fsf@gnu.org>
User-agent: mu4e 1.0; emacs 26.1
From: Alex Sassmannshausen <alex@pompo.co>
To: Ludovic Courtès <ludo@gnu.org>
Subject: Re: bug#22629: “Stable” branch
In-reply-to: <87zhwywe8v.fsf@gnu.org>
Date: Tue, 04 Sep 2018 10:02:31 +0200
Message-ID: <87pnxtu1uw.fsf@pompo.co>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable
X-Spam-Score: 1.3 (+)
X-Spam-Report: Spam detection software, running on the system "debbugs.gnu.org",
 has NOT identified this incoming email as spam.  The original
 message has been attached to this so you can view it or label
 similar future email.  If you have any questions, see
 the administrator of that system for details.
 
 Content preview:  Ludovic Courtès writes: > Hi Alex, > > Alex Sassmannshausen
    <alex@pompo.co> skribis: > >> Ludovic Courtès writes: > > [...] > >>> I
   just had a bright idea (yes!): this can be addressed by writing >>> something
    like this in ~/.config/guix/channels.scm: >>> >>> (map latest-commit-with-substitutes-available
    >>> %default-channels) >>> >>> The hypothetical ‘latest-commit-with-substitutes-available’
    would use >>> (git) and (guix ci) to find the latest commit for which substitutes
    of >>> interest are available, and would return: >>> >>> (channel >>> ;;
   … >>> (commit "cabbag3")) ;the ideal commit >> >> This sounds incredibly
    interesting — and it is testament once again to >> the power of Guix that
    this kind of solution could be feasible! > > Just to be clear: I don’t
   think this would be a substitute for a > “stable” branch; rather, I view
    as a way to have user-defined policies > such as “pull up to the latest
    commit for which there’s a substitute for > IceCat.” [...] 
 
 Content analysis details:   (1.3 points, 10.0 required)
 
  pts rule name              description
 ---- ---------------------- --------------------------------------------------
 -0.0 SPF_PASS               SPF: sender matches SPF record
  1.3 RDNS_NONE              Delivered to internal network by a host with no rDNS
  0.0 T_DKIM_INVALID         DKIM-Signature header exists but is not valid
X-Debbugs-Envelope-To: 26608
Cc: 26608@debbugs.gnu.org, Konrad Hinsen <konrad.hinsen@fastmail.net>,
 22629@debbugs.gnu.org, 32022@debbugs.gnu.org
X-BeenThere: debbugs-submit@debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request@debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit@debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request@debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request@debbugs.gnu.org?subject=subscribe>
Reply-To: alex@pompo.co
Errors-To: debbugs-submit-bounces@debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces@debbugs.gnu.org>
X-Spam-Score: 0.3 (/)

Ludovic Courtès writes:

> Hi Alex,
>
> Alex Sassmannshausen <alex@pompo.co> skribis:
>
>> Ludovic Courtès writes:
>
> [...]
>
>>> I just had a bright idea (yes!): this can be addressed by writing
>>> something like this in ~/.config/guix/channels.scm:
>>>
>>>   (map latest-commit-with-substitutes-available
>>>        %default-channels)
>>>
>>> The hypothetical ‘latest-commit-with-substitutes-available’ would use
>>> (git) and (guix ci) to find the latest commit for which substitutes of
>>> interest are available, and would return:
>>>
>>>   (channel
>>>     ;; …
>>>     (commit "cabbag3"))   ;the ideal commit
>>
>> This sounds incredibly interesting — and it is testament once again to
>> the power of Guix that this kind of solution could be feasible!
>
> Just to be clear: I don’t think this would be a substitute for a
> “stable” branch; rather, I view as a way to have user-defined policies
> such as “pull up to the latest commit for which there’s a substitute for
> IceCat.”

Ah, I understand now.

So the example you provided is a user-defined policy to install the
latest version of Guix that is downloadable using substitutes (if guix
publish has published those already).

As you say, in a similar vein, the end user could for themselves define
a policy that searches for a commit containing a specific successful
build, or a set of specific successful builds.

>> Thinking this through in my head somewhat, I had the following thoughts:
>> - This procedure is invoked client side, where the channel is defined
>> - That means the git searching is done client side, on every invocation
>> of guix (I guess this might be cacheable?)
>
> On every invocation of ‘guix pull’ only.

That makes sense, and is way better than I feared :-)

>> I have no idea what the performance cost would be.  I guess you would
>> use "guix weather" to turn the set of requested packages into a manifest
>> which can then be checked with it.
>
> As I imagine it, the cost would be a few HTTP queries to the Cuirass
> API.  I should try to come up with an example to better explain what I
> had in mind!

Your example helps visualize this, thanks.

Your example depends on there being a jobset that comprises the set of
packages you are interested in testing.

I imagine it is possible to do the same for an individual package / job.

The situation would be different if the end user wanted to perform a
similar operation for an arbitrary set of packages on their end.

It would probably involve something like this (probably naive):

(define (latest-commit-successfully-built-pkg pkg)
  "Return the latest commit for the pkg for which substitutes are
(potentially) available."
  ;;                  Like your version, but magically performs query
  ;;                  for pkg, not the guix-modular-master evaluation
  (let* ((evaluations (latest-evaluations pkg))
         (candidates  (filter-map (lambda (json)
                                    (match (hash-ref json "checkouts")
                                      ((checkout)
                                       (cons (hash-ref json "id")
                                             (hash-ref checkout "commit")))
                                      (_ #f)))
                                  evaluations)))
    (map (match-lambda
            ((evaluation . commit)
             (and (evaluation-complete? evaluation)
                  commit)))
          candidates)))

(any (match-lambda
        ((evaluation . commit) commit)
     (apply lset-intersection equal?
     ;;   Like latest-commit-successfully-built, but takes an
     ;;   individual package name for which we return the
     ;;   commit
     (map latest-commit-successfully-built-pkg
          %set-of-packages))))

Obviously the larger the set, the more requests are required, and the
lower the chance of a commit being available / a downgrade occuring

>> The question of security updates is tricky at the moment already — I
>> would hazard a guess that many people bail out of upgrading when they
>> can't get substitutes for their entire profile / system right now, which
>> means they are not getting security upgrades for package (a) when a
>> substitute for (b) fails.
>
> That’s probably true, and I agree it’s problematic.
>
> What I typically do is “guix pull && guix package -n -u”.  Then I look
> at things that would be built; if, say, LibreOffice is among them, I
> wait for a little while and try again later, until I can get enough
> substitutes.  That usually works okay, but it fails if it turns out that
> one of the dependencies fails to build: substitutes never become
> available in that case.

Interesting.  Do you think this kind of thing might be useful to have in
the Guix manual? Like, in a section about a "typical" desktop end-user
might manage their system day to day?

Alex

Send a report that this bug log contains spam.