#28659 - Content-addressed mirror is not used upon invalid hash

Package	Source(s)		Maintainer(s)
guix		PTS Buildd Popcon

Message #57 received at 28659@debbugs.gnu.org (full text, mbox, reply):

Received: (at 28659) by debbugs.gnu.org; 4 Oct 2017 04:22:45 +0000
From debbugs-submit-bounces@debbugs.gnu.org Wed Oct 04 00:22:45 2017
Received: from localhost ([127.0.0.1]:47192 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.84_2)
	(envelope-from <debbugs-submit-bounces@debbugs.gnu.org>)
	id 1dzbCn-0005EI-HG
	for submit@debbugs.gnu.org; Wed, 04 Oct 2017 00:22:45 -0400
Received: from mail-it0-f50.google.com ([209.85.214.50]:56628)
 by debbugs.gnu.org with esmtp (Exim 4.84_2)
 (envelope-from <maxim.cournoyer@gmail.com>) id 1dzbCk-0005E4-FZ
 for 28659@debbugs.gnu.org; Wed, 04 Oct 2017 00:22:43 -0400
Received: by mail-it0-f50.google.com with SMTP id g18so14028526itg.5
 for <28659@debbugs.gnu.org>; Tue, 03 Oct 2017 21:22:42 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025;
 h=from:to:cc:subject:references:date:in-reply-to:message-id
 :user-agent:mime-version;
 bh=SxCIt8rqMqsyWuN6CZqOa2jlWUNCgl/X5hykfLPDAk8=;
 b=PqGBZvC916zz06aomOcewmI7+QRrS6nxZawtxZWXWPgd6Hg1FGf9yiQfCPhY59K3rG
 v6UOk//EZlcLMhjyYDQhVGcrkw22JigaT40XwlkluaFRgzEhyRXKku4XXx3a8YuUamph
 86BTntElm6GI3QZIrCforv2KgbsmypfmvgA9nkROA+TrclPp/V2i3N21Bmag8Y9adLXZ
 KPBuWBWpFjxPs8txifzwzYQtjZ8cw9xhjT73N7o4XYNnL9ixVOtPDso94CrmpbVfRLpJ
 GuZfwGjgsQSTWJ15y2g/G+dS+cpraexLDkwKornA0PgwzzhaxGiaIB6N9ZjhJeWbnpFJ
 5U3g==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
 d=1e100.net; s=20161025;
 h=x-gm-message-state:from:to:cc:subject:references:date:in-reply-to
 :message-id:user-agent:mime-version;
 bh=SxCIt8rqMqsyWuN6CZqOa2jlWUNCgl/X5hykfLPDAk8=;
 b=Jtb0ckmrCKfrg2jYVI6T1zRDgnnk12Ol78cN6tp8v+AhOBJbkwQWQaXxj6tkaPh/CJ
 FxGkdsXc1pAm+Kadb87e9gCp+HktvWYq0OTejq8tPxQ4HRkU2Y5ZtgE30j7vmcpdFEWS
 s3CZMzqZRCqkPuzaaI/GZwm2Tyfgtq9wg34kurETlObFU7MXz1638D132+DL/arEGXAS
 4ePFeoS4ebd7Mpzur17EabXSfNFUvM2/rCLY5FshxIPqGlRmffckWuRdQLcnPhgfGkhf
 4U1qLFmOdVdP26IpZvMcRdsM73b1RjrWTJPkinL8uIQQF1LprHhb7U92OxotNiFodwtJ
 Xigw==
X-Gm-Message-State: AMCzsaVnIKnYsArsql9WCdLTL7SOhCcQ/pPYanoHb1lmcoZBGkJJ1Brc
 m1JpVYJTyy0AP3Fow+05ir8=
X-Google-Smtp-Source: AOwi7QCp6pEl6t/TnkntKmW22MZS4bpnF5PhcBxSjk+Om6tw85VWmdwxVm33KuI/Bjekss8+iwVPDw==
X-Received: by 10.36.29.137 with SMTP id 131mr27890627itj.91.1507090956674;
 Tue, 03 Oct 2017 21:22:36 -0700 (PDT)
Received: from apteryx (104-222-112-128.cpe.teksavvy.com. [104.222.112.128])
 by smtp.gmail.com with ESMTPSA id v33sm6704548iov.46.2017.10.03.21.22.35
 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256);
 Tue, 03 Oct 2017 21:22:36 -0700 (PDT)
From: Maxim Cournoyer <maxim.cournoyer@gmail.com>
To: Leo Famulari <leo@famulari.name>
Subject: Re: bug#28659: v0.13: guix pull fails;
 libgit2-0.26.0 and 0.25.1 content hashes fail
References: <877ewf18d4.fsf@gnu.org> <87wp4e8yk5.fsf@gnu.org>
 <20171001204237.GA11804@jasmine.lan> <87vajxoavx.fsf@gnu.org>
 <20171002181929.GA10773@jasmine.lan> <87infx2mmt.fsf@gmail.com>
 <20171003142449.GB23431@jasmine.lan>
Date: Wed, 04 Oct 2017 00:22:34 -0400
In-Reply-To: <20171003142449.GB23431@jasmine.lan> (Leo Famulari's message of
 "Tue, 3 Oct 2017 10:24:49 -0400")
Message-ID: <874lrfee45.fsf@gmail.com>
User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/25.2 (gnu/linux)
MIME-Version: 1.0
Content-Type: multipart/mixed; boundary="=-=-="
X-Spam-Score: -0.2 (/)
X-Debbugs-Envelope-To: 28659
Cc: Ludovic Courtès <ludo@gnu.org>, 28659@debbugs.gnu.org
X-BeenThere: debbugs-submit@debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request@debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit@debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request@debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request@debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces@debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces@debbugs.gnu.org>
X-Spam-Score: -0.2 (/)

[Message part 1 (text/plain, inline)]

Leo Famulari <leo@famulari.name> writes:

> On Mon, Oct 02, 2017 at 06:47:06PM -0400, Maxim Cournoyer wrote:
>> Leo Famulari <leo@famulari.name> writes:
>> > I wonder, are there really that many affected packages?
>> 
>> There's a list here:
>> https://github.com/Homebrew/homebrew-core/issues/18044, compiled by one
>> of the homebrew project's maintainers.
>
> I meant, how many Guix packages use the auto-generated GitHub snapshots?
>
> I believe the tell-tale sign is that the download link will have the
> link text 'Source code', as for this release:
>
> https://github.com/libgit2/libgit2/releases/tag/v0.26.0

The following script:

[Message part 2 (text/plain, inline)]

;;; A script to find packages possibly affected by GitHub
;;; infrastructure update that caused minor changes in the
;;; automatically generated tarballs.

(use-modules (ice-9 match)
	     (gnu packages)
	     (guix download)
	     (guix packages))

(define (problematic-uri? uri)

  (define (contains-github-archive? uri)
    (string-match "github.com/.*/archive/" uri))

  ;; URI can be a string or a list of string.
  (match uri
    ((uri1 uri2 ...)			;match list of strings
     (filter contains-github-archive? uri))
    (uri1				;match string
     (contains-github-archive? uri1))))

(define (problematic-github-package? package)
  (let ((source (package-source package)))
    (and (origin? source)
	 (eq? (origin-method source) url-fetch)
	 (problematic-uri? (origin-uri source)))))

(define (problematic-github-packages)
  "List of all the potentially problematic GitHub packages."
  (fold-packages (lambda (p r)
		   (if (problematic-github-package? p)
		       (cons p r)
		       r))
		 '()))
(define (main)
  "Find and print the names of the potentially problematic GitHub packages."
  (let ((packages (problematic-github-packages)))
    (format #t "Number of potentially problematic GitHub packages:~a~%"
	    (length packages))
    (for-each (lambda (p)
		(format #t "~a~%" (package-name p)))
	      packages)))

;;; Run the program.
(main)

[Message part 3 (text/plain, inline)]

outputs that there could be up to 1011 affected packages.

The scripts checks for a url-fetch uri of the form
"github.com/.*/archive/", which seems to be the one used for the
dynamically generated archives.

Here are the first 10 lines of the output:
--8<---------------cut here---------------start------------->8---
Number of potentially problematic GitHub packages:1011
fdupes
cbatticon
sedsed
cpulimit
autojump
sudo
thermald
progress
dstat
[...]
--8<---------------cut here---------------end--------------->8---

I've checked the first few with for example:
--8<---------------cut here---------------start------------->8---
guix build --source --no-substitutes sedsed
--8<---------------cut here---------------end--------------->8---

and they were OK though.

Maxim

Display info messages

Send a report that this bug log contains spam.

debbugs.gnu.org maintainers <help-debbugs@gnu.org>. Last modified: Mon Sep 8 01:21:51 2025; Machine Name: wallace-server

GNU bug tracking system

Debbugs is free software and licensed under the terms of the GNU Public License version 2. The current version can be obtained from https://bugs.debian.org/debbugs-source/.

#28659 Content-addressed mirror is not used upon invalid hash