GNU bug report logs

#70588 Content-addressed mirror is not used upon invalid hash

PackageSource(s)Maintainer(s)
guix PTS Buildd Popcon
Full log

Message #15 received at 70588@debbugs.gnu.org (full text, mbox, reply):

Received: (at 70588) by debbugs.gnu.org; 3 May 2024 18:32:55 +0000
From debbugs-submit-bounces@debbugs.gnu.org Fri May 03 14:32:55 2024
Received: from localhost ([127.0.0.1]:48127 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.84_2)
	(envelope-from <debbugs-submit-bounces@debbugs.gnu.org>)
	id 1s2xiI-0005hF-Jd
	for submit@debbugs.gnu.org; Fri, 03 May 2024 14:32:55 -0400
Received: from mail-lj1-x233.google.com ([2a00:1450:4864:20::233]:39147)
 by debbugs.gnu.org with esmtp (Exim 4.84_2)
 (envelope-from <zimon.toutoune@gmail.com>) id 1s2xiG-0005h8-Fa
 for 70588@debbugs.gnu.org; Fri, 03 May 2024 14:32:53 -0400
Received: by mail-lj1-x233.google.com with SMTP id
 38308e7fff4ca-2e2554b8cdfso1271fa.0
 for <70588@debbugs.gnu.org>; Fri, 03 May 2024 11:32:29 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
 d=gmail.com; s=20230601; t=1714761144; x=1715365944; darn=debbugs.gnu.org;
 h=content-transfer-encoding:mime-version:user-agent:message-id:date
 :references:in-reply-to:subject:cc:to:from:from:to:cc:subject:date
 :message-id:reply-to;
 bh=gxnttXzLj12523Nbhx/rHT+mn3qAeLKpnPJR5cMFBtI=;
 b=dQ4dr3pRGk54c8GyLLhDJHdCdLAD3ev48iKjUO+B41NFsZ7kL3p6t84+0oa1TY1+X0
 mbt55eQarJNd4oGNg0NAQe/Ehi6t905E1gdJkCmBuuS29H7Va+IEQQq3qITgz20lt4/y
 UIJXShewO3RR7q93Gch8A8QLdAcuI1dOAFfTa/wUafi/PBOvO+U0484ikuDPVqQTToYr
 xRn16tcFg0SR0OQAeZGlU7yRM2ub3UWCrDovMvZQkj0w6zPu3x5fvUIeGDqC4SyxN3n1
 8O6P6ECLKY3Uch90nHnQA1HrT4+Bd28YWf4Nc4TcQapsR3sfFAoOOTaV6GQrPj0DmEmA
 Pfrw==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
 d=1e100.net; s=20230601; t=1714761144; x=1715365944;
 h=content-transfer-encoding:mime-version:user-agent:message-id:date
 :references:in-reply-to:subject:cc:to:from:x-gm-message-state:from
 :to:cc:subject:date:message-id:reply-to;
 bh=gxnttXzLj12523Nbhx/rHT+mn3qAeLKpnPJR5cMFBtI=;
 b=vWPPQ0oMFpgazPT3i8DHT2DJwPbs08zLpeXcm5zC0ldSrmiAzyIkrTD8b9fNxgZrsM
 A1ylK7dmDa/Pbt72+y1vkY6CuASGhNcwFhCRJd015cmvQ9w3J8mYPA1r0I0ALwnv+Kl/
 POZEQXIKlfQ/L4ZVqXy/9uxNjkKU+lFuDzFBg9kpBmMiL6TcGMae+Yuo1351lmrjnnkv
 U2FjL2slGhoyC+5VU/g2UnBLaMXlc0YTT/+EydW5BYJiCeZ5bwC3/wxmoDkpNP5WwCs1
 smJ+ruZuO/mR4oNRepvxvXK9q3GE3aSEhAsnNPhvukVeRmu2JQtzxJla3KO8LBi4FMnV
 eGOw==
X-Gm-Message-State: AOJu0YwWvg6/S7J92eyU46BQUI8pTkCWUL7RoeQXg1ruvYO9niaz10Sc
 9dk1E5rbbCrW5nAIXUbS6m+Ksemw0IubhW7FnIoIX4cRJWifACNr
X-Google-Smtp-Source: AGHT+IGt6OeQZhKz2avkRybGIdh6ZlwwwRIZERW+Kqu+aog9tLWQV5KTSiTYqEV6dc5GS0At8O0qpQ==
X-Received: by 2002:a2e:9f17:0:b0:2dd:87a9:f152 with SMTP id
 u23-20020a2e9f17000000b002dd87a9f152mr2200606ljk.2.1714761143647; 
 Fri, 03 May 2024 11:32:23 -0700 (PDT)
Received: from lili (roam-nat-fw-prg-194-254-61-41.net.univ-paris-diderot.fr.
 [194.254.61.41]) by smtp.gmail.com with ESMTPSA id
 j15-20020a05600c190f00b0041bf512f85bsm10183329wmq.14.2024.05.03.11.32.22
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Fri, 03 May 2024 11:32:22 -0700 (PDT)
From: Simon Tournier <zimon.toutoune@gmail.com>
To: 70588@debbugs.gnu.org
Subject: Re: bug#70588: hash mismatch error and stop fallback
In-Reply-To: <87ttjpinb8.fsf@gmail.com> (Simon Tournier's message of "Thu, 25
 Apr 2024 15:28:27 +0200")
References: <87ttjpinb8.fsf@gmail.com>
Date: Fri, 03 May 2024 19:53:39 +0200
Message-ID: <878r0qkcik.fsf@gmail.com>
User-Agent: Gnus/5.13 (Gnus v5.13)
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable
X-Spam-Score: 0.0 (/)
X-Debbugs-Envelope-To: 70588
Cc: ludo@gnu.org
X-BeenThere: debbugs-submit@debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request@debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit@debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request@debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request@debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces@debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces@debbugs.gnu.org>
X-Spam-Score: -1.0 (-)

Hi,

On lun., 02 oct. 2017 at 17:09, ludo@gnu.org (Ludovic Courtès) wrote:

> The problem is that the hash check is performed by guix-daemon itself,
> not by “guix perform-download”.  So when guix-daemon diagnoses a hash
> mismatch, it’s too late and we cannot try again and use the
> content-addressed mirror.

This quote from [1].  Well, why not run a sequence of “guix download”
then “guix build”?  Ah the store item would be hashed twice.  Let try to
avoid this sub-optimized behaviour.

The only way to avoid hashing twice is to extend the protocol of
guix-daemon.  Because currently it reads,

            /* Check the hash. */
            Hash h2 = recursive ? hashPath(ht, actualPath).first : hashFile(ht, actualPath);
            if (h != h2) {
		if (settings.printBuildTrace)
		    printMsg(lvlError, format("@ hash-mismatch %1% %2% %3% %4%")
			     % path % i->second.hashAlgo
			     % printHash16or32(h) % printHash16or32(h2));
                throw BuildError(format("hash mismatch for store item '%1%'") % path);
	    }

with the call under error catch:

        /* Compute the FS closure of the outputs and register them as
           being valid. */
        registerOutputs();

[...]

    } catch (BuildError & e) {


Thus the number of solutions is very limited. :-)

Well, from my understanding, moving elsewhere in guix-daemon the check
of the hash appears to me too cumbersome.  Therefore, the only way is to
replace this ’BuildError’ by something that is not an error and do not
register the item.

Easier to say than to implement.  Again from my understanding, each
strategy for downloading is seen by the daemon as an element of a list
of independent builds and these elements are tried in sequence.
Therefore, we need to special case in order to be able to distinguish
different kind of failures.

All in all, this bug (missing feature):

        Other said, hash mismatch should be raised and obviously the content
        must be discarded; then the hash mismatch should be considered as “not
        found” so let try the next strategy for downloading the content.

is not straightforward to fix. :-)  Need for more thought…


Cheers,
simon


1: bug#28659: v0.13: guix pull fails; libgit2-0.26.0 and 0.25.1 content hashes fail
ludo@gnu.org (Ludovic Courtès)
Mon, 02 Oct 2017 17:09:39 +0200
id:87o9ppoabw.fsf@gnu.org
https://issues.guix.gnu.org/28659
https://issues.guix.gnu.org/msgid/87o9ppoabw.fsf@gnu.org
https://yhetil.org/guix/87o9ppoabw.fsf@gnu.org

Send a report that this bug log contains spam.

debbugs.gnu.org maintainers <help-debbugs@gnu.org>. Last modified: Mon Sep 8 19:45:46 2025; Machine Name: wallace-server

GNU bug tracking system

Debbugs is free software and licensed under the terms of the GNU Public License version 2. The current version can be obtained from https://bugs.debian.org/debbugs-source/.

Copyright © 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson, 2005-2017 Don Armstrong, and many other contributors.