#52555 - [RFC PATCH 0/3] Decentralized substitute distribution with ERIS

Package	Source(s)		Maintainer(s)
guix-patches		PTS Buildd Popcon

Message #175 received at 52555@debbugs.gnu.org (full text, mbox, reply):

Received: (at 52555) by debbugs.gnu.org; 14 Jan 2023 18:42:53 +0000
From debbugs-submit-bounces@debbugs.gnu.org Sat Jan 14 13:42:53 2023
Received: from localhost ([127.0.0.1]:55545 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.84_2)
	(envelope-from <debbugs-submit-bounces@debbugs.gnu.org>)
	id 1pGlUT-0005m0-Av
	for submit@debbugs.gnu.org; Sat, 14 Jan 2023 13:42:53 -0500
Received: from eggs.gnu.org ([209.51.188.92]:52814)
 by debbugs.gnu.org with esmtp (Exim 4.84_2)
 (envelope-from <ludo@gnu.org>) id 1pGlUR-0005lo-OL
 for 52555@debbugs.gnu.org; Sat, 14 Jan 2023 13:42:52 -0500
Received: from fencepost.gnu.org ([2001:470:142:3::e])
 by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256)
 (Exim 4.90_1) (envelope-from <ludo@gnu.org>)
 id 1pGlUM-00046e-FY; Sat, 14 Jan 2023 13:42:46 -0500
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=gnu.org;
 s=fencepost-gnu-org; h=MIME-Version:In-Reply-To:Date:References:Subject:To:
 From; bh=HiZHAUGDgjvZ1UwzLZGZ2RORjTX56178hpofcCl1u3c=; b=jHEb//KYGaR3711uySbq
 Dv6zz65GGk1kArGmcpoWs0ZKnYFsk9trw+ziwxRXBgDTmUE57vNaRnNASTWdBEh02tufvUPdDlPnt
 /vXep4iR23XsubvQ/eDAfQ9fnyh4bq94Ovo9Far8NIwnw5iTP7SKIAvyCWD7Ao2mNh37Ozvzl3+sO
 ccqP1UIxpmx1XVheh8HT9wLpmHoeGPKGitDTLD6pdgHCIbHPQpY08SOPV3oRDaJYU1R6E01q0fY1T
 X+UUa/E3D+YD9Wyzk6Nn7jNVgyqkfQRFh0DNwR811Z/tNmgox1Xn0QAKPzfkbyknQTZzGz15E5tB4
 LGOTwxiA+CpmSg==;
Received: from 91-160-117-201.subs.proxad.net ([91.160.117.201] helo=ribbon)
 by fencepost.gnu.org with esmtpsa (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256)
 (Exim 4.90_1) (envelope-from <ludo@gnu.org>)
 id 1pGlUL-0002YC-5j; Sat, 14 Jan 2023 13:42:46 -0500
From: Ludovic Courtès <ludo@gnu.org>
To: pukkamustard <pukkamustard@posteo.net>
Subject: Re: bug#52555: [RFC PATCH 0/3] Decentralized substitute
 distribution with ERIS
References: <20221229181327.758-1-pukkamustard@posteo.net>
 <20221229181327.758-3-pukkamustard@posteo.net>
Date: Sat, 14 Jan 2023 19:42:43 +0100
In-Reply-To: <20221229181327.758-3-pukkamustard@posteo.net>
 (pukkamustard@posteo.net's message of "Thu, 29 Dec 2022 18:13:21
 +0000")
Message-ID: <87358d7yak.fsf_-_@gnu.org>
User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/28.2 (gnu/linux)
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable
X-Spam-Score: -2.3 (--)
X-Debbugs-Envelope-To: 52555
Cc: 52555@debbugs.gnu.org
X-BeenThere: debbugs-submit@debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request@debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit@debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request@debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request@debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces@debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces@debbugs.gnu.org>
X-Spam-Score: -3.3 (---)

pukkamustard <pukkamustard@posteo.net> skribis:

> * guix/eris.scm: New file.
> * guix/eris/fs-store.scm: New file.
> * Makefile.am (MODULES): Add new files.
> * guix/scripts/publish.scm (bake-narinfo+nar): Use guix-eris-block-reducer.

[...]

> +(define %eris-block-store-directory
> +  (make-parameter
> +   (or (getenv "GUIX_ERIS_BLOCK_STORE_DIRECTORY")
> +       (string-append %state-directory "/eris"))))
> +
> +(define (guix-eris-block-reducer)
> +  "Returns a block reducer that stores blocks of ERIS encoded content."
> +  (eris-fs-store-reducer (%eris-block-store-directory)))

Maybe this should be private to (guix scripts publish)?

Also, the store directory should be /var/cache/guix/publish/eris by
default IMO.

> +(define (eris-fs-store-reducer store-directory)
> +  (case-lambda
> +    (() (mkdir-p store-directory))
> +
> +    ((result) result)
> +
> +    ((_ ref-block)
> +     (let* ((ref (car ref-block))
> +	    (b32 (base32-encode ref))
> +	    (pre (substring b32 0 2))
> +	    (suf (substring b32 2))
> +	    (pre-dir (string-append store-directory "/" pre))
> +	    (path (string-append pre-dir "/" suf))
> +	    (block (cdr ref-block)))
> +
> +       (mkdir-p pre-dir)
> +
> +       (unless (file-exists? path)
> +	 (call-with-output-file path
> +	   (lambda (port) (put-bytevector port block))
> +	   #:binary #t))
> +
> +       #t))))
> +
> +(define (eris-fs-store-ref store-directory)
> +  (lambda (ref)
> +    (let* ((b32 (base32-encode ref))
> +	   (pre (substring b32 0 2))
> +	   (suf (substring b32 2))
> +	   (path (string-append store-directory "/" pre "/" suf)))
> +      (if (file-exists? path)
> +	  (call-with-input-file path
> +	    (lambda (port) (get-bytevector-all port))
> +	    #:binary #t)
> +	  #f))))

Could you add docstrings, remove tabs, and use (ice-9 match) instead of
car/cdr?  :-)

Whole files (blocks, right?) get loaded in memory.  Is that OK or should
it be avoided?

There are time-of-check-to-time-of-use race conditions with those
‘file-exists?’ calls.  In the case of ‘ref’, you can instead write:

  (catch 'system-error
    (lambda ()
      (call-with-input-file …))
    (lambda args
      (if (= ENOENT (system-error-errno args))
          #f
          (apply throw args))))

In the case of ‘reducer’, perhaps it’d be safer to write the block
atomically with ‘with-atomic-file-output’?  (And remove the
‘file-exists?’ check too?)

Write “file” rather than “path” (the latter is used to refer to “search
paths”).  :-)

Ludo’.

Display info messages

Send a report that this bug log contains spam.

debbugs.gnu.org maintainers <help-debbugs@gnu.org>. Last modified: Mon Sep 8 11:22:03 2025; Machine Name: wallace-server

GNU bug tracking system

Debbugs is free software and licensed under the terms of the GNU Public License version 2. The current version can be obtained from https://bugs.debian.org/debbugs-source/.

#52555 [RFC PATCH 0/3] Decentralized substitute distribution with ERIS