#45174 ‘guix substitute’ doesn’t handle HTTP redirects

Message #13 received at 45174@debbugs.gnu.org (full text, mbox, reply):

Received: (at 45174) by debbugs.gnu.org; 12 Jan 2021 21:34:05 +0000
From debbugs-submit-bounces@debbugs.gnu.org Tue Jan 12 16:34:05 2021
Received: from localhost ([127.0.0.1]:33476 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.84_2)
	(envelope-from <debbugs-submit-bounces@debbugs.gnu.org>)
	id 1kzRIj-000456-9l
	for submit@debbugs.gnu.org; Tue, 12 Jan 2021 16:34:05 -0500
Received: from world.peace.net ([64.112.178.59]:39494)
 by debbugs.gnu.org with esmtp (Exim 4.84_2)
 (envelope-from <mhw@netris.org>) id 1kzRIg-00044Z-34
 for 45174@debbugs.gnu.org; Tue, 12 Jan 2021 16:34:04 -0500
Received: from mhw by world.peace.net with esmtpsa
 (TLS1.3:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92)
 (envelope-from <mhw@netris.org>)
 id 1kzRIX-0006RT-Dr; Tue, 12 Jan 2021 16:33:53 -0500
From: Mark H Weaver <mhw@netris.org>
To: Julien Lepiller <julien@lepiller.eu>, Ludovic Courtès
 <ludovic.courtes@inria.fr>
Subject: Re: bug#45174: ‘guix substitute’ doesn’t handle HTTP redirects
In-Reply-To: <20210112181155.05b7992d@tachikoma.lepiller.eu>
References: <87a6uk8ydv.fsf@inria.fr>
 <20210112181155.05b7992d@tachikoma.lepiller.eu>
Date: Tue, 12 Jan 2021 16:32:39 -0500
Message-ID: <87pn294z8t.fsf@netris.org>
MIME-Version: 1.0
Content-Type: text/plain
X-Spam-Score: 0.0 (/)
X-Debbugs-Envelope-To: 45174
Cc: 45174@debbugs.gnu.org
X-BeenThere: debbugs-submit@debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request@debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit@debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request@debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request@debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces@debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces@debbugs.gnu.org>
X-Spam-Score: -1.0 (-)

Hi Julien,

Julien Lepiller <julien@lepiller.eu> writes:

> Here is a patch to fix that issue.  Since ci.guix.info now returns 200,
> it's difficult to test the patch.  [...]
[...]
> From f20e01f2a8df538519660772a7431b53d650d64f Mon Sep 17 00:00:00 2001
> From: Julien Lepiller <julien@lepiller.eu>
> Date: Tue, 12 Jan 2021 18:07:25 +0100
> Subject: [PATCH] substitute: Follow narinfo redirections.
>
> * guix/scripts/substitute.scm (fetch-narinfos): Follow redirections.
> ---
>  guix/scripts/substitute.scm | 38 +++++++++++++++++++++++++++----------
>  1 file changed, 28 insertions(+), 10 deletions(-)
>
> diff --git a/guix/scripts/substitute.scm b/guix/scripts/substitute.scm
> index e53de8c304..790168091e 100755
> --- a/guix/scripts/substitute.scm
> +++ b/guix/scripts/substitute.scm
> @@ -663,18 +663,36 @@ port to it, or, if connection failed, print a warning and return #f.  Pass
[...]
> +        ((301 302 303 307 308)            ; redirect
> +         (let* ((uri (response-location response))
> +                (new-request (build-request
> +                               uri #:headers '((User-Agent . "GNU Guile")))))
> +           (if len
> +               (get-bytevector-n port len)
> +               (read-to-eof port))
> +           (append
> +             (http-multiple-get uri
> +                                handle-narinfo-response '()
> +                                (list new-request)
> +                                #:open-connection
> +                                open-connection-for-uri/cached
> +                                #:verify-certificate? #f)
> +             result)))

Granted, it's been almost six years since I first implemented proper
HTTP redirects for Guix, but as I vaguely recall the URI in the response
may be a relative URI or have some missing components, so in the general
case it must be interpreted relative to the previous URI in accordance
with RFC 3986 section 5.2.

A proper implementation should use 'resolve-uri-reference' from (guix
build download).  Here's the original commit that added that function,
and used it to fix HTTP redirection support in (guix http-client):

  https://git.savannah.gnu.org/cgit/guix.git/commit/?id=04dec194d8e460831ec0695a944d9c7313affea2

Also, keep in mind that multiple redirects may occur, so a proper
implementation requires some kind of loop.  I haven't looked closely
enough at your code above to know whether that case is handled
correctly.  See the relevant code in (guix http-client) for hints.

Anyway, thanks for working on it!

     Regards,
       Mark

Send a report that this bug log contains spam.