#20402 guix cannot download via an http proxy

Message #17 received at 20402@debbugs.gnu.org (full text, mbox, reply):

Received: (at 20402) by debbugs.gnu.org; 30 Apr 2015 22:08:09 +0000
From debbugs-submit-bounces@debbugs.gnu.org Thu Apr 30 18:08:09 2015
Received: from localhost ([127.0.0.1]:57837 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.80)
	(envelope-from <debbugs-submit-bounces@debbugs.gnu.org>)
	id 1Ynwcq-0006ox-UX
	for submit@debbugs.gnu.org; Thu, 30 Apr 2015 18:08:09 -0400
Received: from fencepost.gnu.org ([208.118.235.10]:34955 ident=Debian-exim)
 by debbugs.gnu.org with esmtp (Exim 4.80)
 (envelope-from <ludo@gnu.org>) id 1Ynwcp-0006oq-Ep
 for 20402@debbugs.gnu.org; Thu, 30 Apr 2015 18:08:08 -0400
Received: from reverse-83.fdn.fr ([80.67.176.83]:34952 helo=pluto)
 by fencepost.gnu.org with esmtpsa (TLS1.0:RSA_AES_128_CBC_SHA1:16)
 (Exim 4.71) (envelope-from <ludo@gnu.org>)
 id 1Ynwco-0004OI-Km; Thu, 30 Apr 2015 18:08:07 -0400
From: ludo@gnu.org (Ludovic Courtès)
To: Joshua Randall <jcrandall@alum.mit.edu>
Subject: Re: bug#20402: guix cannot download via an http proxy
References: <CAFBLGSNKzhWGjCiheo8W5sc=tzmcjw6PkczxsDizVw-nKwnSzQ@mail.gmail.com>
 <CAFBLGSN8Gu5-=NsFuHumomtG80jNO94Sjc9AsuSZvkT9R46eEg@mail.gmail.com>
X-URL: http://www.fdn.fr/~lcourtes/
X-Revolutionary-Date: 11 Floréal an 223 de la Révolution
X-PGP-Key-ID: 0x3D9AEBB5
X-PGP-Key: http://www.fdn.fr/~lcourtes/ludovic.asc
X-PGP-Fingerprint: 3CE4 6455 8A84 FDC6 9DB4  0CFB 090B 1199 3D9A EBB5
X-OS: x86_64-unknown-linux-gnu
Date: Fri, 01 May 2015 00:07:59 +0200
In-Reply-To: <CAFBLGSN8Gu5-=NsFuHumomtG80jNO94Sjc9AsuSZvkT9R46eEg@mail.gmail.com>
 (Joshua Randall's message of "Wed, 22 Apr 2015 02:15:58 +0100")
Message-ID: <87r3r1feww.fsf@gnu.org>
User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/24.5 (gnu/linux)
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable
X-Spam-Score: -5.0 (-----)
X-Debbugs-Envelope-To: 20402
Cc: 20402@debbugs.gnu.org
X-BeenThere: debbugs-submit@debbugs.gnu.org
X-Mailman-Version: 2.1.15
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request@debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit@debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request@debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request@debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces@debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces@debbugs.gnu.org>
X-Spam-Score: -5.0 (-----)

Hi,

Sorry for the delay, and thanks for the investigation and patch!

Joshua Randall <jcrandall@alum.mit.edu> skribis:

> This patch modifies http-fetch (in build/download.scm) such that it calls
> Guile's open-socket-for-uri after fixing up the uri so that it always has a
> port specified. I'm not sure how to test the bootstrapping NSS issue that
> required open-connection-for-uri, but my expectation based on reading the
> source is that this alternative should work for Guile > 2.0.7, and I've
> left the original open-connection-for-uri in there for backwards
> compatibility with Guile < 2.0.7. If someone can test this against a
> situation known to have needed the NSS workaround, that would be great.

To name lookup with the bootstrap Guile, one way is to run this:

  $ guix gc -d $(guix build -S -e '(@@ (gnu packages commencement) glibc-final)')
  $ ./pre-inst-env guix build -S \
     -e '(@@ (gnu packages commencement) glibc-final)' --no-substitutes

The second command here uses the bootstrap Guile.

Another approach is this:

--8<---------------cut here---------------start------------->8---
scheme@(guile-user)> ,use(gnu packages bootstrap)
scheme@(guile-user)> ,enter-store-monad
store-monad@(guile-user) [1]> (mlet %store-monad ((guile (package->derivation %bootstrap-guile)))
				(gexp->derivation "foo"
						  #~(begin
						      (mkdir #$output)
						      (pk (getaddrinfo "www.gnu.org" "http")))
						  #:hash-algo 'sha256
						  #:hash (base32 "0mdqa9w1p6cmli6976v4wi0sw9r4p5prkj7lzfd1877wk11c9c73")
						  #:guile-for-build guile))
$5 = #<derivation /gnu/store/5prfiblj6ddziavk9nz31bkgy8jzaghx-foo.drv => /gnu/store/xf3404zw9kfx6a1gcfk6lmqcx6a53ad5-foo 2ae7960>
store-monad@(guile-user) [1]> (built-derivations (list $5))
building path(s) `/gnu/store/xf3404zw9kfx6a1gcfk6lmqcx6a53ad5-foo'

;;; ((#(0 2 1 6 #(2 3497454484 80) #f) #(0 2 2 17 #(2 3497454484 80) #f) #(0 10 1 6 #(10 42541952298791455573290623124440612874 80 0 0) #f) #(0 10 2 17 #(10 42541952298791455573290623124440612874 80 0 0) #f)))
--8<---------------cut here---------------end--------------->8---

Here ‘built-derivations’ fails but the build log shows that
‘getaddrinfo’ succeeded.

Lastly, one can extract
gnu/packages/bootstrap/x86_64-linux/guile-2.0.9.tar.xz and run:

  strace -o log ./bin/guile -c '(getaddrinfo "www.gnu.org" "http")'

Here the log shows that /etc/nsswitch.conf, /etc/services, and
/etc/hosts are accessed and things just work; it does not attempt to
connect to the nscd.

A bit of archeology shows the following timeline:

  1. d14ecda introduces the ‘open-connection-for-uri’ hack (Oct. 2012).

  2. d3b5972 changes libc used to make bootstrap tarballs to use static
     NSS modules (Jan. 2013).

  3. 0621349 updates the bootstrap guile-2.0.9.tar.xz tarballs
     (Nov. 2013), meaning that our current bootstrap Guile indeed uses
     static NSS modules and doesn’t attempt to talk to nscd.

In other words, the hack is no longer needed.

Thus, ‘open-connection-for-uri’ is almost (see below) unneeded now,
which simplifies the solution to the problem you raise.

> I've also changed the only other call to open-connection-for-uri, which is
> in the probe-uri function in scripts/lint.scm - my suspicion is that won't
> be an issue because I'm guessing the lint scripts are not used while
> bootstrapping, so the open-socket-for-uri function will probably be fine
> for the purpose of probing whether a URL is valid.

‘open-connection-for-uri’ also handles TLS connections, which are also
useful for ‘guix lint’, so we cannot completely get rid of it.

Commit d17551d simplifies it so that it is just a wrapper around
‘open-socket-for-uri’.  After that, ‘guix download’ honors $http_proxy.

$https_proxy is not handled yet because that requires more work, and I
do not fully understand how that is supposed to work.  Patch welcome,
though.  :-)

Thanks!

Ludo’.

Send a report that this bug log contains spam.