#20402 guix cannot download via an http proxy

Message #38 received at 20402@debbugs.gnu.org (full text, mbox, reply):

Received: (at 20402) by debbugs.gnu.org; 1 Jul 2015 08:19:31 +0000
From debbugs-submit-bounces@debbugs.gnu.org Wed Jul 01 04:19:31 2015
Received: from localhost ([127.0.0.1]:35161 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.80)
	(envelope-from <debbugs-submit-bounces@debbugs.gnu.org>)
	id 1ZADEw-0004cE-B8
	for submit@debbugs.gnu.org; Wed, 01 Jul 2015 04:19:30 -0400
Received: from eggs.gnu.org ([208.118.235.92]:53757)
 by debbugs.gnu.org with esmtp (Exim 4.80)
 (envelope-from <ludo@gnu.org>) id 1ZADEu-0004c0-AT
 for 20402@debbugs.gnu.org; Wed, 01 Jul 2015 04:19:28 -0400
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
 (envelope-from <ludo@gnu.org>) id 1ZADEm-0000TQ-6Z
 for 20402@debbugs.gnu.org; Wed, 01 Jul 2015 04:19:22 -0400
X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on eggs.gnu.org
X-Spam-Level: 
X-Spam-Status: No, score=-0.6 required=5.0 tests=BAYES_20,RP_MATCHES_RCVD
 autolearn=disabled version=3.3.2
Received: from fencepost.gnu.org ([2001:4830:134:3::e]:38820)
 by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from <ludo@gnu.org>)
 id 1ZADEm-0000TK-3G; Wed, 01 Jul 2015 04:19:20 -0400
Received: from reverse-83.fdn.fr ([80.67.176.83]:35915 helo=pluto)
 by fencepost.gnu.org with esmtpsa (TLS1.2:RSA_AES_128_CBC_SHA1:128)
 (Exim 4.82) (envelope-from <ludo@gnu.org>)
 id 1ZADEl-0003MT-Du; Wed, 01 Jul 2015 04:19:19 -0400
From: ludo@gnu.org (Ludovic Courtès)
To: Claes Wallin (韋嘉誠)
 <gnu@clacke.user.lysator.liu.se>
Subject: Re: HTTP and FTP proxies for guix package / guix-daemon
References: <CAGv_=BodrMu03sbaBVWu0ey+AwKLR9QJ6JzQRW=SX3M7-A5MAw@mail.gmail.com>
 <CAGv_=Bo7WVgbt7-J7M0SSf37xFM4mHXYGwLS4harcjDzzJqsiA@mail.gmail.com>
 <CAGv_=BqrQqCqGKMQz=N6DX9qDGyQz57=3eYBDhbVrewGD2kBew@mail.gmail.com>
 <87381zicgg.fsf@gnu.org>
 <CAGv_=BriLNSHy2MgPioh_hv4xQJSyVXKniOCVTgOsdUTGpU2eA@mail.gmail.com>
 <87616uzfd8.fsf@gnu.org>
 <CAGv_=Bo3rdx1GQ3n7mZrT6io9giiSPU0g8SLDZ_BgNVWJrmuPg@mail.gmail.com>
 <87d20zbj03.fsf@gnu.org>
 <CAGv_=BqVzDp6GGVE=J7Ej0pg3=tVEPU2u3DiwCVPgwNaRuyLsA@mail.gmail.com>
X-URL: http://www.fdn.fr/~lcourtes/
X-Revolutionary-Date: 13 Messidor an 223 de la Révolution
X-PGP-Key-ID: 0x3D9AEBB5
X-PGP-Key: http://www.fdn.fr/~lcourtes/ludovic.asc
X-PGP-Fingerprint: 3CE4 6455 8A84 FDC6 9DB4  0CFB 090B 1199 3D9A EBB5
X-OS: x86_64-unknown-linux-gnu
Date: Wed, 01 Jul 2015 10:19:14 +0200
In-Reply-To: <CAGv_=BqVzDp6GGVE=J7Ej0pg3=tVEPU2u3DiwCVPgwNaRuyLsA@mail.gmail.com>
 ("Claes Wallin \=\?utf-8\?B\?KOmfi+WYieiqoCkiJ3M\=\?\= message of "Wed, 17 Jun
 2015 18:46:17 +0200")
Message-ID: <87pp4cxpbx.fsf@gnu.org>
User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/24.5 (gnu/linux)
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable
X-detected-operating-system: by eggs.gnu.org: Error: Malformed IPv6 address
 (bad octet value).
X-Received-From: 2001:4830:134:3::e
X-Spam-Score: -5.6 (-----)
X-Debbugs-Envelope-To: 20402
Cc: 20402@debbugs.gnu.org
X-BeenThere: debbugs-submit@debbugs.gnu.org
X-Mailman-Version: 2.1.15
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request@debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit@debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request@debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request@debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces@debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces@debbugs.gnu.org>
X-Spam-Score: -5.6 (-----)

"Claes Wallin (韋嘉誠)" <gnu@clacke.user.lysator.liu.se> skribis:

>>> But guix-daemon still doesn't seem to be honoring http_proxy.
>>
>> As a first test, could you check whether ‘guix download’ honors it?
>
> It does.
>
>> A second test would be:
>>
>>   1. Define ‘http_proxy’ in your user shell.
>>   2. From the same shell, run ‘guix build -S gcc --no-substitutes’, say.
>>
>> Does that honor ‘http_proxy’?
>
> http_proxy is set (and exported) in user shell and in the daemon process.

Oops, my instructions were incorrect: ‘http_proxy’ needs to be defined
in the daemon process.

To test, I first run the daemon like this:

  sudo sh -c 'export http_proxy=foobar ; ./pre-inst-env guix-daemon --build-users-group=guixbuild'

and then I run:

--8<---------------cut here---------------start------------->8---
$ ./pre-inst-env guix build -S gcc --no-substitutes
accepted connection from pid 14399, user ludo
La jenaj derivoj estos konstruataj:
   /gnu/store/s32s498y88sidfnrppsn33nfmhlacpzx-gcc-5.1.0.tar.xz.drv
   /gnu/store/z1n2c3hgl7w2cnz9d5szyjwwy8lp0dav-gcc-5.1.0.tar.bz2.drv
@ build-started /gnu/store/z1n2c3hgl7w2cnz9d5szyjwwy8lp0dav-gcc-5.1.0.tar.bz2.drv - x86_64-linux /var/log/guix/drvs/z1//n2c3hgl7w2cnz9d5szyjwwy8lp0dav-gcc-5.1.0.tar.bz2.drv.bz2
starting download of `/gnu/store/6a3d39111w0d5n7rzb5p7aliilw0hx1l-gcc-5.1.0.tar.bz2' from `http://ftpmirror.gnu.org/gcc/gcc-5.1.0/gcc-5.1.0.tar.bz2'...
ERROR: In procedure struct_vtable: Wrong type argument in position 1 (expecting struct): #f
starting download of `/gnu/store/6a3d39111w0d5n7rzb5p7aliilw0hx1l-gcc-5.1.0.tar.bz2' from `ftp://ftp.cs.tu-berlin.de/pub/gnu/gcc/gcc-5.1.0/gcc-5.1.0.tar.bz2'...
ftp://ftp.cs.tu-berlin.de/.../gcc-5.1.0.tar.bz2	  3.9% of 92728.9 KiB (610. KiB/s)  C-c C-cSIGPOLL
--8<---------------cut here---------------end--------------->8---

Here the error message comes from the fact that $http_proxy is a
malformed URL, showing that it *is* honored.  Since the http URL fails,
the download logic falls back to the next URL, which happens to be an
ftp URL, which is unaffected by ‘http_proxy’.

So it’s working as expected here.

>> And a third test is for substitutes: Are they also downloaded via the
>> proxy?  Here you need to:
>>
>>   1. Define ‘http_proxy’ in a root shell.
>>   2. Run ‘guix-daemon’ from this shell.
>>   3. From a user shell, run ‘guix build gcc’, say, which should normally
>>      download a substitute from hydra.gnu.org rather than trying to build
>>      things.
>>
>> Does the substitute download honor ‘http_proxy’?

This test also works as expected here:

--8<---------------cut here---------------start------------->8---
$ ./pre-inst-env guix build -S inkscape
accepted connection from pid 15118, user ludo
La jena derivo estos konstruata:
   /gnu/store/fa6ss1ljwb97ns75332nshskj2bv0dh4-inkscape-0.91.tar.gz.drv
La jenaj derivoj estos elŝutataj:
   /gnu/store/acmzy3nkms38g6vrphh07bczf5s5mbjd-gnutls-3.4.1
   /gnu/store/vx82jvj1xbppnphl8nwjdr7cm7f5bk59-nettle-3.1.1
@ substituter-started /gnu/store/vx82jvj1xbppnphl8nwjdr7cm7f5bk59-nettle-3.1.1 /home/ludo/src/guix/nix/scripts/substitute
found valid signature for '/gnu/store/vx82jvj1xbppnphl8nwjdr7cm7f5bk59-nettle-3.1.1', from 'http://hydra.gnu.org/nar/vx82jvj1xbppnphl8nwjdr7cm7f5bk59-nettle-3.1.1'
downloading `/gnu/store/vx82jvj1xbppnphl8nwjdr7cm7f5bk59-nettle-3.1.1' (1.7 MiB installed)...

[...]

In guix/scripts/substitute.scm:
 182: 2 [#<procedure 1523210 at guix/scripts/substitute.scm:165:9 ()>]
In guix/build/download.scm:
 212: 1 [open-socket-for-uri #]
In web/client.scm:
  73: 0 [open-socket-for-uri #]

web/client.scm:73:16: In procedure open-socket-for-uri:
web/client.scm:73:16: In procedure struct_vtable: Wrong type argument in position 1 (expecting struct): #f
@ substituter-failed /gnu/store/vx82jvj1xbppnphl8nwjdr7cm7f5bk59-nettle-3.1.1 256 fetching path `/gnu/store/vx82jvj1xbppnphl8nwjdr7cm7f5bk59-nettle-3.1.1' failed with exit code 1
guix build: error: build failed: some substitutes for the outputs of derivation `/gnu/store/2v6lls3f8dj1rkpram4jnl8ckqs77r6m-gnutls-3.4.1.drv' failed (usually happens due to networking issues); try `--fallback' to build derivation from source
--8<---------------cut here---------------end--------------->8---

The backtrace again stems from the bogus $http_proxy that I chose (not
nice, but shows that $http_proxy is honored.)

Please let me know if you have evidence that something works differently
for you.

> No. Whenever the daemon is doing the downloading, http_proxy is
> ignored and I get "Temporary failure in name resolution" because the
> VM doesn't have an internet DNS.

[...]

> `/gnu/store/2zkyyw4bq0wzsxdysncrf9lmwl44w5wh-binutils-2.25.tar.bz2'
> from `http://ftpmirror.gnu.org/binutils/binutils-2.25.tar.bz2'...
> ERROR: In procedure getaddrinfo: Temporary failure in name resolution

That’s another, unrelated issue, probably in your network or on your
machine.

Thanks,
Ludo’.

Send a report that this bug log contains spam.