Acknowledgement sent
to Joshua Randall <jcrandall@alum.mit.edu>:
New bug report received and forwarded. Copy sent to bug-guix@gnu.org.
(Tue, 21 Apr 2015 23:01:02 GMT) (full text, mbox, link).
Package: guix
Version: 0.8.1
I am attempting to use guix from within a network that does not allow
outbound http connections except via an http proxy. I am using Guile
v2.0.11, which supports http proxies, so my expectation would be that since
I have http_proxy and https_proxy environment variables set, guix would use
the specified proxy for outbound http connections, but instead it appears
to ignore the proxy settings and attempts to contact the http server
directly, which results in a timeout.
For example, when doing a `guix pull` I got the following:
$ guix pull
starting download of `/tmp/guix-file.l1zwZ7' from `
http://git.savannah.gnu.org/cgit/guix.git/snapshot/guix-master.tar.gz'...
ERROR: In procedure connect: Connection timed out
failed to download "/tmp/guix-file.l1zwZ7" from "
http://git.savannah.gnu.org/cgit/guix.git/snapshot/guix-master.tar.gz"
guix pull: error: failed to download up-to-date source, exiting
It appears that Guile has had support for http proxies in the web client
package since v2.0.10, and although guix is using the http-get method from
Guile, it isn't using the open-socket-for-uri method, which is the one that
implements making a proxy connection. Instead, guix seems to have copied
and modified the code from an older version of open-socket-for-uri
into open-connection-for-uri
(
http://git.savannah.gnu.org/cgit/guix.git/tree/guix/build/download.scm?id=v0.8.1#n153)
and uses that instead. I suspect what has happened is that the Guile
version of open-socket-for-uri has added proxy support since the code was
copied into open-connection-for-uri. One fix would be to port over the
changes to open-socket-for-uri that were made in Guile 2.0.10.
However, it appears from the code comment that that the reason
open-connection-for-uri copies the functionality of open-socket-for-uri is
to avoid NSS lookups for symbolic port arguments, and it looks to me that
since version 2.0.7 of Guile, its open-socket-for-uri can be convinced not
to do NSS lookups as long as (uri-port uri) is not #f (see
http://git.savannah.gnu.org/cgit/guile.git/tree/module/web/client.scm?id=v2.0.7#n53
).
Rather than porting the new code from Guile's open-socket-for-uri, it might
make more sense to just call open-socket-for-uri with a uri that always has
a port (i.e. implement the same hard-coding for http and https in the
http-fetch function to make sure that uri has the default port set - I
notice for some reason Guile's string->uri parser does not set the port for
http and https even though it has the default ports for both set in the
code. I suppose one could use the existing post-2.0.7? test to keep calling
open-connection-for-uri for backwards compatibility with old versions of
Guile (which in any case don't have proxy support, so for my use case it
doesn't matter).
I can try to put together a patch that implements this fix, although I
haven't written scheme in quite a while, so someone else may be better
suited for it.
Cheers,
Josh.
Tags: patch
Please find a proposed patch that should fix this issue. I haven't written
scheme in many years, so please excuse me if I have failed to follow
convention in any way - I won't be offended at all if you want to refactor
it. This patch should apply cleanly on the current git master branch and
also on the v0.8.1 release.
This patch modifies http-fetch (in build/download.scm) such that it calls
Guile's open-socket-for-uri after fixing up the uri so that it always has a
port specified. I'm not sure how to test the bootstrapping NSS issue that
required open-connection-for-uri, but my expectation based on reading the
source is that this alternative should work for Guile > 2.0.7, and I've
left the original open-connection-for-uri in there for backwards
compatibility with Guile < 2.0.7. If someone can test this against a
situation known to have needed the NSS workaround, that would be great.
I've also changed the only other call to open-connection-for-uri, which is
in the probe-uri function in scripts/lint.scm - my suspicion is that won't
be an issue because I'm guessing the lint scripts are not used while
bootstrapping, so the open-socket-for-uri function will probably be fine
for the purpose of probing whether a URL is valid.
After applying this patch, guix should use the http proxy support built in
to Guile >= 2.0.10. This appears to be working for me - I've just started a
bootstrap and it has now started to successfully download packages - so
far, so good.
Please let me know if there is anything else I can do to assist with
getting this issue fixed.
Cheers,
Josh.
After more testing this morning, I see that although the core functionality
is working after applying the proposed patch (i.e. the guix daemon itself
can download things), the derivations are still failing because the
environment is being squashed and so the http_proxy and https_proxy
environment variables are not available to the build. It looks like nix has
a way around this involving setting imputeEnvVars = "http_proxy
https_proxy" so that those variables get copied into the build environment
(for fixed-output derivations only, which makes sense to me).
I'll try to come up with a patch that inserts the imputeEnvVars, but I'm
getting farther from the code I understand so I will probably need some
help in getting it in the right place.
Cheers,
Josh.
On Wed, Apr 22, 2015 at 2:15 AM, Joshua Randall <jcrandall@alum.mit.edu>
wrote:
> Tags: patch
>
> Please find a proposed patch that should fix this issue. I haven't written
> scheme in many years, so please excuse me if I have failed to follow
> convention in any way - I won't be offended at all if you want to refactor
> it. This patch should apply cleanly on the current git master branch and
> also on the v0.8.1 release.
>
> This patch modifies http-fetch (in build/download.scm) such that it calls
> Guile's open-socket-for-uri after fixing up the uri so that it always has a
> port specified. I'm not sure how to test the bootstrapping NSS issue that
> required open-connection-for-uri, but my expectation based on reading the
> source is that this alternative should work for Guile > 2.0.7, and I've
> left the original open-connection-for-uri in there for backwards
> compatibility with Guile < 2.0.7. If someone can test this against a
> situation known to have needed the NSS workaround, that would be great.
>
> I've also changed the only other call to open-connection-for-uri, which is
> in the probe-uri function in scripts/lint.scm - my suspicion is that won't
> be an issue because I'm guessing the lint scripts are not used while
> bootstrapping, so the open-socket-for-uri function will probably be fine
> for the purpose of probing whether a URL is valid.
>
> After applying this patch, guix should use the http proxy support built in
> to Guile >= 2.0.10. This appears to be working for me - I've just started a
> bootstrap and it has now started to successfully download packages - so
> far, so good.
>
> Please let me know if there is anything else I can do to assist with
> getting this issue fixed.
>
> Cheers,
>
> Josh.
>
>
Tags: patch
I've updated my patch to also modify guix/derivations.scm to add a setting
for nix's impureEnvVars, such that it will pass http_proxy and https_proxy
variables from the guix daemon to the build processes (which nix will do
only for fixed-output derivations).
I believe this is working, but I've now run into a third problem: the
bootstrap guile is only v2.0.9, and v.2.0.10 or later is required for proxy
support. What would I need to do to use a later version of bootstrap guile?
Cheers,
Josh.
Subject: Re: bug#20402: guix cannot download via an http proxy
Date: Fri, 01 May 2015 00:07:59 +0200
Hi,
Sorry for the delay, and thanks for the investigation and patch!
Joshua Randall <jcrandall@alum.mit.edu> skribis:
> This patch modifies http-fetch (in build/download.scm) such that it calls
> Guile's open-socket-for-uri after fixing up the uri so that it always has a
> port specified. I'm not sure how to test the bootstrapping NSS issue that
> required open-connection-for-uri, but my expectation based on reading the
> source is that this alternative should work for Guile > 2.0.7, and I've
> left the original open-connection-for-uri in there for backwards
> compatibility with Guile < 2.0.7. If someone can test this against a
> situation known to have needed the NSS workaround, that would be great.
To name lookup with the bootstrap Guile, one way is to run this:
$ guix gc -d $(guix build -S -e '(@@ (gnu packages commencement) glibc-final)')
$ ./pre-inst-env guix build -S \
-e '(@@ (gnu packages commencement) glibc-final)' --no-substitutes
The second command here uses the bootstrap Guile.
Another approach is this:
--8<---------------cut here---------------start------------->8---
scheme@(guile-user)> ,use(gnu packages bootstrap)
scheme@(guile-user)> ,enter-store-monad
store-monad@(guile-user) [1]> (mlet %store-monad ((guile (package->derivation %bootstrap-guile)))
(gexp->derivation "foo"
#~(begin
(mkdir #$output)
(pk (getaddrinfo "www.gnu.org" "http")))
#:hash-algo 'sha256
#:hash (base32 "0mdqa9w1p6cmli6976v4wi0sw9r4p5prkj7lzfd1877wk11c9c73")
#:guile-for-build guile))
$5 = #<derivation /gnu/store/5prfiblj6ddziavk9nz31bkgy8jzaghx-foo.drv => /gnu/store/xf3404zw9kfx6a1gcfk6lmqcx6a53ad5-foo 2ae7960>
store-monad@(guile-user) [1]> (built-derivations (list $5))
building path(s) `/gnu/store/xf3404zw9kfx6a1gcfk6lmqcx6a53ad5-foo'
;;; ((#(0 2 1 6 #(2 3497454484 80) #f) #(0 2 2 17 #(2 3497454484 80) #f) #(0 10 1 6 #(10 42541952298791455573290623124440612874 80 0 0) #f) #(0 10 2 17 #(10 42541952298791455573290623124440612874 80 0 0) #f)))
--8<---------------cut here---------------end--------------->8---
Here ‘built-derivations’ fails but the build log shows that
‘getaddrinfo’ succeeded.
Lastly, one can extract
gnu/packages/bootstrap/x86_64-linux/guile-2.0.9.tar.xz and run:
strace -o log ./bin/guile -c '(getaddrinfo "www.gnu.org" "http")'
Here the log shows that /etc/nsswitch.conf, /etc/services, and
/etc/hosts are accessed and things just work; it does not attempt to
connect to the nscd.
A bit of archeology shows the following timeline:
1. d14ecda introduces the ‘open-connection-for-uri’ hack (Oct. 2012).
2. d3b5972 changes libc used to make bootstrap tarballs to use static
NSS modules (Jan. 2013).
3. 0621349 updates the bootstrap guile-2.0.9.tar.xz tarballs
(Nov. 2013), meaning that our current bootstrap Guile indeed uses
static NSS modules and doesn’t attempt to talk to nscd.
In other words, the hack is no longer needed.
Thus, ‘open-connection-for-uri’ is almost (see below) unneeded now,
which simplifies the solution to the problem you raise.
> I've also changed the only other call to open-connection-for-uri, which is
> in the probe-uri function in scripts/lint.scm - my suspicion is that won't
> be an issue because I'm guessing the lint scripts are not used while
> bootstrapping, so the open-socket-for-uri function will probably be fine
> for the purpose of probing whether a URL is valid.
‘open-connection-for-uri’ also handles TLS connections, which are also
useful for ‘guix lint’, so we cannot completely get rid of it.
Commit d17551d simplifies it so that it is just a wrapper around
‘open-socket-for-uri’. After that, ‘guix download’ honors $http_proxy.
$https_proxy is not handled yet because that requires more work, and I
do not fully understand how that is supposed to work. Patch welcome,
though. :-)
Thanks!
Ludo’.
Information forwarded
to bug-guix@gnu.org: bug#20402; Package guix.
(Thu, 30 Apr 2015 22:13:01 GMT) (full text, mbox, link).
Subject: Re: bug#20402: updated patch to pass http_proxy and https_proxy env
vars to derivation builds
Date: Fri, 01 May 2015 00:12:42 +0200
Joshua Randall <jcrandall@alum.mit.edu> skribis:
> I've updated my patch to also modify guix/derivations.scm to add a setting
> for nix's impureEnvVars, such that it will pass http_proxy and https_proxy
> variables from the guix daemon to the build processes (which nix will do
> only for fixed-output derivations).
I took a slightly different approach in commit c046815 (and also not
passing “impureEnvVars” by default since that would lead to a full
rebuild.)
With commit 0d88313, $http_proxy is honored, but note that it’s the
$http_proxy value in the daemon’s environment that is used, not in the
client’s environment.
> I believe this is working, but I've now run into a third problem: the
> bootstrap guile is only v2.0.9, and v.2.0.10 or later is required for proxy
> support. What would I need to do to use a later version of bootstrap guile?
We will need to update the bootstrap Guile, but that won’t happen until
a later full-rebuild cycle.
You should be able to use substitutes as a workaround in the meantime.
How does that sound?
Thank you!
Ludo’.
Information forwarded
to bug-guix@gnu.org: bug#20402; Package guix.
(Tue, 09 Jun 2015 01:50:04 GMT) (full text, mbox, link).
Subject: Re: guix cannot download via an http proxy
Date: Tue, 9 Jun 2015 03:22:38 +0200
Would support for $ftp_proxy as well require the implementation of new
functionality, or would it be a case of just letting the environment
variable through?
Information forwarded
to bug-guix@gnu.org: bug#20402; Package guix.
(Wed, 10 Jun 2015 02:27:02 GMT) (full text, mbox, link).
To: Claes Wallin (韋嘉誠) <gnu@clacke.user.lysator.liu.se>
Cc: 20402@debbugs.gnu.org
Subject: Re: bug#20402: guix cannot download via an http proxy
Date: Tue, 09 Jun 2015 22:26:38 -0400
Claes Wallin (韋嘉誠) <gnu@clacke.user.lysator.liu.se> writes:
> Would support for $ftp_proxy as well require the implementation of new
> functionality, or would it be a case of just letting the environment
> variable through?
It would require new functionality. Our FTP client would have to be
enhanced to support proxies.
Mark
Information forwarded
to bug-guix@gnu.org: bug#20402; Package guix.
(Sat, 13 Jun 2015 17:45:03 GMT) (full text, mbox, link).
To: Claes Wallin (韋嘉誠)
<gnu@clacke.user.lysator.liu.se>
Cc: 20402@debbugs.gnu.org
Subject: Re: HTTP and FTP proxies for guix package / guix-daemon
Date: Sat, 13 Jun 2015 19:44:44 +0200
(Please always keep the bug email address Cc’d.)
"Claes Wallin (韋嘉誠)" <gnu@clacke.user.lysator.liu.se> skribis:
> I suppose if I have a freshly built guix with "guix package -i guix"
> or "guix package -u guix" it should be using guile 2.0.11 then?
> Actually looking at the head of .../bin/guix it's referring to a
> guile-2.0.11.
Yes, so that should be OK. And it’s Guix > 0.8.2, right?
> But guix-daemon still doesn't seem to be honoring http_proxy.
As a first test, could you check whether ‘guix download’ honors it?
A second test would be:
1. Define ‘http_proxy’ in your user shell.
2. From the same shell, run ‘guix build -S gcc --no-substitutes’, say.
Does that honor ‘http_proxy’?
And a third test is for substitutes: Are they also downloaded via the
proxy? Here you need to:
1. Define ‘http_proxy’ in a root shell.
2. Run ‘guix-daemon’ from this shell.
3. From a user shell, run ‘guix build gcc’, say, which should normally
download a substitute from hydra.gnu.org rather than trying to build
things.
Does the substitute download honor ‘http_proxy’?
Thanks in advance!
Ludo’.
Information forwarded
to bug-guix@gnu.org: bug#20402; Package guix.
(Wed, 17 Jun 2015 16:50:04 GMT) (full text, mbox, link).
Subject: Re: HTTP and FTP proxies for guix package / guix-daemon
Date: Wed, 17 Jun 2015 18:46:17 +0200
Finally got around to this.
On Sat, Jun 13, 2015 at 7:44 PM, Ludovic Courtès <ludo@gnu.org> wrote:
> "Claes Wallin (韋嘉誠)" <gnu@clacke.user.lysator.liu.se> skribis:
>
>> I suppose if I have a freshly built guix with "guix package -i guix"
>> or "guix package -u guix" it should be using guile 2.0.11 then?
>> Actually looking at the head of .../bin/guix it's referring to a
>> guile-2.0.11.
>
> Yes, so that should be OK. And it’s Guix > 0.8.2, right?
# /root/.guix-profile/bin/guix --version
guix (GNU Guix) 0.8.3
# ls -l /root/.guix-profile/bin/guix
lrwxrwxrwx 3 root 999 71 Jan 1 1970 /root/.guix-profile/bin/guix ->
/gnu/store/bja9w08pay8rqfs83f3rzlms9q63kymw-guix-0.8.2.a43b55f/bin/guix
>> But guix-daemon still doesn't seem to be honoring http_proxy.
>
> As a first test, could you check whether ‘guix download’ honors it?
It does.
> A second test would be:
>
> 1. Define ‘http_proxy’ in your user shell.
> 2. From the same shell, run ‘guix build -S gcc --no-substitutes’, say.
>
> Does that honor ‘http_proxy’?
http_proxy is set (and exported) in user shell and in the daemon process.
No.
> And a third test is for substitutes: Are they also downloaded via the
> proxy? Here you need to:
>
> 1. Define ‘http_proxy’ in a root shell.
> 2. Run ‘guix-daemon’ from this shell.
> 3. From a user shell, run ‘guix build gcc’, say, which should normally
> download a substitute from hydra.gnu.org rather than trying to build
> things.
>
> Does the substitute download honor ‘http_proxy’?
No. Whenever the daemon is doing the downloading, http_proxy is
ignored and I get "Temporary failure in name resolution" because the
VM doesn't have an internet DNS.
As I said earlier, guix has guile-2.0.11 in the hashbang. But
guix-daemon seems to be making use of guile-bootstrap.
If I provoke it a bit I get this:
# find /gnu/store/cyppdaliwllm1mzgka3cxwxnddk7j2cl-guile-bootstrap-2.0/
-name '*.scm' -exec touch {} +
# guix package -i texinfo
. . .
;;; note: source file
/gnu/store/cyppdaliwllm1mzgka3cxwxnddk7j2cl-guile-bootstrap-2.0/share/guile/2.0/ice-9/eval.scm
;;; newer than compiled
/gnu/store/cyppdaliwllm1mzgka3cxwxnddk7j2cl-guile-bootstrap-2.0/lib/guile/2.0/ccache/ice-9/eval.go
;;; note: auto-compilation is enabled, set GUILE_AUTO_COMPILE=0
;;; or pass the --no-auto-compile argument to disable.
. . .
;;; it seems /gnu/store/cyppdaliwllm1mzgka3cxwxnddk7j2cl-guile-bootstrap-2.0/share/guile/2.0/ice-9/receive.scm
;;; is part of the compiler; skipping auto-compilation
;;; WARNING: compilation of
/gnu/store/cyppdaliwllm1mzgka3cxwxnddk7j2cl-guile-bootstrap-2.0/share/guile/2.0/ice-9/command-line.scm
failed:
;;; ERROR: failed to create path for auto-compiled file
"/gnu/store/cyppdaliwllm1mzgka3cxwxnddk7j2cl-guile-bootstrap-2.0/share/guile/2.0/ice-9/command-line.scm"
starting download of
`/gnu/store/2zkyyw4bq0wzsxdysncrf9lmwl44w5wh-binutils-2.25.tar.bz2'
from `http://ftpmirror.gnu.org/binutils/binutils-2.25.tar.bz2'...
ERROR: In procedure getaddrinfo: Temporary failure in name resolution
... so it's pretty clear that *something* in there is still
referencing the bootstrap guile. But maybe that's expected? Are the
guile packages built using only the bootstrap and not themselves?
/gnu/store/0aq1kq37b6kr562330xz61rg08l6y5yg-guix-0.8.2.a43b55f.drv
has:
Derive([("out","/gnu/store/bja9w08pay8rqfs83f3rzlms9q63kymw-guix-0.8.2.a43b55f","","")]
. . . ("/gnu/store/4lkrvcnh2y923la3ivsbyv2m2nv3xpjy-guile-2.0.9.tar.xz.drv",["out"])
. . . ("/gnu/store/d3x2skfxhvqj21w39f6bckmxni1q3bbp-guile-2.0.11.tar.xz.drv",["out"])
. . . ("/gnu/store/hvl3gwa57p7k1qnliyy6ymbmz2509azh-guile-2.0.11.drv",["out"])
. . . ("/gnu/store/p4i1inysxx5ra0hvhcrd9liw3a5skcbx-guile-2.0.9.tar.xz.drv",["out"])
. . . ("/gnu/store/wlivhx78f1a05dx0blqy209cb1nqwgf5-guile-2.0.9.tar.xz.drv",["out"])
. . . ["/gnu/store/iylwyjzxcy3hkz1wlgpyk4zmhci97rgk-guix-0.8.2.a43b55f-guile-builder"],"x86_64-linux","/gnu/store/cnqmkmj40jmssnx6fkf9n0n3bqj5x426-guile-2.0.11/bin/guile",["--no-auto-compile","-L","/gnu/store/x2nfxrqhdsvir8l65x19bhnrwd4h41rs-module-import","/gnu/store/iylwyjzxcy3hkz1wlgpyk4zmhci97rgk-guix-0.8.2.a43b55f-guile-builder"],[("GUILE_LOAD_COMPILED_PATH","/gnu/store/07n184fk33b7gqkyrbn6b2ckdfisp09i-module-import-compiled"),("out","/gnu/store/bja9w08pay8rqfs83f3rzlms9q63kymw-guix-0.8.2.a43b55f")])
Why so many different guile drivers? Is that expected?
> Thanks in advance!
No problem!
Information forwarded
to bug-guix@gnu.org: bug#20402; Package guix.
(Wed, 17 Jun 2015 16:52:02 GMT) (full text, mbox, link).
Subject: Re: HTTP and FTP proxies for guix package / guix-daemon
Date: Wed, 17 Jun 2015 18:51:06 +0200
On Wed, Jun 17, 2015 at 6:46 PM, Claes Wallin (韋嘉誠)
<gnu@clacke.user.lysator.liu.se> wrote:
> Why so many different guile drivers? Is that expected?
... derivations. Sorry.
Information forwarded
to bug-guix@gnu.org: bug#20402; Package guix.
(Wed, 01 Jul 2015 08:20:04 GMT) (full text, mbox, link).
To: Claes Wallin (韋嘉誠)
<gnu@clacke.user.lysator.liu.se>
Cc: 20402@debbugs.gnu.org
Subject: Re: HTTP and FTP proxies for guix package / guix-daemon
Date: Wed, 01 Jul 2015 10:19:14 +0200
"Claes Wallin (韋嘉誠)" <gnu@clacke.user.lysator.liu.se> skribis:
>>> But guix-daemon still doesn't seem to be honoring http_proxy.
>>
>> As a first test, could you check whether ‘guix download’ honors it?
>
> It does.
>
>> A second test would be:
>>
>> 1. Define ‘http_proxy’ in your user shell.
>> 2. From the same shell, run ‘guix build -S gcc --no-substitutes’, say.
>>
>> Does that honor ‘http_proxy’?
>
> http_proxy is set (and exported) in user shell and in the daemon process.
Oops, my instructions were incorrect: ‘http_proxy’ needs to be defined
in the daemon process.
To test, I first run the daemon like this:
sudo sh -c 'export http_proxy=foobar ; ./pre-inst-env guix-daemon --build-users-group=guixbuild'
and then I run:
--8<---------------cut here---------------start------------->8---
$ ./pre-inst-env guix build -S gcc --no-substitutes
accepted connection from pid 14399, user ludo
La jenaj derivoj estos konstruataj:
/gnu/store/s32s498y88sidfnrppsn33nfmhlacpzx-gcc-5.1.0.tar.xz.drv
/gnu/store/z1n2c3hgl7w2cnz9d5szyjwwy8lp0dav-gcc-5.1.0.tar.bz2.drv
@ build-started /gnu/store/z1n2c3hgl7w2cnz9d5szyjwwy8lp0dav-gcc-5.1.0.tar.bz2.drv - x86_64-linux /var/log/guix/drvs/z1//n2c3hgl7w2cnz9d5szyjwwy8lp0dav-gcc-5.1.0.tar.bz2.drv.bz2
starting download of `/gnu/store/6a3d39111w0d5n7rzb5p7aliilw0hx1l-gcc-5.1.0.tar.bz2' from `http://ftpmirror.gnu.org/gcc/gcc-5.1.0/gcc-5.1.0.tar.bz2'...
ERROR: In procedure struct_vtable: Wrong type argument in position 1 (expecting struct): #f
starting download of `/gnu/store/6a3d39111w0d5n7rzb5p7aliilw0hx1l-gcc-5.1.0.tar.bz2' from `ftp://ftp.cs.tu-berlin.de/pub/gnu/gcc/gcc-5.1.0/gcc-5.1.0.tar.bz2'...
ftp://ftp.cs.tu-berlin.de/.../gcc-5.1.0.tar.bz2 3.9% of 92728.9 KiB (610. KiB/s) C-c C-cSIGPOLL
--8<---------------cut here---------------end--------------->8---
Here the error message comes from the fact that $http_proxy is a
malformed URL, showing that it *is* honored. Since the http URL fails,
the download logic falls back to the next URL, which happens to be an
ftp URL, which is unaffected by ‘http_proxy’.
So it’s working as expected here.
>> And a third test is for substitutes: Are they also downloaded via the
>> proxy? Here you need to:
>>
>> 1. Define ‘http_proxy’ in a root shell.
>> 2. Run ‘guix-daemon’ from this shell.
>> 3. From a user shell, run ‘guix build gcc’, say, which should normally
>> download a substitute from hydra.gnu.org rather than trying to build
>> things.
>>
>> Does the substitute download honor ‘http_proxy’?
This test also works as expected here:
--8<---------------cut here---------------start------------->8---
$ ./pre-inst-env guix build -S inkscape
accepted connection from pid 15118, user ludo
La jena derivo estos konstruata:
/gnu/store/fa6ss1ljwb97ns75332nshskj2bv0dh4-inkscape-0.91.tar.gz.drv
La jenaj derivoj estos elŝutataj:
/gnu/store/acmzy3nkms38g6vrphh07bczf5s5mbjd-gnutls-3.4.1
/gnu/store/vx82jvj1xbppnphl8nwjdr7cm7f5bk59-nettle-3.1.1
@ substituter-started /gnu/store/vx82jvj1xbppnphl8nwjdr7cm7f5bk59-nettle-3.1.1 /home/ludo/src/guix/nix/scripts/substitute
found valid signature for '/gnu/store/vx82jvj1xbppnphl8nwjdr7cm7f5bk59-nettle-3.1.1', from 'http://hydra.gnu.org/nar/vx82jvj1xbppnphl8nwjdr7cm7f5bk59-nettle-3.1.1'
downloading `/gnu/store/vx82jvj1xbppnphl8nwjdr7cm7f5bk59-nettle-3.1.1' (1.7 MiB installed)...
[...]
In guix/scripts/substitute.scm:
182: 2 [#<procedure 1523210 at guix/scripts/substitute.scm:165:9 ()>]
In guix/build/download.scm:
212: 1 [open-socket-for-uri #]
In web/client.scm:
73: 0 [open-socket-for-uri #]
web/client.scm:73:16: In procedure open-socket-for-uri:
web/client.scm:73:16: In procedure struct_vtable: Wrong type argument in position 1 (expecting struct): #f
@ substituter-failed /gnu/store/vx82jvj1xbppnphl8nwjdr7cm7f5bk59-nettle-3.1.1 256 fetching path `/gnu/store/vx82jvj1xbppnphl8nwjdr7cm7f5bk59-nettle-3.1.1' failed with exit code 1
guix build: error: build failed: some substitutes for the outputs of derivation `/gnu/store/2v6lls3f8dj1rkpram4jnl8ckqs77r6m-gnutls-3.4.1.drv' failed (usually happens due to networking issues); try `--fallback' to build derivation from source
--8<---------------cut here---------------end--------------->8---
The backtrace again stems from the bogus $http_proxy that I chose (not
nice, but shows that $http_proxy is honored.)
Please let me know if you have evidence that something works differently
for you.
> No. Whenever the daemon is doing the downloading, http_proxy is
> ignored and I get "Temporary failure in name resolution" because the
> VM doesn't have an internet DNS.
[...]
> `/gnu/store/2zkyyw4bq0wzsxdysncrf9lmwl44w5wh-binutils-2.25.tar.bz2'
> from `http://ftpmirror.gnu.org/binutils/binutils-2.25.tar.bz2'...
> ERROR: In procedure getaddrinfo: Temporary failure in name resolution
That’s another, unrelated issue, probably in your network or on your
machine.
Thanks,
Ludo’.
Information forwarded
to bug-guix@gnu.org: bug#20402; Package guix.
(Wed, 01 Jul 2015 13:09:02 GMT) (full text, mbox, link).
Subject: Re: HTTP and FTP proxies for guix package / guix-daemon
Date: Wed, 1 Jul 2015 15:08:31 +0200
On Wed, Jul 1, 2015 at 10:19 AM, Ludovic Courtès <ludo@gnu.org> wrote:
> To test, I first run the daemon like this:
>
> sudo sh -c 'export http_proxy=foobar ; ./pre-inst-env guix-daemon --build-users-group=guixbuild'
Could there be a difference because you are testing with the
checked-out, semi-manually built guix and I was checking with a
freshly packaged (pull, package -i guix) guix? Like I said in
http://lists.gnu.org/archive/html/bug-guix/2015-06/msg00036.html ,
guix-daemon as compiled by guix seems to be using guile-2.0.9. Maybe
that only happens due to guix rewriting hashbangs etc, and on a build
outside guix (even when using guix environment) it uses whatever guile
happens to be in the paths?
When I have the time again I will check it exactly the way you are doing it.
> Here the error message comes from the fact that $http_proxy is a
> malformed URL, showing that it *is* honored. Since the http URL fails,
> the download logic falls back to the next URL, which happens to be an
> ftp URL, which is unaffected by ‘http_proxy’.
Ok, that's maybe a clearer way of testing it.
> Please let me know if you have evidence that something works differently
> for you.
I would say that both of us have evidence that it's working and not
working. The conclusion must be that we are not testing the exact same
program.
>> No. Whenever the daemon is doing the downloading, http_proxy is
>> ignored and I get "Temporary failure in name resolution" because the
>> VM doesn't have an internet DNS.
>
> [...]
>
>> `/gnu/store/2zkyyw4bq0wzsxdysncrf9lmwl44w5wh-binutils-2.25.tar.bz2'
>> from `http://ftpmirror.gnu.org/binutils/binutils-2.25.tar.bz2'...
>> ERROR: In procedure getaddrinfo: Temporary failure in name resolution
>
> That’s another, unrelated issue, probably in your network or on your
> machine.
Yes. This is why the proxy is required. There is no internet DNS for
the VM, because it is on an isolated lab network.
When it does go through the proxy, like when running guix download, no
name lookup is done and everything works.
Information forwarded
to bug-guix@gnu.org: bug#20402; Package guix.
(Wed, 01 Jul 2015 14:46:02 GMT) (full text, mbox, link).
To: Claes Wallin (韋嘉誠)
<gnu@clacke.user.lysator.liu.se>
Cc: 20402@debbugs.gnu.org
Subject: Re: HTTP and FTP proxies for guix package / guix-daemon
Date: Wed, 01 Jul 2015 16:45:15 +0200
"Claes Wallin (韋嘉誠)" <gnu@clacke.user.lysator.liu.se> skribis:
> On Wed, Jul 1, 2015 at 10:19 AM, Ludovic Courtès <ludo@gnu.org> wrote:
>
>> To test, I first run the daemon like this:
>>
>> sudo sh -c 'export http_proxy=foobar ; ./pre-inst-env guix-daemon --build-users-group=guixbuild'
>
> Could there be a difference because you are testing with the
> checked-out, semi-manually built guix and I was checking with a
> freshly packaged (pull, package -i guix) guix? Like I said in
> http://lists.gnu.org/archive/html/bug-guix/2015-06/msg00036.html ,
> guix-daemon as compiled by guix seems to be using guile-2.0.9. Maybe
> that only happens due to guix rewriting hashbangs etc, and on a build
> outside guix (even when using guix environment) it uses whatever guile
> happens to be in the paths?
Could you send the store file name of Guix that is being used?
For instance, I have:
--8<---------------cut here---------------start------------->8---
$ sudo deco status guix-daemon
Pasvorto:
Status of guix-daemon:
It is started.
Running value is 31316.
It is enabled.
Provides (guix-daemon).
Requires (user-processes).
Conflicts with ().
Will be respawned.
$ ps 31316
PID TTY STAT TIME COMMAND
31316 ? Ss 0:00 /gnu/store/zg6j61inmmk5kyqgyfjnj7rdxl79i9ps-guix-0.8.2.684bf7c/bin/guix-daemon --build-users-group guixbu
--8<---------------cut here---------------end--------------->8---
That means I’m using commit 684bf7c of Guix, which has the fix, which
came up after the http_proxy commit (commit 0d88313.)
>>> No. Whenever the daemon is doing the downloading, http_proxy is
>>> ignored and I get "Temporary failure in name resolution" because the
>>> VM doesn't have an internet DNS.
>>
>> [...]
>>
>>> `/gnu/store/2zkyyw4bq0wzsxdysncrf9lmwl44w5wh-binutils-2.25.tar.bz2'
>>> from `http://ftpmirror.gnu.org/binutils/binutils-2.25.tar.bz2'...
>>> ERROR: In procedure getaddrinfo: Temporary failure in name resolution
>>
>> That’s another, unrelated issue, probably in your network or on your
>> machine.
>
> Yes. This is why the proxy is required. There is no internet DNS for
> the VM, because it is on an isolated lab network.
Hmm, when using Guile 2.0.11 or later, ‘open-socket-for-uri’ connects to
the proxy (so its name must be available in DNS lookup), not to the
target server. The code in (web client) goes like this:
--8<---------------cut here---------------start------------->8---
(define (open-socket-for-uri uri-or-string)
"Return an open input/output port for a connection to URI."
(define http-proxy (current-http-proxy))
(define uri (ensure-uri (or http-proxy uri-or-string))) ;← proxy
(define addresses
(let ((port (uri-port uri)))
(delete-duplicates
(getaddrinfo (uri-host uri)
(cond (port => number->string)
(else (symbol->string (uri-scheme uri))))
(if port
AI_NUMERICSERV
0))
;; ...
--8<---------------cut here---------------end--------------->8---
Do you see the same DNS issue when running:
http_proxy=http://... guix download http://...
?
Thanks,
Ludo’.
Information forwarded
to bug-guix@gnu.org: bug#20402; Package guix.
(Wed, 01 Jul 2015 14:54:02 GMT) (full text, mbox, link).
Ludovic, sorry for duplicate mail again.
On Jul 1, 2015 4:45 PM, "Ludovic Courtès" <ludo@gnu.org> wrote:
> > Yes. This is why the proxy is required. There is no internet DNS for
> > the VM, because it is on an isolated lab network.
>
> Hmm, when using Guile 2.0.11 or later, ‘open-socket-for-uri’ connects to
> the proxy (so its name must be available in DNS lookup), not to the
> target server. The code in (web client) goes like this:
The proxy is in the DNS.
> Do you see the same DNS issue when running:
>
> http_proxy=http://... guix download http://...
Nope. guix download works. It successfully looks up and connects to the
proxy.
When I next have the chance I will report:
1) Results with guix-compiled guix-daemon
2) Results with dev-compiled guix-daemon
3) /gnu/store hash of 1)
On Jul 1, 2015 4:45 PM, "Ludovic Courtès" <ludo@gnu.org> wrote:
> "Claes Wallin (韋嘉誠)" <gnu@clacke.user.lysator.liu.se> skribis:
>
> > On Wed, Jul 1, 2015 at 10:19 AM, Ludovic Courtès <ludo@gnu.org> wrote:
> >
> >> To test, I first run the daemon like this:
> >>
> >> sudo sh -c 'export http_proxy=foobar ; ./pre-inst-env guix-daemon
> --build-users-group=guixbuild'
> >
> > Could there be a difference because you are testing with the
> > checked-out, semi-manually built guix and I was checking with a
> > freshly packaged (pull, package -i guix) guix? Like I said in
> > http://lists.gnu.org/archive/html/bug-guix/2015-06/msg00036.html ,
> > guix-daemon as compiled by guix seems to be using guile-2.0.9. Maybe
> > that only happens due to guix rewriting hashbangs etc, and on a build
> > outside guix (even when using guix environment) it uses whatever guile
> > happens to be in the paths?
>
> Could you send the store file name of Guix that is being used?
> For instance, I have:
>
> --8<---------------cut here---------------start------------->8---
> $ sudo deco status guix-daemon
> Pasvorto:
> Status of guix-daemon:
> It is started.
> Running value is 31316.
> It is enabled.
> Provides (guix-daemon).
> Requires (user-processes).
> Conflicts with ().
> Will be respawned.
> $ ps 31316
> PID TTY STAT TIME COMMAND
> 31316 ? Ss 0:00
> /gnu/store/zg6j61inmmk5kyqgyfjnj7rdxl79i9ps-guix-0.8.2.684bf7c/bin/guix-daemon
> --build-users-group guixbu
> --8<---------------cut here---------------end--------------->8---
>
> That means I’m using commit 684bf7c of Guix, which has the fix, which
> came up after the http_proxy commit (commit 0d88313.)
>
> >>> No. Whenever the daemon is doing the downloading, http_proxy is
> >>> ignored and I get "Temporary failure in name resolution" because the
> >>> VM doesn't have an internet DNS.
> >>
> >> [...]
> >>
> >>> `/gnu/store/2zkyyw4bq0wzsxdysncrf9lmwl44w5wh-binutils-2.25.tar.bz2'
> >>> from `http://ftpmirror.gnu.org/binutils/binutils-2.25.tar.bz2'...
> >>> ERROR: In procedure getaddrinfo: Temporary failure in name resolution
> >>
> >> That’s another, unrelated issue, probably in your network or on your
> >> machine.
> >
> > Yes. This is why the proxy is required. There is no internet DNS for
> > the VM, because it is on an isolated lab network.
>
> Hmm, when using Guile 2.0.11 or later, ‘open-socket-for-uri’ connects to
> the proxy (so its name must be available in DNS lookup), not to the
> target server. The code in (web client) goes like this:
>
> --8<---------------cut here---------------start------------->8---
> (define (open-socket-for-uri uri-or-string)
> "Return an open input/output port for a connection to URI."
> (define http-proxy (current-http-proxy))
> (define uri (ensure-uri (or http-proxy uri-or-string))) ;← proxy
> (define addresses
> (let ((port (uri-port uri)))
> (delete-duplicates
> (getaddrinfo (uri-host uri)
> (cond (port => number->string)
> (else (symbol->string (uri-scheme uri))))
> (if port
> AI_NUMERICSERV
> 0))
> ;; ...
> --8<---------------cut here---------------end--------------->8---
>
> Do you see the same DNS issue when running:
>
> http_proxy=http://... guix download http://...
>
> ?
>
> Thanks,
> Ludo’.
>
Debbugs is free software and licensed under the terms of the
GNU Public License version 2. The current version can be
obtained from https://bugs.debian.org/debbugs-source/.