#22745 guix http downloads don't resume

Message #5 received at submit@debbugs.gnu.org (full text, mbox, reply):

From: Danny Milosavljevic <dannym@scratchpost.org>

To: bug-guix@gnu.org

Subject: guix http downloads don't resume

Date: Sat, 20 Feb 2016 09:34:45 +0100

Package: guix

By now, guix reconfigure has tried to download the same texlive binary from hydra at least 5 times to the same machine, unsuccessfully breaking after about 1 GB each, for a total of 5 GB, always starting from the beginning.

Would it be possible to just resume?

Additionally, it doesn't seem like it checks the Content-Length in order to find out whether the connection broke before the file was done. Why doesn't it?

The http-client seems to handle chunked encoding - so not sure whether it's a problem with hydra or with guix.

Message #8 received at 22745@debbugs.gnu.org (full text, mbox, reply):

From: Danny Milosavljevic <dannym@scratchpost.org>

To: 22745@debbugs.gnu.org

Subject: error messages

Date: Sat, 20 Feb 2016 09:44:42 +0100

----
Found valid signature for /gnu/store/36wqhbch45x055wj6gfsng00zkwfqg6n-texlive-20150523-texmf.tar.xz
From http://hydra.gnu.org/nar/36wqhbch45x055wj6gfsng00zkwfqg6n-texlive-20150523-texmf.tar.xz
Downloading 36wqhb...-texlive-20150523-texmf.tar.xz (1.76GiB installed)...
 http://hydra.gnu.org/nar/36wqhbch45x055wj6gfsng00zkwfqg6n-texlive-20150523-texmf.tar.xz 1.1MiB/s 13:24 | 893.9MiB transferred
bzip2: Data integrity error when decompressing.
 http://Input file = (stdin), output file = (stdout)wfqg6n-texlive-20150523-texmf.tar.xz 1.1MiB/s 13:24 | 893.9MiB transferred

It is possible that the compressed file(s) have become corrupted.
You can use the -tvv option to test integrity of such files.

You can use the `bzip2recover' program to attempt to recover
data from undamaged sections of corrupted files.

guix substitute: error: corrupt input while restoring '/gnu/store/36wqhbch45x055wj6gfsng00zkwfqg6n-texlive-20150523-texmf.tar.xz' from #{read pipe}#
killing process 3867g/nar/36wqhbch45x055wj6gfsng00zkwfqg6n-texlive-20150523-texmf.tar.xz 1.1MiB/s 13:24 | 893.9MiB transferred
guix system: error: build failed: some substitutes for the outputs of derivation `/gnu/store/83nkdyp9wl6zwflcm416xf1imppp7v9f-texlive-20150523-texmf.tar.xz.drv' failed (usually happens due to networking issues); try `--fallback' to build derivation from source 
---

Also, downloading it by wget right afterwards, it works just fine, all 1.77 GB of it. Wtf?

Message #11 received at 22745@debbugs.gnu.org (full text, mbox, reply):

From: ludo@gnu.org (Ludovic Courtès)

To: Danny Milosavljevic <dannym@scratchpost.org>

Cc: 22745@debbugs.gnu.org

Subject: Re: bug#22745: guix http downloads don't resume

Date: Fri, 25 Mar 2016 09:35:12 +0100

Danny Milosavljevic <dannym@scratchpost.org> skribis:

> By now, guix reconfigure has tried to download the same texlive binary from hydra at least 5 times to the same machine, unsuccessfully breaking after about 1 GB each, for a total of 5 GB, always starting from the beginning.
>
> Would it be possible to just resume?

There does not seem to be an easy way to achieve this.

> Additionally, it doesn't seem like it checks the Content-Length in order to find out whether the connection broke before the file was done. Why doesn't it?

For archives (the /nar/foo URLs), there is currently no ‘Content-Length’
header at all (this is because Hydra, the software, generates those
files on the fly; we should tweak nginx to add a ‘Content-Length’
header, but this seems to require an external nginx plugin.)

However, the {mirror.,}hydra.gnu.org use chunked encoding, indeed, which
allows the HTTP client to detect truncated transfers.

Ludo’.

Message #16 received at 22745@debbugs.gnu.org (full text, mbox, reply):

From: swedebugia@riseup.net

To: 22745@debbugs.gnu.org

Subject: guix http downloads don't resume

Date: Mon, 17 Dec 2018 01:25:57 -0800

Hi

Is this still a problem after Pierres work on texlive and when using the
mirror?

-- 
Cheers 
Swedebugia

Send a report that this bug log contains spam.