Report forwarded
to bug-guix@gnu.org: bug#26136; Package guix.
(Fri, 17 Mar 2017 01:47:02 GMT) (full text, mbox, link).
Acknowledgement sent
to Leo Famulari <leo@famulari.name>:
New bug report received and forwarded. Copy sent to bug-guix@gnu.org.
(Fri, 17 Mar 2017 01:47:02 GMT) (full text, mbox, link).
python-minimal@2.7.13 is failing to build on core-updates:
https://hydra.gnu.org/build/1912761
I saw this near the beginning of the install phase:
------
configure: error: cannot run /bin/sh
/tmp/guix-build-python-minimal-2.7.13.drv-0/Python-2.7.13/Modules/_ctypes/libffi/config.sub
Failed to configure _ctypes module
Python build finished, but the necessary bits to build these modules
were not found:
_bsddb _curses _curses_panel
_sqlite3 _ssl _tkinter
bsddb185 dbm dl
gdbm imageop readline
sunaudiodev zlib
To find the necessary bits, look in setup.py in detect_modules() for the
module's name.
------
And at the end, it fails:
------
if test "xinstall" != "xno" ; then \
case install in \
upgrade) ensurepip="--upgrade" ;; \
install|*) ensurepip="" ;; \
esac; \
LD_LIBRARY_PATH=/tmp/guix-build-python-minimal-2.7.13.drv-0/Python-2.7.13 ./python -E -m ensurepip \
$ensurepip --root=/ ; \
fi
Traceback (most recent call last):
File "/tmp/guix-build-python-minimal-2.7.13.drv-0/Python-2.7.13/Lib/runpy.py", line 174, in _run_module_as_main
"__main__", fname, loader, pkg_name)
File "/tmp/guix-build-python-minimal-2.7.13.drv-0/Python-2.7.13/Lib/runpy.py", line 72, in _run_code
exec code in run_globals
File "/tmp/guix-build-python-minimal-2.7.13.drv-0/Python-2.7.13/Lib/ensurepip/__main__.py", line 4, in <module>
ensurepip._main()
File "/tmp/guix-build-python-minimal-2.7.13.drv-0/Python-2.7.13/Lib/ensurepip/__init__.py", line 205, in _main
default_pip=args.default_pip,
File "/tmp/guix-build-python-minimal-2.7.13.drv-0/Python-2.7.13/Lib/ensurepip/__init__.py", line 108, in bootstrap
_run_pip(args + [p[0] for p in _PROJECTS], additional_paths)
File "/tmp/guix-build-python-minimal-2.7.13.drv-0/Python-2.7.13/Lib/ensurepip/__init__.py", line 31, in _run_pip
import pip
zipimport.ZipImportError: can't decompress data; zlib not available
make: *** [Makefile:927: install] Error 1
phase `install' failed after 29.1 seconds
------
From: Danny Milosavljevic <dannym@scratchpost.org>
To: Leo Famulari <leo@famulari.name>
Cc: 26136@debbugs.gnu.org
Subject: Re: bug#26136: core-updates: python-minimal@2.7.13 fails due to
missing zlib
Date: Fri, 17 Mar 2017 15:08:51 +0100
Hmmm... Python 2.7.13 seems to bundle libffi-3.1. Do we want that?
Also, I checked libffi/configure: If CONFIG_SHELL was set, it would have picked it up.
Also, why does Python 2.7.13 try to configure things in the *install* phase ?
Information forwarded
to bug-guix@gnu.org: bug#26136; Package guix.
(Sat, 18 Mar 2017 17:27:02 GMT) (full text, mbox, link).
On Fri, Mar 17, 2017 at 03:08:51PM +0100, Danny Milosavljevic wrote:
> Hmmm... Python 2.7.13 seems to bundle libffi-3.1. Do we want that?
The python-minimal package *should* use that libffi. I'm not sure why
it's failing now.
> Also, I checked libffi/configure: If CONFIG_SHELL was set, it would have picked it up.
Do you mean if CONFIG_SHELL is not set, so the build process can't find
libffi?
There is also the question about zlib and the bundled pip: why is it
failing now?
I haven't yet found the changes between 2.7.12 and 2.7.13 that are
causing these issues.
From: Danny Milosavljevic <dannym@scratchpost.org>
To: Leo Famulari <leo@famulari.name>
Cc: 26136@debbugs.gnu.org
Subject: Re: bug#26136: core-updates: python-minimal@2.7.13 fails due to
missing zlib
Date: Sat, 18 Mar 2017 21:19:01 +0100
Hi Leo,
>The python-minimal package *should* use that libffi. I'm not sure why it's failing now.
Yeah, sorry. I checked the package definition now and it explicitly disables the system libffi.
>Do you mean if CONFIG_SHELL is not set, so the build process can't find libffi?
Yes, a workaround for the first problem is:
diff --git a/gnu/packages/python.scm b/gnu/packages/python.scm
index be452f062..978aeca47 100644
--- a/gnu/packages/python.scm
+++ b/gnu/packages/python.scm
@@ -192,7 +192,8 @@
'("Lib/subprocess.py"
"Lib/popen2.py"
"Lib/distutils/tests/test_spawn.py"
- "Lib/test/test_subprocess.py"))
+ "Lib/test/test_subprocess.py"
+ "Modules/_ctypes/libffi/configure"))
(("/bin/sh") (which "sh")))
;; Use zero as the timestamp in .pyc files so that builds are
The reason why that helps is because the configure script contains embedded HERE scripts with their own shebangs which patch-shebang doesn't patch.
An alternative workaround is to use this instead:
diff --git a/gnu/packages/python.scm b/gnu/packages/python.scm
index be452f062..04c944d84 100644
--- a/gnu/packages/python.scm12/bin/shore-updates/guix$ /gnu/store/k7029k5va68lka
+++ b/gnu/packages/python.scm
@@ -229,6 +229,8 @@
;; before 1980". Work around this by setting the file times in the
;; source tree to sometime in early 1980.
(lambda _
+ (setenv "CONFIG_SHELL" (which "bash"))
+
(let ((circa-1980 (* 10 366 24 60 60)))
(ftw "." (lambda (file stat flag)
(utime file circa-1980 circa-1980)
>There is also the question about zlib and the bundled pip: why is it failing now?
The above doesn't fix the pip zlib problem. It's very strange: The value of the environment variable C_INCLUDE_PATH includes bzip2 but not zlib. Might make python unable to find zlib.
setup.py does some strange stuff with reading the zlib.h header file manually - maybe that broke. Then it tries to find the library file for libz. The value of the environment variable LIBRARY_PATH doesn't contain zlib either.
Information forwarded
to bug-guix@gnu.org: bug#26136; Package guix.
(Sat, 18 Mar 2017 20:52:01 GMT) (full text, mbox, link).
From: Danny Milosavljevic <dannym@scratchpost.org>
To: Leo Famulari <leo@famulari.name>
Cc: 26136@debbugs.gnu.org
Subject: Re: bug#26136: core-updates: python-minimal@2.7.13 fails due to
missing zlib
Date: Sat, 18 Mar 2017 21:50:57 +0100
Setting the environment variables
CPPFLAGS="-I/gnu/store/y8c62fyb2x4g4qnl54pmlmid1fn8k3mx-zlib-1.2.8/include"
LDFLAGS="-L/gnu/store/y8c62fyb2x4g4qnl54pmlmid1fn8k3mx-zlib-1.2.8/lib"
before running configure makes it work.
(I've also printed inc_dirs in setup.py without that - and zlib didn't appear)
So I guess the general question is how does Python set up inc_dirs ?
And is it normal that C_INCLUDE_PATH and LIBRARY_PATH doesn't contain zlib ? Apparently, since guix-master python2-minimal 2.7.12 doesn't contain it either - and that one works...
So where would it get the other entries?
Information forwarded
to bug-guix@gnu.org: bug#26136; Package guix.
(Sun, 19 Mar 2017 19:21:02 GMT) (full text, mbox, link).
On Sat, Mar 18, 2017 at 09:19:01PM +0100, Danny Milosavljevic wrote:
> An alternative workaround is to use this instead:
>
> diff --git a/gnu/packages/python.scm b/gnu/packages/python.scm
> index be452f062..04c944d84 100644
> --- a/gnu/packages/python.scm12/bin/shore-updates/guix$ /gnu/store/k7029k5va68lka
> +++ b/gnu/packages/python.scm
> @@ -229,6 +229,8 @@
> ;; before 1980". Work around this by setting the file times in the
> ;; source tree to sometime in early 1980.
> (lambda _
> + (setenv "CONFIG_SHELL" (which "bash"))
> +
I think this is the better solution to the libffi / ctypes issue. It
also fixes python-minimal@3 to use the bundled libffi. We added libffi
to python-minimal@3's inputs in
d0b73960db051baa468fc1a0fbab21fa9f89f885 to work around a similar issue.
In Python 3.7, the bundled libffi will be removed, and we will have to
build our minimal Python-3 variant with an external libffi package:
https://bugs.python.org/issue27976
Here's a patch that fixes this problem for python-minimal@2. I don't
want to change python-minimal@3 this late in the core-updates cycle; we
can do it in a python-updates branch after core-updates.
Subject: Re: bug#26136: core-updates: python-minimal@2.7.13 fails due to
missing zlib
Date: Sun, 19 Mar 2017 15:26:36 -0400
On Sat, Mar 18, 2017 at 09:19:01PM +0100, Danny Milosavljevic wrote:
> >There is also the question about zlib and the bundled pip: why is it
> >failing now?
>
> The above doesn't fix the pip zlib problem. It's very strange: The
> value of the environment variable C_INCLUDE_PATH includes bzip2 but
> not zlib. Might make python unable to find zlib.
>
> setup.py does some strange stuff with reading the zlib.h header file
> manually - maybe that broke. Then it tries to find the library file
> for libz. The value of the environment variable LIBRARY_PATH doesn't
> contain zlib either.
Ideally, the minimal Python variants could use the bundled zlib found at
'Modules/zlib'. We sort of gave up on this with python-minimal@3 the
last time we had this problem, but maybe we can solve it now :)
Information forwarded
to bug-guix@gnu.org: bug#26136; Package guix.
(Sun, 19 Mar 2017 20:16:02 GMT) (full text, mbox, link).
On Sun, Mar 19, 2017 at 03:26:36PM -0400, Leo Famulari wrote:
> On Sat, Mar 18, 2017 at 09:19:01PM +0100, Danny Milosavljevic wrote:
> > >There is also the question about zlib and the bundled pip: why is it
> > >failing now?
Here are some build logs showing differences between master and
core-updates.
For 2.7.12, on the master branch, there is this:
------
if test "xinstall" != "xno" ; then \
case install in \
upgrade) ensurepip="--upgrade" ;; \
install|*) ensurepip="" ;; \
esac; \
LD_LIBRARY_PATH=/tmp/guix-build-python-minimal-2.7.12.drv-0/Python-2.7.12 ./python -E -m ensurepip \
$ensurepip --root=/ ; \
fi
Ignoring ensurepip failure: pip 8.1.1 requires SSL/TLS
phase `install' succeeded after 31.5 seconds
------
I guess we didn't notice the lack of TLS support because the bundled pip
is never able to reach the internet when used to build packages.
For 2.7.13 without zlib, on core-updates, it does this:
------
if test "xinstall" != "xno" ; then \
case install in \
upgrade) ensurepip="--upgrade" ;; \
install|*) ensurepip="" ;; \
esac; \
LD_LIBRARY_PATH=/tmp/guix-build-python-minimal-2.7.13.drv-0/Python-2.7.13 ./python -E -m ensurepip \
$ensurepip --root=/ ; \
fi
Traceback (most recent call last):
File "/tmp/guix-build-python-minimal-2.7.13.drv-0/Python-2.7.13/Lib/runpy.py", line 174, in _run_module_as_main
"__main__", fname, loader, pkg_name)
File "/tmp/guix-build-python-minimal-2.7.13.drv-0/Python-2.7.13/Lib/runpy.py", line 72, in _run_code
exec code in run_globals
File "/tmp/guix-build-python-minimal-2.7.13.drv-0/Python-2.7.13/Lib/ensurepip/__main__.py", line 4, in <module>
ensurepip._main()
File "/tmp/guix-build-python-minimal-2.7.13.drv-0/Python-2.7.13/Lib/ensurepip/__init__.py", line 205, in _main
default_pip=args.default_pip,
File "/tmp/guix-build-python-minimal-2.7.13.drv-0/Python-2.7.13/Lib/ensurepip/__init__.py", line 108, in bootstrap
_run_pip(args + [p[0] for p in _PROJECTS], additional_paths)
File "/tmp/guix-build-python-minimal-2.7.13.drv-0/Python-2.7.13/Lib/ensurepip/__init__.py", line 31, in _run_pip
import pip
zipimport.ZipImportError: can't decompress data; zlib not available
make: *** [Makefile:927: install] Error 1
phase `install' failed after 10.8 seconds
------
If you build 2.7.13 with zlib on core-updates (log not *not* attached):
------
if test "xinstall" != "xno" ; then \ case install in \
upgrade) ensurepip="--upgrade" ;; \
install|*) ensurepip="" ;; \
esac; \ LD_LIBRARY_PATH=/tmp/guix-build-python-minimal-2.7.13.drv-0/Python-2.7.13 ./python -E -m ensurepip \ $ensurepip --root=/ ; \ fi
Collecting setuptools Collecting pip Installing collected packages: setuptools, pip Successfully installed pip-9.0.1 setuptools-28.8.0
phase `install' succeeded after 21.6 seconds
------
Subject: Re: bug#26136: core-updates: python-minimal@2.7.13 fails due to
missing zlib
Date: Mon, 20 Mar 2017 00:02:52 +0100
ludo@gnu.org (Ludovic Courtès) skribis:
> From 1cb73a54813136632553f03063abdd5cd105a760 Mon Sep 17 00:00:00 2001
> From: =?UTF-8?q?Ludovic=20Court=C3=A8s?= <ludo@gnu.org>
> Date: Sun, 19 Mar 2017 22:10:12 +0100
> Subject: [PATCH] gnu: python-minimal@2: Add dependencies on libffi and zlib.
>
> * gnu/packages/python.scm (python2-minimal)[inputs]: Add LIBFFI and ZLIB.
> [arguments]: Remove.
Pushed as 5927a843b4a91416b5be9e65c69616447f82e162 in core-updates,
following our discussion on IRC.
Thanks,
Ludo’.
Added tag(s) fixed.
Request was from ludo@gnu.org (Ludovic Courtès)
to control@debbugs.gnu.org.
(Sun, 26 Mar 2017 13:43:02 GMT) (full text, mbox, link).
bug closed, send any further explanations to
26136@debbugs.gnu.org and Leo Famulari <leo@famulari.name>
Request was from ludo@gnu.org (Ludovic Courtès)
to control@debbugs.gnu.org.
(Sun, 26 Mar 2017 13:43:02 GMT) (full text, mbox, link).
bug archived.
Request was from Debbugs Internal Request <help-debbugs@gnu.org>
to internal_control@debbugs.gnu.org.
(Mon, 24 Apr 2017 11:24:04 GMT) (full text, mbox, link).
Debbugs is free software and licensed under the terms of the
GNU Public License version 2. The current version can be
obtained from https://bugs.debian.org/debbugs-source/.