[PATCH 0/3] Use system nss-certs in Python.

  • Done
  • quality assurance status badge
Details
3 participants
  • Efraim Flashner
  • Lars-Dominik Braun
  • Leo Famulari
Owner
unassigned
Submitted by
Efraim Flashner
Severity
normal

Debbugs page

E
E
Efraim Flashner wrote on 3 Apr 22:55 -0700
(address . guix-patches@gnu.org)(name . Efraim Flashner)(address . efraim@flashner.co.il)
cover.1712210069.git.efraim@flashner.co.il
It turns out that the Python ecosystem bundles a version of nss-certs.
This patch series should change it so that it uses the system nss-certs
instead.

Efraim Flashner (3):
gnu: python-certifi: Use system SSL certificates.
gnu: python-pip: Use system SSL certificates.
gnu: python: Use system SSL certificates.

gnu/packages/python-build.scm | 34 +++++++++++++++++
gnu/packages/python-crypto.scm | 34 +++++++++++++++++
gnu/packages/python.scm | 67 ++++++++++++++++++++++++++++++++++
3 files changed, 135 insertions(+)


base-commit: 188d18fc47f0d38edfe06e3e5834fa8587bd300b
--
Efraim Flashner <efraim@flashner.co.il> רנשלפ םירפא
GPG key = A28B F40C 3E55 1372 662D 14F7 41AA E7DC CA3D 8351
Confidentiality cannot be guaranteed on emails sent or received unencrypted
E
E
Efraim Flashner wrote on 3 Apr 22:56 -0700
[PATCH v2 1/3] gnu: python-certifi: Use system SSL certificates.
(address . 70179@debbugs.gnu.org)(name . Efraim Flashner)(address . efraim@flashner.co.il)
aaf898f5494ec9206216be228b84712e459f074f.1712210069.git.efraim@flashner.co.il
* gnu/packages/python-crypto.scm (python-certifi)[replacement]: New
field.
(python-certifi/fixed): Provide a python-certifi which only offers to
use the system's SSL certificates.

Change-Id: Ie1871be42988dff3cccfe24bca626149fee0f371
---
gnu/packages/python-crypto.scm | 34 ++++++++++++++++++++++++++++++++++
1 file changed, 34 insertions(+)

Toggle diff (56 lines)
diff --git a/gnu/packages/python-crypto.scm b/gnu/packages/python-crypto.scm
index 3e1472a6c9..05b6c82fd9 100644
--- a/gnu/packages/python-crypto.scm
+++ b/gnu/packages/python-crypto.scm
@@ -469,6 +469,7 @@ (define-public python-certifi
(package
(name "python-certifi")
(version "2022.6.15")
+ (replacement python-certifi/fixed)
(source (origin
(method url-fetch)
(uri (pypi-uri "certifi" version))
@@ -484,6 +485,39 @@ (define-public python-certifi
is used by the Requests library to verify HTTPS requests.")
(license license:asl2.0)))
+(define python-certifi/fixed
+ (package
+ (inherit python-certifi)
+ (source (origin
+ (inherit (package-source python-certifi))
+ (snippet
+ #~(begin
+ (delete-file "certifi/cacert.pem")
+ (delete-file "certifi/core.py")
+ (with-output-to-file "certifi/core.py"
+ (lambda _
+ (display "\"\"\"
+certifi.py
+~~~~~~~~~~
+This module returns the installation location of SSL_CERT_FILE or
+/etc/ssl/certs/ca-certificates.crt, or its contents.
+\"\"\"
+import os
+
+_CA_CERTS = None
+
+try:
+ _CA_CERTS = os.environ [\"SSL_CERT_FILE\"]
+except:
+ _CA_CERTS = os.path.join(\"/etc\", \"ssl\", \"certs\", \"ca-certificates.crt\")
+
+def where() -> str:
+ return _CA_CERTS
+
+def contents() -> str:
+ with open(where(), \"r\", encoding=\"ascii\") as data:
+ return data.read()")))))))))
+
(define-public python-cryptography-vectors
(package
(name "python-cryptography-vectors")
--
Efraim Flashner <efraim@flashner.co.il> רנשלפ םירפא
GPG key = A28B F40C 3E55 1372 662D 14F7 41AA E7DC CA3D 8351
Confidentiality cannot be guaranteed on emails sent or received unencrypted
E
E
Efraim Flashner wrote on 3 Apr 22:56 -0700
[PATCH v2 2/3] gnu: python-pip: Use system SSL certificates.
(address . 70179@debbugs.gnu.org)(name . Efraim Flashner)(address . efraim@flashner.co.il)
6426f336e0f7547880b312dd8712998546397d84.1712210069.git.efraim@flashner.co.il
* gnu/packages/python-build.scm (python-pip)[replacement]: New
field.
(python-pip/fixed): Provide a python-pip with a patched python-certifi
which only offers to use the system's SSL certificates.

Change-Id: Icea0769b881dc8d760562f0405fa8ea8167a4bd4
---
gnu/packages/python-build.scm | 34 ++++++++++++++++++++++++++++++++++
1 file changed, 34 insertions(+)

Toggle diff (56 lines)
diff --git a/gnu/packages/python-build.scm b/gnu/packages/python-build.scm
index 2ea457cdba..5b71d1502e 100644
--- a/gnu/packages/python-build.scm
+++ b/gnu/packages/python-build.scm
@@ -260,6 +260,7 @@ (define-public python-pip
(package
(name "python-pip")
(version "23.1")
+ (replacement python-pip/fixed)
(source
(origin
(method url-fetch)
@@ -277,6 +278,39 @@ (define-public python-pip
Python Package Index (PyPI).")
(license license:expat)))
+(define python-pip/fixed
+ (package
+ (inherit python-pip)
+ (source (origin
+ (inherit (package-source python-pip))
+ (snippet
+ #~(begin
+ (delete-file "src/pip/_vendor/certifi/cacert.pem")
+ (delete-file "src/pip/_vendor/certifi/core.py")
+ (with-output-to-file "src/pip/_vendor/certifi/core.py"
+ (lambda _
+ (display "\"\"\"
+certifi.py
+~~~~~~~~~~
+This module returns the installation location of SSL_CERT_FILE or
+/etc/ssl/certs/ca-certificates.crt, or its contents.
+\"\"\"
+import os
+
+_CA_CERTS = None
+
+try:
+ _CA_CERTS = os.environ [\"SSL_CERT_FILE\"]
+except:
+ _CA_CERTS = os.path.join(\"/etc\", \"ssl\", \"certs\", \"ca-certificates.crt\")
+
+def where() -> str:
+ return _CA_CERTS
+
+def contents() -> str:
+ with open(where(), \"r\", encoding=\"ascii\") as data:
+ return data.read()")))))))))
+
(define-public python-setuptools
(package
(name "python-setuptools")
--
Efraim Flashner <efraim@flashner.co.il> רנשלפ םירפא
GPG key = A28B F40C 3E55 1372 662D 14F7 41AA E7DC CA3D 8351
Confidentiality cannot be guaranteed on emails sent or received unencrypted
E
E
Efraim Flashner wrote on 3 Apr 22:56 -0700
[PATCH v2 3/3] gnu: python: Use system SSL certificates.
(address . 70179@debbugs.gnu.org)(name . Efraim Flashner)(address . efraim@flashner.co.il)
aac22d9606efdec3fa7e61d1d766dd74bfb6b8d3.1712210069.git.efraim@flashner.co.il
* gnu/packages/python.scm (python)[replacement]: New field.
(python/fixed): Provide a python with a patched python-certifi which
only offers to use the system's SSL certificates.

Change-Id: Ic5bcfb6b32282a7e0628232b1dc4cd60f3f2da52
---
gnu/packages/python.scm | 67 +++++++++++++++++++++++++++++++++++++++++
1 file changed, 67 insertions(+)

Toggle diff (103 lines)
diff --git a/gnu/packages/python.scm b/gnu/packages/python.scm
index 12a5148cb1..3ad4c5d5e7 100644
--- a/gnu/packages/python.scm
+++ b/gnu/packages/python.scm
@@ -96,6 +96,7 @@ (define-module (gnu packages python)
#:use-module (guix gexp)
#:use-module (guix packages)
#:use-module (guix download)
+ #:use-module (guix search-paths)
#:use-module (guix utils)
#:use-module (guix build-system gnu)
#:use-module (guix build-system trivial)
@@ -424,6 +425,7 @@ (define-public python-3.10
(inherit python-2)
(name "python")
(version "3.10.7")
+ (replacement python-3.10/fixed)
(source (origin
(method url-fetch)
(uri (string-append "https://www.python.org/ftp/python/"
@@ -590,6 +592,7 @@ (define-public python-3.10
inputs)))
(native-search-paths
(list (guix-pythonpath-search-path version)
+ $SSL_CERT_FILE
;; Used to locate tzdata by the zoneinfo module introduced in
;; Python 3.9.
(search-path-specification
@@ -982,6 +985,70 @@ (define-public python-3.12
(properties '((cpe-name . "python")))
(license license:psfl)))
+(define python-3.10/fixed
+ (package
+ (inherit python-3.10)
+ (arguments
+ (substitute-keyword-arguments (package-arguments python-3.10)
+ ((#:phases phases)
+ #~(modify-phases #$phases
+ ;; Also remove the bundled CA certificates.
+ ;; TODO: Rename this phase when merging back into python.
+ (replace 'remove-windows-binaries
+ (lambda _
+ ;; Delete .exe from embedded .whl (zip) files
+ (for-each
+ (lambda (whl)
+ (let ((dir "whl-content")
+ (circa-1980 (* 10 366 24 60 60)))
+ (mkdir-p dir)
+ (with-directory-excursion dir
+ (let ((whl (string-append "../" whl)))
+ (invoke "unzip" whl)
+ (for-each delete-file
+ (find-files "." "\\.exe$"))
+ (delete-file whl)
+
+ ;; Search for cacert.pem, delete it, and rewrite the
+ ;; file which directs python to look for it.
+ (let ((cacert (find-files "." "cacert\\.pem")))
+ (unless (null? cacert)
+ (let ((certifi (dirname (car cacert))))
+ (delete-file (string-append certifi "/cacert.pem"))
+ (delete-file (string-append certifi "/core.py"))
+ (with-output-to-file (string-append certifi "/core.py")
+ (lambda _
+ (display "\"\"\"
+certifi.py
+~~~~~~~~~~
+This module returns the installation location of SSL_CERT_FILE or
+/etc/ssl/certs/ca-certificates.crt, or its contents.
+\"\"\"
+import os
+
+_CA_CERTS = None
+
+try:
+ _CA_CERTS = os.environ [\"SSL_CERT_FILE\"]
+except:
+ _CA_CERTS = os.path.join(\"/etc\", \"ssl\", \"certs\", \"ca-certificates.crt\")
+
+def where() -> str:
+ return _CA_CERTS
+
+def contents() -> str:
+ with open(where(), \"r\", encoding=\"ascii\") as data:
+ return data.read()"))))))
+
+ ;; Reset timestamps to prevent them from ending
+ ;; up in the Zip archive.
+ (ftw "." (lambda (file stat flag)
+ (utime file circa-1980 circa-1980)
+ #t))
+ (apply invoke "zip" "-X" whl
+ (find-files "." #:directories? #t))))
+ (delete-file-recursively dir)))
+ (find-files "Lib/ensurepip" "\\.whl$"))))))))))
;; Next 3.x version.
(define-public python-next python-3.12)
--
Efraim Flashner <efraim@flashner.co.il> רנשלפ םירפא
GPG key = A28B F40C 3E55 1372 662D 14F7 41AA E7DC CA3D 8351
Confidentiality cannot be guaranteed on emails sent or received unencrypted
L
L
Lars-Dominik Braun wrote on 4 Apr 18:27 -0700
Re: [bug#70179] [PATCH 0/3] Use system nss-certs in Python.
(name . Efraim Flashner)(address . efraim@flashner.co.il)
Zg9TkkZ6VTNldhTZ@philomena
Hi Efraim,

Toggle quote (4 lines)
> It turns out that the Python ecosystem bundles a version of nss-certs.
> This patch series should change it so that it uses the system nss-certs
> instead.

I would change the comment at the top of core.py so it mentions this is
a Guix-specific version of certifi.py, so it’s clear the package has
been altered. You probably don’t need `_CA_CERTS = None`, since the
try…except clause covers all cases.

Otherwise LGTM.

Lars
E
E
Efraim Flashner wrote on 7 Apr 05:06 -0700
(name . Lars-Dominik Braun)(address . lars@6xq.net)
ZhKMRYXxEarT-9q5@3900XT
On Fri, Apr 05, 2024 at 10:27:46AM +0900, Lars-Dominik Braun wrote:
Toggle quote (13 lines)
> Hi Efraim,
>
> > It turns out that the Python ecosystem bundles a version of nss-certs.
> > This patch series should change it so that it uses the system nss-certs
> > instead.
>
> I would change the comment at the top of core.py so it mentions this is
> a Guix-specific version of certifi.py, so it’s clear the package has
> been altered. You probably don’t need `_CA_CERTS = None`, since the
> try…except clause covers all cases.
>
> Otherwise LGTM.

I left the initial `_CA_CERTS = None` as a sort of initial declaration
of the variable, since I don't really know python that well and I didn't
think it was correct to declare it inside the try…except.

I added the line at the top of core.py saying it was Guix specific and I
also adjusted the commit message for python mentioning the
$SSL_CERT_FILE in the natives-search-paths.

Then I went to build my home-config and I realized what I'd done with
the native-search-paths in python-3.10 and I moved it to the replacement
python so it wouldn't cause a world rebuild.

Patches pushed!

--
Efraim Flashner <efraim@flashner.co.il> רנשלפ םירפא
GPG key = A28B F40C 3E55 1372 662D 14F7 41AA E7DC CA3D 8351
Confidentiality cannot be guaranteed on emails sent or received unencrypted
-----BEGIN PGP SIGNATURE-----

iQIzBAABCAAdFiEEoov0DD5VE3JmLRT3Qarn3Mo9g1EFAmYSjEIACgkQQarn3Mo9
g1H7rQ/9G4GYNDI1gXbG9D3f2cbJd+6PSm/ffAV8YyRXWCxASutvuDQ8/cPuNvzt
YTyrXkA8FZb6GOF7etLsBhy9BrtzO8Z9KSEhH6AybME167GZ4FVzq5qaVz79ZjKq
4mgBLRCFRXdFKpCIyzhS+a/KLZftzbOjMUFGZoiJ+4HpdZHkuEJRF8zjuqqAU6J9
vdAqmkrQdRWelBKE5B2HOQpAoGdOMos53bRmJNwOskvTQcph5LloWV+SquX99UFM
3TsgN1AOilhTtFi03AsluFigXKCKoaAZTcbqq58JgOrtW5czL+oeX9f8wvb63fgY
9ehlc3yuywMqzZ+l2je81bAZxQ9urHszfFcaShny65C3imOfbRdNFJFVYGMDq1xE
wC+Jey7W1JWRUE1UnlbufskeBi6plKyZPObYEHm7OHZC7JpRgYdr/mmkibmY8nCI
InTLn7N4OsGFRyYIs8TRe+/H3QFRm2dnSrTfpqNOBRWDVKUEWL4xFAxncjFLI7pE
SzhSEztM5idaIWdatA1RwB3vPKbgKw6uSctXCwC5N9RrtDks6lAsbX4m5m+qapYd
lW/iwAzGe+QsGtPRQKgGFcICSSnpptIP7bubOcRohsAmraail1zv8lS60W3P7nlt
HTa7j4aK6JbSVNzqw1jhB2J1rZUsYi8w60RKWtxuxOU30vunnVo=
=Rhg+
-----END PGP SIGNATURE-----


Closed
L
L
Leo Famulari wrote on 7 Apr 13:41 -0700
Re: bug#70179: [PATCH 0/3] Use system nss-certs in Python.
ZhME3QWVLAIoiI4I@jasmine.lan
On Sun, Apr 07, 2024 at 03:06:29PM +0300, Efraim Flashner wrote:
Toggle quote (2 lines)
> Patches pushed!

Thanks so much Efraim!
-----BEGIN PGP SIGNATURE-----

iQIzBAABCAAdFiEEaEByLu7k06ZO5T6saqwZY3V/R/8FAmYTBNkACgkQaqwZY3V/
R/8W9BAAsoeZWOcn0oikhI3VCdHEFg9PV7Vjn4joGB9widCoArJ2f27lbTjmJ2/6
8NvDqerPVoF3xnxD6umbKcG5uqCIQoPVhtXaxrs/DENQ47MDus9yuX1dccmQQ7oI
QpPIwff37gSG5px0HPXYD/GkEbOmJMaduVhbxzILeSeb5XFNqrnGYvqCEtqGhYRX
dvksQGVhVVWxAFJbcuOKhu1MxS0FxGlItouJ1+UzuYPOzv7A/wkpTj9q16aSF1BB
VYyCd2PFFfPVtO8wFTJLXgXEAsRqclUKcVp1vbeJQHO+4ndOtkS9aB9riCUkwHir
fKqjrAzBFqqkKeaPcZj4nez9SZtdVh+bxvSWAtiatcgTg+HyKg3fyE/BBaTVwHRe
/KOFFSPcbc+1jQGedh2oM5T6f8O7UBNGeJTVSSEIlOHpX4hxWBi+qJZ9DCBE90e7
uTazw1W4c2eNcLdoyQ6ip4kLNEFyJDZdGs24S66XJOTYSz8BLFbSVddX8Wu8VpNK
gHNK6QisJVfhXenheWpAJrfNY9h8LeC4ty3LUftBVEeanXeo5Lk0+aa7nZKfExOw
5cgH20/8TeAIJSHdyUgXO8fJADxIyKfHFdhsTuzMRLVRtNqUzVaewq74zkSdQmBH
G8EIB8RHFHb1QGlDsCxY4IijgVm0nZo584kZgMSjS3RJJ31bPlM=
=UUx0
-----END PGP SIGNATURE-----


Closed
?
Your comment

This issue is archived.

To comment on this conversation send an email to 70179@patchwise.org

To respond to this issue using the mumi CLI, first switch to it
mumi current 70179
Then, you may apply the latest patchset in this issue (with sign off)
mumi am -- -s
Or, compose a reply to this issue
mumi compose
Or, send patches to this issue
mumi send-email *.patch