[PATCH] gnu: rust: make rust 1.25.0 build reproducible

Done

Details

3 participants

Jakub Kądziołka
Ludovic Courtès
Nikolai Merinov

Owner: unassigned

Submitted by: Nikolai Merinov

Severity: normal

Debbugs page

Nikolai Merinov wrote on 13 Jul 2018 13:15

Recipients:(address . guix-patches@gnu.org)

Message-ID:87a7qlk6tx.fsf@member.fsf.org

* gnu/packages/rust.scm (rust-1.25): switch back to llvm@3.9 still builds with
llvm@6.0 is not reproducible. Remove all llvm@6.0 related twicks.
(rust-1.26): switch to llvm@6.0 still rust-1.26 can build reproducible
binaries with new llvm release. Enable llvm@5.0 related twicks.
---
 gnu/packages/rust.scm | 39 ++++++++++++++++++---------------------
 1 file changed, 18 insertions(+), 21 deletions(-)

Toggle diff (73 lines)diff --git a/gnu/packages/rust.scm b/gnu/packages/rust.scm
index 28bffb44b..1095070eb 100644
--- a/gnu/packages/rust.scm
+++ b/gnu/packages/rust.scm
@@ -545,10 +545,6 @@ jemalloc = \"" jemalloc "/lib/libjemalloc_pic.a" "\"
                                     "0baxjr99311lvwdq0s38bipbnj72pn6fgbk6lcq7j555xq53mxpf")))
     (package
       (inherit base-rust)
-      (inputs
-       ;; Use LLVM 6.0
-       (alist-replace "llvm" (list llvm)
-                      (package-inputs base-rust)))
       (arguments
        (substitute-keyword-arguments (package-arguments base-rust)
          ((#:phases phases)
@@ -559,23 +555,6 @@ jemalloc = \"" jemalloc "/lib/libjemalloc_pic.a" "\"
                    ;; This test wants to update the crate index.
                    (("fn no_index_update") "#[ignore]\nfn no_index_update"))
                  #t))
-             (add-after 'configure 'enable-codegen-tests
-               (lambda _
-                 (substitute* "config.toml"
-                   (("codegen-tests = false") ""))
-                 #t))
-             (replace 'patch-aarch64-test
-               (lambda _
-                 (substitute* "src/librustc_metadata/dynamic_lib.rs"
-                   ;; This test is known to fail on aarch64 and powerpc64le:
-                   ;; https://github.com/rust-lang/rust/issues/45410
-                   (("fn test_loading_cosine") "#[ignore]\nfn test_loading_cosine"))
-                 ;; This test fails on aarch64 with llvm@6.0:
-                 ;; https://github.com/rust-lang/rust/issues/49807
-                 ;; other possible solution:
-                 ;; https://github.com/rust-lang/rust/pull/47688
-                 (delete-file "src/test/debuginfo/by-value-self-argument-in-trait-impl.rs")
-                 #t))
              (delete 'ignore-glibc-2.27-incompatible-test))))))))
 
 (define-public rust-1.26
@@ -585,6 +564,10 @@ jemalloc = \"" jemalloc "/lib/libjemalloc_pic.a" "\"
                                     #:patches '("rust-coresimd-doctest.patch"))))
     (package
       (inherit base-rust)
+      (inputs
+       ;; Use LLVM 6.0
+       (alist-replace "llvm" (list llvm)
+                      (package-inputs base-rust)))
       (arguments
        (substitute-keyword-arguments (package-arguments base-rust)
          ((#:phases phases)
@@ -624,6 +607,20 @@ jemalloc = \"" jemalloc "/lib/libjemalloc_pic.a" "\"
                  (substitute* "src/tools/cargo/tests/testsuite/generate_lockfile.rs"
                    ;; This test wants to update the crate index.
                    (("fn no_index_update") "#[ignore]\nfn no_index_update"))
+                 #t))
+             (add-after 'patch-aarch64-test 'patch-aarch64-llvm6-test
+               (lambda _
+                 ;; This test fails on aarch64 with llvm@6.0:
+                 ;; https://github.com/rust-lang/rust/issues/49807
+                 ;; other possible solution:
+                 ;; https://github.com/rust-lang/rust/pull/47688
+                 (delete-file "src/test/debuginfo/by-value-self-argument-in-trait-impl.rs")
+                 #t))
+             (add-after 'configure 'enable-codegen-tests
+               (lambda* _
+                 (substitute* "config.toml"
+                   ;; codegen test should pass with LLVM 6
+                   (("codegen-tests = false") ""))
                  #t)))))))))
 
 (define-public rust
-- 
2.17.1

Nikolai Merinov wrote on 20 Jul 2018 13:21

Recipients:(address . 32227@debbugs.gnu.org)

Message-ID:876019k698.fsf_-_@member.fsf.org

Hi,

I tried to configure how rust use llvm, but I did not succeed in

it. Looks like llvm@6.0 usage was fixed fully only in rust 1.26.

I already checked once that 1.25 and 1.26 rust releases is reproducible

in my environment with llvm@3.9 and llvm@6.0 respectively. When I posted

this patch I restarted my builds with "--rounds=2".

Regards,

Nikolai

Nikolai Merinov wrote on 22 Jul 2018 02:55

Recipients:(address . 32227@debbugs.gnu.org)

Message-ID:877eln61cv.fsf@member.fsf.org

Hi,

Please, do not merge. Change rejected. With this change 1.27.0 build

still not reproducible. I should try to figure out which llvm

compilation options make builds non-reproducible.

Regards,

Nikolai

Nikolai Merinov <nikolai.merinov@member.fsf.org> writes:

Toggle quote (11 lines)> Hi,
>
> I tried to configure how rust use llvm, but I did not succeed in
> it. Looks like llvm@6.0 usage was fixed fully only in rust 1.26.
>
> I already checked once that 1.25 and 1.26 rust releases is reproducible
> in my environment with llvm@3.9 and llvm@6.0 respectively. When I posted
> this patch I restarted my builds with "--rounds=2".
>
> Regards,
> Nikolai

Ludovic Courtès wrote on 21 Aug 2018 14:19

Re: [bug#32227] [PATCH] gnu: rust: make rust 1.25.0 build reproducible

Recipients:(name . Nikolai Merinov)(address . nikolai.merinov@member.fsf.org)(address . 32227@debbugs.gnu.org)

Message-ID:87ftz7mn89.fsf@gnu.org

Hello Nikolai,

Nikolai Merinov <nikolai.merinov@member.fsf.org> skribis:

Toggle quote (4 lines)

> Please, do not merge. Change rejected. With this change 1.27.0 build

> still not reproducible. I should try to figure out which llvm

> compilation options make builds non-reproducible.

Does the problem still applies to 1.27.0 as currently available in

master?

Thanks,

Ludo’.

Ludovic Courtès wrote on 21 Aug 2018 14:19

Recipients:(name . Nikolai Merinov)(address . nikolai.merinov@member.fsf.org)(address . 32227@debbugs.gnu.org)

Message-ID:87efermn81.fsf@gnu.org

Hello Nikolai,

Nikolai Merinov <nikolai.merinov@member.fsf.org> skribis:

Toggle quote (4 lines)

> Please, do not merge. Change rejected. With this change 1.27.0 build

> still not reproducible. I should try to figure out which llvm

> compilation options make builds non-reproducible.

Does the problem still apply to 1.27.0 as currently available in master?

Thanks,

Ludo’.

Nikolai Merinov wrote on 23 Sep 2018 14:38

Recipients:(name . Ludovic Courtès)(address . ludo@gnu.org)(address . 32227@debbugs.gnu.org)

Message-ID:8736tzlws1.fsf@member.fsf.org

Hello Ludo,

Sorry for long answer. I have no chance to continue ivestigation last
month, but I'll want to provide my old finding:

1. Rust 1.25 build cargo binary (but not rustc itself) in
non-reproducible manner when compiled with llvm-6.0 and llvm-6.1. In
both cases we got non-reproducible code for next functions:
"<env_logger::Logger as log::Log>::log", "git2::panic::check",
"curl::easy::handler::ssl_ctx_cb".
2. Rust 1.26 with llvm-6.0 build is reproducible regardless of which
rustc 1.25 (with llvm-6.0 or llvm-3.9) was used to bootstrap it.
3. Rust 1.27.0 and Rust 1.27.2 again build rustc itself in reproducible
manner, while "cargo" binary differ build to build. I observed
non-reproducible code in "git2::panic::check" function and in
".gcc_except_table" section of resulting binary.

In attachements you can find patch to move rust-1.25 to llvm-3.9, and
patch to update Rust 1.27 to 1.27.2.

From: Nikolai Merinov <nikolai.merinov@member.fsf.org>
To: guix-patches@gnu.org
Subject: [PATCH] gnu: rust: make rust 1.25.0 build reproducible

* gnu/packages/rust.scm (rust-1.25): switch back to llvm@3.9 still builds with
llvm@6.0 is not reproducible. Remove all llvm@6.0 related twicks.
(rust-1.26): switch to llvm@6.0 still rust-1.26 can build reproducible
binaries with new llvm release. Enable llvm@5.0 related twicks.
---
 gnu/packages/rust.scm | 39 ++++++++++++++++++---------------------
 1 file changed, 18 insertions(+), 21 deletions(-)

Toggle diff (73 lines)diff --git a/gnu/packages/rust.scm b/gnu/packages/rust.scm
index 28bffb44b..1095070eb 100644
--- a/gnu/packages/rust.scm
+++ b/gnu/packages/rust.scm
@@ -545,10 +545,6 @@ jemalloc = \"" jemalloc "/lib/libjemalloc_pic.a" "\"
                                     "0baxjr99311lvwdq0s38bipbnj72pn6fgbk6lcq7j555xq53mxpf")))
     (package
       (inherit base-rust)
-      (inputs
-       ;; Use LLVM 6.0
-       (alist-replace "llvm" (list llvm)
-                      (package-inputs base-rust)))
       (arguments
        (substitute-keyword-arguments (package-arguments base-rust)
          ((#:phases phases)
@@ -559,23 +555,6 @@ jemalloc = \"" jemalloc "/lib/libjemalloc_pic.a" "\"
                    ;; This test wants to update the crate index.
                    (("fn no_index_update") "#[ignore]\nfn no_index_update"))
                  #t))
-             (add-after 'configure 'enable-codegen-tests
-               (lambda _
-                 (substitute* "config.toml"
-                   (("codegen-tests = false") ""))
-                 #t))
-             (replace 'patch-aarch64-test
-               (lambda _
-                 (substitute* "src/librustc_metadata/dynamic_lib.rs"
-                   ;; This test is known to fail on aarch64 and powerpc64le:
-                   ;; https://github.com/rust-lang/rust/issues/45410
-                   (("fn test_loading_cosine") "#[ignore]\nfn test_loading_cosine"))
-                 ;; This test fails on aarch64 with llvm@6.0:
-                 ;; https://github.com/rust-lang/rust/issues/49807
-                 ;; other possible solution:
-                 ;; https://github.com/rust-lang/rust/pull/47688
-                 (delete-file "src/test/debuginfo/by-value-self-argument-in-trait-impl.rs")
-                 #t))
              (delete 'ignore-glibc-2.27-incompatible-test))))))))
 
 (define-public rust-1.26
@@ -585,6 +564,10 @@ jemalloc = \"" jemalloc "/lib/libjemalloc_pic.a" "\"
                                     #:patches '("rust-coresimd-doctest.patch"))))
     (package
       (inherit base-rust)
+      (inputs
+       ;; Use LLVM 6.0
+       (alist-replace "llvm" (list llvm)
+                      (package-inputs base-rust)))
       (arguments
        (substitute-keyword-arguments (package-arguments base-rust)
          ((#:phases phases)
@@ -624,6 +607,20 @@ jemalloc = \"" jemalloc "/lib/libjemalloc_pic.a" "\"
                  (substitute* "src/tools/cargo/tests/testsuite/generate_lockfile.rs"
                    ;; This test wants to update the crate index.
                    (("fn no_index_update") "#[ignore]\nfn no_index_update"))
+                 #t))
+             (add-after 'patch-aarch64-test 'patch-aarch64-llvm6-test
+               (lambda _
+                 ;; This test fails on aarch64 with llvm@6.0:
+                 ;; https://github.com/rust-lang/rust/issues/49807
+                 ;; other possible solution:
+                 ;; https://github.com/rust-lang/rust/pull/47688
+                 (delete-file "src/test/debuginfo/by-value-self-argument-in-trait-impl.rs")
+                 #t))
+             (add-after 'configure 'enable-codegen-tests
+               (lambda* _
+                 (substitute* "config.toml"
+                   ;; codegen test should pass with LLVM 6
+                   (("codegen-tests = false") ""))
                  #t)))))))))
 
 (define-public rust
-- 
2.17.1

Toggle diff (15 lines)diff --git a/gnu/packages/rust.scm b/gnu/packages/rust.scm
index 0695f8c7d..4bc8d1d88 100644
--- a/gnu/packages/rust.scm
+++ b/gnu/packages/rust.scm
@@ -631,8 +631,8 @@ jemalloc = \"" jemalloc "/lib/libjemalloc_pic.a" "\"
 
 (define-public rust
   (let ((base-rust
-         (rust-bootstrapped-package rust-1.26 "1.27.0"
-                                    "089d7rhw55zpvnw71dj8vil6qrylvl4xjr4m8bywjj83d4zq1f9c"
+         (rust-bootstrapped-package rust-1.26 "1.27.2"
+                                    "0pg1s37bhx9zqbynxyydq5j6q7kij9vxkcv8maz0m25prm88r0cs"
                                     #:patches
                                     '("rust-coresimd-doctest.patch"
                                       "rust-bootstrap-stage0-test.patch"))))

Regards,

Nikolai

ludo@gnu.org (Ludovic Courtès) writes:

Toggle quote (12 lines)> Hello Nikolai,
>
> Nikolai Merinov <nikolai.merinov@member.fsf.org> skribis:
>
>> Please, do not merge. Change rejected. With this change 1.27.0 build
>> still not reproducible. I should try to figure out which llvm
>> compilation options make builds non-reproducible.
>
> Does the problem still apply to 1.27.0 as currently available in master?
>
> Thanks,
> Ludo’.

Jakub Kądziołka wrote on 23 Feb 2020 06:27

closing

Recipients:(address . 32227-done@debbugs.gnu.org)

Message-ID:20200223142717.h3mvval7iru4o4h4@gravity

These patches seem to have been made obsolete by commit

e027a494fd5e6025e645ce9f36cccca39a77d327 - closing.

Closed

Jakub Kądziołka wrote on 23 Feb 2020 06:29

add wontfix tag

Recipients:(address . control@debbugs.gnu.org)

Message-ID:20200223142934.hwsktvj6alf64cpo@gravity

tags 32227 wontfix

Your comment

This issue is archived.

To comment on this conversation send an email to 32227@patchwise.org

To respond to this issue using the mumi CLI, first switch to it

mumi current 32227

Then, you may apply the latest patchset in this issue (with sign off)

mumi am -- -s

Or, compose a reply to this issue

mumi compose

Or, send patches to this issue

mumi send-email *.patch

is:open	open issues
is:done	closed issues
submitter:<who>	search issue submitter
author:<who>	search by message author
date:yesterday..now	search by issue date
mdate:3m..2d	search by message date