Report forwarded
to bug-guix@gnu.org: bug#22304; Package guix.
(Mon, 04 Jan 2016 16:30:02 GMT) (full text, mbox, link).
Acknowledgement sent
to Ricardo Wurmus <ricardo.wurmus@mdc-berlin.de>:
New bug report received and forwarded. Copy sent to bug-guix@gnu.org.
(Mon, 04 Jan 2016 16:30:02 GMT) (full text, mbox, link).
Subject: Re: bug#22304: Build for Julia is not reproducible
Date: Tue, 05 Jan 2016 23:30:48 +0100
Ricardo Wurmus <ricardo.wurmus@mdc-berlin.de> skribis:
> When building “julia” twice we get different binaries for
>
> $out/lib/julia/sys.so
>
> and
>
> $out/lib/julia/sys-debug.so
Apparently there’s no __DATE__ and __TIME__ in the C code.
I would check if the order of object files on the command line that
links those shared objects is always the same.
Ludo’.
Information forwarded
to bug-guix@gnu.org: bug#22304; Package guix.
(Thu, 04 Feb 2016 14:53:01 GMT) (full text, mbox, link).
Subject: Re: bug#22304: Build for Julia is not reproducible
Date: Thu, 4 Feb 2016 15:51:51 +0100
Ludovic Courtès <ludo@gnu.org> writes:
> Ricardo Wurmus <ricardo.wurmus@mdc-berlin.de> skribis:
>
>> When building “julia” twice we get different binaries for
>>
>> $out/lib/julia/sys.so
>>
>> and
>>
>> $out/lib/julia/sys-debug.so
>
> Apparently there’s no __DATE__ and __TIME__ in the C code.
>
> I would check if the order of object files on the command line that
> links those shared objects is always the same.
As far as I could tell the order of shared objects in two consecutive
runs is the same.
~~ Ricardo
Information forwarded
to bug-guix@gnu.org: bug#22304; Package guix.
(Thu, 28 Nov 2019 17:15:01 GMT) (full text, mbox, link).
To: 22304@debbugs.gnu.org, Ricardo Wurmus <rekado@elephly.net>,
Ludovic Courtès <ludo@gnu.org>
Subject: bug#22304: Julia not reproducible
Date: Thu, 28 Nov 2019 18:14:00 +0100
Hi,
This bug [1] is still present even with the version 1.1.1 of Julia.
[1] http://issues.guix.gnu.org/issue/22304
--8<---------------cut here---------------start------------->8---
guix describe
Generation 57 Nov 25 2019 14:26:15 (current)
guix b5d4d5b
repository URL: https://git.savannah.gnu.org/git/guix.git
branch: master
commit: b5d4d5b9bcf267fddd02fcc14b88eac0bebf979f
--8<---------------cut here---------------end--------------->8---
--8<---------------cut here---------------start------------->8---
guix build julia # populate the store with dependencies
guix build julia --check -K --no-grafts
[...]
guix build: error: derivation
`/gnu/store/xdy3jjz7dzg1lr231b8zbss8xn44ldjj-julia-1.1.1.drv' may not
be deterministic: output
`/gnu/store/s2vj70fgv4v4wq66dbi797ss15f8xd3b-julia-1.1.1' differs from
‘/gnu/store/s2vj70fgv4v4wq66dbi797ss15f8xd3b-julia-1.1.1-check’
--8<---------------cut here---------------end--------------->8---
--8<---------------cut here---------------start------------->8---
diff -r /gnu/store/s2vj70fgv4v4wq66dbi797ss15f8xd3b-julia-1.1.1
/gnu/store/s2vj70fgv4v4wq66dbi797ss15f8xd3b-julia-1.1.1-check
Binary files /gnu/store/s2vj70fgv4v4wq66dbi797ss15f8xd3b-julia-1.1.1/lib/julia/sys.so
and /gnu/store/s2vj70fgv4v4wq66dbi797ss15f8xd3b-julia-1.1.1-check/lib/julia/sys.so
differ
Binary files /gnu/store/s2vj70fgv4v4wq66dbi797ss15f8xd3b-julia-1.1.1/share/julia/test/depot/compiled/v1.1/Bar/HXSAn.ji
and /gnu/store/s2vj70fgv4v4wq66dbi797ss15f8xd3b-julia-1.1.1-check/share/julia/test/depot/compiled/v1.1/Bar/HXSAn.ji
differ
Binary files /gnu/store/s2vj70fgv4v4wq66dbi797ss15f8xd3b-julia-1.1.1/share/julia/test/depot/compiled/v1.1/Baz/rONVA.ji
and /gnu/store/s2vj70fgv4v4wq66dbi797ss15f8xd3b-julia-1.1.1-check/share/julia/test/depot/compiled/v1.1/Baz/rONVA.ji
differ
Binary files /gnu/store/s2vj70fgv4v4wq66dbi797ss15f8xd3b-julia-1.1.1/share/julia/test/depot/compiled/v1.1/Foo/MYb1d.ji
and /gnu/store/s2vj70fgv4v4wq66dbi797ss15f8xd3b-julia-1.1.1-check/share/julia/test/depot/compiled/v1.1/Foo/MYb1d.ji
differ
Binary files /gnu/store/s2vj70fgv4v4wq66dbi797ss15f8xd3b-julia-1.1.1/share/julia/test/depot/compiled/v1.1/Foo/TeeT6.ji
and /gnu/store/s2vj70fgv4v4wq66dbi797ss15f8xd3b-julia-1.1.1-check/share/julia/test/depot/compiled/v1.1/Foo/TeeT6.ji
differ
Binary files /gnu/store/s2vj70fgv4v4wq66dbi797ss15f8xd3b-julia-1.1.1/share/julia/test/depot/compiled/v1.1/Qux/YFfiR.ji
and /gnu/store/s2vj70fgv4v4wq66dbi797ss15f8xd3b-julia-1.1.1-check/share/julia/test/depot/compiled/v1.1/Qux/YFfiR.ji
differ
--8<---------------cut here---------------end--------------->8---
Well, then I try to explore with diffoscope but hum? I do not do
correctly because the resulting diff is 845MB ouch!
All the best,
simon
Information forwarded
to bug-guix@gnu.org: bug#22304; Package guix.
(Fri, 29 Nov 2019 16:20:01 GMT) (full text, mbox, link).
Hi,
zimoun <zimon.toutoune@gmail.com> skribis:
> Binary files /gnu/store/s2vj70fgv4v4wq66dbi797ss15f8xd3b-julia-1.1.1/share/julia/test/depot/compiled/v1.1/Qux/YFfiR.ji
> and /gnu/store/s2vj70fgv4v4wq66dbi797ss15f8xd3b-julia-1.1.1-check/share/julia/test/depot/compiled/v1.1/Qux/YFfiR.ji
> differ
>
>
> Well, then I try to explore with diffoscope but hum? I do not do
> correctly because the resulting diff is 845MB ouch!
Pro tip: just run
diffoscope \
/gnu/store/s2vj70fgv4v4wq66dbi797ss15f8xd3b-julia-1.1.1{,-check}/share/julia/test/depot/compiled/v1.1/Qux/YFfiR.ji
That’ll already be enough info for a start!
Ludo’.
Information forwarded
to bug-guix@gnu.org: bug#22304; Package guix.
(Mon, 02 Dec 2019 17:29:01 GMT) (full text, mbox, link).
Hi Ludo,
On Fri, 29 Nov 2019 at 17:19, Ludovic Courtès <ludo@gnu.org> wrote:
> Pro tip: just run
I think that I have already did the pro tip. :-)
--8<---------------cut here---------------end--------------->8---
diffoscope /gnu/store/s2vj70fgv4v4wq66dbi797ss15f8xd3b-julia-1.1.1{,-check}/lib/julia/sys.so
\
> /tmp/sys.diffoscope
du -sh /tmp/sys.diffoscope
845M /tmp/sys.diffoscope
--8<---------------cut here---------------end--------------->8---
Maybe I am doing incorrectly...
If I understand well [1], the file .ji are "precompiled" (analogy: .pyc). Hum?!
[1] https://docs.julialang.org/en/v1/devdocs/sysimg/
Cheers,
simon
Information forwarded
to bug-guix@gnu.org: bug#22304; Package guix.
(Fri, 24 Jan 2020 18:31:01 GMT) (full text, mbox, link).
Hi,
On Mon, 2 Dec 2019 at 18:28, zimoun <zimon.toutoune@gmail.com> wrote:
> If I understand well [1], the file .ji are "precompiled" (analogy: .pyc). Hum?!
The non-determinism seems coming from these "precompiled" .ji files
and 2 bugs are opened upstream.
https://github.com/JuliaLang/julia/issues/25900https://github.com/JuliaLang/julia/issues/34115
Thanks to Nicolò to pointing out these [1] when working on the version update.
[1] https://debbugs.gnu.org/cgi/bugreport.cgi?bug=38546#80
With lengthy and clear details. :-)
Let start the container with everything useful.
--8<---------------cut here---------------start------------->8---
mkdir -p /tmp/test-julia && cd /tmp/test-julia
guix time-machine --commit=b5d4d5b \
environment -u nixbld --no-cwd --container --network \
--expose=$(realpath determinism) --share=/tmp/test-julia \
--ad-hoc julia coreutils nss-certs curl tar gzip
--8<---------------cut here---------------end--------------->8---
Then, the path to reproduce is odd because there is no Julia packages
in Guix. So we need to use 'Pkg' from Julia which downloads from
elsewhere and the reproducibility of such is not well controlled but
it gives an idea. :-)
--8<---------------cut here---------------start------------->8---
export SSL_CERT_DIR="$GUIX_ENVIRONMENT/etc/ssl/certs"
export SSL_CERT_FILE="$SSL_CERT_DIR/ca-certificates.crt"
export GIT_SSL_CAINFO="$SSL_CERT_FILE"
# populate the Julia package manager
julia --startup-file=no -q -e 'using Pkg; Pkg.instantiate()'
# install something
julia --startup-file=no -q -e 'using Pkg; Pkg.add("Compat")'
# use something / precompiled it
julia --startup-file=no -q -e 'using Compat'
# check the .ji file
md5sum ~/.julia/compiled/v1.1/Compat/*
--8<---------------cut here---------------end--------------->8---
Then each time one runs this block, one gets different hash.
All the best,
simon
Information forwarded
to bug-guix@gnu.org: bug#22304; Package guix.
(Wed, 13 May 2020 18:45:03 GMT) (full text, mbox, link).
To: 22304@debbugs.gnu.org,
Nicolò Balzarotti <anothersms@gmail.com>,
Efraim Flashner <efraim@flashner.co.il>
Subject: Julia v1.4.1 still not reproducible
Date: Wed, 13 May 2020 20:44:32 +0200
Hi Efraim,
On Wed, 13 May 2020 at 13:04, Efraim Flashner <efraim@flashner.co.il> wrote:
> guix build: error: derivation `/gnu/store/z21c12ijspavaclcins5pja8z1s8wbm0-julia-1.4.1.drv' may not be deterministic: output `/gnu/store/ri931fk8hf5wvagk76zhmwbphq2pwgnp-julia-1.4.1' differs from ?/gnu/store/ri931fk8hf5wvagk76zhmwbphq2pwgnp-julia-1.4.1-check?
>
> I built it on bayfront twice. I'll get the diffoscope output once it
> finally builds.
Thank you.
Cheers,
simon
Information forwarded
to bug-guix@gnu.org: bug#22304; Package guix.
(Sun, 17 May 2020 13:56:01 GMT) (full text, mbox, link).
Hi Efraim, Simon,
The attached patches provide Simon's suggested julia-deterministic
build.
With this, I get deterministic julia build, *but* only on a single
machine. Building on a different machine I get a different output. I
still need to investigate why. Also, some of the patches are probably
not needed at all. In the near future, as discussed previously, I aim
at submitting required patches upstream.
I disabled tests in julia-deterministic, so it get built quite fast
(~10minutes). If you want to try to see if this is reproducible//know
how to figure out what might be happening on different machines,
feedback is wellcome!
My ouptut is (patches applied on 75741af9b2):
guix hash -r /gnu/store/mp8hl7j09nfnsxbb3a6ka23ww44gq0z2-julia-deterministic-1.4.1
1k6mak38nkiaj0b9l6rcmxfcjwvbijmczxs703hgawwi6z0jikxl
Thanks, Nicolò
Efraim Flashner <efraim@flashner.co.il> writes:
> On Wed, May 13, 2020 at 08:43:08PM +0200, zimoun wrote:
>> Hi Efraim,
>>
>> On Wed, 13 May 2020 at 13:04, Efraim Flashner <efraim@flashner.co.il> wrote:
>>
>> > guix build: error: derivation `/gnu/store/z21c12ijspavaclcins5pja8z1s8wbm0-julia-1.4.1.drv' may not be deterministic: output `/gnu/store/ri931fk8hf5wvagk76zhmwbphq2pwgnp-julia-1.4.1' differs from ?/gnu/store/ri931fk8hf5wvagk76zhmwbphq2pwgnp-julia-1.4.1-check?
>> >
>> > I built it on bayfront twice. I'll get the diffoscope output once it
>> > finally builds.
>
> I wasn't able to provide the output of diffoscope, it turns out that the
> output was too large to attach.
>
> --
> Efraim Flashner <efraim@flashner.co.il> אפרים פלשנר
> GPG key = A28B F40C 3E55 1372 662D 14F7 41AA E7DC CA3D 8351
> Confidentiality cannot be guaranteed on emails sent or received unencrypted
Information forwarded
to bug-guix@gnu.org: bug#22304; Package guix.
(Sun, 27 Jun 2021 10:41:02 GMT) (full text, mbox, link).
Debbugs is free software and licensed under the terms of the
GNU Public License version 2. The current version can be
obtained from https://bugs.debian.org/debbugs-source/.
From: Nicolò Balzarotti <anothersms@gmail.com>To: Efraim Flashner <efraim@flashner.co.il>, zimoun <zimon.toutoune@gmail.com>Cc: 22304@debuugs.gnu.orgSubject: Re: Julia v1.4.1 still not reproducibleDate: Sun, 17 May 2020 15:51:00 +0200[Message part 3 (text/plain, inline)][0001-gnu-julia-Specify-correct-llvm-version.patch (text/x-patch, attachment)][0002-gnu-Add-julia-deterministic.patch (text/x-patch, attachment)][Message part 6 (text/plain, inline)]