#30107 ant-bootstrap@1.7.1 sometimes fails to build on core-updates

Message #5 received at submit@debbugs.gnu.org (full text, mbox, reply):

From: Chris Marusich <cmmarusich@gmail.com>

To: bug-guix@gnu.org

Subject: ant-bootstrap@1.7.1 sometimes fails to build on core-updates

Date: Sat, 13 Jan 2018 22:58:42 -0800

[Message part 1 (text/plain, inline)]

Hi,

At commit 1b321229f4653c5daa873813e24910789c0b2918 (i.e., the current
tip of the core-updates branch), ant-bootstrap@1.7.1 sometimes fails to
build.  This package is defined in gnu/packages/java.scm, but it is not
exported (i.e., it is used privately within the module).  Note that
according to 'guix refresh', currently 215 packages depend on this
package.

I tried to build this package 147 times.  It failed 5 times, and it
succeeded 142 times.  That's about a 3% failure rate.  All 5 failures
produced the same log output, which I've attached.  My machine is an
x86-64 machine.

-- 
Chris

[ant-bootstrap-1.7.1-segfault.log (application/octet-stream, attachment)]

[signature.asc (application/pgp-signature, inline)]

Message #8 received at 30107@debbugs.gnu.org (full text, mbox, reply):

From: Gábor Boskovits <boskovits@gmail.com>

To: Chris Marusich <cmmarusich@gmail.com>

Cc: 30107@debbugs.gnu.org

Subject: Re: bug#30107: ant-bootstrap@1.7.1 sometimes fails to build on core-updates

Date: Sun, 14 Jan 2018 10:32:03 +0100

[Message part 1 (text/plain, inline)]

I've also seen this once. No idea so far.
I've tried to look around, but found no other mention of this issue.

2018-01-14 7:58 GMT+01:00 Chris Marusich <cmmarusich@gmail.com>:

> Hi,
>
> At commit 1b321229f4653c5daa873813e24910789c0b2918 (i.e., the current
> tip of the core-updates branch), ant-bootstrap@1.7.1 sometimes fails to
> build.  This package is defined in gnu/packages/java.scm, but it is not
> exported (i.e., it is used privately within the module).  Note that
> according to 'guix refresh', currently 215 packages depend on this
> package.
>
> I tried to build this package 147 times.  It failed 5 times, and it
> succeeded 142 times.  That's about a 3% failure rate.  All 5 failures
> produced the same log output, which I've attached.  My machine is an
> x86-64 machine.
>
> --
> Chris
>

[Message part 2 (text/html, inline)]

Message #11 received at 30107@debbugs.gnu.org (full text, mbox, reply):

From: Gábor Boskovits <boskovits@gmail.com>

To: 30107@debbugs.gnu.org

Subject: Another kind of failure

Date: Thu, 18 Jan 2018 10:02:31 +0100

[Message part 1 (text/plain, inline)]

I'm now on 6d49ca16be22e3fb95823ac1780ad9460a18b180.
I also observe another kind of failure now.

After output
Buildfile: build.xml

the build process hangs.

This is also indeterministic, however it is harder to specify the failure
rate here...

[Message part 2 (text/html, inline)]

Message #14 received at 30107@debbugs.gnu.org (full text, mbox, reply):

From: Björn Höfling <bjoern.hoefling@bjoernhoefling.de>

To: 30107@debbugs.gnu.org

Subject: Backtrace

Date: Fri, 26 Jan 2018 11:30:29 +0100

[Message part 1 (text/plain, inline)]

I managed to get a coredump and backtrace, but I'm not able to
extract any useful information. I never went that deep into C
programming. If anyone can get more out of this, attached is the
backtrace, register state and some disassembly.

Björn

[gdb.txt (text/plain, attachment)]

Message #17 received at 30107@debbugs.gnu.org (full text, mbox, reply):

From: Björn Höfling <bjoern.hoefling@bjoernhoefling.de>

To: 30107@debbugs.gnu.org

Subject: How I got the core dump

Date: Sat, 3 Feb 2018 09:36:26 +0100

On request, I'm writing here how I got to that coredump:

My first step was to investigate the build.sh, and I just patched it to
output the full command, stripping of the rest:

diff --git a/bootstrap.sh b/bootstrap.x.sh
index bc54db4..f8c0720 100755
--- a/bootstrap.sh
+++ b/bootstrap.x.sh
@@ -151,18 +151,7 @@ cp src/script/antRun bin
 chmod +x bin/antRun
 
 echo ... Building Ant Distribution
-
-"${JAVACMD}" -classpath "${CLASSPATH}" -Dant.home=. $ANT_OPTS
org.apache.tools.ant.Main -emacs "$@" bootstrap -ret=$?
-if [ $ret != 0 ]; then
-  echo ... Failed Building Ant Distribution !
-  exit $ret
-fi
-
-
-echo ... Cleaning Up Build Directories
-
-rm -rf ${CLASSDIR}
-rm -rf bin
+echo I would do:
+echo "${JAVACMD}" -classpath "${CLASSPATH}" -Dant.home=. $ANT_OPTS
org.apache.tools.ant.Main -emacs "$@" bootstrap echo ... Done
Bootstrapping Ant Distribution

I added the patch into the package definition.

As I have learned yesterday, I could just repack the sources and use

guix --with-source=modified-ant.tar.gz ...

Anyway, I found out it calls:

/gnu/store/088bg6n5llvqn9j7d2740hhhilbqai4a-sablevm-1.13/bin/java-sablevm
-classpath build/classes:src/main:lib/xercesImpl.jar:lib/xml-apis.jar:
-Dant.home=. -Dbuild.compiler=jikes org.apache.tools.ant.Main -emacs
-Ddist.dir=/gnu/store/dxdsdsj4nz7fig92b2xjb7jf7swm5rni-ant-bootstrap-1.7.1
bootstrap

Next, I realized that my Ubuntu+Guix-on-top is eating up my core dumps:

$> cat /proc/sys/kernel/core_pattern 
|/usr/share/apport/apport %p %s %c %d %P

So instead I went into my QEMU machine and continued there.

Set ulimit to unlimited:

ulimit -c unlimited

In sablevm, we need to get debugging infos into it:

Add to it's package definition's #arguments this one:

#:strip-binaries? #f

Rebuild it:

./pre-inst-env guix build -e '(@@ (gnu packages java) sablevm)'
--no-grafts --no-substitutes -K  > sablevm.log 2>&1

Remove your failed builds /tmp/guix-build-* directories, if you have
any.

Then I looped through with this little bash script:

#!/bin/sh

ROUNDS=100

for i in `seq -w 0 $ROUNDS`; do
#    DATE=${date}
#    echo $DATE
    echo -n $i..
    ./pre-inst-env guix build -e '(@@ (gnu packages java)
ant-bootstrap)' --no-grafts --no-substitutes --check -k -K >log-$i.log
2>&1

done;
echo

Then search in the logs:

grep Segementation log-*.log

Hopefully it finds one. Otherwise, repeat step above.
Check that it not onle Segfaults,but also has a "(core dumped)" behind
it. Otherwise, check your ulimit and corefile settings.

The coredump is in the /tmp/guix-build-ant..-n, where n coresponds to
your log-file number.

Finally, exporting the stack trace:

set logging on
set logging file backtrace.log
show logging
bt
info reg
quit


That's it.

Björn

Send a report that this bug log contains spam.