#41484 guix gc fails catastrophically if the disk has no space left

Message #11 received at submit@debbugs.gnu.org (full text, mbox, reply):

Received: (at submit) by debbugs.gnu.org; 23 May 2020 19:57:41 +0000
From debbugs-submit-bounces@debbugs.gnu.org Sat May 23 15:57:41 2020
Received: from localhost ([127.0.0.1]:35774 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.84_2)
	(envelope-from <debbugs-submit-bounces@debbugs.gnu.org>)
	id 1jcaH6-0001iM-VH
	for submit@debbugs.gnu.org; Sat, 23 May 2020 15:57:41 -0400
Received: from lists.gnu.org ([209.51.188.17]:37438)
 by debbugs.gnu.org with esmtp (Exim 4.84_2)
 (envelope-from <raingloom@riseup.net>) id 1jcaH5-0001iF-IW
 for submit@debbugs.gnu.org; Sat, 23 May 2020 15:57:39 -0400
Received: from eggs.gnu.org ([2001:470:142:3::10]:58060)
 by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256)
 (Exim 4.90_1) (envelope-from <raingloom@riseup.net>)
 id 1jcaH5-0004hD-A2
 for bug-guix@gnu.org; Sat, 23 May 2020 15:57:39 -0400
Received: from mx1.riseup.net ([198.252.153.129]:54816)
 by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256)
 (Exim 4.90_1) (envelope-from <raingloom@riseup.net>)
 id 1jcaH3-00076B-Ot
 for bug-guix@gnu.org; Sat, 23 May 2020 15:57:38 -0400
Received: from bell.riseup.net (bell-pn.riseup.net [10.0.1.178])
 (using TLSv1 with cipher ECDHE-RSA-AES256-SHA (256/256 bits))
 (Client CN "*.riseup.net",
 Issuer "Sectigo RSA Domain Validation Secure Server CA" (not verified))
 by mx1.riseup.net (Postfix) with ESMTPS id 49TvKD3NbVzFfWF
 for <bug-guix@gnu.org>; Sat, 23 May 2020 12:57:36 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=riseup.net; s=squak;
 t=1590263856; bh=Ze7tsUgc15uRofwEVuCy2DVUZM4xDTQLrkdgos0TxFo=;
 h=Date:From:To:Subject:In-Reply-To:References:From;
 b=g/KaDFpsF5TOnUjodbe1RAwWGGJwo5pmYguQUAJRt2UfO0d4iNnv9dgdgsu3nUpAE
 DOnaGmoZtVyLoMYt0cOVWKjzX8wV46y2qNSiyK+EVidZ23d2uOR5sxeCT8Mp1wwjqh
 C9cpW1fNWkWmmjpvHVBxovl25CrFMFzi1ODOTlG8=
X-Riseup-User-ID: 609C49D477E72A8B38F11DBAD820BD006876004DCCD57A0F4AD0192EE50C0555
Received: from [127.0.0.1] (localhost [127.0.0.1])
 by bell.riseup.net (Postfix) with ESMTPSA id 49TvKD0DNbzJmhc
 for <bug-guix@gnu.org>; Sat, 23 May 2020 12:57:35 -0700 (PDT)
Date: Sat, 23 May 2020 21:57:33 +0200
From: raingloom <raingloom@riseup.net>
To: bug-guix@gnu.org
Subject: Re: bug#41484: guix gc fails catastrophically if the disk has no
 space left
Message-ID: <20200523215733.2ed2f0de@riseup.net>
In-Reply-To: <A33D3EFC-90DD-4E78-AFE5-AB0BF146373A@vodafonemail.de>
References: <A33D3EFC-90DD-4E78-AFE5-AB0BF146373A@vodafonemail.de>
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: quoted-printable
Received-SPF: pass client-ip=198.252.153.129;
 envelope-from=raingloom@riseup.net; helo=mx1.riseup.net
X-detected-operating-system: by eggs.gnu.org: First seen = 2020/05/23 15:57:36
X-ACL-Warn: Detected OS   = Linux 3.11 and newer
X-Spam_score_int: -27
X-Spam_score: -2.8
X-Spam_bar: --
X-Spam_report: (-2.8 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1,
 DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1,
 RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H3=0.001, RCVD_IN_MSPIKE_WL=0.001,
 SPF_HELO_PASS=-0.001, SPF_PASS=-0.001,
 URIBL_BLOCKED=0.001 autolearn=_AUTOLEARN
X-Spam_action: no action
X-Spam-Score: -1.4 (-)
X-Debbugs-Envelope-To: submit
X-BeenThere: debbugs-submit@debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request@debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit@debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request@debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request@debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces@debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces@debbugs.gnu.org>
X-Spam-Score: -2.4 (--)

On Sat, 23 May 2020 16:49:02 +0200
Stefan <stefan-guix@vodafonemail.de> wrote:

> Hi!
> 
> While trying to build guix, my disk ran out of space.
> 
> building
> /gnu/store/w5l7acwbfvn0n8zj2v9by4j3jwbiimj9-guix-1.1.0-4.bdc801e.drv...
> 100%
> [#########################################################################]note:
> build failure may have been caused by lack of free disk space builder
> for
> `/gnu/store/w5l7acwbfvn0n8zj2v9by4j3jwbiimj9-guix-1.1.0-4.bdc801e.drv'
> failed with exit code 1 build of
> /gnu/store/w5l7acwbfvn0n8zj2v9by4j3jwbiimj9-guix-1.1.0-4.bdc801e.drv
> failed View build log at
> '/var/log/guix/drvs/w5/l7acwbfvn0n8zj2v9by4j3jwbiimj9-guix-1.1.0-4.bdc801e.drv.bz2'.
> guix system: error: build of
> `/gnu/store/w5l7acwbfvn0n8zj2v9by4j3jwbiimj9-guix-1.1.0-4.bdc801e.drv'
> failed
> 
> stefan@guix ~$ df -h /
> Dateisystem                                       Größe Benutzt Verf.
> Verw% Eingehängt auf
> :/volume5/RaspberryPi/Betriebssysteme/guix-system  1,4T    1,4T     0
>  100% /
> 
> OK, well, there’s certainly a lot of garbage to collect. Due to bad
> experiences in the past with the same situation, I first try to
> collect only a little garbage.
> 
> stefan@guix ~$ guix gc -F 100M
> guix gc: 100 MiB werden freigegeben
> finding garbage collector roots...
> note: can't create trash directory: creating directory
> `/gnu/store/trash': No space left on device deleting garbage...
> [0%] deleting
> '/gnu/store/g30aknl5yxk9jbpx2w7r5d0ak6xa5dl2-grub.cfg.lock' [0%]
> deleting
> '/gnu/store/gidc7jrgbnvfm38qx4sc012kn9llgsvz-package-collection-grub-efi-raspberrypi-firmware-u-boot-rpi-3-2.04-collection.lock'
> [0%] deleting
> '/gnu/store/9cpj3ir8fnsg5fiqs3x8alai713xzfz0-grub-keymap.de.lock'
> [0%] deleting
> '/gnu/store/z08s3kwlnpqq6aa1v6ydni5qp2k7gkkb-package-collection-grub-efi-raspberrypi-firmware-u-boot-rpi-3-2.04'
> [0%] deleting
> '/gnu/store/wv7iwd66q6n5775qa4hgaq9j9f2vyvl4-guix-1.1.0-4.bdc801e.lock'
> [0%] deleting
> '/gnu/store/pab2ylsvas12824qw6j8q2h1qm3gh4x6-package-collection-grub-efi-raspberrypi-firmware-u-boot-rpi-3-2.04-union.lock'
> [0%] deleting
> '/gnu/store/3ldyga22s6ym9g2pc9ycjwdwqb4i41v2-package-collection-grub-efi-raspberrypi-firmware-u-boot-rpi-3-2.04.lock'
> [0%] deleting
> '/gnu/store/nfxgqhjyzwngiadn0l180hsbfnw3h4zy-package-collection-grub-efi-raspberrypi-firmware-u-boot-rpi-3-2.04-collection.lock'
> [0%] deleting
> '/gnu/store/pab2ylsvas12824qw6j8q2h1qm3gh4x6-package-collection-grub-efi-raspberrypi-firmware-u-boot-rpi-3-2.04-union'
> [0%] deleting
> '/gnu/store/dahbiixsbi1xhjiim9phl1vginm83bbm-grub.cfg.lock' [0%]
> deleting
> '/gnu/store/61mwka1h77kkhylcs9jj4ij167qgfj39-package-collection-grub-efi-raspberrypi-firmware-u-boot-rpi-3-2.04.lock'
> [0%] deleting
> '/gnu/store/q829h36vnwhbdlnzkgdrx2b51kp2mvdv-grub.cfg.lock' [0%]
> deleting '/gnu/store/ragpi42x39sf5qwcmcmdn751fp4jfhmv-grub.cfg.lock'
> [0%] deleting
> '/gnu/store/ncdsjfl245k6r7zym6w4sr9fyxrwhb38-grub.cfg.lock' [0%]
> deleting
> '/gnu/store/z08s3kwlnpqq6aa1v6ydni5qp2k7gkkb-package-collection-grub-efi-raspberrypi-firmware-u-boot-rpi-3-2.04.lock'
> [0%] deleting
> '/gnu/store/gidc7jrgbnvfm38qx4sc012kn9llgsvz-package-collection-grub-efi-raspberrypi-firmware-u-boot-rpi-3-2.04-collection'
> [0%] deleting
> '/gnu/store/61mwka1h77kkhylcs9jj4ij167qgfj39-package-collection-grub-efi-raspberrypi-firmware-u-boot-rpi-3-2.04'
> [0%] deleting
> '/gnu/store/hmy4ni005digrilzjf93wr9gaj5cq536-computed-files.lock'
> [0%] deleting
> '/gnu/store/hmy4ni005digrilzjf93wr9gaj5cq536-computed-files' error
> (ignored): aborting transaction: cannot rollback - no transaction is
> active guix gc: error: committing transaction: disk I/O error
> stefan@guix ~$ 
> 
> And that’s it. Repeating the guix gc command now results in this
> immediate error:
> 
> stefan@guix ~$ guix gc -F 100M
> warning: SQLite database is busy (SQLITE_PROTOCOL)
> guix gc: error: setting synchronous mode: locking protocol
> stefan@guix ~$ guix gc -F 100M
> warning: SQLite database is busy (SQLITE_PROTOCOL)
> guix gc: error: setting synchronous mode: locking protocol
> 
> A reboot is needed to solve this SQLite error. But retrying all this
> leads to the same situation again and again. It is kind of a deadlock.
> 
> Well, as my guix system boots from an NFS share, in the past I solved
> this by removing other stuff – not related to guix – from the same
> disk. But right now there isn't unrelated stuff left to be deleted. 
> 
> I intend to remove the listed /gnu/store/… files by hand from the NFS
> server side (which is much easier than removing the read-only
> bind-mount.) Is this save at least?
> 
> This is all on a Raspberry Pi 3b aarch64 machine, if it matters.
> 
> 
> Bye
> 
> Stefan
> 
> 
> 
> 

Try deleting /var/log. That's what I did on my tiny Pentium II desktop
with 4 gigs of HDD space.

Send a report that this bug log contains spam.