#78404 - [PATCH 0/2] Go: Module aware build system

Package	Source(s)		Maintainer(s)
guix-patches		PTS Buildd Popcon

Message #5 received at submit@debbugs.gnu.org (full text, mbox, reply):

Received: (at submit) by debbugs.gnu.org; 13 May 2025 09:40:58 +0000
From debbugs-submit-bounces@debbugs.gnu.org Tue May 13 05:40:58 2025
Received: from localhost ([127.0.0.1]:58050 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.84_2)
	(envelope-from <debbugs-submit-bounces@debbugs.gnu.org>)
	id 1uEm89-00087K-KL
	for submit@debbugs.gnu.org; Tue, 13 May 2025 05:40:58 -0400
Received: from lists.gnu.org ([2001:470:142::17]:34926)
 by debbugs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256)
 (Exim 4.84_2) (envelope-from <j@lambda.is>) id 1uEm83-00086u-SG
 for submit@debbugs.gnu.org; Tue, 13 May 2025 05:40:54 -0400
Received: from eggs.gnu.org ([2001:470:142:3::10])
 by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256)
 (Exim 4.90_1) (envelope-from <j@lambda.is>) id 1uEm7r-0000uh-OP
 for guix-patches@gnu.org; Tue, 13 May 2025 05:40:40 -0400
Received: from mx.kolabnow.com ([212.103.80.155])
 by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256)
 (Exim 4.90_1) (envelope-from <j@lambda.is>) id 1uEm7n-0004w0-ME
 for guix-patches@gnu.org; Tue, 13 May 2025 05:40:39 -0400
Received: from localhost (unknown [127.0.0.1])
 by mx.kolabnow.com (Postfix) with ESMTP id 7795A30051EA;
 Tue, 13 May 2025 11:40:30 +0200 (CEST)
Authentication-Results: ext-mx-out013.mykolab.com (amavis); dkim=pass
 reason="pass (just generated, assumed good)" header.d=lambda.is
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=lambda.is; h=
 content-transfer-encoding:content-type:content-type:mime-version
 :message-id:date:date:subject:subject:from:from:received
 :received:received; s=dkim2; t=1747129229; x=1748943630; bh=kByy
 ZYqV0N9BCT9kukwtYoYBZGLA+EtPPxBFe46VQks=; b=dtF/zZMJwc46pyQftwBw
 Gnb++hJGuM6i5b/oq9Q4WLWBb7YRcRe04xwDpOoVTN8VTYWwlyDBcKyc/V7yWMee
 zc1rLvBUOrneiM60PEcxtXa1XKX4bPcR+ZfIJMI4Foeqs4+H4/FSlrAa14rh+I1j
 CKPfwBj7I7/gnGk9IfFtDIfjThV7Aa0JSPHCCCrIFUUNI3HVpMBMaPygpZUFXx7s
 YRNZ0e/tnej42bBUvPolaEhr2Hhmc0o/h8QNSwotEuiqOY6Q2uow7motEq6duC6K
 WuhbwpGC31q871OXvXJXSOWm/VcVFU6hjzqxXcnsqQw1YPYMjq7KRUsw3vIk7wMg
 XQ==
X-Virus-Scanned: amavis at mykolab.com
X-Spam-Flag: NO
X-Spam-Score: 0
X-Spam-Level: 
X-Spam-Status: No, score=0 tagged_above=-10 required=5 tests=[none]
 autolearn=ham autolearn_force=no
Received: from mx.kolabnow.com ([127.0.0.1])
 by localhost (ext-mx-out013.mykolab.com [127.0.0.1]) (amavis, port 10024)
 with ESMTP id 5z85xvDWD2LT; Tue, 13 May 2025 11:40:29 +0200 (CEST)
Received: from int-mx011.mykolab.com (unknown [10.9.13.11])
 by mx.kolabnow.com (Postfix) with ESMTPS id 07B9E30051E3;
 Tue, 13 May 2025 11:40:28 +0200 (CEST)
Received: from ext-subm010.mykolab.com (unknown [10.9.6.10])
 by int-mx011.mykolab.com (Postfix) with ESMTPS id 97BE832085EE;
 Tue, 13 May 2025 11:40:28 +0200 (CEST)
From: Jørgen Kvalsvik <j@lambda.is>
To: guix-patches@gnu.org
Subject: [PATCH 0/2] Go: Module aware build system
Date: Tue, 13 May 2025 11:40:23 +0200
Message-Id: <20250513094023.348947-1-j@lambda.is>
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit
Received-SPF: pass client-ip=212.103.80.155; envelope-from=j@lambda.is;
 helo=mx.kolabnow.com
X-Spam_score_int: -16
X-Spam_score: -1.7
X-Spam_bar: -
X-Spam_report: (-1.7 / 5.0 requ) BAYES_00=-1.9, DKIM_INVALID=0.1,
 DKIM_SIGNED=0.1, RCVD_IN_VALIDITY_CERTIFIED_BLOCKED=0.001,
 RCVD_IN_VALIDITY_RPBL_BLOCKED=0.001, SPF_HELO_NONE=0.001,
 SPF_PASS=-0.001 autolearn=no autolearn_force=no
X-Spam_action: no action
X-Spam-Score: 0.0 (/)
X-Debbugs-Envelope-To: submit
Cc: Jørgen Kvalsvik <j@lambda.is>, steve@futurile.net
X-BeenThere: debbugs-submit@debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request@debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit@debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request@debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request@debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces@debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces@debbugs.gnu.org>
X-Spam-Score: -1.0 (-)

Hi,

This is a new build system for go, based on go modules. I'm submitting
this now to get an issue number, collect some feedback, and iron out the
last few wrinkles, but we have used an internal version of it for a
while and it works well for us. I have tweaked it a bit in preparation
for this patch series, and I do expect it to take a couple of
iterations.

The new build system can use packages built with go-build-system as
inputs, which I think is a prerequisite. There is already a large
collection of go libraries in guix, and we want to use that work
well. That being said, this build system should graudually replace the
go-build-system. Because they are (largely) compatible as inputs,
packages can be migrated gradually. The library or program being built does
not have to be "module aware" (have a go.mod file) to use this build system, a
new go.mod will be created unconditionally.

It addresses most of the shortcomings of the current go build
system. Here's the list from go.scm and how this system is different:

* Avoid copying dependencies into the build environment and / or avoid
  using a tmpdir when creating the inputs union.
    We still copy build dependencies into the build dir, so that go
    build can "fetch" dependencies as it likes. There is a path where
    this can be avoided by using replace directives in go.mod, but this
    is a significant complication and requires build inputs to be proper
    modules, which is not guaranteed.

* Use Go modules
    Yes.

* Re-use compiled packages
    Yes. We install the go build cache under $out/var, and seed
    downstream builds with this cache.

* Avoid the go-inputs hack
    Sort-of? We look at package name, but also the "shape" of the
    package, namely the presence of the src/ directory. This seems to
    work ok.

* Remove module packages, only offering the full Git repos? This is more
  idiomatic, I think, because Go downloads Git repos, not modules.
    There is the go-mod-fetch which downloads modules (as zips). The
    build system does not particularly care about the source, and works
    well both with git clone, hg clone, and url fetch.

* Figurie out how to passthrough --verbosity option to "build" and
  "check"
    Not addressed yet.

* Implement test-backend option, which would be similar to pyproject's
  one, allowing to provide custom test runner.
    Not really, but most go projects are just tested with `go test
    ./...` or `go test ./dir1 ./dir2 ...`. There are options for both
    test flags and test targets. Anything else probably warrants a
    custom check phase.

I have a prototype of guix import go-module ..., too, but it's not quite
ready yet. I will add onto this series a few packages to demonstrate the
build system, and port a few packages from go-build-system to
go-module-build-system.

Jørgen Kvalsvik (2):
  guix: Add downloader for go modules from GOPROXY
  guix: Add module-aware build system for go

 Makefile.am                           |   3 +
 guix/build-system/go-module.scm       | 268 +++++++++++++++
 guix/build/go-module-build-system.scm | 457 ++++++++++++++++++++++++++
 guix/go-mod-download.scm              | 126 +++++++
 4 files changed, 854 insertions(+)
 create mode 100644 guix/build-system/go-module.scm
 create mode 100644 guix/build/go-module-build-system.scm
 create mode 100644 guix/go-mod-download.scm

-- 
2.39.5

Display info messages

Send a report that this bug log contains spam.

debbugs.gnu.org maintainers <help-debbugs@gnu.org>. Last modified: Wed Sep 10 11:01:18 2025; Machine Name: wallace-server

GNU bug tracking system

Debbugs is free software and licensed under the terms of the GNU Public License version 2. The current version can be obtained from https://bugs.debian.org/debbugs-source/.

#78404 [PATCH 0/2] Go: Module aware build system