custom toolchain blog post

  • Done
  • quality assurance status badge
Details
2 participants
  • Ludovic Courtès
  • Mitchell Schmeisser
Owner
unassigned
Submitted by
Mitchell Schmeisser
Severity
normal

Debbugs page

M
M
Mitchell Schmeisser wrote on 24 Feb 2023 10:51
(address . guix-patches@gnu.org)
87sfeuucab.fsf@librem.one
Here is a rough draft of my toolchain post.
I hope it can make an interesting contribution to the Guix literature.

- Mitchell


From 4f6c43091ffd67cdbc5f041e496f61bc8a06070e Mon Sep 17 00:00:00 2001
From: Mitchell Schmeisser <mitchellschmeisser@librem.one>
Date: Fri, 24 Feb 2023 13:02:05 -0500
Subject: [PATCH] website: Add custom toolchain blog post

* website/posts/custom-toolchains-with-guix.md: New file.
---
website/posts/custom-toolchains-with-guix.md | 557 +++++++++++++++++++
1 file changed, 557 insertions(+)
create mode 100644 website/posts/custom-toolchains-with-guix.md

Toggle diff (540 lines)
diff --git a/website/posts/custom-toolchains-with-guix.md b/website/posts/custom-toolchains-with-guix.md
new file mode 100644
index 0000000..f73f1ab
--- /dev/null
+++ b/website/posts/custom-toolchains-with-guix.md
@@ -0,0 +1,557 @@
+# Table of Contents
+
+1. [Overview](#org2633a51)
+2. [Anatomy of a toolchain](#orgc440e9e)
+3. [Bootstrapping a Toolchain](#orgd42b6c3)
+4. [Defining the Packages](#org55042c5)
+ 1. [Binutils](#org67da1ec)
+ 2. [GCC sans libc](#org82d6f83)
+ 3. [Newlib(-nano)](#orgf6bafbc)
+ 4. [Complete toolchain](#org052f2a2)
+5. [Integrating with Zephyr Build System](#orgc3f87f4)
+ 1. [Testing](#org9f3c314)
+
+All code is available at [guix-zephyr](https://github.com/paperclip4465/guix-zephyr) channel.
+
+
+<a id="org2633a51"></a>
+
+# Overview
+
+In order to deploy embedded software using Guix we first need to teach Guix
+how to build it. Since Guix bootstraps everything this means we must teach Guix
+how to build our toolchain.
+
+The [Zephyr Project](https://zephyrproject.org) uses its own fork of GCC with custom configs for
+the architectures supported by the project.
+
+
+<a id="orgc440e9e"></a>
+
+# Anatomy of a toolchain
+
+Toolchains are responsible for taking high level descriptions of programs
+and lowering them down to a series of equivalent machine instructions.
+This process involves more than just a compiler. The compiler uses the `binutils`
+to manipulate it's internal representation down to a given architecture.
+It also needs the use of the C standard library as well as a few other libraries
+needed for some compiler optimizations.
+
+The C library provides the interface to the underlying kernel. System calls like `write`
+and `read` are provided by `Glibc` on most Linux distributions.
+
+In embedded systems smaller implementations like `newlib` and `newlib-nano` are used.
+
+
+<a id="orgd42b6c3"></a>
+
+# Bootstrapping a Toolchain
+
+In order to compile GCC we need a C library that's been compiled for
+our target architecture. How can we cross compile our C library if we
+need our C library to build a cross compiler? The solution is to build
+a simpler compiler that doesn't require the C library to function.
+It will not be capable of as many optimizations and it will be very slow,
+however it will be able to build the C libraries as well as the complete version
+of GCC.
+
+In order to build the simpler compiler we need to compile the `binutils` to
+work with our target architecture.
+The `binutils` can be bootstrapped with our host GCC and have no target dependencies.
+
+[For more information read this.](https://crosstool-ng.github.io/docs/toolchain-construction/)
+
+Doesn't sound so bad right? It isn't&#x2026; in theory.
+However internet forums since time immemorial have been
+littered with the laments of those who came before.
+From incorrect versions of `ISL` to the wrong C library being linked
+or the host linker being used, etc.
+The one commonality between all of these issues is the environment.
+Building a cross toolchain is difficult because isolating build
+environments is difficult.
+
+In fact as of `v0.14.2` the zephyr SDK repository took down the build
+instructions and posted a sign that read
+"Building this is too complicated, don't worry about it."<sup><a id="fnr.1" class="footref" href="#fn.1" role="doc-backlink">1</a></sup>
+
+We will neatly side step all of these problems and not
+risk destroying or polluting our host system with garbage
+by using Guix to manage our environments for us.
+
+Our toolchain only requires the first pass compiler because
+newlib(-nano) is statically linked and introduced to the toolchain
+by normal package composition.
+
+
+<a id="org55042c5"></a>
+
+# Defining the Packages
+
+All of the base packages are defined in `zephyr/packages/zephyr.scm`.
+Zephyr modules are defined in `zephyr/packages/zephyr-xyz.scm`, following
+the pattern of other module systems implemented by Guix.
+
+
+<a id="org67da1ec"></a>
+
+## Binutils
+
+First thing we need to build is the `arm-zephyr-eabi` binutils.
+This is very easy in Guix.
+
+ (define-module (zephyr packages zephyr)
+ #:use-module (guix packages))
+
+ (define-public arm-zephyr-eabi-binutils
+ (let ((xbinutils (cross-binutils "arm-zephyr-eabi")))
+ (package
+ (inherit xbinutils)
+ (name "arm-zephyr-eabi-binutils")
+ (version "2.38")
+ (source
+ (origin (method git-fetch)
+ (uri (git-reference
+ (url "https://github.com/zephyrproject-rtos/binutils-gdb")
+ (commit "6a1be1a6a571957fea8b130e4ca2dcc65e753469")))
+ (file-name (git-file-name name version))
+ (sha256 (base32 "0ylnl48jj5jk3jrmvfx5zf8byvwg7g7my7jwwyqw3a95qcyh0isr"))))
+ (arguments
+ `(#:tests? #f
+ ,@(substitute-keyword-arguments (package-arguments xbinutils)
+ ((#:configure-flags flags)
+ `(cons "--program-prefix=arm-zephyr-eabi-" ,flags)))))
+ (native-inputs
+ (append
+ (list texinfo
+ bison
+ flex
+ gmp
+ dejagnu)
+ (package-native-inputs xbinutils)))
+ (home-page "https://zephyrproject.org")
+ (synopsis "binutils for zephyr RTOS"))))
+
+The function `cross-binutils` returns a package which has been
+configured for the given gnu triplet. We simply inherit that package
+and replace the source.
+The zephyr build system expects the binutils to be prefixed with
+`arm-zephyr-eabi-` which is accomplished by adding another flag to the
+`#:configure-flags` argument.
+
+We can test our package definition using the `-L` flag with `guix build`
+to add our packages.
+
+ guix build -L guix-zephyr zephyr-binutils
+
+ /gnu/store/...-zephyr-binutils-2.38
+
+This directory contains the results of `make install`.
+
+
+<a id="org82d6f83"></a>
+
+## GCC sans libc
+
+This one is a bit more involved. Don't be afraid!
+This version of GCC wants ISL version 0.15. It's easy enough
+to make that happen. Inherit the current version of ISL and swap
+out the source and update the version. For most packages the build process doesn't
+change that much between versions.
+
+ (define-public isl-0.15
+ (package
+ (inherit isl)
+ (version "0.15")
+ (source (origin
+ (method url-fetch)
+ (uri (list (string-append "mirror://sourceforge/libisl/isl-"
+ version ".tar.gz")))
+ (sha256
+ (base32
+ "11vrpznpdh7w8jp4wm4i8zqhzq2h7nix71xfdddp8xnzhz26gyq2"))))))
+
+Like the binutils, there is a function for creating cross-gcc packages.
+This one accepts keywords specifying which binutils and libc to use.
+If libc isn't given (like here), gcc is configured with many options disabled
+to facilitate being built without libc. Therefore we need to add the extra options
+we want <sup><a id="fnr.2" class="footref" href="#fn.2" role="doc-backlink">2</a></sup>
+
+ (define-public gcc-arm-zephyr-eabi-12
+ (let ((xgcc (cross-gcc "arm-zephyr-eabi"
+ #:xbinutils zephyr-binutils)))
+ (package
+ (inherit xgcc)
+ (version "12.1.0")
+ (source (origin (method git-fetch)
+ (uri (git-reference
+ (url "https://github.com/zephyrproject-rtos/gcc")
+ (commit "0218469df050c33479a1d5be3e5239ac0eb351bf")))
+ (file-name (git-file-name (package-name xgcc) version))
+ (sha256
+ (base32 "1s409qmidlvzaw1ns6jaanigh3azcxisjplzwn7j2n3s33b76zjk"))
+ (patches
+ (search-patches "gcc-12-cross-environment-variables.patch"
+ "gcc-cross-gxx-include-dir.patch"))))
+ (native-inputs
+ (modify-inputs (package-native-inputs xgcc)
+ ;; Get rid of stock ISL
+ (delete "isl")
+ ;; Add additional dependencies that xgcc doesn't have
+ ;; including our special ISL
+ (prepend flex
+ perl
+ python-3
+ gmp
+ isl-0.15
+ texinfo
+ python
+ mpc
+ mpfr
+ zlib)))
+ (arguments
+ (substitute-keyword-arguments (package-arguments xgcc)
+ ((#:phases phases)
+ `(modify-phases ,phases
+ (add-after 'unpack 'fix-genmultilib
+ (lambda _
+ (substitute* "gcc/genmultilib"
+ (("#!/bin/sh") (string-append "#!" (which "sh"))))
+ #t))
+
+ (add-after 'set-paths 'augment-CPLUS_INCLUDE_PATH
+ (lambda* (#:key inputs #:allow-other-keys)
+ (let ((gcc (assoc-ref inputs "gcc")))
+ ;; Remove the default compiler from CPLUS_INCLUDE_PATH to
+ ;; prevent header conflict with the GCC from native-inputs.
+ (setenv "CPLUS_INCLUDE_PATH"
+ (string-join
+ (delete (string-append gcc "/include/c++")
+ (string-split (getenv "CPLUS_INCLUDE_PATH")
+ #\:))
+ ":"))
+ (format #t
+ "environment variable `CPLUS_INCLUDE_PATH' changed to ~a~%"
+ (getenv "CPLUS_INCLUDE_PATH"))
+ #t)))))
+
+ ((#:configure-flags flags)
+ ;; The configure flags are largely identical to the flags used by the
+ ;; "GCC ARM embedded" project.
+ `(append (list "--enable-multilib"
+ "--with-newlib"
+ "--with-multilib-list=rmprofile"
+ "--with-host-libstdcxx=-static-libgcc -Wl,-Bstatic,-lstdc++,-Bdynamic -lm"
+ "--enable-plugins"
+ "--disable-decimal-float"
+ "--disable-libffi"
+ "--disable-libgomp"
+ "--disable-libmudflap"
+ "--disable-libquadmath"
+ "--disable-libssp"
+ "--disable-libstdcxx-pch"
+ "--disable-nls"
+ "--disable-shared"
+ "--disable-threads"
+ "--disable-tls"
+ "--with-gnu-ld"
+ "--with-gnu-as"
+ "--enable-initfini-array")
+ (delete "--disable-multilib" ,flags)))))
+ (native-search-paths
+ (list (search-path-specification
+ (variable "CROSS_C_INCLUDE_PATH")
+ (files '("arm-zephyr-eabi/include")))
+ (search-path-specification
+ (variable "CROSS_CPLUS_INCLUDE_PATH")
+ (files '("arm-zephyr-eabi/include"
+ "arm-zephyr-eabi/c++"
+ "arm-zephyr-eabi/c++/arm-zephyr-eabi")))
+ (search-path-specification
+ (variable "CROSS_LIBRARY_PATH")
+ (files '("arm-zephyr-eabi/lib")))))
+ (home-page "https://zephyrproject.org")
+ (synopsis "GCC for zephyr RTOS"))))
+
+This GCC can be built like so.
+
+ guix build -L guix-zephyr gcc-cross-sans-libc-arm-zephyr-eabi
+
+ /gnu/store/...-gcc-cross-sans-libc-arm-zephyr-eabi-12.1.0-lib
+ /gnu/store/...-gcc-cross-sans-libc-arm-zephyr-eabi-12.1.0
+
+Great! We now have our stage-1 compiler.
+
+
+<a id="orgf6bafbc"></a>
+
+## Newlib(-nano)
+
+The newlib package package is quite straight forward (relatively).
+It is mostly adding in the relevent configuration flags and patching
+the files the `patch-shebangs` phase missed.
+
+ (define-public zephyr-newlib
+ (package
+ (name "zephyr-newlib")
+ (version "3.3")
+ (source (origin
+ (method git-fetch)
+ (uri (git-reference
+ (url "https://github.com/zephyrproject-rtos/newlib-cygwin")
+ (commit "4e150303bcc1e44f4d90f3489a4417433980d5ff")))
+ (sha256
+ (base32 "08qwjpj5jhpc3p7a5mbl7n6z7rav5yqlydqanm6nny42qpa8kxij"))))
+ (build-system gnu-build-system)
+ (arguments
+ `(#:out-of-source? #t
+ #:configure-flags '("--target=arm-zephyr-eabi"
+ "--enable-newlib-io-long-long"
+ "--enable-newlib-io-float"
+ "--enable-newlib-io-c99-formats"
+ "--enable-newlib-retargetable-locking"
+ "--enable-newlib-lite-exit"
+ "--enable-newlib-multithread"
+ "--enable-newlib-register-fini"
+ "--enable-newlib-extra-sections"
+ "--disable-newlib-wide-orient"
+ "--disable-newlib-fseek-optimization"
+ "--disable-newlib-supplied-syscalls"
+ "--disable-newlib-target-optspace"
+ "--disable-nls")
+ #:phases
+ (modify-phases %standard-phases
+ (add-after 'unpack 'fix-references-to-/bin/sh
+ (lambda _
+ (substitute* '("libgloss/arm/cpu-init/Makefile.in"
+ "libgloss/arm/Makefile.in"
+ "libgloss/libnosys/Makefile.in"
+ "libgloss/Makefile.in")
+ (("/bin/sh") (which "sh")))
+ #t)))))
+ (native-inputs
+ `(("xbinutils" ,zephyr-binutils)
+ ("xgcc" ,gcc-arm-zephyr-eabi-12)
+ ("texinfo" ,texinfo)))
+ (home-page "https://www.sourceware.org/newlib/")
+ (synopsis "C library for use on embedded systems")
+ (description "Newlib is a C library intended for use on embedded
+ systems. It is a conglomeration of several library parts that are easily
+ usable on embedded products.")
+ (license (license:non-copyleft
+ "https://www.sourceware.org/newlib/COPYING.NEWLIB"))))
+
+And the build.
+
+ guix build -L guix-zephyr zephyr-newlib
+
+ /gnu/store/...-zephyr-newlib-3.3
+
+
+<a id="org052f2a2"></a>
+
+## Complete toolchain<sup><a id="fnr.3" class="footref" href="#fn.3" role="doc-backlink">3</a></sup>
+
+Now that we've got the individual tools it's time to create our complete toolchain.
+For this we need to do some package transformations.
+Because these transformations are going to have to be done for every combination of
+gcc/newlib it is best to create a function which we can reuse for every version
+of the SDK.
+
+ (define (arm-zephyr-eabi-toolchain xgcc newlib version)
+ "Produce a cross-compiler zephyr toolchain package with the compiler XGCC and the C
+ library variant NEWLIB."
+ (let ((newlib-with-xgcc (package (inherit newlib)
+ (native-inputs
+ (alist-replace "xgcc" (list xgcc)
+ (package-native-inputs newlib))))))
+ (package
+ (name (string-append "arm-zephyr-eabi"
+ (if (string=? (package-name newlib-with-xgcc)
+ "newlib-nano")
+ "-nano" "")
+ "-toolchain"))
+ (version version)
+ (source #f)
+ (build-system trivial-build-system)
+ (arguments
+ '(#:modules ((guix build union)
+ (guix build utils))
+ #:builder
+ (begin
+ (use-modules (ice-9 match)
+ (guix build union)
+ (guix build utils))
+ (let ((out (assoc-ref %outputs "out")))
+ (mkdir-p out)
+ (match %build-inputs
+ (((names . directories) ...)
+ (union-build (string-append out "/arm-zephyr-eabi")
+ directories)
+ #t))))))
+ (inputs
+ `(("binutils" ,zephyr-binutils)
+ ("gcc" ,xgcc)
+ ("newlib" ,newlib-with-xgcc)))
+ (synopsis "Complete GCC tool chain for ARM zephyrRTOS development")
+ (description "This package provides a complete GCC tool chain for ARM
+ bare metal development with zephyr rtos. This includes the GCC arm-zephyr-eabi cross compiler
+ and newlib (or newlib-nano) as the C library. The supported programming
+ language is C.")
+ (home-page (package-home-page xgcc))
+ (license (package-license xgcc)))))
+
+This function creates a special package which consists of the toolchain in a special directory hierarchy, i.e `arm-zephyr-eabi/`.
+Our complete toolchain definition looks like this.
+
+ (define-public arm-zephyr-eabi-toolchain-0.15.0
+ (arm-zephyr-eabi-toolchain
+ gcc-arm-zephyr-eabi-12
+ zephyr-newlib
+ "0.15.0"))
+
+To build:
+
+ guix build -L guix-zephyr arm-zephyr-eabi-toolchain
+
+ /gnu/store/...-arm-zephyr-eabi-toolchain-0.15.0
+
+
+<a id="orgc3f87f4"></a>
+
+# Integrating with Zephyr Build System
+
+Zephyr uses CMake as it's build system. It contains numerous CMake
+files in both the so-called `ZEPHYR_BASE`, the zephyr source code
+repository, as well as a handful in the SDK
+which help select the correct toolchain for a given board.
+
+There are standard locations the build system will look for the
+SDK. We are not using any of them. Our SDK lives in the store,
+immutable forever. According to [this](https://docs.zephyrproject.org/latest/develop/west/without-west.html) the variable
+`ZEPHYR_SDK_INSTALL_DIR` needs to point to our custom spot.
+
+We also need to grab the cmake files from the [repository](https://github.com/zephyrproject-rtos/sdk-ng) and create a
+file `sdk_version` which contains the version string `ZEPHYR_BASE`
+uses to find a compatible SDK.
+
+Along with the SDK proper we need to include a number of python
+packages required by the build system<sup><a id="fnr.4" class="footref" href="#fn.4" role="doc-backlink">4</a></sup>.
+
+ (define-public zephyr-sdk
+ (package
+ (name "zephyr-sdk")
+ (version "0.15.0")
+ (home-page "https://zephyrproject.org")
+ (source (origin (method git-fetch)
+ (uri (git-reference
+ (url "https://github.com/zephyrproject-rtos/sdk-ng")
+ (commit "v0.15.0")))
+ (file-name (git-file-name name version))
+ (sha256 (base32 "04gsvh20y820dkv5lrwppbj7w3wdqvd8hcanm8hl4wi907lwlmwi"))))
+ (build-system trivial-build-system)
+ (arguments
+ `(#:modules ((guix build union)
+ (guix build utils))
+ #:builder
+ (begin
+ (use-modules (guix build union)
+ (ice-9 match)
+ (guix build utils))
+ (let* ((out (assoc-ref %outputs "out"))
+ (cmake-scripts (string-append (assoc-ref %build-inputs "source")
+ "/cmake"))
+ (sdk-out (string-append out "/zephyr-sdk-0.15.0")))
+ (mkdir-p out)
+
+ (match (assoc-remove! %build-inputs "source")
+ (((names . directories) ...)
+ (union-build sdk-out directories)))
+
+ (copy-recursively cmake-scripts
+ (string-append sdk-out "/cmake"))
+
+ (with-directory-excursion sdk-out
+ (call-with-output-file "sdk_version"
+ (lambda (p)
+ (format p "0.15.0")))
+ #t)))))
+ (propagated-inputs
+ (list
+ arm-zephyr-eabi-toolchain-0.15.0
+ zephyr-binutils
+ python-3
+ python-pyelftools
+ python-pykwalify
+ python-pyyaml
+ python-packaging
+ dtc))
+ (native-search-paths
+ (list (search-path-specification
+ (variable "ZEPHYR_SDK_INSTALL_DIR")
+ (files '("")))))
+ (synopsis "SDK for zephyrRTOS")
+ (description "zephyr-sdk contains bundles a complete gcc toolchain as well
+ as host tools like dtc, openocd, qemu, and required python packages.")
+ (license license:apsl2)))
+
+
+<a id="org9f3c314"></a>
+
+## Testing
+
+In order to test we will need an environment with the SDK installed.
+We can take advantage of `guix shell` to avoid installing test
+packages into our home environment. This way if it causes problems we
+can just exit the shell and try again.
+
+ guix shell -L guix-zephyr zephyr-sdk cmake ninja git
+
+`ZEPHYR_BASE` can be cloned into a temporary workspace to test our
+toolchain functionality.<sup><a id="fnr.5" class="footref" href="#fn.5" role="doc-backlink">5</a></sup>
+
+ mkdir /tmp/zephyr-project
+ cd /tmp/zephyr-project
+ git clone https://github.com/zephyrproject-rtos/zephyr
+ export ZEPHYR_BASE=/tmp/zephyr-project/zephyr
+
+In order to build for the test board (k64f in this case) we need to
+get a hold of the vendor Hardware Abstraction Layers and CMSIS.<sup><a id="fnr.6" class="footref" href="#fn.6" role="doc-backlink">6</a></sup>
+
+ git clone https://github.com/zephyrproject-rtos/hal_nxp &&
+ git clone https://github.com/zephyrproject-rtos/cmsis
+
+To inform the build system about this module we pass it in with
+`-DZEPHYR_MODULES=` which is a semicolon separated list of paths
+containing a module.yml file.
+
+To build the hello world sample we use the following incantation.
+
+ cmake -Bbuild $ZEPHYR_BASE/samples/hello_world \
+ -GNinja \
+ -DBOARD=frdm_k64f \
+ -DBUILD_VERSION=3.1.0 \
+ -DZEPHYR_MODULES="/tmp/zephyr-project/hal_nxp;/tmp/zephyr-project/cmsis" \
+ && ninja -Cb
This message was truncated. Download the full message here.
L
L
Ludovic Courtès wrote on 27 Feb 2023 04:50
(name . Mitchell Schmeisser)(address . mitchellschmeisser@librem.one)(address . 61765@debbugs.gnu.org)
87fsarmfw6.fsf@gnu.org
Hello Mitchell,

Mitchell Schmeisser <mitchellschmeisser@librem.one> skribis:

Toggle quote (3 lines)
> Here is a rough draft of my toolchain post.
> I hope it can make an interesting contribution to the Guix literature.

Yay, definitely!

Toggle quote (7 lines)
>>From 4f6c43091ffd67cdbc5f041e496f61bc8a06070e Mon Sep 17 00:00:00 2001
> From: Mitchell Schmeisser <mitchellschmeisser@librem.one>
> Date: Fri, 24 Feb 2023 13:02:05 -0500
> Subject: [PATCH] website: Add custom toolchain blog post
>
> * website/posts/custom-toolchains-with-guix.md: New file.

This looks great to me! It’s useful insight for anyone who might want
to add a cross-compilation target to Guix or simply learn how this is
implemented.

Toggle quote (20 lines)
> +++ b/website/posts/custom-toolchains-with-guix.md
> @@ -0,0 +1,557 @@
> +# Table of Contents
> +
> +1. [Overview](#org2633a51)
> +2. [Anatomy of a toolchain](#orgc440e9e)
> +3. [Bootstrapping a Toolchain](#orgd42b6c3)
> +4. [Defining the Packages](#org55042c5)
> + 1. [Binutils](#org67da1ec)
> + 2. [GCC sans libc](#org82d6f83)
> + 3. [Newlib(-nano)](#orgf6bafbc)
> + 4. [Complete toolchain](#org052f2a2)
> +5. [Integrating with Zephyr Build System](#orgc3f87f4)
> + 1. [Testing](#org9f3c314)
> +
> +All code is available at [guix-zephyr](https://github.com/paperclip4465/guix-zephyr) channel.
> +
> +
> +<a id="org2633a51"></a>

You can remove the table of contents and all the HTML snippets that
pandoc added—I don’t think that works as expected with Haunt/CommonMark.

Toggle quote (2 lines)
> +# Overview

You can drop the heading.

Toggle quote (7 lines)
> +In order to deploy embedded software using Guix we first need to teach Guix
> +how to build it. Since Guix bootstraps everything this means we must teach Guix
> +how to build our toolchain.
> +
> +The [Zephyr Project](https://zephyrproject.org) uses its own fork of GCC with custom configs for
> +the architectures supported by the project.

We want blog posts to be widely accessible so I would recommend
providing more context in the intro. In particular, I’d suggest:

1. Mentioning that Guix supports cross-compilation (not everyone is
aware of that), with a link to

2. Adding a couple of sentences saying what Zephyr is (I didn’t know
about it before :-)).

3. Saying a few words as to why Zephyr uses a GCC fork.

4. Clearly stating that you added support for cross-compilation to
Zephyr with Guix in a channel, linking to said channel, and that
this is what the remainder of the post will describe.

You can check out posts at https://guix.gnu.org/blog for inspiration.

Toggle quote (14 lines)
> +# Anatomy of a toolchain
> +
> +Toolchains are responsible for taking high level descriptions of programs
> +and lowering them down to a series of equivalent machine instructions.
> +This process involves more than just a compiler. The compiler uses the `binutils`
> +to manipulate it's internal representation down to a given architecture.
> +It also needs the use of the C standard library as well as a few other libraries
> +needed for some compiler optimizations.
> +
> +The C library provides the interface to the underlying kernel. System calls like `write`
> +and `read` are provided by `Glibc` on most Linux distributions.
>
> +In embedded systems smaller implementations like `newlib` and `newlib-nano` are used.

I’d suggest not using fixed-width font (backquotes) for proper nouns;
you can write “glibc” or “the GNU C Library (glibc)”, “Binutils” or “the
GNU Binary Utilities (Binutils)”, etc.

Toggle quote (35 lines)
> +First thing we need to build is the `arm-zephyr-eabi` binutils.
> +This is very easy in Guix.
> +
> + (define-module (zephyr packages zephyr)
> + #:use-module (guix packages))
> +
> + (define-public arm-zephyr-eabi-binutils
> + (let ((xbinutils (cross-binutils "arm-zephyr-eabi")))
> + (package
> + (inherit xbinutils)
> + (name "arm-zephyr-eabi-binutils")
> + (version "2.38")
> + (source
> + (origin (method git-fetch)
> + (uri (git-reference
> + (url "https://github.com/zephyrproject-rtos/binutils-gdb")
> + (commit "6a1be1a6a571957fea8b130e4ca2dcc65e753469")))
> + (file-name (git-file-name name version))
> + (sha256 (base32 "0ylnl48jj5jk3jrmvfx5zf8byvwg7g7my7jwwyqw3a95qcyh0isr"))))
> + (arguments
> + `(#:tests? #f
> + ,@(substitute-keyword-arguments (package-arguments xbinutils)
> + ((#:configure-flags flags)
> + `(cons "--program-prefix=arm-zephyr-eabi-" ,flags)))))
> + (native-inputs
> + (append
> + (list texinfo
> + bison
> + flex
> + gmp
> + dejagnu)
> + (package-native-inputs xbinutils)))
> + (home-page "https://zephyrproject.org")
> + (synopsis "binutils for zephyr RTOS"))))

Code snippets should be written like this, without leading indentation:

```scheme
(define …)
```

This will enable syntax highlighting.

Toggle quote (7 lines)
> +We can test our package definition using the `-L` flag with `guix build`
> +to add our packages.
> +
> + guix build -L guix-zephyr zephyr-binutils
> +
> + /gnu/store/...-zephyr-binutils-2.38

Likewise:

```
guix build foo
```

Toggle quote (4 lines)
> +# Integrating with Zephyr Build System
> +
> +Zephyr uses CMake as it's build system. It contains numerous CMake

s/it's/its/

One thing that’s not clear to me: with this in place, can you do “guix
build --target=arm-zephyr-eabi hello”, for instance? If not, what’s
missing to support it?

It would be great if you could finish with a short conclusion stating,
for instance, the key takeaway message, lessons learned, and/or your
thoughts on how this could benefit others in the broader community.

I wonder if it would be worth mentioning
one would go about adding a module. WDYT?

Could you send an updated patch?

Thanks for contributing this article!

Ludo’.
L
L
Ludovic Courtès wrote on 2 Mar 2023 10:14
control message for bug #61765
(address . control@debbugs.gnu.org)
87o7pb3tsi.fsf@gnu.org
tags 61765 + moreinfo
quit
M
M
Mitchell Schmeisser wrote on 27 Feb 2023 07:35
Re: bug#61765: custom toolchain blog post
(name . Ludovic Courtès)(address . ludo@gnu.org)
87k003xgs0.fsf@librem.one
L
L
Ludovic Courtès wrote on 11 Mar 2023 04:11
(name . Mitchell Schmeisser)(address . mitchellschmeisser@librem.one)(address . 61765@debbugs.gnu.org)
87zg8jwkqc.fsf@gnu.org
Hi Mitchel,

(Please keep the issue Cc’d.)

Apologies for the delay!

Mitchell Schmeisser <mitchellschmeisser@librem.one> skribis:

Toggle quote (18 lines)
>> One thing that’s not clear to me: with this in place, can you do “guix
>> build --target=arm-zephyr-eabi hello”, for instance? If not, what’s
>> missing to support it?
>
> You cannot. I do not know how to describe it in a succinct way but
> suffice to say the applications need to know they are targeting Zephyr
> when they are written. The application will include a `prj.conf` which
> is analogous to a custom defconfig in the Linux kernel.
> Either in this file, or somewhere in the CMakeLists.txt a `BOARD`
> variable is set to specify a specific board definition.
> These board definitions contain information about the architecture and
> the CMake scripts themselves pick the toolchain.
>
> It's not that it's impossible to implement something like `guix build
> --target=arm-zephyr-eabi k64f-hello-world` but the k64f-hello-world
> would be written in such a way that the target is implicit in the
> package.

OK. To put it differently, a typical POSIX program won’t work on
Zephyr; programs have to target the Zephyr interfaces, right?

Toggle quote (6 lines)
> The way I envision the `--target/system` flags being used in this
> context is `guix build --target=arm-linux-gnueabihf k64f-hello-world`
> which will still produce the correct firmware but will allow the
> firmware to be staged to another machine which will be responsible for
> the final deployment over some transport.

Or rather:

guix build --target=arm-zephyr-eabi k64f-hello-world

?

Toggle quote (15 lines)
>> I wonder if it would be worth mentioning
>> <https://guix.gnu.org/manual/en/html_node/Platforms.html> too, and how
>> one would go about adding a module. WDYT?
>
> I considered trying to add Zephyr platforms but I'm not sure it's worth
> the effort.
> In addition to the patch to the website I attached another post(in org)
> which describes how I integrated this toolchain into the Guix
> infrastructure to allow defining firmware packages.
> Maybe there will be additional information in there which can help you
> understand where I'm going with all of this.
>
> There will be a part 3 (and possibly more) about how to practically use
> this stuff in a real project.

Woow. :-)

Toggle quote (7 lines)
> From 0920ec7d951354c94c3da277d58e54b587522622 Mon Sep 17 00:00:00 2001
> From: Mitchell Schmeisser <mitchellschmeisser@librem.one>
> Date: Mon, 27 Feb 2023 10:20:32 -0500
> Subject: [PATCH] website: Add toolchain blog post
>
> website/blog/custom-toolchains-with-guix.md: New file

I pushed it under the drafts directory for now, to leave others a bit
more time to comment before we publish. I followed up with a commit
editing things a bit, mostly fixing typographical issues,
spelling/capitalization, code formatting, and removing tabs.


(BTW, I made slight modifications to some of the code snippets to! One
package was using (append (list …) (package-native-inputs …)), which
really works “by chance” I guess; you should use ‘modify-inputs’
instead.)

Let me know if anything’s amiss!

Thanks,
Ludo’.
M
M
Mitchell Schmeisser wrote on 11 Mar 2023 08:50
(name . Ludovic Courtès)(address . ludo@gnu.org)(address . 61765@debbugs.gnu.org)
A10CB170-D3A4-4B79-B380-D35ECD54D5B3@librem.one
Hi Ludo,
Toggle quote (4 lines)
>
> OK. To put it differently, a typical POSIX program won’t work on
> Zephyr; programs have to target the Zephyr interfaces, right?

This is mostly correct. I think a lot of my conceptual difficulties come from the fact that west/zephyr try to tackle the issues of software deployment and dependency management at the project level. What I mean to say is that in the rest of the software world we have things like auto tools or pkg-config to help with component composition but Zephyr tries to do all composition through several thousands of lines of cmake. Every bit as complex as the kernel kconfig but even more so since modules can be scattered across the file system.
One of the selling points of zephyr is that your application is abstracted from the hardware and can run on multiple boards without modification. However to make this happen you need to do a lot of work on the application side.
Below you can see an example. You have to provide the proper kernel config for every target you want to support.

Using guix there may be a more elegant solution to these problems but maybe not.

Thank you for all your feedback,
Mitchell


Toggle quote (83 lines)
> On Mar 11, 2023, at 7:12 AM, Ludovic Courtès <ludo@gnu.org> wrote:
>
> Hi Mitchel,
>
> (Please keep the issue Cc’d.)
>
> Apologies for the delay!
>
> Mitchell Schmeisser <mitchellschmeisser@librem.one> skribis:
>
>>> One thing that’s not clear to me: with this in place, can you do “guix
>>> build --target=arm-zephyr-eabi hello”, for instance? If not, what’s
>>> missing to support it?
>>
>> You cannot. I do not know how to describe it in a succinct way but
>> suffice to say the applications need to know they are targeting Zephyr
>> when they are written. The application will include a `prj.conf` which
>> is analogous to a custom defconfig in the Linux kernel.
>> Either in this file, or somewhere in the CMakeLists.txt a `BOARD`
>> variable is set to specify a specific board definition.
>> These board definitions contain information about the architecture and
>> the CMake scripts themselves pick the toolchain.
>>
>> It's not that it's impossible to implement something like `guix build
>> --target=arm-zephyr-eabi k64f-hello-world` but the k64f-hello-world
>> would be written in such a way that the target is implicit in the
>> package.
>
> OK. To put it differently, a typical POSIX program won’t work on
> Zephyr; programs have to target the Zephyr interfaces, right?
>
>> The way I envision the `--target/system` flags being used in this
>> context is `guix build --target=arm-linux-gnueabihf k64f-hello-world`
>> which will still produce the correct firmware but will allow the
>> firmware to be staged to another machine which will be responsible for
>> the final deployment over some transport.
>
> Or rather:
>
> guix build --target=arm-zephyr-eabi k64f-hello-world
>
> ?
>
>>> I wonder if it would be worth mentioning
>>> <https://guix.gnu.org/manual/en/html_node/Platforms.html> too, and how
>>> one would go about adding a module. WDYT?
>>
>> I considered trying to add Zephyr platforms but I'm not sure it's worth
>> the effort.
>> In addition to the patch to the website I attached another post(in org)
>> which describes how I integrated this toolchain into the Guix
>> infrastructure to allow defining firmware packages.
>> Maybe there will be additional information in there which can help you
>> understand where I'm going with all of this.
>>
>> There will be a part 3 (and possibly more) about how to practically use
>> this stuff in a real project.
>
> Woow. :-)
>
>> From 0920ec7d951354c94c3da277d58e54b587522622 Mon Sep 17 00:00:00 2001
>> From: Mitchell Schmeisser <mitchellschmeisser@librem.one>
>> Date: Mon, 27 Feb 2023 10:20:32 -0500
>> Subject: [PATCH] website: Add toolchain blog post
>>
>> website/blog/custom-toolchains-with-guix.md: New file
>
> I pushed it under the drafts directory for now, to leave others a bit
> more time to comment before we publish. I followed up with a commit
> editing things a bit, mostly fixing typographical issues,
> spelling/capitalization, code formatting, and removing tabs.
>
> https://git.savannah.gnu.org/cgit/guix/guix-artwork.git/tree/website/drafts/custom-toolchains-with-guix.md
>
> (BTW, I made slight modifications to some of the code snippets to! One
> package was using (append (list …) (package-native-inputs …)), which
> really works “by chance” I guess; you should use ‘modify-inputs’
> instead.)
>
> Let me know if anything’s amiss!
>
> Thanks,
> Ludo’.
Attachment: file
M
M
Mitchell Schmeisser wrote on 13 Mar 2023 06:52
(name . Ludovic Courtès)(address . ludo@gnu.org)(address . 61765@debbugs.gnu.org)
e0a66bab-2bca-cbd2-2e3b-c893c0391a1d@librem.one
Here are two patches with minor fixes.

Sorry about the code formatting... Does anyone have any tips for getting
emacs to format scheme code in markdown?


Thanks,

Mitchell


On 3/11/23 07:11, Ludovic Courtès wrote:
Toggle quote (75 lines)
> Hi Mitchel,
>
> (Please keep the issue Cc’d.)
>
> Apologies for the delay!
>
> Mitchell Schmeisser <mitchellschmeisser@librem.one> skribis:
>
>>> One thing that’s not clear to me: with this in place, can you do “guix
>>> build --target=arm-zephyr-eabi hello”, for instance? If not, what’s
>>> missing to support it?
>> You cannot. I do not know how to describe it in a succinct way but
>> suffice to say the applications need to know they are targeting Zephyr
>> when they are written. The application will include a `prj.conf` which
>> is analogous to a custom defconfig in the Linux kernel.
>> Either in this file, or somewhere in the CMakeLists.txt a `BOARD`
>> variable is set to specify a specific board definition.
>> These board definitions contain information about the architecture and
>> the CMake scripts themselves pick the toolchain.
>>
>> It's not that it's impossible to implement something like `guix build
>> --target=arm-zephyr-eabi k64f-hello-world` but the k64f-hello-world
>> would be written in such a way that the target is implicit in the
>> package.
> OK. To put it differently, a typical POSIX program won’t work on
> Zephyr; programs have to target the Zephyr interfaces, right?
>
>> The way I envision the `--target/system` flags being used in this
>> context is `guix build --target=arm-linux-gnueabihf k64f-hello-world`
>> which will still produce the correct firmware but will allow the
>> firmware to be staged to another machine which will be responsible for
>> the final deployment over some transport.
> Or rather:
>
> guix build --target=arm-zephyr-eabi k64f-hello-world
>
> ?
>
>>> I wonder if it would be worth mentioning
>>> <https://guix.gnu.org/manual/en/html_node/Platforms.html> too, and how
>>> one would go about adding a module. WDYT?
>> I considered trying to add Zephyr platforms but I'm not sure it's worth
>> the effort.
>> In addition to the patch to the website I attached another post(in org)
>> which describes how I integrated this toolchain into the Guix
>> infrastructure to allow defining firmware packages.
>> Maybe there will be additional information in there which can help you
>> understand where I'm going with all of this.
>>
>> There will be a part 3 (and possibly more) about how to practically use
>> this stuff in a real project.
> Woow. :-)
>
>> From 0920ec7d951354c94c3da277d58e54b587522622 Mon Sep 17 00:00:00 2001
>> From: Mitchell Schmeisser <mitchellschmeisser@librem.one>
>> Date: Mon, 27 Feb 2023 10:20:32 -0500
>> Subject: [PATCH] website: Add toolchain blog post
>>
>> website/blog/custom-toolchains-with-guix.md: New file
> I pushed it under the drafts directory for now, to leave others a bit
> more time to comment before we publish. I followed up with a commit
> editing things a bit, mostly fixing typographical issues,
> spelling/capitalization, code formatting, and removing tabs.
>
> https://git.savannah.gnu.org/cgit/guix/guix-artwork.git/tree/website/drafts/custom-toolchains-with-guix.md
>
> (BTW, I made slight modifications to some of the code snippets to! One
> package was using (append (list …) (package-native-inputs …)), which
> really works “by chance” I guess; you should use ‘modify-inputs’
> instead.)
>
> Let me know if anything’s amiss!
>
> Thanks,
> Ludo’.
From 25a65c6ace2a3df43d6c3c6d6e23062a51419025 Mon Sep 17 00:00:00 2001
From: Mitchell Schmeisser <mitchellschmeisser@librem.one>
Date: Mon, 13 Mar 2023 09:39:22 -0400
Subject: [PATCH 2/2] website: custom-toolchains-with-guix: Update reference
URL

* website/drafts/custom-toolchains-with-guix.md: Changed url from
"latest" to 3.1 which was the current version at the time of writting.
---
website/drafts/custom-toolchains-with-guix.md | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)

Toggle diff (15 lines)
diff --git a/website/drafts/custom-toolchains-with-guix.md b/website/drafts/custom-toolchains-with-guix.md
index fb491c6..ffc3b1a 100644
--- a/website/drafts/custom-toolchains-with-guix.md
+++ b/website/drafts/custom-toolchains-with-guix.md
@@ -447,7 +447,7 @@ for a given board.
There are standard locations the build system will look for the SDK. We are not using any of them.
Our SDK lives in the store, immutable forever.
-According to [the Zephyr documentation](https://docs.zephyrproject.org/latest/develop/west/without-west.html), the variable `ZEPHYR_SDK_INSTALL_DIR` needs to point to our custom spot.
+According to [the Zephyr documentation](https://docs.zephyrproject.org/3.1.0/develop/west/without-west.html), the variable `ZEPHYR_SDK_INSTALL_DIR` needs to point to our custom spot.
We also need to grab the CMake files from the
[repository](https://github.com/zephyrproject-rtos/sdk-ng)
--
2.39.1
L
L
Ludovic Courtès wrote on 15 Mar 2023 07:45
(name . Mitchell Schmeisser)(address . mitchellschmeisser@librem.one)(address . 61765@debbugs.gnu.org)
87r0tq3wfc.fsf_-_@gnu.org
Hi,

Mitchell Schmeisser <mitchellschmeisser@librem.one> skribis:

Toggle quote (3 lines)
> Sorry about the code formatting... Does anyone have any tips for
> getting emacs to format scheme code in markdown?

No tips, other than making sure to untabify. :-)

Toggle quote (8 lines)
> From 0f6c28345a51e20004bc16b73bda1c0e5ccb7f4c Mon Sep 17 00:00:00 2001
> From: Mitchell Schmeisser <mitchellschmeisser@librem.one>
> Date: Mon, 13 Mar 2023 09:36:36 -0400
> Subject: [PATCH 1/2] website: custom-toolchains-with-guix: Code fix
>
> * website/drafts/custom-toolchains-with-guix.md: Removed unnecessary
> native-inputs from gcc-arm-zephyr-eabi-toolchain code block.

[...]

Toggle quote (9 lines)
> From 25a65c6ace2a3df43d6c3c6d6e23062a51419025 Mon Sep 17 00:00:00 2001
> From: Mitchell Schmeisser <mitchellschmeisser@librem.one>
> Date: Mon, 13 Mar 2023 09:39:22 -0400
> Subject: [PATCH 2/2] website: custom-toolchains-with-guix: Update reference
> URL
>
> * website/drafts/custom-toolchains-with-guix.md: Changed url from
> "latest" to 3.1 which was the current version at the time of writting.

Applied, and finally published:


Thank you!

Ludo’.
L
L
Ludovic Courtès wrote on 15 Mar 2023 07:46
control message for bug #61765
(address . control@debbugs.gnu.org)
87pm9a3wen.fsf@gnu.org
close 61765
quit
M
M
Mitchell Schmeisser wrote on 15 Mar 2023 18:55
Re: bug#61765: custom toolchain blog post
(name . Ludovic Courtès)(address . ludo@gnu.org)(address . 61765@debbugs.gnu.org)
717EED68-1B47-4BEE-A47F-762702439926@librem.one
Ah I’m so honored, thank you for the encouragement!

-Mitchell

Toggle quote (37 lines)
> On Mar 15, 2023, at 10:46 AM, Ludovic Courtès <ludo@gnu.org> wrote:
>
> Hi,
>
> Mitchell Schmeisser <mitchellschmeisser@librem.one> skribis:
>
>> Sorry about the code formatting... Does anyone have any tips for
>> getting emacs to format scheme code in markdown?
>
> No tips, other than making sure to untabify. :-)
>
>> From 0f6c28345a51e20004bc16b73bda1c0e5ccb7f4c Mon Sep 17 00:00:00 2001
>> From: Mitchell Schmeisser <mitchellschmeisser@librem.one>
>> Date: Mon, 13 Mar 2023 09:36:36 -0400
>> Subject: [PATCH 1/2] website: custom-toolchains-with-guix: Code fix
>>
>> * website/drafts/custom-toolchains-with-guix.md: Removed unnecessary
>> native-inputs from gcc-arm-zephyr-eabi-toolchain code block.
>
> [...]
>
>> From 25a65c6ace2a3df43d6c3c6d6e23062a51419025 Mon Sep 17 00:00:00 2001
>> From: Mitchell Schmeisser <mitchellschmeisser@librem.one>
>> Date: Mon, 13 Mar 2023 09:39:22 -0400
>> Subject: [PATCH 2/2] website: custom-toolchains-with-guix: Update reference
>> URL
>>
>> * website/drafts/custom-toolchains-with-guix.md: Changed url from
>> "latest" to 3.1 which was the current version at the time of writting.
>
> Applied, and finally published:
>
> https://guix.gnu.org/en/blog/2023/building-toolchains-with-guix/
>
> Thank you!
>
> Ludo’.
?
Your comment

This issue is archived.

To comment on this conversation send an email to 61765@patchwise.org

To respond to this issue using the mumi CLI, first switch to it
mumi current 61765
Then, you may apply the latest patchset in this issue (with sign off)
mumi am -- -s
Or, compose a reply to this issue
mumi compose
Or, send patches to this issue
mumi send-email *.patch