GNU bug report logs

#25327 cargo build-system should be able to filter out target.cfg(windows) dependencies

PackageSource(s)Maintainer(s)
guix PTS Buildd Popcon
Reply or subscribe to this bug. View this bug as an mbox, status mbox, or maintainer mbox

Report forwarded to bug-guix@gnu.org:
bug#25327; Package guix. (Mon, 02 Jan 2017 00:46:02 GMT) (full text, mbox, link).


Acknowledgement sent to ng0 <ng0@libertad.pw>:
New bug report received and forwarded. Copy sent to bug-guix@gnu.org. (Mon, 02 Jan 2017 00:46:02 GMT) (full text, mbox, link).


Message #5 received at submit@debbugs.gnu.org (full text, mbox, reply):

From: ng0 <ng0@libertad.pw>
To: bug-guix@gnu.org
Subject: cargo build-system should be able to filter out target.cfg(windows) dependencies
Date: Mon, 02 Jan 2017 00:46:09 +0000
The cargo build-system should be able to filter out Windows
dependencies.
Ignore blocks with this in Cargo.toml files:
[target.'cfg(windows)'.dependencies]

until the next [ ] starts.

I also found that most *32-sys packages (currently ALL *32-sys
I've seen) are windows specific.
As the build system is simple enough to not complain, you just
have to learn about this while doing the work.

-- 
♥Ⓐ  ng0
PGP keys and more: https://n0is.noblogs.org/ http://ng0.chaosnet.org




Information forwarded to bug-guix@gnu.org:
bug#25327; Package guix. (Fri, 18 Dec 2020 20:03:01 GMT) (full text, mbox, link).


Message #8 received at 25327@debbugs.gnu.org (full text, mbox, reply):

From: zimoun <zimon.toutoune@gmail.com>
To: ng0 <ng0@libertad.pw>, Hartmut Goebel <h.goebel@crazy-compilers.com>, John Soo <jsoo1@asu.edu>
Cc: 25327@debbugs.gnu.org
Subject: Re: bug#25327: cargo build-system should be able to filter out target.cfg(windows) dependencies
Date: Fri, 18 Dec 2020 20:56:37 +0100
Hi,

On Mon, 02 Jan 2017 at 00:46, ng0 <ng0@libertad.pw> wrote:
> The cargo build-system should be able to filter out Windows
> dependencies.
> Ignore blocks with this in Cargo.toml files:
> [target.'cfg(windows)'.dependencies]
>
> until the next [ ] starts.
>
> I also found that most *32-sys packages (currently ALL *32-sys
> I've seen) are windows specific.
> As the build system is simple enough to not complain, you just
> have to learn about this while doing the work.

This bug is really old and the story about cargo has a bit evolved.  Is
it still relevant with the recent additions?

If no, feel free to close it.  And without more comment, I will close it
after the usual 2 weeks delay.


All the best,
simon




Information forwarded to bug-guix@gnu.org:
bug#25327; Package guix. (Sat, 19 Dec 2020 13:11:01 GMT) (full text, mbox, link).


Message #11 received at 25327@debbugs.gnu.org (full text, mbox, reply):

From: Hartmut Goebel <h.goebel@crazy-compilers.com>
To: zimoun <zimon.toutoune@gmail.com>, ng0 <ng0@libertad.pw>, John Soo <jsoo1@asu.edu>
Cc: 25327@debbugs.gnu.org
Subject: Re: bug#25327: cargo build-system should be able to filter out target.cfg(windows) dependencies
Date: Sat, 19 Dec 2020 14:10:00 +0100
Am 18.12.20 um 20:56 schrieb zimoun:
> Is it still relevant with the recent additions?

I just checked this with sequoia 0.20.0: The package "winapi" is still 
downloaded and compiled - even if obviously not used sicne on Linux.

-- 
Regards
Hartmut Goebel

| Hartmut Goebel          | h.goebel@crazy-compilers.com               |
| www.crazy-compilers.com | compilers which you thought are impossible |





Information forwarded to bug-guix@gnu.org:
bug#25327; Package guix. (Mon, 11 Jan 2021 12:46:02 GMT) (full text, mbox, link).


Message #14 received at 25327@debbugs.gnu.org (full text, mbox, reply):

From: zimoun <zimon.toutoune@gmail.com>
To: Hartmut Goebel <h.goebel@crazy-compilers.com>
Cc: 25327@debbugs.gnu.org, ng0 <ng0@libertad.pw>, John Soo <jsoo1@asu.edu>
Subject: Re: bug#25327: cargo build-system should be able to filter out target.cfg(windows) dependencies
Date: Mon, 11 Jan 2021 13:41:53 +0100
Hi Hartmut,

On Sat, 19 Dec 2020 at 14:10, Hartmut Goebel <h.goebel@crazy-compilers.com> wrote:
> Am 18.12.20 um 20:56 schrieb zimoun:
>> Is it still relevant with the recent additions?
>
> I just checked this with sequoia 0.20.0: The package "winapi" is still
> downloaded and compiled - even if obviously not used sicne on Linux.

Should the remove of this “winapi” happen at the build-system level or
at the package per package level?

Other said, are the Windows dependencies normalized by Cargo?

All the best,
simon






Information forwarded to bug-guix@gnu.org:
bug#25327; Package guix. (Mon, 11 Jan 2021 13:35:01 GMT) (full text, mbox, link).


Message #17 received at 25327@debbugs.gnu.org (full text, mbox, reply):

From: John Soo <jsoo1@asu.edu>
To: zimoun <zimon.toutoune@gmail.com>
Cc: Hartmut Goebel <h.goebel@crazy-compilers.com>, ng0 <ng0@libertad.pw>, 25327 <25327@debbugs.gnu.org>
Subject: Re: bug#25327: cargo build-system should be able to filter out target.cfg(windows) dependencies
Date: Mon, 11 Jan 2021 05:33:52 -0800
[Message part 1 (text/plain, inline)]
     Hi zimoun,  

  
I think that the configuration phase will fail even if conditional compilation dependencies are missing.    It’s worth a shot though!
  

  
- John
     
[Message part 2 (text/html, inline)]

Information forwarded to bug-guix@gnu.org:
bug#25327; Package guix. (Thu, 24 Mar 2022 18:50:01 GMT) (full text, mbox, link).


Message #20 received at 25327@debbugs.gnu.org (full text, mbox, reply):

From: Nicolas Graves <ngraves@ngraves.fr>
To: 25327@debbugs.gnu.org
Cc: Maxime Devos <maximedevos@telenet.be>
Subject: windows-related dependencies in crates
Date: Thu, 24 Mar 2022 18:18:59 +0100
Hi all,

I've delved into the subject the past week and now have a clearer view
on it :

- windows (and mac) dependencies are not needed when compiling a single
  package. It is possible to bypass it without changing the
  build-system, by a few means :

           - either by using the #:cargo-build-flags argument (doesn't
             work everytime, seems to work only seldomly with
             --target unix)
             In this option, one would also need to modify Cargo.toml
             file before the package phase, since `cargo package`
             vendors all possible dependencies to ensure a package on
             all environments.
             For instance (1) :
             #:phases
             (modify-phases %standard-phases
               (add-before 'package 'unix-cargo-toml
                 (lambda _
                   (delete-file "Cargo.toml")
                   (substitute* "Cargo.toml.orig"
                     (("^mac-notification-sys.*") "")
                     (("^chrono.*") "")
                     (("^winrt-notification.*") ""))
                   (rename-file "Cargo.toml.orig" "Cargo.toml")
                   #t)))

           - either by using the environment variable
             "CARGO_BUILD_RUSTFLAGS"  during the build phase, like that
             for instance (2) :
             #:phases
             (modify-phases %standard-phases
               (add-before 'build 'unix-cargo-toml
                 (setenv
                  "CARGO_BUILD_RUSTFLAGS"
                  (string-append
                   (getenv "CARGO_BUILD_RUSTFLAGS")
                   "--cfg 'target_os=\"linux\"'"))))
             In this case, the package phase would also require the same
             change as in (1). Note that this might not be backward
             compatible further down the line (seen in the cargo
             manual).

For some reason, these two means don't seem to propagate when using such
packages as inputs.

There are also the options to use a snippet or a patch on the source
itself. The snippet on the source could look like (1), or the patch as
its effects, and we would still need to use (2). Haven't experimented
much with that yet.

In any case, all this seems kinda bulky for an edit on an imported base,
and since we have lots of information with `cargo metadata`, I imagine
it would be possible to modify the build-system itself, so that it only
considers `target_os = "linux"` or possibly redox / bsd / dragonfly if
guix considers porting to these platforms.

I have 2 possible ideas :

1) Maybe something like adding a phase after unpack, which would :

- parse `cargo metadata` to isolate only needed dependencies (by maybe
generating and applying a patch ? or maybe there's a cargo command for
generating a Cargo.toml based on modified metadata ?). In any case,
modifying the Cargo.toml to only keep relevant information.

2) Maybe rethinking the package phase entirely instead of relying on
`cargo package --no-verify --no-metadata` which seems to not allow
options without interferring with the Cargo.toml file.

In both cases, we would then need to :
- modify the phase were environment variables are defined to add (2).
- modify the crate import script accordingly (not trivial).

As for the second hypothesis, according to the cargo manual, what `cargo
package` does is the following :

1. Load and check the current workspace, performing some basic checks.
Path dependencies are not allowed unless they have a version key. Cargo
will ignore the path key for dependencies in published
packages. dev-dependencies do not have this restriction.

2. Create the compressed .crate file.
a. The original Cargo.toml file is rewritten and normalized.
b. [patch], [replace], and [workspace] sections are removed from the manifest.
c. Cargo.lock is automatically included if the package contains an
  executable binary or example target. cargo-install(1) will use the
  packaged lock file if the --locked flag is used.
d. A .cargo_vcs_info.json file is included that contains information about
  the current VCS checkout hash if available (not included with
  --allow-dirty).

3. Extract the .crate file and build it to verify it can build.
- This will rebuild your package from scratch to ensure that it can be
  built from a pristine state. The --no-verify flag can be used to skip
  this step.

4. Check that build scripts did not modify any source files.

Step 3 and 4 are actually not performed, since the build-system calls
cargo package with the --no-verify flag. The checks in step 1 doesn't
seem that essential on guix, but I imagine we could replicate
them. Clearly the essential step here is 2, but since we get a crate,
steps a. and b. should actually already be done at the source level, and
step d. is not essential because of origin's field version. Step 2c is
not essential either since Cargo.lock file is actually deleted during
the build system.

I might try to implement these changes, although I do not have enough
time and experience to do that quickly, neither enough horsepower to try
to rebuild everything after that. I included Maxime Devos in the
conversation, would it be possible to fork guix to develop this and try
to rebuild every rust package after that on a guix build farm ? Sorry,
haven't got in touch with guix developers yet, still imposter's
syndrome effect, but implementing this doesn't seem too hard. What would
I need to do to get going on that ?

Cheers,

Nicolas Graves




Information forwarded to bug-guix@gnu.org:
bug#25327; Package guix. (Thu, 24 Mar 2022 19:11:01 GMT) (full text, mbox, link).


Message #23 received at 25327@debbugs.gnu.org (full text, mbox, reply):

From: Maxime Devos <maximedevos@telenet.be>
To: Nicolas Graves <ngraves@ngraves.fr>, 25327@debbugs.gnu.org
Subject: Re: windows-related dependencies in crates
Date: Thu, 24 Mar 2022 20:10:43 +0100
[Message part 1 (text/plain, inline)]
Nicolas Graves schreef op do 24-03-2022 om 18:18 [+0100]:
> I included Maxime Devos in the
> conversation, would it be possible to fork guix to develop this and try
> to rebuild every rust package after that on a guix build farm ?

In the past, for some build system changes, separate branches have been
created that are automatically built by ci.guix.gnu.org.  A non-
committer and a committer could then collaborate together (on
guix-devel@gnu.org / guix-patches@gnu.org), though only the committer
can
actually push the changes.  If, at some point, the changes appear to be
‘ready’, the build on ci.guix.gnu.org can be started to build the
substitutes.  If there are no regressions and the builds completed, the
branch can be merged.

TBC, I'm not a committer, but I'd be willing to review FWIW (I don't
understand cargo).

Greetings,
Maxime.
[signature.asc (application/pgp-signature, inline)]

Information forwarded to bug-guix@gnu.org:
bug#25327; Package guix. (Sat, 08 Oct 2022 15:17:02 GMT) (full text, mbox, link).


Message #26 received at 25327@debbugs.gnu.org (full text, mbox, reply):

From: zimoun <zimon.toutoune@gmail.com>
To: Nicolas Graves <ngraves@ngraves.fr>
Cc: Maxime Devos <maximedevos@telenet.be>, 25327@debbugs.gnu.org
Subject: Re: bug#25327: cargo build-system should be able to filter out target.cfg(windows) dependencies
Date: Sat, 08 Oct 2022 16:15:50 +0200
Hi,

On Thu, 24 Mar 2022 at 18:18, Nicolas Graves <ngraves@ngraves.fr> wrote:

> I might try to implement these changes, although I do not have enough
> time and experience to do that quickly, neither enough horsepower to try
> to rebuild everything after that. I included Maxime Devos in the
> conversation, would it be possible to fork guix to develop this and try
> to rebuild every rust package after that on a guix build farm ? Sorry,
> haven't got in touch with guix developers yet, still imposter's
> syndrome effect, but implementing this doesn't seem too hard. What would
> I need to do to get going on that ?

Just to know what is the status of this bug#25327 [1]?  Well, I have not
followed closely some recent changes about Rust and Guix.  Maybe, some
patches are already around?

Cheers,
simon

1: <http://issues.guix.gnu.org/issue/25327>




Information forwarded to bug-guix@gnu.org:
bug#25327; Package guix. (Sat, 08 Oct 2022 16:04:01 GMT) (full text, mbox, link).


Message #29 received at 25327@debbugs.gnu.org (full text, mbox, reply):

From: Maxime Devos <maximedevos@telenet.be>
To: zimoun <zimon.toutoune@gmail.com>, Nicolas Graves <ngraves@ngraves.fr>
Cc: 25327@debbugs.gnu.org
Subject: Re: bug#25327: cargo build-system should be able to filter out target.cfg(windows) dependencies
Date: Sat, 8 Oct 2022 18:03:20 +0200
[Message part 1 (text/plain, inline)]
On 08-10-2022 16:15, zimoun wrote:
> Hi,
> 
> On Thu, 24 Mar 2022 at 18:18, Nicolas Graves <ngraves@ngraves.fr> wrote:
> 
>> I might try to implement these changes, although I do not have enough
>> time and experience to do that quickly, neither enough horsepower to try
>> to rebuild everything after that. I included Maxime Devos in the
>> conversation, would it be possible to fork guix to develop this and try
>> to rebuild every rust package after that on a guix build farm ? Sorry,
>> haven't got in touch with guix developers yet, still imposter's
>> syndrome effect, but implementing this doesn't seem too hard. What would
>> I need to do to get going on that ?
> 
> Just to know what is the status of this bug#25327 [1]?  Well, I have not
> followed closely some recent changes about Rust and Guix.  Maybe, some
> patches are already around?

antioxidant-build-system (*) automatically ignores all dependencies it 
can't find (including Windows dependencies), and as such solves this.

(*) close to 100%, the main remaining problem is lack of support for 
workspaces; I'm currently looking for packaging a topological sorting 
algorithm but it depends on other unpackaged things etc.  Other problems 
should be rather minor.

Greetings,
Maxime.
[OpenPGP_0x49E3EE22191725EE.asc (application/pgp-keys, attachment)]
[OpenPGP_signature (application/pgp-signature, attachment)]

Send a report that this bug log contains spam.


debbugs.gnu.org maintainers <help-debbugs@gnu.org>. Last modified: Wed Apr 16 04:04:49 2025; Machine Name: wallace-server

GNU bug tracking system

Debbugs is free software and licensed under the terms of the GNU Public License version 2. The current version can be obtained from https://bugs.debian.org/debbugs-source/.

Copyright © 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson, 2005-2017 Don Armstrong, and many other contributors.