GNU bug report logs

#72994 [PATCH] gnu: emacs-julia-snail: Vendor julia libraries.

PackageSource(s)Maintainer(s)
guix-patches PTS Buildd Popcon
Reply or subscribe to this bug. View this bug as an mbox, status mbox, or maintainer mbox

Report forwarded to andrew@trop.in, cox.katherine.e+guix@gmail.com, liliana.prikler@gmail.com, guix-patches@gnu.org:
bug#72994; Package guix-patches. (Tue, 03 Sep 2024 04:34:02 GMT) (full text, mbox, link).


Acknowledgement sent to Danny Milosavljevic <dannym@friendly-machines.com>:
New bug report received and forwarded. Copy sent to andrew@trop.in, cox.katherine.e+guix@gmail.com, liliana.prikler@gmail.com, guix-patches@gnu.org. (Tue, 03 Sep 2024 04:34:02 GMT) (full text, mbox, link).


Message #5 received at submit@debbugs.gnu.org (full text, mbox, reply):

From: Danny Milosavljevic <dannym@friendly-machines.com>
To: guix-patches@gnu.org
Cc: Danny Milosavljevic <dannym@friendly-machines.com>
Subject: [PATCH] gnu: emacs-julia-snail: Vendor julia libraries.
Date: Tue, 3 Sep 2024 02:02:18 +0200
* gnu/packages/patches/emacs-julia-snail-1.3.1-vendor.patch: New file.
* gnu/local.mk (dist_patch_DATA): Add it.
* gnu/packages/emacs-xyz.scm (emacs-julia-snail)[arguments]<#:phases>[vendor]:
New phase.
[inputs]: Add julia-cstparser, julia-tokenize.
[propagated-inputs]: Remove julia-cstparser, julia-tokenize.
[source]: Add patch.

Change-Id: I61c0a4493fcfd219809bcc35f8765be0928a7e81
---
 gnu/local.mk                                  |  1 +
 gnu/packages/emacs-xyz.scm                    | 25 ++++++--
 .../emacs-julia-snail-1.3.1-vendor.patch      | 57 +++++++++++++++++++
 3 files changed, 78 insertions(+), 5 deletions(-)
 create mode 100644 gnu/packages/patches/emacs-julia-snail-1.3.1-vendor.patch

diff --git a/gnu/local.mk b/gnu/local.mk
index 0c4ab96bf3..8647956f2f 100644
--- a/gnu/local.mk
+++ b/gnu/local.mk
@@ -1179,6 +1179,7 @@ dist_patch_DATA =						\
   %D%/packages/patches/emacs-git-email-missing-parens.patch	\
   %D%/packages/patches/emacs-helpful-fix-tests.patch	\
   %D%/packages/patches/emacs-highlight-stages-add-gexp.patch	\
+  %D%/packages/patches/emacs-julia-snail-1.3.1-vendor.patch	\
   %D%/packages/patches/emacs-json-reformat-fix-tests.patch	\
   %D%/packages/patches/emacs-kv-fix-tests.patch	\
   %D%/packages/patches/emacs-lispy-fix-thread-last-test.patch   \
diff --git a/gnu/packages/emacs-xyz.scm b/gnu/packages/emacs-xyz.scm
index 370daaf50e..1837c759d2 100644
--- a/gnu/packages/emacs-xyz.scm
+++ b/gnu/packages/emacs-xyz.scm
@@ -14656,22 +14656,37 @@ (define-public emacs-julia-snail
               (file-name (git-file-name name version))
               (sha256
                (base32
-                "0h5lwc2hsk4rc82idhf9qj9553v7x76wmy9x2z1h60pdd93ilcyr"))))
+                "0h5lwc2hsk4rc82idhf9qj9553v7x76wmy9x2z1h60pdd93ilcyr"))
+              (patches
+               (search-patches "emacs-julia-snail-1.3.1-vendor.patch"))))
     (build-system emacs-build-system)
     (arguments
      (list
+      #:phases
+      #~(modify-phases %standard-phases
+          (add-after 'validate-compiled-autoloads 'vendor
+            (lambda* (#:key inputs outputs #:allow-other-keys)
+              (let* ((out (assoc-ref outputs "out"))
+                     (base (dirname (car (find-files out "JuliaSnail.jl"))))
+                     (vendor (string-append base "/vendor")))
+                (mkdir-p vendor)
+                (copy-recursively (string-append (assoc-ref inputs "julia-cstparser")
+                                                 "/share/julia/loadpath/CSTParser")
+                                  (string-append vendor "/CSTParser"))
+                (copy-recursively (string-append (assoc-ref inputs "julia-tokenize")
+                                                 "/share/julia/loadpath/Tokenize")
+                                  (string-append vendor "/Tokenize"))))))
       #:include
       #~(cons* "^JuliaSnail\\.jl" "extensions" %default-include)))
     (inputs
-     (list emacs-dash emacs-s emacs-spinner emacs-xref))
+     (list emacs-dash emacs-s emacs-spinner emacs-xref julia-cstparser
+           julia-tokenize))
     (propagated-inputs
      (list libvterm
            emacs-julia-mode             ;required by parser
            emacs-parsec                 ;required by parser
            emacs-popup
-           emacs-vterm
-           julia-tokenize
-           julia-cstparser))
+           emacs-vterm))
     (home-page "https://github.com/gcv/julia-snail")
     (synopsis "Development environment and REPL interaction package for Julia")
     (description "This package provides a development environment and REPL
diff --git a/gnu/packages/patches/emacs-julia-snail-1.3.1-vendor.patch b/gnu/packages/patches/emacs-julia-snail-1.3.1-vendor.patch
new file mode 100644
index 0000000000..eaa38fb868
--- /dev/null
+++ b/gnu/packages/patches/emacs-julia-snail-1.3.1-vendor.patch
@@ -0,0 +1,57 @@
+Author: Danny Milosavljevic <dannym@friendly-machines.com>
+Date: 2024-09-03
+License: GPL3+
+
+Previously, we propagated julia-cstparser and julia-tokenize. This would only
+work if the user had installed julia in their profile (because only then
+JULIA_LOAD_PATH of julia-cstparser would be propagated).
+That doesn't seem likely or desireable.
+
+It turns out that it's not difficult to make julia read vendored packages.
+Let's do that (since there's a comment inside JuliaSnail.jl that warns about
+CSTParser APIs being easily incompatible, vendoring is better anyway).
+
+diff -ru orig/4l8xr3ldffb30a44x157lj3asaj3ykls-emacs-julia-snail-1.3.1-checkout/JuliaSnail.jl 4l8xr3ldffb30a44x157lj3asaj3ykls-emacs-julia-snail-1.3.1-checkout/JuliaSnail.jl
+--- orig/4l8xr3ldffb30a44x157lj3asaj3ykls-emacs-julia-snail-1.3.1-checkout/JuliaSnail.jl	2024-09-03 00:02:03.798373662 +0200
++++ 4l8xr3ldffb30a44x157lj3asaj3ykls-emacs-julia-snail-1.3.1-checkout/JuliaSnail.jl	2024-09-03 01:05:26.310766009 +0200
+@@ -30,12 +30,12 @@
+    catch err
+       if isa(err, ArgumentError)
+          if isfile(joinpath($dir, "Project.toml"))
+-            # force dependency installation
+-            Main.Pkg.activate($dir)
+-            Main.Pkg.instantiate()
+-            Main.Pkg.precompile()
+-            # activate what was the first entry before Snail was pushed to the head of LOAD_PATH
+-            Main.Pkg.activate(LOAD_PATH[2])
++             # force dependency installation
++             if isa(err, ArgumentError)
++                 error("Vendored dependencies not found. Please make sure the vendored packages are located in the 'vendor' directory.")
++             else
++                 rethrow(err)
++             end
+          end
+       end
+    finally
+@@ -51,11 +51,16 @@
+ end
+ 
+ @with_pkg_env (@__DIR__) begin
+-   # list all external dependency imports here (from the appropriate Project.toml, either Snail's or an extension's):
+-   import CSTParser
+-   # check for dependency API compatibility
+-   !isdefined(CSTParser, :iscall) &&
+-     throw(ArgumentError("CSTParser API not compatible, must install Snail-specific version"))
++    @with_pkg_env (joinpath(@__DIR__, "vendor", "CSTParser")) begin
++        @with_pkg_env (joinpath(@__DIR__, "vendor", "Tokenize")) begin
++            println(LOAD_PATH)
++            # list all external dependency imports here (from the appropriate Project.toml, either Snail's or an extension's):
++            import CSTParser
++            # check for dependency API compatibility
++            !isdefined(CSTParser, :iscall) &&
++                throw(ArgumentError("CSTParser API not compatible, must install Snail-specific version"))
++        end
++    end
+ end
+ 
+ 

base-commit: b833aaaee7c95ec0339428a6b602f26831494798
-- 
2.45.2





Information forwarded to guix-patches@gnu.org:
bug#72994; Package guix-patches. (Mon, 16 Sep 2024 08:25:01 GMT) (full text, mbox, link).


Message #8 received at 72994@debbugs.gnu.org (full text, mbox, reply):

From: Ludovic Courtès <ludo@gnu.org>
To: Danny Milosavljevic <dannym@friendly-machines.com>
Cc: Katherine Cox-Buday <cox.katherine.e+guix@gmail.com>, 72994@debbugs.gnu.org, Liliana Marie Prikler <liliana.prikler@gmail.com>, Andrew Tropin <andrew@trop.in>
Subject: Re: [bug#72994] [PATCH] gnu: emacs-julia-snail: Vendor julia libraries.
Date: Mon, 16 Sep 2024 10:24:04 +0200
Hey Danny,

Danny Milosavljevic <dannym@friendly-machines.com> skribis:

> * gnu/packages/patches/emacs-julia-snail-1.3.1-vendor.patch: New file.
> * gnu/local.mk (dist_patch_DATA): Add it.
> * gnu/packages/emacs-xyz.scm (emacs-julia-snail)[arguments]<#:phases>[vendor]:
> New phase.
> [inputs]: Add julia-cstparser, julia-tokenize.
> [propagated-inputs]: Remove julia-cstparser, julia-tokenize.
> [source]: Add patch.
>
> Change-Id: I61c0a4493fcfd219809bcc35f8765be0928a7e81

[...]

> +          (add-after 'validate-compiled-autoloads 'vendor
> +            (lambda* (#:key inputs outputs #:allow-other-keys)
> +              (let* ((out (assoc-ref outputs "out"))
> +                     (base (dirname (car (find-files out "JuliaSnail.jl"))))
> +                     (vendor (string-append base "/vendor")))
> +                (mkdir-p vendor)
> +                (copy-recursively (string-append (assoc-ref inputs "julia-cstparser")
> +                                                 "/share/julia/loadpath/CSTParser")
> +                                  (string-append vendor "/CSTParser"))
> +                (copy-recursively (string-append (assoc-ref inputs "julia-tokenize")
> +                                                 "/share/julia/loadpath/Tokenize")

Use ‘search-input-directory’ rather than ‘assoc-ref’.

> +++ b/gnu/packages/patches/emacs-julia-snail-1.3.1-vendor.patch
> @@ -0,0 +1,57 @@
> +Author: Danny Milosavljevic <dannym@friendly-machines.com>
> +Date: 2024-09-03
> +License: GPL3+
> +
> +Previously, we propagated julia-cstparser and julia-tokenize. This would only
> +work if the user had installed julia in their profile (because only then
> +JULIA_LOAD_PATH of julia-cstparser would be propagated).
> +That doesn't seem likely or desireable.
> +
> +It turns out that it's not difficult to make julia read vendored packages.
> +Let's do that (since there's a comment inside JuliaSnail.jl that warns about
> +CSTParser APIs being easily incompatible, vendoring is better anyway).

I’m not sure I understand.  For most programming languages (Python,
etc.), we use propagated inputs like this.  How does that differ?

Normally we unbundle (“unvendor”?) libraries and this seems to go in the
opposite direction, right?

Thanks,
Ludo’.




Information forwarded to guix-patches@gnu.org:
bug#72994; Package guix-patches. (Mon, 16 Sep 2024 18:02:02 GMT) (full text, mbox, link).


Message #11 received at 72994@debbugs.gnu.org (full text, mbox, reply):

From: dannym@friendly-machines.com
To: Ludovic Courtès <ludo@gnu.org>
Cc: Katherine Cox-Buday <cox.katherine.e+guix@gmail.com>, 72994@debbugs.gnu.org, Liliana Marie Prikler <liliana.prikler@gmail.com>, Andrew Tropin <andrew@trop.in>
Subject: Re: [bug#72994] [PATCH] gnu: emacs-julia-snail: Vendor julia libraries.
Date: Mon, 16 Sep 2024 20:01:16 +0200
Hi Ludo,

On 2024-09-16 10:24, Ludovic Courtès wrote:
>> +                (copy-recursively (string-append (assoc-ref inputs 
>> "julia-tokenize")
>> +                                                 
>> "/share/julia/loadpath/Tokenize")
> 
> Use ‘search-input-directory’ rather than ‘assoc-ref’.

Will do!

> I’m not sure I understand.  For most programming languages (Python,
> etc.), we use propagated inputs like this.  How does that differ?
> 
> Normally we unbundle (“unvendor”?) libraries and this seems to go in 
> the
> opposite direction, right?

The idea is to make Guix more user-friendly.

I touched this only after I myself got bitten by the following:

I have guix home (with only a few packages--notably no julia (!) or 
gcc-toolchain
or anything like that in it),
and I have emacs set up with emacs-buffer-env and the following 
configuration:

  (setq buffer-env-script-name '("manifest.scm" ".envrc"))

In a julia project, manifest.scm contains:

  (specifications->manifest
    (list "julia"))

The idea is that emacs-bufferenv will automatically load that manifest 
when I
open a file in that directory within emacs--and then have julia-snail 
REPL
(including sending expression on point to the REPL) work.

Since the emacs julia-snail REPL integration needs to find the bounds of 
the expression,
it has to have a parser for the Julia language.

However, what happens now is that because I don't have julia-cstparser 
and julia-tokenize
in guix home, it won't work.

But from the standpoint of the profile I shouldn't have julia-cstparser 
and julia-tokenize
in my home profile since I don't personally directly use it (it's a 
requirement of emacs-julia-snail--I didn't even know what 
julia-cstparser is a few days ago :) ).

But nevermind that.  Even if I added julia-cstparser and julia-tokenize 
to my
guix home profile, it would NOT work.  The reason is because julia 
itself is not part of
the guix home profile, and the "julia" package would have a search-path 
specification that
would set some environment variables that julia needs to find 
julia-cstparser and
julia-tokenize.
Since the package is not there (in guix home) it won't set the 
variables.

The idea of this patch is that emacs will automatically set up whatever 
it needs to in order
to make the REPL work (including the finding of the boundaries of the 
current expression
that the emacs package needs for "send expression at point to REPL") 
without weird
dependencies bleeding explicitly into the user profile or into the local 
directory's manifest.

In my opinion, the user should not need to be concerned with what 
julia-cstparser is in this case.

Emacs should just magically load whatever it needs for the Julia REPL to 
work (whether
that's written in Julia or Python or whatever is an implementation 
detail--you COULD
have a Julia parser written in Python. Notably, most parsers for any 
language in
IntelliJ IDEA are written in Java--the Python parser, Rust parser etc).

That said, I'm totally open to other ways to do it.

The obvious alternative would be for the user to put julia-cstparser and 
julia-tokenize
into every Julia project's manifest.scm . But the error message is 
inscrutable if you
don't have those in there[1]--and I have no idea how any normal person 
would figure out
that julia-cstparser is needed in the first place.

[1] julia-snail--send-to-server: No Julia REPL buffer *julia* found; run 
julia-snail
Starting Julia process and loading Snail...
julia-snail--repl-enable: The REPL terminal buffer is inactive; 
double-check julia-snail-executable path

(In buffer "*scratch*" : )
julia-snail-executable
"julia"

(note: that "julia" excutable IS available in the manifest of the buffer 
env--and launching "julia" does work when launched manually within 
emacs)




Information forwarded to guix-patches@gnu.org:
bug#72994; Package guix-patches. (Tue, 17 Sep 2024 17:21:02 GMT) (full text, mbox, link).


Message #14 received at 72994@debbugs.gnu.org (full text, mbox, reply):

From: Liliana Marie Prikler <liliana.prikler@gmail.com>
To: dannym@friendly-machines.com, Ludovic Courtès <ludo@gnu.org>
Cc: Katherine Cox-Buday <cox.katherine.e+guix@gmail.com>, 72994@debbugs.gnu.org, Andrew Tropin <andrew@trop.in>
Subject: Re: [bug#72994] [PATCH] gnu: emacs-julia-snail: Vendor julia libraries.
Date: Tue, 17 Sep 2024 19:19:04 +0200
Hi Danny,

Am Montag, dem 16.09.2024 um 20:01 +0200 schrieb
dannym@friendly-machines.com:
> The idea of this patch is that emacs will automatically set up
> whatever it needs to in order to make the REPL work (including the
> finding of the boundaries of the current expression that the emacs
> package needs for "send expression at point to REPL")  without weird
> dependencies bleeding explicitly into the user profile or into the
> local directory's manifest.
You could write a meta package à la "julia stuff for emacs-julia-snail"
and mention that in the description, no?  I think bundling (or
propagating) these dependencies from the package is likely not going to
do us favours in weird edge cases where you actually want more julia
packages, e.g. in the julia package that you may or may not be hacking
on, which has other dependencies :)

Cheers




Information forwarded to guix-patches@gnu.org:
bug#72994; Package guix-patches. (Thu, 17 Oct 2024 07:44:02 GMT) (full text, mbox, link).


Message #17 received at 72994@debbugs.gnu.org (full text, mbox, reply):

From: Ludovic Courtès <ludo@gnu.org>
To: Liliana Marie Prikler <liliana.prikler@gmail.com>
Cc: Katherine Cox-Buday <cox.katherine.e+guix@gmail.com>, 72994@debbugs.gnu.org, dannym@friendly-machines.com, Andrew Tropin <andrew@trop.in>
Subject: Re: [bug#72994] [PATCH] gnu: emacs-julia-snail: Vendor julia libraries.
Date: Thu, 17 Oct 2024 09:42:45 +0200
Hi Danny & Liliana,

Liliana Marie Prikler <liliana.prikler@gmail.com> skribis:

> Am Montag, dem 16.09.2024 um 20:01 +0200 schrieb
> dannym@friendly-machines.com:
>> The idea of this patch is that emacs will automatically set up
>> whatever it needs to in order to make the REPL work (including the
>> finding of the boundaries of the current expression that the emacs
>> package needs for "send expression at point to REPL")  without weird
>> dependencies bleeding explicitly into the user profile or into the
>> local directory's manifest.

Thanks for explaining!

> You could write a meta package à la "julia stuff for emacs-julia-snail"
> and mention that in the description, no?

Yes, sounds like a good idea to me.  WDYT, Danny?

Ludo’.




Send a report that this bug log contains spam.


debbugs.gnu.org maintainers <help-debbugs@gnu.org>. Last modified: Mon Nov 4 22:17:16 2024; Machine Name: wallace-server

GNU bug tracking system

Debbugs is free software and licensed under the terms of the GNU Public License version 2. The current version can be obtained from https://bugs.debian.org/debbugs-source/.

Copyright © 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson, 2005-2017 Don Armstrong, and many other contributors.