GNU bug report logs

#39258 Faster guix search using an sqlite cache

PackageSource(s)Maintainer(s)
guix-patches PTS Buildd Popcon
Full log

Message #218 received at 39258@debbugs.gnu.org (full text, mbox, reply):

Received: (at 39258) by debbugs.gnu.org; 27 Mar 2020 16:27:26 +0000
From debbugs-submit-bounces@debbugs.gnu.org Fri Mar 27 12:27:26 2020
Received: from localhost ([127.0.0.1]:33288 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.84_2)
	(envelope-from <debbugs-submit-bounces@debbugs.gnu.org>)
	id 1jHrpN-0000Vd-O3
	for submit@debbugs.gnu.org; Fri, 27 Mar 2020 12:27:26 -0400
Received: from mugam.systemreboot.net ([139.59.75.54]:36920)
 by debbugs.gnu.org with esmtp (Exim 4.84_2)
 (envelope-from <arunisaac@systemreboot.net>) id 1jHrpH-0000V4-9p
 for 39258@debbugs.gnu.org; Fri, 27 Mar 2020 12:27:20 -0400
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
 d=systemreboot.net; s=default; h=Content-Transfer-Encoding:Content-Type:
 MIME-Version:References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From:Sender
 :Reply-To:Content-ID:Content-Description:Resent-Date:Resent-From:
 Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Id:List-Help:
 List-Unsubscribe:List-Subscribe:List-Post:List-Owner:List-Archive;
 bh=+xeMBbi0lnhx5F/Pbxsj5gXUpnAYL2wwHS+Oi9X11KI=; b=Mn/IRFNQScuBTd594sqeuVyXxL
 Nd4HWtEpXJppFEHuWW6qEiVZ4ILFNmh2UB/9/jqHp4irzu2wYt1pdMEWr/ZJcBYqu+N/bimzcWang
 7ZJGrxvjZ/ExSuDiucJjkljp3ywl/GK+vBArd2+Vgp0yxrJZKzQwt05UDEj4bLrDs+qY=;
Received: from [192.168.2.1] (helo=steel.lan)
 by systemreboot.net with esmtpsa  (TLS1.3) tls TLS_AES_256_GCM_SHA384
 (Exim 4.93) (envelope-from <arunisaac@systemreboot.net>)
 id 1jHrp6-0044U6-9U; Fri, 27 Mar 2020 21:57:08 +0530
From: Arun Isaac <arunisaac@systemreboot.net>
To: 39258@debbugs.gnu.org
Subject: [PATCH v3 3/3] guix: Use package metadata cache for package search.
Date: Fri, 27 Mar 2020 21:56:54 +0530
Message-Id: <20200327162654.18785-4-arunisaac@systemreboot.net>
X-Mailer: git-send-email 2.25.1
In-Reply-To: <20200327162654.18785-1-arunisaac@systemreboot.net>
References: <20200327162654.18785-1-arunisaac@systemreboot.net>
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit
X-Spam-Score: 0.0 (/)
X-Debbugs-Envelope-To: 39258
Cc: Arun Isaac <arunisaac@systemreboot.net>, mail@ambrevar.xyz, ludo@gnu.org,
 zimon.toutoune@gmail.com
X-BeenThere: debbugs-submit@debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request@debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit@debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request@debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request@debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces@debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces@debbugs.gnu.org>
X-Spam-Score: -1.0 (-)
* guix/scripts/package.scm (process-query): Call search-packages and
display-package-search-results instead of find-packages-by-description and
display-search-results respectively.
* guix/ui.scm (package-metadata->recutils): New function.
(%package-metrics): Use package-metadata record field accessors.
(package-relevance): Rename argument package to package-metadata.
(display-package-search-results): New function.
---
 guix/scripts/package.scm |   5 +-
 guix/ui.scm              | 132 ++++++++++++++++++++++++++++++++++++---
 2 files changed, 125 insertions(+), 12 deletions(-)

diff --git a/guix/scripts/package.scm b/guix/scripts/package.scm
index 110d4f2977..c11f92f5a2 100644
--- a/guix/scripts/package.scm
+++ b/guix/scripts/package.scm
@@ -7,6 +7,7 @@
 ;;; Copyright © 2016 Benz Schenk <benz.schenk@uzh.ch>
 ;;; Copyright © 2016 Chris Marusich <cmmarusich@gmail.com>
 ;;; Copyright © 2019 Tobias Geerinckx-Rice <me@tobias.gr>
+;;; Copyright © 2020 Arun Isaac <arunisaac@systemreboot.net>
 ;;;
 ;;; This file is part of GNU Guix.
 ;;;
@@ -770,9 +771,9 @@ processed, #f otherwise."
                                       (_                   #f))
                                     opts))
               (regexps  (map (cut make-regexp* <> regexp/icase) patterns))
-              (matches  (find-packages-by-description regexps)))
+              (matches  (search-packages (current-profile) regexps)))
          (leave-on-EPIPE
-          (display-search-results matches (current-output-port)))
+          (display-package-search-results matches (current-output-port)))
          #t))
 
       (('show requested-name)
diff --git a/guix/ui.scm b/guix/ui.scm
index 1e24fe5dca..934699f065 100644
--- a/guix/ui.scm
+++ b/guix/ui.scm
@@ -14,6 +14,7 @@
 ;;; Copyright © 2019 Chris Marusich <cmmarusich@gmail.com>
 ;;; Copyright © 2019 Tobias Geerinckx-Rice <me@tobias.gr>
 ;;; Copyright © 2019 Simon Tournier <zimon.toutoune@gmail.com>
+;;; Copyright © 2020 Arun Isaac <arunisaac@systemreboot.net>
 ;;;
 ;;; This file is part of GNU Guix.
 ;;;
@@ -112,6 +113,7 @@
             package-synopsis-string
             string->recutils
             package->recutils
+            package-metadata->recutils
             package-specification->name+version+output
 
             supports-hyperlinks?
@@ -122,6 +124,7 @@
             relevance
             package-relevance
             display-search-results
+            display-package-search-results
 
             with-profile-lock
             string->generations
@@ -1484,6 +1487,75 @@ HYPERLINKS? is true, emit hyperlink escape sequences when appropriate."
             extra-fields)
   (newline port))
 
+(define* (package-metadata->recutils p port #:optional (width (%text-width))
+                                     #:key
+                                     (hyperlinks? (supports-hyperlinks? port))
+                                     (extra-fields '()))
+  "Write to PORT a `recutils' record of <package-metadata> object P, arranging
+to fit within WIDTH columns.  EXTRA-FIELDS is a list of symbol/value pairs to
+emit.  When HYPERLINKS? is true, emit hyperlink escape sequences when
+appropriate."
+  (define width*
+    ;; The available number of columns once we've taken into account space for
+    ;; the initial "+ " prefix.
+    (if (> width 2) (- width 2) width))
+
+  ;; Note: Don't i18n field names so that people can post-process it.
+  (format port "name: ~a~%" (package-metadata-name p))
+  (format port "version: ~a~%" (package-metadata-version p))
+  (format port "outputs: ~a~%" (string-join (package-metadata-outputs p)))
+  (format port "systems: ~a~%"
+          (string-join (package-metadata-supported-systems p)))
+  (format port "dependencies: ~a~%"
+          (string-join (package-metadata-dependencies p) " "))
+  (format port "location: ~a~%"
+          (or (and=> (package-metadata-location p)
+                     (if hyperlinks? location->hyperlink location->string))
+              (G_ "unknown")))
+
+  ;; Note: Starting from version 1.6 or recutils, hyphens are not allowed in
+  ;; field identifiers.
+  (format port "homepage: ~a~%" (package-metadata-home-page p))
+
+  ;; TODO: Print license
+  ;; (format port "license: ~a~%"
+  ;;         (match (package-metadata-license p)
+  ;;           (((? license? licenses) ...)
+  ;;            (string-join (map license-name licenses)
+  ;;                         ", "))
+  ;;           ((? license? license)
+  ;;            (let ((text (license-name license))
+  ;;                  (uri  (license-uri license)))
+  ;;              (if (and hyperlinks? uri (string-prefix? "http" uri))
+  ;;                  (hyperlink uri text)
+  ;;                  text)))
+  ;;           (x
+  ;;            (G_ "unknown"))))
+  (format port "synopsis: ~a~%"
+          (string-map (match-lambda
+                        (#\newline #\space)
+                        (chr       chr))
+                      (or (and=> (package-metadata-synopsis p) P_)
+                          "")))
+  (format port "~a~%"
+          (string->recutils
+           (string-trim-right
+            (parameterize ((%text-width width*))
+              (texi->plain-text
+               (string-append "description: "
+                              (or (and=> (package-metadata-description p) P_)
+                                  ""))))
+            #\newline)))
+  (for-each (match-lambda
+              ((field . value)
+               (let ((field (symbol->string field)))
+                 (format port "~a: ~a~%"
+                         field
+                         (fill-paragraph (object->string value) width*
+                                         (string-length field))))))
+            extra-fields)
+  (newline port))
+
 
 ;;;
 ;;; Searching.
@@ -1528,34 +1600,74 @@ score, the more relevant OBJ is to REGEXPS."
 (define %package-metrics
   ;; Metrics used to compute the "relevance score" of a package against a set
   ;; of regexps.
-  `((,package-name . 4)
+  `((,package-metadata-name . 4)
 
     ;; Match against uncommon outputs.
-    (,(lambda (package)
+    (,(lambda (package-metadata)
         (filter (lambda (output)
                   (not (member output
                                ;; Some common outpus shared by many packages.
                                '("out" "doc" "debug" "lib" "include" "bin"))))
-                (package-outputs package)))
+                (package-metadata-outputs package-metadata)))
      . 1)
 
     ;; Match regexps on the raw Texinfo since formatting it is quite expensive
     ;; and doesn't have much of an effect on search results.
-    (,(lambda (package)
-        (and=> (package-synopsis package) P_)) . 3)
-    (,(lambda (package)
-        (and=> (package-description package) P_)) . 2)
+    (,(lambda (package-metadata)
+        (and=> (package-metadata-synopsis package-metadata) P_)) . 3)
+    (,(lambda (package-metadata)
+        (and=> (package-metadata-description package-metadata) P_)) . 2)
 
     (,(lambda (type)
-        (match (and=> (package-location type) location-file)
+        (match (and=> (package-metadata-location type) location-file)
           ((? string? file) (basename file ".scm"))
           (#f "")))
      . 1)))
 
-(define (package-relevance package regexps)
+(define (package-relevance package-metadata regexps)
   "Return a score denoting the relevance of PACKAGE for REGEXPS.  A score of
 zero means that PACKAGE does not match any of REGEXPS."
-  (relevance package regexps %package-metrics))
+  (relevance package-metadata regexps %package-metrics))
+
+(define* (display-package-search-results matches port
+                                 #:key
+                                 (command "guix search"))
+  "Display MATCHES, a list of <package-metadata>/score pairs.  If PORT is a
+terminal, print at most a full screen of results."
+  (define first-line
+    (port-line port))
+
+  (define max-rows
+    (and first-line (isatty? port)
+         (terminal-rows port)))
+
+  (define (line-count str)
+    (string-count str #\newline))
+
+  (let loop ((matches matches))
+    (match matches
+      (((package-metadata . score) rest ...)
+       (let* ((links? (supports-hyperlinks? port))
+              (text   (call-with-output-string
+                        (lambda (port)
+                          (package-metadata->recutils package-metadata port
+                                                      #:hyperlinks? links?
+                                                      #:extra-fields
+                                                      `((relevance . ,score)))))))
+         (if (and (not (getenv "INSIDE_EMACS"))
+                  max-rows
+                  (> (port-line port) first-line) ;print at least one result
+                  (> (+ 4 (line-count text) (port-line port))
+                     max-rows))
+             (unless (null? rest)
+               (display-hint (format #f (G_ "Run @code{~a ... | less} \
+to view all the results.")
+                                     command)))
+             (begin
+               (display text port)
+               (loop rest)))))
+      (()
+       #t))))
 
 (define* (display-search-results matches port
                                  #:key
-- 
2.25.1





Send a report that this bug log contains spam.


debbugs.gnu.org maintainers <help-debbugs@gnu.org>. Last modified: Thu Jul 10 13:11:35 2025; Machine Name: wallace-server

GNU bug tracking system

Debbugs is free software and licensed under the terms of the GNU Public License version 2. The current version can be obtained from https://bugs.debian.org/debbugs-source/.

Copyright © 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson, 2005-2017 Don Armstrong, and many other contributors.