Introspecting Emacs Packages
This all started when someone on the Internet asked "How can I figure out which
functionality I'm seeing is from which package?" A seemingly simple question.
They want to know because they're trying to filter out some of the default
content provided by doom's SPC b B command, which is a switch-to-buffer kind
of thing:
Now I'm trying to clear the list of hundreds of files in switch buffer (
b B) ,which i assume is just stored in a list somewhere, but i don't know where to narrow down the search, because I don't know if this is coming from a particular package I should know about or from part of doom itself. I could easily fix it myself if I knew where to look.
My initial answer was relatively short and straightforward:
One way is to use which-key to figure out which function is being called for a keybinding, which will usually contain the package name. Another way is to turn on the profiler, do whatever thing you’re doing, and then check out the profile to see which functions have been called. Another way is to use describe-mode to see if you’re in a major mode defined by some package, so in your switch buffer example, SPC b B, then M-x describe-mode.
Once you have a function or functions to look at, the easiest thing to do is SPC h f to do describe-function, then go and view the source to see what it’s doing if needed.
However, OP wanted something a little more direct:
The profiler is useful, but it sorts the results and only displays them after it is stopped. Is there anyway I can view in real-time a buffer of the elisp code in the order that it executes?
I don't know how to get exactly what they asked for (if anyone knows, please email me), but I do think that, even without being real-time, the profiler is really useful, so I did some exploration of how to use the profiler results effectively. Realistically, I'd probably just read the code, which I go into a bit at the end.
Everything that follows is a lightly edited version of my (absurdly excessive) response.
Introspecting Profiler Results
I don't know of a way to sort the profiler in order of execution, although it
does show you what called what, which can be helpful in figuring out what the
main things running are. For the SPC b B example, if I run profiler-start,
select cpu, press SPC b B, then M-x and profiler-stop, then
`profiler-report, I get something that looks like this at the top level:
713 63% + command-execute
307 27% + timer-event-handler
102 9% + ...
3 0% + redisplay_internal (C function)
Since you're looking for what's actively done, we can ignore the
timer-event-handler section and drill into the command-execute section.
The only thing in there is funcall-interactively, which only contains a call
to consult-buffer:
713 63% - command-execute
713 63% - funcall-interactively
713 63% + consult-buffer
So, right off the bat we know that consult is the main package handling
showing the buffer list. At this point, I'd go and read up on what it is and
what it does so that I'd know whether it's the thing I care about, but we can
continue to dig deeper in the profile to figure out much the same.
(Note that you can press RET on any of these functions to go to the
definition, which should often be more than enough to tell you what the function
is doing and whether you care about it, but let's keep following the profile).
consult-buffer calls just one function, consult--multi (the -- is a
convention indicating a "private" function in a package), which calls only two
functions: consult--read and consult--multi-candidates.
713 63% - consult-buffer
713 63% - consult--multi
711 63% + consult--read
2 0% + consult--multi-candidates
Okay, so what do each of these do? If I expand down into consult--read, it's
calling all consult- functions until eventually it gets to completing-read,
which we can assume is builtin function since it doesn't have a prefix:
711 63% - consult--read
711 63% - consult--read-1
711 63% - consult--with-preview-1
711 63% - #<compiled -0xb03ad39761aa1b4>
711 63% + completing-read
The docs for that say:
Signature
(completing-read PROMPT COLLECTION &optional PREDICATE REQUIRE-MATCH INITIAL-INPUT HIST DEF INHERIT-INPUT-METHOD)
Documentation
Read a string in the minibuffer, with completion.
PROMPT is a string to prompt with; normally it ends in a colon and a space. COLLECTION can be a list of strings, an alist, an obarray or a hash table. COLLECTION can also be a function to do the completion itself. PREDICATE limits completion to a subset of COLLECTION. See try-completion, all-completions, test-completion, and completion-boundaries, for more details on completion, COLLECTION, and PREDICATE. See also Info node (elisp)Basic Completion for the details about completion, and Info node (elisp)Programmed Completion for expectations from COLLECTION when it's a function.
So, this is the thing that handles autocomplete in the minibuffer. The question
is where is it getting its candidates. The other consult function that gets
called, consult--multi-candidates, seems like a good bet. Let's see what
functions it calls:
2 0% - consult--multi-candidates
2 0% - seq-do
2 0% - mapc
2 0% - #<compiled 0x117611e5295de284>
1 0% - #<compiled 0x1da8f3f42ff4e3cf>
1 0% mapcar
1 0% - #<compiled -0x17a520717cbc2782>
1 0% - consult--project-root
1 0% - doom-project-root
1 0% - let
1 0% + projectile-project-root
seq-do (if we look at its definition) applies a function to each element of a
sequence and then returns the sequence. mapc is basically the same thing, and
looking at seq-do, it pretty much just calls mapc. Anyway, via that, we call
one compiled function, which calls two more compiled functions. We can't really
tell what the first one is doing, but the second one is calling
consult--project-root, which via doom-project-root, calls
projectile-project-root, so this is presumably getting candidates for the
current project.
Okay, so so far we have functions from consult, doom, projectile, and
emacs itself. Do we have any other packages to consider? Well, if we expand
completing-read, to see what it calls, we see a few more:
711 63% - completing-read
711 63% - completing-read-default
711 63% - apply
711 63% - vertico--advice
711 63% - #<subr completing-read-default>
699 62% + command-execute
3 0% - vertico--exhibit
2 0% - vertico--arrange-candidates
1 0% - vertico--affixate
1 0% - #<compiled 0x19402bb672750fef>
1 0% - mapcar
1 0% - #<compiled 0x34087b8b533b33e>
1 0% - marginalia--cached
1 0% marginalia-annotate-consult-multi
1 0% vertico--display-candidates
1 0% ws-butler-global-mode-check-buffers
1 0% - redisplay_internal (C function)
1 0% - eval
1 0% doom-modeline-segment--modals
So completing-read calls vertico-advice ("advice" means a function override,
so this is presumably a vertico function used to override a builtin function),
which calls some other vertico functions. Based on the names alone, it looks
like vertico handles the minibuffer layout. You can verify this by looking at
the docs for vertico.
We've also got marginalia, which if we read up on vertico we'll learn is used
to display extra information about the candidates (e.g. file size, file type,
etc.).
We've got a ws-butler-global-mode-check-buffers, which is probably irrelevant
to us, because if we look at ws-butler, it's purpose is to manage EoL and EoF
whitespace. We've also got redisplay_internal, which you'll often see in a
profile. Based on the name we might assume that it handles redrawing the frame,
but as we can see it only calls something related to the modeline, so we don't
care about it right now.
Okay, so we've got consult, which contains the main function being called,
consult-buffer. It uses at least projectile to find candidates. It calls
completing-read to handle autocomplete. vertico overrides the
completing-read functionality to provide a nice buffer, with extra info from
marginalia.
So this is all fun, but as you can see we occasionally run into a compiled function, which doesn't help us figure out what's going on.
This gets us back to the main approach I would normally use over the profiler,
which is starting from one function and then following the code. Here, we could
know either via describe-key or the profile that we start at consult-buffer,
so let's look at that and see what we can see.
Introspecting a Function
As long as we know where we start, we can usually figure out what we need. In
this case, we're starting from consult-buffer.
The main thing consult-buffer does is this:
(when-let (buffer (consult--multi consult-buffer-sources
:require-match
(confirm-nonexistent-file-or-buffer)
:prompt "Switch to: "
:history 'consult--buffer-history
:sort nil))
consult-buffer-sources, if we press K on it to get its help
(+lookup/documentation in doom, or we could just use describe-variable),
says:
consult-buffer-sources is a variable defined in consult.el.
Value (consult--source-hidden-buffer consult--source-buffer consult--source-recent-file consult--source-bookmark consult--source-project-buffer consult--source-project-recent-file +vertico--consult-org-source)
Original Value (consult--source-hidden-buffer consult--source-buffer consult--source-recent-file consult--source-bookmark consult--source-project-buffer consult--source-project-recent-file)
Documentation
Sources used by consult-buffer.
See consult--multi for a description of the source values.
Okay, so let's look at consult-multi, since it's the main player anyway. The
key bit of its docs is:
The function returns the selected candidate in the form (cons candidate source-value). The sources of the source list can either be symbols of source variables or source values. Source values must be plists with the following fields:
Required source fields:
- :category - Completion category.
- :items - List of strings to select from or function returning list of strings.
So, consult-buffer-sources is a list of sources, where each source defines
either items to choose from or a function returning items to choose from.
We can see what a source looks like if we look at one of the already defined
sources, let's say consult--source-buffer, which if we do SPC h v
(describe-variable) and look at, is:
(:name "Buffer" :narrow 98 :category buffer :face consult-buffer :history buffer-name-history :state consult--buffer-state :default t :items
#[0 "\300\301\302\303\304$\207"
[consult--buffer-query :sort visibility :as buffer-name]
5])
So we can see we've got our name and category and such, but the :items
function it calls is compiled, so it doesn't help us much. However, the variable
help tells it's it's defined in consult.el with a nice link we can click on,
so if we follow it and take a look at the definition, we get something a lot
more useful:
(defvar consult--source-buffer
`(:name "Buffer"
:narrow ?b
:category buffer
:face consult-buffer
:history buffer-name-history
:state ,#'consult--buffer-state
:default t
:items
,(lambda () (consult--buffer-query :sort 'visibility
:as #'buffer-name)))
"Buffer candidate source for `consult-buffer'.")
Okay! Much better. :items is a lambda that calls consult--buffer-query. We
can go look at that to see what it does, but essentially this is the idea. We
now what what we would want to hack on if we wanted to hack on different things.
If we want to add more items to the popup, we'd add a source to
consult-buffer-sources. If we want to filter or otherwise change something
that's already there, we'd want to advise (override) or replace one of the
existing source functions. If we want to change something about how sources are
displayed, we'd want to go digging in vertico and maybe marginalia.
Summary
Emacs is self-documenting and infinitely extensible, but it's not always easy to find what you need to know. We covered the basics of exploring the profiler output, which can help us figure out which functions are called whenever we perform some action. That in turn can guide is to the documentation and source code for those functions, which we can usually use to figure out whatever we need.