Managing a bibliography of BiBTeX entries with Denote
Introduction
When the Denote package first came on the scene, I thought to myself “Ah! This would be a perfect way to manage my bibliography!”. Some background information, I am a PhD student, and I figured that its file naming scheme would be well suited for saving the pdfs of the different papers I read. What with its tagging system, I could easily find any paper’s pdf provided I gave it sufficiently informative tags.
Which I did. My denote-directory
was soon full of pdfs with
informative tags and titles to make it easy to search for them
again. The only problem was that I never did. And when I did, I
found myself wanting the paper’s BiBTeX entry (for when it finally
came to cite them in my own work) instead of the actual pdf
itself. The latter could be easily gotten online, putting my habit of
hoarding pdfs into question. Additionally, when I went to write some
notes about a given paper (using Denote, of course), I would make a
new note that would link to the pdf file. A link that was almost never
followed.
As a result, I decided to rethink how I manage my bibliography, and that is what I try to describe below.
Given their importance to my workflow, my bibliography management system now centres around BiBTeX entries, and not the pdfs of the papers they refer to. Here is a typical workflow of adding a publication to my bibliography:
- I come across an interesting paper online that I’d like to save to my bibliography.
- After copying its BiBTeX entry from the webpage to the clipboard, I switch to emacs.
- I then open up an org-capture template that asks for the BibTeX (which I paste in), a title (defaults to the title of the entry), and some tags.
- A capture window opens with a Denote note that has the inserted BiBTeX enclosed in an org source block.
- I then write any notes I may have (if any) and close the capture
window with
C-c C-c
.
Implementation
To implement this workflow, I first define the org capture template
function, which dynamically creates the Denote note template from the
given BiBTeX. The word ’template’ reminds me of the denote-templates
user option, but the templates we’re creating are dynamic, so we need
something different. Come in then org capture template functions to
save us.
Here is its definition:
(defun namilus-denote-org-capture-biblio () "Ask the user for a bibtex entry, title, and keywords, and creates a denote note template with: 1. The bibtex included inside an org bibtex source block. 2. The keyword \"biblio\" and the bibtex entry's sanitised key as part of the denote file's tags. If the bibtex entry entered by the user is empty or doesn't match the regexp, only the \"biblio\" keyword is added, along with whatever other keywords entered by the user." (let* ((bibtex (namilus-denote-bibtex-prompt)) (title (denote-title-prompt (namilus-denote-bibtex-title bibtex))) (keywords (append (denote-keywords-prompt) (namilus-denote-biblio-keywords bibtex))) (front-matter (denote--format-front-matter title (denote--date nil 'org) keywords (format-time-string denote-id-format nil) 'org))) (setq denote-last-path (denote--path title keywords (file-name-as-directory (denote-directory)) (format-time-string denote-id-format) 'org)) (denote--keywords-add-to-history keywords) (concat front-matter (namilus-denote-bibtex-org-block bibtex))))
The function namilus-denote-org-capture-biblio
prompts for the
BiBTeX entry, a title (which is auto-completed from the inputted
BiBTex), and some keywords. In addition to the keywords supplied by
the user, this template function also adds two tags:
- The ’biblio’ tag, in order to distinguish bibliography entries from other notes in my zettelkasten.
- The ’sanitised’ BiBTeX key. Sanitisation involves the removal of non alphanumeric characters from the key so it can be used as a Denote file tag.
The motivation behind adding a BiBTeX key tag is so that given that entry’s key, I can easily find its corresponding note.
The functions that read input from the user, sanitise the BiBTeX,
extract the key, generate the file tags, and create the org source
block are given below. A lot of this code is taken straight from
denote.el
and refashioned for my purposes. Ah, the principles of
software freedom in action!
(defun namilus-denote-bibtex-prompt (&optional default-bibtex) "Ask the user for a bibtex entry. Returns the sanitised version. See `namilus-denote-sanitise-bibtex' for details." (let* ((def default-bibtex) (format (if (and def (not (string-empty-p def))) (format "Bibtex [%s]: " def) "Bibtex: ")) (sanitised-bibtex (namilus-denote-sanitise-bibtex (read-string format nil nil def)))) (if sanitised-bibtex sanitised-bibtex (error "Invalid BiBTeX")))) (defun namilus-denote-sanitise-bibtex (bibtex) "Returns a santised version of BIBTEX. Sanitisation entails remove all non alpha-numeric characters from the bibtex key, and returning this updated bibtex entry. If BIBTEX is not a valid bibtex entry, returns nil." (when (string-match "@.*{\\(.*\\)," bibtex) (let* ((key (match-string-no-properties 1 bibtex)) (sanitised-key (replace-regexp-in-string "[^A-Za-z0-9]" "" key))) (replace-regexp-in-string key sanitised-key bibtex)))) (defun namilus-denote-bibtex-key (bibtex) "Returns the bibtex key from BIBTEX." (when (string-match "@.*{\\(.*\\)," bibtex) (match-string-no-properties 1 bibtex))) (defun namilus-denote-bibtex-title (bibtex) "Returns the bibtex title from BIBTEX." (when (string-match "\\s *title\\s *=\\s *{\\(.*\\)}," bibtex) (match-string-no-properties 1 bibtex))) (defun namilus-denote-biblio-keywords (bibtex) "Returns a list of strings \"biblio\" and the key from the BIBTEX entry, otherwise, just returns a list consisting of the string \"biblio\"." (let ((bibtex-key (namilus-denote-bibtex-key bibtex))) (if bibtex-key `("biblio" ,bibtex-key) '("biblio")))) (defun namilus-denote-bibtex-org-block (bibtex) "Returns a string representing an org `bibtex' source block encompassing BIBTEX, a string of a bibtex entry." (concat "#+begin_src bibtex\n" bibtex "\n#+end_src"))
Next comes adding the org capture template, which is relatively straightforward:
(add-to-list 'org-capture-templates '("B" "Bibliography (with Denote) BibTeX" plain (file denote-last-path) #'namilus-denote-org-capture-biblio :no-save t :immediate-finish nil :kill-buffer t :jump-to-captured nil))
Extracting BiBTeX entries to make a .bib
file
You might now ask “This is all well and good but what do you do when
it comes to write and actually cite these notes? You can’t cite org
files in LaTeX as far as I know…”. This is very true. It would be
nice to say, mark all ’biblio’ tagged entries in my denote-directory
in Dired and extract out their BiBTeX entries into a single .bib
file. This niceity we can make a reality with emacs. Here is the
emacs-lisp to do so:
(defun namilus-denote-biblio-read-bibtex (file) "Reads the bibtex entry from a given Denote FILE. Does so by searching for a org bibtex source block and returns the contents therein." (with-temp-buffer (insert-file-contents file) (let ((contents (buffer-string))) (when (string-match "#\\+begin_src.*bibtex\\(\\(.*\n\\)*\\)#\\+end_src" contents) (match-string-no-properties 1 contents))))) (defun namilus-denote-generate-bibliography (denote-biblio-files bibliography-file) "Writes the org bibtex source blocks located in each of DENOTE-BIBLIO-FILES to BIBLIOGRAPHY-FILE." (with-temp-file bibliography-file (dolist (file denote-biblio-files) (let ((bibtex (namilus-denote-biblio-read-bibtex file))) (if bibtex (insert bibtex))))))
With namilus-denote-generate-bibliography
defined, we can pass it a
list of denote files and a target bibliography file and it will go
through each file and add the bibtex entry to the target
bibliography. Something akin to a multi-file org tangle.
Next I define the interactive function that grabs the marked Dired
files, and asks the user for a .bib
file to save the BiBTeX entries
in. With this method, we can also generate .bib
files for specific
bibliography items e.g those tagged with a specific keyword.
(defun namilus-denote-bibliography-file-prompt (&optional default-bibliography-file) "Ask the user for a bibliography file." (let* ((def default-bibliography-file) (format (if (and def (not (string-empty-p def))) (format "Bibliography file [%s]: " def) "Bibliography file: "))) (expand-file-name (read-file-name format nil def)))) (defun namilus-denote-dired-generate-bibliography-from-marked () (interactive) (namilus-denote-generate-bibliography (dired-get-marked-files) (namilus-denote-bibliography-file-prompt)))
Conclusion
This method of bibliography management I think is better than keeping
just a plain .bib
file, but is similar to having one large org file
with each reference being a top level heading i.e the guiding
principle behind Nicolas Rougier’s org-bib-mode
package. Here the
approach is “one-file-per-entry” instead of
“one-heading-per-reference”, with all headings in the same file.
Peter Prevos’ package denote-citar also allows for taking notes about
a particular BiBTeX entry, but assumes you already have a
citar-bibliography
full of BiBTeX entries, and requires the use of
the citar package as well. Prevos’ package was certainly an
inspiration for this post, with the focus of the workflow I’ve
described being to quickly save a BiBTeX entry and its corresponding
notes, as well as create .bib
files on-the-fly from them.
By using Denote to create these note files, we can also benefit from the features that Denote has to offer like filtering by title and keywords, as well as linking between related entries. This is not to say that you can’t do those things with a single org file i.e by using regular org heading tags and heading links, but I chose this workflow because I like having the standalone note files. You might then say “exporting a subtree would also result in a single file for a given heading”, but hey, that’s the beauty of emacs - the choice is up to you…