Repository files navigation

Muttlight is a MacOS application that improves search and preview for emailfiles stored inMailDir format.Specifically, it allows the associated file extensions to be specified andregistered via a graphical user interface, provides a Spotlight importer toextract meta data, integrates with Quick Look to provide previews andthumbnails, and allows files to be opened directly in Mutt.

Muttlight Launcher was created with Sveinbjörn Þórðarson'sPlatypus application. It responds to‘open’ requests on MailDir messages by running a simple (naive) shell scriptto launch anIterm2 session, or failing that, anative Terminal session, with Mutt open on the selected message.

Documentation is available elsewhere.

Pull requests are welcome.

1. Install prerequisites

brew install gettextbrew install ncurses

2. Clone (or link) the Mutt source code into asrc/mutt subdirectory

wget -qO- ftp://ftp.mutt.org/pub/mutt/mutt-1.8.3.tar.gz \  | tar xvz -C ./src && ln -s mutt-1.8.3 src/mutt

3. Build Mutt

(cd src/mutt && ./configure && make)

4. Build Muttlight

cd srcmake

Install Muttlight simply by copying the Muttlight.app bundle (directory) to$HOME/Applications.

Mutt is a text-based email client that works as wellin a terminal on MacOS as on any other system. It provides an efficient,configurable, programmable, and keyboard-only interface, seamlessintegration of powerful texts editors likeVim, and anon-vendor-specific and easy-to-synchronize storage format, namelyindividual MIME files inMailDirs.

Unfortunately, mail messages stored in MailDir format are not wellintegrated into the MacOS search (Spotlight) and preview (Quick Look)features. Mails appear in Spotlight results, since they are indexed, buttheir encoded filenames are all but unreadable and their contents are notdisplayed. Renaming these files to have the.eml extension causes them tobe properly indexed, displayed, and previewed by exploiting plugins in thenative Mail application. But, this violates the MailDir format and rendersthe files unreadable by Mutt and similar applications.

Two possible solutions suggest themselves.

1. Introduce a version of the MailDir specification that requires.emlextensions and modify Mutt and similar applications accordingly.

2. Somehow get the Mail application to also index these MailDir fileswithout renaming them.

The first solution is possible since Mutt is open-source software, but iteither complicates system configuration, since Mutt must be installed withpatches, or requires upstream acceptance of the patches. It would,furthermore, be necessary to patch mail delivery agents likefetchmail andpostfix. But, is it reasonably for a relativelyminor issue on a particular operating system to spread to such long-standingand widely-used pieces of software?

The second solution would be ideal, but unfortunately I was not able to makeit work. I tried registering file extensions in the finder, with theduti application, and using theUTTypeConformsTofeature of MacOS applications. The Mail application is closed source and soit is not possible to investigate it directly or to patch it. If you knowhow to do this or you work for Apple and can influence the application,pleasecontact me! I would be very pleased to makeMuttlight redundant by finding a better solution to the problem describedabove.

Muttlight represents a third solution: an application that registers itselfas the owner of MailDir files and leverages Mutt's source code to improvetheir integration into the MacOS search and preview features.

A filename in theMailDir format endswith a suffix of the form:2,DFPRST, where the six final letters are flagsthat are either present or absent (giving 64 different combinations). Sincefile extensions on MacOS begin with a., it is also necessary to considerthe characters to the left of the suffix. They are sometimesmbox, butmore often they are (part of) the host name on which the file was originallycreated (to ensure distinct names across multiple systems). Files in thenew subdirectory are named without a:2,DFPRST suffix but usuallyinclude a host name. Obviously, the host name extensions will vary acrossinstallations. Muttlight provides a GUI that searches the system for MailDirfiles, summarizes the extensions that are being used, and allows users toselect those which should be treated. Note that theDoveCot version of theMailDir scheme cannot be handled since it embeds file size information inthe extension.

Once the MailDir filename ‘extensions’ have been registered to the Muttlightfile type (org.tbrk.muttlight.email), the system will rely on Muttlight toextract metadata and provide preview images. Both features are provided byplugins that exploit the existing Mutt source code to parse and display mailmessages.

Metadata is extracted from the mail header and any plain text, html, orenriched parts. Attachment names are indexed, but not their contents.The various parts are decoded before being indexed. This should improve thequality of search results, particularly for emails in character sets otherthan ASCII. I do not know whether the Mail application indexes mailattachments but this seems difficult to achieve using the public Spotlightinterfaces (since it would be necessary to recursively call other importerplugins).

The Quick Look plugin exploits (...hacks around...) the Mutt pager to givepreviews that, while not as elegant as the native Mail ones, closelyresemble the text-based display in the Mutt client. This style may even bepreferable to some users. Rudimentary parsing is applied to colorize theheader fields, attachment status lines, and quoted replies.

Altogether, these features, together with the Mutt Launcher component, allowa rapid configuration and natural integration of MailDir contents into theMacOS user interface.

There are, of course, other ways to index and search mail messages inMailDir format. For instance, I already usemairix on both MacOS and Linux,and it works well once you get the hang of it. It has the advantage ofworking specifically with mail files and of showing the results directlywithin Mutt.

The advantage of Spotlight is that the indexes are kept up-to-date moredynamically and they span all files and file types on a system. Muttlightallows more convenient browsing of mail messages when they come up amongstother results (pdfs, text files, source code, etcetera).

Runmdimport -L to see the list of installed Spotlight plugins.The list should include.../muttlight.app/Contents/Library/Spotlight/muttlight.mdimporter.

Runmdls on a MailDir file to check that the plugin is working correctly.ThekMDItemContentType field should beorg.tbrk.muttlight.email and theother fields (kMDItemAuthors,kMDItemDisplayName, etcetera) should bevalid. To log the importing process and to see whether the plugin is beingcalled, runmdimport -d 4 <filename>.

Runqlmanage -m | egrep --color '.*muttlight.*|$ to see the list ofinstalled Quick Look plugins (and to highlight the muttlight entry).The list should includeorg.tbrk.muttlight.email -> .../muttlight.app/Contents/Library/QuickLook/muttlight.qlgenerator.It may be necessary to reset the plugin manager by runningqlmanage -r(andqlmanage -r cache).

Open a MailDir directory in Finder to check that thumbnails are beinggenerated correctly, press⌘-Y to preview a file. The thumbnails andpreviews should show the messages more or less as they appear in mutt.

The Quick Look plugin can be debugged by runningqlmanage -d 4 -p <filename>. This will show detailed logging. There may bea delay before the preview is displayed. Add-o . to the command line todump the generated files to disk.

Runmdfind <keywords> to list indexed files that contain the givenkeywords. Entering the same keywords into the Spotlight search box(⌘-Space) should display the same list. When muttlight is functioningcorrectly, any mail files in the search results are named according to theirsubject (and not their MailDir filename) and their contents should bepreviewed as in the mutt pager. It will probably be necessary toreindex your MailDir directoriesbefore the search results are named correctly (themds andmds_storesprocesses become active during reindexing).

The following command deletes and regenerates the Spotlight index on theroot volume (regeneration may take some time).

sudo mdutil -E /

Run the following command and search formuttlight to see registered fileextensions and plugin “claims”.

/System/Library/Frameworks/CoreServices.framework/Frameworks/LaunchServices.framework/Versions/A/Support/lsregister -dump | less

Reset the entire Launch Services database:

/System/Library/Frameworks/CoreServices.framework/Frameworks/LaunchServices.framework/Support/lsregister -kill -r -domain local -domain system -domain user