random musings from a primate

Review: An Attic Called DEVONthink

Sherlock Holmes, by J. Frank Wiles (Shadow of Fear, The Strand, 1914)“You see,” he explained, “I consider that a man’s brain originally is like a little empty attic, and you have to stock it with such furniture as you choose. A fool takes in all the lumber of every sort that he comes across, so that the knowledge which might be useful to him gets crowded out, or at best is jumbled up with a lot of other things so that he has a difficulty in laying his hands upon it. Now the skilful workman is very careful indeed as to what he takes into his brain-attic. He will have nothing but the tools which may help him in doing his work, but of these he has a large assortment, and all in the most perfect order. It is a mistake to think that that little room has elastic walls and can distend to any extent. Depend upon it there comes a time when for every addition of knowledge you forget something that you knew before. It is of the highest importance, therefore, not to have useless facts elbowing out the useful ones.”

- Sherlock Holmes to Watson during their first case, A Study in Scarlet

From hints in the Canon, I’m positive that Holmes nurtured a own home-grown content management system (CMS) of notes, newspaper clippings, pages torn from journals, snippets from medical textbooks, monographs on fingerprints and head measurements, observations on mud types and tobacco ashes, criminal trial transcriptions, and so on. Some of this was no doubt kept in his “attic” for casework, but there was much that didn’t fit (such as the fact that the Earth revolved about the sun, he claimed) and –should a pertinent nugget need to be recalled to solicit a possible solution for a case– that would require safekeeping for ready reference and analysis later.

I’ll leave the theorising as to whether Holmes would even bother with computers in this day and age to those Baker Street professionals who take great joy in debating such topics. However, I can’t help but wonder what sort of system he, as a knowledge worker, would utilise today.

I guess one might refer to me as a “knowledge worker” in the purer sense of the term (if you can indeed filter out the buzzword poisoning). Most of my time is spent instructing, consulting, preparing coursework, developing (mainly educational) multimedia, researching, writing, and coming up with solutions to uncommon problems. To do this requires a tremendous body of knowledge and information, very little of which I can actually retain in the little attic of my mind. Over the past few years, I’ve tried quite a number of ways to store and retrieve information, and they all have their pros and cons. In particular, I am very impressed with Tinderbox and the way in which it leverages agents, queries which act upon its freeform database and associated metadata to produce groups of links to items that match (and that can be further utilised in other scripts, like exporting to HTML for a website). However, I’ve never been able to find a content management system that also does an excellent job of handling non-textual media and concordance –finding items that are similar, based upon word relationships drawn out by context. That is, until now. Its name is DEVONthink.

Because of the necessity for heavy-duty content management in a recent contract, I’ve spent quite a bit of quality time recently with DEVONtechnologies’ DEVONthink. I had tried the 1.7 series a while back, and found it rather lacking for my purposes. However, the most recent versions have seen quantum leaps forward, especially with a number of features that are helped along by Apple’s new Safari/WebKit and OS X 10.4.x. While the basic DEVONthink 1.9.3 is quite suitable for most purposes (especially since it has received a number of features hitherto reserved for the behind-schedule Pro version), the professional version certainly piques my interest. I spent a week exploring the demo of the standard version, and the past few month using the DEVONthink Pro beta. While the latter is definitely not quite ready for prime time –there are quite a number of minor glitches, and some features that are not implemented or honed yet– I think I can say with a great degree of certainty that this is the type of application for which I have been searching several years.

Obviously, my needs may not be the same as yours. Here is a list of the most important criteria upon which I must determine an application’s suitability for knowledge management, at least for my purposes, and how DEVONthink (or DTPro) fills those needs.

  1. Full textual support: This goes without saying, since most of my material will be text in some way. The system should make allowances for plain text, RTF and HTML, and be able to somehow import Word .DOC and PDF files without too many extra steps. Reading this material should be easy on the eyes, with decent aliased fonts, multiple zoom levels and various ways of viewing information (for example, outline views vs 3-pane views).

    DT has no problem with the cut-and-paste, import and writing of plain text, RTF and HTML. For example, I can copy a body of text from a web page (in Safari) or a word processor, and paste it straight into an RTF page in DT. It retains most formatting, links, colours, and even graphics. The Word import uses OS X’s native text conversion abilities to translate .DOC files into RTF, and PDF files are likewise no problem. Since DT is a native Cocoa application, you have full font anti-aliasing, and can use styles for paragraph spacing and so forth. Multiple font sizes and zooms are available, and there are several views for laying out the various panes to show information in different ways (with extra views available in DTPro).

  2. Web integration: A goodly portion of my digital information comes to me via webpages, either as snippets of data that I trip across in online journals, wikis, blogs, news articles or static pages, or via email, most of which I read in a browser using Gmail. All of this needs to collected and categorised with due attribution (URL, date, author, etc.).

    There are a number of mechanisms for important web-based data into DT, depending on what you want and how you want it. For example, I can drop an URL straight from Safari into a DT group (that is, a topic folder). Clicking on that item will then call up a webpage live for browsing. I can then right-click and “Capture Page”, which saves a local copy of the page as an archive, complete with graphics. Or, I can select some text and –via an provided AppleScript– send the text/graphics to a new text or RTF entry, or even append to an existing one. I can also use scripts to send the page straight from Safari to a selected or predefined group within DT. Any type of URL import or capture will also save a copy of the exact URL as metadata, so you know exactly where it came from, and can refresh to a new copy if you wish. (Note: because Firefox is not a 100% Cocoa application, it doesn’t support a number of OS X services and abilities that Safari does. This means that I’ve actually been spending far more time in the latter lately; it hasn’t been too unpleasurable.)

    I should mention that DEVONtechnologies sells a product called DEVONagent, a web search agent which is supposed to integrate quite well with DT in ways beyond a standard browser. While I did download the demo for it, I must admit that I haven’t spent more than an hour with the application: I’m so familiar with the strengths and foibles of Google that I have a hard time wrapping my mind around why I’d user another application to search. However, given how strong the company’s other offerings are, I have no doubt that I’ll devote some time to it in the future. (The two are available as a bundle from DEVONtechnologies’ shop.)

  3. Non-textual file support: I also create or use a number of other files, including various graphic formats, PDF files, OPML, video, audio, spreadsheets, charts, presentations, and so on. These need to be stored, categorised and annotated.

    This is one of the areas where DT really shines over many of its competitors. You can drag any of these files straight into DT and –depending on your settings– either link to the originals, have them copied into the DT file storage, or insert them directly into the database itself. Scripts allow items to be added from the Finder as well. PDF files can be converted into regular text for indexing, and you can view the PDF files right inside of DT. Graphics are displayed as thumbnails, and you can view full size or zoomed versions. Any QuickTime-supported video or audio file can be viewed inside the system as well. DT seems to have support for more file types than any other application I’ve tried so far; file formats that are unknown are simply linked into DT for viewing in external programs, but you can add annotations, comments, etc., for searching, and then classify and categorise them appropriately.

  4. Writing tools This system should ideally hold my writings, as I consider them as another part of the work I must collate and reference. In peak form, I write over 20 original pages a day, and need an application that does not impede my flow, but yet has enough capability that I’m not aggravated by a lack of basics. As such, I need a simple word processor that can accept the occasional graphic or chart, and the system should certainly allow me to view multiple files at the same time (a major fault with a couple of other applications that I’ve tried). Also, I don’t want tools distracting me or getting in my way: I need to concentrate on content, not fonts or ligatures.

    DT has a good balance of writing tools, in my opinion. If I’m writing in RTF, I can choose to show the “Ruler”, which has controls for styles, spacing, lists and justification. I can turn on the “spell-check as I write”, do a find/replace, insert graphics easily, choose fonts, do highlighting and so forth. However, there aren’t a half-dozen panels splattered all over my screen to distract me from the task at hand: writing. DTPro also has a full-screen mode, all the rage amongst writers nowdays seeking to avoid distraction.

    One handy writing feature I have to mention is DT’s wiki abilities. You can create wiki links to items within the database quite easily, and even give a document multiple aliases that allow you to reference it under different names (which Pascal Vernier notes is handy for uses like bibliographies).

  5. Rapid data entry and management: All the above needs to be stored, categorised and catalogued extremely quickly, within seconds. I cannot spend all my time cutting and pasting, adding keywords, creating meta-tags, and massaging information to fit.

    In addition to the cut-and-paste and import tools mentioned above, DT has many ways to create items, as well as a fairly easy classification system to push items into the right topic. (It has an auto-classify feature that I don’t dare turn on yet, but later I’ll get up my courage.) Once an item is imported or entered, you can Cmd-Shift-I to see its info box, and add comments, aliases, an URL and so forth. This is mainly only needed if the content is not text (and therefore difficult to index).

  6. Nested topics: There should be multiple tiers of topics, subtopics, subsubtopics, and so on, so I can group information how I see fit, and drill down to an appropriate snippet in the hierachy. For example: “Technology -> Education -> Learning Management Systems -> Open Source LMS -> Moodle”. My “sweet spot” seems to be four or five tiers deep.

    This was one of the major problems I had with AquaMinds’ Notetaker. I wanted to be able to create nested topics, but wasn’t able to go any deeper than sections, subsections and pages. Sure, I could put different topics into different “notebooks” and select each one from the library, but that didn’t seem to be too wholistic to me, nor condusive to the ways in which I manage information. DT allows me to nest topics quite deep, and so I can categorise and group topics in a neatly-maintained taxonomic hierarchy.

  7. System integration: There should not only be an easy way of storing information from the file system, but also retrieving information (i.e., copying data) into the file system. For example, a graphic could be dragged from the application straight into a folder for synchronising with a webserver. Synchronisation with an external file folder would be an added bonus.

    To store information from the file system, either use an import script or drag and drop the item from the Finder into the appropriate place in DT. To move items out, simply drag the item from DT into the Finder. I tried this with a mounted FTP server, and was easily able to upload a graphic right from DT to my webserver. DTPro also has a file synchronisation feature forthcoming, whereby it can “watch” a folder and sync files to and from DTPro to the folder.

  8. Data surety: The system should allow for peace of mind via: a) intermittent (preferably user-defined) back-ups; and b) easy export to non-proprietary forms, like RTF, HTML, JPG and so on.

    DT standard allows you to back up and optimise the database whenever you want. (You can set the preferences to say how many back-ups you wish to keep.) Also, the folks at DEVON Technologies believe that your data shouldn’t be held captive, and that’s a refreshing change from the multitude of commercial applications that hold your work hostage within a locked-in proprietary system (forcing upgrades, enforcing loyalty, and so forth). With this in mind, they allow you to easily export folders and files. Personally, I’ve only tried this once, but it seemed to work well. You can also concatenate entries; for example, you can write chapters of a novel in several different documents, select them all, and then export them as one RTF file.

  9. Semantic “searching” of material: The search should be smart enough to find items by meaning, not just exact words or regular expressions. For example, a search for “primate” should also find texts that contain ape, monkey, chimpanzee and so forth, but not necessarily contain the word “primate” itself. This semantic connection can be made either through concordance derived from context, or from dictionary lists. In either case, it should be very fast.

    This seems to be a particular strength of DEVONthink on OS X (and Nota Bene on Windows, someone mentioned to me). Concordance in DT seems to be generated based upon words in a similar context. For example, it’s quite natural that articles about primates would also use the words “ape”, “monkey” and so on, and the proximity of these words within articles would indicate a relationship. Or so the theory goes: it sounds perfectly sound to me. The only problem with this is that the concordance must be built over time: the more text and articles you create or import, the better the system becomes as recognising these relationships. At first, it’s a little hit-or-miss. That being said, the thousand or so articles I have within my system have already led to a drastic improvement in DT’s ability to find similar pieces.

    Speaking of seaches, this is one area in which I find fault in DT’s current version: non-semantic searching. The base search function allows you to find exact words via ALL or ANY, a phrase (words in an exact order), or wild cards. However, it lacks decent boolean and more advanced search techniques. For example, I’d like to search for primate AND (monkey* OR chimpanzee*) NOT 'a million monkeys typing'. Mind you, I’m sure that this would require quite a bit of CPU as your database swells in size, but it’s very hard to refine word searches otherwise. I hear that DT is getting the boolean search capabilities of its counterpart DEVONagent, so this is definitely a step in the right direction.

  10. Smart folders: I need to be able to find, group and save items gathered from the system, essentially query result lists that refer to items stored elsewhere. For instance, I might have a smart folder called Primates that contains all the articles found in the search mentioned above. Whenever I click on this folder, I should see all the relevant items from all throughout the system on this topic in one place.

    This is a feature in DEVONthink Pro. It consists of groups (folders) which automatically “trigger” AppleScripts whenever they are opened. This script can be a regular query, a request to download feeds, a gathering of to-do items with checkboxes (theoretically, as I haven’t tried this) and so on. However, because the current beta of DTPro is currently lacking meaningful metadata capabilities and therefore the ability to search using it, it can only find items by querying regular content and comments. From perusing the forums and the README file, it would appear that keywords, tags, categories and (hopefully) the ability to search using this information is forthcoming. In an email to Eric Boehnisch-Volkmann, President of DEVONtechnologies, I asked about this. He responded:

    The final version of DEVONthink Professional shall feature categories, that could be used for what you’re suggesting, and a future version is supposed to also have the possibility to add custom metadata fields.

  11. Alternate data forms: Although I definitely require a freeform database, it is sometimes helpful to have a more structured form for storing and manipulating data, like that you might use for a bibliographic entry, a contact, or an application form.

    The DTPro betas have just started to incorporate something like this. It certainly shows promise, but I’m going to hold off expressing any opinions until it’s more complete.

  12. Dependable developers: The system should be developed by a team whom I can trust, and they should respond well to user feedback, incorporate improvements on a frequent basis, and be dedicated to the product for the long term.

    The owner, chief developer and an “evangelist” hang out on the DEVONtechnologies Forum, and take time, effort and care to respond to users. Ideas for future additions to the application are often acknowledged with gratitude, problems tend to be quickly solved, and (although there aren’t a lot of members) there appears to be a sense of community at work. Regarding frequent improvements to the software, I can personally vouch for the vast number of changes since the 1.7 version not so long ago, and few point versions I’ve seen and used lately have demonstrated a continual commitment to adding functionality and addressing user concerns. True, the Pro version is quite a bit behind schedule, but many of the Pro-only features have actually been added to the standard version in the meantime, so users certainly shouldn’t feel neglected.

    In passing, I just wanted to mention a particular topic in the forum: it was announced that DEVONtechnologies had a new partial owner, a U.S. firm who does work as a contractor to the Pentagon. Now, you can imagine the feelings of many international (and more liberal American) users: the thought of doing business with a company that has anything to do with the Iraq war, the prison scandals, bullying (“You’re either with us or against us”) and an irrefutable record of power-mongering and curtailing of personal freedoms, well… it’s not unforseen that strong opinions and resentment should occur. The president of the firm posted his feelings on the matter and how his company was handling the situation, and his posts were passionate and devoid of the PR spin that I had half-expected. I must say that not only did I overcome any feelings of apprehension, but I actually formed a much stronger sense of respect and trust in them.

  13. A dedicated tool: I want a tool for managing information, not planning my schedule, laying out newsletters, retouching red-eye, or finding studs in my wall. I simply want a quick and powerful system to store, annotate, categorise, organise and retrieve multiple types of data.

    Although I’m sure that some AppleScript or Automator wizards will no doubt be able to twist DEVONthink to do many things (a little AppleScript that’s provided to translate languages is a nice example), its central focus is on managing and organising information, not creating blogs, setting up GTD, or any other peripheral task. Although some people will no doubt push the application in innovative ways, the DEVONtechnologies folks know where their primary focus lies.

    Please note that the above is not a slight against an application like Tinderbox, which is extremely powerful and adaptable (and –with the proper encouragement, scripts and attachments– could act, no doubt, as a stud-finder). The point is, I want a freeform database that’s easy to use with many types of data without extra clicks, writing import/export scripts, many hours of (albeit pleasurable) experimentation, or being distracted by all kinds of alluring but complicated geek tools, all stock-in-trade of Tinderbox. In effect, Tinderbox is much like the Emacs of personal content management systems. But, like Emacs, its power and peculiar ways of doing things are often far too complex for simple tasks. DEVONthink, on the other hand, lets me concentrate on gathering, creating and organising content with intuitive and almost reflexive methods.

  14. OS X friendly: While I would generally prefer for the system to be cross-platform and server-enabled for multiple users, my main “hunting and gathering” machine is a Mac, and so this application should at least take advantage of Mac OS X services, scriptability, integration, and other enhancements.

    This is one of the major reasons why I ultimately chose DEVONthink instead of Tinderbox. True, the latter can be controlled via powerful internal scripting variables, agents and other mechanisms (witness how people can actually produce quite functional blogs with the tool), but it is not aware of the Cocoa services and AppleScripting that make working with DEVONthink such a joy. For example, I can drag an “import droplet” into my dock, and whenever I drag a file from my Finder onto it, DEVONthink will copy it into the DT database directories, index the contents, and even create RTF/text from certain types of binary files like PDF and Word docs. The application has an embedded (Safari) web browser, and can view pages right in the main window, as well as capture full web pages –complete with graphics– for offline or archival use. It works with Tiger’s PDFKit to view and manipulate PDF files. It uses OS X’s WebKit to allow you to create HTML files right in the application, with access to source coding. You can cut-and-paste material from a Cocoa browser like Safari, and all formatting and graphics can be preserved. With the new Pro betas, there are quite a lot of AppleScripts included to use and learn from; the AppleScript dictionary for it is quite impressive. In other words, it is very OS X friendly, and a good citizen.

So there you have it. Aside from a few issues surrounding searching and metadata, which are currently in development, DEVONthink fits my criteria almost perfectly.

Dealing with a CMS should be a long-term commitment. Once you have begun to insert entries, categorise data, and feed it daily, the last thing you want to do is export the whole mess and start again with something else. It is a sign of my high opinion of DEVONthink Pro that –for the first time– I feel perfectly comfortable in entering all my more important data into it without any apprehension of a time when I’d switch to something else. (Although, since the application is still in beta, I’m not foolish enough not to back up the my daily.)

So there you have it. A great personal content management system that is able to handle almost any type of data and find those relevant associations that aid and enrich almost any project. It is an infinite and well-organised attic. Holmes, I think, would appreciate DEVONthink Pro.

9 Comments

  1. May 24, 2005    

    I wish I could give DEVONthink a spin, but as I’ve no Mac, I’m out of luck. Alas, “OS X friendly” also means OS X only.

  2. May 26, 2005    

    As a contrast to Sherlock’s theory, here’s a competing point of view from two characters in Robertson Davies’ novel “Tempest-Tost”:

    “Oho, now I know what you are. You are an advocate of Useful Knowledge.”

    “Certainly.”

    “You say that a man’s first job is to earn a living, and that the first task of education is to equip him for that job.”

    “Of course.”

    “Well, allow me to introduce myself to you as an advocate of Ornamental Knowledge. You like the mind to be a neat machine, equipped to work efficiently, if narrowly, and with no extra bits or useless parts. I like the mind to be a dustbin of scraps of brilliant fabric, odd gems, worthless but fascinating curiosities, tinsel, quaint bits of carving, and a reasonable amount of healthy dirt. Shake the machine and it goes out of order; shake the dustbin and it adjusts itself beautifully to its new position.”

    >>Thanks for a great blog.

  3. May 26, 2005    

    Ha! Excellent, Mike!

    From that point of view, I think DEVONthink would also make an excellent dustbin. (Although I’m not sure that’s an endorsement they’d wish to use on their website.)

  4. May 28, 2005    

    … elementary, my dear Johnston!

  5. July 13, 2005    

    Hi, nice summary.

    Can you tell me how can I find the “import droplet” you were talking about. Is it a Pro feature?

    Thanks!

  6. July 27, 2005    

    Your article was written in May, however, I just read it today. Very good viewpoints. I have DEVONthink, I may just have to upgrade to DEVONthink Pro after reading your observations. I see that you mentioned Aquamind’s Notetaker. I would like to know what you think of Circus Ponies’ NoteBook, that is, if you have looked into it.

  7. August 12, 2005    

    Thank you Douglas,

    I’ve been a DT devote for a couple of year now. It has grown as my needs have expanded. However, I’ve never been able to quite describe it. Your piece is wonderful and I’ll probably end up sending this url to a number of people over time.

    I’ve been using the Pro beta through several incarnations and find it excellent. I too have used Tinderbox and loved its sophistication but, too, have wished that it took more advantage of the built-in goodies of OS X as well as handled a wider range of file formats. I even chafed against the same shortcoming is Circus Ponies’ Notebook and AquaMinds’ Notetaker that you mention.

    In short, you have beautifully articulated what I’ve been trying to say for some time. Thank you.

  8. Durbrow's Gravatar Durbrow
    August 17, 2005    

    Excellent review. Just a quick note and query. Circus Ponies Notebook does not suffer from the same flaw as AquaMinds NoteTaker. One can have a rich hierarchy of levels on the Contents Page.

    I do would love to see the author’s take on the new Pro version (and whether it offer significantly more than the Personal edition)

  9. JV's Gravatar JV
    October 3, 2005    

    Great review.

    As a soon-to-be switcher (I have an iMac at home but don’t get to spend a lot of time with it; I’m planning on buying a Powerbook as a writing machine and all-around brain), I’m torn between DT and Tinderbox. I’ve experimented with DT a good bit more, probably because its demo version is much more functional than TB’s. I’ve had a great time entering blog posts into DT, organizing them, and exporting them to ecto for posting to my blog. I could easily see myself writing *everything* in DT.

    I haven’t really tried using DT as a general-purpose notetaking or brainstorming app. TB seems more suited to that, but it’s also byzantine, emacsian and intimidating — as you’ve already noted. I can’t afford to buy both, nor would I want to store my data in two places. I’m leaning towards DT at the moment, but have never been a big fan of hierarchy. I hope you’re right about category and tag support in the final release of DT Pro.

    All that being said, could you be more specific about your experience with TB? How did you attempt to use it? Any gripes other than usability?

3 Trackbacks

  1. ...pickhits... on May 22, 2005 at 2:52 pm
  2. a million monkeys typing » DEVONthink Pro Public Beta 2 on July 28, 2005 at 3:41 pm
  3. D*I*Y Planner on October 17, 2005 at 2:44 pm
  4. The Swamp » Blog Archive » Life Hacks on October 23, 2005 at 6:43 pm
  5. D*I*Y Planner on December 5, 2005 at 8:44 am
  6. a million monkeys typing » 43 Folders on DEVONthink and Smart Groups on May 23, 2006 at 12:11 pm
  7. DEVONthink Professional half-price today only! — Jim Gibbon.com on October 26, 2007 at 3:26 pm
  8. Friday Shorts 28 March | Fragments From Floyd on March 28, 2008 at 7:15 am

Leave a Reply

May 2005
S M T W T F S
« Apr   Jun »
1234567
891011121314
15161718192021
22232425262728
293031  

Categories

Flickr

Driving in Yellowknife

Driving in Yellowknife

At the Snow Castle

At the Snow Castle

Musk Ox Skull

Thaw 4

Cameron River, Early Spring

Thaw 3

Thaw 2

Thaw 1

Yellowknife, Past Midnight

Durrell Wharf, Newfoundland

Rose Hips in Autumn

The Coming of the Storm

Water Lilies

A Zen Moment

Twillingate Lighthouse

Winter Brook

Irises 1

Irises 2