Category Archives: Personal Information Management

Themes for study and learning in February

The themes I wanted to work on  for January were:

  1. Continue with Presence and Deep Work.   I got a lot of reading done on this and some good work in January.  Going forward I’ll be experimenting with strategy and tactics for increasing my Deep Work time (and my presence with respect to Deep Work and, really, everything).
  2. Fascism and Totalitarianism.   Didn’t get far with this, since I wanted to start with Hanna Arendt and (surprise surprise!) it’s in great demand at my libary, so I haven’t risen to the top of the queue.  I’ll continue this thread in February
  3. The Body.   4HB was a bit disappointing on second reading.  Tim Ferriss is a great showman and he has all kinds of cool hacks, but for my Body scheme I’m moving forward with more classic approaches: Weight Watchers, YAYOG (You Are Your Own Gym), and the “Younger Next Year” approach to working (back) up to fitness. 

Themes for February

  1. Continue with Fascism and Totalitarianism.  Hopefully Arendt will become available soon at the library (or I may just have to spring for it).  Open to other suggestions
  2. PowerPoint innards.  I have a scheme to code a web app which will check your PowerPoint deck for “5 common Intelligent Pitching flaws” per my work on Intelligent Pitching over the last couple of years.  See back posts for more.
  3. Poker.  I’m in a regular poker game but not getting any better at it.  Time to buckle down and do some reading and deliberate practice.

Welcome your thoughts…

PIM projects and PIM events: how to organize?

I’ve become a big fan of hierarchical todo-list managers.  It’s great to go down from “All of Life” through Roles to Goals to Projects to Tasks to Next Actions.  I don’t know if it helps me be more productive, but it certainly feels good to think that everything’s connected somehow.

Except some things are not just tree-connected.  They’re graph-connected.

What’s the difference?  A graph can have several “ancestors” (maybe better to call them “predecessors”) for each node.  So a task like “send article to Joe” can belong to “cultivate relationship with Joe” and “send article to tech visionary friends.”

An example of graph structure which most GTD-oriented software supports is “contexts” vs. “projects”.  My project might be to finish the sprinkler controller, but different parts of that project will take place in different contexts: Workshop, Internet, Computer, InMyHead.  Contexts are usually implemented as labels.

One graph-structured problem I’m wrestling with now is the distinction between a project and an event.

I might have a project, say, to improve my friendships with several friends.  And I may have an event where I get together for drinks with some or all of them.

With a hierarchical PIM and contexts, I have a couple of choices about how to implement this:

  1. The Event could own the Projects.  In this case, “Get together for drinks” owns all of the people I’m going to have drinks with.  Problem: it’s inelegant, and it’s transient.  Once the event is over, I have to stick the various friends someplace else.  One would think that the friendships were the more important or less transient or more durable thing and should therefore go above the event in the hierarchy, but there’s no easy way to get a number of parents in a hierarchy to own (or share ownership of) a child node.
  2. The Event could be a context, a label.  Probably a better solution, in that a context is sort of like an event — it’s a physical or logical place where certain tasks can be done — but I’ve already overloaded contexts in my PIM and it’s starting to break down.
  3. The PIM I use now, MLO, allows a node to own an unrelated node elsewhere in the hierarchy, but these relationships are kind of hidden in the software (maybe an afterthought for the developers) and hard to manage.  One wants the graph relationships to be first-class citizens within the software.

Any thoughts?

Complexity — and simplicity — in a PIM

My to-do lists get more and more complex over time.

I’ve explained here why I like My Life Organized (or “MLO” for short).  Briefly, MLO allows you to organize your tasks into a hierarchy, so that bigger tasks can be divided into smaller tasks and tasks without an organizing principle can be grouped into higher-level “projects” or simply “folders”.

But a consequence of this for me — and I think for any PIM-head who takes this stuff seriously — is that the list of tasks invariably grows and becomes more complex.

Most PIM software has a way of ranking or sorting tasks: by “importance”, by “urgency”, by overdue-ness, by all of the above.  It would be nice to build in to the PIM software some kind of scoring for simplicity as well.  One PIM tree is simpler than another if it isn’t as deeply nested, or if there isn’t a large degree of co-dependence between the various tasks, or <pick your simplicity function here>.

You could then have the software detect and warn you if you were adding on complexity at an alarming rate, and give you some tactics (or even semi-automated processes) for simplifying your PIM tree.

Be careful what you wish for, I guess.  I used some PIM software for a while which would actually put your tasks into the calendar based on the importance sort (and available time slots: each task had to be assigned a length).  Tasks could be split up to some extent, and it was nifty for a couple of days, but then it really got like Dr. Evil’s “unattended killer machine” that was supposed to do in Austin Powers and never did.  It was a major task trying to re-jigger the tasks so they would fit into the schedule where I knew they should go anyhow.  And finally, to achieve simplicity, I threw the software overboard and moved on.

Maybe a simplicity automaton would work better than an automatic scheduler, but I suspect it too could be become a tyrant if wholly automated.  Best to make software our companion and not our master.

Down Payment on a PIM use case discussion

My friend Bob and I are teaching together the next couple of days, and I know he gets excited about PIM stuff.  Not quite a PIM-head, perhaps, but a PIM-head-in-training.

We had a not-altogether-clear preliminary conversation about what PIM use case would make Bob buy PIM software.

What excites him, like many, is making better use of his contacts.  Matt may be a doc at Hopkins with life-science research cred (complete fictional example btw) but he’s also an expert on smoking meat.    And his wife is a patent attorney.

You could imagine several ways to code up Matt as a contact (using categories or labels or the like) but two things rise up to bite you: 1) you don’t know all the uses of Matt you’re going to make downstream (you may take an interest in scuba later on, and Matt is a certified dive instructor) and 2) the collection of all the possible labels becomes unwieldy itself; you can’t remember how to code up a smoked-meats expert and end up inventing several labels and making the mess worse.  You now have to scroll down through a hairy ontology-ish beast in order to search or browse, and chances are you won’t.

This was a down payment on a further discussion.  Someday when magically there’s more time Bob and I are going to sit down for a hour or two and work through the various use cases.  It’ll be fun.

The disconnect between to-dos, habits, and calendar entries

I’m still vexed by the disconnect between these three kinds of commitments, which I discussed in an earlier post.

I still use one app for to-dos (MLO) and one for habits (most recently Habitica, as suggested by Chris Bailey.  I’ve had the feeling with my to-do app that things are somehow converging: I’m getting more and more of the features I want each time I do a kaizen improvement and switch to a somewhat better app.

I don’t get the same feeling at all with the habits apps.  I want two things from a habits app which I don’t care about (as much) with a to-do app:

  1. Fuzzy recurrence.  I’ve talked about that here: basically the ability to specify things like “every 2-3 day” or “at least 6 times a week”.
  2. Accountability.  I want some mechanism in the app that makes me feel good when I do the habit and makes me feel bad when I leave the habit out.

These things wouldn’t be terrible in a to-do app, but they’re not as important.  With a habits app, you’re interested in showing up, in Jerry Seinfeld’s unbroken chain.

Habitica gets a C- for fuzzy recurrence.  There are “dailies”, which are things that have to be done every day and if you miss you get dinged.  And there are habits, where you get + or – (depending on the habit) if you engage in the habit, whenever.  There’s some ability to pick which days of the week the habits apply, but it’s far from a rich set of fuzzy recurrence operations.

But the daily-ness of the app (and maybe its very clunkiness) helps me with the accountability.  I have a drive to check off those daily things, because I know the system has no flexibility, and I know I’ll be punished if I don’t.

Punished how?  The gamification elements in the app help me with staying accountable.   I lose Strength and Health if I don’t do my good dailies or if I do do my bad dailies.  Ditto with the habits, although it’s less hortatory with the habits (maybe because they don’t need to be done every day).

My ultimate PIM would combine some of this stuff with the to-do functionality.  Right now I have some things I want to do every day that are in both MLO and Habitica.  And, btw, perhaps the ultimate PIM could link accountability to bank account or reputation or other stuff that some of the “do or die” apps have.

And then having that stuff project to my calendar would be the Holy Grail.

Intentional Objects and Relationship Objects

I’m beginning to think about how to conceptualize the world of personal information management, and I’m finding some food for thought.

First of all, I was quite interested in a Viewpoint by Yoav Shoham in Communications of the ACM (not sure if non-members can read it, but the link is here).

Yoav and two co-founders started Timeful, a PIM company sold to Google in 2015.  His Viewpoint was about Timeful’s approach to generalizing time events.

What they zeroed in on was the intentional aspect of calendar events.  A meeting, a habit, a bucket-list entry, all have a time and calendar element, but they vary in other aspects, including their relationship to our intentions.  Timeful reified these as “Intentional Objects” or IO’s, which were the foundation of the Timeful design.

Sadly, all one can see of this design is a screenshot or two from Yoav’s papers, because Google, after it acquired Timeful, shut down its app and threw the cloak of secrecy over how the work would be developed within the ‘Plex.  Their right to do so, of course, but something of a pathetic move for a company with their stature and cash flow.

Timeful, I’m surmising, used the various aspects of an IO to plan automated scheduling for all of one’s time-based events: the weekly status meeting, the visit to the gym, the daily block of work on the Great American Novel.  A very interesting line of thinking, and an interesting app (although I must say my tastes run more to Intelligent Assistants than to Omniscient AIs: I would rather have the app perform as an amanuensis than as an opaque oracle, but maybe that’s what the Timeful team had in mind, or maybe that’s what Google has in mind.)

In any case, I started thinking about contacts in the same way that the Timeful team was thinking about calendar entries: think outside the box, try to generalize, try to see what human purposes are being served by maintaining a contact list.

A couple of reflections:

  1. I have contacts in my contact list where I’m interested in the individual, where I’m interested in the couple, and where I’m interested in the household (e.g., neighbors).  Why do we have to treat these all the same way in a contact list, and how do we design a “Relationship Object” that will  capture the nuances of these differences and allow our software to work expressively with them.
  2. There are lots of non-humans that want me to treat them like humans.  In particular, corporations and brands want me to Like them, to white-list them, and to store them as contacts just like my individuals, couples, and households.  Are these really the same thing?  Why not?  Is it because of the asymmetry in our relationship (they can yammer at me; I can’t really yammer at them, for example), or is it because of the power asymmetry?  Is there a power dimension to all contacts?  How to model that?
  3. How about contacts that can order me around (IRS) or even kill me (Selective Service, for those of us who remember the draft)?  How do we model that?  Do their “requests” (demands, really) have more urgency than others’?  More importance?

Love your thoughts on all this, and on how IO’s and RO’s and <other>O’s would interact in the PIM of tomorrow…

Starting up the “Recurrent Tasks” PIM project

So, some developments on the PIM front.

First, off, in conversation with Larry Fitzpatrick I’ve decided to bite off an actual project, as a way of jumpstarting my efforts to write the ultimate PIM as well as to re-start as a coder and — who knows? — feather my nest in other ways as well.

I had been thinking about a project of scraping social media — Facebook, LinkedIn, Pinterest, Twitter, etc. — to generate personal “Klout-like” stats.

Why scrape?  As I began to look into it, it seemed that most of these sites had APIs for business use — Fan pages, etc. — but nothing for personal use.  Want to figure out engagement for your Facebook pages?  They’ve got an API for that.  Wanto to figure out engagement with you as a person.  SOL.

Larry talked me out of that.  He said that there was a lot of “pumping out entropy” in any scraping effort, and I would end up with a needs-to-be-maintained-constantly hack which didn’t prove any concepts or really make much progress toward the goal of the ultimate PIM machine.

So he asked me to say the first thing that came into my head for a more substantive theoretical piece of the PIM pie, and I said “recurrent to-dos”.

I blogged about them here.  They’re theoretically interesting.  They actually lie at the heart of what makes me most interested in one or another PIM product (so they differentiate one’s hack).  And they are pretty kludge-y in most PIMs, from Outlook to Todoist to Toodeldoo and beyond.

So I started thinking about it.  I had also been reading some of Paul Graham’s essays (if you haven’t read them, they’re wonderful; go here), and he had written about using LISP for coding his ecommerce platform (eventually bought by Yahoo).

It was like Proust’s madeleine.  It took me back to an era where I wrote in LISP and, more importantly, thought in LISP.  And in LISP, recurrent objects are kind of trivial.

Trivial in the sense that there’s no difference between a function and a variable in LISP: generators look syntactically just like atoms (except that they may have arguments, etc.).  The language doesn’t make you jump through hoops to use a function where you might use a variable (or vice versa).

So maybe LISP would be a good medium for experimenting with how to do recurrent PIM patterns, since a PIM pattern may be nothing but a generator for to-do events (I say “may be” because one wants to undo recurrent to-dos, update them, and other operations TBD).

So I was wondering if Graham were really right and it would be possible to right a major server-side app in LISP.

And then it hit me: JavaScript treats functions just the way LISP does!

This could all be done in JavaScript.

So now I can kill three birds with one stone: get started re-sharpening my coding skills, come up to speed on JavaScript, and move forward on the PIM project.


Next step is to do some experiments with a Recurrent_Generator class.

Recurrence patterns in PIMs

A really essential element in any kind of personal information manager is a recurrence pattern.  You know: “every 3 days”; “at the beginning of the month”; “2 weeks after I’m done.”

For some kinds of PIM apps — habit managers come to mind — you might even say that the richness of the recurrence pattern repertoire is the main element driving me to pick one over the other.

The problem is that doing recurrence patterns is hard, and doing them right is even harder.  They are perverse.

Recurrence patterns inherit at least some of their perversity from the calendar itself.  We have years that don’t divide into weeks.  We have months that don’t divide into weeks either, and that don’t even have a uniform length.  We have weekdays and weekends.  We even have different clocks, since our legacy clock the Earth is slowing down.

(Scientists add a second to the year every few years.  I have this picture of a team in white coats bringing over a bar of light a light-second long and opening an access cover and throwing into the stream of light that is Time.  Given how long a light-second is, that’s a pretty big team.)

But then the nature of recurrence adds some extra perversity all its own.

In the first place, some recurrences are regular and some are driven by completion of the event.  “Every two days” is not the same as “two days after I finish.”

And some recurrences have more oomph (a technical term!) than others: you’ve got to take your medicine every day, you ought to exercise every day (but the sky won’t fall if you don’t), and it would be nice to write a handwritten thank-you note after every dinner party.

(Perhaps the oomph is best handled not in the recurrence mechanism itself but in a orthogonal priority or urgency mechanism.  Even so, it would be nice to express the oomph at the same time you created the recurrence pattern (but see Syntax and Command Language below).)

And then, as if the foregoing weren’t enough, some recurrences are fuzzy.  “Every 2-3 days.”  “4 times a week max.”

I once used a habit manager app for a few months just because it had some cool fuzzy recurrence features.  It didn’t interoperate with anything else, it only ran on iOS, and it didn’t sync with the cloud — all flaws that eventually drove it down.  But those fuzzy recurrences were pretty cool: I miss them still.

So these are variety issues.  Recurrence patterns are hard in part because of the variety.

But there are (at least?) three issues with recurrence patterns aside from their variety:

  1. Controller interface.  A lot of recurrence providers try to offer a natural-language interface to recurrence patterns.  “Just type in a phrase that describes when your task has to be done”.    I’m not blown away by these (and I’m an old NLP guy from the ’80s).  You really can’t say what you mean often: “The 2nd Saturday of each month” may well parse to “every other Saturday” in one system and to its intended meaning in another.  Having a form to fill out is better, because you can at least see all the options that can be generated, but forms don’t do well with fuzzy recurrences.
  2. Internal Representation.  Whatever the implementation of a task or an event, a recurring task or event is a different kind of animal: it’s not a data structure, it’s an object.  Outlook, for example, doesn’t even seem to store recurring objects in the same file as task or calendar objects.  When you migrate, they don’t always migrate together.
  3. “Round Trip”.  The different representations of recurring objects means that at some point they will need to “compile” to their non-recurring cousins.  One would want this to be a round trip in the sense that you could traverse from the recurrence to the end objects and also reconstitute the recurrence from the end objects.  No one does this.  Once a recurring object has been instantiated, there’s no going back.

For all these reasons, recurring patterns are hard, and getting them right is an architectural key to a great PIM.

Tagging by Doing

Part of building a better PIM infrastructure is undoubtedly solving the “tagging problem.”

The tagging problem is: how do you get users of a PIM system to tag their data 1) promptly 2) accurately 3) with regard to downstream users, and 4) MECE-ly without requiring them to use a heavy-duty ontology of some sort?

  1. Promptly.  The best thing is to tag data at the point of production, at least for salience to the person entering the data.  For downstream taggers, promptness is also of value as the information comes to them.
  2. Accurately.  There are two senses of accurate (maybe more?): Accuracy1 is making sure that data with the same semantics gets the same tags.  Accuracy2 is making sure that the tag is appropriate to the data.
  3. Helpful Downstream.  In a perfect world, the tagging with all be helpful to future users of the data.
  4. MECE-ly.  MECE is McKinsey’s acronym for “mutually exclusive, collectively exhaustive” which describes a set of tags which, for want of a better word, are what linear algebra would call a basis for a knowledge space.  Every item fits into one of the tags, and no data fits into two of the tags.

The tagging problem has no real solution, and the lack of a solution may be the downfall of many a knowledge system, whether personal or tribe.

Solutions are elusive because it’s not immediately in the interests of the tagger to do the tagging, even when it might be in their long-term interest.  Of course, it’s always in the long-term interest of any user of the system.

But what could you do if tagging occurred as an automatic by-product of some other operation?  One that users wanted to do?

I once trained an open-source Bayesian categorizer to distinguish spam from bacn from desirable mail in my inbox.  I did it by doing what I would have done anyhow — dragging the incoming mail into good, bacn, and spam folders.  But while I was doing that the classifier was changing its weights (or whatever classifiers do) and automatically improving its ability to triage my mail.

Could that same technique be applied to more complicated tagging problems?  Not sure.  It needs some thought.

Microsoft search stinks, except for OneNote

First of all, I’m not a MIcrosoft basher.

I’m not even a big foe of Microsoft.  I made a good living from programming with Microsoft products for years in the ’80’s and ’90’s, and I don’t like to bite the hand that fed me.

But, more than that, there are Microsoft tools I’ve liked, some I’ve loved, and some I still love (Excel, yes, and Word too).

However, Microsoft search, almost across the entire company, is terrible.  Bing search is inferior.  Visual Studio search is impossible.  Desktop search didn’t work well at all.  Windows 8/10 search is very very lame.

With one exception, perhaps: as I wrote here, I’ve moved off of Evernote back to OneNote, and OneNote search is pretty darn good.

First of all, it’s better than Evernote search.

Evernote search is quirky and clunky.  I type in a word that’s in the text and in a tag, and then when I click on the tag (thinking I’ll get all the results with that tag) I inexplicably get nothing at all.  I have to go back and say “search by tag” and then scroll down and find the tag and click on it.  I don’t get it.

In OneNote, you type in a word, you get a list of pages grouped by salience (at the top are pages with the search term in a title, then in the body, then in less-visited places (Recycle bin, Misplaced sections, etc.).  You click on a search result and you go to that page.  Take a note from OneNote, guys at Evernote.  It doesn’t have to be clunky.