Nick Winter’s Big Success with Percentile Feedback

I have posted several times about using what I call percentile feedback to boost productivity. Percentile feedback means comparing your current performance to your previous performance using a percentile. If the current performance is in the middle of your previous performances, the percentile is 50, for example. Percentile feedback is easy to understand (scores above 50 are better than average) and is sensitive to small improvements — so even small improvements are rewarded. My implementation had three other helpful features: 1. It adjusted for the time I woke up to make different days more comparable. 2. It measured efficiency (time working/time available) to further improve comparability across days. 3. It was graphical. I made a graph of efficiency throughout the current day versus previous days. It greatly increased how much I worked every day.

I love it and wish I had it for everything I measure. Unlike so many feedback systems, it is realistic and encouraging. I found it worked extremely well — to my surprise, actually. It’s not so surprising I would think of it because it vaguely resembles an animal-learning procedure. (Animal learning is my area of expertise within psychology.)

Nick Winter, one of the developers of Skritter (which I use), recently started to use it. He gave a much-too-short QS talk about it in Pittsburgh a month ago. I asked him about his experience. He is as enthusiastic as I am. He wrote:

The percentile feedback has been a huge success–I’m getting way more done than I ever did, and I’m much better at prioritizing toward my main project. Seeing the graph going in real time has been much better at making me aware of what I need to do to hit high targets each day. I will do a full writeup on this, and on my self experiments, when I finish this iOS app and stop focusing so much on work. The short teaser goes something like this:
Phase 0: just tracking normal work at end of day in a Google Doc, average 2 hours a day on iOS development
Phase 1: tracking normal work and iOS dev separately in the Google Doc, average 4 hours a day on iOS development
Phase 2: using Beeminder to have better graphing and goal incentive for iOS dev, average 5 hours a day
Phase 3: first three weeks of using percentile feedback, average 6.4 hours a day
Phase 4: second three weeks of using percentile feedback, deciding to really push it based on the positive feedback from my metrics (more productivity, more happiness), average 9.4 hours a day
So now I’m getting close to averaging 70 hours of focused iOS dev a week and it feels great. In a normal work place, “time spent working” != “productivity”, but for me they’re very similar as long as my energy is good, which it almost always is now.
The surprising insight is that changing the way that I measured my work performance–from spreadsheet, to better spreadsheet, to graph, to better graph–has had such a huge impact. I have been working on maximizing work productivity for four years, ever since starting the startup, but in the last six months I’ve become radically more effective. I love the percentile feedback graph design!

You can see his implementation on his homepage.

7 thoughts on “Nick Winter’s Big Success with Percentile Feedback

  1. The percentile feedback actually worked very well for me.
    I used your R scripts on Max OS X.

    I have a two-display set-up, so I dragged the window with the chart to the second display, so that it wasn’t obscured by other windows. This was a critical factor for me to make this system effective. When I started to see the productivity chart all the time, it started to be much more motivating.

  2. I’m very interested in this but I’m not sure it’s a feature or a bug that the system results in feeling “great” about working 70 hours a week.

    Perhaps it is a way to become pleasantly addicted to extremely hard work, but that’s a dangerous power. I was working about 60-75 hours/week doing ios development over Jan & Feb, and in the end I was sick (but I’m not in my exactly young).

    Also, I worry that using monitoring to drive up efficiency means you get very efficient at what you can monitor, and crowd out all slack which is valuable over the longer term for creativity even if it’s benefit harder to monitor. Reading this post, for instance, would not count as productive.

    That said, if there were a good app for this, I’d be trying it tomorrow.

  3. Alexis, I can see your point about motivation hacks potentially cutting off useful signals sent by unmotivation, and about blindness to the unmeasured areas of life. I use other tools (like daily journaling, happiness/health tracking, and time set aside for faraway thinking) to be strategic about those things, which I think is better. But certainly it should be pointed out that one should not blindly hack productivity without another way of getting perspective.

Leave a Reply

Your email address will not be published. Required fields are marked *