When Monitoring a Behavior Makes it Worse

I've been doing some idle life-logging experimentation for the past two months with a Google Form and some simple Matlab analysis (project repo here). It was partly motivated by trying to operationalize some of my thinking around habit formation and falling off the wagon/getting back on, and partly by the vague idea that I might in the future unbundle Tempo into small idea chunks and rebundle those into an app, instead of writing a second edition. In the two months, I learned a few interesting lessons big and small. Some were about the right way to log and analyze behaviors, others were about the nature of behaviors and habits themselves. All very interesting. But perhaps the most interesting thing I learned was about the very idea of monitoring behaviors (with anything from a diary to an app) is that if you measure a behavior, it generally gets worse before it gets better, if it gets better at all. Kinda like if you think too much about how you work the brake and accelerator while driving, you'll suddenly start fumbling/jerking awkwardly like a student driver. I think there are two things going on here:

1. When you bring up any habit for conscious inspection with a tool, you regress from unconscious competence to conscious incompetence (see shu-ha-ri). This happens because most of your later mastery is unconscious, and paying conscious attention to what you're doing suspends the unconscious parts.
2. When the habit is a creative habit, there is an additional factor. For an uncreative habit, feedback of error via inspection or monitoring triggers dumb corrective actions. If you're drifting out of your lane and your fancy new car beeps, you just steer back in. But if your monitoring is telling you that your "hit rate" for successful blog posts as a fraction of all blog posts is falling, there is no obvious action you can take to fix it. So being sensitized to the gap just increases anxiety, which makes performance worse.

The first is a manageable problem in a thoughtfully designed tool that foregrounds and manages the trade-off by setting the right expectations: "warning: this logging/monitoring app will make things worse before it makes it better, like any skill-learning aid." The second is a much more serious one. When the right response to a feedback error is a creative action, the tradeoff is between knowing more about the "stuck" situation versus heightened anxiety that prevents you from doing much with the data. Arguably, in this regime, the right way to handle the tradeoff is to turn off the monitoring and go open-loop for a while, trusting creative play behaviors to generate an event that unsticks you. I think this is why common self-improvement goals like weight loss run aground once you hit the existing homeostasis point.  If your body set-point is say 150 lb and you are at 155 lbs due to too much Thanksgiving and Christmas over-eating, the diet-and-exercise routine in response to what you see on the scale everyday is enough to get you back to 150 lb. But if you're hovering in the noise zone around 150 lbs (say +/- 2 lbs) and want to move the setpoint itself to 140 lbs, you need a creative lifestyle shift. Watching the scale daily is not helpful in achieving this goal. You need something else. I am not entirely sure about how to approach this interesting problem, but for starters, I think it's useful to segment tools and behavior modification projects into two kinds: sustaining projects (no set points need to move, no creativity needed) and disruptive projects (set points need to move, creative insights needed). They are two very different regimes of behavior modification, and inspection/feedback/monitoring tools work very differently in the two regimes. I believe we fall off the wagon when we have to shift between these regimes.