Not long ago, Howard Wainer, the statistician I mentioned recently, learned that his blood sugar was too high. His doctor told him to lose weight or risk losing his sight. He quickly lost about 50 pounds, which put him below 200 pounds. He also started making frequent measurements of his blood sugar, on the order of 6 times per day, with the goal of keeping it low.
It was obvious to him that the conventional (meter-supplied) analysis of these measurements could be improved. The conventional analysis emphasized means. You could get the mean of your last n (20?) readings, for example. That told you how well you were doing, but didn’t help you do better.
Howard, who had written a book about graphical discovery, made a graph: blood sugar versus time. It showed that his measurements could be divided into three parts:
measurement = average + usual variation + outlier (= unusual variation)
Of greatest interest to Howard were the outliers. Most were high. They always happened shortly after he ate unusual food. Before a reading of 170, for example, he had eaten a pretzel. He had not realized a pretzel could do this. He stopped eating pretzels.
When Howard told me this, it was like a door had opened a tiny crack. Recently a deep-sea treasure-hunting company found a shipwreck off the coast of Spain. They named it Black Swan, apparently a reference to Nassim Taleb’s book. Shipwrecks are black swans on the ocean floor; black-swan weather had sunk the ship. For Howard, outliers were another kind of buried treasure: the key to saving his sight.
It isn’t just Howard. Outliers are buried treasure in all science. They are a source of new ideas, especially the new ideas that lead to whole new theories. The Shangri-La Diet derived from an outlier: Unusually low hunger in Paris. My self-experimentation about faces and mood started with an outlier: One morning I felt remarkably good. My discovery that standing improved my sleep started with a series of days when I slept unusually well.
Modern statistics began a hundred years ago with the t test and the analysis of variance and p values — very useful tools. Almost all scientists use them or their descendants. Almost all statistics professors devote themselves to improvements along these lines. However, conventional statistical methods, the t test and so on, deal only with usual variance. (Exploratory data analysis is still unconventional.) As Taleb has emphasized, outliers remain not studied, not understood, and, especially, not exploited.
I didn’t lose the weight first and spot blood sugar outliers afterwards. They happened at the same time and were related. Losing weight is hard. It is hard because if you weigh yourself in the morning and then eat a double bacon cheesburger, extra fries and a thick shake andf weigh yourself again, nothing has changed. Similarly, if you eat very carefully all day and then weigh yourself again at night, nothing has changed. So there is no immediate feedback to reward good behavior our discourage bad. Blood sugar is fast — if you eat something you shouldn’t you find out immediately. And the long-term consequences of poor eating habits are dire (you go blind and your legs get amputated before dying prematurely (see Jackie Robinson as one very sad example). By controlling blood sugar I was forced into healthy eating (a Hobbitt diet — eating 5-6 small meals a day) and the weight fell away at a bit over 1/2 lb/day. I went from 240 to 191 in a few months. Unfortunately, despite maintaining a very vigorous exercise regime, I lost muscle as well as fat. Now, a year later I am still between 190 and 195 and blood sugar is well controlled. And although I can do as many push-ups now as I could before, I can’t bench press what I used to.
Seth: I love the metaphor of outliers as buried treasure. The problem is to distinguish between junk that you dig up and really valuable stuff.
Howard: Immediate feedback is very important in game design (players get frustrated and give up if they can’t figure out what the effects of their actions are). I think Seth has actually blogged a bit about the concept before … Putting these sort of situations in place in everyday “games” is one element that increases motivation. Runners often use a similar sort of approach by measuring their heartbeat rate.
@Howard Wainer
You may want to check out two fascinating blogs. They might be able to assist you with your weight loss, retaining your sight (None of my business, but I assume this is diabetes related) and general health.
Both are well-written, somewhat out of the mainstream thought, for an educated audience and employ statistics and logical reasoning in their arguments.
1) Art de Vany’s Blog on Evolutionary Fitness and Diet – https://www.arthurdevany.com/
Excerpt from an interview on T-Nation
Charles: I don’t know if I read in the first chapter of your book on in the blog, you said that when you were in your 40′s you started making changes in your lifestyle. I wanted to ask you what was the catalyst for those changes and what did you start doing differently?
Art: That’s very interesting, because it was probably when my wife developed type-1 diabetes and we moved to California in ’84. She had developed it 2 years prior to that.
My son had been an infant onset juvenile diabetic at the age of 2, so I began studying metabolism quite intensively, just trying to keep track of his health and keep him healthy.
And then having my wife have the same problem develop, obviously it’s in her genetic stock on that side of the family, the autoimmune illness.
Charles: Sure.
Art: I, by that time, accumulated a lot of evidence as to what foods elevated what glucose, and began systematically eliminating those, even with my son. But we became more ruthless at it when my wife also developed type-1 diabetes.
So, if you just look at the evidence, somebody who can’t control their blood sugar adequately, because they don’t respond with an insulin response, as do normal people, then what you find is the kinds of foods that I eliminated from the diet were all of the things apparently so heavily criticized for having high glycemic index or high glycemic load.
So, that was the beginning of it. And then, I migrated towards a hunter/gatherer model, simply by looking at the evolutionary history and what kinds of foods our ancestors ate.
2) The Hyperlipid Blog – https://high-fat-nutrition.blogspot.com/
Written by a man named Peter who is a physiologist and biochemist training. He follows an 80% fat diet and is constantly trying to prove himself wrong ala Karl Popper.
Two posts on diabetes:
https://high-fat-nutrition.blogspot.com/2008/03/diabetes-and-hunger.html
https://high-fat-nutrition.blogspot.com/2008/03/diabetes-and-cardiac-apotosis.html
Best of luck.
Seth – two thoughts on outliers.
1. An outlier in statistical jargon is often called a contaminant — as in a contaminated normal distribution. One goal of statisitical analysis is to reduce the influence of outliers on the overall estimate. This is typically done by separating outliers from the main body of the data for different treatment. Contaminant is not meant as pejorative. In south Africa good building stone is sometimes contaminated by diamonds.
2. I am reminded of Isaac Asimov’s observation that scientific discovery is almost never accomplanied by “Eureka.” Much more often it is signaled by “that’s funny.”
Hope you are well,
H