Month: March 2007

How Good is the Atkins Diet?

Seth Roberts2 Comments

A new study, just published in JAMA, compares several popular diets: Atkins, Zone, Ornish, and LEARN (a conventional-wisdom-type diet based on “national guidelines,” according to the paper). The Atkins diet did much better than the other three. The results were quite a bit more positive for Atkins than an earlier comparative study where compliance was poor, weight loss was minimal, and no diet was clearly better than the rest. The Atkins Company, not surprisingly, is pleased with the new study; they have put it in their research library.

Here is what the researchers concluded from their data: “A low-carbohydrate, high-protein, high-fat diet may be considered a feasible alternative recommendation for weight loss” (from the abstract — the meaning of “alternative” is not explained).

However, a graph in the paper (Figure 2 for those of you with access) makes a very important point that the researchers don’t mention: Persons on the Atkins diet weighed more after 12 months on the diet than after 6 months. After 6 months, in other words, the lost weight was coming back. The regain is not small: From Month 6 to Month 12 the Atkins dieters regained about one-quarter of the weight they had lost. At the end of the study (Month 12), they had lost about 10 pounds.

My interpretation is that the Atkins Diet works for two reasons: 1. The food is new. The flavors of the new food are not yet associated with calories. The novelty wears off, of course. This is why some of the lost weight was regained. 2. High-glycemic-index foods (such as bread and potatoes) are eliminated. This produces permanent weight loss, but not a lot. When I started to eat low-glycemic-index foods I lost 6 pounds, which I never regained. A 6-pound loss is not terribly different from the 10 pounds (average) lost by study participants.

In a newspaper article, the study’s lead author mentioned the regain:

As the study progressed, [Christopher Gardner, an assistant professor of medicine at the Stanford Prevention Research Center] said, some dieters put back on some of the weight they had lost early in the year.

That’s misleading. It wasn’t “some dieters” — it was a trend shown by the whole group. But at least he (kinda) mentioned it.

Jane Jacobs on College

Seth Roberts2 Comments

Jane Jacobs, the urban and economic theorist, wrote:

Only in stagnant economies does work stay docilely within given categories. And wherever it is forced to stay within prearranged categories — whether by zoning, by economic planning, or by guilds, associations or unions — the process of adding new work to old can occur little if at all.

In the case of college, the “work” is post-high-school education. College students are not forced to join a union but the need for credentials forces them to attend college, where, as Jacobs correctly predicts, a narrow range of subjects is taught in a narrow range of ways. Take my department (psychology at UC Berkeley). As one of my students, a psychology major, asked, why isn’t there a course about relationships? That’s what’s really important, he said. Yes, why not? There has never been such a course at Berkeley nor, to my knowledge, at any other elite university. What a curious omission. And why do practically all classes involve lectures, reading assignments, and tests? Aren’t there a thousand ways to teach and learn? I think Jacobs has the answer: Work has been forced to stay within prearranged categories — categories that seem increasingly outdated. The pattern of chapters in almost all introductory psychology textbooks (which cost about $100) derives from the 1950s!

An earlier post by me about college. Other people’s comments. Jane Jacobs on the food industry and scientific method.

Andrew Gelman Interviews Me About TV and Mood

Seth Roberts2 Comments

Andrew did this interview for Stay Free!, a magazine about media and consumerism, in 2000. They didn’t publish it.

AG Why don’t you start by describing your method of using TV watching to cure depression?

SR To feel better, you watch faces on TV in the morning and avoid faces (televised and real) at night. TV faces are beneficial in the morning and harmful at night only if they resemble what you would see during an ordinary conversation. The TV faces must be looking at the camera (both eyes visible) and close to life-size. (My experiments usually use a 27-inch TV.) Your eyes should be about three feet from the screen. Time of day is critical–if you see the TV faces too early or late they will have no effect. The ave contact with other people has a big effect on when we are awake; and (c) there are many connections between depression and circadian rhythms. Depression is closely connected with insomnia, for instance.

AG I generally think of TV as an evil, addictive presence in American life. Do you think there’s something dangerous about giving TV this “badge of approval” as a medical treatment?

SR It’s not quite a “badge of approval.” Seeing faces on TV at night–which of course is when most people watch–is harmful, my research suggests, if the faces are close to life-size. And they often are. Maybe TVs will be made with variable picture sizes–one size for morning, another size for night. When I watch TV at night (very rare), I stay as far away as possible.

AG I mean, if this method really worked, I could imagine the Depression Network running talk shows in the morning that are basically infomercials for Prozac or whatever. Would you worry about that?

SR No. I watch faces on TV every morning and would appreciate more choice. I suspect the morning shows would not be Prozac infomercials, however, because the people watching would not be depressed.

One thing that bothers people about your plan is the idea of TV as a substitute for human contact. I think that most of us–even people who spend a lot of time watching TV–find this idea upsetting. It’s like “Brave New World” and virtual reality. Are you at all bothered by recommending to depressed people that they sit inside watching TV?

“Substitute for human contact”? True, but why is that so bad? Reading–which TV critics, many of them writers, seem invariably to like–is also a substitute for human contact, of course. Agriculture is a substitute for hunting and foraging. Vitamin pills substitute for food. Civilization is all about substitutes–about being able to fulfill needs in many ways.

Still, I think watching faces on TV in the morning is only a partial solution to the problem of depression, just as nutritional supplements (e.g., iodized salt, folate added to flour) are only partial solutions to the problems caused by a poor diet. A fuller solution would include changing when most people work. The usual pattern is work (morning and afternoon) then socialize (evening). A better pattern would be socialize (early morning) then work (late morning to early evening)–and go to bed early. I do my little bit for the revolution by inviting friends to brunch rather than dinner. The revolution would also include picture phones with life-size faces.

I heard you say once that depression is ten times as common now as it was 100 years ago. Where do you get that information from?

Many articles have made that point. One of them is: Klerman, G. L, & Weisman, M. M. (1989). Increasing rates of depression. Journal of the American Medical Association, 216, 2229-2235.

If depression is a consequence of modern life, do you think there’s something strange about seeking a technological solution for it? It’s sort of like saying, people are too atomized, so let’s solve the problem with even more solitude?

It is one of many technological solutions to problems caused by “atomization”–people being farther apart. Telephones, air travel, and email are other examples. So it isn’t strange. If my subjects are any guide, watching TV for an hour every morning would not increase the solitude of most depressed persons. They are already alone during that time.

Would listening to the radio be OK?

No. You have to see faces.

Have you ever tried to get your research sponsored by TV stations or networks or, for that matter, a publication like TV Guide?

No, but I once put a “TV is good” ad (ABC) on my bulletin board.

Andrew Gelman on Web Trials and the Shangri-La Diet

Seth Roberts2 Comments

Andrew Gelman is a professor of statistics at Columbia University. Years ago we co-taught a seminar about left-handedness. His blog. This interview took place via instant messaging in February, 2007 and has been edited slightly.

SR I want to ask your opinion of web trials. People go to a website where they choose or are randomly assigned a treatment. Then they come back and report the results.

AG Then the records of their choices and outcomes are made publicly available.

SR Yes. And there would probably be some summary of the results prepared by experts. It wouldn’t be just raw data.

COMPARING TO CURRENT STATE OF THE ART IN MEDICAL RESEARCH

AG We could compare to the current state of the art in medical research, which I think is to have some moderately large randomized clinical trials, each of which is published in a journal, followed by a meta-analysis of these trials. A difficulty with the current state-of-the-art is that sample sizes in clinical trials seem to be simultaneously too small and too large. Too small in that results tend to be just barely statistical significant (and often not significant for subgroups), so that you can’t really put your faith in one study, hence the need for meta-analysis. Too large in that each study is unwieldy, takes a huge amount of effort and doesn’t allow for much learning and experimentation during the study.

SR. A famous epidemiologist [Richard Doll] once said that if the effect is strong, you don’t need a big study.

AG In some way the high cost is a good barrier in that people have to think seriously and justify what they want to do. On the other hand, within any particular research plan, it would seem to limit the possibility for innovation.

Speaking generally, a challenge is to integrate clinical judgment (including ideas of experimentation and trying different things with different patients) with scientific goals such as replicability.

Also, there are well-known cognitive illusions in clinical judgment, which is what motivates the evidence-based-medicine movement (for randomized trials, public records of data, etc.) in the first place.

SR How do web trials fit into the picture you have drawn?

AG Ideally, web trials are intermediate between controlled randomized trials on one hand, and full recording of observational data on the other. If people are really volunteering to be randomized, then they follow the protocol, then this is a clean randomized expt (albeit not blinded, an issue I’d like to raise with you). In practice there will be lots of selection, dropout, measurement error, etc., which moves it toward an observational study. The dispersed nature of the data collection is similar to (in fact, more dispersed than) the idea of individual clinicians recording their experiences and outcomes into a centralized databased. That is, the data collection is dispersed, the database is centralized.

SR A web trial would have more regularity — less variation — across subjects than observations collected from individual doctors. Because everyone would get the same instructions. Whereas different doctors are obviously going to give different instructions (for the same nominal treatment).

AG Yes. That’s why I said the web trial is in between.

DIFFICULTIES WITH BLINDING

SR In the area of blinding I think a web trial would be better than the conventional double-blind clinical trial. If the goal is to guide practice. In practice patients are not blinded. Blinding is a tool to equate expectations. Better to equate expectations by comparing different treatments both believed to be effective.

AG One of the difficulties with your self-experimentation is that there’s no blinding at all. Similarly with these trials. Some of it is the nature of your treatments, but perhaps with some effort you could come up with blinded versions.

SR In my self-experimentation the expectations are equal in the different conditions, in many cases.

AG For example, consider the recent self-experiment that you describe on your blog, where you try different oils and measure your balance. I’d believe these results a lot more if you blinded the treatments.

SR Sure, blinding would help in that case, I agree. I plan to do something like that. But blinding is not necessary to equate expectations. For example, I tried many ways of losing weight. In every case I expected it to work. Some ways worked much better than others. It is this comparison of the effects of different treatments that is interesting. In general expectations cannot be very powerful or there would be no problems left to solve. Expectations are powerful in a few areas and seem to have no effect in many areas. I don’t mean we should ignore them; but to emphasize them as a big deal is not what the evidence suggests. In any case in web trials the participants would only be randomized (or choose) treatments they thought might work

AG There’s some work by Rubin and other statisticians on “broken randomized trials” which can more generally be thought of as experiments that have partial randomization.

SR I think of web trials as giving “entrants” (or subjects) a choice: to be or not to be randomized. Then when it’s all over you compare the two groups.

AG That makes sense. You’ll still have some problems: 1. People not following protocol. 2. Non-blindness of treatments. 3. Other problems, I’m sure, which I can’t think of offhand.

SR Well, these are equal for all conditions so they shouldn’t distort anything

AG In a controlled trial you can deal with some of these things: 1. In a controlled trial you can have more interactions with the experimental subjects, thus maybe more likely they’ll follow protocol. 2. In a controlled trial you can (sometimes) ensure blindness. In general, I don’t think you can get away with assuming that biases cancel out.

ANALYZING DATA FROM WEB TRIALS

AG Your web trials should give us a big juicy source of data that can be thrown at a stat Ph.D. student as a thesis project, perhaps! My intuition as an amateur sociologist of applied statistics is that an exemplary applied analysis is a good way to kick-start the study of a statistical problem.

SR What’s an example of such a kick-start? That’s an interesting point.

AG I’m thinking of the hierarchical models that were fit by Lindley, Novick, Rubin, and others in the late 1960s thru early 1980s to educational data. These provided examples for people to follow–templates–as well as demonstrations that these methods really worked. There were various interesting disciussions of these models in the stat literature, in particular I’m thinking of a paper by Rubin on law school validity studies in J. Amer. Stat. Assoc. from 1980 that had several discussants.

SR Yes, it is true that the data from web trials would be complex and interesting in new ways and accessible to everyone.

AG Yes, having available data is another plus–that’s really a new feature which should help. Now back to the warnings. A very well known example is the Nurses Health Study, an observational study that found that taking post-menapausal drugs was associated with lower heart-attack risks (and lower death rates). But when a big randomized expt was done, no association was found. Actually, taking the drugs slightly increased cancer risk, I believe. See here.

I talked with various people about this, and there are different potential explanations for the discrepancies. One story is that the women who took the drugs were otherwise healthier, more health conscious, etc.–even after controlling for whatever pre-treatment variables they controlled for. Another story is that the populations of the 2 studies were different (in particular, in their average ages), and perhaps the drugs are beneficial for some ages but not others. (Incidentally, the drugs were not originally intended to reduce heart-attack risk. This was an unexpected effect (or non-effect), I believe.)

Anyway, the people I trust on these matters (notably John Carlin) believe that the difference is because of “selection”, i.e., the drugs don’t really reduce heart attack risk. But the observational study led people to recommend the drugs. So this is a big example where the obs study was misleading.

SR: Did the randomized study conclusively rule out the effect size seen in the correlational study? or did it simply find no effect?

AG I’m not sure. My impression is that the expt actually contradicted the obs study–a stat signif negative effect for one, and a stat signif positive effect for the other–not just that there was significance for the expt and no signif for the obs study–but I never really looked into it.

SR I’d like to return to the issue of blind vs don’t blind. You believe any experiment where subjects are not blind to the treatment has a problem?

AG Yes, if knowledge of the treatment could affect the outcome (for example, through motivation). I worry about it for your diet and depression studies.

SR Well, in much research the first question is whether there is a useful effect. later experiments deal with mechanism. I was under the impression that what matters is to equate expectations across conditions and that blinding is just one way to do this.

AG Maybe you’re right, I’m not actually up on this literature. I know that Paul Rosenbaum has written about it.

** MORE ON BLINDNESS: CONSIDERING THE SHANGRI-LA DIET **

AG My knowledge of it is not particularly sophisticated. For your diet and depression studies, there are obvoious stories based on motivation.

I wouldn’t go so far as some people and simply dismiss your results. But the concerns are natural, I think. It’s a little different than the problem with the Nurses study. Here I’m worried about motivation, there the issue was selection.

Although there’s a possible selection problem in your study too, in that the people (including you) doing the Shangri-La Diet might be those who are ready to try something new and lose weight.

SR There are a lot of people who are always ready to try something new and lose weight.

AG Again, this could be tested with a blinded study. For example, half the people get the oil apart from a meal, half get the oil with the meal. Not that this would solve all problems of interpretation. . . .

For example, Caroline thinks that your diet works, but that the reason why it works is that it stops people for snacking for a 2-hour period (before and after the oil) and also focuses people on their snacking.

SR If anyone thinks that — and it is a perfectly reasonable thing to think if you are just starting to learn about it — then they can replace the oil with water and see if they continue to lose.

AG To answer your comment (”there are a lot of people who are always ready to try something new and lose weight”): yes, I remember you saying this before, and this is a big reason I wouldn’t dismiss your results immediately. But, still, people willing to try this wacky new thing might be special (on average). To put it another way, I expect there were similar successes with people trying Scarsdale, Atkins, etc.

SR I’m sure that people who try my diet are unusual early adopter types. I think Atkins has some truth to it — some reasons it would actually work. I don’t know enough about Scarsdale to comment. My theory says that merely changing what you eat (to foods with unfamiliar or at least less familiar flavors) should lower your set point.

AG Sure, but you had another point which was that these were people for whom nothing worked before. I was just using these diets as examples of other things that worked when nothing worked before. It relates to the historical perspective of new diets as things that will work for a few years before burning out. Possibly because the new diets can motivate people.

SR I tend to think they burn out because the new food becomes familiar.

AG I’m not saying that this is necessarily true of your diet–yours might be different–I’m just giving a historical control to give insight as to how there could really be motivational issues.

SR That’s true, research to distinguish my explanation of the burn out and a motivational one could be done but of course hasn’t been.

AG Your story, “they burn out because the new food becomes familiar”, is plausible. It’s also plausible that it’s easier to motivate yourself with a plan that’s new and different.

SR I hope there will be studies of whether the theory behind my diet is correct. These would essentially be studies that test the prediction that familiarity matters. This is a prediction that other theories do not make.

AG Yeah, based on reading the appendix to your book, there’s still some research synthesis that needs to be done (presumably with the help of animal studies).

SR I agree.

BACK TO WEB TRIALS

SR Web trials are relatively early in the research chain and they are relatively practical. In these cases you don’t worry a lot about mechanism, you worry much more about efficacy — is there an effect?

AG Regarding the analysis of web trials, it would be interesting to look at other examples of partially randomized experiments. Rubin and Hill and others worked on a study of school choice where they looked into some of these issues. It was a study that randomized some aspects of which kids went to which schools, but parents had some choices too.

In medicine and also in economics/public-policy, there has been a lot of interest in recent years in trying to get inside this sort of study rather than just relying on the “intent to treat” or explicit randomization.

SR “get inside this sort of study”–what do you mean?

AG: I mean, look at what treatments are actually chosen by the individuals in the study, not just looking at what treatments they were assigned to.

SR Could you sum up why you like the idea of web trials?

AG 1. Lots of data. 2. Motivates people to randomize, to apply the treatment, and to record results. 3. More generally, gets people involved in the project as participants, not just “subjects”

SR Those are good points, thanks.

AG Thank you for giving me the opportunity to think about these things. I’m still struggling with the question, “Are medical experiments too small or too big (in number of subjects)?”. As discussed here.

The Trouble With College

Seth Roberts3 Comments

Yesterday I heard something — a very ordinary bit of info — that neatly summed up the trouble with college. Someone told me about a friend of hers who was a graduate student in English at Berkeley. Her friend taught a small class of freshman and sophomores. He was enthusiastic about what he was teaching, but his students were not. He couldn’t make them enthusiastic, even a little. They just sat there. When I started teaching at Berkeley, I had a similar experience. My first class was introductory psychology. Over the first few months, I came to see that my students, almost all of them, had different interests than me. I thought X and Y were fascinating; they didn’t.

No one is at fault here, of course. It’s perfectly okay that the grad student enthused about something that leaves his students cold. It is perfectly okay that I liked Research X and Y but Research X and Y bored my students. Nothing wrong with any of this — in fact, we need diversity of thought and knowledge, which grows from diversity of interests. We need diversity of thought and knowledge because we have many different problems to solve.

At fault is a system (Berkeley and similar colleges) that fails to value that diversity. (In fact, it doesn’t even notice the diversity, except in a one-dimensional way: how much students resemble their professor.) Even worse, the system tries to reduce diversity of thought because it tries to make students think like their professors. Why should the 20 (or 800) students in one class be forced to learn the same material? The students vary greatly. Forcing all of them to learn the exactly same stuff is like forcing all of them to wear exactly the same clothes. It can be done, especially if rewards and punishments (i.e., grades) are used, but it’s unwise. Just as feeding children a poor diet stunts physical growth, forcing college students to imitate their professors, instead of letting them (or even better, helping them) grow in all directions, stunts intellectual growth.

I wrote about these issues here and gave a related talk about human evolution. Aaron Swartz and I have ideas about a better way, and how to get there, which I will blog about. I will tell a 10-minute story about this as part of the Porchlight story-telling series on March 26 (Monday), 8:00 pm, Cafe du Nord, 2170 Market Street, San Francisco ($12 admission).

A New Way to Quit Smoking?

Seth Roberts12 Comments

A few days ago on the Dean Edell radio show, I’m told, Dean Edell told his listeners that nicotine patches don’t cause any addiction problems; people just don’t get addicted to them. To anyone who has read The Shangri-La Diet this will sound eerily familiar: Dr. William Jacobs, a professor of psychiatry and addiction researcher at the University of Florida, told me that no one gets addicted to unflavored sugar water, although lots of people get addicted to Coke, Pepsi, and other forms of flavored sugar water.

These examples suggest is that it isn’t the drug (sugar, nicotine) that causes addiction, it’s the signal of the drug — the conditioned stimulus (CS), to use animal-learning jargon. No signal, no addiction. In the case of sugar water, it’s very clear: Digestion of calories provides little or no pleasure. Ingestion of sweet-tasting things provides just a little pleasure. Ingestion of a flavor that has been paired with calories many times, such as the flavor of Coke, provides a lot of pleasure. The pattern with nicotine may be similar: Nicotine itself provides little or no pleasure. It is learned signals of nicotine — events repeated followed by nicotine — that can be very pleasant.

The practical application is that you may not need nicotine patches to quit smoking. It may be enough to hold your nose while you smoke. (The nose-clipping that SLD forum readers are familiar with.) When you smoke, the smell may become the CS. With this way of smoking you could have cigarettes whenever you wanted. You’d just come to want them less and less.

Likewise, it may be possible to get rid of an addiction to coffee by holding your nose while you drink it.

Thanks to Carl Willat.

Science in Action: Omega-3 (time of day effect)

Seth Roberts2 Comments

Flaxseed oil seems to have detectable effects within hours. For example, I increased the dose in the evening and my balance was better the next morning. To get some sense of the time course of the effect, I varied the time of day that I took the flaxseed. I usually took it around 10 pm; I tried 10 am instead. I continued to test my balance around 7 am.

Here are the results from an ABA experiment.

Taking 3 tablespoons at 10 am produced better results than taking the same amount at 10 pm. I fit lines with equal slope to both the A (10 pm) and B (10 am) treatments, as the graph shows. The two lines had different intercepts, p < .05.

Although 10 am produces better balance, it produces worse sleep — more evidence that the sleep improvement and the balance improvement are due to different mechanisms. I want both improvements, so I am going to split my dose — half in the morning, half in the evening.

Of course, the fact that time of day of flaxseed oil matters is more reason to think that presence/absence of flaxseed oil matters. It is very hard to explain these results in terms of expectations: I had no reason to expect one time to be better than the other.