THE question of whether we are cured by a medicine or treatment can be a surprisingly slippery one. Most people instinctually trust their internal experiences and their ability to link cause and effect. If they took a pill and then soon felt better they assume that the pill must have been the cause of their improvement. Unfortunately, the placebo effect (which causes people feel better entirely due to their expectations of feeling better), as well as a variety of other related effects expose the subtle but serious flaws in this reasoning. Matters get even worse when it is not even our own experiences that we are relying on, but rather the testimonials of others. Consider the following points:
(1) When people are treated with sugar pills or other inert substances but are told they are taking genuine medicine they frequently report that their symptoms improve or disappear. The size of this effect depends on which disorder is being treated, and perhaps also on psychological factors such as how many sugar pills were taken, whether the pills appeared to be of a known brand†, and even the color of the pills† †. It is debated how strong the true placebo effect really is and how much of people’s reported improvement is due to other factors (such as patients misreporting their symptoms, experimenter bias, the subjectivity of symptoms, reversion to the mean, self-termination of disease, natural symptom fluctuations, etc.)†, but none the less, a large number of studies have made it abundantly clear that bogus treatments frequently cause people to report feeling better † † † † † † †. If a friend says that a medicine made them feel better, that response is consistent with what people often say when they receive no medicine at all.
(2) Many common conditions, such as the cold and flu, are self-terminating, meaning that you will typically get better from them no matter what treatment is taken. Suppose, for example, that I give you powdered asparagus as an experimental (and bogus) treatment for the flu. If you happen to feel better the next morning and have faith in the curative powers of that green vegetable or my (non-existent) healing powers, you may attribute your improved condition to the treatment even if you would have felt just as well had you taken nothing at all. Different people can have very different responses to the same disease, and whereas one person will get sick for only a very short period, another person’s symptoms will linger for much longer. Hence, it can be very difficult to tell whether your body has been cured by a treatment or whether your body has simply healed itself.
(3) Our lives tend to fluctuate so much that it can be extremely challenging to sort out which of the changes in our behavior or bodies are responsible for the changes in the way we feel. Did I feel better because I started sleeping more, or because of that new medicine I’ve been taking, or because I’ve lost weight, or because I am less stressed, or because my hormone levels have changed, or simply because of natural fluctuations in my illness? Questions like these often cannot be answered with confidence.
(4) Many symptoms that we experience, such as pain, stress and fatigue are subjective in nature, and therefore are difficult to track, remember accurately, and explain. Is the pain I now experience really less severe than it used to be, or is it just a different sort of pain? Am I really feeling better than I was three months ago, or have I simply forgotten what I felt like back then? Is my illness cured, or are my symptoms just bothering me less this week?
(5) Diagnoses are often tentative. When you go to the doctor with flu like symptoms, she will probably tell you that you have the flu. However, there are many different strains of the flu, each with their own unique characteristics, and there are many diseases that cause flu-like symptoms. She tells you that you have the flu because that seems the most likely to her given the evidence available, but that does not mean that she is definitely right. While medical technology today has some excellent diagnostic tools, in many cases they are not used for what appear to be minor ailments (due to cost considerations), and some diseases, such as Lyme disease, can be very tricky to diagnose†. Furthermore, even when effective tests do exist, they never have 100% accuracy. Unfortunately, if you do not know with complete confidence what disease a person is suffering from, you can never know with complete confidence whether the treatment they were given was effective. Sometimes self-terminating illnesses will be mistaken for treatable, but non-self terminating ones, leading a patient to the mistaken impression that they have been cured by their medication when in fact it was useless.
To illustrate the problem of misdiagnosis, consider the case of a man who tests positive on the PSA test, which is considered to be “the most effective test currently available for the early detection of prostate cancer†.” Upon hearing this frightening news, let’s assume our man (who prefers an “alternative and holistic” approach to medicine) seeks out his neighborhood naturopath who grinds some roots up into a paste for him. After taking this concoction for a year, the man goes back to his practitioner only to discover that his prostate cancer is “cured”. It would not be surprising if we soon find this “lucky” fellow touting his “miraculous healing” at dinner parties or selling the amazing curative root paste via informercials or on the internet. But by far the most likely explanation for the man’s healthy state is that he never had prostate cancer in the first place, as about 7 out of 10 people who have positive PSA test results do not have the disease†. When we consider scenarios like this one, it is not difficult to see why there should be a large number of well meaning individuals who tout suspicious and likely worthless miracle treatments cures. There will always be those individuals who seemed to recover from an incurable disease upon taking sham medicine simply due to the fact that they never had the illness in the first place.
(6) Amazingly, incredibly, astoundingly coincidental and unlucky things do happen. If I were to roll a ten sided die ten times, it is possible that it land on each digit of my telephone number in order. The odds of this occurring are ten billion to one against, but it definitely could happen. In fact, with a world population of over five billion people†, with each of these people witnessing many different events everyday, we should expect occurrences at this enormous level of coincidence and absurdity to happen on a daily basis. With the rate at which remarkable stories travels in our connected world, it is not unlikely that information about at least a few of these true but seemingly impossible events will eventually find its way to us. So what are you to think if your friend’s typically deadly cancer suddenly disappears? If a misdiagnosis can be ruled out, then after taking the time to rejoice in your friend’s wonderful good fortune you can be further awed by considering the shockingly unlikely event that you had the opportunity to witness.
(7) Anyone who has spent much time around humans should know that their testimony is not always without error. We exaggerate, we misremember, we misinterpret events, we draw false conclusions, we extrapolate from tiny amounts of data, sometimes we even lie. When a friend tells you that his headaches were cured by taking some vitamins, does he really know that to be the case? Perhaps his headaches only felt better after taking the pills but were not actually cured, and he simply is trying to tell a more compelling story. Or perhaps he misinterpreted the events and the headaches just happened to get better for some other reason. Who knows? Maybe he’s even got a case of them in the car that he’s “willing” to sell you.
(8) People are much more likely to talk about the medicines that cured their ills than those that they took for a while with no results. Hence, when a friend recommends a miracle diet to us that “worked for her”, it is good to ask ourselves how many other people we know who tried the same diet with little result. Even if we could poll all of our friends who have tried the diet and ask them what they thought of it, such an informal study would lack the rigorous controls, careful population selection, and statistical analysis used in real research studies that would allow us to draw firm conclusions from it.
(9) For chronic or long term conditions it is not uncommon for people to only seek out treatment when their symptoms are “acting up”, or are particularly worse than usual. Of course, that means that no matter what treatment is given their symptoms are likely to improve (and sometimes rapidly) by falling back down to their usual or average level of discomfort. This phenomenon is sometimes referred to as “reversion to the mean”. Unfortunately, a patient may misinterpret this return to their usual level as an improvement caused by whatever treatment they were given.
Hopefully, after this discussion, it will be clear that when someone says “I was cured”, what they usually should have said is “I feel better”, since linking cause to effect when it comes to medicine can be such a difficult task. When someone tells you that they know a treatment works because it worked for them, it may be helpful to remind yourself of all the ways mentioned above that confusion can occur, and be cautiously skeptical until you can put your hands on much stronger evidence of the treatments efficacy. For all of the reasons above, it is remarkably easy to find someone who believes they have benefited or claims to have benefited from a particular treatment, and hence testimonials are extremely weak forms of evidence. Any website that attempts to use testimonials to sell treatments for medical problems should be viewed with extreme suspicion!
The only way to know with much confidence that a treatment is reliably effective for a particular disorder is to try it on a large group of people (preferably hundreds or even thousands). To rule out the possibility of all of the effects mentioned above it is absolutely crucial that the study be “controlled” by giving half of the population (chosen at random) the real treatment under consideration, and giving the other half a placebo or previously established standard treatment. Since a patient’s expectations can effect their reported symptoms, the study will also needed to be “blind” if possible so that no patient knows which of the treatments he is receiving. Better yet, the study should be “double-blind” so that even the researchers and those administering the treatment do not know which treatment is given to each test subject. This helps prevent a subject’s reported symptoms from being influenced by the expectations of those administering treatment, and helps prevent researchers from accidently injecting their biases into the data. Finally, the results of the study should be analyzed by a statistician who can then determine how sure one can be that the treatment under consideration is truly better than a placebo or other standard treatments. Ideally, even the statistician should not know which treatment is which (this is sometimes known as a “triple-blind” experiment) so that her preconceived notions cannot effect her analysis. At last, the results should be published in an established peer reviewed journal regardless of whether the results were negative or positive, so that others can examine the study for methodological flaws, and so that other researchers can easily see the conclusions of all studies relating to a given subject. All too often negative results are never published, which can greatly mislead other researchers about the true effectiveness of treatments. Sometimes an ineffectual treatment will be lent a false air of legitimacy when researchers study it using small, uncontrolled, unrandomized and unblinded trials, and then publishing only those result that show the treatment is effective while trashing the negative ones. Hence, to the untrained observer it may seem as though the medicine has been “scientifically proven” to work, when in fact nothing has been demonstrated except that badly designed studies lead people to false conclusions. Tactics such as these may explain the continued existence of some useless forms of alternative medicine that are popular today.
Unfortunately it is quite common (especially in the area of alternative medicine) that the rigorous protocols just mentioned are neglected or glossed over. While it is certainly the case that some treatments which have not yet been tested in large, randomized, controlled, blinded, peer reviewed studies do in fact work, there is no substantial reason for you to believe that they do! Eventually someone will get around to doing the tests, and until that time your health is very likely going to be a lot better off if you stick to only those treatments that have been carefully and painstakingly demonstrated to be effective. Even large and well designed studies occasionally come to false conclusions due to the random nature of testing, human error and the complexity of biological systems, but the evidence derived from good studies should be hundreds or thousands of times more convincing than mere anecdotes or personal experience. Since humans do often make mistakes, and occasionally corrupt studies by purposefully or subconsciously injecting their own biases into the data, the very best thing is to wait until multiple, independent studies have been performed by different researchers that come to the same conclusion. Of course, extra caution is called for when dealing with those who have a monetary interest in the outcome of their own conclusions. For example, we should be wary if a drug has been studied only by the doctor who invented it, or if The American Journal of Homeopathic Medicine claims that homeopathy should, in many cases, replace the use of conventional medicine.