Evaluating Treatment Effectiveness

For a therapy to be considered effective, it needs to show benefits across multiple studies conducted on a range of samples. Once many studies have been conducted on a specific form of treatment, researchers often conduct a meta-analysis. Meta-analysis is a statistical technique used to combine and analyze data from multiple studies on the same subject. For example, a single study may examine the usefulness of a single drug for the treatment of mild depression in people between 25 and 65 years old. But a meta-analysis might statistically combine the results of 100 depression-treatment studies. Meta-analyses can be used to explore questions that could not be answered with a single study. Depending on the studies included, a meta-analysis could explore the benefits of one drug compared to another or of one drug compared to cognitive-behavioral therapy (CBT). Meta-analyses can also be used to explore whether a treatment works better for mild or severe symptoms or for different age groups. The statistical techniques used in meta-analysis produce a number called the effect size. The effect size indicates the magnitude of the difference between groups. For example, a large effect size for a comparison of CBT to a waitlist condition (delayed or no treatment) would indicate that CBT was meaningfully better at reducing symptoms than doing nothing.

Meta-analyses finding small effect sizes have called the benefits of antidepressants into question. Particularly for people with mild to moderate symptoms, there seems to be little clinical difference between giving a person an antidepressant or a placebo (a fake pill).

A panoramic study is a combination of meta-analyses. Psychiatrist Stefan Leucht and colleagues have conducted several panoramic studies. In a comparison of psychiatric drugs to medical drugs, they found that psychiatric drugs typically beat placebo by about the same amount as drugs for medical conditions. In a review of 60 years of data on the use of antipsychotics for treating schizophrenia, they found that twice as many patients improved with drugs compared to placebos. Even so, only a minority of patients experienced a good response. Many continued to have significant symptoms.

Many meta-analyses show that psychotherapy is effective. Across disorders about 80% of people improve with treatment, and about 5–10% of clients suffer adverse effects. However, some researchers have raised concerns about the research base. They note that negative results are less likely to be published and also raise concerns that researchers are often biased in favor of a particular therapeutic approach. Research suggests that most forms of therapy are better than no therapy. Even treatments intended as control conditions, such as simply listening to a person talk, are better than nothing. This is likely because there are many nonspecific factors that contribute to positive outcomes. These include empathy, warmth, instilling hope, and building a genuine connection.

However, not all forms of therapy are equally effective. Many conditions require more than just a warm and connected therapeutic relationship. In general, behavioral and cognitive-behavioral therapies show the strongest effect sizes. This is especially true for anxiety disorders, disruptive behavior problems, autism, and sexual dysfunction. The most effective treatments for depression focus either on building strong social connections, engaging in positive activities, or managing negative thoughts. For serious mental illnesses such as schizophrenia and bipolar disorder, medications show the clearest benefit. Talk therapy can also help by improving medication compliance and helping people to build stronger coping skills and relationships.