By Bruce Ramshaw, MD
In September 2010, a 44-year-old academic superstar was named dean of the Tilburg School of Social and Behavioral Sciences faculty at Tilburg University in Tilburg, the Netherlands. Just one year earlier, this acclaimed social psychology researcher, Diederik Stapel, received the Career Trajectory Award from the Society of Experimental Social Psychology. Stapel moved to Tilburg University in 2006 and started TiBER, the Tilburg Institute for Behavioral Economics Research. By the pinnacle of his career, Stapel had authored and co-authored dozens of papers, some published in the most prominent journals, such as Science. The problem was that Diederik Stapel was a fraud. For more than a decade, Stapel made up data for his studies, regularly hoodwinking his co-authors, colleagues and students alike. Why would a recognized brilliant student and young researcher do this? He was clearly beyond capable of producing valuable scientific research. Why would he risk so much when he had the ability to do the work honestly?
Stapel did genuinely want to publish honest research and he did attempt to present the complexity of multiple variables interacting to produce varied and unpredictable results. In an interview in an article in The New York Times, Stapel gives reasons explaining his actions that help to also explain many of the other fraudulent activities that we see in published research and in our world in general.
From the Times article:
In his early years of research—when he supposedly collected real experimental data—Stapel wrote papers laying out complicated and messy relationships between multiple variables. He soon realized that journal editors preferred simplicity. “They are actually telling you: ‘Leave out this stuff. Make it simpler.’” So, Stapel decided it would be better for his career to make the results of his studies simple to understand. He chose to make things up because that is what the editors, and presumably the journal readers, wanted to read.
Also from the Times article:
[Stapel] insisted that he loved social psychology but had been frustrated by the messiness of experimental data, which rarely led to clear conclusions. His lifelong obsession with elegance and order, he said, led him to concoct sexy results that journals found attractive. “It was a quest for aesthetics, for beauty, instead of truth,” he said.
Although blatant fraud, as in the case of Diederik Stapel, does exist, it is not very common. A much more common problem in medical research is that the simplistic conclusions of published studies do not completely make sense when tested in the real world of patient care. Our traditional clinical research methods seek to prove or disprove a hypothesis to produce generalizable medical knowledge: that is, scientific medical truths that will apply to most (or to average) patients. Complexity science shows how incomplete this kind of thinking is when applied to the real (complex) world of patient care. Patients bring variability into the process and local variables make processes different in different clinical settings, even when the same disease is being treated with the same test or treatment.
This simplistic thinking is easy to identify in our world. Anytime you hear a direct cause-and-effect story in the news, this simplistic thinking is at play. For example, if the stock market goes up due to a favorable jobs report, you can be sure that the real factors that contributed to the effect of a stock market increase are much more complex than that simple report. Yet it seems that our mass media reporters suffer from the same simplistic thinking as scientific journal editors when reporting on things like movements in the stock market. In the financial market, a simplistic understanding might only hurt someone financially. In health care, a simplistic understanding will potentially cause financial, physical and even emotional harm as well as waste in our health care system.
For even the most beneficial tests and treatments in health care, there still will be a group of patients who suffer harm and a much larger group who will have no benefit (waste) from a test or treatment. Until now, such unnecessary harm and waste generally has been acceptable within our health care system. We have overlooked the fact that even the most cost-effective and beneficial treatment strategies can have complex outcomes that lead to harm and waste in various ways.
I’ll use vaccines as an example. Most people accept that vaccines have been a valuable treatment strategy for a number of diseases, especially for the most vulnerable in our society, including children. It is unconscionable to fail to support a treatment strategy that can eradicate a devastating disease such as polio. And the work of innovators, such as Jonas Salk; pharmaceutical companies; and public health organizations have saved millions of people from contracting a variety of diseases that would cause pain, suffering, disability and death. The Bill and Melinda Gates Foundation, with its multiple interests, has as a primary focus the funding of vaccination programs in impoverished areas throughout the world.
But like all other things we do in health care, in addition to the good that they do, vaccines can cause harm and be wasteful. Each specific vaccine results in its own variable proportion of benefit, waste and harm. But, clearly every vaccine does result in benefit, waste and harm. Despite the real and potential benefits of vaccines, it is clearly wasteful to vaccinate a person, child or adult, who would not contract the disease that is targeted for prevention by the vaccine. And there is a group (or subpopulation) of people who will be harmed by the act of receiving a vaccination. This is where it gets more complex. The simplistic thinking that vaccines are clearly good and so everyone should be vaccinated does not make sense for those children who are harmed and for the parents who see their children harmed by a treatment that helps millions of other people. It is very difficult to convince the mother of a child who suffers or dies as a result of receiving a vaccine that vaccinating everyone is justified solely on the basis of the numbers of other children and parents who are helped.
It is important to note that I am not talking about the sensationalized “vaccines can cause autism” debate. The decisions from multiple legal cases have not supported that theory and some of the organizations developed to help vet that theory have been disbanded. However, these same vaccines that have not been shown to cause autism do cause harm and death. Some children with autism and other complications from receiving vaccinations have been awarded compensation because of the recognition that vaccines cause harm and death. On Oct. 1, 1988, the National Childhood Vaccine Injury Act of 1986 (Public Law 99-660) created the National Vaccine Injury Compensation Program (VICP). There are numerous rules and restrictions, but one thing is clear: Vaccines cause harm and death. By 2010, nearly $2 billion had been awarded to victims of harm from vaccines. The fund is created by an excise tax levied on the producers of vaccines, at $0.25 per vaccine. This must seem a welcome tradeoff to minimize the product liability potential if this program did not exist, especially because the maximum compensation is $250,000 regardless of the degree of harm or suffering, including death.
... To Thoughtful, Reflective Curiosity
Using a paternalistic view of health care makes this complex issue even more complex. To push a treatment on a very diverse society, highlighting the benefits of a treatment and not transparently disclosing the potential harm can lead to unintended harm and consequences that will paradoxically prevent achieving the goal of 100% compliance for vaccination.
I was interviewing a very bright and competitive applicant for our surgery residency program recently and she was talking about some of the challenges she experienced caring for patients in an urban setting (where she is currently in medical school). She is from a more rural area and she was expressing frustration with the lower socioeconomic class of people who were less educated than she was used to. The example she gave to validate her frustration was a group of parents who refused vaccinations for their children. I asked her what accountability we had as health care professionals who did not define the harm vaccines can cause and did not disclose that there is a fund that has paid out almost $2 billion for those who have been harmed by vaccines to these parents. We no longer live in the dark ages (pre-Internet)—people do find this information and if we deny or are not well informed about the harm and waste our recommendations for medical care can cause, then we can become the bad guy, as if we are intentionally hiding information.
We are a major part of the problem when we do not understand the complexity of the tests and treatments we prescribe and recommend. This very bright medical student had no knowledge of the VICP and only knew about potential vaccine harm through the mass media presentation of the autism–vaccine theory. I do not blame her at all. I blame us—the seasoned medical educators who are committed to high-quality patient care, education and research. We need to evolve beyond our simplistic understanding of the results and application of medical research and apply a much more complete understanding of our world.
In general surgery, we have numerous examples of how simplistic interpretations of medical research can cause harm and waste. To finish this dialogue about why we need to evolve our understanding in health care, I will present the complex research surrounding the use of screening mammography. I am intimately familiar with the use of mammography for the detection of early-stage breast cancer because one of the pioneers in this effort was my senior partner in my surgical practice for eight years.
A. Hamblin Letton was the president of the American Cancer Society and was present at the signing of the National Cancer Act with Richard Nixon in 1974. He truly believed that if we could get all women to have mammograms, we would find and cure breast cancer before it could harm these women. It was a very noble effort that I completely believed during my early surgical career. Unfortunately, like all other tests and treatments in health care, this is a much too simplistic understanding of reality. This simple screening test has caused immense harm and waste for women and our system, and unfortunately, those women who have been harmed and those who received mammograms unnecessarily are unknown and unknowable with our methods of medical research. If a respected organization recommends that a woman should have a screening mammogram if she is between the ages of 40 and 69 and the primary care physician follows that recommendation for every woman, then many of these women will be harmed by getting a mammogram and most of these women will be put through an unnecessary test.
For essentially every test and treatment we have in health care, there are basically three subpopulations of patients who undergo a test or receive a treatment. First, there is a group that benefits from the test or treatment, but there is also a group that does not benefit (this is waste in our system), and finally, there is a group of people who are harmed by the test or treatment (directly or indirectly). Until now, our simplistic thinking has allowed us to rationalize that the waste and harm was just a necessary evil to help those patients who benefit from a test or treatment. Who could argue that a few unnecessary mammograms are justified to save a woman’s life? But complexity science argues, and the data from the use of vaccines and more than 30 years of screening mammography have shown, that it is not so simple and we are perpetrating a degree of waste and harm in patient care that is not sustainable and not ethical.
To understand how to define the subpopulations that are harmed and those who do not benefit (causing waste), a more complete understanding of research and data analysis is required. In Part 2 of “Understanding,” I will describe complex systems data analytics, or how to understand data from real patient care—nonlinear and messy, but real.
Diederik Stapel was a fraud, but he is not a villain. The villain in our world is not a person or an organization. The villain is our lack of understanding of complexity. Stapel’s desire to seek success by accommodating the desire to read simple results of complex biologic processes is the fault of n o one individual but the fault of all of us who participate in the application of biological sciences. That Diederik Stapel did this to gain a meteoric rise in academic stature is as much a reflection of our lack of understanding as it is his personal character flaw. When we gain a more complete understanding of health care and our world, we will not only not allow simple-minded efforts like that of Diederik Stapel to achieve undeserved rewards, but we will also begin to address the waste and harm that is caused every day in our system that results from a much too simplistic understanding of how we care for patients and how we try to improve patient outcomes.
Dr. Ramshaw is chairman and chief medical officer, Transformative Care Institute (nonprofit) and Surgical Momentum LLC (for profit), and co-director, Advanced Hernia Solutions, Daytona Beach, Fla.