How should we innovate? How should we introduce new procedures and devices? We live in a world that is rapidly changing. When I began my career as a surgeon, we made big incisions; we used crude surgical tools, such as needles and thread; and we recorded our results on paper. Who would’ve dreamed that we would be using mini-incisions, stapling devices, sitting at a robotic console instead of at the side of our patient, and dictating into our watches only to have it show up in every computer in the hospital? Yes, modern surgery is moving along at an unbelievable pace due to new instruments, new procedures and new innovations. Surgery as I know it has markedly changed over my almost 50-year career.
Vote for the winner!What are the mechanisms by which innovation occurs, and are there adequate safeguards in place to prevent us from taking our patients down the wrong path? Will safeguards and bureaucracy, in fact, stifle the development of new procedures and devices? In oth er words, how do we continue to maintain the progress of surgery and still prevent our patients from suffering potential injury and complications? This is the subject of this month’s debate. “Curbing our enthusiasm” for innovation of new surgical procedures and devices: Is it a good thing or a bad thing for us to slow down the train of progress?
For this month, I have picked two outstanding surgical scholars who have taken very different approaches to where we have been and where we are going. Hopefully, thinking about this important issue and debating it from both sides will help us solve this dilemma in the future. There is no doubt that we need to continue to innovate and improve our surgical practices, but we must do this in a way that does not harm our patients or us as surgeons. I think you’ll find in this month’s debate some middle ground that may guide us in the future. Again, we look forward to continuing this debate online or on Twitter. This issue is too important for us not to continue the conversation.
Pismo Beach, Calif.
Send comments about this debate or ideas for future debate topics to khorty@mcmahonmed.comA Good Thing: Critical Evaluation Is Necessary Before Widespread Adoption of New Techniques and Technology
Innovation of new technology, medical devices and surgical procedures is absolutely necessary in our field; it’s how we make progress and improve patient care. However, the ecosystem by which innovation is assessed and refined in the medical world is flawed. This has been well described. There are competing agendas resulting in conflicts of interest that must be managed by surgeons, professional societies, industry and regulatory bodies. Furthermore, a new medical device or surgical technique that does not add value to the system by improving the safety and efficacy, or reducing the cost of an operation, is not truly innovative. Yet, it is often ushered in as the next greatest thing by the surgical community before any supporting evidence, leaving me to ask why.
The stages of surgical innovation have been well described as moving from the generation of a hypothesis with an early promising report to being accepted conclusively as a new standard without formal testing. As the life cycle continues and comparative effectiveness data begin to emerge slowly through appropriately designed trials, the procedure or device is often ultimately abandoned (Table).1,2 This inefficient process continues as each new device and surgical procedure begins the same plodding journey, with the overwhelming majority in time not living up to the hype. At the end of this cycle, there are significant profit margins for industry, potential growth in surgical volume and notoriety for some surgeons and hospital systems, and occasional unnecessary harm to patients.
Table. 7 Stages in the Life Cycle of an Innovative Medical Device or Surgical Procedure1,2
The “promising report”—an interesting observation that prompts further, similar case reports.
Professional adoption—widespread use of the procedure by physicians and surgeons.
Public acceptance without formal critical evaluation—third-party payors begin to cover the procedure, but it still usually remains without formal critical evaluation.
The randomized controlled trial—inquiring minds finally decide to evaluate the procedure in a formal study.
Professional denunciation of the procedure—the randomized controlled trial finds the procedure suboptimal.
Erosion of support and discredit—the procedure falls from favor and is replaced, usually with a new procedure at the beginning of its own life cycle, and thus also unevaluated.
Acceptance as a “standard procedure”—the procedure has now become entrenched. Case series accumulate, but formal critical evaluation still does not exist. Powerful forces—both surgical egos and commercial interests—champion the procedure and discourage criticism.
Perhaps the most notable example of the harm that this cycle has caused patients is the surgical treatment of stress urinary incontinence in women. This story should be a learning point for all surgeons, as it brings into play multiple issues that were never properly managed, resulting in significant patient harm and eventually an uproar, leading to significant increases in regulation. Gynecologic surgery saw an explosion of stress urinary incontinence surgery after the development of operation-specific “kits.” The kit was thoughtfully designed to include a comprehensive array of all the supplies necessary to perform a pubovaginal sling. Yet, conspicuously absent was the presentation of any objective evidence to support the safety and efficacy of the procedure, or even the suggestion that such discourse was necessary or desirable. Industry recognized the potential market available and developed elaborate marketing campaigns, investing millions of dollars in surgical key opinion leaders and an economic story that promised increased volume and profits for the hospitals and surgeons. The end of this story is well known to the medical community and, sadly, has resulted in significant long-term pain and suffering for many unsuspecting women.
This tale highlights some glaring flaws in the federally regulated system that manages the introduction of innovative medical devices to the market. Indeed, the use of medical devices in the United States is largely operating under a “let the buyer beware” approach rather than the kind of careful scientific evaluation that the public expects.1 Unlike drugs, which must be demonstrated by clinical trials to be both safe and efficacious prior to widespread release, the majority of new medical devices are routed through the 510(k) pathway and do not carry the same burden of proof. This does not result in FDA approval, but instead only FDA clearance to sell a device—a seemingly minor, but in truth profoundly impactful distinction.
The designation of “FDA approved” conveys that a drug has been thoroughly vetted through a rigorous testing process before coming to market. In contrast, “FDA cleared” is more of a political approval for a company to market a device and results in legal permission, while the ethical dilemma of whether to use the device lies with the physician. In other words, the surgeon is left to determine whether or not they should use the device, arguably in absence of sufficient evidence to support the notion that the device is even safe or efficacious! Regulatory bodies then anticipate that the medical community will demand data prior to implementing these devices, while the medical community presumes that the device would not be available without adequate preclinical information.
Alarmingly, much of the medical community is unaware of the process by which devices are cleared for market, and thus are unlikely to request preclinical information. Further complicating the system is that most devices are cleared based on the device manufacturer providing evidence that the new device is of substantial equivalence to a predicate device, or one that has been previously marketed. Thus, the medical device manufacturer claims that its “innovative” product is in fact so similar to a preceding device on the market that it should not have to undergo substantial preclinical testing and is therefore safe to bring to market. Moreover, how can a substantially equivalent device be truly innovative?
The Institute of Medicine shares a similar concern, concluding that the 510(k) process “lacks the statutory basis to make it a reliable screen of safety and effectiveness … as long as the standard for clearance is substantial equivalence to any previously cleared device.”3 Critics would suggest that FDA approval should be required for any new device to enter the market. Still, applying the same rigorous standard of preclinical evaluations required to obtain true FDA approval for all medical devices would be cost-prohibitive, and would likely stifle medical device innovation, preventing new efficacious devices from making it to the market. And yet, how are we to move forward without strong evidence to support the safety and effectiveness of new devices?
Given that it is impractical to obtain preclinical information from adequately powered randomized controlled trials for every new medical device, an alternative way to handle this issue is to develop robust registries capable of supporting adequate postmarketing surveillance of devices brought to market through the current FDA clearance designation. This would involve collecting meaningful data in real time upon release of these products. These efforts should be required by the FDA and supported by industry, and the surgeon should be ethically bound to prospectively collect their data in national registries when evaluating innovative medical devices and surgical procedures. The end result would be a powerful system by which industry stakeholders are enabled to bring new innovation to market, surgeons are empowered with knowledge of how the devices are performing in order to deliver informed clinical decisions, and the return on investment is the true holy grail of medical care: patient safety. Ideally, this will not require a mandate from the regulatory bodies, and surgeons and industry can collaborate in meaningful ways to reshape the process of innovation in the surgical community with the patient’s best interest in mind.
If the culture of surgical innovation were to change with all stakeholders taking an active role in evaluating safety and efficacy in real time post-market, the system could sustain meaningful innovation in a more practical and harm-free manner. Innovation must continue to occur, and both industry and surgeons, by taking an active role in contributing to and critically evaluating postmarketing data, must be the engines by which this critically important process moves forward.
References
- Am J Obstet Gynecol. 2010;202(1):30.e1-4.
- Milbank Mem Fund Q-Health Soc. 1981;59(3):374-411.
A Bad Thing: However, Curbing One’s Enthusiasm Is a Matter of Perspective. Ask Larry David.
Consider surgical practice if we had waited for proof about inhalation anesthetics. In the 1840s, William Morton anesthetized a variety of animals, which gave him enough confidence—not proof—to anesthetize patients in his office for dental procedures, and eventually gain an invitation for a public demonstration in the Bullfinch Amphitheater of Massachusetts General Hospital. Dr. Morton, however, arrived late to his own demonstration because he had to wait for an instrument maker to complete a new inhaler, without proof it was better.
After this, increased use of ether had plenty of benefits, but also gave rise to some problems. In a quest for better anesthetic agents, Edinburgh obstetrician James Simpson learned about chloroform. “To test this agent, Simpson and his friends inhaled it after dinner at a party in Simpson’s home on the evening of November 4, 1847. They promptly fell unconscious and, when they awoke, were delighted with their success. Within two weeks, Simpson submitted his first account of chloroform’s use to The Lancet.”1 Since then, hundreds of millions of patients have benefited from inhalation anesthetics, yet to this day, their mechanism of action remains a mystery. Imagine if we had waited for proof.
Now let’s analyze the current practice of obtaining this proof—the randomized controlled trial (RCT). Arguably, the most well-designed and executed study in hernia disease was the laparoscopic versus open inguinal hernia repair trial performed at Veterans Affairs hospitals across the United States.2 This study set out to answer the question of which technique was better. So after several years of planning, enrolling patients and analyzing the data, the results were published. And what typically happens after several years of practice? Experiential learning and evolution, because in reality, surgical practice evolves, with or without proof. And,when considering value, we must include cost. For example, the VA hernia trial cost millions of U.S. tax dollars. But it’s worth it, right? In order to definitively answer the question of which hernia repair is best. And that answer was, according to the conclusion of the authors and The New England Journal of Medicine, that open repair was superior. But superior for whom? Because it was a contrived population of men within the VA system, it should be better for those patients. But it wasn’t. Unpublished data presented at the American College of Surgeons meeting in New Orleans, following publication of the results, revealed that some of the centers had 0% recurrence rates for the laparoscopic approach and more than 10% recurrence rates for the open approach. Imagine those patients being offered only the approach deemed to be “superior.” And what defines superior?
In this trial, “superior” was defined as a lower recurrence rate among a group of surgeons operating on some men with inguinal hernias in the VA system. But why do patients have hernia repairs? The reason we fix hernias (or treat any medical condition) is to solve an existing problem or prevent a problem. For hernia repair, that is typically for pain relief and/or prevention of acute incarceration. Therefore, the most meaningful outcomes would be pain and recurrence. In the VA trial, the results are clearly reported for both metrics.2 Chronic pain at two years was 14.3% for patients having an open procedure and 9.8% for those who had a laparoscopic repair. Recurrence rates were 4.9% and 10.1%, respectively. When considering meaningful outcomes together (pain and recurrence), both methods of repair had failure rates that were essentially identical: 19.2% for open and 19.9% for laparoscopic. So for unknown reasons, the most prestigious medical journal in the world only reported recurrence rates. I don’t think this serves patients’ best interests.
But this is not a new problem. The innovative master surgeon, Edoardo Bassini, when implementing an ingenious method of inguinal hernia repair, only reported half of his results as well. His recurrence rates were low, but he failed to mention chronic pain. Even as recently as in the past month, I have unfortunately been told firsthand from a patient about how they revisited their hernia surgeon for pain, only to be told the operation was a success because there was no recurrence, and they would have to live with the pain. Turning a blind eye to patient symptoms because one technique has been proven to be “superior” is not the best path to improving the value of our care for the patient.
Proof can be harmful at the regulatory level as well, due to clinical “proof” that a device is substantially equivalent to a similar device that is already legally marketed. When the FDA clears a device for sale and marketing, such as a hernia mesh, they have pored over data supplied by the manufacturer to ensure “safety and efficacy.” This works most of the time, but overreliance on data sometimes can get in the way of common sense.
Proof can be harmful at the regulatory level as well, due to clinical “proof” that a device is substantially equivalent to a similar device that is already legally marketed. When the FDA clears a device for sale and marketing, such as a hernia mesh, they have pored over data supplied by the manufacturer to ensure “safety and efficacy.” This works most of the time, but overreliance on data sometimes can get in the way of common sense.
Ethicon introduced an ultra-lightweight coated polypropylene mesh to the market after receiving FDA clearance. The mesh was deemed substantially equivalent to existing devices, and rapidly became the market leader in terms of sales. The problem is that hernias are not all substantially equivalent to one another, and this mesh was marketed for use in all hernias. Even if the hernia required a stronger mesh, surgeons blindly accepted the approval from the FDA as “proof” that the new mesh was good enough. What subsequently happened was outcomes data from European databases documented higher failure rates due to the weak mesh. The mesh was then recalled from the market (after thousands of implants), sparking a mesh hysteria manifesting in a seemingly endless series of hernia mesh lawsuit commercials, in addition to anxiety among patients and their surgeons regarding preoperative discussions about their hernia repair.
So it’s not an either–or situation; it’s more like “and–both.” We do need safety data, but proof that the intervention or device is superior to existing practice prior to use in humans will slow innovation and deny patients access to cutting-edge technology. And this will ultimately hurt more people than it helps.
References
- Best Pract Res Clin Anaesthesiol. 2005;19(3):323-330.
- N Engl J Med. 2004;350(18):1819-1827.
Dr. Rosen’s Rebuttal to Dr. Earle
Dr. Earle brings up some great points in his argument, and at the end of reading it, I think he and I agree on many issues. Should all new innovations require definitive “proof”? No. However, if we accept that system, there needs to be a transparent process that allows patients to feel that these innovations are being carefully assessed on an ongoing basis. Furthermore, that system should be able to identify and address safety signals as they become apparent, in a timely fashion. Currently, that postmarketing surveillance system does not exist, and we are essentially walking on a tightrope with no safety net. The vast majority of times, innovations will be safe and no one will fall and get injured. But when a safety event occurs, the outcomes are typically dramatic, and the system over-corrects. This tends to result in public outcry, and the response is often overreaching and has significant legal and financial ramifications.
It is also interesting that he brings up an example of a physician who actually tested the drug on himself prior to clinical use to provide “proof.” Would physicians be willing to place a prosthetic mesh in themselves first, before widespread use? I doubt it and don’t think that is a reasonable solution. Physicians must also realize how little data it takes for us to start doing something “new,” while the burden of proof to abandon a procedure or device we think is working can be overwhelming. While I would disagree with Dr. Earle’s assumption that all FDA-cleared devices have had extensive clinical data reviewed prior to clearance, I do believe that the system promotes safe innovation in most cases. However, in his example of the recent mesh recall, without the robust registry system in Europe, we would not have identified that safety signal and more patients would have experienced undue harm.
Dr. Earle’s Rebuttal to Dr. Rosen
Dr. Rosen has made some excellent points, and he correctly pointed out the flawed system. His reference to the seven-stage life cycle of surgical innovation is interesting, but also flawed.
Consider laparoscopic cholecystectomy. While in medical school, an attending surgeon mentioned it was being developed in France, while demonstrating how difficult proper operative exposure was gained through our right subcostal incision. Around that same time, a German surgeon was convicted of manslaughter after a complication from a laparoscopic cholecystectomy. Then I learned alongside my trainers as they converted to laparoscopy, and more recently have had the honor of assisting two entire countries convert to a laparoscopic approach for cholecystectomy. The benefits of this innovation, which fortunately did not follow the seven-stage life cycle, are well known.
One of my mentors, Dr. Felicien M. Steichen, is one of the fathers of surgical stapling. During an interview with him, we began talking about cases, as surgeons do. I referred to the standard side-to-side, functional end-to-end stapled small bowel anastomosis. He immediately raised his hand with his index finger extended to pause my story. With a downward gaze, he raised his eyes toward mine as he stood, without changing the position of his head, and cracked a smile. He made his way to a file cabinet and pulled out his original article describing this technique,1 and told me the story of his innovation. And during the production of his memorial video,2 Leon Hirsch, the founder of the U.S. Surgical Corp. pondered the difficulties of getting innovation to the market in today’s regulatory environment. Fortunately, this technique and device have not followed the seven-stage life cycle.
My colleague is also correct that the FDA 510(k) pathway is flawed. Having been recently employed by the FDA as a medical officer, I can attest the agency recognizes that fact, and increasingly operates in a manner that favors safety and innovation over bureaucratic check boxes. In December 2016, during a time of great political division, Congress passed the 21st Century Cures Act, designed “to accelerate the discovery, development, and delivery of 21st century cures, and for other purposes.”3 In January 2018, the FDA published its strategic priorities. The agency stated they will be successful if “by December 31, 2020, more than 50% of manufacturers of novel technologies for the U.S. market intend to bring their devices to the U.S. first or in parallel with other major markets.”4
Interpretation of the data driving Dr. Rosen’s life cycle may also be flawed. The recalled transvaginal mesh, for example, may have had poor results due to the technique, not the device itself. A recent review stated that “currently lack of available evidence on other single-incision slings prevents any meaningful conclusions.”5 And the data show most women benefited. Indeed, the situation is far more complex than the presence or absence of the device itself on the market, emphasizing that a recall is not in the best interest of the majority of women with stress urinary incontinence.
Finally, Dr. Rosen’s comments regarding registries are on the right track. The concept of real-world data is necessary, but all current registries are limited by sample size and potentially biased. The owners of the registries can have conflicts of interest, particularly when funded solely by industry or insurance companies. To date, however, they currently represent the best real-world data there are, although access is limited. But we already have all of the real-world data in our collective electronic health record (EHR); we just don’t have access to the fragmented databases. The holy grail for improvement of health care is a single or integrated EHR. Because we cannot control any complex system, we must collect all the data, analyze all the data, and apply all of the analysis. Wash, rinse, repeat—forever. This continuous quality improvement uses feedback from our own outcomes and will improve quality and reduce cost, but it will take an act of Congress. EHR integration should be something for which we all advocate. This is the real world, and in the real world, we are all patients, too.
References
- Surgery. 1968;64(5):948-953.
- Tribute to Felicien Steichen. https://youtu.be/ F1AFS7LufVg. Accessed August 17, 2018.
- U.S. Congress. www.congress.gov/ 114/ bills/ hr34/ BILLS-114hr34enr.pdf. Accessed August 17, 2018.
- FDA Centers for Devices and Radiological Health. 2018-2020 Strategic Priorities. www.fda.gov/ downloads/ AboutFDA/ CentersOffices/ OfficeofMedicalProductsandTobacco/ CDRH/ CDRHVisionandMission/ UCM592693.pdf. Accessed August 17, 2018.
- Cochrane Database Syst Rev. 2017;7:CD008709.