back to top
Monday, 2 December, 2024
HomeArtificial Intelligence (AI)Regulatory challenges with rapid rise of AI technology

Regulatory challenges with rapid rise of AI technology

Medical device technology is now deeply entrenched in many patients’ healthcare and can have an extraordinary impact on their lives, with artificial intelligence (AI) programs that can scan MRIs for cancer, Apple AirPods serving as hearing aids, and devices that decode the electrical blips of the brain to restore speech to those who have lost it.

Yet as advancements become more tangible, reports The New York Times, regulation of the devices has commanded increasing attention at the US Food and Drug Administration (FDA), and the use of AI in medicine and brain implants are among the issues and products that a new head of the FDA’s device division will need to address when she assumes the role.

Dr Michelle Tarver, a 15-year-veteran of the agency, is stepping into that spotlight at a critical time, taking over from Dr Jeffrey Shuren, who forged deep ties with the device industry, escalated the pace of approvals, and made the agency more approachable to companies.

Some of those device makers were represented by Shuren’s wife and her law firm, posing ethical conflicts for him that continue to draw scrutiny.

Congressional lawmakers and consumer advocates have also become increasingly concerned about the device industry’s influence over the sprawling division, which has a budget of about $790m and a staff of 2 500.

Device safety and standards for agency approvals of products as intimate as heart valves or neural implants will be at the forefront of the division’s mission in the coming years.

Among the issues Tarver will encounter:

Brains, computers and Elon Musk

Few devices will require such intense oversight as one of the most breathtaking technologies in development: brain-computer interfaces that dip into the surface layers of the brain to decode its electrical noise – and return function to people who have lost it.

Researchers from a number of teams have demonstrated the capability to restore the voice and speech of a man with ALS, to enable another paralysed man to walk, and to help a man who is paralysed below the neck to play computer games by simply thinking about steering left or right.

The medical device division is playing a crucial role in this field by authorising and overseeing trials that evaluate the devices’ safety and effectiveness and, at some point in the future, deciding whether they can be sold.

Perhaps no company developing a device is more high-profile than Elon Musk’s company, Neuralink. It is developing the brain-computer device that enabled an Arizona man to play video games with his mind and is also beginning work on a device Musk hopes could restore vision.

Neuralink has already received the green light from the agency to implant its device, which is inserted in a quarter-width hole bored into the skull, in a second patient.

Depending on the outcome of the presidential election, Musk could gain considerable sway across several federal agencies overseeing his businesses, including Tesla, SpaceX and presumably Neuralink, which could give him leverage over competitors.

The weight of industry influence

In two years, Tarver is expected to take the lead in the next set of high-stakes negotiations that determine how the FDA spends billions of dollars collected from the drug and device industries that the agency regulates.

The negotiations have grown in importance to the FDA, with industry funds now providing $362m –or nearly half of the device division’s budget of $790m – and an overall payment that amounts to nearly half of the agency’s annual budget of $7.2bn.

The process is akin to the Olympics of policymaking for the FDA, culminating in agreements that must be passed by Congress to keep the agency running. The funds support the hiring of hundreds of staff assigned to maintain the brisk pace of product reviews.

The arrangement has its supporters, who note that the money allows the FDA to be competitive in hiring scientists who can keep up with the rapid flow of innovation in biotechnology and other fields.

But it has also drawn criticism over concerns that it puts the FDA to work for largely for-profit industries and compromises the agency’s efforts to protect public health.

Lingering ethics issues

Those concerns loom a bit larger as Tarver steps into the post of outgoing device division director Shuren. He has overseen and taken part in a negotiation process with a legal client of his wife’s often at the bargaining table, The New York Times found.

An investigation found that he failed to follow agency ethics rules in some instances when his work overlapped with that of his wife, Allison Shuren, a prominent lawyer. The findings prompted lawmakers to seek a review by the inspector general of the Department of Health and Human Services.

One of Ms Shuren’s clients has been Alcon, a producer of medical devices, including lenses implanted in the eye and lasers used in eye surgery.

During user-fee meetings in 2021, Alcon executives negotiated with the FDA on behalf of two medical device trade groups representing hundreds of companies. It was the only company that sent two representatives, even though it is far smaller than some others, like Medtronic and Johnson & Johnson, that were at the table.

Dr Shuren negotiated in person with Alcon and other companies in 2016, agency records show. After each cycle of talks, he presented the agreement to Congress, according to agency transcripts of his testimony.

Federal ethics laws bar officials from working on government matters where a spouse has a financial interest that affects one company or a discrete group. The FDA has said that Dr Shuren has “not participated in matters specific to Alcon”.

Asked about whether the agency had concerns about the potential for ethical and financial conflicts given his involvement in the talks, the agency declined to comment. Neither Shuren nor his wife responded to requests for comment.

Agreements reached in the deal-making in recent years include an FDA commitment to decide approvals on 95% of low- to moderate-risk devices within 90 days. Another agreement led to a Third Party Review programme that allows outside companies to make initial product device approval decisions that are finalised by the agency.

The ballooning field of AI

The rapid clip of product authorisations has brought the division under scrutiny in the most traditional quarters of medicine and in the most advanced.

Harvard researchers recently reviewed dozens of cardiology device recalls and found that the FDA had deemed many of the devices to be of moderate risk, though they turned out to be deadly.

An editorial by Dr Ezekiel Emmanuel, a former federal health official and vice provost at the University of Pennsylvania, called on the FDA to place safety over speed.

The FDA said it disagreed with an assertion in the study that devices similar to those already marketed need to be thoroughly tested in people.

Doctors and researchers vetting agency-cleared AI programmes have also found the agency’s review records lacking. As they consider using such tools in patient care, a lot of answers they seek about how the programmes work are nowhere to be found in agency approval records.

Many of those programmes are considered low or moderate risk, and hundreds have been authorised under the agency’s 510(K) programme, in which products are typically authorised in 90 days. They include software programs meant to spot cancers and other problems on MRIs, CT scans and other images.

Stanford researchers published a study recently noting that a vast majority – 96% of nearly 700 – of AI programs authorised by the FDA had no information about race or ethnicity, “exacerbating the risk of algorithmic bias and health disparity”.

The agency said the publicly released summaries criticised in the study were merely brief descriptions that did not reflect the extent of staff reviews that can amount to thousands of pages about the software programs.

Researchers from Mass General Brigham and elsewhere published a report concluding that information from the FDA about the performance of certain programs was too sparse to justify using in medical practice.

Still, the promise of AI in healthcare has generated sky-high interest, and the FDA has discussed its use in drug development and employing it internally to catch “cheating” in product applications, said Dr Robert Califf, the agency’s commissioner, last month.

Califf acknowledged the agency’s limitations in regulating the vast reach of AI programs, including how they function when they are broadly deployed. Evaluating the scope of these programs extends far beyond the agency, he said, and if the FDA were to “keep eye on 100% of it, we would need an agency two to three times bigger than it currently is”.

 

Harvard study – Most recalled cardiovascular devices gained FDA approval with little to no clinical evidence (Open access)

 

Stanford study(Open access)

 

The New York Times article – From A.I. to Musk’s Brain Chips, the F.D.A.’s Device Unit Faces Rapid Change (Restricted access)

 

See more from MedicalBrief archives:

 

3M pays $6bn for faulty earplugs

 

Massive Philips recall highlights flaws in medical device surveillance

 

Billions paid out in the US to replace defective heart devices

 

US lawmaker probes FDA inspection of Musk’s Neuralink

 

FDA approves AI device to check skin cancer

 

 

 

 

MedicalBrief — our free weekly e-newsletter

We'd appreciate as much information as possible, however only an email address is required.