The Robots Are Coming, Part 2

Last week we tiptoed up to some of the problems that increasing automation in health care may uncover. This week let’s talk briefly about proposed FDA action to address them.

FDA approvals of automated and artificially intelligent health care platforms have been accelerating, and in the flurry of activity that accompanies all Presidential transfers of power, the outgoing Trump administration had controversially proposed waiving much of the regulatory oversight of medical artificial intelligence tools. With a new administration in place, the original WhiteHouse.gov link I’d originally included in this post no longer exists, and the FDA has pivoted hard in the other direction, releasing a new five-part action plan laying out its efforts to regulate products that incorporate “Artificial Intelligence/Machine Learning (AI/ML)-Based Software as a Medical Device (SaMD).” Let’s go through them one at a time:

  1. One of the big challenges in regulating artificial intelligence is that an AI program is, by definition, not a static product the way a drug or a physical device is. Once the FDA approves a drug, we know that we’ll be dispensed the same drug ten years from now that we’ll get today.

    But artificial intelligence learns over time. So as the dataset that a software platform “learns from” grows, the interpretation of that data by the program may change. We call this process of gradual improvement through multiple steps an “iterative” process. So the FDA has expressed an expectation that manufacturers and the FDA be able to transparently monitor performance through this iterative process in hopes of maximizing safety while allowing the gradual change and improvement of the platform.

  2. When you go to an ATM you have the expectation that you can put any bank card into the ATM and check your bank balance, even if your ATM card was issued from a different bank than the machine. When you put a CD into a CD player, you know it will play, regardless of whether the machine is a Sony or a Panasonic. But this isn’t true of every device. Electronic health records are notoriously finicky about what data they can exchange, for example. So the FDA has asked manufacturers to use “Good Machine Learning Practice (GMLP)” to “encourage harmonization of the development of programs through consensus standards efforts.” These include harmonization of data management, feature extraction, training, interpretability, evaluation, security, and documentation standards.

  3. Many artificial intelligence/machine learning platforms are trained on existing datasets. The patients who contributed their data were real, but the data is now so divorced from any living creature that it is easy to think of it in hard, mathematical terms rather than as attached to a real, living person who had thoughts, feelings, emotions, and parents. So the FDA has asked that manufacturers take a “patient centered” approach to how these technologies interact with people. What this precisely means is still under discussion, but broadly it seems to mean transparency to users (that is, you’ll know when your data is being used, and you’ll know when a machine is helping make decisions in your care), usability (meaning the operation of the software won’t be a mystery), equity (everyone gets a fair shake at representation within the software’s training dataset, for example), trust, and accountability.

  4. The machines we build carry our biases. You may have read about bias in software used in sentencing for convicted criminals. The software was meant to reduce bias in sentencing, but since it was trained on a dataset that displayed the bias of past sentencing, the software itself was biased against certain groups of people. Or you may have seen news of this in facial recognition algorithms, which only become adept at recognizing faces when intentionally exposed to diverse faces. A good example of a similar phenomenon in medicine is a tool developed to predict knee pain in patients with osteoarthritis. A commonly used tool built on data from mostly white, British patients was found to be less accurate than a similar tool that was trained on data that included more Black and low-income patients. The new, more diversely trained model roughly doubled the likelihood that an evaluated Black patient would be considered eligible for surgery. With this in mind, the FDA has pledged to “evaluate and address algorithmic bias and to promote algorithm robustness,” specifically as it relates to race, ethnicity, and socioeconomic status, to avoid biases present in the health care system from seeping into algorithms.

  5. Since many currently commercially available AI and machine learning products were approved based on their performance with historical datasets, not based on prospective testing on real patients the way a drug or another device would have historically been, we don’t know for certain how they’ll do in the real, nitty-gritty care of patients. But even with the traditional model of drug testing prior to approval, a need for post-approval testing is not unprecedented. Roughly a fifth of medications prescribed, for example, are used “off-label,” meaning they’re used for something other than the purpose for which the FDA originally approved them. Gabapentin, for example, is frequently used for pain relief, but it’s FDA approvals are only for seizure disorder and for a specific type of pain syndrome called “postherpetic neuralgia.” With this in mind, the FDA has pledged to monitor “Real-World Performance.” This not only allows the FDA to monitor the performance of the device in terms of accuracy of its recommendations, but it allows the FDA to monitor just exactly how the devices are being used. As far as I can tell, real-world performance data monitoring at this point is voluntary but encouraged. Depending on how well this voluntary system works, FDA intends to develop a framework for any mandatory prospective reporting in the future.

 Will any of this solidify public and physician trust in artificial intelligence? I don’t know. My hunch is that trust on the physician side will hinge more on the positive or negative effect of AI on clinics’ bottom line. And public trust of technology seems to depend more on convenience than on the good or ill intentions of the company. Few of us complain about Google, for example, because even though Google knows a lot about us it makes certain parts of our lives, like the composition of this blog post on Google Docs, better.

As the Medical Director of the Kansas Business Group on Health I’m sometimes asked to weigh in on hot topics that might affect employers or employees. This is a reprint of a blog post from KBGH.

The Robots Are Coming, Part 1

We are mostly techno-optimists here at KBGH. We have talked about the possibility of technology saving the aging primary care workforce by augmenting their skills in certain areas. But if you are a techno-pessimist, you might think more in terms of what automation or robots will do to certain jobs, the way factory automation has decreased the availability of jobs in manufacturing. (Or you may worry about safety because of bias in machines or a Tom Selleck-Runaway style robot rebellion. Great movie with Wichita native Kirstie Alley. I digress.)

My training is in endocrinology and metabolism, disorders of the finely tuned feedback loops of chemical messengers in the body. About half of most endocrinologists’ practices is the care of diabetes mellitus, a collection of metabolic defects that cause excess sugar to build up in the blood and cause blindness, kidney disease, and nerve damage, among other devastating problems. When I was in training in the mid-aughts, a big part of my day was spent managing insulin pumps, small pager-sized devices worn by some diabetic patients that deliver precise doses of insulin to meet their dietary and exercise patterns. My job was to observe blood glucose levels the patients took from fingersticks and coach patients on how to change their pump settings. For all their sophistication, insulin pumps were still pretty manual.

In the last few years, though, a new type of insulin pump has emerged. We call them “closed-loop” devices because, when paired with an implanted sensor that constantly tracks the patient’s blood sugar levels, the pumps can make adjustments to insulin infusion rates without the wearer even being involved. For now, the automation in the devices is mostly confined to rates of insulin infusion when the wearer is not eating or exercising. But pumps that can detect food intake and activity and make rapid adjustments are just around the corner. Insulin pumps will eventually use artificial intelligence, complex intelligence uninvolved with messy human emotionality or consciousness, to make adjustments in the background that are seemingly unrelated to our traditional understanding of diet, exercise, or adherence to therapy. Accidentally load your insulin pump with insulin that is slightly out of date and less potent than last week’s supply? The machine may detect it and adjust your rates of infusion to make up the difference. Have a family tragedy that increases your stress hormone levels, causing blood sugars to spike? The insulin pump’s “brain” may be able to detect this and bring your sugars back to normal without your input or recognition.

And this is just the tip of the iceberg in terms of medical technology. Already “decision support systems,” like the alert your doctor gets through her electronic health record to make sure you get your cancer screenings, show small, persistent improvements in overall care. And as we touched on in previous blog posts, robots have proved themselves to be superior to humans in a range of medical tasks, from finding diabetic eye disease to detecting bleeding in brains on CT scans.

All this will require a cultural shift. Often patients express frustration at having “only seen the PA” when they have gone to the doctor, in spite of ample evidence that physicians’ assistants and nurse practitioners provide excellent care, sometimes exceeding the quality of care of physicians in trials. Our culture currently places value on face-to-face time with the physician. And Americans are anxious about the potential safety of driverless cars in spite of the fact that human-driven cars currently kill more than 30,000 people a year. So how will we respond to robots guiding certain potentially high-risk parts of our care like insulin adjustment or detection of bleeding in radiology studies? Maybe we will give them the same brush-off we sometimes give PAs. Or maybe we will accept their input the way we have accepted advertising algorithms from Facebook and Google. It is completely possible that medical professionals will resent medical robots the way we resent automation for taking away factory jobs. To tiptoe into these ideas we will talk about a potential regulatory framework next week.

As the Medical Director of the Kansas Business Group on Health I’m sometimes asked to weigh in on hot topics that might affect employers or employees. This is a reprint of a blog post from KBGH.

Will technology save the aging primary care workforce?

As the Medical Director of the Kansas Business Group on Health I’m sometimes asked to weigh in on topics that might affect employers or employees. This is a reprint of a blog post from KBGH:

The issue we’re facing

The primary care physician workforce in Kansas–family doctors, internists, and pediatricians–is aging. Of the 1,976 primary care physicians in Kansas as of April 2020, 15.6 percent are already over 65, and 39.2 percent are over 55. The simple demographics of this are intimidating: even though they provide the most essential, cost-effective care in medicine, only 43 percent of practicing physicians in the U.S. are primary care providers, similar to the average of eleven Organization for Economic Cooperation and Development countries. But the fraction of graduating students entering primary care is steadily decreasing.  Even more ominously, older physicians are much more likely to be harmed by infectious diseases like SARS-CoV2, the causative virus behind COVID-19, adding to the inevitable workforce turnover caused by death. This all portends an uncertain future for primary care provision in many Kansas communities, since Kansas is already underserved relative to most other states at baseline.

As if that weren’t enough to worry about, physician skills appear to deteriorate over time. A 2017 study in the British Medical Journal found, for example, that elderly Medicare beneficiaries’ hospital adjusted 30-day mortality rates were 10.8% for physicians aged <40 and rose steadily to 12.1% for physicians aged ≥60, a 15% relative increase in risk for patients cared for by older doctors. Not only that, but costs of care were slightly higher among older physicians. This may not simply be due to age-related decline; it could be that younger doctors were trained in a way that improved their care. For example, “evidence-based medicine” is an integral part of medical training in the modern era. Older doctors who were not trained under this philosophy are demonstrably less likely to follow evidence-based care guidelines. This is hard for me to read. Statistically, I am likely a worse doctor than I was fifteen years ago. But I digress.

What can be done about this problem?

The Association of American of Medical Colleges, predictably, has argued for years that the solution is to train more physicians, by two mechanisms: first, the AAMC advocates for increasing the cap on Medicare funding that limits the number of residents at a given institution. Second, the AAMC supports greater incentives such as scholarships and loan repayment for primary care providers working in underserved areas. Examples of this are the Kansas Medical Student Loan Program, which pays for medical school for a limited number of students in return for an agreement to practice primary care in underserved areas in Kansas; and the Kansas Bridging Plan, which gives resident physicians additional funding during their training in exchange for a three-year commitment to practice in a rural community. On the federal level, the AAMC advocates for increased recruitment of international medical graduates, who already represent about a quarter of practicing physicians in America, through programs like the J-1 Visa Waiver program.

Others point toward increased use of non-physician practitioners like physician assistants (PAs) and advanced practice registered nurses (APRNs). This is clearly the preferred short-term option. PAs and APRNs require drastically less training than physicians, which eliminates the seven-year gap between policy and practice that we see in traditional medical training. And the health outcomes of patients seen by non-physician providers seem to be roughly equivalent to those of patients seen by doctors. Another British Medical Journal systematic review of randomized trials and observational studies–one of several such reviews in various journals, all with similar conclusions–concluded that “Patients are more satisfied with care from a nurse practitioner than from a doctor, with no difference in health outcomes.”

But long-term, if the skills of physicians like me decline with age, we can be certain the skills of other providers fall as well. How do we ensure that quality care continues to be delivered over the lifespan of the practitioner?

Automation may be the answer

Let’s look at my specialty, endocrinology. Six years ago, when I left full-time practice, the management of blood glucose levels was mostly an intuitive art/science, driven mostly by the experience of the physician-patient dyad. But in the last few years we’ve seen the emergence of “smart” glucometers that quadruple the likelihood of of a patient controlling their blood sugars while reducing their risk of dangerous low blood sugars. We’ve seen the development of automated insulin devices in the hospital that outperform conventional treatment of blood glucose levels. The FDA approved an artificial intelligence-based device to scan and diagnose the eyes of diabetic patients with diabetic eye changes (the most frequent complication of diabetes) without even having an ophthalmologist or optometrist involved. Newer, even more innovative, devices are in development, such as an app that can allegedly detect the presence of lung disease by the sound of a patient’s cough.

Some of these devices will pan out in the long run, while others won’t. But even a conservative projection is cause for optimism. It is not unreasonable to predict that practitioners with far less training than physicians will have the tools and skills to provide very competent care–elements of both primary care and specialty care–in the near future. Technology must be carefully monitored by humans, but its abilities do not decline with age. On the contrary, a given technology’s performance today is the worst that it will likely ever be. Best Buy will sell faster computers next month than it does today, and faster yet a year from now. And automated devices aren’t resistant to delivering evidence-based care; it is programmed in. I welcome the Rise of the Robots.

Links for Tuesday, November 21, 2017: more on the new HTN guideline, Gymnastics coaches throwing robot shade, the last iron lungs, Germany bans smartwatches, and Raymond Chandler hated US healthcare

Thoughtful post on the new HTN guideline by Dr. Allen Brett

Representative quote: "Consider, for example, a healthy white 65-year-old male nonsmoker with a BP of 130/80 mm Hg, total cholesterol level of 160 mg/dL, HDL cholesterol of 60 mg/dL, LDL cholesterol of 80 mg/dL, and fasting blood glucose of 80 mg/dL — all favorable numbers. The calculator estimates his 10-year CV risk to be 10.1%, making him eligible for BP-lowering medication under the new guideline. To my knowledge, no compelling evidence exists to support drug therapy for this person."

A gymnastics coach says the Boston Dynamics robot flip was a 3.5/5.0

'In a back salto, says Mazloum, “you want to be able to go as high as you can, and you want to be able to land as close to where you take off as possible.” To do that, the gymnast has to squat, throw her arms up by her ears so her body is a straight line (in gymnast-speak, opening the shoulder angle and the hip), then contract into a “closed” position again. By these standards, Atlas’ trick is “not the cleanest flip,” explains Mazloum.

Here’s Mazloum’s critique: Atlas didn’t quite get to that open position, “so it didn’t really get the full vertical that we look for. That’s why it went backwards a little bit.”'

The last of the iron lungs

Get your kids vaccinated for polio, folks.

Germany has banned smartwatches for kids

If I understand this correctly, it is not because smartwatches cause kids to be distracted monsters (although I don't doubt that that statement is at least a little bit true). The decision stems from the capability of bad guys to hack in and monitor the location of little Dick and Jane:

You have to wonder who thought attaching a low-cost, internet-enabled microphone and a GPS tracker to a kid would be a good idea in the first place. Almost none of the companies offering these “toys” implement reasonable security standards, nor do they typically promise that the data they collect—from your children—won’t be used be used for marketing purposes. If there ever was a time to actually sit down and read the terms and conditions, this was it.
Get your shit together, parents.

Asking parents to destroy them might be a bit of an overreaction, though.

Raymond Chandler paints a dark picture of American healthcare in a newly-discovered story

The title, "It’s All Right – He Only Died," sounds like the title of a video residencies would show interns to convince them that quality improvement and patient safety are part of their job.

The doctor who turned away the patient, Chandler writes, had “disgrace[d] himself as a person, as a healer, as a saviour of life, as a man required by his profession never to turn aside from anyone his long-acquired skill might help or save”.