A discussion arose on the Qt-interest mailing list a while back about medical device companies using Qt. Given that damned few people read that list I decided to post the bulk of it here. I’ll leave their name out. If you really gotta know you can search the October 2022 archive.
The assertion:
A poster (troll?) writes that few medical device companies use Qt. That is
simply incorrect.I know developers at 4 medical device companies of which 3 use Qt: Qt 5, Qt
poster
6, and don’t know if the 3rd transitioned to Qt 6 yet.
Since this is a blog, we can start out with the cheap shot. Unless you are talking about the number of dudes looking to kick your ass in a bar fight, 4 is not a big number. According to an entity which monitors such things, as of the start of 2023 there were 933 medical device manufacturers in the United States.
Fear
Every developer working on medical devices and automotive safety/self-driving systems must live with the fear that can be found at this link. You worked on a device that killed people and a regulatory body recalled/pulled it from the market. We have to be Mercedes developers, not Tesla developers. Far too many getting into this market are “Tesla developers.” They think Agile is a valid software development methodology (it’s not) and that it is okay to ship half (or less)-baked software and systems to customers. It’s not okay. It’s never okay to have this stat. While we are at it, sure as Hell not okay to have enough of a body count for a Web site like this one to exist.
If having worked on a medical device that ends up in an FDA Spotlight story like this one doesn’t bother you in the least, you shouldn’t be allowed near a keyboard even to check email. People who are damned sick need the most complex of devices. They come into a doctor/hospital/whatever medical service and they get told they need this device. Said device is supposed to be thoroughly tested and vetted. The FDA has to approve it after all. They have every reason to trust it and it is your job as a developer to see that trust is not misplaced. Far too many Agile developers in this market segment. Trust in their software is simply false hope.
The Misinterpretation
Where this reader failed to follow is to the end of the sentence. I and others had told them “We no longer see medical device companies using Qt for new product development.” To understand that you have to understand the life cycle of a medical device. I’ve been in that field for about a decade now. Others will have different names for the stages, but the stages remain the same.
Medical Device Life Cycle – Death to Birth
Defunct/Discontinued
Either the FDA pulled this from market or the manufacturer ceased making it. Might be superceded by later products. Might have had to been destroyed completely as part of market removal. Documentation, maintenance kits, and training are no longer produced.
Heritage
Development team disbanded/off to other things. Product is still manufactured/sold as are maintenance kits. Training and documentation is available. This is a device that is out to pasture. It will continue generating revenue without receiving any enhancements. There are no new FDA filings for these devices.
Legacy
Many are tempted to call this “current” but Legacy is more accurate. A device moves from “New Product” or “De Novo” to here. They spend the majority of their life here before becoming Heritage. At this stage of life the devices still have teams. New versions are repeatedly introduced. This is the sweet spot because you don’t have to do the full “New Product” filing and testing.
Many companies play fast and loose with “minor enhancement” filings. Changing how a device detects occlusion from tried and true to something new certainly wouldn’t qualify as a “minor enhancement” in my book.
You can generally ride the sweet spot for 6-10 years before someone at the FDA forces you to go through a “New Product” filing. There are only so many “minor changes” they will let you make before they make you call it a completely new product.
New Product
- The FDA made you submit under this category
- You have made medical devices before but not this device. You mined the FDA filing database for all of the appropriate information so you too can turn a profit selling devices of this type.
- You’ve never made a medical device before but this non-invasive diagnostic/monitoring device looks highly profitable so you mined the filling database as above.
Only real difference between 2 and 3 is that 3 will almost definitely have to go through a full audit.
When mining the device databases you search for what tools the De Novo filers are using. That’s the cutting edge stuff that will be good for many years if the software has already been certified or proven with the FDA. You can re-use/reference their proof if you use that exact same software down to the last digit of the version.
De Novo
Nobody has made this before.
There may be other surgical robots on the market but yours is the only one that can, say, perform a heart transplant all by itself. There are many ultrasound applications out there, but your device uses ultrasound to 99.9999% accurately diagnose Alzheimer’s. etc.
This is a rough filing path, even rougher than 3 with audit. My current/most recent project is going down this path now.
The documentation you have to create is massive because you have to document to a level allowing any other manufacturer to make a similar device if they can avoid stepping on your patents. If your device is diagnostic in nature, you have to formally provide and prove the values/ranges/information displayed.
Everybody has seen a glucose tester. A drop of blood in the test strip and a number comes up on the screen. The how and the test strip are up to individual manufacturers. The drop of blood, however, has to create the same number (within small margin of error) on each and every manufacture’s device. It cannot generate 104 on one and 222 on another.
The company that filed the De Novo for the first glucose tester had to develop that industry standard as part of their De Novo filing process and they had to do it in a provable manner.
Life Cycle Summary
A medical device is “born” as either a New Product or a De Novo FDA filing process. You have to actually “file” the documentation before you write your first line of code. All of this stuff goes into a Design History File. (Really more of a directory or document database on disk these days.)
Once you complete your software and/or hardware development a manufacturing line is set up. This has to be the exact same production line that will be used to manufacture the device once approved. You can’t “build a few by hand” for certification and clinical trials. The FDA mandates what gets tested and certified actually comes off the production line.
Third party/external QA and clinical trials could be short, or lengthy and expensive. That really depends on the type of device and what you get approved as a formal test plan. I think we can all agree that a no-contact thermometer doesn’t need as much testing as a heart transplant surgical robot.
After all of the testing and trials are complete, you can start up full production and sell your device. It’s now your Legacy. In general you will have 6-10 years where you can push out new versions of this product as “minor enhancements” with a much shorter filing/testing/approval process.
Minor Enhancement
There simply are too many companies playing fast and loose with the definition of a “minor enhancement.” As a software developer, there is no way I would classify changing an infusion pump from using the standardized pressures used most everywhere for occlusion alarms to “learning what an occlusion is” as a “minor enhancement.” I didn’t read every link and every word in that FDA recall, but a quick skim leaves me with that impression. Maybe they were doing that for years, but in that case, why wasn’t QA already asking/testing “What happens if we turn it on with an occlusion already present?”
I understand there is an MBA (probably from Keller) with a spreadsheet proving how much more profit the company will make by using Agile and skipping testing, but you have to be able to sleep at night. I guess if one is void of ethics the body count doesn’t bother them. If you have ethics you can’t really use Agile in the medical device world. Those “User Story” solutions aren’t visible to QA or regulators. That’s where the body count sky rockets.
The Lies We Accept
Humans are unbelievably gullible. They will accept any lie without question if it happens to make their lives easier or is something they wish to believe. Here’s the biggest lie that kills the most patients.
Oh, we’ll be fine. We simply won’t use that.
Gullible humans and management
Take a good read of QTBUG-12055. Not just the image pasted here, the entire bug and its history.
First reported against Qt version 4.5.0 it spent a lot of years being denied as a bug despite being reproducible. This was an insidious patient killing bug ranked “Somewhat important” in the bug database. (Read through the history, I seem to remember it was marked unimportant during the years it was denied.)
I encountered sooooo many shops who told themselves the above lie and believed they were “safe” because they would not use QTextStream in their project. Read all the way through the bug. This bug had to do with the single-threadiness of Qt. Delete actions were queued across threads. This meant any thread using something from another thread (generally via Signal/slot connection) did not have their connection cut when the object was deleted.
More Detail
QTextStream was just the easiest to reproduce the issue with. You opened a logging file. QTextStream was using it from some other thread. Now it came time to close the log file and open a new so you could send the old up to some periodic reporting/archive system. A common use case, especially in the medical device world.
Your main thread dutifully closes and kills off the file object, then creates another and sends out signals that all logging should now use this file. Trouble is the deletion of the first and the connecting to the new were queued events in other threads. Now you crash because you try to write to a closed file held by a deleted object.
The Fix Wasn’t Much Better
“The fix” which came about 11 years after initial reporting. Now they perform the delete and disconnects as a “direct connection.” Okay, this kinda stops a crash. But now your QTextStream doesn’t have any place to write while the event loop for the thread is waiting for execution time. You will lose log messages. If that log is of a patient having a heart attack it is important.
The Lies Don’t Stop
Now that Qtc is focused on a hand polished turd called QML, the bug database will climb to the stratosphere. If you want to know why read about Drunk Driving Across All Three Lanes. Management likes to hire people who never went to college because they are cheaper. Problem is cheaper people don’t learn C++, they want to use JavaScript or some other scripting language like Python.
It doesn’t matter what part of Qt a bug is reported against. As you’ve seen with the QTextStream issue, the bugs tend to be deeply rooted in Qt. Just avoiding whatever it was reported against doesn’t actually avoid the bug.
Summary – Medical Device Companies Using Qt
I’ve used Qt and written about it for many years now. There was a time pre-QML where it was a fine OpenSource project headed in the right direction. Not so much today. When medical device projects reach out today Qt is almost never mentioned. I’m actually going to be taking over one that tried to us Qt and failed to deliver. Don’t know exactly what happened, I just know their client is ripping it out of their hands and giving it to a firm I work with.
As I and others stated to the individual, we no longer see medical device companies using Qt for New Product/De Novo developments. It is still being used in the Legacy world, but that has a limited future. During the days of Qt 4.8 OpenSource and Widgets it seemed like every medical device company was using Qt. You would get 2-3 emails/phone calls per week about projects while you were working at your current client site.
Not today during the Qt 6.x era. Most of it went away during the Qt 5.x era along with the perpetual license and OpenSource LTS. The perpetual license fees were astronomical. Last client I worked for that bought one during 5.x era reportedly paid $650,000 for a 5-user royalty-free license. Really shocking if you visit here today.
Yeah, that 21583 is what the bug database returned for REPORTED, OPEN, IN PROGRESS, REOPENED. That’s gotta make pee run down the collective leg of the risk management team. And that is for SOUP (Software Of Unknown Provenance) . You are responsible for testing and certifying it. Given the massively expensive license and/or perpetual royalties, how can you justify using something that hasn’t gotten its entire code base IEC 62304 certified?
The QNX Difference
Contrast that with QNX Neutrino RTOS for medical devices. They actually get that independently certified. You are buying both not having to do that and an insurance policy that if the OS turns out to be at fault, Blackberry will be liable, not you. That’s something worth paying for! Don’t know what it costs, but it is worth something.
For certain someone will be able to scour the planet and find one, maybe five, doing New Product or De Novo with Qt today, but out of 933, that isn’t much. Most of the openings you see now are Legacy headed toward Heritage.
In speak other geeks will understand, Qt has become the FORTRAN and Ada of the medical device world. Some companies are trapped by/with the tool and new companies/products avoiding. Boeing still hires FORTRAN and Ada developers because they have to, but that is changing.
Final Word
Programming languages and tools have a good run for a couple of decades, that’s it. After that they are, in Gartner speak, “Non-strategic Divest.” There is no company creating a shiny new CRM, ERP, Accounting, Warehouse Management, whatever system starting out today saying “I know! We’ll write everything in COBOL!”
