Sepsis is hard to spot. Whats interesting about this article is that once you get into the details of whatt happened on the patients second visit, its largely about the hospital information systems and how they got in the way.
An automated alert popped up warning that the doctors should consider Sepsis. That alert essentially then blocked progress, and the doctors ended up (essentially) ticking the 'not sepsis' box so that they could get on with their (reasonable) next step which was either ordering an x-ray or starting antibiotics. Then somehow after that, sepsis did not get re-considered.
It was Banerjee’s task to document Sam’s care, and as he began to do so, a pop-up appeared on his computer screen. Sam’s fever and heart rate had triggered an automated warning for sepsis, a potentially life-threatening condition in which the immune system has a dangerous reaction to an infection. It requires speedy intervention. To help the hospital comply with state-mandated sepsis regulations, the pop-up provides a checklist of tests and orders used to identify and treat sepsis.
Agyare had instructed Banerjee to hydrate Sam right away but to wait for the results of Sam’s lab work before ordering a chest X-ray or the strong antibiotics used to treat sepsis.
But Banerjee, a novice, got stuck. He couldn’t figure out how to navigate the template to make some but not all of the auto-populated orders. “This was my first patient that triggered the sepsis pathway,” he explained, in testimony. So he asked Connor Welsh, a third-year resident, for help.
At 8:50 p.m., Welsh showed Banerjee how. From his own computer, he clicked into a field on Sam’s chart to assert that sepsis was not likely: “Based on my evaluation,” the automated note said, “this patient does not meet clinical criteria for bacterial sepsis.” And then Welsh recorded what Banerjee said Agyare had said earlier: “Likely viral syndrome. Workup pending.” Welsh’s name appears on the note, but in his deposition he said he never interacted with Sam. Senior residents often help junior ones in this way, he said. “I signed this note based on the discussion with the provider, Dr. Banerjee, based on his evaluation and the medical management of Mr. Terblanche,” he testified.
...
Sam’s chart is 51 pages long, a catalog of billing codes and abbreviations, check-boxes and shorthand, updates and addenda. The record of the second visit contains numerous contradictions: Sam’s heart rate was documented at 126, yet Banerjee clicked the box “normal.” In one place it says Sam didn’t have a cough, while in another it says he did. The signatures of doctors who testified they never saw Sam — including one who was not in the hospital that night — accompany notes. Vital signs were ordered and not taken, as was an EKG.
> An automated alert popped up warning that the doctors should consider Sepsis. That alert essentially then blocked progress, and the doctors ended up (essentially) ticking the 'not sepsis' box so that they could get on with their (reasonable) next step which was either ordering an x-ray or starting antibiotics.
Man, I feel like I've been trying in vain to fight pop-ups for my whole software development career. Now we have an example where, at worst a pop-up got someone killed, and at best it was part of the chain of events that got someone killed. I don't know what it is that draws product designers to keep reaching for that horrible UX pattern, but it's got to be stopped. Nobody reads these things so a popup is the worst place to put important information that the user needs to read!
I think a lot of pop-up usage comes from company lawyers trying to cover butts: "Well, regulation says that users need to be informed of XYZ, so just stick a pop-up there. Then we can tell the regulator 'Hey at least we did our part to inform the user.'"
It's even worse, in a way. Mt Sinai uses Epic, as do many (most?) other larger hospitals in the US.
The stops - which in my experience (anesthesiologist, not ER doc, so I don't get sepsis warnings but I do get some of my own) are not popups, per se, but warnings that prevent you from leaving the screen until you have dealt with them. In this case, he could not place an order for at least some of the sepsis bundle of orders without placing all of them or making the sepsis warning go away - permanently. And the inexperienced trainee was told by the experienced supervising physician not to order the antibiotics and x-ray until at least preliminary labs had come back. At least for the antibiotics, this is good stewardship - we don't want to be giving people antibiotics for viral illnesses.
I have fought against incomprehensible ordering systems so much that when I order a chest X-ray (usually to confirm the placement of a large central IV), I have found it best to call the radiology technician and tell them what I want and let them order it under my name, because if I don't, I'll inevitably screw up some minor detail and they will have to re-order it anyway. "Chest x-ray to confirm internal jugular central venous catheter placement" (well, "CXR to check IJ CVC") is what I would have written in a paper chart; now it wants to know vast amounts of detail that I often don't know.
So this story comes down to a gripe about data systens that is as old as these systems themselves: the designers and implementers come up with data models and workflows that feel like they cover every possible case and they deliver a system that is a perfectly rigid implementation of these flows because they seem complete. Inevitably, these systems meet the messy real world and users find themselves unable to do what they need to do for the occasinal really weird case and get hopelessly stuck because they run up against stubborn rule checks.
This plays out every time when two workers look at each other and go "How do I enter XYZ without ABC?" - "You can't do this. Here's our workaround".
It’s not even a weird case. I have to specify, but don’t care if it’s back-to-front or front-to-back. Or where the x-ray is taken (yes, doctors have to specify that). That matters to the radiologist reading it (plain x-rays are like a camera obscura, so things close to the emitter cast a larger shadow), but for my purposes, it doesn’t matter because the level of detail I need is not that.
So in your case the SW is a poor fit. That feels underwhelming from a package that is used widely and presumably should have the flexibility to adapt to the requirements of very different customers.
Yes, all major inpatient EHRs do have the flexibility to adapt to those requirements. But that level of customization is itself a huge effort, and many hospitals lack the resources to do it well. The other problem is that hospital administrators and IT staff often give in to internal pressure from clinicians to customize every little thing even when it doesn't actually improve productivity or care quality. Sometimes it's better for humans to adapt to the software rather than vice versa. "Vanilla is the best flavor."
As a clinician, I wholeheartedly agree that adapting myself to the software is usually better than spending tons of money to try to make the software fit me, but at the same time the software really should fit the general norm of practice style.
This is more about unnecessary complexity. Medicine is hierarchical and like any hierarchy those at the top should be giving orders that leave lots of leeway for those who have to make them happen. EHRs often push all that onto the physician who never learned how to do that and really shouldn’t.
It’s like the old story about a green lieutenant in the army. Commander says, Lieutenant Smith, I want a flagpole on our parade ground. The dumb lieutenant tells their sergeant precisely how to do it. The smart lieutenant says, “Sergeant, the commander wants a flagpole right here. Make it happen.”
Whenever a question arises (and it will), the sergeant with a dumb lieutenant has to go back and clear every step. The smart lieutenant’s sergeant doesn’t have to call until they encounter something they don’t know how to do or can’t approve on their own authority.
> the designers and implementers come up with data models and workflows that feel like they cover every possible case and they deliver a system that is a perfectly rigid implementation of these flows because they seem complete. Inevitably, these systems meet the messy real world and users find themselves unable to do what they need to do for the occasinal really weird case and get hopelessly stuck because they run up against stubborn rule checks.
No. Those systems have nothing to do with the real world. They are just a project manager's idea about "the real world". Have you used any Microsoft product lately ? Every day it tells you: "Hey , i have a new feature".
But that is 41% for a single vendor out of 10. So market leader. And it looks like the data is based on hospital installations but not normalized based on quantity of patient served or patient case complexity.
Edward’s Hospital in Plainfield, Il and Mt Sinai are two different use cases and should not count equally.
That is something to keep an eye on. The system that owns the hospital where 95% of my work happens uses Epic and they subsidize it for hospitals that aren’t formally part of our system but could be a good referral base.
Catholic hospital, so it has its own internal issues, but it’s definitely not PE-controlled.
I think the main problem with pop-ups that makes them a bad place to put important (any?) information is that they interrupt the user's workflow - which is, indeed, kinda the point of the pop-up. But what is a normal human's reaction to this. They are in the middle of trying to do the thing they want to do. To continue with this, they need to get the pop-up out of the way, so they just click the button.
I run into this ALL THE TIME during the normal course of using software. I am trying to do something, then I get a pop-up about something-or-another that the developer clearly thought was important for me to see, but I click past it so I can get back to what I was doing. A lot of times is these "look at this cool new feature" kind of things - ARGH! And the worst part is, I might actually WANT to come back and learn about the new feature, but often I only get that one chance, and I'm too busy on my important task to focus on it right now.
Actually - a specific example of this just case to mind: I want to take a screenshot with Snagit - so I press PrtSn. But now Snagit pops up and asks me if I want to update. Sure, maybe I do - but not now! Now I just want to take my damn screenshot! So I click 'no.' And then I don't see it again until the next time I want to take a screenshot, and also do not want to be interrupted in my 5sec tasks with a 5min update.
Much better is to provide the information in a conspicuous, hard-to-miss area of the UI, but which DOESN'T block my workflow. Like, literally just put the text right on the form/interface in bold, red font. Like for Snagit - don't make it a popup update notification. Just put a bold/red link on the capture window and editor UI that says "Update"
Be typing a prompt and get 1) Do Action 2) Always Do Action 3) Don't Do Action
But if, like me, one looks at the keyboard while typing, you might not notice and be typing away and it has stolen your focus. Worse it used to default to 1) when you pressed enter and so you might have agreed to anything.
Oh man! Focus stealing is the worst! Bitwarden used to do this while I was typing my master password, stealing focus in the middle of what I'm typing to ask if I want to interrupt what I'm trying to accomplish to get their next minor upgrade. Which, of course typing my password very quickly I often hit enter at that moment, forcing me down an upgrade path. Beside the point of having my typing part of my password into a context it wasn't meant for. I think they've fixed it though.
Since you can’t reliably catch 100% of tricks 100% of the time, continuing to do so is effectively guaranteering yourself to be tripped up in that 1 out of 100 times.
I remember a time when I thought we were past the Windows 98/XP popup malware era. Now every single website and application loves to shove popups in my face whenever I'm trying to accomplish a quick task. I don't think I have ever read anything other than the buttons to figure out which ones not to press.
It occurs to me that maybe user behavior metrics are to blame, because our default response to these stupid intrusions (click any button to make it go away) appears as "engagement" to the devs/markting people: "oh look, people are clicking our popup buttons!" so then they are encouraged to add more of them. "Look how much engagement we're driving!"
The problem is that they actually work. I've been involved in having to implement these nightmares and it turns out that normal people do actually interact with them and it does result in increased revenue.
I am currently waiting for an oven technician to show up to help fix an oven that refuses to operate until I complete a certain cleaning cycle. But the cleaning cycle is blocked on a pop up that cannot be dismissed. I think it’s trying to helpfully give me instructions, but I don’t need the instructions — I already followed them and now it should be the oven’s turn.
> don't know what it is that draws product designers to keep reaching for that horrible UX pattern, but it's got to be stopped. Nobody reads these things so a popup is the worst place to put important information that the user needs to read!
Hi, Product Manager and paramedic here.
The type of popup you are talking about is not what is happening here. It's a modal dialog requiring the provider to answer multiple questions to evaluate their patient (see my other comment in this thread).
And then it will warn you that the patient meets SIRS or sepsis criteria. It's not quite as simple as "regulation says user needs to get warned of blah".
As a product designer, I can think of 2 reasons why the person that came up with this flow might've made it a popup:
1) That's the default design system pattern for alerts, so whoever was the designer just went with it.
2) There's other alert patterns (alert bar, toast, etc), but sepsis was deemed to be so dangerous to the patient that it deserved to have its own special friction inducing UI element to alert doctors to take action.
>I don't know what it is that draws product designers to keep reaching for that horrible UX pattern
There are legitimate cases for alert modals like this one, but this definitely is an example as to when it shouldn't be used.
Sepsis is indeed an immediate life threatening condition, and I bet the alert was added as some kind of legal/medical ‘oh shit’ type of condition - without thinking through the consequences from a UX perspective.
Have you been on the other side of this? I get dozens of sepsis alerts a day, usually on the same patient, and the criteria that triggers them is so broad and non specific they are functionally useless. Each alert locks down the entire system ironically preventing you from reviewing what triggered it in the first place. You cannot do anything until it is addressed and you are forced to commit to an action without all the data because of it like administer a medication or order fluids, which may not be appropriate. Lots of things mimic sepsis criteria including but not limited to decompensated cirrhosis, HF, cancers, leukemias. The worst is that they don’t even pop up at the right time, they usually pop up usually way after the sepsis has been treated. In the past year, I’ve only had about a half dozen appropriate sepsis pop ups among the hundreds I’ve received.
It blocks the system with a demanded action, but doesn't even show you what triggered the alert condition? I would completely expect a "List of conditions that suspect sepsis" and get those details up front and center.
I'd be putting in medical records "Due to software popping up an un-dismissable sepsis screen that does not show details, I dismissed it due to needing the data it was flagged on".
Yes, really. I’ve had to restart my Citrix session to make it go away or dismiss it like the writer did.
You’ll get something like “sepsis criteria triggered by wbc 13, cr 1.5, hr 101, rr 22.” And that’s it - usually in the middle of a night on a new patient I just got a page for. Can’t open documentation to see the patients med history.
It’s ridiculous. I’m not using Epic but I am using a major EMR.
To be fair I’ve written almost exactly what you mentioned out of sheer frustration once or twice but it’s not ideal
Yes, in almost every case, the default "popup" GUI library call is also a modal dialog. You cannot access anything else, anywhere else, in the entire program (even if the program had multiple separate windows open). All you can do is read the dialog's text, and hit the "ok" button to dismiss it (or pick from one of a set of "buttons" that are shown on the popup to dismiss it).
The worst ones also do a global grab, with the result that you can't even switch away to another unrelated application on the system without first "interacting" with and dismissing the popup.
To see the version that is built into Javascript in the browser, put the following into the URL field of a new bookmark, and save the new bookmark.
javascript:(function(){alert("hello");})();
Then, while here on HN (or anywhere else), click that new bookmark you just made, which will pop up the default built in Javascript alert box, and try to interact with the rest of the page it pops up in front of.
UI engineers need to be held legally responsible for false alarms like this.
My older car regularly hallucinates an incoming frontend collision and takes over the speedometer with a flashing red/black screen.
The new one (Kia) overrides the steering and forces the car to depart the lane (usually over double yellow lines).
If the alert regularly produces false positives then such behavior (and the behaviors of these EMRs) should open the vendor to civil and criminal liability. The courts should just assume the behavior will lead to loss of life, in the same way as discharging a firearm randomly in the city might.
It probably makes sense to have a short grace period to push a patch. Maybe one week after 0.1% of users complain?
If your vehicle has a safety defect then you should file a formal complaint with the NHTSA. These things won't get fixed unless customers follow the process.
Safety defect: my Tesla has a really distracting giant screen in the drivers field of view. WTF?
Closed: Working as Intended, Elon has us by the balls.
It’s similar to alarm fatigue in the ED. In most, every piece of equipment is alerting on at least one thing at any given moment, if just because the patients pulse-ox sensor is not attached super well.
But they came in because of a diabetic emergency and we’re just waiting to make sure the treatment worked and 99% of the time it does, so no one really cares, but the UX around silencing it isn’t great (and may be a liability if used). Like in this example.
Because maybe this is one of those 1/1000 cases where the insulin didn’t work, and they lost all peripheral circulation and that pulse-ox sensor’s bad reading is warning you that they are about to lose all their fingers.
Appreciate the added and specific context. I'm not in the medical field but I have worked on EMRs.
What, by your estimation, would be the better user experience for alerting the imminently life threatening situation?
It sounds like a signal to noise ratio with false positives, but IMO I'd rather a provider be at least given the time to pause and consider the diagnosis. I'm not sure about the optimal way to do it.
We are grading a tool which might not need to be there at all. It shouldn't get points for coming in six hours later saying that the patient being treated for sepsis might have sepsis. See what I'm saying? There's a cost to every flag you wave.
UX is sort of the issue in the sense that people have been trained to ignore pop-ups because they are used for so much irrelevant garbage that it is a "boy who cried wolf" situation. The questions of: "How do you force a computer user to take something seriously?" is fundamentally a UX question.
When people encounter a jarring interaction with a computer, most people's default response is to blame the computer for not doing what they expected it to... even when the computer is telling them that they need to do something differently. This makes it very difficult to guide users into changing their behavior. And if the user has experienced this message erroneously in the past, they have been conditioned to presume it is erroneous.
Anyone who has ever worked helpdesk can tell you that people call in with "computer errors" all of the time that are simply messages telling the user to do something... but it doesn't occur to them to actually follow the instructions. This is a fundamental HCI issue that is tough to solve, and usually isn't solved by someone who is just building a form to check off a compliance item.
If you are actually trying to do something to resolve the situation, why would you want a progress stopping pop-up forcing you out of actually doing the thing you need to do? Especially if it forces you to pick out of two options when the thing you are trying to do is figure out which option is appropriate?
Using EMR systems are nightmares for anyone actually doing things, but great for figuring out what was documented later. The two are rarely related.
If someone is trying to land a plane, would a pop-up about an engine fire that takes over all control input also be a good idea?
Have you read any of the details? The provider was unable to take appropriate action because the progress halting popup was preventing taking that action.
Of course I did. You're wrong. The provider CHOSE not to take the appropriate action. The appropriate action was ordering the medication because the automated diagnoses was in fact correct.
PS - I would suggest avoiding rhetoric like "did you even read" as per the guidelines. It's just a waste of everyone's time.
> Agyare had instructed Banerjee to hydrate Sam right away but to wait for the results of Sam’s lab work before ordering a chest X-ray or the strong antibiotics used to treat sepsis.
> But Banerjee, a novice, got stuck. He couldn’t figure out how to navigate the template to make some but not all of the auto-populated orders. “This was my first patient that triggered the sepsis pathway,” he explained, in testimony. So he asked Connor Welsh, a third-year resident, for help.
> At 8:50 p.m., Welsh showed Banerjee how. From his own computer, he clicked into a field on Sam’s chart to assert that sepsis was not likely: “Based on my evaluation,” the automated note said, “this patient does not meet clinical criteria for bacterial sepsis.”
The provider was instructed (possibly wisely) to take some of the automated sepsis actions and to defer others. The computer did not allow this without generating an automated note saying “ this patient does not meet clinical criteria for bacterial sepsis.”
Imagine if your email UI auto-classified one particular email as so high priority that it would allow you to do nothing else, including reading other emails, until you archived the priority email and, to for good measure, would auto-reply “no action needed” when you hit archive. You get a priority email, and you determine that the correct action is to wait up to 20 minutes for another email that you expect and, while waiting 20 minutes, to read other emails from the same sender. But you can’t because your UI won’t allow it. So you archive it, send the “no action needed” reply (which is outrageously inappropriate but you don’t actually have any control here), and hope you remember in 20 minutes while you are horribly overworked.
ISTM the patient’s family should consider suing the software vendor.
Kind of my thinking here... the fact that popups are abused for BS doesn't mean that they don't serve a purpose and this is one of those times.
The fact that it was confusing and a second Doctor went ahead and bypassed it altogether that hadn't seen the patient is a bit problematic to me.
Beyond this, as much as I empathize with the family here, people still die... I had two similar ER visits in my life and it's only chance I wasn't sent home both times... when I came down with Guilliam-Barre and when I had ketoacidosis. The former, was about to be sent home when one of the residents recognized my symptoms (sudden onset weakness all over). In the latter, urgent care said it was just a cold/virus etc.. but I couldn't even keep water down, after 4 days I couldn't stand up and called for an ambulance I dropped over 50# of weight in under a week. I remember thinking to myself "if I go back to sleep I won't wake up."
> The fact that it was confusing and a second Doctor went ahead and bypassed it altogether that hadn't seen the patient is a bit problematic to me.
Doctors are humans, and humans have cultures. In this culture, it was normal for people to dismiss popups that seemed to be distracting, and one human shared with another human this useful tidbit of information.
People need to get real about who doctors are. They are not super-people.
The system was set up in such a way that the popup became a distraction, and humans naturally want to get rid of distraction.
I’m really shocked that nobody stopped and thought “doctors have special training and should be expected to diagnose and treat Sepsis if reminded about the possibility.” I don’t think it was appropriate to block the entire flow, and I’m surprised nobody thought it was possible to want some but not all of the actions for sepsis. If the developers are really making popups that make life or death decisions then maybe EMR customizations should have a tighter safety class.
If it were me, I would have also named the developer or administrator who implemented the popup. It sounds like a key stumbling block in a medical device.
> that draws product designers to keep reaching for that horrible UX pattern
Because in almost every GUI library, they are the default built-in "alert" setup that can be used. Almost any other alert system other than a "popup" has to be coded, and the designers and coders take the easy way out and just use the "built in popup widget" already in the library.
It's more that modal alerts are easier to reason about. With some other kind of notification you'd need to think about what the user should or should not be able to do until the alert condition is dealt with and what the consequences of dong it without dealing with the alert should be. It's generally easier to make the user deal with the special condition now so that the rest of the code can assume it either doesn't apply or has been dealt with.
In rare cases that's not possible; four that I know of:
- the action is controlling something in the physical world such as a CNC milling machine, or
- the action involves acting on somebody else's computer, for example sending an email, or
- the action involves securely deleting information so that future adversaries who obtain your data storage medium can't recover it, or
- data storage space is so limited that you don't have room to log an undo record.
Obviously none of those were the case here, but when there are, there are well-known techniques for reducing the risk. For example, you can include by default a "cooling off" period to cancel the email send in or restore from nightly backups, or figure out how to do a "dry run" without a cutting tool in the chuck to see if it looks like your CNC program is going to smash the mill.
You can put a molly-guard over inevitably destructive actions; that's why the IBM PC's reset keystroke is a three-key chord, and Emacs asks for you to type "yes" or "no" rather than "y" or "n" in certain cases. (Although in many of Emacs's cases, being able to undo would have been better. Freeing up the memory of a closed buffer with unsaved changes, for example, could almost always wait a few minutes!)
It's also important in such cases for the user to be able to clearly see all the relevant information.
In this specific case it seems a particularly poor implementation of a classic ordered and/or nested Checklist. If the order matters, remind people checking later steps that they missed a previous step. If nesting matters and there's some if->then logic in the checklist based on the answer, add more checklist steps as necessary. If neither the order or the nesting matters, then why is it a modal step?
You could simulate the destructive actions instead of asking beforehand and make user confirm all destructive action before proceeding to actually executing them.
This is why I think that we're getting to the point where software developers need to be trained in ethics and licensed in the same way as professional engineers are around the world. The people building the tools need to be held accountable when design choices kill people.
Most software development is not much like engineering. That's probably the right choice for low-stakes consumer products, but wouldn't be accepted in avionics and shouldn't be accepted in medical software that impacts life safety.
The hospital software that I'm familiar with is absolutely terrible and would never have been accepted in an avionics environment. It is an endless pile of nightmarishly wrong UI/UX choices coupled with opaque documentation and updates that confuse the users. They also make money hand over fist.
Ethics is applicable to many situations outside of life critical systems. Engineers take ethics very seriously because it is important, and I think it is important for software developers as well. The first time I worked closely with a Professional Engineer on a project, it was eye opening as to how the concern of doing things the ethical way shaped the interaction... Everything from identifying potential business conflicts to ensuring data collection for inputs to the design process was done correctly and traceable.
Corporations already push enough random HR requirements on tech employees. I can't see how having a semester or 2 of ethics courses is particularly onerous.
I think including ethics in the curriculum at both the high school and university level is a great idea. I do not think requiring software developers to be licensed is a good idea at all.
Safety-critical software projects should have a licensed engineer in a supervisory role.
Licensing takes many forms. Some are onerous, some are not. Working At Heights training takes 0.5-1 day of training every 3 years where I live, and it's mandated because it saves lives. Yes, it's a form of licensing, but it's an easy box to check because it's accessible.
For example: running experiments on humans without their prior consent is considered unethical. However, large tech firms routinely run A / B tests on their users without providing Informed Consent. If software developers were trained like engineers, they would be ethically obligated to obtain Informed Consent prior to engaging in this kind of conduct. More importantly, when software developers realize they were being experimented upon (as happened in the Linux kernel community a while back), they were justifiably outraged.
I know this isn't going to be a popular position to hold here, but there's a lot of harm being done by unethical practices that are currently widespread in big tech. Ephemeral ads that prey on the elderly would be considered Not Good by anyone that has seen a parent fall prey to them, yet there doesn't seem to be any concern whatsoever amongst the industry giants themselves to prevent this practice. So long as an ad brings in money, it's good to run seems to be the bar for advertisers at present, and I don't think that's good for society.
When there are no ethical considerations given to the consequences of an action, unethical outcomes are inevitable. Fixing that starts with learning, which is something every great developer already does.
> More importantly, when software developers realize they were being experimented upon (as happened in the Linux kernel community a while back), they were justifiably outraged.
Wait, what happened here? Do you have more details?
Name a consumer product that doesn’t impact life safety.
Phone notifications do. Logging out software while someone’s driving does. Every button on your car impacts life safety. Every interlock in your kitchen does.
Remember, each of these things gets rolled out 100M’s of times, so one-in-a-million scenarios kill 100’s of people.
To an extent, but only if the software meets the criteria for being legally classified as a "medical device". And at the lower classification levels the regulations are very light. You don't necessarily have to prove any level of reliability or usability.
Ex employee of the division of imagine devices and software reliability here. I have nothing to add, and I'm not qualified to confirm or deny your comment. Just interesting to see a mention of my past life
But thats part of the problem, as well as part of the solution. The FDA regulations say the software must do this and must do that, and you end up with thousands of pop up alerts. I think the regulations are well meaning but they way they translate into software is not realistic. In the article an FDA-regulated pop-up is part of the story.
medical opinion is that sepsis was not the cause of death despite the family's insistence.
it is likely a rare condition that the doctors missed. this case is sad but being fixated on one diagnosis and building the case around that is just trying to pin blame.
It may be that the ER's rapid throughput meant they adhere to the heuristic quoted in the article
"When you hear hoofbeats, think of horses, not zebras."
-- Anon. (saying in medicine)
whereas here, it was the (more rare) zebra, but nobody could take the time to do DD (differential diagnosis, i.e. to tease apart what can and cannot be the case).
This is stated as the likely result in the article, complete with the quote.
The article notes that the ER, at that stage of the visit, is not tasked with a diagnosis but deciding whether to admit to the hospital or discharge.
The complaint is that, sepsis or not, horse or zebra, the symptoms presented were severe enough to warrant further tests (such as a chest x-ray, if for no reason other than to rule something out) as part of a hospital admission. Those tests might also have been inconclusive, and the patient might still have died, but it would have at least reflected the severity of what was presented.
Instead, in a chaos of paperwork and a supervising physician who overruled every warning flag in favor of discharging the patient, the kid got sent back to die alone in his dorm room.
This article brought up the sickening memory of taking my frail elderly mom to the er with similarly severe-but-general symptoms.
The er was crowded and the hospital was crowded. The er clearly couldn’t treat her beyond basics, she needed to be admitted and monitored. But, as a sympathetic resident told me before mysteriously disappearing “there’s no space upstairs, I’ll try to get your mom in somehow”.
I worked with EHRs at the time and knew how to advocate. They kept trying to discharge my deeply ill mom without explanation and bumping into my objections, I was talking to a different nurse or social worker or resident every 3 hours round the clock. I felt scared to leave even for a short time lest they expel her.
In the end, I needed to go home to sleep and they discharged her at 6am, and when I arrived they had her bundled up and already waiting to be taken home, shivering and ashen. All they told me was that there’s no diagnosis, no reason to admit her and no beds anyway, she just needs to rest and have fluids, try urgent care if needed.
Multiple social workers sympathetically assured me and my mom’s aide that we were good people for being up to taking care of my mom at home, so they could tick a discharge box. We emphatically were not.
In the end, eventually, she was ok. The experience was harrowing. Many people talked to me but no one engaged with us, the interest was clearly in getting my mom out. It felt cruel and uncaring.
I’m surprised the article doesn’t address the “refusal to admit” angle. It used to be that you could admit patients for care and monitoring without a diagnosis, but this simply isn’t a thing anymore. So, deeply ill people who for whatever reason don’t have access to adequate care and monitoring from a caretaker at home are simply surrendered to their fate.
Hospitals are fundamentally paper pushing bureaucracies that systematically don't give a fuck about patients, and the "case managers" and "social workers" are the worst of the lot. A hospital gave my dad covid in the middle of 2020, then insisted he be discharged still covid positive and unable to stand on his own (the PTs had just let him languish). That was some great public health policy right there. The only consolation was that it was summer so I could keep good airflow through the house. I avoided getting it, as far as I could tell. Found a freelance aide that didn't mind the rona, and we eventually got him back on his feet. And once you see how abjectly horrible this system is, you can't avoid noticing all of those terrible dynamics even in much lower-stakes interactions.
They key element was the introduction of financial decisions into healthcare. This is all in the name of 'efficiency' and never mind the body count. The best thing that can happen to a health care worker is to be subjected to the system themselves.
The health care worker is not the one in charge of these decisions though. I don't consider an executive or administrator a "health care worker" so maybe that is who you mean?
IME, health care workers (doctors and nurses) often defend many of the habits and attitudes encouraged by the system and hated by patients, because the pressure on them has banded them into an "us vs them" or "soldier vs civilian" mentality. Also because the current situation has kicked off a cycle of antagonism between them and patients and especially the people advocating for said patients.
> Also many families try to push elderly to hospital for few days.
And rightly so. As people age and their health deteriorates, often a few day's monitoring and nursing care can forestall downward spirals, or catch sudden downturns so family can react appropriately, instead of the usual sudden crises and desperate scrambles. This is nursing, and hospitals used to do it, which I think is what your comment implies. But these days, without the right golden-key "diagnosis", nothing happens, and those diagnoses are certainly deployed "strategically".
There's no other accessible institution that really provides this kind of care, thus dumping responsibility on the beleaguered "community", as the social workers call it. If you happen to have insufficient wealth and willing empowered family to take care of this, I strongly suggest not getting sick. Or old.
Read all the issues with his diagnosis. One way or another the staff wasn't doing what the record says they did. How could you possibly get to the diagnosis if the tests your claim ordered was never done?
I am an inpatient RN. This conversation is interesting for several reasons: laypeople, unless in hospital as a patient frequently, will have no idea how the "systems" in a hospital intertwine and work together - if we're lucky enough to have them do that. (I wish medical professionals had the energy to talk more about the minutiae of their work just so people better understood.) The "systems" I'm talking about are specialties and roles as well as computer this and technology that. Ethics taught to administrators as well as coders would change a LOT about what we are charting - fact is, hospitals are doing more today to cut costs and avoid liability than they are doing to put patients first. This is obvious just by the fact that they keep declaring they are putting patients first. Rule of thumb: the more the thing is advertised, the less likely it is true despite the understanding that THAT thing is extremely important to their target audience. (not to mention outcomes and statistics show who is getting good care and who is not -lots more to do with finances and financial resources than evidence-based practice, seems to me.) To the point of this article: While I'm charting, I get pop-ups CONSTANTLY. Most are from the software company wanting to give me a tour of the programs features while I'm literally just trying to find a note written by physical therapy, or chart vital signs - I would LOVE to have a proficient skill in navigating EPIC but no hospital where i've worked has given me training beyond the first week. No return to chart training after I've worked with their system for a while, later, when I would know what I'm looking to improve. Other pop-ups are for sepsis alerts as this discussion is about, or (for a RN) fall alerts or skin alerts... All the things that are part of my job and training to be regularly assessing. Here is my summary: What I have seen, for docs and especially for RNs is very obviously just micro-managing to insane degrees of interference, when what works to achieve the right process and best outcome is solid training, retraining, accountability by real people and with respect, and excellent leadership. I DID once work at a hospital where they had enough staff to follow up, answering questions staff had (nursing care or software hacks) and -most important- leadership that did not intimidate, and that could and would talk with the staff member not meeting standards, in real time, getting them past whatever hangup or misunderstanding gets in the way of excellent practice. Follow up and follow up again, with the attitude of teaching (versus punitive micro-management by those whose priority is the bottom line) and supporting the staff to do the right and best thing. Pop-ups and multiple clicks to say "ok" and "yes, I really do mean to do this thing" and "for real please confirm!" add too many wasted minutes, interrupting my thought processes every day day when seconds count for someone's life-saving treatment. I don't know if there is any other industry outside of medicine and nursing where the institution itself literally just adds one road block in front of another, keeping us from focusing and doing what our critical thinking and training have taught us to do. ...don't get me started on Moral Distress and Secondary Trauma because of ignorance around letting us care. . .
did you read the article? i have a medical background and his hematology results does not support sepsis. the family pointing blame at the hospital for ignoring the sepsis automated warning is barking up the wrong tree and probably why the hospital ignored them.
not saying the hospital is faultless because they clearly failed in this case but as in any courtroom if you charge a criminal with the wrong crime you are bound to lose.
I think the real implications are much more chilling. As much as we like to believe otherwise, there is always a chance that a seemingly-healthy college kid will drop dead of something that even the best doctor wouldn't have anticipated.
And as much as we would like to believe otherwise, the modern healthcare system is riddled with problems that no technology or checklists will fix. It doesn't take someone's death to verify this- just go read your own charts and discharge papers. Even for something relatively routine there are bound to be inaccuracies. Doctors know this, which is why they spend so much time doing handoffs and interviewing patients.
We pretend that the medical 'record' is infallible, helping to reduce the mental load on doctors while protecting them from liability. But as this case shows, the 'record' is both inaccurate and not useful in showing fault. It's a paper tiger. I'm not saying we should scrap the whole system, but I do think it needs to be examined in a data-driven manner.
As a surgeon, one truly humbling fact about humans is we are simultaneously incredibly fragile and impossibly resilient. You will be shocked at what people can survive and what flimsy things kill people
Almost 30 years ago I went to a Penn and Teller magic show. They did their bullet catch (classed up but basically the same trick people have been doing for a century) and other stuff I don't really remember.
The trick I vividly remember was just Penn standing behind a table, putting a piece of green cloth (like a surgical thing) over a water balloon, and then giving a long speech of all the damage that friends of his had survived, as he stabbed the balloon (under the cloth) repeatedly in time with his speech, and talked about the wonder of medical science, and how doctors he knew had saved people from all these horrendous accidents and damaging the balloon in sync with every example.
And then he removed his hands from the table, holding them up to the audience, leaving the balloon still under the surgical barrier, and said "And the other thing that doctors will tell you, if get a couple of beers into them, is that sometimes people just die for no reason at all." And the balloon collapsed right on cue.
I can't seem to find a video of it but I remember it clearly.
Technology can fix plenty. I have narcolepsy and was able to diagnose myself after multiple doctors had failed, and luckily I finally found a specialist in narcolepsy who could actually confirm it. Wouldn't be possible without Google (which any of those docotors could have used but didn't). I would probably still be searching for a doctor to actually do his job. Technology can solve plenty of problems. What we need to do is get the doctors out of the loop.
And for every person who correctly diagnoses themselves despite at odds with what a doctor is telling them, you have 10 people undergoing unnecessary, dangerous treatments because the misdiagnosed themselves.
I was in my case (and I didn't even have AI back then) after multiple doctors failed miserably. Of course I needed to get the actual doctor diagnosis to get insurance to pay for my treatment, but that's bureaucracy, not medical science.
No. I can transcribe every interaction with 100% accuracy, but if those notes aren't used in any way it's not data-driven. This article shows that the notes are inaccurate, suggestions using the notes are routinely ignored, and that doctors and legal review think this process is acceptable. There is no professional or legal liability if the records are wrong. And yet if you talk to a medical professional they'll explain that the records are to establish a legal paper trail if anything goes wrong.
Some executive(s) have been told that detailed medical records are the solution to so many problems in modern medicine. But they lack either the guts or the expertise to make sure that these systems are actually accomplishing what they set out to do.
>suggestions using the notes are routinely ignored
Of course. The records are known to not be 100% accurate. Any conclusion you derive from them will be faulty.
>There is no professional or legal liability if the records are wrong.
Again, of course. In many cases it may not even be possible to show a record is incorrect. For example, if the record doesn't say a test was performed, but the patient insists that it was, is the record wrong, or is the patient mistaken? Or a doctor could incorrectly write down something that only he saw, such as a blood pressure value on a gauge.
I would guess a key obstacle to eliminating all these inaccuracies is that doctors don't see strict record-keeping as actually useful in helping patients. Every minute that they're taking notes of dubious future utility is a minute they could spend seeing a patient.
The real issue is the administration of the hospital sees every minute the doctors spend taking better records instead of seeing another patient as a loss of ability to bill someone's insurance for that time.
I'm sure there's many doctors who would like to take better notes if they were allowed the time to do so.
Maybe the case for better records reducing costs to insurance by assisting in prevention / early intervention is a path forward?
People die of missed sepsis all the time, so if you want to lean on your "medical background" to claim this was not the case, you better elaborate what exactly that background is
OK fair enough, but the detail in the article about the hospital information system and 'Note bloat' are still very interesting to me. I've seen stories like this before - when everything triggers an alert, people start ignoring the alerts. (edit: it was this story: https://medium.com/backchannel/how-technology-led-a-hospital... )
> The signatures of doctors who testified they never saw Sam — including one who was not in the hospital that night — accompany notes.
I'd bet 10 to 1 this is due to residents or fellows copy-pasting prior notes forward. An extremely common albeit rarely problematic practice that is nevertheless lazy and underpoliced.
As someone at an academic institution, this drives me insane. I mean, have some sort of pride in your work. Copy forward but make the necessary changes!!
As someone who’s worked on EHRs, I can tell you that doctors rely on “carry forward” and templated notes and other entry types.
They are fixated on fast and smooth workflow, both because they want the computer out of their face, they want software to help them jump through the ever growing number of mandatory hoops, and they are under serious pressure to keep their numbers high.
Doctors that I know don't necessarily mind carry forward notes if it is in a standard template, and it's been reviewed for accuracy. The doctors I know rightly hate when trainees (or occasionally but rarely other attendings) do either of these:
1. Carry forward non-templated notes that reuses another physician's prose with zero or near-zero updates as it invites mis-interpretation from colleagues.
2. Carry forward templated or non-templated information that clearly hasn't been reviewed, best indicated by signatures/names of physicians that had nothing to do with the note as in this article or dates that are clearly wrong.
And it's doubly-bad when it's non-templated notes with incorrect physician names and/or dates.
Just to be precise. The sepsis alarm was not blocking x-ray or antibiotics but rather suggesting them.
“””
Agyare had instructed Banerjee to hydrate Sam right away but to wait for the results of Sam’s lab work before ordering a chest X-ray or the strong antibiotics used to treat sepsis.
“””
Most sepsis alert implementations ironically do block review of the data to see if the sepsis is real, what triggered the alert, and what treatments are appropriate. Part of the sepsis recommendations always proposed by the EMR is to give lots and lots of IV fluids, even if the patient is in decompensated heart failure which would make it worse
The question should be why was he trying to disregard the orders that were part of the standard protocol for a possible sepsis situation - just accept them all as intended.
The system was poorly designed; as pointed out elsewhere: ordering antibiotics before lab results come back is a bad practice. However, the particular sepsis popup required antibiotics to be ordered, and lab results hadn't come back yet, but another procedure (x-ray) needed to be ordered immediately.
Hard disagree. The information system and its popup was vital information that the attending failed to act on. This anecdote serves as an example of user error and incompetence. And I’m a huge hater of popups.
What? It's simple enough that it's taught to EMTs with ~160 hours of education (I'm a paramedic and EMS instructor and evaluator):
Temperature <96.8 or >100.4
Heart rate >90
Respiratory rate >20
WBC count > 12000
Add confirmed or suspected source of infection. Simplified, each additional match increases the suspicion.
The issue, as described later, is not that it's hard to spot, or the SIRS criteria tool didn't flag it, it's that the doctor didn't do their job or document correctly.
"VS were ordered and not taken"? What kind of ER is this?
My most recent septic patient had literally zero SIRS criteria, and I strongly did not suspect a bacterial source of infection. Isolated hypotension, nothing else. Said she felt great and asked to go home
To date she's grown out GPCs on 8/10 blood cultures.
Sepsis actually is hard.
Also, it sounds like a bacterial source was not suspected in this case (and was validated by the cultures being negative and the autopsy), so AFAICT this was not sepsis.
> Also, it sounds like a bacterial source was not suspected in this case (and was validated by the cultures being negative and the autopsy), so AFAICT this was not sepsis.
Am confused by this. Sepsis can be a response to bacterial, viral, or fungal infection, no?
Let's do a chronological analysis of some prior definitions of sepsis.
The first one, from the 1990s, utilized an elevated white blood cell count plus three clinical variables (temperature, heart rate, and respiratory rate). This definition is very broad; statistically speaking, it's very sensitive but has low specificity.
The most recent definition describes sepsis as 'life-threatening organ dysfunction caused by a dysregulated host response to infection.' Septic shock is defined as a subset of sepsis in patients who have a vasopressor requirement and a lactate level greater than 2 mmol/L.
Scores such as NEWS, SOFA, and qSOFA exist, but they primarily assess disease severity and prognosis for patients who are already in a hospital setting.
It is very important to always maintain a high degree of suspicion for sepsis, but it seems to me that few clinicians would have had a strong suspicion of it in this case...
You're right, and I should be more specific - what I referred to was things like SIRS, SOFA, and qSOFA (which was being introduced at the BLS level around here in 2022) for "suspicion scores".
But even then:
> but it seems to me that few clinicians would have had a strong suspicion of it in this case...
Tachycardic, febrile and with a suspected infection?
The issue here seemed to me to be two-fold, misdiagnosis of a viral infection versus bacterial, but in the setting of treating for a bacterial infection to then be consciously overlooking multiple markers for sepsis?
I 100% get where you are coming from; however, in the ER setting, multiple non life threatening conditions could cause these symptoms (I assume paramedics don't get as much experience in in-hospital settings, please correct me if I am wrong as I am not from the US!)
That said, there was a cascade of errors that ultimately led to this very very unfortunate outcome (that maybe could have been prevented, or maybe not)
Had the chest x-ray been ordered perhaps his enlarged heart could have been noticed in due time.
I do not mean to be pretentious in any way. I hope my english does not interfere with this.
You're absolutely fine! Definitely a lot other things can also have those influences on temperature, HR. Our EMS system started teaching SIRS (and then qSOFA) to EMS providers to have them tell the charge nurse that the patient "meets Sepsis criteria", because this was being missed in triage and septic patients were spending hours on hours waiting for care.
Paramedics don't get very much in-hospital time here, other than getting intubation experience in the OR with an anesthesiologist.
I appreciate your input, it does not come across at all pretentious - and as a paramedic, I would never want to "not learn more medicine".
An automated alert popped up warning that the doctors should consider Sepsis. That alert essentially then blocked progress, and the doctors ended up (essentially) ticking the 'not sepsis' box so that they could get on with their (reasonable) next step which was either ordering an x-ray or starting antibiotics. Then somehow after that, sepsis did not get re-considered.
https://archive.is/tJePt#selection-1465.0-1491.52
It was Banerjee’s task to document Sam’s care, and as he began to do so, a pop-up appeared on his computer screen. Sam’s fever and heart rate had triggered an automated warning for sepsis, a potentially life-threatening condition in which the immune system has a dangerous reaction to an infection. It requires speedy intervention. To help the hospital comply with state-mandated sepsis regulations, the pop-up provides a checklist of tests and orders used to identify and treat sepsis.
Agyare had instructed Banerjee to hydrate Sam right away but to wait for the results of Sam’s lab work before ordering a chest X-ray or the strong antibiotics used to treat sepsis.
But Banerjee, a novice, got stuck. He couldn’t figure out how to navigate the template to make some but not all of the auto-populated orders. “This was my first patient that triggered the sepsis pathway,” he explained, in testimony. So he asked Connor Welsh, a third-year resident, for help.
At 8:50 p.m., Welsh showed Banerjee how. From his own computer, he clicked into a field on Sam’s chart to assert that sepsis was not likely: “Based on my evaluation,” the automated note said, “this patient does not meet clinical criteria for bacterial sepsis.” And then Welsh recorded what Banerjee said Agyare had said earlier: “Likely viral syndrome. Workup pending.” Welsh’s name appears on the note, but in his deposition he said he never interacted with Sam. Senior residents often help junior ones in this way, he said. “I signed this note based on the discussion with the provider, Dr. Banerjee, based on his evaluation and the medical management of Mr. Terblanche,” he testified.
...
Sam’s chart is 51 pages long, a catalog of billing codes and abbreviations, check-boxes and shorthand, updates and addenda. The record of the second visit contains numerous contradictions: Sam’s heart rate was documented at 126, yet Banerjee clicked the box “normal.” In one place it says Sam didn’t have a cough, while in another it says he did. The signatures of doctors who testified they never saw Sam — including one who was not in the hospital that night — accompany notes. Vital signs were ordered and not taken, as was an EKG.