Human Factors and Cognitive Safety in Clinical AI in Viet Nam: Managing Automation Bias, Overreliance, Alert Fatigue and Deskilling
IMPLEMENTATION INSIGHTS
Human Factors and Cognitive Safety in Clinical AI in Viet Nam: Managing Automation Bias, Overreliance, Alert Fatigue and Deskilling
IMPLEMENTATION INSIGHT
Last reviewed: 3 October 2026
A clinical AI system can be accurate and still be unsafe in practice. The problem may not be a defective model. It may be the way a recommendation is presented, the time available to review it, the number of competing alerts on the screen, the authority a clinician feels to disagree, or what happens to professional skills after months or years of repeated reliance on automation.
A clinician may accept an incorrect recommendation because the system is usually right. Another may begin ignoring alerts after seeing too many low-value ones. A junior clinician may have fewer opportunities to develop a skill that AI routinely performs. An experienced clinician may recognize that an AI output is wrong but hesitate to override it because the technology has become embedded in institutional workflow. These are human-factors risks, and model accuracy alone cannot address them.
They also explain why keeping a clinician formally “in the loop” should not be treated as proof that clinical AI is safe. A 2026 framework in npj Digital Medicine argues that meaningful oversight depends on four interlocking conditions: epistemic capacity, cognitive space, decisional authority and intervention effectiveness. In practical terms, people need to understand what they are reviewing, have enough time and attention to review it, have real authority to disagree, and be able to intervene before an unsafe AI-mediated action affects care.
An emerging concept goes further: cognitive safety. Rather than asking only whether AI improves today's decision, cognitive safety asks whether repeated AI use preserves the clinician's ability to reason independently, recognize uncertainty, challenge incorrect automation and continue functioning safely when AI is wrong or unavailable. A 2026 article proposed cognitive safety as a longitudinal property of the clinician-AI-organization sociotechnical system. It remains an emerging human-factors concept, not a term defined in Vietnamese AI legislation.
For healthcare organizations in Viet Nam, this creates a practical governance question: How can AI improve clinical work without quietly weakening the human capabilities needed to detect its failures?
Human factors are part of clinical AI safety, not separate from it
Clinical AI is often evaluated through measures such as discrimination, sensitivity, specificity, calibration, error rates or workflow efficiency. Those measures matter, but they describe only part of what happens once a system enters clinical care. In practice, performance becomes a property of a human–technology–workflow system. The same model can produce different consequences depending on who uses it, where it appears in the workflow, how its output is presented, whether independent judgment occurs first, how much time the clinician has and whether disagreeing with the AI is operationally easy.
A 2026 scoping review of 140 empirical studies of human–AI collaboration in healthcare found that evidence was concentrated in diagnostic interpretation, with fewer studies in screening and triage, therapeutic decision-making and administrative workflows. Benefits frequently depended on task fit, workflow integration, training and appropriately calibrated trust. The review also found that accountability and patient-safety issues were often discussed but rarely evaluated empirically.
The implication is important. Evidence that an AI system performs well on its own does not automatically show that clinicians using it will perform better. Conversely, concern about human-factors risk does not mean AI inevitably makes clinical decision-making worse. Well-designed collaboration can improve performance. What increasingly matters is the human–AI team operating in the real workflow, not the model in isolation.
Automation bias and overreliance are problems of calibrated trust
Trust is necessary for useful technology. A system that clinicians consistently distrust is unlikely to add much value. The patient-safety problem is miscalibrated trust, relying on AI more, or less, than its actual performance and the clinical situation justify.
Automation bias describes a tendency to give undue weight to an automated recommendation or reduce independent checking because automation is present. In clinical practice, this could mean following an incorrect recommendation despite contradictory evidence or failing to notice something important because the AI did not flag it. When a system performs well most of the time, rare errors may become harder to detect because users can gradually learn that accepting its recommendation is usually efficient.
Recent evidence illustrates why this matters. A multicenter study published on 24 September 2026 evaluated junior clinicians' ability to identify deliberately introduced hallucinations in generative-AI responses across simulated clinical scenarios. The results raised substantial concerns about the assumption that a clinician placed downstream of an AI output will reliably catch its errors. Because this was a simulation involving junior clinicians rather than routine clinical care, its findings should not be generalized directly to Vietnamese healthcare or to all clinicians.
Overreliance is broader than automation bias. It can develop when a workflow progressively transfers too much cognitive or operational responsibility to automation. The organization itself may unintentionally encourage this. If AI advice appears first, is visually prominent, is easier to accept than to override or is embedded in institutional expectations, clinicians may learn that following the system is the normal path. If disagreement requires additional documentation or creates delay, the interface and workflow are shaping behavior even if an override button technically exists.
Workload matters as well. A clinician may have theoretical authority to challenge an AI output but not enough cognitive space to reconstruct the reasoning independently under real-time pressure. This is one reason meaningful oversight cannot be reduced to the presence of a human name beside the final decision.
Workflow sequence can also influence reasoning, but there is no simple rule that AI should always appear first or always appear second. A 2026 randomized controlled trial involving 70 clinicians compared AI-first and AI-second diagnostic workflows. Both AI-assisted workflows improved diagnostic performance compared with conventional resources, while overall performance did not significantly differ between the two AI workflows. The study also showed that workflow structure changes how clinicians and AI influence one another, reinforcing the need to evaluate sequencing for the actual task rather than assume one universal design is safest.
The appropriate sequencing may therefore differ across radiology, prescribing, triage, clinical documentation and generative decision support.
Alert fatigue can turn AI assistance into background noise
Alert fatigue is not unique to AI. Healthcare has struggled for years with excessive electronic warnings and decision-support notifications. AI can add another layer of predictions, risk scores, prioritization signals and recommendations to already crowded clinical interfaces.
When alerts become too frequent, poorly targeted or clinically irrelevant, users may increasingly dismiss them. Eventually, a genuinely important warning risks receiving the same attention as many low-value ones.
A 2026 systematic review in the Journal of the American Medical Informatics Association examined 22 reviews of physician-directed clinical alerts. Alert quantity, override rate and acceptance rate were the most commonly reported measures, yet only one included review provided an operational definition of alert fatigue. The authors recommended considering a significant, sustained decline in appropriate alert responses from an established baseline, rather than treating override rates alone as proof that alert fatigue exists.
This distinction matters for AI governance. A high override rate may indicate fatigue, but it could also mean the AI is generating poor recommendations. A very low override rate may reflect good system performance, but it could also reflect automation bias or a workflow in which disagreement is difficult. Alert metrics therefore need to be interpreted alongside clinical appropriateness, workload, timing and user behavior.
Not every clinical AI system produces alerts, so alert fatigue should not be treated as a universal AI risk. It is particularly relevant to clinical decision support, deterioration prediction, triage, medication safety and other functions that interrupt clinicians or repeatedly request action.
Deskilling is a longer-term risk, and the evidence should not be overstated
Clinical expertise develops through repeated exposure, independent interpretation, feedback, difficult cases and correction of mistakes. If AI routinely performs part of that work, clinicians may receive fewer opportunities to build or maintain the corresponding skill.
A 2026 scoping review of physician deskilling concluded that empirical evidence remains limited but identified evidence and concerns across specialties including radiology, pathology and endoscopy. The review emphasized reduced opportunities for hands-on learning, automation bias and the possibility that performance may deteriorate when AI support is removed, while also calling for stronger longitudinal evidence and competence monitoring.
A separate 2026 systematic review of generative AI in healthcare identified automation bias, deskilling concerns and effects on diagnostic reasoning as recurring themes. It also found that much of the available literature remained observational, simulation-based or conceptual rather than based on long-term real-world deployment.
The appropriate conclusion is therefore not that AI inevitably deskills clinicians. Deskilling is a plausible risk with emerging empirical support, particularly where automation replaces cognitive work that clinicians may still need to perform independently.
The issue may be particularly important for trainees and early-career clinicians. A senior specialist may already have developed expertise through years of unaided practice, while a trainee entering an AI-rich environment may experience a different learning pathway. Healthcare organizations should therefore ask not only, “Does this AI save time?” but also, “Which human capability will receive less practice if we use it this way?”
Cognitive safety extends patient safety over time
Traditional AI evaluation often asks whether an output is correct today. Cognitive safety introduces another question: What is repeated interaction with the system doing to the people who must supervise it tomorrow?
The 2026 proposal on cognitive safety describes it as the capacity of a clinician–AI–organization system to support or improve clinical performance without eroding independent hypothesis generation, uncertainty calibration, reasoned dissent, metacognitive control and resilient performance when AI is wrong or unavailable.
This matters because a system could create short-term improvement while increasing long-term dependency. An AI image-analysis tool, for example, might improve detection while clinicians gradually receive less unaided practice. The immediate clinical outcome may improve, yet the organization may become increasingly dependent on the technology. If the system later becomes unavailable, is withdrawn or is wrong in an unusual case, loss of independent capability may become a patient-safety issue.
Cognitive safety therefore connects directly with downtime, fallback planning and decommissioning. A fallback workflow exists only on paper if the people expected to use it no longer have sufficient skill or confidence to perform it safely.
This does not mean every task must remain manual merely to preserve competence. Some tasks can appropriately be automated. The governance question is which human capabilities remain essential for verification, exception handling, escalation and safe care when automation fails or becomes unreliable.
Viet Nam's AI framework supports human oversight, but cognitive safety is not a statutory category
Viet Nam's Law on Artificial Intelligence No. 134/2025/QH15 was adopted on 10 December 2025 and has been effective since 1 March 2026. It establishes the national risk-based framework under which more detailed requirements apply according to the system's classification and use.
For high-risk AI systems, Decree No. 142/2026/NĐ-CP requires providers to establish and maintain risk-management arrangements appropriate to intended use, deployment scope and risk. The provider's risk-management system includes, among other things, mechanisms for human oversight and intervention appropriate to the system's risk, while providers must also give deployers the information needed for safe operation and management of identified risks. Deployers have their own operational responsibilities: they must supervise system operation for errors and emerging risks, establish and maintain human oversight and intervention during use, and apply risk-control measures within their authority.
The Decree also gives useful context for what effective human control can look like. In the criteria used to determine whether a system should be proposed for inclusion in the high-risk list, one relevant exclusion concerns systems with technical mechanisms and operational procedures that enable an authorized person to independently review, intervene in, reject or adjust the system's decision before it takes effect. This is a classification criterion, not a universal statutory definition of meaningful human oversight, but it illustrates the regulatory importance placed on genuine intervention capacity.
These provisions should not be generalized to mean that every AI system used in healthcare is legally high-risk. Decision No. 33/2026/QĐ-TTg, effective from 15 August 2026, contains the current statutory high-risk list; its healthcare section currently contains two entries concerning specified AI-assisted surgical and robotic systems under defined conditions.
Human-factors safeguards may nevertheless be appropriate for many other clinical AI systems as organizational patient-safety controls, even when those systems are not legally classified as high-risk.
The National AI Ethics Framework under Circular No. 05/2026/TT-BKHCN provides another relevant governance reference. Its direct scope covers AI used for state management or the provision of public services, while Vietnamese and foreign organizations and individuals conducting AI research, development, provision, deployment or use in Viet Nam outside that direct scope are encouraged to apply it. The accompanying guidance tells users to treat AI as a support tool rather than a complete substitute for human judgment in important decisions and to verify AI information, recommendations and outputs, particularly for health-related content.
What Vietnamese law does not currently do is establish “cognitive safety” as a separate statutory category, prescribe a standardized automation-bias assessment, or require every healthcare organization to operate a formal deskilling-monitoring program.
These are organizational human-factors and patient-safety controls that may appropriately go beyond minimum legal requirements.
Meaningful oversight requires knowledge, time, authority and usable controls
A clinician cannot supervise an AI system meaningfully simply by being present in the workflow. Users need enough knowledge to understand what the AI is intended to do, where and how it was evaluated, its important limitations and the types of failure they should expect. Generic AI-awareness training is not enough for a clinically consequential system; users need knowledge relevant to the particular technology and workflow.
They also need sufficient time and cognitive space. If a doctor is expected to verify a long AI-generated clinical summary in seconds between competing tasks, human review may formally exist while providing little real protection. Interface and workflow design therefore become part of the safety control.
Clinicians also need genuine authority to disagree, and intervention must be practical. Override behavior should not automatically be viewed as noncompliance. If users believe disagreement with AI will be questioned, penalized or treated as inefficient, oversight becomes weaker. Likewise, human control is largely theoretical if an automated system has already changed a priority, generated an order, sent a communication or triggered a downstream action before anyone can intervene.
The governance question is therefore not simply, “Was a clinician present?” It is “Did the organization make meaningful independent judgment and intervention possible?”
Design workflows that preserve clinical reasoning
Healthcare organizations should consider human factors when deciding where and how AI appears in a workflow. For some high-consequence tasks, obtaining an independent human assessment before displaying AI advice may reduce anchoring. In other circumstances, early AI input may improve performance or prevent premature closure. The 2026 randomized workflow trial demonstrates why sequencing should be tested in context rather than converted into a universal design rule.
Organizations can also consider cognitive forcing functions selectively. A 2026 paper on operational safety in clinical AI discusses approaches such as making AI information available on demand, requiring a provisional clinician assessment before the AI output is displayed, or introducing a deliberate delay in selected workflows. These mechanisms are intended to interrupt heuristic acceptance and create space for analytical reasoning.
These safeguards should be proportionate. Friction added everywhere can itself increase workload and alert fatigue. The aim is not to make AI difficult to use; it is to create moments of reflection where unexamined automation could meaningfully harm a patient.
Train clinicians to challenge AI, not only to operate it
AI training often focuses on where to click, how to prompt and what the system can do. Human-factors safety requires another form of preparation: how the system can fail and what the clinician should do when it does.
Training can expose users to realistic failure cases, including plausible incorrect recommendations, incomplete summaries, misleading confidence, atypical patients and cases outside the intended or evaluated use. Simulation can test whether clinicians identify these problems under realistic time pressure instead of merely asking whether they understand a policy.
This matters because AI output can remain fluent and convincing even when wrong. The 2026 multicenter hallucination study provides a strong reason for healthcare organizations to test the assumption that clinicians will reliably detect AI errors rather than treat human review as an automatic safeguard.
Organizations should also create psychological safety around disagreement. A clinician who repeatedly observes questionable recommendations should be able to report the pattern without being told that the system has already been “validated.” Overrides, disagreements and near misses can be valuable governance signals.
The objective is calibrated reliance: clinicians use AI where it adds value, question it when uncertainty warrants it and retain the authority to place independent clinical judgment above an automated recommendation.
Preserve the human capabilities that still matter without AI
A mature AI implementation plan should identify which competencies need to remain available even when AI performs part of routine work. Depending on the specialty, this may include interpreting images without AI, independently constructing a differential diagnosis, recognizing deterioration without an automated score, reviewing AI-generated documentation critically, managing unusual cases outside intended use or completing a workflow safely during downtime.
The appropriate preservation strategy will vary. It may involve periodic unaided practice, simulation, supervised training, competency assessment, deliberate exposure to difficult or edge cases, or minimum practice requirements for skills that remain safety-critical.
Training environments deserve particular attention. AI may improve a trainee's performance on a task today while changing how that trainee develops expertise over time. The design question can therefore be different for an experienced specialist using AI as augmentation and a learner using the same tool before the underlying skill has fully developed.
Deskilling should be assessed longitudinally rather than assumed from AI use alone. Reduced manual activity is not itself proof that competence has been lost. The relevant question is whether clinicians continue to possess the skills necessary when AI advice is unavailable, uncertain or incorrect.
Monitor the human–AI system, not only the model
Post-deployment monitoring often concentrates on technical model performance. Human-factors monitoring should sit beside it.
Organizations can examine how often clinicians accept or override AI outputs, review latency, patterns of AI–human agreement and disagreement, differences across patient groups or scenarios, whether alerts are increasingly ignored, staff-reported verification burden and whether clinical performance changes when AI is unavailable. These measures should not be interpreted in isolation.
A falling override rate may mean system performance has improved, but it could also indicate growing overreliance. A rising override rate could reflect poor calibration, alert fatigue or appropriate clinical resistance. Faster review may signal efficiency, but it could also suggest superficial checking. The meaningful-oversight framework therefore recommends interpreting measures such as override frequency, review latency, concordance and discordance, subgroup patterns and AI-associated incidents as connected signals rather than standalone scorecards.
Alert monitoring requires the same caution. The 2026 JAMIA review shows why override and acceptance rates alone cannot establish alert fatigue; what matters is the appropriateness of the response and whether that appropriate response deteriorates from an established baseline.
The aim is to detect when the relationship between people and technology is changing, not merely whether the algorithm's AUROC remains stable.
A practical Human Factors and Cognitive Safety Gate
Before deploying or materially changing clinically consequential AI, healthcare organizations can ask:
Task: What cognitive work does the clinician perform today, and which part will AI perform, influence or remove?
Consequence: What happens if the clinician accepts an incorrect AI output or fails to notice missing information?
Independent judgment: Does the workflow preserve independent clinical reasoning where it matters, or does AI anchor the decision too early?
Presentation: Could confidence language, defaults, visual prominence or interface design encourage automatic acceptance?
Cognitive space: Do users realistically have enough time and attention to evaluate the output in the actual workflow?
Alert burden: Will the system add warnings, risk scores or interruptions to an already crowded environment, and how will low-value alerts be controlled?
Authority: Can clinicians disagree, override, escalate or stop the AI without disproportionate effort or organizational penalty?
Verification burden: How much additional checking does AI create, and has that work been considered in staffing and workflow design?
Skill preservation: Which human capabilities may receive less practice, and which remain necessary for exception handling, downtime or independent care?
Training: Have users practiced identifying realistic AI failures, not only normal system operation?
Learners and other user groups: Could trainees, junior clinicians or other users interact differently with the system and require different safeguards?
Monitoring: Which human–AI interaction signals will be reviewed after deployment, and who will interpret them?
Fallback: Can clinicians still provide safe care if the AI is unavailable, suspended or withdrawn?
Learning: How will overrides, disagreements, near misses and unexpected user behavior feed back into training, workflow design and governance?
The Human Factors and Cognitive Safety Gate is a practical organizational framework proposed for healthcare governance. It is not a statutory assessment method established by Vietnamese law.
What this means for healthcare organizations in Viet Nam
Viet Nam is entering a period in which AI adoption and AI governance are developing at the same time. The AI Law has been effective since March 2026, Decree No. 142 since May 2026, the National AI Ethics Framework since March 2026 and the current high-risk AI list under Decision No. 33 since August 2026.
On 28 August 2026, the Prime Minister also approved the National Artificial Intelligence Strategy to 2030, with a vision to 2045, reinforcing Viet Nam's broader policy direction toward AI development, deployment and capability building.
As clinical use grows, healthcare organizations should resist a simple assumption: if a human remains in the workflow, the human will automatically make the system safe.
Current evidence gives no basis for that assumption. Clinicians can be influenced by automated recommendations, fail to identify plausible AI errors, experience excessive alert burden and lose opportunities to maintain important skills. At the same time, appropriately designed human–AI collaboration can improve performance.
The governance objective is therefore not to maximize human control for its own sake or minimize the use of AI. It is to create a clinical system in which human judgment and AI capability remain complementary, appropriately trusted and able to correct one another when either side is wrong.
The most useful leadership question is not simply, “Is there a clinician in the loop?”
It is, “Does our workflow preserve the clinician's knowledge, time, authority and skill to recognize when the AI should not be trusted?”
That is the level at which human factors become patient safety.
Professional education notice
This article is intended for professional education and knowledge exchange. It does not constitute individualized legal, regulatory, clinical, occupational, human-factors or medical-device advice.
The terms cognitive safety, automation bias, overreliance, alert fatigue, deskilling and the Human Factors and Cognitive Safety Gate are used here as human-factors and organizational-governance concepts. “Cognitive safety” is an emerging concept in the 2026 medical-AI literature and is not a statutory category defined by Viet Nam's Law on Artificial Intelligence, Decree No. 142/2026/NĐ-CP or Decision No. 33/2026/QĐ-TTg.
Not every AI system used in healthcare is legally classified as high-risk. Internal human-factors controls may appropriately extend beyond statutory requirements where proportionate to clinical consequence, patient safety and organizational risk.
The international studies cited here come from different healthcare systems and include simulations, controlled experiments, reviews and emerging real-world evidence. Their findings can inform governance but should not be assumed to directly represent clinician behavior or the prevalence of human-factors risks in Viet Nam. Local workflow evaluation, workforce assessment and post-deployment monitoring remain important.
References
[1] National Assembly of Viet Nam. Law No. 134/2025/QH15 on Artificial Intelligence. Adopted 10 December 2025; effective 1 March 2026.
[2] Government of Viet Nam. Decree No. 142/2026/NĐ-CP Detailing a Number of Articles and Measures for Implementation of the Law on Artificial Intelligence. Issued 30 April 2026; effective 1 May 2026.
[3] Prime Minister of Viet Nam. Decision No. 33/2026/QĐ-TTg Promulgating the List of High-Risk Artificial Intelligence Systems. Issued 30 June 2026; effective 15 August 2026.
[4] Ministry of Science and Technology of Viet Nam. Circular No. 05/2026/TT-BKHCN Promulgating the National Artificial Intelligence Ethics Framework. Issued and effective 10 March 2026.
[5] Prime Minister of Viet Nam. Decision No. 1671/QĐ-TTg Approving the National Artificial Intelligence Strategy to 2030, Vision to 2045. Issued 28 August 2026.
[6] van de Sande D, Economou-Zavlanos N, van Genderen ME. Meaningful oversight of medical AI beyond human in the loop. npj Digital Medicine. 2026;9:569.
[7] Corrao S. Accuracy alone is not enough: cognitive safety of artificial intelligence in medicine. Recenti Progressi in Medicina. 2026;117(9):390–395.
[8] Zhang Y, Zhou J, Zhang Z, et al. A multicenter assessment of human oversight of generative AI outputs in simulated clinical decision making. npj Digital Medicine. Published 24 September 2026.
[9] Everett SS, Bunning BJ, Jain P, et al. From tool to teammate in a randomized controlled trial of clinician-AI collaborative workflows for diagnosis. npj Digital Medicine. 2026;9:409.
[10] Strong J, Rogers H, Sun E, et al. Human-AI Collaboration in Healthcare: A Scoping Review. npj Digital Medicine. Published 20 June 2026.
[11] Ray CE, Wilson GM, Hughes AM, et al. Alert fatigue measurement in clinical decision support: a systematic review. Journal of the American Medical Informatics Association. 2026;33(8):1523–1531.
[12] Heudel PE, Crochet H, Filori Q, Bachelot T, Blay JY. Artificial intelligence in medicine: a scoping review of the risk of deskilling and loss of expertise among physicians. ESMO Real World Data and Digital Oncology. 2026;12:100693.
[13] Al-Anezi FM. Generative Artificial Intelligence in Healthcare: Automation Bias, Deskilling, and Cognitive Implications — A Systematic Review. Journal of Healthcare Leadership. 2026;18:590498.
[14] Kim YT, Kim H, Bahl M, et al. Defining operational safety in clinical artificial intelligence systems. npj Digital Medicine. 2026;9:281.