Investigation report: Clinical decision making - diagnosis and treatment of pulmonary embolism in emergency departments

A pulmonary embolism (PE) is a blockage in the arteries of a person’s lung, usually caused by a blood clot. Patients with a PE may have a range of symptoms (health problems that the patient experiences) and signs (things that clinicians find through examinations and tests). These include the classic symptoms of PE such as breathlessness and chest pain. PE may be difficult to diagnose and, if not recognised, can cause significant harm.

The purpose of this investigation is to identify learning that could support clinicians to make decisions about the diagnosis and treatment of PE patients in emergency departments (EDs). This learning may also apply to the treatment of other conditions where patients have non-specific symptoms.

Clinicians diagnose a PE by assessing a person’s symptoms and signs, and then requesting appropriate tests. The decisions made during the diagnosis and treatment of PE can be influenced by several factors including the expertise of the staff who assess the patient, and the context within which they are working.

HSIB identified a safety risk related to delayed or missed diagnosis and treatment of PE across several areas of healthcare where patients may first seek help. A review of events reported into the Strategic Executive Information System (a national database of patient safety incidents) identified EDs as one of these areas.

At the time of investigation access to EDs was limited because of the COVID-19 pandemic, and the investigatory approach was therefore adapted. Rather than examining a specific event, the investigation reviewed 14 serious incident reports obtained from hospitals across England. This review identified the terms of reference for a national investigation which aimed to:

• examine clinical decision making in the diagnosis and treatment of PE and the role of expertise (significant knowledge and skill that supports effective and practical decision making) using an Applied Cognitive Task Analysis (ACTA)
• identify factors in the wider healthcare system that support or inhibit diagnostic decision making when staff are seeing patients with non-specific symptoms and signs that may suggest PE.

This investigation was a first opportunity for HSIB to apply a specific methodology (ACTA) to examine decision making in healthcare.

• Recognising that a person may have a PE is challenging, particularly for less experienced staff and when the person’s signs and symptoms are non-specific or atypical.
• Deciding whether to initiate treatment for a suspected PE requires a decision that balances risks, and this decision can benefit from expert knowledge and skill.
• Despite expertise and the available tools to help identify patients who may have a PE, a small number of PEs may always be missed.
• Experts use different thought processes and show different behaviours when making decisions compared to more novice staff.
• Decision-making skills in healthcare are commonly developed through experience, without formal training or opportunities to practise making decisions.
• Simulation-based learning has the potential to help staff acquire decision-making skills more quickly.
• Other industry sectors, such as aviation and the fire service, aim to accelerate the development of decision-making skills through structured training and the use of ‘generic decision tools’ for analytical decisions.
• EDs do not always provide the conditions which support the development of decision-making skills.
• Decision making in EDs is affected by workload, workforce availability, and performance targets.
• ED staff asked for further guidance to be provided on the use of decision aids to support the diagnosis of PE.
• The design of ED processes influences the decisions staff make. There is no standard model of initial patient assessment in EDs; this contributes to variation in the requesting of tests which can affect later decisions.
• Pathways for the diagnosis and treatment of PE in outpatient settings may create a safety risk where patients are discharged on anticoagulation medicines without a confirmed diagnosis; the capacity of imaging services is a significant contributor to this.
• Loss of clinical information when a patient’s care is handed over was identified as a further safety risk. This can contribute to harm if tests, such as D-dimer (a blood test used as part of the assessment of likelihood of PE), are not followed up.
• Work procedures for the diagnosis and treatment of PE are not routinely designed in line with human factors principles to support their access and use.
• The physical design of environments may also affect decision making.

HSIB wishes to acknowledge the challenges staff face in EDs when caring for patients with complex health conditions in the context of limited resource and capacity. This has been further exacerbated by the COVID-19 pandemic. HSIB recognises that these challenges mean the safety recommendations and safety observations in this report will be difficult to implement at the time of publication. Publication has gone ahead to ensure learning is shared and for the findings to be acted upon when capacity allows.

1.1.1 Venous thromboembolism (VTE) is a collective term for the formation of blood clots (thrombus) which can travel through and then block (occlude) a person’s blood vessels. This process is known as embolism. VTE includes deep vein thrombosis (DVT, blood clots usually in the lower leg/calf) and pulmonary embolism (PE). A PE is a blockage in the arteries of a person’s lung usually caused by a blood clot.

1.1.2 The incidence (occurrence rate) of PE per year is around 60 to 70 cases per 100,000 people (Howard et al, 2018). The individual risk of a person developing a VTE is influenced by their risk of forming blood clots.

PE risk factors and signs/symptoms

1.1.3 A PE is the partial or complete blockage of single or multiple pulmonary (lung) blood vessels, usually from a DVT that travels from a person’s leg to their lung. There are multiple factors that increase the risk of a person developing a PE, these include (Clarity Informatics, 2020):

• cancer
• immobility
• inflammation including infection
• inherited disorders of blood clotting
• major injury or injury to the lower limbs
• obesity
• oestrogen containing medicines
• pregnancy
• previous VTE in the individual or their family
• surgery, particularly bone and emergency.

1.1.4 Clinicians use symptoms (health problems that the patient experiences) and signs (things that clinicians find through examinations and tests) to help them make a diagnosis. The classic symptoms of PE (see table 1) include pleuritic chest pain (pain that is worse on breathing in), shortness of breath and haemoptysis (coughing up blood). However, patients may also show symptoms and signs that are few, unexpected (atypical), or do not point at a particular diagnosis (non-specific). Patients may also arrive at an emergency department following a collapse and cardiac arrest. This variation can make diagnosis challenging.

Diagnosing PE

1.1.5 The National Institute for Health and Care Excellence (NICE) provides guidance on the diagnosis and treatment of adults with PE (National Institute for Health and Care Excellence, 2020a). NICE states that patients seeing a clinician with symptoms and/or signs of PE should have a full medical history, physical examination and chest X-ray.

1.1.6 Where there is a suspicion of PE, NICE recommends the use of a two-level Wells score (see table 2). The Wells score is a validated tool to help predict the likelihood of a PE when used in conjunction with a D-dimer blood test (see 1.1.7). Where the Wells score is more than four points, PE is likely and patients should be offered radiological imaging (a scan). Where the Wells score is four or less, PE is unlikely and a D-dimer blood test can be considered.

1.1.7 D-dimer is a protein that is made when a blood clot breaks down. A D-dimer blood test checks for the levels of this protein in the blood. This test is widely available in hospitals and results are usually available within a few hours. A negative D-dimer test result suggests no PE is present in a patient who is thought to be unlikely to have a PE (based on Wells score). If the D-dimer is positive, NICE recommends that the patient has a scan to look for a PE (National Institute for Health and Care Excellence, 2020a).

1.1.8 D-dimers are not specific to PE. A person’s D-dimer levels can also be raised by factors such as age, ethnicity, pregnancy, recent surgery, certain bleeding disorders, cancer, heart disease, trauma, aortic blood vessel problems and COVID-19.

1.1.9 The main imaging technique used to support the diagnosis of PE is computerised tomography pulmonary angiography (CTPA). CTPA provides images of a patient’s pulmonary (lung) blood vessels. Where filling of blood vessels is not seen, this can suggest a PE (see figure 1). The ventilation (V)/perfusion (Q) scan is an alternative for certain patients who cannot or should not have a CTPA (National Institute for Health and Care Excellence, 2020a). V/Q scans were not considered in this investigation.

Figure 1 CTPA images showing a possible pulmonary embolism (Choong et al, 2008)

Treating PE

1.1.10 Anticoagulation medicines are the main treatment for suspected and confirmed PE. These medicines reduce the formation of further blood clots. Due to the risk of clots increasing in size while patients are undergoing tests for suspected PE, NICE recommends interim treatment with anticoagulation in certain circumstances (National Institute for Health and Care Excellence, 2020a):

• if a patient has a likely PE and cannot have an immediate scan
• if a patient is unlikely to have a PE (they have a Wells score of four or less) and a D-dimer result cannot be obtained within 4 hours.

1.1.11 After a diagnosis of PE is confirmed, patients continue treatment with longer-term anticoagulant medicines. Other treatments for life-threatening PEs include medicines to break down clots (thrombolytics) and procedures to directly remove clots.

Outpatient diagnosis and treatment of PE

1.1.12 Traditionally patients with a suspected PE would have been admitted to hospital. More recently there has been a move to managing many conditions outside of hospital when patients are well enough to be at home. This improves patient experience and reduces demand on hospitals.

1.1.13 In 2018 the British Thoracic Society (BTS) developed guidance for the initial treatment of patients with PE as outpatients (Howard et al, 2018). NICE recommends considering ‘outpatient treatment for suspected or confirmed low-risk PE’ and recommends using a validated tool to determine patients who are low risk of ‘poor outcomes’ from a PE (National Institute for Health and Care Excellence, 2020a). The BTS provides suggestions on what validated tools can be used (Howard et al, 2018).

COVID-19 and PE

1.1.14 During the COVID-19 pandemic it became apparent that the virus affected the formation of blood clots. COVID-19 can lead to patients developing PE, and other clots, through immune-related clot formation. The virus can also result in increased D-dimers in the blood because of inflammation.

1.1.15 Studies have explored the presence of PE in COVID-19 with variable results. These have been summarised in various reports and systematic reviews (for example, Tholin et al, 2021; Jiménez et al, 2021; Malas et al, 2020). Clot formation is likely related to the severity of a patient’s COVID-19 infection.

1.1.16 The symptoms that a patient may have with COVID-19 may be similar to those of a PE. COVID-19 patients may be breathless and have a cough. This can make distinguishing between the diagnoses difficult. As described above, patients may also have COVID-19 and a PE at the same time.

1.2.1 This investigation is focused on type-one emergency departments (EDs). Type-one EDs are consultant-led, available 24 hours a day and can treat any emergency patient. As of 2019/20 there were 134 hospital trusts with type-one EDs in England (NHS Digital, n.d.).

1.2.2 In England, at the time of writing, most EDs are directed by a national operating standard under which 95% of people attending an ED should be admitted to hospital, transferred or discharged within 4 hours of arrival (NHS England and NHS Improvement, 2020). In December 2021 there were 1,250,669 attendances at major EDs in England with 73.3% of attendances meeting the national operating standard (NHS England and NHS Improvement, n.d.).

ED processes

1.2.3 When patients arrive at an ED, staff will first prioritise and direct them to the most appropriate place for treatment. This happens through an ‘initial assessment’ process (see figure 2) and triage. Initial assessment aims to rapidly identify those patients who require immediate treatment, and also helps to ensure patients are seen and treated quickly.

1.2.4 Following initial assessment, patients will be diagnosed and treated, and then discharged or referred on for specialist care. Patients may be seen by an ED doctor, advanced clinical practitioner (ACP) or general practitioner (GP) in the ED.

PE pathways

1.2.5 To support the standardised assessment of patients with different symptoms, EDs have clinical pathways and work procedures. These include pathways and procedures for the diagnosis and treatment of PE, usually based on NICE guidance.

1.2.6 PE pathways include a combination of tools to support diagnostic decision making (such as Wells score), D-dimer blood testing and radiological imaging (scans). PE pathways support the standardised evaluation of patients and reduce the risk of subsequent death from PE (van Belle et al, 2006). These pathways will not identify every patient with a PE, but will help identify the majority.

1.3.1 Decision making is ‘the process of reaching a judgment, choosing an option or course of action to meet the needs of a given situation’ (Clinical Human Factors Group, 2018). In this report, the phrase ‘clinical decision making’ is used to describe how healthcare staff seek information, judge information, decide on possible diagnoses and treatment, and implement a plan.

1.3.2 Decision making has been extensively researched and traditionally is seen to involve two approaches (Kahneman, 2012):

• System 1 (fast): recognising patterns and making rapid and intuitive decisions.
• System 2 (slow): developing ideas for what may be needed for less recognisable situations and testing it by gathering and analysing information (also referred to as analytical decision making).

1.3.3 A person’s decisions are influenced by their recognition of and familiarity with a situation. Unfamiliar situations are more likely to result in slower, analytical thinking to reach a decision (system 2). When a situation is recognised, rapid and intuitive decisions (system 1) are more likely to follow, supported by unconscious mental shortcuts, known as ‘heuristics’. Heuristics support decision making in situations where there is limited time or information (an example is provided in figure 3).

Figure 3 Example of a heuristic (Clinical Human Factors Group, 2018)

1.3.4 Heuristics are useful because they can help staff reach a diagnosis and start treatment quickly. However, if the outcome of the heuristic is not as intended, with hindsight it may be called a bias (Klein, 1989). Bias is often the result of limited information and limited prompts.

1.3.5 More recently, the traditional beliefs around clinical decision making have been challenged because they may not account for how staff make decisions in the ‘real world’. Newer models incorporate the role of intuition, the influence of emotions, self-regulation and the real-world context (Patel et al, 2015).

Recognition-primed decision making

1.3.6 An individual’s clinical decision making is influenced by their knowledge, experience and memory, and the situation within which they find themselves. Naturalistic decision making (NDM) refers to how people make decisions in complex and real-world situations (Klein, 2008). Research into NDM led to the development of the ‘recognition-primed decision making’ (RPD) model.

1.3.7 The RPD model was developed to help understand how experts make decisions in complicated, information limited, and/or time-critical situations (Klein, 1997). An expert is someone who has significant knowledge about and skill in a particular subject or area, leading to high-quality performance.

1.3.8 When confronted with a situation, experts will often recognise the situation leading to a course of action. RPD helps describe how experts recognise, make sense of, and identify an appropriate course of action through three variations depending on the situation (Helander, 2006; Klein, 1999):

• Simple match – the person experiences a situation that they perceive as familiar and typical, resulting in a familiar course of action.
• Sensemaking required – the person experiences a situation which they interpret to be slightly different from what they are familiar with or from what they expected, resulting in gathering further information and engaging in further assessment before implementing a familiar course of action.
• Evaluate the action – the person experiences a familiar situation, but there are factors that mean they cannot implement the familiar course of action. The course of action is modified and the person mentally evaluates or simulates whether it will work before implementing it.

Table 3 provides some real-world examples.

Expert versus novice decision makers

1.3.9 Expert decision makers are those with significant knowledge and skill that support effective and practical decision making. Expertise is different to experience, which relates to the acquiring of knowledge and skill over years. Years of experience do not necessarily mean someone is an expert, but it can contribute to the development of expertise.

1.3.10 To develop decision making as a skill, practice is required. This practice includes developing appropriate heuristics for rapid decision making, and using structured approaches to analytical decision making. Skill development progresses through levels from novice to expert (see figure 4). More novice or beginner decision makers may be knowledgeable, but have limited real-world practice; they therefore need to be supervised.

1.3.11 Novice decision makers tend to default to following rules, meaning a situation is matched with a rule to help decide on an action. Checklists and procedures (cognitive aids) support rule-based decisions. While rule-based decision making allows rapid action, there is a risk that rules can be misapplied, or that no rule can be found to match a situation. In complex healthcare situations not all situations can be anticipated and therefore analytical decision making is required.

1.3.12 More expert decision makers have a broader mental repertoire to help them deal with situations. They are therefore able to deal with more complexity. Because of increased practice and experience, their heuristics are less prone to being misapplied. They may also be more able to identify when a situation is outside of their mental repertoire and requires practiced analytical decision making.

2.1.1 Following HSIB’s investigation into the ‘Management of venous thromboembolism risk in patients following thrombolysis for an acute stroke’ (Healthcare Safety Investigation Branch, 2020a), a further safety risk was identified of delay in the recognition of PE.

2.1.2 HSIB reviewed the Strategic Executive Information System (StEIS, the national reporting system for serious incidents) for other incidents involving a delay in the diagnosis of PE (see appendix 7.1 for the search criteria used). This identified several incidents, particularly in EDs, before and during the COVID-19 pandemic.

2.1.3 As a result of this information, it was recommended to HSIB’s Chief Investigator that a national investigation should be undertaken. The safety risk met the following criteria for a national investigation.

Outcome impact – what was, or is, the impact of the safety issue on people and services across the healthcare system?

• The incidence (occurrence rate) of PE per year is around 60 to 70 cases per 100,000 people (Howard et al, 2018). While overall trends may be decreasing, it presents a significant burden on public health services (Barco et al, 2020).
• PE is a significant condition associated with a risk of death (Heit et al, 2016). PE can also result in other complications including heart rhythm abnormalities (arrhythmias), heart failure, and lung tissue damage (pulmonary infarction).
• If undiagnosed or untreated, PE is associated with higher risks of death or long-term harm. Incidents reported on StEIS demonstrate the continuing harm associated with delays to PE diagnosis and its treatment.
• PE has a significant psychological impact on patients, families and staff. Patients with a PE may have psychological distress (for example, Tran et al, 2021). Healthcare staff who may have missed a PE when diagnosing a patient may also experience distress (Healthcare Safety Investigation Branch, 2021a).

Systemic risk – how widespread and how common a safety issue is this across the healthcare system?

• In 2019 the National Confidential Enquiry into Patient Outcome and Death (NCEPOD) identified delays in the processes for treating patients with PE in almost 40% of the 526 cases it reviewed (National Confidential Enquiry into Patient Outcome and Death, 2019).
• NCEPOD also identified that for just under 20% of patients there was a delay in starting anticoagulation medication, and there was variation in how patients were selected for outpatient care pathways.
• The incidents reported on StEIS demonstrated a risk in EDs. ED environments may be higher risk because patients may visit the ED with non-specific symptoms.

Learning potential – what is the potential for an HSIB investigation to lead to positive changes and improvements to patient safety across the healthcare system?

• The data demonstrates a need to understand why delays to diagnosis and treatment of PE occur in EDs and how they can be mitigated. An investigation offers the potential to understand contributory factors.
• COVID-19 may also impact on the diagnosis of PE (see 1.1.14). Understanding the contributory factors in delays to diagnosis and treatment of PE may add value to the treatment of PE in patients with COVID-19.

Investigative approach

2.2.1 The investigation was carried out between October 2020 and May 2021. The investigation had two parts:

1. Review of serious incidents (local factors): a review of serious incidents in England to identify local factors that contributed to incidents related to diagnosis and treatment of PE in EDs. The findings defined the terms of reference for part two.
2. National investigation: a national investigation into the factors that influenced ED staff decision making in the diagnosis and treatment of PE.

Review of serious incidents (local factors)

2.2.2 The investigation approached NHS hospitals to request copies of serious incident investigation reports where the incident:

• occurred during 2020 in the NHS in England
• involved ED and the diagnosis and/or treatment of PE, and
• was associated with delays and/or missed diagnosis and/or treatment.

2.2.3 The investigation analysed the serious incident reports to identify the factors that contributed to the incidents. The reports were analysed using the Systems Engineering Initiative for Patient Safety (SEIPS) (Holden et al, 2013; Carayon et al, 2006) as a framework (see appendix 7.2). The approach was similar to that of other HSIB reports (Healthcare Safety Investigation Branch, 2021b). Findings are presented in section 3.

Terms of reference for national investigation

2.2.4 The serious incident reports demonstrated various factors that influenced the occurrence of the incidents. A consistent finding across the serious incident investigations was that ‘incorrect’ decisions were made in the diagnosis and treatment of PE. The aims of the national investigation were therefore to attempt to understand the nuances of the decisions made in EDs and factors that could affect them. The terms of reference were to:

• examine clinical decision making in the diagnosis and treatment of PE and the role of expertise
• identify factors in the wider healthcare system that support or inhibit diagnostic decision making when staff are seeing patients with non-specific symptoms and signs that may suggest PE.

National investigation

Examining clinical decision making

2.2.5 Applied Cognitive Task Analysis (ACTA) was used to examine clinical decision making. ACTA is a technique to analyse the cognitive elements of a task. It considers how expertise supports the undertaking of cognitive tasks and why such tasks may be difficult for novices.

2.2.6 ACTA methodology is provided in a toolkit (Militello et al, 1997; Militello and Hutton, 1998) with guidance for use. The steps of an ACTA, and how the investigation undertook the analysis, are provided in appendix 7.3. The methodology includes three methods and the investigation used two of these in the cognitive task diagram (a visual representation of the tasks involving thinking) and knowledge audit (interviews with staff to explore expertise using probing questions). Knowledge audit interviews were undertaken with clinical experts and novices in EDs (see table 4).

2.2.7 The terms ‘expert’ and ‘novice’ were used in line with the original ACTA model. To distinguish between levels of expertise the investigation was guided by Dreyfus’ model (Dreyfus, 2004) (see section 1.3 and figure 4). Staff were considered to be either experts or novices:

• Experts – senior nursing staff; advanced clinical practitioners; senior doctors in emergency medicine, including consultants.
• Novices – students of any profession; newly qualified nurses, doctors and advanced clinical practitioners; and those with limited situational knowledge and skill in emergency medicine.

Identifying wider system factors

2.2.8 Alongside the ACTA, the investigation undertook observations and interviews (see table 4) to identify factors in the wider healthcare system that influenced decision making. Interviews were also undertaken with national bodies and subject experts. Evidence was further gathered from academic and policy literature.

2.2.9 The investigation visited six EDs across four hospital trusts in England. The EDs were geographically spread and of different sizes with between 150 and 650 attendances a day (see table 4).

Analysis

2.2.10 Interviews were undertaken using a semi-structured approach and observations using an observation template based on SEIPS. Field notes were taken and collated for analysis.

2.2.11 The learning, safety observations and safety recommendations in this report were shared with stakeholders. This allowed further corroboration of findings, adjustment where necessary, and formation of safety recommendations that have been agreed by relevant national bodies.

Limitations

2.2.12 The investigation’s findings are specific to decision making in EDs. This was the focus of the investigation and where the lines of inquiry led. Evidence collection was predominantly qualitative through the interviews, observations and ACTA. The findings are therefore based on staff experiences, perceptions and narrative. However, they have been verified with national experts, subject matter advisors and relevant national bodies.

2.2.13 The investigation acknowledges that there will be other factors that influence decision making outside of those considered in this report and that similar challenges will be seen in other parts of healthcare. The findings of this report may have wider benefit for decision making outside of the ED.

2.2.14 Safety recommendations have been made where the investigation has appropriate evidence and following engagement with the relevant national bodies. Safety observations represent learning from the investigation that may be of benefit to organisations and the wider healthcare system.

3.1.1 Martin was a husband and father of two. He was 51 years old, and described by his Wife as “clever, but humble, and well-liked by everybody”. He was active in the local community and also enjoyed playing cricket and golf. He had been working from home during the COVID-19 pandemic as his office had been closed.

3.1.2 Martin was rarely unwell, but while working from home on a Friday, he started feeling short of breath. Martin and his Wife thought he could have COVID-19, so booked a test which was undertaken on the Sunday. Two days later the test came back as negative. Martin was still breathless so he rang NHS 111 and was advised to go to the local emergency department (ED).

3.1.3 Martin attended the ED on his own because of COVID-19 restrictions. He had some tests and was discharged home. Martin told his Wife that he had undergone a heart tracing, chest X-ray and blood tests, but the doctors could find nothing wrong so thought it might be a chest infection.

3.1.4 Over the following days Martin remained short of breath, but was still able to get on with daily life. Eight days after going to the ED, Martin drove into work for a meeting, but when he returned to his car became unwell. An ambulance was called which took him to the ED. Martin’s Wife received a call from the hospital asking her to pack a bag for him because he was going to need to stay in hospital.

3.1.5 About 30 minutes later Martin’s Wife received another call from the hospital to tell her that Martin had become seriously unwell. She was asked to get to the hospital as soon as possible and arrived about 45 minutes later. Martin had suffered a cardiac arrest and was being moved to intensive care. Martin did not recover and died later the same day. A post-mortem report showed Martin had died from a pulmonary embolism (PE).

3.1.6 An investigation was undertaken by the hospital to find out whether there had been an opportunity to identify a PE during Martin’s first visit to the ED. The hospital found that a PE had been considered and a D-dimer blood test to rule-out a PE had been planned, but was not requested. Martin’s care was verbally handed over to another member of staff at shift change. Martin was later discharged as all his tests were normal, but staff did not know that a D-dimer test had not been undertaken.

3.1.7 The hospital’s investigation highlighted issues relating to:

• who orders a D-dimer blood test and when
• handover of outstanding tests required
• high workload
• lack of relatives in ED to provide a further information
• overcrowding of the ED.

3.2.1 The investigation reviewed 14 serious incident reports, referred to here as ‘reports’. Each report investigated the care of a patient who had an incorrect, missed or delayed diagnosis or treatment of PE in an ED. Several of the reports described patients who had died as a result of a PE. Some patients died from other conditions alongside a PE, or died from a condition incorrectly suspected to be a PE.

3.2.2 The reports commonly considered how the process of decision making by staff resulted in delayed or missed diagnoses of PE, and explored the associated contributory factors to a greater or lesser degree. The reports also considered the process of developing treatment plans for patients with suspected PE. Figure 5 provides an overview of the processes and outcomes seen in the reports.

3.3.1 Diagnostic decision making refers to how ED staff use information to conclude whether a patient may have a PE. Decision aids and tools may be used to support decision making; these aids and tools may be available in procedures and guidelines.

3.3.2 The reports included missed diagnoses of PE, including where an alternative diagnosis was suspected, and delayed diagnoses of PE. Where a PE was not initially suspected, but was subsequently found, alternative initial diagnoses included chest infections (including COVID-19), muscular pain and anxiety. Reports also included examples of patients who were suspected of having a PE, but an alternative diagnosis was later found, such as aortic dissection (a tear that allows blood to flow between the layers of the wall of a person’s aorta, which is the largest artery in the body).

3.3.3 Several contributory factors were found to have influenced the decisions made by staff in the diagnosis of PE. The recurrent themes were:

• COVID-19 was perceived to have increased the risk of missing a PE because the infection caused similar symptoms and could occur at the same time as a PE. There was limited national guidance to help clinicians identify patients with a PE and COVID-19.
• Demand on EDs and the national operating standard were described as creating ‘pressure’ which limited time to ‘pause’ and consider the range of possible diagnoses.
• Staffing shortages nationally and locally of senior doctors limited opportunities for in-person reviews of patients meaning they were assessed by more novice staff.
• ED diagnostic and treatment pathways did not always prompt clinicians to consider PE when patients had symptoms that were unexplained or not typical of PE. Outpatient PE pathways did not always consider the risk of discharging a patient with an unknown diagnosis who was on anticoagulation medication, while they waited to have a scan. Examples included a case where the patient was later found to have an aortic dissection.
• Conflict between ED and radiology departments was described, creating barriers to requesting imaging. Scans were a limited resource.
• Decision making tools, cognitive aids and risk assessments to support diagnosis of PE were often not accessed or used.
• D-dimer blood tests influenced consideration of PE. Where a D-dimer had been requested during the initial assessment of a patient, PE became the lead potential diagnosis and where a D-dimer was not requested, PE was given less consideration. An elevated D-dimer was assumed to be related to PE and did not lead staff to consider other causes, such as aortic dissection.
• Review of results of tests or observations outside expected ranges did not always take place before a patient was given a diagnosis or discharged. Findings that were outside of expected ranges were not always accounted for. Examples included the D-dimer and fast heart rates.
• Patient symptoms were not always typical of PE or were non specific. Other symptoms included nausea and vomiting, abdominal pain and dizziness. Patients often had risk factors for blood clots, but some of these may have not been recognised, such as being less active during COVID-19.

3.3.4 Most reports contained a focus on how staff ‘biases’ had resulted in PE being considered less likely or other diagnoses being considered more likely. Prompts influencing bias and leading staff to consider alternative diagnoses to a PE included COVID-19, tender chest walls (muscular-related diagnosis), and mental health histories (anxiety-related diagnosis). The focus of investigations on individual bias limits learning by reducing consideration of the wider healthcare system. Instead, investigations should seek to identify the conditions that contribute to bias, and address them to support effective decision making. This includes supporting staff to develop more sophisticated and appropriate heuristics (see 1.3.3).

3.4.1 The reports included examples of delays in giving anticoagulation medication, or anticoagulation being given incorrectly, to patients with suspected or confirmed PE. Reports also described limited imaging capacity that meant immediate scans were uncommon where patients were not unwell.

3.4.2 Underdosing of anticoagulation was seen where prescribing was based on the estimated weight of the patient. This safety risk was identified nationally in 2010 (National Patient Safety Agency, 2010). Reports described insufficient emphasis on weighing patients, poor weighing equipment, limited prompts to identify which patients need weighing, and difficulties weighing unwell patients.

3.4.3 Three reports also identified challenges with the administration of thrombolysis (a medicine to break down blood clots) for patients with a PE in emergency situations. Staff found barriers to accessing medicines because of unclear local processes for access to emergency medicines, limited signposting of medicine locations, and local training that did not include how to access the medicines.

3.5.1 The investigation’s review of the reports provided an understanding of local and some national factors that contributed to the serious incidents. The investigation recognises that the number of reported incidents was small compared to the number of patients who arrive at EDs with a suspected PE. Some of the reports also represented patients who did not have the typical symptoms of PE, which made diagnosis more challenging.

3.5.2 From the review of the reports, the investigation concluded that:

• Decision making in EDs is challenging, particularly in the diagnosis of suspected PE where patients may have non-specific or atypical symptoms and signs.
• Local investigations often attributed incorrect decisions to ‘biases’ with limited exploration of how staff made decisions in practice.
• Staff did not always use the tools available to them to help make decisions, such as decision aids and guidelines.
• There are specific, wider ED factors that potentially contribute to incorrect decisions such as demand on services, workload, process and procedure design, conflict between departments, and access to support.

3.5.3 The findings suggested that there would be value in the investigation seeking to understand, in more depth, how ED staff make decisions, how staff can be supported to make effective decisions, and the wider factors that may influence those decisions.

4.1.1 HSIB considered decision making in EDs in a previous investigation, ‘Delayed recognition of acute aortic dissection’ (Healthcare Safety Investigation Branch, 2020b). Similar to that previous investigation, this investigation found staff using different approaches to decision making depending on the situation, their expertise and experience.

4.1.2 This investigation identified examples of staff using recognition-primed decision making (RPD, see 1.3.6 to 1.3.8). This was particularly noted when EDs were busy, when there were lots of patients waiting, and in urgent situations. Seeing patients with familiar signs and symptoms resulted in familiar actions.

4.1.3 When staff saw patients with unfamiliar signs and symptoms, particularly those that were less urgent, they described a more analytical approach. The investigation observed staff thinking through symptoms and signs, talking out loud, and discussing options with colleagues. While this approach was more comprehensive, it was time consuming and sometimes still did not result in the intended outcomes.

4.1.4 During observations the demand on EDs was seen to increase during the day. As demand grew, the available time to gather information about patients was reduced. The use of RPD in these situations supported rapid decisions, but some decisions were based on limited information. There was a risk that heuristics (see 1.3.3) may be misapplied as demonstrated in the serious incidents in section 3.

4.1.5 Staff also told the investigation how they managed uncertainty. Decisions were commonly based on limited information and without a full awareness of the situation. Staff aimed to identify common diagnoses and exclude “worst-case scenarios” (life-threatening conditions).

4.1.6 Staff described how managing uncertainty meant taking risks. For example, they knew that they could not know everything about a patient’s medical history, and were not able to test every patient for a PE. This demonstrated the need to make ‘trade-offs’, balancing factors such as accuracy, efficiency and thoroughness.

4.1.7 More novice staff and students were also heard matching patients to textbook descriptions of diagnoses, sometimes using educational resources. The ACTA subject matter advisor described these as rule-based decisions which are common for novices, but may risk rules being inappropriately applied to less familiar, more complex or nuanced situations. For example, novices generally associated a raised D-dimer with a potential PE, not considering other potential diagnoses.

4.1.8 Students and doctors in their early years of practice recited classical textbook features of PE to the investigation. These were chest pain, breathlessness and coughing up blood (haemoptysis). The investigation asked experts about the signs and symptoms of PEs and heard that these classic features were rarely all present; often a patient would have breathlessness alone, or a fast heart rate. Patients could also have atypical signs and symptoms such as described in the serious incident reports. The research suggests that 97% of patients with PE have at least one of the three classic features (BMJ Best Practice, 2021).

The investigation used ACTA (see 2.2.5 to 2.2.7) to examine the key decisions made by ED staff relating to the diagnosis and treatment of PE. During the development of a cognitive task diagram (see figure 6), ED staff identified three cognitive tasks as the most challenging. They told the investigation that these tasks were influential in the diagnosis and treatment of suspected PE and involved decisions that needed the most expertise. The tasks were:

• initial assessment of the patient, including selecting investigatory tests
• further assessment and review of preliminary test results
• prescribing of anticoagulation medication while waiting for the scans or results of tests (referred to as ‘interim coagulation’), particularly if the patient is to be discharged for outpatient care.

Knowledge audits (see appendix 7.3) were undertaken across six EDs with a range of expert and novice staff. The aim of this was to examine decision making within the cognitive tasks identified above. Appendix 7.4 provides a fuller summary of the findings.

Initial assessment and selection of investigatory tests

4.2.1 Initial assessment of patients arriving at EDs included triage, streaming and some rapid clinical assessment (see section 1.2). Rapid clinical assessment included taking a short history, measuring of vital signs and initial tests.

4.2.2 Initial assessment was commonly undertaken by nurses with varying levels of experience. Nurses described how assessing patients with non-specific symptoms required expertise to ensure appropriate tests were requested, particularly when there were time pressures.

4.2.3 The investigation found that the decision to do a D-dimer test and early suspicion of a PE was influenced by workload, staff experience and ED performance targets. The decision had ramifications for later assessment of patients, influencing the later consideration of PE. Nursing staff rarely found out the outcomes of their decisions or what happened to patients.

4.2.4 Expert ED nursing staff described how they tried to focus on symptoms rather than diagnoses, recognising that fixating on a diagnosis too early may influence later decisions. They described that they would only request a D-dimer test after consultation with a senior doctor as they recognised the need for an assessment to be undertaken to determine its need.

4.2.5 Novice ED nursing staff told the investigation that they sometimes found the initial assessment of patients difficult. They felt that training had not taught them how to assess symptoms and signs to select investigatory tests; rather they were expected to follow published work procedures. The ACTA subject matter advisor told the investigation that the work procedures in settings such as EDs are unlikely to be able to provide the necessary detail to help staff care for patients with the variety and complexity of conditions that they see in EDs. This meant that training was important.

Further assessment and review of results

4.2.6 Further assessment of patients was undertaken after the initial assessment. At this point patients were asked to provide a full description of their symptoms, were examined, and investigatory tests results were reviewed. Further tests were then requested and initial treatment was started if required.

4.2.7 Further assessment was undertaken by staff with a range of experience. They included advanced clinical practitioners (ACPs), GPs and doctors of all levels. The ED subject matter advisor told the investigation that this assessment was often undertaken by novice staff, particularly for well patients who may have subtle symptoms and signs.

4.2.8 ED staff described that the role of further assessment was to pull together information to identify a main diagnosis. They considered that prolonged experience of working in ED may help the gaining of familiarity with the varied signs and symptoms that patients with PE may show. They also described that expertise was needed to help ensure atypical symptoms and signs, and observations and test results outside of expected ranges, were identified, understood, and acted on appropriately.

4.2.9 The investigation was told that the suspicion of, and decision to pursue, a diagnosis of a PE was influenced by workload, years of experience, expertise in the skill of decision making, ED performance targets, available test results and previous decisions about a patient’s potential diagnosis.

4.2.10 Expert staff described the challenge of building a mental list of potential diagnoses for a patient with non-specific symptoms when information was limited. They likened this to situations that GPs may commonly encounter (see appendix 7.4 for a knowledge audit undertaken with a GP). They knew they could rarely be 100% sure of a diagnosis, but aimed to refine their mental list using specific questions to account for each symptom, sign and test result. They were concerned with any unexplained finding or finding that did not match with a patient’s age, fitness levels or their own reflections on what was wrong with the patient.

4.2.11 Experts also used subtle cues to build their understanding of how unwell a patient may be. These cues may be missed by someone with less experience. For example, watching patients walk or talk gave insights into how unwell they were. Some experts also actively sought peer review (opinion from expert colleagues) to check and challenge their diagnoses.

4.2.12 For novice staff, the investigation identified several factors that made further assessment difficult (see appendix 7.4); a summary is presented here. Novice staff described receiving little training and practice in making decisions. Their education had led them to look for “textbook” presentations. They therefore might reassure themselves that a significant diagnosis is not present in patients who do not “fit the rules”. For example, several novices told the investigation that they would not expect a patient who looked well to have a PE or aortic dissection.

4.2.13 Novice staff also described how the tests requested at a patient’s initial assessment may influence their thinking. For example, the investigation observed that PE had been added to a list of potential diagnoses for a patient after blood test results had been received. The doctor described that they had not initially thought the patient had a PE, but because they had a raised D-dimer test result (ordered at initial assessment), they felt obliged to consider PE as a diagnosis. Alternative causes of the raised D-dimer result were not considered.

4.2.14 Experts told the investigation that they thought novices felt that they had to make a diagnosis to account for a patient’s symptoms and test results. This meant that certain tests, such as D-dimer, were sometimes used outside of their intended scope for reassurance. Experts considered that novices may feel uncomfortable exposing their uncertainty about a diagnosis.

4.2.15 The investigation was also told about the influence of COVID-19 on decision making. Early in the pandemic a consultant said: “We have all become novices.” The new disease was not part of the experts’ experiences and accounting for the new symptoms and signs was therefore challenging. The similarity of PE and COVID-19 symptoms also made identifying PE difficult during the peaks of the pandemic.

Prescribing of interim anticoagulation

4.2.16 The National Institute for Health and Care Excellence (NICE) describes the role of interim anticoagulation for patients with suspected PE and the circumstances in which it should be prescribed and administered (see 1.1.10) (National Institute for Health and Care Excellence, 2020a). The decision to prescribe will be made by a doctor or ACP.

4.2.17 Experts described being uncomfortable about prescribing anticoagulation to a patient with an unknown diagnosis. They therefore tried to balance the risk of giving anticoagulation to a patient who ultimately may not need it or to whom it may cause harm, with not giving it to a patient who may need it. The investigation attempted to examine experts’ individual strategies to balance risks; nothing formal was described. Some experts used the term “gut feelings” which the investigation considered described their mental shortcuts based on experience.

4.2.18 For novices, some described that prescribing interim anticoagulation was “easy” because NICE told them what to do. The ACTA subject matter advisor suggested that this may be because, unlike experts, they had limited experience of the potential risks. Other novices described the decision to prescribe as challenging; they felt that their training had not equipped them to make judgements about clinical risk.

Summary of findings

4.2.19 The ACTA showed how decisions made in the diagnosis and treatment of PE in ED are influenced by the expertise (knowledge and skill) of decision makers, the available information relating to a patient, and interpretation of and action taken on that information. Decisions were further influenced by the individual circumstances of the staff, training, experience, and wider factors in the healthcare system.

What do experts do?

4.2.20 When presented with familiar situations, experts described knowing what common courses of action to follow. Where situations were less familiar and when there was time, experts described thinking through possibilities in a more analytical way. They were often unable to describe how they structured that analytical thinking.

4.2.21 The terms “gut feeling” and “gestalt” (here meaning an appreciation of the whole problem, rather than its parts) were heard several times. They related to how decision makers built overall perceptions of situations based on factors such as experience, history and examination findings, and other situational prompts. The ACTA subject matter advisor described that a gestalt allows for interpretation of the various sources of information, not just individual symptoms.

4.2.22 The ACTA identified ways experts think and act to support their decision making. These findings may be of value when considering how to develop expert decision-making skills. In summary, experts:

• were alert to the possibility of life-threatening diagnoses and sought to rule these out
• sought to account for unexplained symptoms, signs and results
• were comfortable saying they were unsure of a diagnosis
• recognised the risk of fixating on a diagnosis and compared information critically
• sought opportunities to challenge their decisions with peers
• appreciated when their cognitive abilities may be being affected
• were less worried about making unpopular decisions
• took into account various sources of information including their own knowledge from training, previous experience where applicable, and organisational rules and guidance.

Why are decisions sometimes difficult?

4.2.23 Through the ACTA, novice staff told the investigation of limited recognition of the role of heuristics, limited training in how to make analytical decisions and limited opportunities to develop their decision-making skills in training environments and in clinical practice. These gaps in training and development were further explored by the investigation and findings are described in section 4.3.

4.2.24 The ACTA again demonstrated wider factors in the healthcare system that further influenced the decision making of novices and experts. Several of these factors were similar to those identified in the review of the serious incidents, including demand for ED services, capacity in the ED, COVID-19 and standardised processes. These factors were explored further and findings are described in section 4.4.

This section describes the findings from the investigation’s exploration of how healthcare staff are trained in decision making, and how expert skills are developed. Five factors are explored in turn (see figure 7). While the investigation’s focus was on decision making in the diagnosis and treatment of PE in EDs, development of more expert decision-making skills may further support staff when they see patients with other non-specific or atypical symptoms and signs.

Decision making in healthcare curricula

4.3.1 The ACTA found that staff had limited familiarity with how people make decisions, and the role of heuristics. Staff also described limited opportunities to practice making decisions. The investigation therefore aimed to understand how the skill of decision making is taught to students and staff.

4.3.2 The investigation reviewed several healthcare curricula and standards for training (General Medical Council, 2021; 2018; 2017; Joint Royal Colleges of Physicians Training Board, n.d.; Royal College of Emergency Medicine, 2021a; 2019; Royal College of Nursing, n.d.; Royal College of Physicians, n.d.; 2021). These curricula covered medicine, nursing, advanced clinical practitioners (ACPs – staff with non-medical backgrounds who have skills and knowledge to undertake advanced roles) and physician associates (healthcare professionals who work alongside doctors). They all included decision-making competencies to a lesser or greater degree. For example, the ACP curriculum described knowledge expectations such as to ‘define the steps of diagnostic reasoning’ and ‘demonstrate ability to identify one’s own biases and inconsistencies in clinical reasoning’ (Royal College of Emergency Medicine, 2019).

4.3.3 Students and novice doctors told the investigation that their training had not taught them the theory behind how people make decisions, and had not prepared them to make decisions that would affect patients. They described limited appreciation of the differences between the types of decision making or the factors that influence decision making, and had received limited opportunities to practise making decisions.

4.3.4 The investigation was told that national curricula and standards were often written by professional bodies. However, implementation and delivery of training is the responsibility of the individual training establishment, for example a university. Professional bodies carry out inspections to assure themselves that establishments are appropriately delivering their curricula.

4.3.5 The investigation found a gap between the expectations of curricula, particularly those focused on medical training, around decision-making skills, and how prepared novice doctors felt to make decisions in clinical practice. The investigation identified inconsistencies in how decision making is taught at undergraduate and postgraduate levels. The development of decision-making skills was heard to be via ‘on the job’ experience, implicitly learnt, and with a belief that years of experience equated to expertise.

Simulation and decision making

4.3.6 ED leads told the investigation about the use of simulation to train staff in common ED situations. Simulations are ‘pretend’ versions of healthcare situations that give staff a chance to practise how they would respond. They take place either in the clinical environment or a bespoke simulation suite. Every ED the investigation visited had access to simulation facilities.

4.3.7 ED staff told the investigation that they generally welcomed simulation and saw value in it. However, they were rarely able to access it because the need for care provision and workforce shortages often meant simulation and other mandatory training was cancelled. When simulation did take place, it was rarely multi-professional – that is, it only involved some groups of staff. This created unrealistic scenarios because unlike real ED situations, the simulations did not involve the wider multidisciplinary team.

4.3.8 Where ED staff had experienced simulation sessions, these focused on emergency teamworking scenarios such as cardiac arrests (when a person’s heart stops beating) and trauma. Staff had not experienced any other types of simulation, such as less critical scenarios, those involving patients with non-specific signs and symptoms, or the use of simulation for the evaluation of systems and processes.

4.3.9 The research literature provides examples of the positive role of simulation in developing decision-making skills (for example, Murray et al, 2018). Health Education England (HEE) has described the potential benefits of simulation in the development of professional capabilities, multi-professional team working and evaluation of systems and processes. However, HEE has also described unequal access to simulation across the country, varying views on its value, and no agreed approach for how best to implement programmes. In response HEE has developed a vision for simulation in England (Health Education England, 2020).

Serious games

4.3.10 HEE told the investigation about other training opportunities available to develop decision-making skills. These included ‘serious games’, which are games that have been developed for the purpose of something other than entertainment, for example education.

4.3.11 Serious games support learning because they are motivating, active, attention grabbing, can be repeated and allow for immediate feedback (Drummond et al, 2017). They are used in several industries; for example, the nuclear industry uses virtual facilities to train inspectors. In healthcare they have been used for patient, public and staff education (Sharifzadeh et al, 2020).

4.3.12 HEE said that serious games could be valuable in helping healthcare novices to develop decision-making skills. This is supported by research that shows the positive effects of serious games on developing decision-making skills (for example, Kaczmarczyk et al, 2016; Crichton et al, 2000). Further positives include limited need for expensive simulation equipment and the option to undertake training as and when the learner wishes (Wang et al, 2016).

Developing decision making in other industries

4.3.13 The ACTA subject matter advisor told the investigation that some other industries deliver training to help accelerate the development of decision-making expertise. The investigation recognises that learning from other industries may not be directly transferrable to healthcare and that any transfer requires evaluation to ensure safety and effectiveness. However, learning from other industries may provide insights and be of value to healthcare.

4.3.14 Research shows that it is possible to help people to accelerate the development of their decision-making skills. For example, studies suggest that ‘decision-skills training’ (Phillips et al, 2004) and ‘accelerating the acquisition of expertise’ (Hoffman et al, 2013) can help people to develop recognition-primed decision making skills and more sophisticated heuristics, with a reduced likelihood of them being misapplied.

4.3.15 To understand how industries accelerate skills development, the investigation engaged with military helicopter aviation (case study 1) and the fire service (case study 2). In summary, the investigation found that there were established approaches to help people with limited experience to develop their decision-making skills. The industries developed understanding of decision making, more sophisticated heuristics for recognition-primed decision making, and analytical decision-making skills by:

• Teaching students how people make decisions, the role of heuristics and potential decision-making traps.
• Using mnemonics and ‘generic decision tools’ to support structured, analytical decision making. These methods and tools are embedded in teaching and practice.
• Including use of the generic decision tools in staff induction and competency assessment. This leads to repetitive practice and the tools are also used to provide feedback by instructors.
• Requiring real-world and simulated application of generic decision tools to maintain competencies, with a minimum yearly simulation requirement. Simulations offer exposure to novel, rare and common situations, enabling staff to recognise them more easily in real life.

4.3.16 The use of simulation in these industries was a critical part of initial and ongoing staff training, and mandatory competency assessments. Staff who did not meet the necessary level of competency would be prevented from undertaking certain responsibilities until the competency was met.

Accelerating decision-making skills in healthcare

4.3.17 The evidence gathered by the investigation suggests that there may be opportunities to support healthcare staff to develop their decision-making skills through:

• the use of simulation to expose staff to situations, for example emergency scenarios, to develop recognition
• the practice of using analytical decision making, for example when considering non-specific or atypical signs and symptoms such as those of PE, where there is time for staff to consider the diagnosis and discuss options with their supervisors.

4.3.18 The learning from the case study industries shows potential value in a comprehensive education focus on understanding decision making, a generic decision tool, and practice in making decisions in a safe environment. The investigation found that while healthcare curricula describe decision-making competencies, novice staff did not feel confident making decisions.

4.3.19 The ACTA subject matter advisor told the investigation that generic decision tools are useful to structure analytical decision making, particularly when the decision maker is confronted with new, unfamiliar situations with time to consider actions. They act as ‘scaffolding’ for novices.

4.3.20 Various generic decision tools exist outside of healthcare such as DECIDE (define, establish, consider, identify, develop and evaluate) (Guo, 2020), OODA loops (observe, orient, decide and act), and TDODAR (see figure 8). The investigation found no established generic decision tool in healthcare and no formal training in using tools to make analytical decisions. Some staff were aware of the tools used by other industries.

Figure 8 TDODAR mnemonic (Civil Aviation Authority, 2014)

4.3.21 The investigation reviewed the research literature for evidence of impact of generic decision tools such as TDODAR, DECIDE and OODA in industry and healthcare. While the tools are discussed positively by many authors and international bodies, the investigation was not able to find any evaluation studies. In general, tools were felt to support analytical thinking, but potentially required ‘updating’ to bring them in line with modern thinking around decision making (Bryant, 2006). They also required training in their use. Research literature was available for the fire service’s decision control process (see case study 2) (Cohen-Hatton and Honey, 2015).

4.3.22 The investigation found that there may be opportunities to improve the use of simulation to support the development of decision-making skills in healthcare. HEE told the investigation that it was keen to enhance the use of simulation and to introduce alternative learning methods to support education in decision making. The investigation recognises the need for staff to have protected time to access training for their development.

4.3.23 The findings led the investigation to make the following safety recommendation and safety observation. The investigation worked with HEE to form the safety recommendation with the aim of supporting staff in EDs to develop their decision-making skills and their recognition of conditions such as PE when faced with non-specific or atypical symptoms and signs. Regarding the safety observation, the investigation recognises the importance of developing an evidence base for the use of a generic decision tool in healthcare to ensure appropriate and safe implementation.

This section describes the findings in relation to factors in the wider healthcare system that support or inhibit diagnostic decision making in EDs. Five factors are explored in turn (see figure 9). These factors emerged from the review of the serious incidents, the ACTA and observations in EDs.

Demand and decision making

4.4.1 The investigation witnessed the significant demand on EDs, which was a result of high patient numbers. ED staff told the investigation that high demand affects the time and information available to them to make decisions. As EDs became busier throughout the day, staff were seen to spend less time with each patient. As a result the thoroughness of assessments was reduced to improve efficiency; this is a recognised risk (Hollnagel, 2009).

4.4.2 Since May 2021 the numbers of patients visiting EDs ‘have been the highest on record for their respective month’ (Royal College of Emergency Medicine, 2021b). Reports by national bodies describe how demand has exceeded capacity, with inadequate resources, workforce shortages, insufficient hospital beds and limited understanding of local demand contributing to the pressures on EDs (Moulton and Mann, 2021; Royal College of Emergency Medicine, 2020).

4.4.3 The examination of ED demand and capacity was outside of the scope of this investigation. However, the investigation recognises that, no matter the expertise of decision makers, degraded decision making may sometimes occur if demand and resource challenges are not addressed.

4.4.4 ED staff told the investigation that the national operating standard for EDs (see 1.2.2) was another factor that influenced their decisions. Staff described having to make rapid decisions about patients who had been in their EDs for almost 4 hours. The investigation found that staff sometimes felt pressurised to meet the standard and described a sometimes punitive response if it was not met.

4.4.5 The national operating standard in England is contentious. There is evidence that it has benefited patients by reducing long waits and overcrowding, which are known to be associated with harm. However, the standard has not been consistently met for some time (NHS England and NHS Improvement, n.d.) and may have led to a focus on performance, rather than quality of care and good outcomes for patients (Moulton and Mann, 2021). At the time of writing there are proposals for the standard to be replaced (NHS England and NHS Improvement, 2020; O’Dowd, 2020) and new operational priorities are planned (NHS England and NHS Improvement, 2022).

National guidance and decision making

4.4.6 The National Institute for Health and Care Excellence (NICE) provides guidance on the diagnosis and treatment of adults with PE (and deep vein thrombosis) (National Institute for Health and Care Excellence, 2020a). The investigation found that ED staff were familiar with the guidance, but rarely applied the decision-making scores and criteria described in it.

Demand and decision making

4.4.1 The investigation witnessed the significant demand on EDs, which was a result of high patient numbers. ED staff told the investigation that high demand affects the time and information available to them to make decisions. As EDs became busier throughout the day, staff were seen to spend less time with each patient. As a result the thoroughness of assessments was reduced to improve efficiency; this is a recognised risk (Hollnagel, 2009).

4.4.2 Since May 2021 the numbers of patients visiting EDs ‘have been the highest on record for their respective month’ (Royal College of Emergency Medicine, 2021b). Reports by national bodies describe how demand has exceeded capacity, with inadequate resources, workforce shortages, insufficient hospital beds and limited understanding of local demand contributing to the pressures on EDs (Moulton and Mann, 2021; Royal College of Emergency Medicine, 2020).

4.4.3 The examination of ED demand and capacity was outside of the scope of this investigation. However, the investigation recognises that, no matter the expertise of decision makers, degraded decision making may sometimes occur if demand and resource challenges are not addressed.

4.4.4 ED staff told the investigation that the national operating standard for EDs (see 1.2.2) was another factor that influenced their decisions. Staff described having to make rapid decisions about patients who had been in their EDs for almost 4 hours. The investigation found that staff sometimes felt pressurised to meet the standard and described a sometimes punitive response if it was not met.

4.4.5 The national operating standard in England is contentious. There is evidence that it has benefited patients by reducing long waits and overcrowding, which are known to be associated with harm. However, the standard has not been consistently met for some time (NHS England and NHS Improvement, n.d.) and may have led to a focus on performance, rather than quality of care and good outcomes for patients (Moulton and Mann, 2021). At the time of writing there are proposals for the standard to be replaced (NHS England and NHS Improvement, 2020; O’Dowd, 2020) and new operational priorities are planned (NHS England and NHS Improvement, 2022).

National guidance and decision making

4.4.6 The National Institute for Health and Care Excellence (NICE) provides guidance on the diagnosis and treatment of adults with PE (and deep vein thrombosis) (National Institute for Health and Care Excellence, 2020a). The investigation found that ED staff were familiar with the guidance, but rarely applied the decision-making scores and criteria described in it.

Scores and criteria supporting decisions

4.4.7 Where there is a low suspicion of PE and another diagnosis is more likely, NICE recommends the use of the pulmonary embolism rule-out criteria (PERC) to help determine whether further tests for PE are needed (National Institute for Health and Care Excellence, 2020a, 2020b). NICE states that PERC should be considered where ‘the clinician estimates the likelihood of PE to be less than 15% based on the overall clinical impression, and other diagnoses are feasible’. ED staff described concerns about the guidance for PERC that limited their use of it to inform their decisions:

• In practice determining a ‘less than 15%’ likelihood of PE in a patient is challenging. NICE does not provide further guidance on this, but does provide the background to the recommendation (National Institute for Health and Care Excellence, 2020b).
• The criteria for when PERC can and cannot be used is unclear. ED staff described uncertainty about situations such as patients with active cancer or who are on the oral contraceptive pill.

4.4.8 The ED subject matter advisor told the investigation that they had seen PERC support clinical decision making and promote evidence-based care. However, the ED and haematology subject matter advisors said that it can be inappropriately applied where staff do not understand its role and there is limited clarity around its use. The investigation also noted that even when correctly applied, there will still be a small proportion of patients who have a PE but meet the conditions of PERC (Kline et al, 2004).

4.4.9 Where there is a suspicion of PE, NICE recommends the use of the Wells score (see 1.1.6) to guide whether a D-dimer test may be used (National Institute for Health and Care Excellence, 2020a). During its observations, the investigation did not see staff documenting a Wells score before requesting a D-dimer test and did not hear it being used to support diagnostic decisions for PE.

4.4.10 ED staff described concerns about the validity of the Wells score that limited their use of it. Because it is over 20 years old (Wells et al, 2001), staff questioned whether it accounted for the types of patients now commonly seen in EDs such as those with obesity, on the oral contraceptive pill, with suspected but not confirmed cancer, and with reasons for not developing a high heart rate.

4.4.11 Several consultant doctors told the investigation of other scoring systems that they perceived as more valid than the Wells score. The haematology subject matter advisor and another national expert in thrombosis told the investigation that the Wells score was still the most appropriate to use.

Decision to start patients on interim anticoagulation

4.4.12 The ACTA identified that the decision to give a patient interim anticoagulation was sometimes challenging for ED staff. Once a PE has been suspected, NICE recommends anticoagulation if a scan cannot be done immediately (if a patient is likely to have a PE), or if a D-dimer test result is not available within 4 hours (if a patient is unlikely to have a PE) (National Institute for Health and Care Excellence, 2020a). Regarding imaging, the guidance does not define a timeframe for ‘immediately’; it may be that a computerised tomography pulmonary angiography (CTPA) is ordered immediately, but that limited capacity means a delay.

4.4.13 ED doctors and ACPs were familiar with NICE’s recommendation for anticoagulation pending CTPA, but shared occasions where they had withheld anticoagulation. They described their concerns about the safety of a “blood thinning medicine” where a diagnosis is unknown. They felt that NICE’s recommendation did not consider the risk to patients of taking anticoagulation if their diagnosis turned out to be something other than PE. In addition, the recommendation did not state the circumstances in which anticoagulation should not be administered.

4.4.14 The haematology subject matter advisor stated that the risk from a single dose of low molecular weight heparin (commonly used for interim anticoagulation) was small (Chai-Adisaksopha et al, 2014; Agnelli et al, 2013; Büller et al, 2012) unless there were obvious risk factors, for example major recent surgery. A comprehensive history was needed to identify bleeding risk and other diagnoses. However, it was acknowledged that a comprehensive history requires time which may be limited in an ED.

4.4.15 In relation to the administration of anticoagulation pending a D-dimer result, ED staff were often unaware of this NICE recommendation and suspected that it was aimed at primary care. NICE told the investigation that the guidance was for all providers of primary, secondary and tertiary care (for example, GPs, hospitals and specialist care centres) (National Institute for Health and Care Excellence, 2020a).

The influence of COVID-19 on decision making

4.4.16 ED staff shared concerns that the emergence of COVID-19 may invalidate some of the recommendations in NICE guidance for assessing the risk of PE. This had the potential to impact on their decisions when diagnosing PE. With the risk of blood clots associated with COVID-19, staff questioned whether D-dimer result thresholds and the Wells score were still appropriate.

4.4.17 NICE and the subject matter advisors told the investigation that there was limited research into the validity of scoring systems and D-dimer tests in COVID-19. There are differing opinions about assessment of the risk of blood clots in patients with the virus (Patel et al, 2021) and various societies have published advice (for example, British Society of Thoracic Imaging, 2021).

Summary

4.4.18 NICE told the investigation that its document provides guidance based on consideration of available evidence, but that clinicians must make decisions appropriate to the individual circumstances they are presented with. However, NICE welcomes opportunities to clarify and make its guidance more usable. The investigation found that clarity on some aspects of the NICE guidance would be welcomed to support ED staff in diagnosing and treating suspected PE. As a result, HSIB makes the following safety recommendation with the intent that it clarifies guidance where possible to support decision making, and identifies whether more recent evidence is available to inform guidance updates.

4.4.7 Where there is a low suspicion of PE and another diagnosis is more likely, NICE recommends the use of the pulmonary embolism rule-out criteria (PERC) to help determine whether further tests for PE are needed (National Institute for Health and Care Excellence, 2020a, 2020b). NICE states that PERC should be considered where ‘the clinician estimates the likelihood of PE to be less than 15% based on the overall clinical impression, and other diagnoses are feasible’. ED staff described concerns about the guidance for PERC that limited their use of it to inform their decisions:

• In practice determining a ‘less than 15%’ likelihood of PE in a patient is challenging. NICE does not provide further guidance on this, but does provide the background to the recommendation (National Institute for Health and Care Excellence, 2020b).
• The criteria for when PERC can and cannot be used is unclear. ED staff described uncertainty about situations such as patients with active cancer or who are on the oral contraceptive pill.

4.4.8 The ED subject matter advisor told the investigation that they had seen PERC support clinical decision making and promote evidence-based care. However, the ED and haematology subject matter advisors said that it can be inappropriately applied where staff do not understand its role and there is limited clarity around its use. The investigation also noted that even when correctly applied, there will still be a small proportion of patients who have a PE but meet the conditions of PERC (Kline et al, 2004).

4.4.9 Where there is a suspicion of PE, NICE recommends the use of the Wells score (see 1.1.6) to guide whether a D-dimer test may be used (National Institute for Health and Care Excellence, 2020a). During its observations, the investigation did not see staff documenting a Wells score before requesting a D-dimer test and did not hear it being used to support diagnostic decisions for PE.

4.4.10 ED staff described concerns about the validity of the Wells score that limited their use of it. Because it is over 20 years old (Wells et al, 2001), staff questioned whether it accounted for the types of patients now commonly seen in EDs such as those with obesity, on the oral contraceptive pill, with suspected but not confirmed cancer, and with reasons for not developing a high heart rate.

4.4.11 Several consultant doctors told the investigation of other scoring systems that they perceived as more valid than the Wells score. The haematology subject matter advisor and another national expert in thrombosis told the investigation that the Wells score was still the most appropriate to use.

Decision to start patients on interim anticoagulation

4.4.12 The ACTA identified that the decision to give a patient interim anticoagulation was sometimes challenging for ED staff. Once a PE has been suspected, NICE recommends anticoagulation if a scan cannot be done immediately (if a patient is likely to have a PE), or if a D-dimer test result is not available within 4 hours (if a patient is unlikely to have a PE) (National Institute for Health and Care Excellence, 2020a). Regarding imaging, the guidance does not define a timeframe for ‘immediately’; it may be that a computerised tomography pulmonary angiography (CTPA) is ordered immediately, but that limited capacity means a delay.

4.4.13 ED doctors and ACPs were familiar with NICE’s recommendation for anticoagulation pending CTPA, but shared occasions where they had withheld anticoagulation. They described their concerns about the safety of a “blood thinning medicine” where a diagnosis is unknown. They felt that NICE’s recommendation did not consider the risk to patients of taking anticoagulation if their diagnosis turned out to be something other than PE. In addition, the recommendation did not state the circumstances in which anticoagulation should not be administered.

4.4.14 The haematology subject matter advisor stated that the risk from a single dose of low molecular weight heparin (commonly used for interim anticoagulation) was small (Chai-Adisaksopha et al, 2014; Agnelli et al, 2013; Büller et al, 2012) unless there were obvious risk factors, for example major recent surgery. A comprehensive history was needed to identify bleeding risk and other diagnoses. However, it was acknowledged that a comprehensive history requires time which may be limited in an ED.

4.4.15 In relation to the administration of anticoagulation pending a D-dimer result, ED staff were often unaware of this NICE recommendation and suspected that it was aimed at primary care. NICE told the investigation that the guidance was for all providers of primary, secondary and tertiary care (for example, GPs, hospitals and specialist care centres) (National Institute for Health and Care Excellence, 2020a).

The influence of COVID-19 on decision making

4.4.16 ED staff shared concerns that the emergence of COVID-19 may invalidate some of the recommendations in NICE guidance for assessing the risk of PE. This had the potential to impact on their decisions when diagnosing PE. With the risk of blood clots associated with COVID-19, staff questioned whether D-dimer result thresholds and the Wells score were still appropriate.

4.4.17 NICE and the subject matter advisors told the investigation that there was limited research into the validity of scoring systems and D-dimer tests in COVID-19. There are differing opinions about assessment of the risk of blood clots in patients with the virus (Patel et al, 2021) and various societies have published advice (for example, British Society of Thoracic Imaging, 2021).

Summary

4.4.18 NICE told the investigation that its document provides guidance based on consideration of available evidence, but that clinicians must make decisions appropriate to the individual circumstances they are presented with. However, NICE welcomes opportunities to clarify and make its guidance more usable. The investigation found that clarity on some aspects of the NICE guidance would be welcomed to support ED staff in diagnosing and treating suspected PE. As a result, HSIB makes the following safety recommendation with the intent that it clarifies guidance where possible to support decision making, and identifies whether more recent evidence is available to inform guidance updates.

• Recognising that a person may have a PE is challenging, particularly for less experienced staff and when the person’s signs and symptoms are non-specific or atypical.
• Deciding whether to initiate treatment for a suspected PE requires a decision that balances risks, and this decision can benefit from expert knowledge and skill.
• Despite expertise and the available tools to help identify patients who may have a PE, a small number of PEs may always be missed.
• Experts use different thought processes and show different behaviours when making decisions compared to more novice staff.
• Decision-making skills in healthcare are commonly developed through experience, without formal training or opportunities to practise making decisions.
• Simulation-based learning has the potential to help staff acquire decision-making skills more quickly.
• Other industry sectors, such as aviation and the fire service, aim to accelerate the development of decision-making skills through structured training and the use of ‘generic decision tools’ for analytical decisions.
• EDs do not always provide the conditions which support the development of decision-making skills.
• Decision making in EDs is affected by workload, workforce availability, and performance targets.
• ED staff asked for further guidance to be provided on the use of decision aids to support the diagnosis of PE.
• The design of ED processes influences the decisions staff make. There is no standard model of initial patient assessment in EDs; this contributes to variation in the requesting of tests which can affect later decisions.
• Pathways for the diagnosis and treatment of PE in outpatient settings may create a safety risk where patients are discharged on anticoagulation medicines without a confirmed diagnosis; the capacity of imaging services is a significant contributor to this.
• Loss of clinical information when a patient’s care is handed over was identified as a further safety risk. This can contribute to harm if tests, such as D-dimer (a blood test used as part of the assessment of likelihood of PE), are not followed up.
• Work procedures for the diagnosis and treatment of PE are not routinely designed in line with human factors principles to support their access and use.
• The physical design of environments may also affect decision making.

SEIPS is a systems engineering approach with human factors principles embedded within it (see figure 11). SEIPS describes how components of the work system produce work processes which result in different outcomes. Work system factors are described below (Holden et al, 2013; Carayon et al, 2006):

• person(s): the people working in the particular system and the patient
• tasks: undertaken by the persons which may vary in complexity or variety
• tools and technology: used to undertake the tasks which may vary in usability and functionality
• internal environment: the physical space around the persons, for example layout, noise and temperature
• organisation: conditions external to the persons to support the organisation of, for example, resources and activity
• external environment: factors outside of the healthcare institution that might include policy, societal or economic factors.

Processes can be physical, cognitive, or behavioural and lead to outcomes for the patients, professionals or healthcare institutions. The interactions between the various components of the work system lead to different outcomes, both positive and negative. The framework includes feedback loops which represent the adjustments systems make over time.

Overview of the method

An Applied Cognitive Task Analysis (ACTA) involves the following steps (Stanton, 2013):

• Observation of the task of interest with development of a task diagram with expert input.
• Identification of the steps in the task that require cognitive skills.
• Exploring of the cognitive aspects of the task where expertise is used, what those areas of expertise are and how they are used (termed the knowledge audit). Various probes are used to support this (see table 5).
• Identification of difficult cognitive elements, common errors and potential cues or strategies to overcome the difficulty.
• Simulation may also be used where experts are presented with typical scenarios and the steps they describe to manage it are examined from a cognitive perspective.

The investigation’s ACTA

The investigation’s ACTA approach was:

• Training of the investigation team by a subject matter advisor.
• Identification of cognitive elements following observations in two emergency departments (EDs).
• Development of a cognitive task diagram.
• Validation of the cognitive task diagram at the previously visited EDs.
• Knowledge audits with experts and novices across six EDs.
• Discussion of scenarios and their challenging cognitive elements.
• Development of a cognitive demands table and presentation of findings.

The following sections provide supporting information to section 4.2 in the main report.

Initial assessment

What do experts do?

The investigation was told that expert nursing staff:

• recognise that early focus on a diagnosis, such as PE, may influence later decisions.
• seek to identify symptoms to guide the selection of tests that may later support diagnosis.
• recognise that a D-dimer test has a specific use, and only request one following expert consultation.

Why is it difficult for novices?

The investigation was told by novice staff that initial assessment is difficult for novice nurses because:

• They are not trained to assess symptoms and select tests; instead training focuses on triage.
• It is seen as an isolated process without recognition of its influence on later diagnosis and treatment.
• They rarely receive feedback on whether a D-dimer test was appropriate, and do not know the outcome of their patients.
• Blood order sets (requesting forms with pre-determined blood tests depending on the patient’s symptoms) and strict procedures lead to automatic behaviours without consideration of implications.
• Because of the national operating standard, ED processes are performance focused rather than outcome focused.

Further clinical assessment

What do experts do?

The investigation was told that expert staff (consultants/senior doctors/experienced advanced clinical practitioners):

• Recognise that they can never be 100% sure of a diagnosis without relevant tests, but will have a comfort threshold for when they feel testing is needed based on experience.
• Use discriminatory questioning to account for each symptom, sign and investigation result. Any left unexplained require further investigation. For example, unexplained tachycardia (fast heart rate).
• Recognise that observations and investigation results must be interpreted with knowledge of the individual patient and what is normal for them.

• Recognise the potential for an undiagnosed and significant condition in patients with non-specific symptoms and risk factors. They use ‘gut feeling’.
• Look for subtle clues to support their understanding of whether a condition is present.
• Recognise that well patients can still have a significant condition.

• Appreciate current context, for example the COVID-19 pandemic, on patients’ symptoms.
• Understand, recognise and adapt to the constraints of the healthcare system, such as CTPA access and the national operating standard.
• Appreciate the specific role of D-dimers in the diagnostic pathway for PE and are comfortable following it.

• Seek feedback on outcomes of the patients they have treated to reflect on their decisions, and learn from when they were wrong.
• Discuss with fellow experts when they are uncertain about what decision to make to access multiple opinions.

Why is it difficult for novices?

The investigation was told that further assessment is difficult for novice staff because:

• They may see few or no PEs or aortic dissections in their career.
• Training focuses on textbook symptoms and signs suggestive of PE and aortic dissection, leading to rule-based decisions.
• They may reassure themselves that a significant illness/condition is not present in patients who do not fit the rules or who seem well.

• Training focuses on common conditions and so novices may not expect young, fit and well people to have a PE.

• Limited experience results in fitting symptoms, signs and investigation results to a ‘best fit’ diagnosis without accounting for all unusual test results, symptoms or signs.

• Limited experience may also mean higher confidence in a diagnosis having not seen the variable signs and symptoms that someone with a condition may have.
• Documentation of suspected diagnoses on referral letters and initial assessment documents, and what tests have already been ordered, may cue a novice to consider a specific diagnosis to the detriment of another.
• Less experienced staff may require reassurance for their decisions which may mean ordering tests such as D-dimer outside of their intended use.
• Clinical demand limits opportunities for structured case reviews with experts. Experts may only have capacity to discuss patients, rather than see them in person with the novice.
• Staff may be worried about what others think of them and negative responses to previous decisions may alter future decisions.

General practitioner perspective

ED staff told the investigation that their assessments were often supported by the availability of tests and several hours to observe patients. In contrast they described that GPs in primary care have short appointments within which to make decisions and limited access to tests. To examine the primary care perspective, the investigation approached an experienced GP.

The GP described various cues, beyond standard history and examination, that developed his suspicion of a significant medical problem. He walks to the waiting room to collect patients providing early information through cues such as social interaction and the effects of exertion when walking. He described how he “teases out” the potential causes behind symptoms and signs, and is suspicious of those without explanation. He spoke of being concerned by the patient who “is not quite right”. Examples included a young, fit adult with lower than usual oxygen saturations, or an active person who is still able to exercise, but not as intensely.

The GP shared that with experience comes learning from unintended outcomes; these shape levels of perceived risk and make clinicians suspect conditions such as PE. As an experienced practitioner, he recognised the importance of “gut instinct” and that as he has become more experienced, he has felt less inhibited to trust it. This has also meant he has become less worried about what others think as long as he is doing what he feels is best for the patient. A novice may be worried about what others think and may fit unexplained symptoms and signs to a diagnosis. Novices may not tease out the specifics of a patient’s signs and symptoms, focusing on common conditions and accepting them at face value.

Finally, the GP described the context within which his decisions are made. Telephone consultations have become more common, but have removed visual cues. There is also significant demand for GP services, making decisions within short appointments difficult.