Background
Dizziness and imbalance are prevalent symptoms that pose significant diagnostic and management challenges in primary care settings. The absence of readily available specialists often leads to inadequate diagnosis and management. Decision Support Systems (DSS) offer a promising avenue to assist primary care physicians in accurately diagnosing and effectively managing patients experiencing these symptoms through the utilization of personalized patient data.
Aim
This study aims to evaluate the diagnostic precision and practical application of the EMBalance DSS as a tool to enhance the diagnosis and management of common vestibular disorders within primary care environments.
Methods
Patients presenting with persistent dizziness were recruited from primary care facilities across Germany, Greece, Belgium, and the UK. Participants were randomly assigned to either a group where primary care clinicians assessed patients with the aid of the EMBalance DSS (+ DSS) or a group where assessments were conducted without the DSS (− DSS). Subsequently, neuro-otology/audiovestibular medicine specialists, blinded to the initial assessments, performed comprehensive clinical evaluations on each patient. These expert evaluations served as the “gold standard” against which the diagnostic accuracy of the + DSS group, the − DSS group, and the DSS as a standalone diagnostic tool was measured.
Results
A total of 194 individuals, ranging in age from 25 to 85 years (mean age 57.7 years, SD = 16.7 years), participated in the study. 100 participants were assigned to the + DSS group and 94 to the − DSS group. The diagnosis provided by primary care physicians in the + DSS group aligned with the expert diagnosis in 54% of cases, compared to a 41.5% agreement rate in the − DSS group (odds ratio 1.35). Similar positive trends favoring the + DSS group were noted in terms of patient management and referrals for further specialist care. Notably, the standalone DSS demonstrated superior diagnostic and management accuracy compared to the + DSS group, suggesting potential benefits when used directly.
Conclusion
The study reveals trends indicating improved diagnostic accuracy and management of vestibular disorders when the EMBalance DSS is utilized in primary care. While the DSS shows promise for facilitating timely and effective care for patients with dizziness in primary care settings, further refinement is necessary to enhance its diagnostic accuracy and optimize its clinical utility.
Trial registration number
NCT02704819 (clinicaltrials.gov).
Keywords: Dizziness, Diagnosis, Randomized control trial, Decision support system, Primary Care Physician Diagnosis Decision Support
Background: The Challenge of Vestibular Disorder Diagnosis in Primary Care
Dizziness and imbalance represent significant health concerns with substantial socioeconomic implications. Affecting up to 40% of individuals by the age of 60, these symptoms are among the most frequent reasons for seeking medical consultation [1, 2]. Patients experiencing vertigo may consult primary care physicians up to 9.6 times and specialists up to 7.2 times, often undergoing as many as six laboratory-based diagnostic procedures [3]. The impact extends beyond healthcare visits, with 80% of affected adults requiring time off work [4] and nearly half reporting considerable disruptions to their social and family lives, sometimes necessitating career changes or even job abandonment [5]. Furthermore, individuals with dizziness and imbalance face an elevated risk of cognitive and psychological impairments [6, 7].
Despite the high prevalence and potentially debilitating effects of these conditions, achieving an accurate diagnosis and appropriate treatment plan can be a lengthy process, often requiring an average of 4.5 healthcare provider visits [8]. Primary care physicians, who are often the first point of contact for these patients, can feel overwhelmed by the complexities inherent in diagnosing dizziness and imbalance. This challenge stems from the subjective nature of symptom reporting, the wide array of potential underlying pathologies, the intricate mechanisms governing balance control, and limitations in specialized medical expertise within primary care settings [9, 10]. These factors collectively contribute to delayed diagnosis and suboptimal management of vestibular disorders [8].
While the majority of acute vertigo cases presenting in emergency departments are benign and linked to vestibular conditions such as Benign Paroxysmal Positional Vertigo (BPPV), Meniere’s disease, acute unilateral vestibulopathy, and vestibular migraine, stroke accounts for a significant minority, estimated between 4–15% [11]. Posterior fossa strokes (affecting the brainstem and cerebellum), despite being less frequent, carry a high mortality rate of 3–19% [12, 13] and necessitate swift diagnosis and intervention to prevent further neurological damage and promote recovery [14]. Misdiagnosis of posterior fossa stroke is more likely when dizziness is a primary presenting symptom [15]. To address this critical diagnostic need, structured approaches, such as the TiTraTE algorithm, have been proposed for evaluating vertigo in acute settings. These algorithms emphasize the importance of establishing symptom onset, evolution, triggers, and conducting appropriate examinations [16]. The HINTS (Head Impulse Test, Nystagmus, Test of Skew) examination, a combination of three oculomotor signs, demonstrates superior sensitivity in identifying posterior fossa strokes compared to early MRI-DWI (100% vs. 72%) [17]. The significant burden imposed by vestibular conditions on healthcare systems, economies, and society underscores the urgent need for improved diagnostic and management strategies [10, 18].
Advances in computer science and artificial intelligence offer potential solutions to these diagnostic challenges through the development of clinical decision support systems [19]. DSS are designed to facilitate clinical diagnosis, therapeutic decisions, and treatment planning by leveraging personalized patient data [20, 21]. These systems strategically codify and manage biomedical knowledge, employing computer modeling tools, medical data processing techniques, and artificial intelligence methodologies to address complex clinical scenarios [22–24]. The recent coronavirus pandemic has further accelerated the adoption of telehealth solutions within the vestibular field. A task force of vestibular and eye movement experts has proposed remote assessment strategies for dizzy patients using virtual platforms. These strategies include diagnostic and triage protocols to expedite face-to-face outpatient assessments based on symptom characteristics and signs [25]. The task force concluded that virtual platforms could effectively support eye movement examinations, including nystagmus, saccades, smooth pursuit, binocular alignment tests, and head thrust tests. Furthermore, a feasibility study exploring smartphone-based video recordings for positional testing in screening non-acute BPPV has shown promising initial results [26]. However, while these advancements provide valuable groundwork, they have not yet fully translated into robust, validated DSS tools for widespread clinical use in primary care.
Currently, only a limited number of DSS tools are specifically designed for diagnosing vestibular disorders [27, 28]. A notable ongoing clinical study in Germany is evaluating a system incorporating a computerized clinical decision system, a mobile application, and an interdisciplinary educational program developed by the German Center for Vertigo and Balance Disorders (DSGZ) (Computerized clinical decision system and mobile application with expert support to optimize management of vertigo in primary care: study protocol for a pragmatic cluster-randomized controlled trial [29]. With the exception of this ongoing German study, existing DSS tools have not been rigorously validated in real-world clinical settings where non-specialist physicians utilize them to obtain clinical information and formulate diagnoses.
Moreover, the majority of existing DSS primarily focus on diagnosis, often neglecting to provide comprehensive management support, including rehabilitation guidance for patients with vestibular disorders.
The EMBalance project [27, 30] was initiated to address these gaps by developing and validating a web-based platform specifically for primary care physicians. This platform aimed to facilitate early diagnostic evaluation and effective management of balance disorders. This paper presents the proof-of-concept clinical evaluation of the EMBalance DSS, conducted according to the previously published study protocol [31].
Aims: Evaluating Diagnostic Accuracy and DSS Usability
The primary objective of this study was to determine the diagnostic accuracy of primary care physicians when utilizing the EMBalance DSS (+ DSS) compared to standard assessment methods without DSS support (−DSS) for patients presenting with vestibular disorder symptoms. Diagnostic accuracy was quantified by assessing the level of agreement between the non-specialist physicians’ final diagnoses and the “gold standard” diagnoses established by specialists in neuro-otology (primary outcome measure).
The secondary aims focused on evaluating the usability and effectiveness of the DSS in various aspects of patient care:
- To compare the primary care clinical diagnoses of the + DSS group versus the − DSS group, analyzing overall diagnostic agreement, as well as sensitivity, specificity, positive predictive value (PPV), and negative predictive value (NPV) for individual disorder diagnoses (odds ratio).
- To assess the diagnostic accuracy of the EMBalance DSS as a standalone tool, independent of clinician interpretation.
- To evaluate the level of agreement between the overall management plans proposed by non-specialist physicians in the + DSS group and the “gold standard” management plans determined by specialists.
- To examine the level of agreement between the management plans suggested by the standalone DSS tool and the “gold standard” management plans established by specialists.
- To compare the frequency of referrals to secondary care for management between the + DSS and − DSS groups.
Methods: Randomized Controlled Trial Design and Procedures
Study Design and Settings
This clinical study employed a randomized controlled trial design. The EMBalance study was concurrently conducted in primary and tertiary care centers across the United Kingdom, Germany, Greece, and Belgium. Table 1 details the participating clinical settings.
Table 1. Participating Clinical Settings
| Institution | Primary care setting | Tertiary care setting |
|---|---|---|
| Greece (UoA1) | Hippocrateio Hospital | Hippocrateio Hospital |
| Belgium (UA2) | Antwerp University Hospital | Antwerp University Hospital |
| Germany (UKFLR3) | Freiburg University Medical Centre | Freiburg University Medical Centre |
| UK (UCL4) | Keats Group Practice Hampstead Group Practice Parliament Hill Practice James Wigg Practice Ampthill Practice West Hampstead Medical Centre Brondesbury Medical Centre |
National Hospital for Neurology and Neurosurgery Royal National Throat, Nose and Ear Hospital |
1University of Athens, 2University of Antwerp, 3University of Freiburg, 4University College London
Ethical approval was granted by the Yorkshire and The Humber—Bradford Leeds Research Ethics Committee (approval No. 16/YH/0051). The trial was registered on clinicaltrials.gov (ref. number: NCT02704819). The EMBalance DSS underwent review and received approval from the Medicines and Healthcare products Regulatory Agency (MHRA), based on its classification as a diagnostic support tool intended to augment, not replace, clinician decision-making.
Participants: Inclusion and Exclusion Criteria
Patients presenting to primary care with balance-related symptoms were recruited based on the following criteria:
Inclusion criteria
- Age: 18–90 years
- Capacity to understand provided information and consent to participate
- Acute onset vertigo (single or multiple attacks; defined as a sensation of movement/illusion; onset within 1 month prior to study recruitment) or chronic dizziness (defined as a sensation of disturbed spatial orientation without a false sense of motion, lasting more than 3 but less than 12 months before recruitment) exacerbated by head movements
- Sub-acute presentation of vertigo or dizziness (as defined above, with duration 0 to 3 months prior to recruitment) without prior presentation to emergency services
Exclusion criteria
- Participants with learning disabilities, dementia, or uncontrolled psychiatric disorders
- Pregnant and breastfeeding women
- Patients unable or unwilling to provide informed consent
Study Groups: Intervention and Control
Consenting patients were randomly assigned to one of two study groups:
- Intervention group (+ DSS): Patients evaluated by a non-specialist physician with the support of the EMBalance DSS.
- Control group (−DSS): Patients evaluated by a non-specialist physician without the support of the DSS.
Theoretical Basis of the Intervention: EMBalance DSS Development
The overarching concept of the EMBalance project and the methodology for developing the DSS have been previously detailed [31, 27]. In brief, participating medical partners established and agreed upon “gold standard” diagnostic criteria and treatment guidelines for balance disorders, aligning with the nomenclature, classifications, and recommendations of the Bárány Society (http://www.baranysociety.nl/), a leading international neuro-otological society focused on evidence-based consensus and standardization in vestibular science and clinical practice.
Clinical partners then compiled extensive retrospective clinical data from 984 patients diagnosed with various vestibular disorders. This data encompassed medical history, signs and symptoms, audio-vestibular test results, imaging studies, and questionnaire responses. This rich dataset populated a purpose-built repository designed based on user requirements and usage scenarios identified for the EMBalance project. The repository stored patient demographics, clinical history (symptoms, examinations, etc.), pre-existing conditions, medications, and data related to diagnosis and treatment planning generated by the DSS reasoning engine. Data mining techniques were employed to identify and extract key parameters crucial for developing and training the algorithms embedded within the DSS. These algorithms, combined with clinically relevant parameters defined by the medical partners, pre-defined the range of diagnostic and management decisions the DSS could formulate based on patient clinical data [27]. These core components were then integrated with user-friendly interfaces to create the complete EMBalance DSS platform [31].
The EMBalance DSS Platform: Modules and Features
The EMBalance DSS is a multilingual web-based platform comprising three primary modules [27, 30]:
- Diagnostic Module: Guides primary care physicians through a structured history taking and clinical examination process tailored for vestibular disorders. It incorporates evidence-based algorithms and clinical guidelines to generate potential diagnoses with associated probability estimations (high, medium, or low certainty).
- Management Module: Provides evidence-based management recommendations aligned with the proposed diagnoses. These recommendations encompass pharmacological treatments, dietary interventions, vestibular physiotherapy, and referral guidelines.
- Educational Module: Offers access to educational resources, including instructional videos on vestibular examination techniques and information on various vestibular disorders, designed to enhance the knowledge and skills of primary care physicians in this domain.
Table 2. Diagnostic Accuracy of Non-Specialist Physicians with DSS Support (+ DSS Group)
| Diagnostic category | Sensitivity | Specificity | PPV | NPV |
|---|---|---|---|---|
| BPPV | 73% (19/26) | 89%(66/74) | 70.3% (19/27) | 90% (66/73) |
| PVD | 31.4% (11/35) | 92% (60/65) | 64.7% (11/17) | 72.3% (60/83) |
| BVF | 50% (2/4) | 98.9% (95/96) | 66.6% (2/3) | 97.9% (95/97) |
| VM | 29.6% (8/27) | 97.2% (71/73) | 72.7% (8/11) | 79.7% (71/89) |
| MD | 100%(5/5) | 96.8%(92/95) | 62.6% (5/8) | 100%(92/92) |
| Pontine/cerebellar lesion | 83.3% (5/6) | 90.3% (84/93) | 35.7% (5/14) | 97.6% (84/86) |
| PPPD | 50% (2/4) | 82.3% (79/96) | 22% (2/9) | 97.5% (79/81) |
| Cumulative measures | 48.5% (52/107) | 92.3% (547/592) | 58.4% (52/89) | 91% (547/601) |
MD: Meniere’s disease, BPPV: Benign paroxysmal positional vertigo, PPPD: Persistent postural perceptual dizziness, BVF: Bilateral vestibular failure, PVD: Peripheral vascular disease, VM: Vestibular migraine.
Outcome Measures: Primary and Secondary Endpoints
Primary Outcomes
The primary outcome measure was diagnostic accuracy, assessed by:
- Overall agreement between the diagnoses established by non-specialist physicians (both + DSS and − DSS groups) and the “gold standard” diagnoses determined by neuro-otology specialists, adhering to published evidence-based guidelines.
Secondary Outcomes
Secondary outcome measures focused on DSS usability and its impact on various aspects of clinical practice:
- Comparison of primary care clinical diagnoses between the + DSS and − DSS groups, evaluating sensitivity, specificity, PPV, and NPV for overall diagnosis (all diagnostic categories combined).
- Diagnostic accuracy of the EMBalance DSS as a standalone tool, comparing its sensitivity, specificity, PPV, and NPV for various diagnoses (proposed with high and medium certainty) against those values in the − DSS group.
- Comparison of the level of agreement between management plans developed by non-specialist physicians in the + DSS group and the “gold standard” management plans established by specialists.
- Comparison of the level of agreement between management plans suggested by the standalone DSS tool and the “gold standard” management plans determined by specialists.
- Comparison of the number of referrals to secondary care for management between the + DSS and − DSS groups.
Sequence Generation, Randomization, Allocation Concealment, and Blinding
Randomization sequences were generated independently for each participating center by research fellows not involved in patient diagnosis or management, using Research Randomizer v4.0 software. Eligible patients were randomized in a 1:1 ratio. Allocation concealment was ensured by placing group assignments in sequentially numbered, opaque, sealed envelopes, opened by the non-specialist physician at the time of recruitment. Patient identification trial numbers were assigned to each envelope for retrospective monitoring. Specialists providing the “gold standard” diagnoses were blinded to the initial non-specialist physician assessments and DSS usage.
Study Procedure: Patient Flow and Assessments
The study flow is illustrated in Figure 2. Clinical research nurses were responsible for patient recruitment, obtaining informed consent, and providing patients with their randomized allocation sequence and study ID in sealed envelopes.
On the day of recruitment, patients were examined by non-specialist physicians, either with or without DSS support, based on randomization. Non-specialists formulated diagnoses and care plans with and without DSS assistance. Physicians using the DSS were informed that it would aid in information gathering and would likely suggest multiple potential diagnoses (with probability estimations) and/or recommend specialist referral or further investigations (e.g., MRI). Non-specialists were instructed to exercise their clinical judgment, choosing from DSS-suggested diagnoses or formulating their own.
Non-specialist physicians then prescribed treatment plans for each participant, or opted to refer patients to specialist care. For the + DSS group, physicians could adopt or modify the DSS-proposed management plan, which included pharmacological treatment, dietary advice, and/or vestibular physiotherapy. Within seven days, patients attended a specialist neuro-otology clinic for expert evaluation. Specialists provided the “gold standard” diagnosis and management plan after conducting comprehensive audio-vestibular tests and other necessary assessments. “Gold standard” diagnoses and management plans adhered to Bárány Society recommendations and systematic reviews conducted by the study authors. Specialists compared their “gold standard” assessments to the non-specialist physicians’ diagnoses and management plans, blinded to DSS usage. Discrepancies were resolved by the expert, who determined the final diagnosis and management plan according to evidence-based guidelines. Management included vestibular physiotherapy, ranging from specialized, supervised physiotherapy to generic Cawthorne–Cooksey exercises provided via booklet [33], depending on the clinical setting. All patients were reviewed by specialists at a 3-month follow-up, irrespective of the applied management plan.
Statistical Analysis
Data analysis was performed using IBM SPSS Statistics v22.0 and StatXact statistical software. Descriptive statistics, odds ratios (OR), and confidence intervals (CI) were calculated to assess agreement between specialist and non-specialist diagnoses and management plans, and referral rates. Sensitivity, specificity, PPV, and NPV were calculated to evaluate diagnostic accuracy of non-specialists and the standalone DSS. Statistical significance was set at p < 0.05.
Results: EMBalance DSS Improves Diagnostic and Management Trends
A total of 200 participants were recruited and randomized. Six cases were excluded (five lost to specialist evaluation, one withdrawal), leaving 194 cases for analysis (100 + DSS, 94 − DSS). Expert review was blinded to non-specialist decisions. The mean age was 57.7 years (SD = 16.7), with no significant age difference between groups (p = 0.53). 37% of participants were male and 63% female.
Primary Outcomes: Diagnostic Accuracy Enhancement with DSS
The non-specialist physician’s proposed diagnosis (cumulative across all diagnostic categories) agreed with the expert diagnosis in 54% (N = 54) of cases in the + DSS group, compared to 41.5% (N = 39) in the − DSS group (Tables 2 and 3), with an odds ratio of 1.35 (95% CI 0.76–2.42).
Table 3. Diagnostic Accuracy of Non-Specialist Physicians without DSS Support (− DSS Group)
| Diagnostic category | Sensitivity | Specificity | PPV | NPV |
|---|---|---|---|---|
| BPPV | 58.3% (14/24) | 90%(63/70) | 66.6% (14/21) | 86.3% (63/73) |
| PVD | 44.4% (8/18) | 88.1% (67/76) | 47% (8/17) | 87% (67/77) |
| BVF | 100% (1/1) | 100% (93/93) | 100% (1/1) | 100% (93/93) |
| VM | 35% (6/17) | 93.5% (72/77) | 54.5% (6/11) | 86.7% (72/83) |
| MD | 33.3%(2/6) | 90.9%(80/88) | 20% (2/10) | 95%(80/84) |
| Pontine/cerebellar lesion | 50% (5/10) | 96.4% (81/84) | 62.5% (5/8) | 94% (81/86) |
| PPPD | 7.6% (1/13) | 95% (77/81) | 20% (1/5) | 97.4% (77/79) |
| Cumulative measures | 41.5% (37/89) | 93.6% (533/569) | 50.6% (37/73) | 92.6%(533/575) |
MD: Meniere’s disease, BPPV: Benign paroxysmal positional vertigo, PPPD: Persistent postural perceptual dizziness, BVF: Bilateral vestibular failure, PVD: Peripheral vascular disease, VM: Vestibular migraine.
Secondary Outcomes: Sensitivity, Specificity, PPV, and NPV Analysis
Sensitivity, specificity, PPV, and NPV values per diagnostic category are presented in Table 2 (+ DSS group) and Table 3 (− DSS group). Diagnostic sensitivity in the − DSS group was below 60% for six out of seven diagnoses. The + DSS group showed sensitivity exceeding 70% for Menière’s disease (100%), BPPV (72%), and pontine/cerebellar lesions (83.3%). The standalone DSS tool demonstrated sensitivity above 70% for five diagnoses.
Secondary Outcomes: Standalone DSS Diagnostic Accuracy
Diagnostic accuracy measures for the DSS proposed 1st line (high certainty) and 2nd line (medium certainty) diagnoses are shown in Table 4. Cumulative sensitivity for the standalone DSS was 62%, with an odds ratio of 3 (95% CI 1.67–5.53).
Table 4. Diagnostic Accuracy of Standalone DSS (1st and 2nd Line Diagnoses)
| Diagnostic category | Sensitivity | Specificity | PPV | NPV |
|---|---|---|---|---|
| BPPV | 80% (20/26) | 68.9% (51/74) | 47.6% (20/42) | 87.9% (51/58) |
| PVD | 51.4% (18/35) | 92.3% (60/65) | 85.7% (18/21) | 75.9% (60/79) |
| BVF | 75% (3/4) | 91.6% (88/96) | 25% (3/12) | 100% 88/88 |
| VM | 44% (12/27) | 90.4% (66/73) | 60% (12/20) | 82.5% 66/80 |
| MD | 100% (5/5) | 75.7% (72/95) | 17.8% (5/28) | 100% (72/72) |
| Cerebellar/pontine lesion | 86% (6/7) | 61.2% (57/93) | 14.6% (6/41) | 96.6% (57/59) |
| PPPD | 75% (3/4) | 85.4% (80/96) | 15.7% (3/19) | 98.7% (80/81) |
| Cumulative | 62% 67/108 | 80% (474/592) | 80.7% (67/183) | 83% (474/517) |
MD: Meniere’s disease, BPPV: Benign paroxysmal positional vertigo, PPPD: Persistent postural perceptual dizziness, BVF: Bilateral vestibular failure, PVD: Peripheral vascular disease, VM: Vestibular migraine.
Agreement between the DSS proposed 1st and 2nd line diagnoses and expert diagnoses was observed in 63% of cases, significantly higher than disagreement (37%, p = 0.009). The difference between correct cumulative DSS diagnoses and correct diagnoses by non-specialists without DSS was statistically significant (p = 0.0039).
Secondary Outcomes: Management Agreement and Referral Rates
Correct management by non-specialists (agreement with expert management) was observed in 48% (N = 48) of + DSS cases vs. 31% (N = 29) of − DSS cases (Figure 3), with an odds ratio of 2.07 (95% CI 1.15–3.72).
In cases with correct diagnoses in the + DSS group (N = 54), DSS-proposed management was correct in 37 cases and incorrect in 12 (five missing). In the − DSS group with correct diagnoses (N = 39), management was correct in 25 and incorrect in 13 (one missing), OR 0.6237, CI 0.2450–1.5878.
Referral rates to specialist services were significantly lower in the + DSS group (2%) compared to the − DSS group (12.8%) (Figure 4), odds ratio 7.17 (95% CI 1.56–32.96).
Figure 4: Patient referral percentage for management in + DSS and − DSS groups. NR: no referral, R: referral.
Discussion: EMBalance DSS as a Promising Tool for Primary Care
This study represents the first completed clinical evaluation of a novel DSS designed to improve the diagnosis of vestibular disorders in primary care settings. The findings indicate positive trends towards enhanced diagnosis and management in the + DSS group compared to the − DSS group. Furthermore, the standalone DSS demonstrated even greater diagnostic and management accuracy than when used in conjunction with physician decision-making, highlighting its potential as a powerful diagnostic aid.
Effective management of dizziness is contingent upon identifying the underlying cause. However, dizziness complaints in primary care are diverse [34], and accurate diagnosis can be challenging with limited diagnostic resources [35]. Machine learning algorithms and predictive models have previously shown utility in tertiary healthcare settings [28, 36–38]. A key strength of the EMBalance DSS lies in its attempt to address barriers related to symptom definition and diagnostic strategies, aiming to improve diagnostic success rates. This initial validation study provides encouraging evidence supporting this approach.
DSS as a Supportive Tool for Primary Care Physicians
Diagnostic evaluations by non-specialist physicians, when compared against specialist “gold standard” diagnoses, tended to be more accurate in the + DSS group (54% correct) than in the − DSS group (41.5% correct). While a trend towards improved sensitivity and PPV was observed in the + DSS group across all diagnostic categories, this trend was not statistically robust. One possible explanation for this weaker trend could be that non-specialists did not consistently adopt the DSS-proposed diagnoses, even when presented with high or moderate certainty levels. Another contributing factor might be the use of specialized neuro-otological terminology within the DSS, which may have been unfamiliar to some primary care physicians. Recent research has shown that specific language patterns are associated with different vestibular conditions [37], and understanding these nuances is critical for accurate diagnosis. While patients often use the generic term “dizziness” to describe their symptoms, the varied differential diagnoses and potential knowledge gaps among non-specialist primary care physicians in vestibular disorders can impede effective information gathering and accurate diagnosis, even with DSS support. User feedback collected separately within the EMBalance project indicated that the DSS’s front-end graphical user interface was user-friendly and easy to understand, suggesting that program language comprehension was not a significant barrier. The DSS backend incorporated clinical data from 984 patient records, characterized by approximately 350 features. It is plausible that a larger dataset and/or inclusion of additional relevant features could further enhance diagnostic accuracy. Furthermore, potential inconsistencies in history taking and clinical examination techniques among non-expert clinicians might have influenced diagnostic outcomes. While the EMBalance DSS offered instructional videos demonstrating correct clinical examination techniques, accessible within the interface and via a YouTube channel (https://www.youtube.com/channel/UCXFf98Ktus48Ut9a5Nbm4sA), physicians were not explicitly required to review these videos prior to DSS use. These factors will be addressed in future iterations of the DSS to improve its effectiveness as a primary care physician diagnosis decision support tool.
Standalone DSS Performance: High Sensitivity
The standalone DSS tool, providing 1st and 2nd line diagnostic suggestions, exhibited higher sensitivity compared to the − DSS non-specialist group (Odds Ratio of 3), although this was accompanied by a reduction in specificity. The overall sensitivity of the standalone DSS was 62%, comparable to findings from a previous study by Feil et al. [28]. The EMBalance DSS standalone tool demonstrated high sensitivity (> 80%) for diagnosing Meniere’s disease, cerebellar pontine lesions, and BPPV, and moderate sensitivity (> 70%) for PPPD and BVF diagnoses. However, sensitivity remained relatively low for peripheral vestibular disorders (PVD) and vestibular migraine (VM), aligning with sensitivity levels reported in other studies [39]. Further research could explore how the expertise level of non-specialist physicians and increased experience with DSS usage over time might affect diagnostic outcomes.
Diagnostic Accuracy for Specific Vestibular Entities
For primary care physicians, including general practitioners (GPs), key clinical priorities include ruling out life-threatening conditions, diagnosing treatable specific diseases, and identifying chronic dizziness early to prevent its progression [34]. Analysis of the DSS’s diagnostic capabilities for individual vestibular entities revealed high sensitivity and reasonable specificity for certain high-impact diagnoses. Notably, the DSS demonstrated strong performance in identifying cerebellar-pontine lesions, which are potentially life-threatening and require prompt diagnosis [12, 13]. The sensitivity of non-specialist physicians in the − DSS group for diagnosing these lesions was only 50%, while the EMBalance DSS achieved a significantly higher sensitivity. This improved performance may be attributed to the DSS’s incorporation of relevant clinical history and examination elements based on the TiTrate and HINTS rules [16], combined with advanced data mining techniques [27]. The DSS also exhibited high sensitivity for common, treatable vestibular disorders like BPPV and PPPD [40, 41], as well as rarer conditions like Meniere’s disease. These findings suggest that the DSS holds considerable promise for enhancing the diagnosis of these specific vestibular conditions in primary care settings, as supported by the results of this randomized clinical trial.
Improved Management Plans with DSS Support
Management plans were significantly improved in the + DSS group compared to the − DSS group (odds ratio 2.07). This finding is consistent with a meta-analysis of 138 clinical trials of non-vestibular DSSs, which reported improved treatment quality in 46 of these studies [42]. The management results underscore that, despite the high prevalence of dizziness and vertigo in general practice, non-specialist physicians’ utilization of optimal vestibular management strategies remains suboptimal. Furthermore, the proportion of patients referred for further assessment and management was significantly lower in the + DSS group (2%) compared to the − DSS group (12.7%). Reducing unnecessary referrals to specialist care is a key indicator of DSS success [42]. The average number of healthcare provider visits required to achieve a correct diagnosis and initiate appropriate treatment for vestibular disorders is approximately 4.5 in both the US and the UK [8]. Therefore, the observed improvements in management accuracy and reduced referral rates with DSS use could translate into substantial cost savings for healthcare systems and society. Management decision accuracy was also high for the standalone DSS, reaching 75%. Factors such as mistrust of new technologies [43] could potentially influence the adoption of DSS management recommendations, as non-specialist physician management decisions, while improved with DSS support, were still less accurate (56%) than the standalone DSS (75%). Clinician perception of DSS utility may also evolve with experience, with perceived benefits potentially increasing over time compared to initial usage [44]. Further investigation into changes in non-expert management decision-making at the start versus later stages of DSS use could provide valuable insights into user adoption patterns.
Overall, the study highlights that optimal vestibular management is often lacking for patients managed solely in primary care or by non-specialists. This represents a significant concern, as these patients are at a threefold increased risk of developing psychological sequelae, such as anxiety, panic disorder, and depression [52], and chronic dizziness [53], leading to substantial socioeconomic costs [54]. Early diagnosis and appropriate management are crucial for these patients. The EMBalance DSS demonstrates promise in addressing these needs. However, further improvements in diagnostic accuracy are necessary before widespread clinical implementation, and the study findings should be replicated in larger, multicenter trials.
Limitations and Future Research Directions
The introduction of computer-aided systems into clinical practice may, to some extent, disrupt the patient-doctor relationship [21]. The EMBalance DSS design incorporated features such as a user-friendly interface, streamlined data entry, and allocated consultation time to mitigate this potential limitation and minimize patient discomfort. Furthermore, the user manual provided to participating physicians included a dedicated section addressing this issue and suggesting strategies to mitigate this risk.
Resistance to adopting information technology solutions, even when rigorously validated, is a recognized challenge that needs to be addressed during implementation [43]. Successful adoption may necessitate educational initiatives and proactive customer acquisition strategies to fully realize the potential of these technologies. Another potential limitation influencing diagnostic accuracy, for both non-specialists and the DSS, is the possibility that non-specialist physicians’ inexperience may affect their ability to accurately elicit key clinical information, such as phrasing appropriate questions about crucial symptoms suggested by the DSS. This could inadvertently bias the DSS’s diagnostic predictive capabilities [37]. The presence of multiple overlapping vestibular disorders, such as the co-occurrence of vestibular migraine and Meniere’s disease [55], presents an additional diagnostic challenge for non-specialists. Future research should investigate these factors to inform and guide subsequent DSS iterations and validate findings through additional studies. The current study did not track missing data rates within the EMBalance DSS fields populated by clinicians, which could also influence diagnostic outcomes. Exploring this aspect in larger studies could help determine the optimal number of populated features required for accurate diagnosis. Finally, the COVID-19 pandemic has underscored the growing need for remote diagnostic tools and the potential value of detailed, personalized, and digitally recorded data for precision healthcare [56]. A modified version of the EMBalance DSS could be particularly well-suited for telehealth applications and warrants further exploration in this context.
Future iterations of the EMBalance DSS will incorporate measures to enhance diagnostic accuracy. The training dataset used to develop the DSS prototype will be expanded for each diagnostic category. The DSS will include additional review questions for important features that show discrepancies (e.g., reported vertigo lasting hours with a positive Dix-Hallpike test). It will also integrate specific rule-based components to highlight critical features from the patient history and clinical examination, aligning with recommendations from the Consensus on Virtual Management of Vestibular Disorders [25]. This will result in a “hybrid” system, combining rule-based logic with data mining techniques to improve predictive capabilities. Furthermore, based on feedback from target users at the end of the study and mirroring approaches in more recent studies like the PoiSe study [29], an introductory training course will be developed for target users to explain the DSS structure and optimal data population methods.
Conclusion: EMBalance DSS Shows Promise for Vestibular Disorder Management in Primary Care
The EMBalance DSS provides a structured and comprehensive diagnostic and management framework for a wide range of vestibular disorders. The diagnostic and treatment protocols embedded within the system have been developed through collaborative input from the EMBalance consortium, adhering to national and international guidelines. This proof-of-concept study demonstrated a trend towards improved diagnosis of vestibular patients with DSS use compared to traditional methods, achieving statistical significance when considering both first- and second-line diagnoses accepted by primary care physicians. The DSS also facilitated the provision of significantly improved management strategies. Implementing decision support systems like the EMBalance DSS in primary care settings, for both simple and complex cases (particularly those showing limited improvement after three months of follow-up), has the potential to enhance patient diagnosis, alleviate symptoms, and positively impact associated socioeconomic costs and patient quality of life. While further development is needed to optimize its diagnostic accuracy, the EMBalance DSS holds substantial promise for ensuring timely and effective diagnosis and management of vestibular disorders in primary care. Its potential relevance is further amplified in the context of the COVID-19 pandemic, highlighting the need for innovative telehealth solutions. Emerging interactive patient communication methods spurred by the pandemic, coupled with DSS and AI strategies like EMBalance, may pave the way for significant advancements in patient care in the future.
Acknowledgements
We gratefully acknowledge the support of the Ménière’s Society, the participating clinical centers, clinical colleagues, and the patients who generously contributed to this study.
Abbreviations
BPPV Benign paroxysmal positional vertigo
BVF Bilateral vestibular failure
DSS Decision support system
− DSS Control group: ‘non-specialist doctors without the support of the DSS’
- DSS Intervention group: ‘non-specialist doctors with support from the DSS’
MD Meniere disease
MHRA Medicines and Healthcare products Regulatory Agency
MV Vestibular migraine
PPPD Persistent postural perceptual dizziness
PVD Peripheral vestibular disorder
RCT Randomized control trial
Author contributions
DEB, DK, TB, CM, FW, DF, DK, and LML were principal investigators, designed studies, oversaw data collection and analysis, drafted (DEB) and revised the paper. NK, NM, BI, LC, VM and LM conducted the clinical study, collected and analyzed data, and revised the paper. TE provided technical support, analyzed results, and revised the paper. IN contributed to clinical study design and conduct, and results analysis. VVP and PVDH oversaw the clinical study, analyzed results, and revised the paper. All authors approved the final draft.
Funding
This study was supported by a European Commission FP7 Grant Agreement 610454.
Availability of data and materials
Data supporting the findings are available upon request from University College London, subject to data access restrictions. The data are not publicly available.
Declarations
Conflict of interest
The authors declare no competing interests.
Ethics approval and consent to participate
Ethical approval was obtained from the Yorkshire and The Humber—Bradford Leeds Research Ethics Committee (approval No. 16/YH/0051). The trial was registered in clinicaltrials.gov (NCT02704819). The EMBalance DSS was approved by the MHRA as a diagnostic support tool, not a substitute for clinical judgment.
Informed consent
All participants provided informed consent after receiving written information and oral explanation.
Consent for publication
Not applicable.
References
Associated Data
Data Availability Statement
The data that support the findings of this study are available upon request from the University College London, but restrictions apply to the availability of these data. The data underlying this article cannot be shared publicly.
