Hong
Kong Med J 2018 Dec;24(6):584–92 | Epub 9 Nov 2018
DOI: 10.12809/hkmj187533
© Hong Kong Academy of Medicine. CC BY-NC-ND 4.0
ORIGINAL ARTICLE
Validation and modification of the Ottawa subarachnoid
haemorrhage rule in risk stratification of Asian Chinese patients with
acute headache
HY Cheung, MCEM1; CT Lui, FRCEM, FHKAM
(Emergency Medicine)1; KL Tsui, FRCS (Edin), FHKAM (Emergency
Medicine)2
1 Accident and Emergency, Tuen Mun
Hospital, Tuen Mun, Hong Kong
2 Accident and Emergency, Pok Oi
Hospital, Yuen Long, Hong Kong
Corresponding author: Dr CT Lui (luict@ha.org.hk)
![](https://www.hkmj.org/sites/default/files/mjimage/pdficon.png)
Abstract
Objective: To validate the
Ottawa subarachnoid haemorrhage (SAH) rule in an Asian Chinese cohort
and to explore the roles of blood pressure and vomiting in prediction of
SAH in patients with non-traumatic acute headache.
Methods: A retrospective cohort
study was conducted in two regional hospitals. All patients aged ≥16
years who presented with non-traumatic acute headache to the study
centres from July 2013 to June 2016 were included. A logistic regression
model was created for the variables of the Ottawa SAH rule and other
potential predictors, including vomiting and systolic blood pressure
(SBP) >160 mm Hg. Model discrimination was evaluated using the area
under the receiver operating characteristic curve. Net reclassification
improvement and integrated discrimination improvement indices were
evaluated. The model’s diagnostic characteristics, including
sensitivities and specificities, were evaluated.
Results: A total of 500 eligible
headache cases were included, in 50 of which SAH was confirmed (10%). In
addition to the predictors of the Ottawa SAH rule, vomiting and SBP
>160 mm Hg were found to be significant independent predictors of
SAH. Net reclassification improvement and integrated discrimination
improvement indices indicated that including vomiting and SBP >160 mm
Hg would improve the model prediction. The Ottawa SAH rule had 94%
sensitivity and 32.9% specificity. The modified Ottawa SAH rule that
included both vomiting and SBP >160 mm Hg as criteria improved
sensitivity to 100%, specificity to 13.1%, positive predictive value to
11.3%, and negative predictive value to 100%.
Conclusions: The Ottawa SAH rule
demonstrated high sensitivity. Addition of vomiting and SBP
>160 mm Hg to the Ottawa SAH rule may increase its sensitivity.
New knowledge added by this study
- The Ottawa subarachnoid haemorrhage (SAH) rule is highly sensitive with high negative predictive value for prediction of SAH in Asian Chinese patients presenting with acute headache.
- Modification of the Ottawa SAH rule by adding vomiting and acute hypertension may further improve the negative predictive value and accuracy. Further validation on an external cohort is required.
- The Ottawa SAH rule is applicable for risk stratification of patients presenting with acute headache in emergency and primary care settings, which can provide a reference for referral and prioritisation of imaging.
Introduction
Patients frequently present to emergency
departments (EDs) with headache. About 4.5% of total ED attendance in the
United States is attributable to headache,1
1% to 6% of which is caused by non-traumatic subarachnoid haemorrhage
(SAH).2 3
4 Among the volume of
neurologically intact patients with severe acute headache, identifying the
10% with ‘walking SAH’—patients with SAH but maximum Glasgow Coma Scale
score and normal neurological examination—is particularly difficult.5 Specifically, those patients have good Hunt and Hess
grading and generally better prognosis.6
Failure to identify those SAH patients would jeopardise those patients’
otherwise good outcomes. A case series demonstrated that 25% of aneurysmal
patients with SAH were misdiagnosed during their initial medical
evaluations, 38% of which had clinical grade 1 or 2 at the time of
misdiagnosis.7 Overall, 24% of
patients deteriorated before the correct diagnosis was made, with poor or
worse final outcomes. Most of the misdiagnoses of SAH cases were caused by
failure to perform computed tomography (CT) imaging. Another study
reported that 12% of patients with SAH were initially misdiagnosed, of
which 19% had normal mental status at first contact, and these
misdiagnoses were associated with worse quality of life at 3 months and
increased risk of death or severe disability at 12 months.4 Again, failure to conduct CT scanning was the most
common cause, accounting for 73% of diagnostic errors. However, conducting
a CT scan on every single patient who attends an ED for headache may not
be practical, in consideration of radiation exposure to patients and
resource implications. For such purposes, clinical prediction rules
including the Ottawa SAH rule have been developed for identification of
low-risk patients who can be discharged safely without a CT scan or other
imaging (Table 1).4
However, those clinical prediction rules have not been well validated in
the Asian Chinese population. The primary objective of the current study
is to validate the Ottawa SAH rule in the Asian Chinese population. The
secondary objective is to identify possible modifications to improve its
accuracy.
Methods
Study design and setting
This was a retrospective cohort study conducted in
the EDs of two regional hospitals in Hong Kong. With daily attendance of
approximately 600 and 350 patients, respectively, Tuen Mun Hospital and
Pok Oi Hospital together serve a population of over 1 million. All
patients are coded according to principal diagnosis following the
International Classification of Diseases, Ninth Edition (ICD-9).8 Case recruitment had two phases. First, we searched the
hospital’s electronic database for ICD-9 codes indicating headache
symptoms, related syndromes, and diseases that may present with a primary
complaint of headache (online Supplementary Appendix). In the second phase
of case inclusion, the medical records of all cases retrieved in the first
phase were screened for eligibility to be included in the current study
according to the inclusion and exclusion criteria.
Inclusion criteria
All patients aged ≥16 years who presented with
acute headache to the study centres from July 2013 to June 2016 were
included. Acute headache was defined as non-traumatic headache that
reached maximal intensity within 1 hour, with an interval of <14 days
from headache onset to presentation. The clinical details of each
retrieved case were screened for inclusion eligibility by both written and
electronic medical records. Cases with obvious pathology (eg, frontal
sinusitis) were excluded. Exclusion criteria included age <16 years,
history of trauma within the last 7 days (collapse associated with
headache onset leading to head injury was not an exclusion), history of
previous SAH, known cerebral aneurysm or cerebral neoplasm, >14 days
since symptom onset, altered mental state, Glasgow Coma Scale score <15
on presentation, and new focal neurological signs. Patients were still
included if they had recurrent ED attendance during the study period.
Data collection
A detailed manual review of written and electronic
medical and radiological records was conducted to obtain the following
data for eligible cases: duration and quality of headache, presence of
thunderclap headache, neck pain, limited range of neck movement, loss of
consciousness (LOC), onset with exertion, arrival by ambulance, failure of
ambulation in previously ambulatory patients, associated symptoms of
dizziness and vomiting, history of benign headache syndrome, concomitant
anticoagulant usage or known bleeding diathesis, presenting Glasgow Coma
Scale score, blood pressure and heart rate (the first recorded values in
the ED), neurological deficits, imaging, lumbar puncture, definitive
diagnosis, and neurological outcome. Criteria not documented in the
medical records were presumed to be absent. Standardised data collection
forms were deployed for data entry by a single investigator.
Definition of outcome
Subarachnoid haemorrhage was defined in accordance
with Perry et al9: subarachnoid
blood on CT scan, xanthochromia in cerebrospinal fluid, or red blood cells
in the final tube of cerebrospinal fluid, with positive angiography
findings (ie, an aneurysm or arteriovenous malformation on cerebral
angiography). All CT films were reviewed by both an experienced emergency
physician and a radiology fellow, with outcome decided by consensus.
The study outcomes included the sensitivity of the
Ottawa SAH rule (Table 1) and the impact on the accuracy of the
Ottawa SAH rule of addition of the following clinical predictors to the
proposed clinical decision rule: systolic blood pressure (SBP) >160 mm
Hg, diastolic blood pressure >100 mm Hg, vomiting, failure of
ambulation in previously ambulatory patients, bleeding diathesis or on
anticoagulants, and existence of a benign headache disorder that could
account for the headache.
Statistics
R 3.4.1 for Windows (R Foundation for Statistical
Computing, Vienna, Austria) was employed for analysis, and a 5%
significance level was adopted. Continuous data were presented as mean and
standard deviation if normally distributed. Categorical variables were
shown as frequencies and percentages. For univariate analysis, comparison
was performed between patients in the non-SAH and SAH groups using
independent samples t tests, Chi squared tests, and Fisher’s exact
test where appropriate. Predictors that were significant in the univariate
analysis were entered into the logistic regression model by a forward
stepwise method based on likelihood ratios. Adjusted odds ratios (AORs)
and P values were calculated for each predictor. The Hosmer-Lemeshow
goodness-of-fit test was adopted for model calibration. Model
discrimination was evaluated by the area under the receiver operating
characteristic (ROC) curve of the predicted probabilities. Collinearity
was explored with variance inflation factors. Net reclassification
improvement and integrated discrimination improvement indices were
calculated to assess the improvement of model prediction with the addition
of significant variables to the Ottawa SAH rule.
The Ottawa SAH rule (Table 1) was applied to calculate the sensitivity,
specificity, positive and negative predictive values, positive and
negative likelihood ratios, and their corresponding 95% confidence
intervals (95% CIs). The modified Ottawa SAH rule was created on the basis
of the additional independent predictors included in the logistic model
and the diagnostic characteristics evaluated.
Sample size calculation
Sample size was calculated to yield 80% power at a
5% significance level. These calculations assumed SAH prevalence of 6.5%,
and the original derivation paper of the Ottawa SAH rule achieved 100%
sensitivity and 15.3% specificity.4
With two-tailed hypothesis testing, to achieve the same specificity and
10% variation of sensitivity, a sample of 500 subjects with 31 cases of
SAH would be required. Sample size calculation was performed with NCSS
PASS 11 (Version 11.0.10).
Results
A total of 1816 potential headache cases during the
study period were retrieved from the hospitals’ databases. After a
detailed review of clinical information, 500 eligible cases were included
in the analysis, in 50 of which SAH was confirmed (10%) [Fig
1]. There were missing values for major components of the Ottawa SAH
rule in 16% of the included cases. Two out of the 50 SAH cases had
negative CT results (Fisher scale 1), and both of these cases were
detected by xanthochromia in cerebrospinal fluid extracted by lumbar
puncture. In terms of angiographic findings, angiography was not performed
in five patients; six patients had normal angiograms; two had
arteriovenous fistulae; one had moyamoya disease; two had cerebral amyloid
vasculopathy; and the SAHs of the remaining 34 (68%) patients were
aneurysm-related.
The demographic and clinical characteristics of the
included cohort are shown in Table 2. The SAH group was contrasted with the
non-SAH group in terms of various clinical characteristics. Lumbar
puncture was performed in 50 patients, and two patients had xanthochromia
in the extracted cerebrospinal fluid. Another 48 patients had SAH
diagnosed by CT. Logistic regression (Table 3) revealed that LOC (AOR=16.3; P<0.001)
and thunderclap headache (AOR=12.4; P<0.001) were the strongest
predictive parameters for SAH. In addition to the parameters in the Ottawa
SAH rule, vomiting and SBP >160 mm Hg were demonstrated to be
independent predictors of SAH. The Hosmer-Lemeshow goodness-of-fit test
demonstrated satisfactory model calibration (P=0.986). Area under the ROC
curve for the Ottawa SAH rule was 0.819 (95% CI=0.782-0.851) [Fig
2]. The variance inflation factors of predictors ranged from 1.03 to
1.15, indicating non-significant multicollinearity. The modified Ottawa
SAH rule was defined as positive prediction of SAH with the occurrence of
any of the following criteria: vomiting, SBP >160 mm Hg, or any of the
parameters of the Ottawa SAH rule (Table 1). The area under the ROC curve of the
modified Ottawa SAH rule in combination with vomiting and SBP >160 mm
Hg increased to 0.870 (95% CI=0.837-0.898). Non-parametric comparison of
the areas under the ROC curves demonstrated a statistically significant
difference (P=0.041). The net reclassification improvement index was 0.158
(95% CI=0.007-0.309; P=0.040), and the integrated discrimination
improvement index was 0.622 (95% CI=0.353-0.891; P<0.001). Both indices
indicate that the addition of vomiting and SBP >160 mm Hg would improve
the model’s discriminatory and predictive capacity.
![](https://www.hkmj.org/system/files/hkmj187533-table-3.jpg)
Table 3. Logistic regression model for prediction of subarachnoid haemorrhage in patients with acute headache
![](https://www.hkmj.org/system/files/hkmj187533-fig-2.jpg)
Figure 2. Receiver operating characteristic curves of prediction of SAH by the Ottawa SAH rule and modified Ottawa SAH rule
Table 4 describes the diagnostic characteristics of
various clinical prediction rules that predict SAH in patients with acute
headache. The Ottawa SAH rule achieved 94% (95% CI=82.5-98.4%) sensitivity
and 32.9% (95% CI=28.6-37.5%) specificity. The modified Ottawa SAH rule in
combination with both vomiting and SBP >160 mm Hg produced sensitivity
of 100% (95% CI=91.1-100%) and specificity of 13.1% (95% CI=10.2-16.7%).
![](https://www.hkmj.org/system/files/hkmj187533-table-4.jpg)
Table 4. Diagnostic characteristics of clinical prediction rules for SAH in patients with acute headache
For 349 out of the 500 included patients, positive
predictions would have been rendered by the Ottawa SAH rule, as one or
more of its criteria were satisfied (69.8%). For 441 out of the 500
patients, positive predictions would have been rendered by satisfying one
or more criteria of the modified Ottawa SAH rule (88.2%). The clinical
implication is that 69.8% and 88.2% of the included patients would have
required CT according to the Ottawa and modified Ottawa SAH rules,
respectively (Table 4). In our cohort, CT was performed in the ED
on a total of 481 (96.2%) patients. Thus, application of the Ottawa and
modified Ottawa SAH rules can reduce CT administration by 26% and 8%,
respectively.
Three patients with SAH were not identified by the
Ottawa SAH rule. All three of them were relatively young (aged 20-38
years). Two of them were initially discharged home, with initial negative
imaging for SAH, and were diagnosed upon re-attending the accident and
emergency departments. Of those two, one patient later developed signs of
meningeal irritation with xanthochromia revealed by lumbar puncture, and
the other patient was called back to the hospital 7 days later after a
retrospective CT report found hydrocephalus and collapsed in the medical
ward. A repeat CT scan showed diffuse SAH. Those patients’ mild symptoms
at first presentation that led to their initial discharges might account
for their falsely negative Ottawa SAH rule findings. In addition, patients
who wanted to go home might have been more tolerant of pain and more
reluctant to describe alarming symptoms. The former patient was initially
diagnosed with reversible cerebral vasoconstriction syndrome, and SAH was
noted when the patient subsequently re-attended the ED. This might explain
his atypical presentation in comparison with other patients with SAH who
had ruptured intracranial aneurysms and tended to present with more florid
symptoms from the beginning. The latter patient was diagnosed with a
ruptured anterior communicating artery aneurysm. The third patient had a
positive CT scan, and digital subtraction angiography showed a ruptured
distal internal carotid aneurysm. All three patients experienced vomiting,
and two of them had SBP >160 mm Hg.
Discussion
Various investigations have attempted to identify
clinical parameters to predict SAH among patients presenting with
non-traumatic headache. The identified predictors include: age >40
years, neck pain or stiffness, LOC, onset with exertion, arrival by
ambulance, vomiting at least once, diastolic blood pressure >100 mm Hg,
and SBP >160 mm Hg.3 10 11 In the
last decade, clinical prediction rules have combined various predictors to
provide better accuracy in diagnosis of SAH.4
12 13
The Ottawa SAH rule is one of the best known ones, and it has been subject
to external validation.4 An
external validation study of the Ottawa SAH rule reported sensitivity of
100% and specificity of 7.6%.14
The SAH incidence rate in our study was noticeably
higher than that in the original derivation study and other validation
studies.11 14 15 This is
not consistent with previous epidemiological studies that reported Chinese
people to have a lower rate of aneurysmal SAH than that of other
populations.16 One possible
explanation is that our inclusion criteria were more stringent, as
headache cases with obvious accountable causes were excluded. The mean age
and SBP of our cohort were similar to those of other studies.12 13 14 15 Only 8.8%
of our included patients reported thunderclap headache, much lower than
the 78% to 89% reported in previous studies.4
14 However, the causes of headache
in patients with non-SAH headache are mostly benign, and the pattern of
benign headache syndromes may be different in the Chinese population
compared with that in Caucasians (ie, migraine is more prevalent in
Caucasians).17 The rate of onset
during exertion was much lower in our cohort than in those of previous
studies, as were neck pain and reduced range of movement.14 15 The exact
reason for the latter is unknown. One possible explanation is that there
are more cervicogenic headaches and occipital neuralgia in the other study
cohorts. Conversely, we reported higher rates of arrival by ambulance,
vomiting, and LOC,14 15 although one study excluded unwitnessed LOC.14 Loss of consciousness, thunderclap headache,
vomiting, SBP >160 mm Hg, neck pain or limited neck range of movement,
and age >40 years were found to be significant independent predictors
of SAH. Onset during exertion had a high AOR, but its statistical
significance was limited by its low incidence and limited sample size.
Both the original derivation study and the
validation study by Bellolio et al14
reported that the Ottawa SAH rule had 100% sensitivity.15 In contrast, our study found that the Ottawa SAH rule
was only 94% sensitive. The sensitivity of our study was limited by the
low rates of onset during exertion, thunderclap headache, and neck pain or
limited neck range of movement in our included patients. This might be
partly attributable to our retrospective design, as unreported criteria
were assumed to be absent. Moreover, the term thunderclap headache might
be interpreted differently by different patients. Specificity was higher
in our study than in others (32.9% in our study compared with 8%-15% in
other studies).14 15 This may be attributed to the difference in the
clinical characteristics of patients with non-traumatic headache in the
Asian Chinese population compared with those in the original derivation
study and other validation studies, which were mostly Caucasians.
Adding two independent predictors for SAH (vomiting
and SBP >160 mm Hg) to the Ottawa SAH rule to produce a Modified Ottawa
SAH rule improved its accuracy in terms of sensitivity. We found that
three patients with SAH could not be identified by the Ottawa SAH rule.
All three were relatively young, and two of them presented initially with
mild symptoms and were discharged after brain CT did not show SAH. One
patient was diagnosed with reversible cerebral vasoconstriction syndrome,
which might present differently from the more common aneurysmal SAH. In a
validation study,12 20% of
patients with SAH were CT-negative, and most of them had posterior
communicating artery aneurysm or normal digital subtraction angiography.
However, our three patients did not share those clinical characteristics.
Integration of vomiting and SBP >160 mm Hg to the model detected those
cases and further improved sensitivity to 100% in our cohort.
The specificity of the Ottawa SAH rule was
demonstrated to be low (15%) in the original derivation study.4 This implies that among patients without SAH, only 15%
had negative predictions by the Ottawa rule, while the remaining 85% had
positive predictions. The high false-positive prediction rate may have
implications in terms of excessive unnecessary CT scans ordered: it may
result in unnecessary radiation exposure, and the surge of CT requests
might strain ED resources. With higher specificity in our cohort, we found
that application of the Ottawa and modified Ottawa SAH rules can reduce CT
use by 26% and 8%, respectively. A UK study found that if the Ottawa SAH
rule had been applied, the CT investigation rate would have been much
higher (59% to 74%) than the actual rate of 37%.18
Another UK study reported a similar CT investigation rate of 61.7% with
the application of the Ottawa SAH rule, which was significantly higher
than the rate of 54.2% in actual practice.19
A review surmised that while the Ottawa SAH rule seemingly can rule out
SAH, in actual practice, it might increase the frequency of CT
investigations.11 However, there
is still a lack of impact analysis regarding the effects of the Ottawa SAH
rule on patients’ neurological outcomes and mortality.
A clinical decision rule was recently proposed by
Kimura et al20 in 2016. In their
1561-patient multicentre observational study, the authors aimed to
identify concrete, unambiguous predictors for SAH, avoiding subjective
terms like ‘thunderclap headache’. The EMERALD (Emergency Medicine,
Registry Analysis, Learning and Diagnosis) SAH rule criteria are SBP
>150 mm Hg, diastolic blood pressure >90 mm Hg, blood sugar >115
mg/dL (6.9 mmol/L), or serum potassium <3.9 mEq/L (3.9 mmol/L).
Hyperglycaemia has been well reported in patients with SAH. Most studies
have focused on the prognostic value of blood glucose, but there have been
no other reports on the use of glucose levels for assistance with SAH
diagnosis in the literature. Similarly, hypokalaemia has been reported in
patients with SAH, which was postulated to be related to increased
catecholamine secretion after SAH, resulting in higher intracellular
potassium uptake and reduced serum potassium levels. While it requires
blood sampling, the EMERALD SAH rule has been reported to have 100%
sensitivity and 14.5% specificity, the latter of which is higher than that
of the Ottawa SAH rule (8.8% in the study). Thus, more unnecessary CT
scans could be avoided with the implementation of simple bedside point of
care testing. However, the biological plausibility and external validity
of this study might be affected by the lack of evidence about the
mechanism of hyperglycaemia and hypokalaemia in patients with SAH, and
further, patients with known diabetes mellitus were not excluded from this
study. Patients with known cerebral aneurysm or new focal neurological
deficits were also not excluded. Because CT scans would almost certainly
be ordered for those patients, it might restrict this rule’s usefulness
for detection of ‘walking SAH’. We cannot evaluate this rule, as our
cohort was unlikely to undergo glucose and potassium sampling in the ED,
and so far, we have not found any external validation studies for this
clinical decision rule. Nevertheless, it is worth exploring whether the
addition of blood glucose and potassium levels to the Ottawa SAH rule
could improve its specificity and reduce unnecessary CT administration.
Limitations
This study has several limitations. First, its
retrospective design is prone to information bias. In total, 16% of the
predictors were missing values in this study. With missing values, the
prevalence of the predictors may be reduced, with potential effects on
their diagnostic accuracy. Second, tracing of the outcome of whether or
not the patients had SAH was limited because of the study’s retrospective
nature. Further, if patients with SAH did not re-attend public hospitals
but received treatment in private hospitals, the outcomes may have been
missed. Third, as one study centre contains a neurosurgical department,
there is a risk of referral bias. Cases diagnosed in other hospitals and
referred to the study centre were excluded. In addition, as both study
centres are located in the same cluster, SAH cases diagnosed at one study
centre are often transferred to the other study centre for neurosurgical
consultation at the ED there. Great care was taken to crosscheck between
patients at the two study centres to avoid duplicate entry.
Although we exhaustively searched for all eligible
cases, there was still a chance of selection bias, as some cases that were
eligible according to the inclusion criteria may have been missed. While
the Ottawa SAH rule is very sensitive, it is only applicable to a very
specific group of patients with headache. Patients with headache that took
marginally >1 hour to peak would be excluded. This greatly limits the
rule’s clinical applicability throughout the population: one validation
study reported that only 9% of patients with headache in an ED were
applicable.14 The modified Ottawa
SAH rule lacks an external cohort for validation in this study, and
validation with an independent multicentre prospective cohort would be
required to establish external validity.
In conclusion, the Ottawa SAH rule demonstrated
high sensitivity. Addition of vomiting and SBP >160 mm Hg to the Ottawa
SAH rule as criteria may increase its sensitivity.
Author contributions
All authors made substantial contributions to the
concept or design of the study, acquisition of data, analysis or
interpretation of data, drafting of the article, and critical revision for
important intellectual content.
Declaration
All authors have disclosed no conflicts of
interest. All authors had full access to the data, contributed to the
study, approved the final version for publication, and take responsibility
for its accuracy and integrity.
Funding/support
This research received no specific grant from any
funding agency in the public, commercial, or not-for-profit sectors.
Ethical approval
Approvals from the Hospital Authority New
Territories West Cluster Ethics Committee were obtained.
References
1. Perry JJ, Stiell IG, Wells GA, et al.
Attitudes and judgment of emergency physicians in the management of
patients with acute headache. Acad Emerg Med 2005;12:33-7.
2. Vermeulen M, van Gijn J. The diagnosis
of subarachnoid haemorrhage. J Neurol Neurosurg Psychiatry 1990;53:365-72.
Crossref
3. Perry JJ, Stiell IG, Sivilotti ML, et
al. High risk clinical characteristics for subarachnoid haemorrhage in
patients with acute headache: prospective cohort study. BMJ
2010;341:c5204. Crossref
4. Perry JJ, Stiell IG, Sivilotti ML, et
al. Clinical decision rules to rule out subarachnoid hemorrhage for acute
headache. JAMA 2013;310:1248-55. Crossref
5. Edlow JA, Malek AM, Ogilvy CS.
Aneurysmal subarachnoid hemorrhage: update for emergency physicians. J
Emerg Med 2008;34:237-51. Crossref
6. Hunt WE, Hess RM. Surgical risk as
related to time of intervention in the repair of intracranial aneurysms. J
Neurosurg 1968;28:14-20. Crossref
7. Mayer PL, Awad IA, Todor R, et al.
Misdiagnosis of symptomatic cerebral aneurysm. Prevalence and correlation
with outcome at four institutions. Stroke 1996;27:1558-63. Crossref
8. International Classification of
Diseases. Ninth Edition. Geneva, Switzerland: World Health Organization;
1977.
9. Perry JJ, Sivilotti ML, Sutherland J, et
al. Validation of the Ottawa Subarachnoid Hemorrhage Rule in patients with
acute headache. CMAJ 2017;189:E1379-85. Crossref
10. Lui CT, Tsui KL, Kam CW. Nuchal pain
predicts subarachnoid haemorrhage in severe headache patients. Hong Kong J
Emerg Med 2008;15:212-7. Crossref
11. Carpenter CR, Hussain AM, Ward MJ, et
al. Spontaneous subarachnoid hemorrhage: a systematic review and
meta-analysis describing the diagnostic accuracy of history, physical
examination, imaging, and lumbar puncture with an exploration of test
thresholds. Acad Emerg Med 2016;23:963-1003.Crossref
12. Mark DG, Hung YY, Offerman SR, et al.
Nontraumatic subarachnoid hemorrhage in the setting of negative cranial
computed tomography results: external validation of a clinical and imaging
prediction rule. Ann Emerg Med 2013;62:1-10. Crossref
13. Kelly AM, Klim S, Edward S, Millar N.
Sensitivity of proposed clinical decision rules for subarachnoid
haemorrhage: an external validation study. Emerg Med Australas
2014;26:556-60. Crossref
14. Bellolio MF, Hess EP, Gilani WI, et
al. External validation of the Ottawa subarachnoid hemorrhage clinical
decision rule in patients with acute headache. Am J Emerg Med
2015;33:244-9. Crossref
15. Kowalski RG, Claassen J, Kreiter KT,
et al. Initial misdiagnosis and outcome after subarachnoid hemorrhage.
JAMA 2004;291:866-9.Crossref
16. de Rooij NK, Linn FH, van der Plas JA,
Algra A, Rinkel GJ. Incidence of subarachnoid haemorrhage: a systematic
review with emphasis on region, age, gender and time trends. J Neurol
Neurosurg Psychiatry 2007;78:1365-72. Crossref
17. Stewart WF, Lipton RB, Liberman J.
Variation in migraine prevalence by race. Neurology 1996;47:52-9. Crossref
18. Matloob SA, Roach J, Marcus HJ,
O’Neill K, Nair R. Evaluation of the impact of the Canadian subarachnoid
hemorrhage clinical decision rules on British practice. Br J Neurosurg
2013;27:603-6. Crossref
19. Yiangou A, Nikolenko N, Noreikaite J,
Thondam S. Impact of subarachnoid haemorrhage Canadian clinical decision
rule for investigation of acute headache, a retrospective case note
review. Lancet 2017;389:S103. Crossref
20. Kimura A, Kobayashi K, Yamaguchi H, et
al. New clinical decision rule to exclude subarachnoid hemorrhage for
acute headache: a prospective multicenter observational study. BMJ Open
2016;6:e010999. Crossref